RAG / Retrieval-Augmented Answer Checklist

Use this to check an answer produced by a system that retrieves documents/context and then generates a response from them. - Is every factual claim in the answer actually supported by the retrieved context — not filled in from the model's general knowledge when the context didn't contain it? - If the retrieved context didn't contain enough information to answer, does the answer say so, rather than confidently answering anyway? - Are citations/sources (if shown) pointing to the passage that actually supports the specific claim next to it, not just a loosely related document? - Did retrieval actually pull the right documents for this query — check a sample manually if the answer seems off; a wrong answer is often a retrieval problem, not a generation problem. - Is outdated or superseded information in the retrieved context being presented as current, when a newer document in the corpus contradicts it? - Does the answer correctly handle a query that spans multiple retrieved documents, or does it only reflect the first/most similar one? - For a query with no good match in the corpus, does the system say it doesn't know, instead of generating a plausible-sounding but ungrounded answer? A RAG system that never says "I don't have information on that" is a warning sign, not a feature — check specifically for that failure mode with a query you know isn't covered.