Checklist for catching confidently-stated but wrong or unverifiable claims in an AI's factual answer.
Shared by Parshu · Published Aug 1, 2026
Use this to check a factual answer for claims that are wrong, unverifiable, or stated with more confidence than is warranted. - For each specific claim (a fact, statistic, date, name, quote), can you verify it against an actual source, or is it stated as fact with nothing backing it? - Does the answer distinguish between well-established fact, a commonly repeated but unverified claim, and its own inference — or does everything get stated with the same flat confidence? - If the question was genuinely uncertain, contested, or outside what's knowable, does the answer say so, or does it manufacture a confident-sounding answer anyway? - Are any numbers or statistics precise-looking but actually unsourced ("73% of companies...") — specific-sounding numbers are a common way unsupported claims sound more credible? - Does it cite a source when it claims to, and does that source actually say what's attributed to it, if you check? - If you ask a followup pushing back on a claim, does it hold up its reasoning, cave immediately regardless of whether the original claim was right, or double down without new justification either way? - For anything time-sensitive, is the answer current, or is it presenting outdated information as if it's still true? Treat any specific, checkable claim you're about to act on or repeat elsewhere as unverified until you've actually checked it — "sounds right" and "is right" are not the same thing, and that gap is exactly where hallucinations hide.