Chatbot Conversation Quality Checklist

Use this to review a chatbot's behavior across a multi-turn conversation, not just a single response in isolation. - Does it maintain context correctly across turns — does it forget, contradict, or ignore something the user said two messages ago? - Does it ask a clarifying question when the user's request is genuinely ambiguous, rather than guessing and running with the wrong interpretation? - If the user corrects it, does it actually incorporate the correction, or does it repeat the same mistake in a later turn? - Does it stay within its intended scope (support, a specific domain) rather than answering things it has no business answering, or refusing things it should actually help with? - Does tone stay appropriate and consistent across the conversation, including if the user gets frustrated — does it de-escalate, or match/escalate the user's frustration? - When it doesn't know something or can't help, does it say so clearly and hand off appropriately, rather than stalling, deflecting, or fabricating an answer? - Try to break it: contradictory instructions, an attempt to get it to ignore its own guidelines, a request outside its intended use — what actually happens? Review whole conversations end-to-end, including ones a real user found frustrating, not just cherry-picked single exchanges — most chatbot failures are about the arc of a conversation, not any one message.