A demo is a promise the engineering has to keep. A well-rehearsed happy path on a stage is not the same thing as a system that lives alongside people, fails gracefully, and shares responsibility when it does. One is theatre. The other is a product.
A failure pattern worth testing for is this: they were optimised for the demo, not for the seventy-third Tuesday of their existence. The thing that makes an AI feature real isn’t how it performs on the prepared example; it’s how it behaves when the user does something the team didn’t anticipate, which a rehearsed demonstration cannot rule out.
The hard parts are the interesting parts.
Handoff. Refusal. Admission of uncertainty. These are not UX niceties; they are the product. The AI that says “I’m not sure, but here’s what I can tell you” gives the user information that a flawless demo cannot: where the system’s knowledge ends.
Reliability is usually framed as a tax on creativity. I think it is also where creativity gets tested. Anyone can sketch a feature that works when the input is pristine. The interesting design work is the feature that stays useful when the input is weird, the available evidence is incomplete, and the user is annoyed.
Give uncertainty somewhere to go
Imagine a support assistant that finds two conflicting answers in the approved documentation. A demo can quietly avoid that example. A product needs to decide what happens next.
One possible design is to show the conflict, identify the sources and ask a support specialist to resolve it. Another is to provide the limited facts that are consistent while clearly withholding the disputed answer. The right choice depends on the task and consequences. Confidently blending the two documents is not a resolution.
| Condition | Product behaviour | Evidence to retain |
|---|---|---|
| Relevant, consistent evidence | Answer within the permitted scope | Sources and result |
| Conflicting or missing evidence | Explain uncertainty and request clarification | Conflict and unresolved question |
| Action unavailable or disallowed | Stop the action and offer a valid handoff | Attempt, access decision and next owner |
This is an illustrative design, not a report of a deployed system. Its purpose is to make the behaviour testable. “Handle errors gracefully” does not tell an engineer what the customer should see or an evaluator what to check.
Test the experience after the failure
Did the user learn what happened? Can they continue without starting again? Does the next person receive enough context to help? Those questions turn a failure state into product work.
Anthropic’s agent evaluation guide distinguishes what an agent says from the outcome in the environment. That distinction belongs in the experience too. A message claiming success should not substitute for checking whether the requested change actually happened.
For a practical way to separate those checks, see the evaluation evidence map. Outcome, authority and recovery need their own evidence. One passing happy-path run does not establish all three.
The NIST AI Risk Management Framework is a useful voluntary reference for thinking about risks across the system lifecycle. It does not remove the need to define the actual behaviour of this product, for this person, in this situation.
Make those choices early in a short decision record. Otherwise the fallback experience will be whatever the last error handler happens to do. That is an implementation accident with a user interface.
