Skip to content

The confident wrong answer

When an agent hits its limit or can't make progress, the dangerous outcome is not a crash, it's returning a half-finished answer reported as success, so a failure reaches the user looking like a result.

incident

An agent that couldn't find the refund policy answers anyway, and a customer is told the wrong thing with full confidence.

where_are_you

The maturity ladder, worst to best

These are the rungs the Agent Reliability Scorecard scores for this dimension, straight from the tool. Find where your agent sits, then aim for the top rung.

  1. It returns whatever half-answer it's holding, reported as success

  2. It throws and the whole request just fails

  3. It returns a clear "couldn't complete" with the partial state

  4. It escalates to a human or a fallback with its state intacttarget

the_fixMake the failure branch a contractWhen the agent hits its ceiling, hand off to a human or a fallback with its state intact. The difference between escalating and silently returning a half-answer is the difference between graceful degradation and a confidently wrong output.

why_it_matters

Every agent has a ceiling: the iteration cap, a tool that's down, a question it can't resolve. What happens at that ceiling is a design choice, and the worst choice is the invisible one, where the agent hands back whatever partial state it's holding and the calling code treats it as a normal success. A confidently wrong answer is more expensive than an error, because nobody knows to distrust it.

Make the failure branch a contract. When the agent can't complete, it should say so explicitly and hand off, to a human, to a fallback path, or back to the user, with its partial state intact so the next step can continue rather than restart. Graceful degradation is a feature you build, not a default you get. The difference between escalating and silently half-answering is the difference between a system you can trust and one you can't.

ceiling hit → half-answerreturned as successescalate with state intact instead

faq

Questions & answers

How should an AI agent handle a task it can't complete?
It should fail loudly, not quietly. Return a clear "couldn't complete" with the partial state, and escalate to a human or a fallback path rather than emitting a half-answer as if it succeeded. The calling code needs to be able to tell success from failure, which it can't if every run returns something that looks like an answer.
Why is a confident wrong answer worse than a crash?
A crash is visible and gets handled; a wrong answer delivered with confidence propagates. The user acts on it, downstream systems store it, and nobody flags it because nothing looked broken. Designing the ceiling to escalate rather than half-answer keeps failures detectable.

Fixing one failure mode is a day. Hardening the whole loop is the work.

I review where your agent loop terminates, how it escalates, how it fences tool output, idempotency on the side effects, and the evals that catch a regression before users do. Book a call, or leave your email.

Book a call

No spam. You'll get a reply from me.

Prefer proof first? See how this plays out in real case studies →