Why AI chatbots make things up — and how citations fix it

In 2024, an airline's chatbot invented a bereavement refund policy. A court ordered the airline to honor it. The bot hadn't been hacked or misconfigured in any dramatic way — it did exactly what language models do by default: it produced the most plausible-sounding answer, and plausible is not the same as true.

If you're putting AI in front of your customers, this is the risk that matters. Not that the bot will be unhelpful — that it will be helpfully, confidently wrong, in your brand's voice, about your own policies.

Hallucination isn't a bug — it's the default

Language models are trained to continue text convincingly. Ask one a question outside its knowledge and it doesn't experience a gap; it fills the gap with the statistically likely shape of an answer. Fine for brainstorming. Catastrophic for a returns policy.

Vendors often respond with 'better prompts' or 'guardrails' — instructions layered on top of the same mechanism. That reduces the frequency of invention. It can't eliminate it, because the underlying objective — always produce an answer — hasn't changed.

The structural fix: retrieval with receipts

The fix that actually changes the failure mode is architectural. First, restrict the agent to answering from a specific corpus — your website, your docs, your policies — retrieved at question time. Second, require it to cite the passage behind each claim. Third, when retrieval comes back empty, the correct output is 'I don't know,' not improvisation.

Citations do two jobs at once. For your visitor, they turn the answer from an assertion into something checkable — click through and read the source. For you, they make every conversation auditable: when an answer looks off, the citation shows you exactly which page produced it, which usually means the page needs fixing, not the bot.

"I don't know" is a feature

Teams sometimes worry that a refusing bot looks weak. In practice, the opposite: visitors trust an assistant more after seeing it decline once, because it proves the confident answers mean something.

Every refusal is also free product research — a logged, precise gap in your content. Write the missing page, the next crawl picks it up, and the same question gets a cited answer next week. That loop, answer-or-log-the-gap, is how the agent and the website improve each other.