Why AI makes things up (hallucinations) and five habits to make it happen less

In 2023, a New York lawyer filed a brief in federal court citing several rulings that supported his case. None of them existed. He had found them by asking ChatGPT, which not only invented them with plausible names, dates and docket numbers but also confirmed they were real when he asked. The case, known as Mata v. Avianca, ended with a financial sanction for the lawyers.

In 2024, a Canadian tribunal ordered Air Canada to compensate a passenger because the virtual assistant on its website had described a bereavement fare policy that did not match the real one. The airline argued that the chatbot was responsible for its own answers; the tribunal did not accept that.

Both stories have the same origin: a hallucination, an invented answer that the model presents with complete confidence. Understanding why it happens helps you prevent it rather than discover it when it is too late.

Why AI makes things up

It writes what is plausible, not what is verified

A language model generates text by predicting, piece by piece, the most likely continuation (we explain it step by step in What is a large language model). It has no internal step where it checks a database of verified facts. If you ask it for a legal citation, the most likely continuation is something with the exact shape of a legal citation. Whether it exists is not part of the calculation.

It has been rewarded more for answering than for doubting

There is a second, less intuitive cause. In 2025, OpenAI researchers published an analysis that attributes much of the hallucination problem to how models are trained and evaluated: most tests score answers as right or wrong, with no credit for "I don't know". In that system, guessing pays off, just like on a multiple-choice exam with no penalty for wrong answers. The model learns that a confident answer scores better than an honest one.

It fills gaps to be helpful

When your question is missing information, the model tends to fill it with reasonable assumptions instead of asking you. If you ask for "the price of this tool's business plan" without giving it the pricing page, it will give you a number that looks real.

Where it is most likely to make things up

Not every question carries the same risk. Be most on guard when you ask for:

• Citations, references and bibliographies. Book titles, scientific papers, court rulings, specific URLs.
• Exact figures. Percentages, prices, dates, market statistics.
• Facts about little-known people or companies. The less they appear online, the more it improvises.
• Local or very specific rules. A country's tax requirements, a city's regulations, a company's internal policies.
• Recent events. Anything after its knowledge cutoff, if the app does not run a search.
• Details inside long documents. It may attribute to page 40 something that page 4 says, or mix up two sections (we cover this in Tokens and the context window).

By contrast, the risk is low when you ask for things that do not depend on a specific fact: ideas, rewrites of your own text, structures, drafts, explanations of very well-known concepts.

Five habits to make it invent less

1. Give it the source instead of asking it to remember

This is the change with the biggest impact. If you paste the contract, the pricing page or the report, the model works from that text and not from its statistical memory. This is called grounding, anchoring the answer to a source. RAG systems do the same thing automatically with your documents.

2. Give it explicit permission not to know

Anthropic's documentation on reducing hallucinations recommends exactly this: allow the model to admit uncertainty. One line is enough: "If you do not have enough information to answer confidently, say so and explain what is missing. I prefer that to an invented answer."

3. Ask for verbatim quotes before the conclusion

With a document in front of it, ask first for the literal passages that support the answer and then for the answer. If it cannot find a passage, it should not claim anything on that point. And you can check each quote in seconds with your browser's search.

4. Separate facts from opinions in the answer

Ask it to mark which statements are checkable facts and which are inferences or recommendations. As well as helping you review, it forces the model to treat what it presents as fact more carefully.

5. Use web search when the fact is current or specific

If your assistant has web search, turn it on for prices, regulations, news or company data, and ask it to link each claim to the page it comes from. Open at least the links for the facts you will use: a real link does not guarantee the page says what the answer claims.

A prompt that brings the five habits together

Answer the question using only the material I give you below.

Rules:
- Before answering, copy word for word the passages you will use.
- Mark each statement as [FACT] if it comes from the material or
  [INFERENCE] if it is your own conclusion.
- If the material does not let you answer something, write "Not in the
  material" and do not fill it in with general knowledge.
- Do not invent figures, names, dates or links.

Material:
[PASTE THE TEXT OR DOCUMENT HERE]

Question: [YOUR QUESTION]

To check an answer you already have, use the Fact-check claims template. To research a topic from scratch with real sources, use Deep research with sources.

What these habits do not solve

Reducing is not eliminating. Even with the source in front of it, a model can misread a table, confuse two similar figures or summarise in a misleading way. So the final rule is simple: responsibility for what you publish, sign or decide is still yours. When an answer is going to leave your chat and affect other people, check it with the method in How to check an AI answer before you use it.

Keep reading

• How to check an AI answer before you use it: the method for reviewing what it already told you.
• What is a large language model (LLM): why it writes the plausible rather than the verified.
• How to write a prompt that works the first time: context, examples and format to ask better.
• How to research with AI without losing rigour: primary sources, searches and checking.
• In the glossary: hallucination and grounding.

Sources

• Kalai, A. T., Nachum, O., Vempala, S. S. and Zhang, E. (2025). Why Language Models Hallucinate. arXiv.
• Anthropic. Reduce hallucinations. Claude documentation.
• Mata v. Avianca, Inc., No. 22-cv-1461 (PKC), United States District Court for the Southern District of New York, sanctions order of 22 June 2023.
• Moffatt v. Air Canada, 2024 BCCRT 149, Civil Resolution Tribunal of British Columbia. Coverage: Global News (February 2024).