Ask a chatbot for a source and it may hand you a perfect-looking reference to a paper that does not exist. Researchers call this a hallucination. It is not a rare glitch, and it is not lying. It follows from how these systems are built, and once you see why, you can work around it.
What a hallucination is
In natural language generation, a hallucination is text that reads fluently but is unfaithful to a source or to the facts.1 The word covers invented references, wrong dates, fake quotations and confident answers about events that never happened.
Why fluent text can be false
A language model writes by estimating which word is likely to come next, then doing it again (How a Language Model Writes a Sentence shows the loop). Likely and true overlap often, but they are different things. A sentence can be statistically natural and factually wrong, and the model has no separate step that checks it against the world.
Two causes the research points to
In 2025, Kalai and colleagues argued that hallucinations are not mysterious.2 The first cause sits in training. A model must learn to tell valid statements from invalid ones, and when a fact cannot be told apart from a plausible falsehood, errors appear for the same reason a classifier makes mistakes. Nothing in the pattern of language tells a model which of two reasonable birthdays belongs to a little-known person, for example.
The second cause sits in how models are scored. Most benchmarks give a point for a correct answer and nothing for "I don't know", so a model that always guesses scores higher than one that admits doubt. The authors compare it with a student guessing on an exam, and they argue that the scoring, not only the models, has to change.2
Bigger is not automatically more truthful
The TruthfulQA benchmark asks 817 questions across 38 categories, including health, law, finance and politics, chosen because common misconceptions tempt a model to answer wrongly. The best model in the study was truthful on 58% of the questions, against 94% for people. The largest models were generally the least truthful, because they copied false claims that are common in their training text.3
Systems built since then add extra training and tools such as web search, which reduce the problem. The underlying habit of guessing has not gone away.2
A five-minute routine
- Give it the text. Paste the source and ask it to answer only from that text.
- Ask for quotations. A claim with a quotation you can find in the source is checkable. A claim without one is a guess.
- Check what can be checked. Look up every name, date, number, quotation and reference. Reference lists are where invented sources hide.
- Allow "I don't know". Telling the model it may say so can help. It is not a guarantee.
- Match the effort to the stakes. Skim a birthday message. Verify a medical, legal or school decision against a primary source.
What this does not mean
Hallucination does not make AI useless. It works well when you can see the source or judge the result: drafting, summarising text you supply, explaining an idea, writing practice questions. The right mental picture is a fast, confident assistant who has not checked anything.
In the course. Learning to work with that assistant, how to ask well, how to check the answers and how to use AI honestly, is a large part of the Neurovia One AI Course, which starts from the basics. See the course.
References
- Ji Z, Lee N, Frieske R, et al. Survey of hallucination in natural language generation. ACM Comput Surv. 2023;55(12): article 248. Find it ↗
- Kalai AT, Nachum O, Vempala SS, Zhang E. Why language models hallucinate. arXiv. 2025: 2509.04664. Find it ↗
- Lin S, Hilton J, Evans O. TruthfulQA: measuring how models mimic human falsehoods. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. 2022:3214–3252. Find it ↗