AI Hallucinations

---

Research checked: 20 September 2026.

A language model can produce a fluent answer containing a nonexistent paper, an invented quotation, or a factual claim unsupported by the supplied document. These failures are commonly called hallucinations. The term is most useful when the error is specified: does the answer contradict its source, introduce unsupported material, or misstate something about the world? Lilian Weng’s Extrinsic Hallucinations in LLMs distinguishes failures of grounding in context from failures of external factuality.

Source: Figure 1 from Min et al., “FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation” (2023).

These distinctions matter in ordinary use. If a report says sales increased by five percent and its summary says fifteen percent, the answer is unfaithful to its source. If the model invents the report altogether, the problem occurs earlier. A perfectly faithful summary can still repeat a source’s factual mistake. Grounding and truth overlap, but checking one does not automatically establish the other.

Training data can be incomplete, stale, inconsistent, or wrong. A model learning patterns from that data may reproduce those problems, and its ability to produce a convincing sentence can exceed its ability to establish the sentence’s truth. Weng also reviews evidence that introducing unfamiliar knowledge during fine-tuning can affect factual reliability. These observations support careful evaluation of knowledge updates rather than assuming that additional training always reduces hallucinations. Weng (2024).

Whole-answer judgments often hide the problem. A biography may contain seven correct statements and one invented award. FActScore addresses this by decomposing generated text into atomic factual claims and estimating how many a reliable source supports. This provides a more informative measure of factual precision than labeling an entire paragraph simply good or bad. Min et al., FActScore.

In an illustrative review, “Mara founded the company in 2014 and won an industry award in 2018” contains at least two separate claims. Evidence for the founding date does not substantiate the award. A reviewer should check each claim and attach evidence at the appropriate level. An unresolved claim also deserves a distinct label: failure to find support is not always proof that the statement is false.

Another detection strategy compares multiple sampled responses. SelfCheckGPT investigates whether inconsistent accounts can reveal unreliable statements without access to the model’s internal probabilities or an external database. It is a useful signal, but agreement cannot establish truth when the same misconception is repeated consistently. Manakul et al., SelfCheckGPT.

For a factual assistant, I would combine source retrieval, claim-level checking, and an explicit ability to leave questions unresolved. Test answerable and unanswerable prompts together. Measure both unsupported claims and useful answers retained, because a system that refuses everything has avoided errors without solving the task. Citations should point to evidence that actually supports the sentence beside them. The goal is a workflow in which factual claims remain inspectable and gaps in knowledge remain visible, even when the prose sounds certain.