Hallucination

Ask an AI about a real court case and it might give you a thorough, well-written answer - case name, year, ruling, legal reasoning. It might also have invented every single detail. The answer sounds authoritative. The case does not exist. This is an AI hallucination, and it is one of the most important things to understand about how these systems work.

The reason this happens is tied directly to how AI models are built. They learn by reading enormous amounts of text and getting very good at predicting what words should come next, given the words that came before. This makes them excellent at generating fluent, plausible-sounding responses. The problem is that "plausible" and "accurate" are not the same thing - and the model was never directly taught to tell the difference.

When a model does not know something, it does not naturally say "I do not know." Instead, it generates the most statistically likely continuation of your prompt, which often sounds like a confident, detailed answer. There is no internal alarm that goes off when the model crosses from fact into fabrication. It just keeps generating the next word, and the next, with the same confident tone throughout.

This problem is especially dangerous with specific details: exact dates, precise numbers, names of people, citations to studies or legal cases. These are exactly the things people most need to get right, and they are also exactly the things most likely to be confidently wrong. A hallucinated statistic in a business report or a made-up legal precedent in a brief can cause serious damage.

Techniques like giving the AI real documents to read before answering, asking it to cite specific sources, and always having a human review important outputs all help reduce the risk. But no approach available today eliminates hallucinations entirely. Any time you use AI for something where accuracy really matters, the output needs to be checked - either by a person or by another automated system.