What is an AI hallucination?
An AI hallucination is output from an AI model that sounds confident and plausible but is factually wrong or invented, such as a fake citation, an incorrect figure or a policy that does not exist.
Language models are built to produce likely-sounding text, not to check facts. When they lack the right information, they often fill the gap with something that reads well and happens to be false. The danger is the tone: a made-up statute or an invented warranty term arrives in the same confident prose as a correct one, with no warning sign attached.
Courts have sanctioned lawyers for filing briefs with citations an AI tool invented, which is part of why ABA Formal Opinion 512 stresses that lawyers remain responsible for checking AI output. The same principle applies in every field: a person accountable for the work verifies anything factual before it reaches a client, a court, a patient or a regulator.
You can lower the rate but not reach zero. Ground the model in your own documents, ask it to cite the passage it used, tell it to say when it does not know, and keep tasks narrow. Then measure: sample outputs weekly and track a correction rate. If corrections stay high, the task may not suit AI yet.
Example
For example, a plumbing company's website chat assistant might be asked whether a water heater repair is covered under warranty. Without access to the actual warranty terms, it could invent a coverage period. A safer design limits the assistant to approved FAQ content and hands warranty questions to the office, with the customer's details already captured.