A hallucination in AI is a situation in which a language model delivers false or invented information, worded in a way that sounds plausible and confident. OpenAI describes it as plausible but false statements. It is not a rare glitch but a feature of how models are built and evaluated.
In short
- A hallucination is an answer that sounds right and isn't: a name, number, source or detail the model made up.
- According to OpenAI, there are two main causes: arbitrary facts that can't be learned from text patterns, and evaluation methods that reward guessing over "I don't know".
- Hallucinations can't be eliminated completely, but they can be reduced: provide a source, let the model say it doesn't know, and check.
- In a business, the rule is to keep human approval over anything that is published, sent or costs money.
The illustration: a student who won't leave a blank
Picture an exam in which a wrong answer and a blank answer both score zero. What will a smart student do? Always guess. In the best case they're right, and they have nothing to lose.
OpenAI argues that this is exactly what happens to models. In its research it gives an example: a model guessing a person's birthday might succeed with a chance of one in 365, while an answer of "I don't know" is guaranteed to score zero. And in tests that measure only accuracy, the guesser looks better than the careful one.
Why it happens: two causes
First cause: training
No pattern to learn
- The model learns patterns from text
- Spelling and punctuation are a stable pattern
- An arbitrary fact, like a person's birthday, has no pattern
Second cause: measurement
Guessing pays off
- Most evaluations count only correct answers
- "I don't know" gets zero, like an error
- A model that guesses ranks higher
OpenAI's conclusion is that the evaluation methods also need to change: to reward calibrated expression of uncertainty, not false confidence. There is no promise here that the problem will disappear, and OpenAI itself notes that hallucinations remain a challenge for all language models.
What does a hallucination look like in a business?
A promise that doesn't exist
- Telling a customer about a returns policy you don't have
- A discount nobody approved
An invented detail in content
- A "study" or figure with no source
- A product spec that doesn't match the catalog
A wrong number in a report
- A summary that fills in a missing figure
- A comparison to a period that wasn't checked
A source that doesn't exist
- A link or quote that looks real
- The name of a law or regulation that wasn't checked
How to reduce hallucinations
Anthropic publishes practical techniques in its documentation:
- Allow "I don't know". State explicitly in the instruction that if information is missing, it can say so.
- Ground it in quotes. For a long document, first ask for word-for-word quotes, and only then an analysis that relies on them.
- Require a source for every claim. And if no quote supports a claim, delete it.
- Limit it to the material you provided. Instruct it to use only the attached documents and not general knowledge. This is also how RAG works.
- Check twice. Run the same request several times and compare. Contradictions are a warning sign.
Anthropic itself adds that the techniques reduce hallucinations significantly but don't eliminate them, and that critical information should be verified, especially for important decisions.
Before you publish or send AI output
Common myths about hallucinations
Myth"The model is lying"
In practiceLying requires intent. The model doesn't intend to mislead, it produces a plausible continuation, and sometimes the plausible one isn't true.
Myth"A new, advanced model doesn't make things up"
In practiceModels improve, but hallucinations don't disappear. Anthropic's documentation also says the techniques reduce and don't eliminate.
Myth"You can spot a hallucination by its style"
In practiceA hallucination sounds exactly like a correct answer, in a confident tone. You check against a source, not by feel.
Myth"If you provide documents, there are no more hallucinations"
In practiceIt helps a lot, but the model can still misinterpret or fill in what's missing. So you keep demanding a source and checking.
Related concepts
- LLM: the model that writes the answer
- RAG: giving the model the right document
- AI agent: a model that carries things out, with approval
- Computer Use: when the model acts on a screen
- Prompt injection: when content misleads the model
Platin Digital is an OpenAI Select Partner. In the agents we set up, content for publishing, a message to a customer and a budget change always wait for your approval, for the reason described here.
Frequently asked questions
What is a hallucination in AI?
A hallucination in AI is a situation in which a language model delivers false or invented information in plausible, confident wording, such as a name, figure or source that doesn't exist. According to OpenAI, it stems partly from arbitrary facts not being learnable from text patterns, and from evaluation methods that reward guessing.
Why does ChatGPT make things up?
Because it was built to produce a plausible continuation of text, not to check facts against a database. When it lacks information, the common evaluation methods encourage it to guess instead of saying it doesn't know. The more you provide it with a source, as in RAG, the smaller the chance of error.
Can hallucinations be prevented completely?
No. They can be reduced by allowing "I don't know", quotes, requiring a source and limiting it to the documents provided. Both Anthropic and OpenAI note that the problem remains, so critical information is checked.
How do you stay safe when using AI in a business?
You divide the work: the model prepares a draft, and a person approves everything that is published, sent to a customer or touches money. You check numbers and sources, and start with tasks where a small error isn't costly. More in the explainer on AI agents.
