Why Do LLMs Hallucinate? A Student Who Can't Turn In a Blank Exam

Why do LLMs hallucinate?

The way I see it, you can picture an LLM as a student like this: he’s read a great many books, but his memory has been compressed, and he himself doesn’t know which parts have actually grown fuzzy; worse still, he’s been required to—during the exam—“never turn in a blank sheet.”

An infographic that uses a student, adrift in a sea of books and required to “leave nothing blank and answer every question,” as a metaphor for why LLMs hallucinate, listing six causes such as compressed memory and mixed-up sources

Its Strength Isn’t “Knowing Whether It Knows”

This student’s real strength isn’t judging whether he “actually knows,” but stringing words together fluently so that the answer looks like a reasonable one.

So when the material itself is vague, the sources are muddled, or the very first step of reasoning goes wrong, what follows may—for the sake of keeping a consistent tone—grow ever more complete, and ever more confidently wrong.

The Key Is to Narrow Its Room to “Patch Things In Recklessly”

For this reason, when using AI, the point isn’t just to toss out a line like “find me the answer.” It’s to find ways to narrow its room to “patch things in recklessly.”

A better approach is to:

structure the question, spelling out the background, the constraints, and the output format;
when there’s too much material, feed it in batches so it can process step by step;
and, more importantly, provide correct sources and require it to “answer according to the sources,” flagging anywhere it’s unsure.

Treat It as an Assistant That Needs a Clear Assignment

To put it another way: treat AI as an assistant that “needs to be given a clear question, reference materials, and a defined scope for its answer.”

The looser the question, the more easily it patches things in; the more clearly you frame the scope, the more likely it is to genuinely help.

In other words, if the person asking can design the question to be more like a “fill-in-the-blank” than a boundless “essay question,” then the answers AI gives will usually be more convergent, and closer to a more precise result.