The Jagged Intelligence of LLMs: Why AI Is a Genius One Moment and a Fool the Next
The same LLM can, on one hand, methodically refactor a hundred thousand lines of code, and on the other, offer advice as absurd as “walk over to the car wash to wash your car.” This enormous gap in capability is what’s called “jaggedness.”
The Key Isn’t Difficulty, but “Whether It’s Been Practiced”
What’s interesting is that the key isn’t “whether humans find the task hard or easy,” but whether the task falls within the model’s training-data distribution.
Andrej Karpathy—a founding member of OpenAI and former director of AI at Tesla—points to two main causes:
- Verifiability: code has a clear right-or-wrong signal—does it compile, do the tests pass—which makes it easy for reinforcement learning (RL) to train and reinforce over and over.
- Economic factors: capabilities with commercial value get prioritized by frontier labs and packed into the training-data distribution; the common sense of everyday life, meanwhile, tends to get overlooked.
Smooth on the Track, Hacking Away the Moment It Strays
The result is this: when the model stays on the track that reinforcement learning has paved, it flies fast and smooth; but the moment it strays off the track, it’s like someone hacking wildly through a jungle with a machete—on the surface it looks as though it’s diligently getting things done, but what it’s actually doing is utterly absurd.
So to truly use an LLM well and avoid stepping on landmines, you have to build a “sufficiently accurate” mental model of its capabilities: knowing which tasks keep it on the track, and which tasks have actually led it into the jungle.