CONCEPT
Jagged Intelligence
Andrej Karpathy’s adjective for the uneven capability profile of large language models—brilliant on some peaks, startlingly weak in adjacent valleys, with no smooth surface connecting them and no representative point from which a single test can tell you about the whole.
The intelligence of
large language models is not uniform. It does not rise and fall together the way human competence tends to. A model can solve a problem that would stump a graduate student and then fail at a task a child would find trivial, sometimes within the same conversation.
Andrej Karpathy calls this jagged intelligence, and the word does real work: it corrects two symmetrical errors people make when they encounter these systems. The first error extrapolates from the peaks—watching a model write an elegant essay and concluding that something approaching a mind is clearly present. The second extrapolates from the valleys—watching a model miscount letters in a word and concluding that the whole thing is a parlor trick. Both errors assume the intelligence is uniform, that a single sample tells you about the whole. Karpathy’s point is that neither the peaks nor the valleys are representative, because there is no representative point. The