CONCEPT
The Performance-Competence Gap
Rodney Brooks’s diagnosis of the most reliable source of overestimation in artificial intelligence: the systematic inference, calibrated for judging humans, that impressive performance on a demonstrated task reflects broad general competence—an inference that fails completely when applied to machines.
The performance-competence gap names the chasm between what a machine demonstrates it can do and what it is actually capable of across the full range of situations the demonstration implies. When a human performs impressively on one task, we reliably infer a great deal about what else they can do: skills come bundled in humans in predictable ways, so a person who can read a menu can read a novel. Rodney Brooks identified that this inference—finely tuned by evolution and experience for judging other humans—fails completely when applied to machines. A system that performs impressively on a curated benchmark may have near-total incompetence on adjacent tasks that a human performer would obviously also master, because machine competence does not generalize the way human competence does. The instinct to assume otherwise is, in Brooks’s accounting, the single most reliable source of overestimation in the entire history of
artificial intelligence. Every major hype cycle he has witnessed, from