CONCEPT
Complementary Learning Systems
McClelland, McNaughton, and O’Reilly’s 1995 neuroscientific theory explaining why the brain needs two distinct memory systems—a slow, distributed neocortical learner and a fast hippocampal store—and why the absence of the second is the deepest structural limitation of current large language models.
The most embarrassing failure of early
connectionist networks was not the limitation Minsky and Papert had proven but a different one, discovered from inside: catastrophic interference. Train a distributed network on one set of examples, then train it on a new set, and the new learning overwrites the old. The very property that makes distributed learning powerful—overlapping, superimposed weights that let the system generalize—is the property that makes it catastrophically forgetful.
McClelland, McNaughton, and O’Reilly turned this failure into an explanation of brain architecture. In a 1995 paper in
Psychological Review that became one of the most cited in cognitive neuroscience, they proposed that the brain solves catastrophic interference by running two complementary systems: the neocortex, a slow learner that extracts statistical structure through gradual interleaving of experience, and the hippocampus, a fast learner that captures new events rapidly in sparse, non-overlapping form that does not interfere with existing knowledge. New memories are