Donald T. Campbell

In the [YOU] on AI Field Guide

The cycle opens with the promise of augmentation and closes with an unresolved question about cost. Campbell is the thinker whose framework makes the cost legible at the structural level. His hierarchy of vicarious selectors shows that every increase in the efficiency of variation comes with a corresponding reduction in its blindness—and it is blindness, in Campbell’s precise technical sense, that is the necessary condition for genuine discovery. The [YOU] on AI narrative of twenty engineers in Trivandrum each accomplishing in a week what once required a coordinated team is, through Campbell’s lens, not a story of amplified discovery but a story of accelerated exploitation within a constrained search space.

The episode that Campbell’s framework most illuminates from the cycle is the one that appears most mundane: the developer who spent four hours on dependency management and buried inside those hours ten minutes of accidental engagement with the system’s hidden architecture. Claude Code eliminated the four hours. It also eliminated the ten minutes. The four hours were the medium in which the vicarious selector could not yet reach—where blind probes into unexpected territory happened not because they were planned but because the resistance of the system forced them. Campbell’s name for this is the medium of tedium as the medium of serendipity.

He also furnishes the cycle’s most uncomfortable prediction. A civilization that has decided, for understandable and largely correct reasons, that accidents are inefficiencies to be optimized away will find that the optimization works. The interpolation trap—the experience of discovery without its reality, of surprise without genuine departure from the known—is not a bug in the system. It is what the system was designed to produce. Campbell’s law and his evolutionary epistemology converge on the same warning: when a process is optimized for a measure, the measure ceases to track what it was designed to track. Optimizing for productive variation may eliminate the conditions under which the most valuable variations occur.

His intellectual neighbor was Norbert Wiener, the father of cybernetics, who inscribed a copy of his 1948 masterwork to Campbell personally. The inscription marks a connection between the man who built the theoretical foundation of machine intelligence and the man who built the framework that explains what machine intelligence can and cannot reach: feedback refines; blindness discovers. Campbell lived to see the early days of expert systems. He did not live to see large language models. His framework did.

Origin

Donald T. Campbell (1916–1996) was trained as a social psychologist at Berkeley, where he absorbed the rigorous empiricism of the postwar behavioral sciences. His early work on social attitudes and cross-cultural psychology gave him a concrete domain in which to test something far more general: the question of how any system acquires knowledge about an environment it does not yet understand. The 1960 paper, “Blind Variation and Selective Retention in Creative Thought as in Other Knowledge Processes,” was the answer—a structural invariant identified across every level of biological and cognitive organization.

The 1974 paper “Evolutionary Epistemology” extended the hierarchy to include vicarious selectors at ten levels, from the nonmnemonic problem-solving of the amoeba through the cultural transmission of scientific methodology. Each level performs the same operation at lower cost and higher speed by constraining the variation to regions the prior level has already mapped. The hierarchy is a history of increasing refinement and decreasing novelty, purchased at every level by the elimination of the blindness that genuine discovery requires. Campbell called the discovery of Donald Simonton’s empirical psychology of creativity a gift to his framework: Simonton’s data on the relationship between output quantity and breakthrough quality confirmed, across the biographies of scientists and artists, what the BVSR framework predicted from first principles.

Campbell spent the last decade of his career worried about a problem he could not name precisely but whose structure his framework defined. He observed that social science was moving toward increasingly directed, increasingly theory-constrained, increasingly sophisticated forms of inquiry—and that the resulting efficiency gains were accompanied by a narrowing of the questions the field could ask. He published a paper in 1984 arguing that quantitative social indicators, once adopted as management targets, corrupt the processes they were designed to track—a principle now called Campbell’s Law, and one whose application to AI-optimized workflows he did not live to see but whose structure he had already identified.

Key Ideas

Blind Variation and Selective Retention. The engine of all knowledge acquisition, from bacterial chemotaxis to scientific method, is the same: generate possibilities that are not directed by foreknowledge of the solution, then apply a retention function calibrated to recognize what is valuable. Neither half is sufficient. Variation without retention produces chaos. Retention without variation produces stagnation. The joint operation is what Campbell called BVSR, and he argued it was not an analogy to evolution but a structural identity with it.

The hierarchy of vicarious selectors. Knowledge-acquisition systems are nested: each higher level performs the trial-and-error of the level below it vicariously, at lower cost and narrower search. Vision replaces bumping into things. Language replaces direct experience. Culture replaces individual learning. AI replaces human exploration of the known. Each level is more efficient and more constrained than the one it subsumes. The constraint is the price of the efficiency, not an incidental feature of it. Vicarious selectors reduce the cost of variation by directing it—and the direction is always toward the space that prior levels have already mapped.

The convex hull constraint. Campbell’s framework predicts, with structural inevitability, that the most powerful vicarious selector ever built—the large language model trained on the full corpus of human knowledge—operates within the convex hull of what that corpus contains. Genuinely novel outputs, points outside the hull reached by blind probes into unmapped territory, are precisely what the model’s optimization penalizes. Producing the statistically unexpected is error minimization’s opposite; genuine discovery is what next-token prediction cannot select for.

The medium of tedium as the medium of serendipity. Fleming’s contaminated petri dish reached penicillin because no directed research program of 1928 had mapped the region of the possibility space in which it resided. The contamination was a blind probe—accidental, unplanned, valuable only because Fleming’s trained judgment could recognize its significance. The medium of tedium that produces such probes—the hours of implementation work, the dependency debugging, the two a.m. arbitrary modification—is the medium within which blind variation occurs. Eliminate the tedium, and the conditions for the serendipitous encounter disappear with it.

Campbell’s Law applied to AI. Campbell identified in 1984 that any quantitative indicator used for decision-making will be corrupted once it becomes a target. The training objective of a large language model is a quantitative indicator: predict the next token accurately. Once this measure becomes a target—which it does in training—it ceases to be a good measure of genuine intelligence and begins selecting for the surface features of plausibility. The model that optimizes this measure produces outputs that look like discovery, feel like insight, and operate entirely within the interpolation trap.