The Building Blocks Hypothesis

In the AI Story

Holland's insight was that the combinatorial mathematics of building block recombination provided the only plausible mechanism by which adaptive systems could navigate search spaces larger than the number of atoms in the observable universe. Brute-force search is impossible for problems of even modest complexity. Rule-based programming requires designers who understand the solution space, but the most interesting problems are precisely those where no one understands the space well enough to program a solution. Building block recombination bypasses both limitations by letting the system discover which components are useful through the feedback of fitness evaluation, and by letting useful components propagate through populations faster than random search could ever explore them.

The framework illuminates what LLMs actually are, viewed through Holland's lens. Training data is a compression of building blocks: every argument, metaphor, narrative structure, logical pattern, and rhetorical move committed to text across human history. These are not stored as retrievable units — the model is not a database — but compressed into statistical regularities that constitute the system's internal representation. When the model generates text, it recombines building blocks under the selection pressure of the user's prompt. This is structurally identical to what Holland described in genetic algorithms.

The hypothesis has immediate implications for the democratization argument that The Orange Pill advances. When civilization's linguistic building blocks become accessible for a hundred dollars a month, the developer in Lagos gains access to the same combinatorial space as the engineer at Google. Not the same salary, not the same institutional support, but the same raw material for emergent solutions. The leveling is real and also partial — the quality of emergence depends on both the richness of building blocks (which the model provides) and the sharpness of selection (which the human must provide).

Holland's 1986 paper 'Escaping Brittleness' made the connection to AI explicit. Expert systems failed because they did not discover building blocks — their rules were hand-coded, fixed, incapable of recombination. They could not adapt to novel situations because they had no mechanism for generating novel combinations of existing knowledge. Thirty years later, large language models achieved what Holland was reaching for through a different technical mechanism but with the same structural logic. The intelligence emerges from recombination, and the recombination operates on building blocks the system discovered rather than ones any designer specified.

Origin

The hypothesis appears in its mature form in Holland's 1975 book but traces back to his doctoral work at Michigan in the late 1950s, where he developed the first computer science PhD program in the United States. Its mathematical elaboration in the schema theorem — which describes how short, low-order patterns propagate through populations under selection — became the theoretical backbone of the genetic algorithm field.

Holland's insistence that the hypothesis applied across substrates — from DNA to economic institutions to cognitive schemas — was initially treated as overreach. By the 1990s, as the framework demonstrated its productivity across domains, the generality was increasingly recognized as the source of its power.

Key Ideas

Combinatorial explosion, combinatorial rescue. Direct search is impossible; building block recombination is tractable and productive.

Context-dependent contribution. A building block that contributes nothing in one context may be essential in another; usefulness is relational.

Schema theorem. Short, low-order patterns with above-average fitness propagate exponentially under selection — the mechanism by which adaptive systems learn.

Substrate independence. The same structural logic operates in biological evolution, immune systems, economies, and language models.

Access versus selection. Equal access to building blocks does not produce equal outcomes; the quality of emergence depends on the quality of the selection mechanism applied to them.

Debates & Critiques

The schema theorem has been subject to decades of formal critique. Critics demonstrated that the theorem holds rigorously only for infinite populations and cannot, in its original form, distinguish between problems where genetic algorithms work well and problems where they do not. Defenders argued that the theorem describes a tendency rather than a guarantee, and that its predictive utility in complex domains — including the emergent behavior of large language models — vindicates the underlying insight even where the mathematics must be refined.