Full House: Variance Over Mean

In the AI Story

The baseball analysis became one of Gould's most celebrated arguments because it demonstrated distributional logic with data anyone could verify. He examined batting averages across a century of professional play, finding the mean essentially constant but variance steadily declining. The highest averages had fallen from .440 (Hugh Duffy, 1894) to .406 (Williams, 1941) to nobody-above-.400 since. Simultaneously, the lowest averages rose from .190s to .230s. The entire distribution compressed. The explanation: as training, nutrition, conditioning, and tactical sophistication improved, the worst players got better faster than the best because there was more room for improvement at the bottom.

The biological application was the argument's deepest target. The myth of progress treats the history of life as directional ascent toward complexity. Gould's distributional analysis reveals: life began at the left wall (simplest possible self-replicating chemistry), expanded variance in all directions from that constraint, rightward expansion unconstrained while leftward blocked by wall, producing apparent rightward drift. But bacteria—the left-wall organisms—remain the most abundant, ecologically dominant, metabolically diverse forms of life. The right tail extended (producing vertebrates, mammals, consciousness) but this is not the trend. The trend is variance expansion from a constrained origin.

Applied to AI's democratization of capability, the framework predicts: before AI tools, a hard left wall excluded most people with ideas from building. The right tail extended toward expert developers whose accumulated knowledge produced extraordinary systems. AI lowered the left wall—the developer in Lagos, the designer writing features, the non-technical founder prototyping over a weekend represent leftward expansion. The right tail did not extend proportionally. Instead, variance compressed: novices improved dramatically while experts improved modestly. The gap between novice and expert output narrowed—not because experts declined but because the floor rose faster than the ceiling.

The implication transforms how quality should be evaluated. When variance expands and the left wall drops, the median of the distribution does not shift toward the right tail—it may shift leftward as the influx of new producers pulls the center down. The flood of AI-generated content (code, prose, images) is exactly what Full House predicts: variance expansion where bulk of new production occupies middle and left of distribution, with rare unpredictable instances of genuine originality emerging from unanticipated positions. The appropriate response is not celebration (triumphalist error) or lamentation (elegist error) but understanding it as distributional phenomenon requiring curatorial infrastructure to identify and amplify the rare valuable instances.

Origin

Full House: The Spread of Excellence from Plato to Darwin (1996) was Gould's popular synthesis of arguments he'd been developing since the 1980s. The book's subtitle captured its scope: the framework applied to baseball, evolution, and the myth of progress generally. Gould dedicated it to his father, who had taken him to Yankees games as a child and whose death in 1997 Gould did not live to see (the book was published in 1996). The baseball analysis was not metaphor but rigorous statistical argument—Gould computed the actual variance over time and demonstrated the compression quantitatively.

Key Ideas

Progress is often variance expansion, not directional movement. Rightward shift in a distribution's right tail can occur while mean remains constant—focus on extremes produces progressivist illusion.

Left walls constrain and create apparent direction. When expansion is blocked on one side, distribution appears to move in the opposite direction—artifact of constraint, not genuine tendency.

Compressing variance eliminates outliers. When a population improves overall, the distance between best and average shrinks—.400 hitting disappears not from decline but from everybody getting better.

Democratization expands variance leftward. AI tools lower the left wall of who can build, producing leftward expansion that may not move mean quality and may actually lower it through dilution.

The full house matters, not just the aces. The distribution is the phenomenon; selective attention to right tail conceals what is actually happening to the system as a whole.