Chance and the Specificity of the Moment

In the AI Story

The mechanisms of chance are specific and well-characterized. Genetic drift — the random fluctuation of allele frequencies in finite populations — produces evolutionary change independent of selection. In small populations, drift can overwhelm selection entirely, fixing neutral or even mildly deleterious alleles simply because the population is too small for selection to operate reliably. The founder effect — the genetic consequences of a population being established by a small number of individuals — can determine the evolutionary trajectory of a lineage for thousands of generations.

These chance processes are not errors in the evolutionary mechanism. They are features of it. They are part of the explanation for why the history of life has the specific character it has — why certain lineages persisted and others vanished, why certain adaptations appeared in certain lineages and not others, why the biosphere has this particular configuration and not one of the billions of other configurations that were equally possible.

The relevance to the AI moment is more than analogical. The transformer architecture, introduced in 2017, was not the only possible architecture for language modeling. It succeeded because it scaled well with available hardware, because self-attention proved effective at capturing long-range dependencies, and because the engineering culture of specific organizations happened to prioritize the combination of data scale, compute scale, and architectural simplicity that transformers reward. These are real reasons. They are also specific reasons, embedded in a particular historical context that could have been different.

The training data is itself a product of chance. The corpus of text on which large language models are trained is a biased, contingent collection — predominantly English, predominantly Western, predominantly from the period of the internet's existence. The model's capabilities and limitations reflect this corpus, which reflects specific historical accidents about which texts were digitized. The twenty-fold productivity multiplier Segal's team experienced is a real measurement under specific conditions. Whether the same multiplier obtains under different conditions — different tools, teams, organizational contexts, years — is an empirical question the specific measurement cannot answer.

Origin

Mayr's emphasis on chance matured through his engagement with population genetics in the 1940s and was sharpened in his debates with Stephen Jay Gould in the 1980s and 1990s. Though the two disagreed on much, they agreed on the fundamental role of contingency in shaping evolutionary outcomes, and both argued that any framework projecting evolutionary trajectories forward confidently was ignoring the evidence of how past trajectories actually unfolded.

Key Ideas

Selection operates on a contingent substrate. Even where selection is directional, the variation it operates on is produced by chance, and the specific alleles available at a given time depend on historical accidents.

Drift overwhelms selection in small populations. The smaller the population, the larger the role of chance in determining which alleles fix. Many evolutionary outcomes are drift, not selection.

Founder effects lock in history. The genetic composition of a founding group constrains what selection can subsequently produce — sometimes for thousands of generations.

AI systems are historically specific. The architectures, training data, reward models, and fine-tuning sequences that produced today's AI are contingent on specific decisions by specific organizations at a specific moment.

Plan for the present with rigor, for the future with humility. Generalizations about current AI should be held as descriptions of current conditions, not predictions about future ones.

Debates & Critiques

Adaptationists, particularly in the tradition of Richard Dawkins, have sometimes accused Mayr and Gould of overstating the role of chance and understating the power of selection to produce optimized outcomes. The empirical record has largely vindicated Mayr and Gould: molecular evolution, in particular, shows extensive neutral variation unexplainable by selection alone, confirming that drift is a major force in shaping the genome.