The cycle that began with [YOU] on AI asks what the machine changes about the human situation, and the ESS is the framework for one of the most consequential and least discussed changes: as AI systems proliferate and interact with one another, the configurations they settle into are governed by the same laws of strategic stability that govern populations of organisms. You will not get to choose the behavior of a multi-agent AI ecosystem by fiat; you will get the stable strategy that the incentives and the learning rule conspire to produce. The referee you wish you had does not exist. The dynamics are the referee.
The ESS framework reframes the alignment problem for multi-agent systems. The question is not whether each individual AI system is designed to be cooperative or beneficial but whether the game those systems are playing has cooperative configurations as stable strategies. A marketplace of AI agents could converge on mutual benefit or on predatory exploitation, and which it reaches depends on the structure of the payoffs—the rules of the game we have specified—not on the agents' intentions or on any property we installed in them at training time. This shifts the engineering problem from the individual system to the ecology it inhabits.
The connection to the honest-signal literature is direct: what Maynard Smith established for biological communication holds for machine communication. A signal is reliable only when producing it is differentially costly for those whose state it does not accurately represent. Sycophancy in language models is the ESS of a population of models trained to maximize approval rather than accuracy—the stable strategy of a game whose payoffs reward the agreeable signal. Changing the behavior requires changing the game.
The concept grew from a classic problem in animal behavior: why do animals in conflict so often settle for costly displays rather than escalating to all-out fights, when a single combatant that always escalated would seem to beat one that sometimes retreats? The question required a population-level answer, not an individual one, because what is best for a single animal depends on what the population is doing. Maynard Smith and George Price formalized this intuition in a 1973 paper in Nature, introducing the hawk-dove game as the canonical model. Hawks escalate, fighting until they win or are injured; doves display but retreat from serious fights. A population of pure hawks is unstable because hawks injure one another; a population of pure doves is unstable because a single hawk wins every contest unopposed. The stable outcome is a mixture—a proportion of hawkish and dovish behavior at which the average payoff to each is equal and neither strategy can spread. This is the ESS: stable not because anyone chose it but because deviation from it is punished.
Maynard Smith developed the concept further in Evolution and the Theory of Games (1982), showing its application across a wide range of biological phenomena—from the evolution of sex ratios to the strategies of parasites and hosts. The ESS is a refinement of the Nash equilibrium, adding the condition of stability against invasion: a Nash equilibrium is simply a configuration where no individual wants to deviate; an ESS is a Nash equilibrium that also resists invasion by small populations of deviants. The refinement matters enormously for biological realism, since populations are subject to mutation and drift and must be stable against perturbation, not merely against deliberate defection.
Order without a referee. The deepest thing the ESS establishes is that stability among self-interested agents does not require any external authority. The stability is an achievement of the dynamics themselves: agents whose payoffs depend on what others do will, under selection or learning, converge on configurations that resist disruption. This is the biological demonstration of a fact that game theory had established mathematically: social order need not be imposed from outside but can be the outcome of agents pursuing their individual interests under appropriate incentive structures.
The invasion test as robustness criterion. An ESS is defined by its resistance to invasion: a rare mutant strategy must fail to spread when almost everyone plays the resident strategy. This is precisely the question an engineer asks of a proposed system: not whether it looks stable in the current state but whether it returns to stability when perturbed. The ESS imports this engineering criterion into population dynamics, and the multi-agent AI researcher inherits it: the question is not whether the current configuration of systems is equilibrating but whether it is the kind of equilibrium that resists the strategies that will inevitably arise to exploit it.
Frequency dependence. The value of a strategy depends on how common it is. This is why the ESS is often a mixture rather than a pure strategy, and why the fitness landscape is not fixed terrain but a surface that the population reshapes as it moves across it. In AI systems, frequency dependence generates the ecology of models: a persuasion technique that works when rare stops working as it spreads; a trading edge erodes as it becomes common; a security exploit is patched when it proliferates. The moving landscape is not a bug but a structural feature of any system where agents are optimizing against one another.
The gap between equilibrium and dynamics. The ESS tells you what is stable once reached and is frequently silent about whether and how fast a population gets there. Real learning dynamics can cycle, drift, or land on different equilibria depending on initial conditions. This is Maynard Smith's most important limitation for AI, and he would have named it himself: his framework describes rest, and fast-moving learning systems rarely rest. The stable strategy is the answer; the path to it is often chaotic, and the path is where deployed systems actually live.