CONCEPT
Minimax Robustness
Abraham Wald’s worst-case decision principle applied to AI safety: the insistence that systems operating in safety-critical domains must be evaluated against their worst-case behavior rather than their average-case performance, and that a favorable expected outcome provides no protection when the tail contains catastrophe and the experiment is run once.
When you do not know which state of the world you face, and the worst case is catastrophic and unrecoverable, the rational decision is not to optimize for the expected outcome—it is to minimize the maximum harm.
Abraham Wald formalized this intuition as the minimax principle in the 1940s, drawing on von Neumann's game theory to cast statistical decision-making as a zero-sum game in which nature, playing adversarially, will select the state of the world that maximizes the decision-maker's loss. The decision-maker's rational response is to choose the action whose worst possible outcome is least bad—to minimize the maximum loss. This is not pessimism. It is the decision theory of the single irreversible draw: the setting in which you do not get to average over multiple trials, in which being right on average provides no protection if the single draw is catastrophic. Minimax robustness is the extension