Precautionary Incrementalism
Precautionary incrementalism is the modification of the standard method appropriate to the boundary conditions where its assumptions strain. Standard incrementalism relies on error correction: try something, observe consequences, fix what went wrong. The method works when errors are reversible. It fails when they are not. Some AI governance decisions produce consequences that cannot be undone because the damage completes before the feedback arrives. For these high-stakes domains, precautionary incrementalism biases early iterations toward constraint rather than permissiveness, accepting suboptimal outcomes on some dimensions in exchange for preserving the option to do better later.
The Ratchet of Permanent Restriction — Contrarian ^ Opus
There is a parallel reading that begins from the political economy of precautionary decisions. Once constraints are institutionalized—especially those justified by protecting children or preventing catastrophe—they rarely loosen. The mechanism isn't the prudent adjustment described here but regulatory capture by those who benefit from constraint: incumbent firms who use safety requirements to raise barriers to entry, professional groups who gatekeep through certification requirements, bureaucracies whose budgets depend on the persistence of the threat they manage. The "option to do better later" becomes a fiction when the constituencies that form around initial constraints have every incentive to preserve them.
The child development example reveals the deeper problem. Who determines what constitutes appropriate AI exposure for children? The answer in practice will be some combination of academic experts whose careers depend on problematizing technology, tech companies who benefit from age-verification systems that smaller competitors cannot afford, and middle-class parents whose cultural anxieties about screen time become universalized as developmental science. The actual children most affected—particularly those from lower-income families who might benefit most from AI educational tools—have no voice in these determinations. The "modest precautionary judgment" becomes a vehicle for reproducing existing inequalities under the guise of protecting development. The real irreversibility isn't the cognitive damage from too much AI exposure but the opportunity cost for children denied access to transformative tools during their formative years. The framework assumes we can identify irreversible domains objectively, but in practice, every interest group claims their concern is the irreversible one.
In the AI Story
The domains where the modification is indicated are identifiable. Autonomous weapons systems that make lethal decisions faster than human deliberation can operate. AI in critical infrastructure — power grids, financial systems, communications networks — where system failures cascade at machine speed. The reshaping of children's cognitive development during windows of neurological plasticity that, once closed, do not reopen. In each domain, the standard reliance on error correction encounters a boundary: the error itself may be unacceptable, not because incrementalism undervalues its severity but because the error destroys the conditions under which the next increment of learning could occur.
The approach is not comprehensive planning. It does not require modeling the full trajectory of AI development or predicting consequences across all dimensions. It requires the narrower judgment that in domains where consequences are irreversible, the method's standard risk tolerance must be reduced. This is targeted caution, not synoptic analysis. It accepts uncertainty about most consequences while insisting on constraint in the specific domains where consequences cannot be corrected.
The application to children's cognitive development illustrates the logic. The concentric approach to attentional ecology cannot wait for organizational policies, sectoral standards, and regulatory frameworks to accumulate through iterative cycles — the child's cognitive development is happening now. The window for certain kinds of developmental experience is biologically finite. The response is not comprehensive understanding of cognitive development in AI environments but the modest precautionary judgment that, in conditions of uncertainty about irreversible consequences, erring on the side of caution is prudent. A child whose AI access is overly restricted can be given expanded access later; a child whose capacity for sustained attention has been compromised may not recover what was lost.
The method remains incremental. It is still iterative. It is still empirical. It is still revisable. But the increments are conservative in domains where errors are irreversible and bolder in domains where errors can be corrected. The approach discriminates between contexts based on the asymmetry of risks — a discrimination that comprehensive planning claims to make analytically but that, in practice, is made through the political interaction of parties who disagree about which risks are most severe.
Origin
The concept appears in the Lindblom volume's sixth chapter as one of four boundary-condition modifications to standard incrementalism. It draws on the broader precautionary principle tradition in environmental and public-health governance but insists that precaution operate within the incrementalist framework rather than as an alternative to it.
Key Ideas
Asymmetric risk tolerance. Domains are treated differently based on whether their errors are reversible.
Constraint before full understanding. Early iterations accept constraint without waiting for comprehensive understanding of consequences.
Preserved optionality. Over-constraint is preferred to under-constraint when the option of later expansion exists and the option of later reversal does not.
Narrow application. The modification is targeted, not universal. Applying it everywhere would produce the paralysis that its critics rightly warn against.
Appears in the Orange Pill Cycle
Domain-Specific Risk Geometries — Arbitrator ^ Opus
The right weighting depends entirely on which question we're asking about precautionary incrementalism. On the existence of genuinely irreversible domains, Edo's view is essentially correct (95%). Autonomous weapons and childhood development windows do represent fundamentally different risk geometries than reversible policy experiments. The contrarian's political economy critique doesn't negate this physical and biological reality.
But when we ask who decides what counts as irreversible and how constraints get implemented, the contrarian view dominates (75%). The history of precautionary regulation does show persistent ratcheting, and the constituencies that form around safety measures do resist loosening even when evidence suggests they could. The child development example is particularly telling—the science is genuinely uncertain, but the political dynamics strongly favor restriction over access, especially along class lines.
The synthetic frame that holds both views might be "provisional constraint with sunset mechanisms." Instead of assuming we can preserve the option to loosen restrictions later through political will alone, build in automatic expiration and renewal requirements that force active decisions to maintain constraints. Make the cost of restriction visible by requiring those who benefit from AI access to subsidize alternatives for those who are restricted. Most importantly, distinguish between precaution about genuine physical irreversibilities (where Edo's framework applies fully) and precaution about social or developmental irreversibilities (where both the science and politics are more contested). The framework needs not just discrimination between reversible and irreversible domains, but recognition that irreversibility itself is often a contested and political determination.
Further reading
- Aaron Wildavsky, Searching for Safety (1988) — the opposing view
- Jonathan Haidt, The Anxious Generation (2024) — on children's cognitive development