CONCEPT

Burden of Proof Asymmetry

The structural imbalance in production environments by which the party wishing to proceed bears no special evidentiary burden while the party wishing to stop must produce compelling novel evidence — the mechanism that makes reasonable exceptions rational and their accumulation catastrophic.

Vaughan identified burden of proof asymmetry as the institutional structure that converts production pressure into decision-level distortion. At NASA, the engineer who wished to proceed could point to the accumulated record of successful flights, the engineering analyses classifying anomalies as within acceptable limits, and the production schedule that rewarded forward motion. The engineer who wished to stop had to demonstrate, with quantitative evidence compelling enough to override the record, that the specific conditions of this launch exceeded the established limits. The asymmetry was not a policy; it was a feature of the institutional environment, as ambient and invisible as the air in the room.

In The You On AI Field Guide

The asymmetry is structural rather than designed — no one decided that stopping should require more evidence than proceeding; the structure emerged from the institutional reality that proceeding produces visible measurable outputs while stopping produces invisible unmeasurable protections.

The AI transition has reproduced the asymmetry with particular severity. The developer who deploys AI-generated code after functional testing points to track record; the developer who wishes to conduct comprehensive review must justify the delay against visible competitive costs while the risk of proceeding remains speculative. The evidence against proceeding is, by its nature, harder to produce than the evidence for proceeding, because the evidence against is predictive while the evidence for is historical.

Aviation safety reform since the 1970s has partially addressed the asymmetry through crew resource management: any crew member who sees a risk is empowered to stop the operation, shifting the burden to those who wish to continue. This redistribution required decades of cultural change following multiple disasters and near-disasters.

In AI-augmented work, the redistribution is harder because production pressure has migrated inward. The engineer cannot empower herself to stop herself; the institutional structure that would need to impose the pause is the same structure being shaped by practitioners who do not want to pause. The conventional reform mechanism — empowering individuals to stop — does not function when the individual is both the one who needs to stop and the one who wants to proceed.

Origin

The concept was implicit in Vaughan's Challenger research and was formalized through her subsequent theoretical work on institutional decision-making. The asymmetry's operation has been documented across industries including healthcare (stop-the-line authority in hospitals), aviation (crew resource management), and nuclear power (stop-work authority).

Key Ideas

Structural, not designed. The asymmetry emerges from the visibility gap between the outputs of proceeding and the protections of stopping.

Predictive versus historical. Evidence against proceeding is predictive and harder to produce; evidence for proceeding is historical and accumulates automatically.

Partially addressable. Aviation and healthcare have reformed the asymmetry through explicit stop-work authority, redistributing the burden.

Resistant in AI work. The migration of production pressure inward makes conventional stop-work reforms less effective because the individual cannot empower herself against herself.

Invisible until retrospect. The asymmetry is recognized clearly only after failures reveal which side of the burden was carrying the protection.

In The You On AI Field Guide

Origin

Key Ideas

Related Entries

Further Reading