Common-Mode Failure

In the AI Story

The distinction between independent and correlated failure is the mathematical core of common-mode analysis. If two systems have a 1% independent failure rate each, the probability of simultaneous failure is 0.01% — one hundred times safer than either alone. If the same failure rate is fully correlated — both systems failing together or not at all — the probability of simultaneous failure is 1%, offering no improvement over a single system. The apparent redundancy is cosmetic; the real protection depends on whether the failures are independent.

Traditional organizations had a kind of accidental diverse redundancy in the diversity of their specialists. A backend engineer's assumptions were different from a frontend engineer's assumptions, which were different from a data engineer's assumptions. The differences were not engineered for safety; they emerged from different training, different experience, different domain languages. But the consequence was structural safety: an architectural mistake that seemed reasonable to one specialist would often look wrong to another, and the boundary between their domains provided a natural detection mechanism.

The AI-augmented generalist eliminates the diversity along with the boundaries. She uses the same AI tool, the same conversational frame, the same cognitive architecture across every domain. An assumption that is wrong in a way that Claude does not flag is wrong in every domain simultaneously, because Claude is the common mode. The fintech incident described in Chapter 2 — where settlement logic errors passed through tests authored by the same engineer using the same tool reflecting the same architectural assumptions — is a textbook common-mode failure: the tests and the code shared a cognitive lineage, so the failure propagated through the entire detection system rather than being caught by it.

The prescription is epistemic diversity: maintaining independent perspectives in the system even when efficiency argues against it. Segal's instinct to keep team size despite the multiplier is this kind of maintenance. The HRO capability of deference to expertise — flowing authority to whoever has the most relevant independent knowledge in a crisis — requires that such independent expertise exists in the organization, which requires that the organization has not eliminated it in pursuit of efficiency.

Origin

The concept emerged from nuclear engineering in the 1960s and 1970s, when early reactor designs assumed redundancy would provide safety without examining whether the redundant systems were truly independent. Investigations of near-misses and accidents consistently found shared vulnerabilities: shared power supplies, shared instrumentation, shared software, shared human operators under common stress.

Key Ideas

Redundancy is not independence. Multiple backup systems sharing a common vulnerability provide no real protection against that vulnerability.

Cognitive common-mode. A single mind operating across multiple domains is, structurally, a common-mode cognitive system.

Shared training sets as shared vulnerability. AI tools trained on the same data produce correlated errors across every domain they are used in.

Diverse redundancy as prescription. Real protection requires redundancy whose components do not share vulnerabilities.

Silent failure propagation. Common-mode failures often do not produce the alarm signals that single-system failures trigger, because the alarm systems share the common mode.