The Compounding Degradation Loop

Automation's threat to human capability is not a single process but a system of three interacting mechanisms, each of which degrades a different dimension of what Lisanne Bainbridge called the operator's evaluative contribution—the capacity to detect errors, diagnose their cause, and assert corrective judgment against a system's confident output. Skill decay is the deterioration of manual and cognitive capabilities that are not exercised; trust miscalibration is the drift of the operator's confidence in the automated system beyond what the system's actual reliability warrants; feedback attenuation is the reduction in the quality and timeliness of information the operator receives about the accuracy of her evaluative judgments. Each of these mechanisms is damaging in isolation. Together, they form a compounding degradation loop in which each mechanism amplifies the others: skill decay reduces detection capability, which allows more errors to pass undetected, which biases the evidence toward AI correctness, which increases trust miscalibration; increased trust reduces monitoring effort, which further reduces detection and accelerates skill decay; feedback attenuation prevents any of