The Training Problem Under Automation

In The You On AI Field Guide

Conventional training assumes a model in which expertise is acquired through explicit instruction, controlled exposure to canonical cases, and supervised practice. This model works for well-defined skills in stable domains. It fails for the kind of expertise Bainbridge identified as essential for exception handling: the tacit, pattern-based judgment built through encounter with the full distribution of situations, including the unanticipated ones.

The problem has four components. First, representativeness: training scenarios must accurately represent the exceptions operators will actually face, but the essential feature of those exceptions is that they are unrepresentative. Second, frequency: exceptions must be encountered often enough to build fluent response, but making them frequent contradicts the reliability that makes automation worthwhile. Third, stakes: training lacks the stress and consequence of real incidents, and performance under stress differs from performance in training. Fourth, transmission: the tacit knowledge seniors built cannot be directly transmitted to juniors through explicit instruction.

Bainbridge's partial answer was a training philosophy that accepted these limits rather than pretending to overcome them. Training should focus on building mental models of how the system actually works — not just its normal operation but its failure modes, its degradation patterns, its relationships to the physical processes it controls. Training should expose operators to as many anomalies as possible, including deliberately introduced ones, accepting that the exposure remains insufficient. Training should preserve the mentoring relationships through which tacit knowledge could still be transmitted, accepting that transmission remains imperfect.

In the AI era, the training problem has acquired new urgency. The apprenticeship problem in software development, diagnostic medicine, and legal research is the training problem in its contemporary form. The juniors being trained today will, in ten years, be the seniors on whom AI-era organizations depend for judgment — and they will have built their judgment on a foundation that no previous generation of professionals has relied upon. Whether that foundation will bear the weight is the open question of the transition.

Origin

Bainbridge developed the training problem in her 1983 paper and elaborated it in subsequent work through the 1990s. The framework has been adopted across safety-critical training programs in aviation, nuclear power, and medicine, and has more recently been invoked in discussions of AI's impact on medical residency, junior developer training, and professional education generally.

Key Ideas

Exceptions cannot be fully simulated. The essential feature of the exceptions that matter is that no one anticipated them — simulations can only rehearse the exceptions someone imagined.

Tacit knowledge resists transmission. The pattern libraries that experts use to recognize exceptions are built through encounter, not instruction; curricula cannot teach what only years of direct experience deposits.

Generational erosion compounds the problem. Each generation of operators trained in a more automated environment has less direct experience than the one before, and the cumulative loss is irreversible on institutional timescales.

Mentoring is the partial remedy. Where tacit knowledge can be transmitted at all, it is transmitted through sustained relationships between experienced and developing practitioners — the infrastructure AI-era cost pressure is most efficient at dissolving.