Naturalistic Decision Making

In The You On AI Encyclopedia

NDM's core methodological innovation was the Critical Decision Method, a structured interview technique developed by Klein and colleagues that walked practitioners backward through specific challenging incidents to surface the cues they attended to, the patterns they recognized, and the decisions they made. The method revealed cognitive architectures that laboratory studies had missed because they had eliminated the conditions — time pressure, ambiguity, ill-defined goals, changing conditions, high stakes — under which field expertise actually operates.

The movement's contrast with classical decision theory was sharp. Classical theory evaluated decisions against normative standards of rationality derived from economic models. NDM evaluated decisions against the standard of what worked in the actual conditions practitioners faced. The shift in evaluative standard was consequential: practices that looked irrational by classical standards — reliance on intuition, satisficing rather than optimizing, resistance to explicit comparison — turned out to be both prevalent and effective under field conditions.

NDM has direct relevance to the AI transition because the deployment of AI systems in expert domains typically treats decision-making as the classical theorists framed it — a process of generating options, comparing alternatives, and selecting the optimum. AI tools designed on this framework miss what NDM documents: that expert decision-making operates through recognition and simulation, that it depends on experiential foundations that classical analysis cannot formalize, and that it is systematically undermined by systems that substitute for the experiential process rather than supporting it.

Klein's work with DARPA's Explainable AI program applied NDM's framework to the question of what users need in order to oversee AI systems. The answer — accurate mental models of the system's competence boundaries — shaped the design of tools like AIQ that target the experiential foundations of effective human-AI collaboration rather than merely the technical transparency of AI outputs.

Origin

The NDM movement crystallized at a 1989 conference organized by Klein, Orasanu, Calderwood, and Zsambok that brought together researchers working on expert cognition in domains where classical decision theory had proven inadequate. The resulting 1993 volume, Decision Making in Action, established NDM as a coherent research program and the RPD model as its flagship framework.

The movement drew on earlier work in expertise research — Herbert Simon's bounded rationality, Adriaan de Groot's chess studies, Chi, Glaser, and Farr's expertise research — but its distinctive contribution was the rigorous study of expertise under the time-pressured, high-stakes conditions where it most matters.

Key Ideas

Field over laboratory. Expert cognition must be studied in the conditions where it operates, not under controlled laboratory simplifications.

Recognition over comparison. Experts recognize and simulate rather than generating and comparing options.

Satisficing over optimizing. Effective decisions find workable options, not optimal ones.

Critical Decision Method. Structured retrospective interviews can surface the cognitive processes of expert decision-making.

Implications for AI design. Systems built on classical decision-theoretic assumptions miss how expertise actually operates.