Naturalistic Decision Making
Naturalistic Decision Making (NDM) emerged in the late 1980s as a research movement focused on decision-making in real-world settings rather than laboratory paradigms. Its practitioners — Klein, Judith Orasanu, Caroline Zsambok, and others — shared the conviction that classical decision theory, with its emphasis on rational choice among clearly defined options, had misrepresented how experienced practitioners actually operate. NDM research, conducted through field interviews and structured observation in firegrounds, intensive care units, military command posts, and cockpits, revealed that experts do not compare alternatives; they recognize patterns and mentally simulate actions. The movement's foundational 1989 conference produced the intellectual framework within which Klein's RPD model, the pattern library concept, and the research program on sensemaking developed.
In the AI Story
NDM's core methodological innovation was the Critical Decision Method, a structured interview technique developed by Klein and colleagues that walked practitioners backward through specific challenging incidents to surface the cues they attended to, the patterns they recognized, and the decisions they made. The method revealed cognitive architectures that laboratory studies had missed because they had eliminated the conditions — time pressure, ambiguity, ill-defined goals, changing conditions, high stakes — under which field expertise actually operates.
The movement's contrast with classical decision theory was sharp. Classical theory evaluated decisions against normative standards of rationality derived from economic models. NDM evaluated decisions against the standard of what worked in the actual conditions practitioners faced. The shift in evaluative standard was consequential: practices that looked irrational by classical standards — reliance on intuition, satisficing rather than optimizing, resistance to explicit comparison — turned out to be both prevalent and effective under field conditions.
NDM has direct relevance to the AI transition because the deployment of AI systems in expert domains typically treats decision-making as the classical theorists framed it — a process of generating options, comparing alternatives, and selecting the optimum. AI tools designed on this framework miss what NDM documents: that expert decision-making operates through recognition and simulation, that it depends on experiential foundations that classical analysis cannot formalize, and that it is systematically undermined by systems that substitute for the experiential process rather than supporting it.
Klein's work with DARPA's Explainable AI program applied NDM's framework to the question of what users need in order to oversee AI systems. The answer — accurate mental models of the system's competence boundaries — shaped the design of tools like AIQ that target the experiential foundations of effective human-AI collaboration rather than merely the technical transparency of AI outputs.
Origin
The NDM movement crystallized at a 1989 conference organized by Klein, Orasanu, Calderwood, and Zsambok that brought together researchers working on expert cognition in domains where classical decision theory had proven inadequate. The resulting 1993 volume, Decision Making in Action, established NDM as a coherent research program and the RPD model as its flagship framework.
The movement drew on earlier work in expertise research — Herbert Simon's bounded rationality, Adriaan de Groot's chess studies, Chi, Glaser, and Farr's expertise research — but its distinctive contribution was the rigorous study of expertise under the time-pressured, high-stakes conditions where it most matters.
Key Ideas
Field over laboratory. Expert cognition must be studied in the conditions where it operates, not under controlled laboratory simplifications.
Recognition over comparison. Experts recognize and simulate rather than generating and comparing options.
Satisficing over optimizing. Effective decisions find workable options, not optimal ones.
Critical Decision Method. Structured retrospective interviews can surface the cognitive processes of expert decision-making.
Implications for AI design. Systems built on classical decision-theoretic assumptions miss how expertise actually operates.
Debates & Critiques
The enduring tension between NDM and the heuristics-and-biases tradition associated with Kahneman was partially resolved through Klein and Kahneman's 2009 adversarial collaboration, which established conditions under which intuitive expertise is reliable. The broader debate about whether AI development should be informed by NDM's findings — about the experiential foundations of expert cognition — remains largely unresolved, with most AI research proceeding within frameworks that NDM's empirical work has substantially challenged.
Appears in the Orange Pill Cycle
Further reading
- Klein, G., Orasanu, J., Calderwood, R., & Zsambok, C. E. (Eds.). (1993). Decision Making in Action: Models and Methods. Ablex.
- Zsambok, C. E., & Klein, G. (Eds.). (1997). Naturalistic Decision Making. Erlbaum.
- Klein, G. (2008). Naturalistic decision making. Human Factors, 50(3), 456–460.
- Lipshitz, R., Klein, G., Orasanu, J., & Salas, E. (2001). Taking stock of naturalistic decision making. Journal of Behavioral Decision Making, 14(5), 331–352.