Lucy Suchman

In the [YOU] on AI Field Guide

The cycle that began with [YOU] on AI describes a phase transition: for the first time in history, a person can describe what she wants in natural language and receive a working artifact in return. The translation barrier between human intention and machine execution has collapsed. Suchman is the sharpest theorist of what lived inside that barrier—and therefore of what is at stake when it falls. Her framework shows that the barrier was not merely an inconvenience. It was the territory through which intelligence developed.

Her central diagnosis, extended into the AI era, runs as follows. When a practitioner navigated the old barrier—debugging code, managing dependencies, troubleshooting systems—she was engaged in situated action: improvising responses to specific circumstances that deposited layers of understanding no curriculum could replicate. Each unexpected failure was a formative encounter. When large language models navigate that territory on the practitioner’s behalf, the output is produced and the understanding is not. AI outputs are plans, not actions: they address described situations, not encountered ones, and the map will always be simpler than the territory.

The Deleuze episode recounted in The Orange Pill—where Claude produced a philosophically inaccurate reference that “worked rhetorically”—is, in Suchman’s precise terms, a case of a plan masquerading as an action: an output statistically plausible over the training corpus, untested against the actual territory of Deleuzian philosophy. Segal caught it because he possessed the situated knowledge the evaluation required. Her framework asks—relentlessly—what happens when the evaluator does not.

Her lens connects across the cycle’s gallery of thinkers. Where Judea Pearl shows that AI cannot climb from pattern-matching to causal reasoning because it lacks a model of mechanism, Suchman shows it cannot cross from generated output to situated action because it lacks a body in a world. And where the cycle’s concept of improvisation as expertise identifies the human contribution that matters most in the age of AI, it is Suchman who supplies the empirical and theoretical foundation for why improvisation cannot be automated without destroying the practitioner who would otherwise develop it.

Origin

Suchman arrived at Xerox PARC in 1979 as an anthropologist—an outsider in a laboratory of engineers who understood machines from the inside out. Management had hired her because they faced a problem their technical expertise could not solve: the Alto workstation and the Star information system, the most advanced personal computers in existence, defeated ordinary users despite excellent engineering. Suchman spent months watching people interact with these machines and discovered that the engineers’ model of the user—a plan-executing agent who needed better procedures—was wrong. Users did not follow procedures; they interpreted them, projecting social intelligence onto machine outputs, forming hypotheses, adjusting when results were unexpected. The machine had no interpretive capacity; all the understanding was on the human side. She called this asymmetry between the human’s rich situated activity and the machine’s procedural responsiveness the central problem of human-machine interaction.

Her 1987 book made the argument formally, drawing on ethnomethodology—the sociological study of how people produce the orderly character of ordinary life—and on the work of Julian Orr, who had documented how Xerox photocopier technicians navigated the gap between the service manual and the specific, particular, always-different machine in front of them. Plans and Situated Actions provoked a formal rebuttal from Herbert Simon and Aran Vera in 1993; Suchman responded in the same journal. The debate was not academic. It was about the nature of intelligence itself.

She remained at PARC for decades, producing a body of research that expanded from photocopiers to surgery, air traffic control, and eventually to military AI systems. Her 2007 second edition of Plans and Situated Actions and her 2023 essay “The Uncontroversial ‘Thingness’ of AI” bring her analysis into direct engagement with the present moment, arguing that even critical discourse about AI reifies it as a coherent autonomous entity and thereby conceals the specific sociomaterial assemblages—training data, corporate decisions, interface designs, user practices—through which AI actually operates.

Key Ideas

Situated action. The foundational thesis: competent action arises not from executing a prior plan but from improvised, moment-by-moment responsiveness to the specific circumstances the actor actually faces. A plan is not the determinant of action but a resource for it—something the practitioner may consult, modify, or abandon depending on what the situation demands. This is the claim that dismantled the planning paradigm of classical AI and that locates the intelligence of human performance precisely in the gap between any map and the territory it represents.

AI outputs are plans, not actions. The sharpest application of her framework to the present moment: every AI-generated artifact—code, analysis, text—addresses the described situation, not the encountered one. The deployment environment is always richer than the description. The gap between the plan and the actual situation is where human situated judgment becomes indispensable, and it is also the gap that practitioners are losing the opportunity to develop as AI handles the navigation on their behalf.

Open worlds and closed worlds. AI succeeds “to the extent that the worlds in which systems operate have been effectively closed.” A closed world is one in which the variables are known, the contingencies bounded, and a plan can specify the action in advance. Human practice always occurs in open worlds. The fundamental limitation of any AI system is not computational power but the necessary simplification that any representation imposes on the open world it represents.

The asymmetry of human-machine interaction. From the photocopier study to the large language model: the human brings her full social intelligence to every interaction, projecting understanding, intention, and reciprocity onto machine outputs that have none of these properties. The outputs of sufficiently sophisticated systems can sustain the projection across extended interactions, producing what feels like genuine intellectual partnership while the interpretive labor remains entirely on the human side. This asymmetry is the structural cause of the over-trust the cycle identifies as among AI’s deepest risks.

The sociomaterial assemblage. AI is not a thing but a configuration of hardware, training data, corporate decisions, interface designs, and user practices. Treating it as a coherent autonomous entity—even critically—conceals the specific decisions and structures that shape what it does and who benefits. Accountability requires disaggregating the assemblage: not “AI did X” but “this company, with this data, through this interface, produced this outcome for these users.”