Thick Alignment

In the [YOU] on AI Field Guide

[YOU] on AI rests on the conviction that these tools will belong to all of us only if all of us insist on a hand in shaping them. Thick alignment names the mechanism by which that insistence becomes structural rather than rhetorical. It reframes what “safe AI” means: not a system that does not deceive or harm its builders, but a system whose objectives have been legitimated by the people it governs. The cycle asks what it would mean to see the machine clearly; thick alignment asks what it would mean to govern it well, and argues that the two questions cannot be separated.

Where algorithmic governance now touches credit, policing, healthcare, and hiring, thin alignment produces systems that do what their builders want while systematically failing the people most affected. Thick alignment demands that those people have genuine standing to shape outcomes—not the comment period nobody reads, not the advisory board with no power, but input that can change what gets built before it is deployed. Nelson knows the standard is rarely met. She holds to it because the alternative—a technological future designed for the public without the public—is not a future a democracy can accept while remaining one.

Origin

Nelson developed the concept by noting that the word “alignment” had done remarkable work in the AI field while smuggling in a hidden assumption: that the values to which a system should be aligned are already given, obvious, or safely determined by experts. In its standard technical usage, alignment means getting a system to pursue the objectives its designers specify, avoid the behaviors they forbid, and remain controllable in their hands. Stated this way it sounds unobjectionable. Nelson’s objection is not that this goal is wrong but that it is radically incomplete—and that the incompleteness is disguised by the reassuring sound of the word.

The concept draws on her earlier work tracing how the prestige of science has been used to make contestable claims appear as settled facts. In medicine, the language of objective science cloaked profoundly unequal treatment—some bodies cared for, others neglected or harmed, all under the banner of a single impartial science. Thick alignment is her attempt to prevent the same dynamic from recurring in AI: to insist that the value questions cannot be absorbed into the technical work and thereby hidden from view, but must be answered through a process that includes those who will live with the result.

Key Ideas

The specification is the problem. Thin alignment takes the values question as answered and focuses on the engineering question. But the specification of the objective is precisely what is not settled. An objective function encodes someone’s notion of what counts as success; a training dataset is a record of a particular society’s choices; a safety definition reflects whose harms the designers have learned to see and whose they have not. The engineering follows the specification; the specification follows power; and power follows who is in the room. Thick alignment moves the conversation upstream, to the room.

Affected communities as experts. Nelson’s most consequential argument is that on the question of values, technical expertise confers no special authority. A system’s designers may be world-class engineers; that expertise tells them nothing about whether a given distribution of benefits and harms is just. The person whose loan was denied by an algorithm knows something about that system that its designers do not—what it is like to be on the receiving end, what the error costs, what recourse was or was not available. That knowledge is a form of expertise, and a governance process that excludes it is systematically incomplete.

Legitimacy as infrastructure. A technology deployed without legitimate public standing is fragile—it generates the kind of resistance and backlash that ultimately undermine deployment. Democratic legitimacy, on this account, is not a luxury that successful technology can do without; it is part of the infrastructure of durable deployment, and skipping it stores up costs to be paid later. Thick alignment is therefore not only a moral demand but a practical one: systems that have been thickly aligned are more trustworthy and therefore more durable than those that have not.