You On AI Field Guide · The Optimization Trap The You On AI Field Guide Home
TxtLowMedHigh
CONCEPT

The Optimization Trap

Rob Reich’s diagnosis of the technology industry’s defining intellectual failure: that optimizing for a measurable metric is always a value choice disguised as a neutral engineering achievement—and that what the metric does not measure is often what matters most.
The optimization trap is Rob Reich’s name for the political sleight of hand at the center of the technology industry’s self-understanding: the substitution of what companies care about for the values that a democratic society might choose to prioritize, carried out under the cover of neutral engineering methodology. When a company optimizes for engagement, it has made a decision that engagement matters more than truth, more than mental health, more than the quality of public discourse. The decision is presented as a product choice rather than a political choice—as the neutral maximization of a measurable variable rather than the assertion of a value system. The trap springs when the metric goes up and the thing the metric does not measure goes down, and the optimization is declared a success because the metric is the only thing being evaluated. In the AI transition, the trap becomes most visible when productivity is the optimization target: the twenty-fold
← Home0%
CONCEPTBook →

Keep reading with YOU ON AI

Unlock the full book, 10,000+ field-guide entries, and a 1000+ thinker library. If you have a book code, register now — it takes a minute.

Register with book code Sign in