Adaptive Preferences

In the AI Story

The concept cuts against one of the deepest assumptions of market economics: that preference satisfaction reliably indicates well-being. Sen demonstrated that preferences can adapt to constraints in ways that reproduce the constraints rather than evaluating them. A person who has been deprived of a capability may stop valuing the capability, and the stopping-valuing may appear to justify the continued deprivation. The preference is genuine. The satisfaction is real. The deprivation, however, remains.

The AI application is precise and disturbing. Users of AI tools who report high satisfaction may be reporting satisfaction that has adapted to a contracted capability set. The student who has always used AI to generate essays may not value the capability of writing, because the capability has never been developed and its absence has never been felt. The developer who has always used AI to generate code may not value deep architectural understanding. The preference adapts to the available capability set, and the adapted preference obscures the capabilities that were lost or never developed.

The Berkeley study on AI adoption documented this pattern empirically, though not in these terms. Workers reported satisfaction with AI-augmented work even as the same study documented boundary erosion, task seepage, intensified multitasking, and accumulating fatigue. The satisfaction and the degradation coexisted. The workers valued what the tool provided — speed, output, breadth — and had ceased to value what the tool eroded. The preference adapted. The new capability set became the standard against which satisfaction was measured.

The implications for AI evaluation are profound. User satisfaction metrics, Net Promoter Scores, engagement statistics — these cannot be taken at face value as measures of well-being. They measure preference satisfaction in a context where preferences are being shaped by the very technology whose effects are being evaluated. The tool that produces the adaptation registers as successful according to the metrics the adaptation produces. Sen's framework insists that satisfaction must be supplemented by independent assessment of capability — assessment the person whose preferences have adapted cannot reliably conduct on her own behalf.

Origin

Sen introduced the concept prominently in his 1985 Dewey Lectures ('Well-being, Agency and Freedom') and developed it across subsequent work. Martha Nussbaum extended the analysis, particularly in the context of gender and development.

Key Ideas

Preferences adapt to constraints. The deprived often stop desiring what they have been denied.

Satisfaction is unreliable. A person can be genuinely satisfied with a deeply constrained life if her expectations have adjusted to the constraint.

Utilitarianism fails here. Any welfare framework based on preference satisfaction is blind to adaptive deprivation.

Independent assessment is required. The person whose preferences have adapted cannot be the sole judge of her own well-being.

Debates & Critiques

Critics have argued that the concept risks paternalism — who decides when a preference is 'adapted' rather than authentic? Sen's response has been that the determination belongs to democratic deliberation informed by evidence about objective conditions, not to unilateral expert judgment.