Surveillance Capitalism

In the AI Story

Every commercial language model, recommendation system, and content ranker is embedded in surveillance capitalism. The training data comes from surveillance; the economic justification for investment comes from prediction and influence; the alignment question becomes, Zuboff argues, inseparable from the question of whose interests these systems serve.

Zuboff's analysis has become increasingly applicable since 2022, as generative-AI systems have entered the same commercial context they critique. Training data for frontier models is harvested from the same substrate — user-generated content, behavioral traces, scraped records — that Zuboff's earlier analysis centered on. The question of whether generative AI is itself a surveillance-capitalist technology is contested; the question of whether it was built on surveillance-capitalist infrastructure is not.

Origin

Zuboff's analysis builds on her earlier work on information technology and power (In the Age of the Smart Machine, 1988). The full theory is in The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of Power (2019).

Key Ideas

Behavioral surplus. Data beyond what's needed to provide the service, extracted and used for prediction.

Instrumentarianism. Zuboff's term for the new form of power: not control via rules (totalitarianism) but control via prediction and nudge.

Division of learning. Who knows what about whom, and who decides what to do with that knowledge.

Right to the future tense. Zuboff's framing of autonomy as the right to an unpredicted, un-nudged future.

Consent as design. Zuboff argues that "meaningful consent" is structurally impossible in surveillance capitalism because the economic value depends on extracting more than any user would knowingly trade. The AI-training analog is sharper: users who wrote text decades ago did not and could not consent to their text training future language models, because the technology did not yet exist.