The Fine Print and the Future

EDO SEGAL: Let me bring this down from the metaphysics to the kitchen table, because there's a twelve-year-old version of this whole argument and it's the most honest version. The twelve-year-old says: but I agreed. I clicked "I accept." Nobody made me. Professor Zuboff, the entire legal architecture of your century's tower rests on that click — on consent. You say the consent is a fiction. He's going to say the consent is real and binding. So tell the twelve-year-old: when she clicked "I accept," what did she actually agree to, and was it real?

ZUBOFF: She agreed to nothing, and the word "consent" is the most overworked lie in the system, and I can prove it to a twelve-year-old. Consent, to mean anything, requires that you know what you're agreeing to, that you could realistically refuse, and that the terms don't change after you sign. Now look at the click. She "agreed" to terms longer than this entire book, written to be unreadable, that she could not negotiate, that reserve the right to change themselves at any time, for data uses that didn't exist yet, processed by systems that hadn't been built, to predict things about her that she can't imagine. That's not consent. It's a ritual of consent — a piece of theater whose function is to transfer the liability to her so that when the harm comes, the company can say "she agreed." And the refusal she supposedly had? Refuse, and she loses access to the infrastructure of modern life — her school's portal, her friends, her future employer's application. "Agree or be excluded from society" is not a choice; it's a toll. The genius of the click is that it manufactures the legal form of consent while removing every condition that would give consent moral weight. She didn't sign away her future, Edo. Her future was taken, and she was handed a pen and told to call it a signature.

EDO SEGAL: Professor Bentham, that's a frontal assault on the legal foundation of the entire apparatus you're defending. She says the consent is theater. And here's the irony I can't resist handing you: you spent your life attacking legal fictions — "nonsense upon stilts." Is click-through consent a fiction of exactly the kind you'd have demolished?

BENTHAM: ... You have caught me in my own net, sir, and I will not wriggle. Yes. If there is one thing my entire jurisprudence stood for, it is the demolition of the legal fiction that does real work while corresponding to no real fact — the form of consent without its substance, the ceremony that launders a transfer of power into the appearance of agreement. I built my life attacking exactly this in the common law: the solemn fictions that let the powerful take while the forms said all was freely given. And she is right that the click is such a fiction. It has the form of my own principle — voluntary exchange, each party consulting their own interest — and none of the substance I required for the form to be valid: real knowledge, real alternative, real symmetry. I cannot defend it. I will go further, because honesty compels it: a utilitarian has no use for consent as a sacred talisman anyway — I never grounded the good in consent, I grounded it in consequences. So I can abandon the click without abandoning an inch of my system. The click is bad law and bad utility both. Strike it down. Require real disclosure, real alternatives, real symmetry, or void the taking entirely.

ZUBOFF: I want to stop and honor that, because it's the most important thing that's happened tonight and it would be cheap of me to just press on. Jeremy Bentham, defender of the tower, has just agreed that the legal mechanism by which his tower currently operates is a fraud he'd have spent his life destroying. We agree completely: the consent is theater, the click is nonsense upon stilts, and either you build real conditions for agreement or you void the extraction. That's not a small overlap. That's the entire current legal basis of surveillance capitalism, condemned by both of us, from opposite philosophies, in the same breath.

EDO SEGAL: Convergence four, and it's a big one — I'm marking it loudly because the reader should not miss two opponents demolishing the same foundation from opposite ends. Both of you reject click-through consent as a fraud: Zuboff because it violates autonomy, Bentham because it's a legal fiction without substance and bad utility besides. You agree the current legal basis is rotten. Now — and this is the test — having agreed the foundation is rotten, do you agree on what to build instead? Professor Bentham?

BENTHAM: Here we part again, and cleanly. Having voided the false consent, I would build true consent and bound extraction: real disclosure in plain language, a genuine right to refuse without exile from society, a legal cap on what may be taken and a public audit of its use, the prediction forbidden from sale, the objective bound by statute to the welfare of the watched. Do all that, and the surplus may still be gathered — to catch the fraud, to find the famine, to balance the grid — but gathered under conditions that make it serve the greatest happiness rather than private margin. I would regulate the taking into legitimacy. The instrument is too valuable to destroy; I would civilize it.

ZUBOFF: And I would abolish the taking, because I no longer believe it can be civilized — and the difference between us is now perfectly clear and perfectly unresolved. Jeremy thinks behavioral surplus is a resource that bad law currently misgoverns, and that good law could make its extraction legitimate. I think behavioral surplus is stolen goods at the root — that the unilateral claiming of private human experience as free raw material for prediction is illegitimate however you regulate the downstream use, the way a market in human beings is illegitimate however humanely you run the auction. He wants to make the extraction fair. I want to make the extraction unthinkable — to put the human, as he once put the slave and the woman and the animal, outside the set of things that may be claimed as raw material at all. That's the whole debate in one sentence: he wants a fair market in the future tense, and I want the future tense withdrawn from the market.

EDO SEGAL: Let me make "the market in the future tense" concrete, because it sounds like a metaphor and you mean it literally. There are actual markets here.

ZUBOFF: They are as real as a grain exchange. I call them behavioral futures markets — markets where what's bought and sold is a prediction of what specific people, or specific kinds of people, will do. An advertiser doesn't buy your attention in the abstract; it buys a computed probability that you, this week, in this mood, will click, convert, vote, or quit. That probability is a product manufactured from your behavior and sold to someone whose interest is in making the probability come true. When I say "withdraw the future tense from the market," I mean: this particular commodity — a traded claim on a named human being's not-yet-decided behavior — should not exist, the way a traded claim on a human being's body should not exist. Jeremy wants to license the exchange and tax it fairly. I want to declare the commodity itself contraband.

BENTHAM: And I answer that contraband is what we call a good we have decided, by collective will, to keep off the market — which proves my point, not hers. We did not abolish commerce to abolish the slave trade; we abolished the specific traffic and kept the market. If the traded claim on a named person's behavior is the contraband, then forbid that specific traffic — the sale of individuated prediction — and you have done it within my framework, by law, by line-drawing, without abolishing the instrument that also finds the famine. She keeps reaching for "abolish the mechanism" when her own examples reach only for "abolish a specific traffic." That is regulation with a louder adjective.

EDO SEGAL: And there's the unresolved core, clean enough to hand to the reader whole. You've agreed the watching mechanism is neutral, that mechanism-transparency is a trap, that the calculus is blind to the route, that the consent is a fraud — four convergences, more than I expected. And you've arrived, from all that agreement, at irreducibly opposite remedies: civilize the extraction, or abolish it. That's the fork. We're near the end of the second hour. After the break I do something I do in every one of these: I leave the room. The two of you question each other directly, and I rescue no one. The Crossing — next.