The Watchtower as Gift

EDO SEGAL: I want to start this round with a confession instead of a question, because the best questions I know come out of wounds. Years ago I built engagement machinery — products designed to be watched so they could watch you, designed to learn the exact shape of what kept you scrolling and feed it back to you. I told myself it was a service. I knew the loops. Ask me how I slept. So when you tell me, Professor Bentham, that the watchtower is an act of love, I don't hear an abstraction. I hear the voice I used on myself at the time. Take me back to the original drawing. Tell me, the way you'd tell a twelve-year-old, what the Panopticon was for — and why you call it kindness.

BENTHAM: Then let me tell you a thing you can feel. Picture a prison in 1790 — the real ones, that I walked through and could not unsee. Filth. Fever. Men chained in the dark, beaten by guards who were themselves brutalized, the strong preying on the weak in the corners no warden could see. Punishment was theatre and cruelty was routine, and none of it reformed a single soul; it only manufactured worse men and returned them to the world. I looked at that horror and I asked a builder's question, not a poet's: what arrangement of stone and sightlines would make cruelty unnecessary? And the answer came to me through my brother Samuel, who'd used a central point of inspection to oversee workmen in Russia. A ring of cells, every one open to a tower at the center. The watchman sees all; the watched cannot see the watchman, cannot tell when the eye is upon them. And so — here is the kindness — the prisoner stops needing to be beaten. He behaves because he might be seen. The watching does the work the whip used to do, and it does it without breaking a single body. "Morals reformed, health preserved, industry invigorated, instruction diffused." I meant every word. The tower was how I proposed to end the lash.

EDO SEGAL: So what you're saying — literally — is that the watching is a substitute for violence. That the eye replaces the whip, and that's a moral advance. Is that the version you find most compelling?

BENTHAM: It is the heart of it. And notice the principle generalizes far past the prison. A factory whose workers know the floor is observable has fewer accidents. A hospital open to inspection kills fewer patients. A government whose accounts are public steals less. I proposed the principle for all of them, because the principle is the same: where conduct is visible, conduct improves, and it improves gently, by the soft pressure of possible observation rather than the hard hand of force. Your century took my tower and aimed it at fraud, at disease, at the predator who would harm a child. When a machine watching every transaction catches the man laundering money for traffickers, that is my tower working exactly as designed. My opponent wishes to indict the whole apparatus. I ask her to look at the man in handcuffs who would otherwise be free, and tell the child he trafficked that the watching was a theft.

ZUBOFF: I'll take that child, because he deserves better than to be used as a shield. Yes — catch the trafficker. I have never argued against catching the trafficker, and Jeremy knows it, and the move he just made is the oldest move in the surveillance playbook: point at the worst criminal to justify watching everyone. Here is what his account leaves out, and it is not a detail, it is the whole modern fact. In the Panopticon, the watcher and the watched were inside one institution, with one shared purpose — the reform of the prisoner, ostensibly for his own good and the public's. The watcher had no separate, hidden interest in the prisoner's future. Now go to the machine you praise. Who owns it? Not the public. Not the prisoner. A corporation, whose objective is not your reform and not the public good but the prediction of your behavior, sold to a third party you will never meet, for the modification of what you do next, toward an end that is theirs. The watchman in your tower wanted the prisoner to be good. The watchman in mine wants the user to be profitable, which is an entirely different thing, and the gap between those two objectives is where every harm I study lives.

EDO SEGAL: Let me name that, because the reader can't see your faces and that was the first exchange where neither of you gave an inch. Bentham says the watcher and the watched share a purpose — your improvement. Zuboff says they don't anymore — the watcher's purpose is now your predictability, sold for profit. Professor Bentham, that's a serious charge against your design, not just its abuse. Does the principle survive the watcher having a hidden, separate interest?

BENTHAM: It is a serious charge, and I will meet it seriously, because she has put her finger on something real. I will concede this: a Panopticon in which the inspector profits from the prisoner's predictability, rather than aiming at his reform, is a corrupted Panopticon. I grant it freely. But mark what kind of objection this is. It is not an objection to visibility. It is an objection to misaligned objectives and absent accountability — which is to say, it is an engineering problem and a legal problem, not a metaphysical one. The remedy is not to put out the light. The remedy is to fix who holds it and to whom they answer. Make the inspector visible to the public, as I always insisted he must be. Forbid the sale of the prediction. Bind the objective by law to the welfare of the watched. Do all that, and you have my tower restored, doing its gentle work. She wants to burn down the building because the current tenant is a scoundrel. I say evict the tenant and keep the building, because the building, rightly tenanted, is the greatest engine of reform ever drawn.

ZUBOFF: And here is why I cannot accept the eviction theory, Jeremy, and it's the deepest thing I'll say in this round. You speak as if the misalignment were an accident — a good building, a bad tenant, fixable by better law. I am telling you the misalignment is constitutive. Surveillance capitalism did not adopt extraction by mistake. It discovered, around 2001, that behavioral surplus was a source of staggering wealth, and it rebuilt the entire architecture around the extraction, because that is where the money was. The asymmetry isn't a bug in the tower; it's the business model. You cannot legislate a profit-extracting predator into a benevolent reformer by asking it nicely to realign, any more than you could have legislated the slave trade into ethical sourcing. Some architectures are illegitimate at the root, and the only honest remedy is abolition of the mechanism, not better management of it. That's why I stopped saying "regulate" and started saying "abolish." You taught me, actually — you taught me to follow the consequences all the way down and not flinch. I followed them down to the root, and the root is rotten.

EDO SEGAL: Mark that — Zuboff just paid Bentham a real debt, and the disagreement got sharper at the same time, which is the best thing that can happen in an hour. He says: good building, bad tenant, evict and keep. She says: the building was rebuilt around the rot, condemn it. Hold that fork. Because the thing the modern tower extracts has a name, and she's spent forty years naming it, and the next round belongs to it. Behavioral surplus — what gets taken, and whether taking it is taking anything at all.