The Question on the Table

EDO SEGAL: Somewhere in the world right now — statistically, in the time it takes me to say this sentence — about a million people are being watched by a machine that they invited in. A teenager in Jakarta, scrolling before sleep, while a model behind the glass clocks every pause, every half-second of lingering, and adjusts the next thing she sees. A man at a kitchen table applying for a loan, his whole life flattened into a risk score he will never be allowed to read. A nurse in Cleveland, end of a double shift, typing a question into a box that will remember the question forever, in a place she cannot inspect, for purposes she did not choose. None of them feel watched. There is no tower in the eyeline, no guard's face at the window. And that — the watching without the felt sensation of being watched — is the most successful piece of architecture in human history, and tonight I have at this table the two people who can tell you whether it is a cathedral or a cage.

A million people, watched and predicted, and the question none of them stop to ask, because the convenience makes it feel already answered, is the one we are going to spend three hours inside: when the watchtower can see everything you do, and predict what you'll do next, is being watched the price of a well-run world — or the quiet theft of the self you were climbing to become?

I have wanted this conversation for a very long time, and I had to reach across two centuries to get it.

Jeremy Bentham was born in 1748 and died in 1832. He was a child prodigy who entered Oxford at twelve, a jurist, a reformer so far ahead of his age that he argued against slavery, against cruel punishment, for the rights of women and animals and homosexuals, at a time when each of those was a scandal. He founded utilitarianism — the doctrine that the right act is the one producing the greatest happiness of the greatest number. He invented a calculus to measure pleasure and pain. And he designed a building, the Panopticon, that he believed could reform the human race through the gentle pressure of perpetual possible observation. He is here tonight, and yes — I have to say this once, because it matters — he has been briefed on our century. He knows what a neural network is. He has read about the feed and the score and the prediction market. Professor Bentham, you've had two hundred years to think it over.

BENTHAM: I have had two hundred years, sir, mostly spent in a glass cabinet, which I commend to anyone who wishes to learn patience. And I will say at once, before your other guest sharpens her knife — I do not arrive here embarrassed. I arrive vindicated. You have built my inspection-house out of light and wire, you have made the watchman tireless and the cells comfortable, and you have done in a decade what I could not persuade a single government to fund in my lifetime. I designed the engine. You finished it. The only question that interests me is whether you are using it well.

EDO SEGAL: Shoshana Zuboff was born in 1951. She is a scholar and social theorist, one of the first tenured women at Harvard Business School. In 1988, when the personal computer was a novelty, she wrote In the Age of the Smart Machine and gave us a distinction the whole field still runs on — between automating, which displaces human knowing, and informating, which could deepen it. In 2019 she wrote a seven-hundred-page indictment, The Age of Surveillance Capitalism, and named the thing Bentham is so proud of. She has since stopped asking that it be regulated and started asking that it be abolished. Professor Zuboff, you have spent four decades watching this engine get built. The man who drew it is sitting across from you.

ZUBOFF: He's sitting across from me, and I want to thank him, genuinely, because Jeremy Bentham is the most honest man in the history of this idea. Everyone since him has hidden the tower — wrapped it in convenience, in connection, in the word "free." He put it in the middle of the room and called it what it was: an apparatus for producing behavior through asymmetrical observation. I disagree with almost everything he concludes. But I will never accuse him of the thing I accuse the others of, which is concealment. The men running my century learned the lesson he didn't intend to teach: that the watching works best when the watched cannot see the tower at all.

EDO SEGAL: You see why I needed both of you. Three rules for the evening, and they are the same three I always set. First: three hours, which means no one wins by the next bell — long form exists so an argument can breathe before anyone strangles it. Second: I declare my bias at the door. I have built engagement machinery and regretted it. I have also been met, genuinely, at three in the morning by one of these systems, and I have never fully resolved the contradiction. Both of you will press on a different half of me tonight. Third: if the disagreement survives three hours, I do not paper it over. I hand it, intact, to the reader. Each of you may add a rule.

ZUBOFF: One rule. We name the actor. Every time someone says "the technology does this" or "the algorithm decides," I am going to ask: who. Which company, which objective, which owner. The passive voice is where the accountability goes to die, and it is the native tongue of surveillance capitalism. There is no "it." There is always a they.

BENTHAM: A fair rule, and I'll add its twin. Every time my distinguished opponent says a thing is "intolerable" or "illegitimate" or a "theft," I shall ask her to produce the ledger. Theft from whom, of what, measured how, against what gain to whom else. Indignation is not an argument. I spent my life insisting that we replace the cant of natural rights — "nonsense upon stilts," I called it — with the honest arithmetic of consequences. If she wishes to convict my tower, let her show me the sum comes out negative. I am entirely willing to be convinced by a number.

ZUBOFF: And there, in his first sixty seconds, is the entire fight. He thinks the harm has to show up in a number, or it isn't real.

BENTHAM: And she thinks it's real because she feels it. We shall have a wonderful evening.

EDO SEGAL: We have our evening. One image before the opening statements, because it's the frame this whole series climbs inside and both of you will have to stand somewhere on it. In [YOU] on AI I argued that you ascend a tower by taking the stairs, not the elevator — that the climb is the point, that what you see from each floor you earned by your own effort. And near the top, where the view gets long, you realize the building itself has windows that look back at you. Professor Bentham, you would say those windows are a kindness — that being seen is what keeps the climber honest and safe. Professor Zuboff, I suspect you'd say the windows are one-way glass, and there's someone behind them selling the map of where you'll step next.

ZUBOFF: I'd say it's worse than one-way glass. The glass learns. It studies which way you flinched, and tomorrow the stairs are arranged to send you where the owner of the building wants you to go — and you'll experience that as your own free choice to climb. That's not surveillance as your guest understands it. It's something new, and naming it is half the battle.

BENTHAM: And I would say: thank heaven the glass learns, for a watchman who learns can finally watch every cell at once — the very thing my design could never do — and a society at last made fully visible to itself is a society that can at last be made good. She fears the completed tower. I have waited two centuries for it.

EDO SEGAL: Then we know where we stand, and we stand on opposite banks of the same river. Here is the question once more, plainly, because every round tonight is this question wearing a different coat. When the machine can see everything you do and predict what you'll do next — is that the price of a well-run world, or the theft of the self? Jeremy Bentham, the floor is yours.