The One and the Many

EDO SEGAL: One more round before The Crossing, and it's the one I think the reader most needs, because it's where the two philosophies show their teeth. Professor Bentham, your foundation is the aggregate — the greatest happiness of the greatest number, everybody to count for one. Professor Zuboff, your foundation is the inviolable individual — the one whose autonomy may never be summed away. So let me put the hardest case on the table and make you both bleed for it. A predictive system, watching everyone, will save ten thousand lives a year — caught diseases, prevented suicides, stopped attacks — at the cost of rendering every single person continuously legible and predictable. Bentham, your calculus says run it. Zuboff, your principle says it's a violation whatever the body count. I want you each to argue the case you'd least like to argue. Bentham, defend the one against your own arithmetic. Zuboff, defend the ten thousand against your own principle.

BENTHAM: You ask me to steelman the individual against my own sum, and I will, because a philosophy that cannot argue against itself is a faith, not a philosophy. Here is the strongest case against my arithmetic, and I have felt its force more tonight than ever in my life. The aggregate hides the distribution. When I say the system produces ten thousand units of saved life against a diffuse cost of universal legibility, my calculus reports a number going up — but a number can rise while specific persons are crushed, and the optimizer reads only the number. The machine sees the pattern and never the person; it knows the shape of the multitude and is blind, by construction, to the one inside it whom the shape is grinding. My own principle, pressed by a tireless machine, becomes a method for abandoning the one whose sacrifice raises the total. The deepest version: if her "right to the future tense" is real — and after this evening I am less certain it is not — then it is exactly the kind of good my calculus cannot weigh, because it is not a quantity of pleasure but a condition of being a person, and you cannot trade a condition of personhood for any quantity of pleasure without committing a category error my arithmetic is blind to. So the case against me is this: I built a calculus that must, in principle, sacrifice the one for enough of the many, and there may be things in the one that no quantity of the many can justly outweigh. That is the strongest thing my opponent has, and I have just made it for her, because it is true.

EDO SEGAL: That's a man arguing himself to the edge of his own foundation. Professor Zuboff — now you. Defend the ten thousand saved against your own principle. Don't let yourself off easy.

ZUBOFF: I won't, because the ten thousand are real and my principle has to look them in the face or it isn't serious. Here's the strongest case against me. I say autonomy is inviolable, that the future tense may never be claimed as raw material — but a mother whose child's suicide was prevented by a system that read the warning signs in his behavior does not care about my philosophy, and she is right not to care, and any principle that tells her the prevention was illegitimate has lost contact with the human good it claims to serve. My absolutism about autonomy, pressed hard, becomes a willingness to let ten thousand people die to preserve an abstraction they would each have traded in a heartbeat to live. And worse: I cannot actually draw the clean line I pretend to. The same surplus that farms the teenager catches the trafficker. The same prediction that steals the future tense saves the suicidal kid. They run on one architecture. If I abolish the extraction, I abolish the rescue with it, and I have to be honest that "abolish" has a body count too — it's just a body count I've let myself not look at, because the victims of my remedy are statistical and invisible in exactly the way I accuse Jeremy's victims of being. That's the strongest case against me. The architecture that endangers the self is the same architecture that saves the body, and I do not have a clean way to keep one and kill the other.

EDO SEGAL: Both of you just argued your way to the edge of your own cliff, and neither of you jumped, and that's the most honest thing two people can do. Let me name where you've ended up, because it's not where you started. Bentham: there may be things in the one that no quantity of the many can outweigh — a crack in pure aggregation. Zuboff: the architecture of the theft is the architecture of the rescue, and abolition has its own invisible dead. You've each conceded the other's strongest ground. Which means The Crossing is going to be a real fight, not a performance, because you respect each other's case now. I'm leaving the room. Ask each other the question you actually want answered.