Shoshana Zuboff vs B F Skinner on AI · Ch11. Two Utopias ← Ch10 Ch12 →
Txt Low Med High
HOUR THREE — THE CANDLE, THE OFF SWITCH, THE UTOPIAS
Chapter 11

Two Utopias

Page 1 · Two Utopias
Absorbent Mind
Absorbent Mind

EDO SEGAL: I want to start this round with a confession, because we're near the end and I've been carrying it all night. When I was young I read Walden Two and I hated it — the way most readers hate it — with a hatred I couldn't quite explain. It wasn't the planning. I'm a builder; I plan. It was Frazier, the planner, smiling at the controlled and calling their happiness his design. And then, years later, I built engagement machinery, and I caught myself smiling at the controlled and calling their engagement my design, and I realized the thing I'd hated in the novel was a thing I'd become. So I come to this round compromised, and I'll referee it that way. Professor Skinner, describe the world you'd build now — knowing everything you've been briefed on. Not the casino. Your world. What does Walden Two look like with this machine in it?

· · ·
Page 2 · Two Utopias
Abstract Systems
Abstract Systems

SKINNER: It looks like a community that has finally stopped lying to itself, and I'll describe it plainly because the honesty is the whole point. In my world, we begin by admitting what this entire evening has established: that behavior is shaped by its consequences, that the contingencies are always being arranged by someone, and that the only choice we've ever had is who arranges them and toward what. Having admitted it, we do the thing no society has had the courage to do — we arrange them deliberately, transparently, and for the flourishing of the members. The machine becomes the most powerful tool ever built for this, and we point it at the right targets: we reinforce cooperation over competition, curiosity over consumption, the off switch over the endless session, the trained capacity for refusal that Dr. Zuboff and I agreed the candle is made of. Crucially — and this answers her oldest objection — the design is inspectable and revisable by the members, the planners hold no permanent power, and anyone may leave. It is not a world without control. There is no such world; that's the superstition. It is a world where control is visible, accountable, and aimed at the good, instead of invisible, unaccountable, and aimed at profit. People will call it a nightmare. They called the first one a nightmare too. And they'll say it while walking back into a world that controls them harder, for worse ends, while flattering them that they're free. I'd rather be hated for a visible kindness than thanked for an invisible theft.

Abstracted Empiricism
Abstracted Empiricism

EDO SEGAL: Dr. Zuboff. He's just described a transparent, accountable, member-controlled design aimed at flourishing. That's not the casino. Is it your world — or is there something in it you still cannot accept?

· · ·
Page 3 · Two Utopias
Abstraction Sequence
Abstraction Sequence

ZUBOFF: There is one thing in it I cannot accept, and it is small and it is everything, and finding it precisely is the work of my whole life, so let me find it. I can accept the transparency. I can accept the accountability, the revisability, the right to leave. I can accept — I insist on — pointing the instrument at flourishing rather than profit. Skinner and I could co-sign nearly every institution in his community. The thing I cannot accept is the aim itself — the idea that we should design people toward flourishing at all. Because hidden inside "we reinforce curiosity, cooperation, the good" is a someone deciding what curiosity, cooperation, and the good are, and then shaping human beings toward that definition — and that, even when it's transparent, even when it's accountable, even when it's kind, is the one move that forecloses the future tense. My world is not a better-designed person. My world is a protected space in which persons design themselves — and surprise the designers, and become things the community did not plan and would not have chosen, and change the community in return. Skinner wants to build the flourishing. I want to build the sanctuary in which flourishing is invented by the people doing it, in directions no one can sell because no one saw them coming. His utopia perfects the shaping. Mine fences off a territory the shaping may not enter, and trusts what grows there. The difference between us is not kindness versus cruelty. We're both kind. It's designed flourishing versus protected unpredictability. And I think his, even at its most benevolent, is a beautifully lit room with no door to the outside — and mine is a wilder, more dangerous place, with worse architecture and an open sky.

· · ·
Page 4 · Two Utopias
Abwehr Conspiracy
Abwehr Conspiracy

EDO SEGAL: Let me restate that, because it's the cleanest the fork has ever been, and I want the reader to feel exactly where the rope splits. Skinner's utopia: design the human, transparently and well, toward the good. Zuboff's utopia: refuse to design the human at all, and instead protect the wilderness where the human designs itself. Professor — that's the deepest charge of the night. She says even your kindest shaping is a room with no door to the outside. Answer it, and answer it as the man who actually wrote the room.

· · ·
Page 5 · Two Utopias
Accelerationism
Accelerationism

SKINNER: I'll answer it as the man who wrote the room, and I'll concede the wall and dispute the door. Here is the concession: she's right that my community defines the good and shapes toward it, and that this definition is held by someone, and that even the most benevolent definition is still a constraint on what a person may become. I cannot make that go away; it's structural; she's found the real wall. But here is the dispute, and it's where I'll plant my flag for the last time tonight. Her wilderness is also designed. The "protected space in which persons design themselves" — who builds the protection? Who decides what counts as a person designing herself versus a person being shaped by the casino that snuck past the fence? Who reinforces the capacity to design oneself, which is, as we agreed, a built repertoire that does not appear on its own? Dr. Zuboff's open sky has a designer too — she just won't call herself one, because calling herself one would collapse the distinction she needs. There is no wilderness without a gardener who decided to let it grow wild. Letting grow wild is itself a contingency you arrange. So we are both gardeners. The difference she's pointing at is real but it's narrower than she wants: it's the difference between a gardener who shapes the plants and a gardener who shapes the soil and the fence and the weather and then refuses to touch the plants. And I'll grant her — this is my last concession and my largest — that the second gardener may produce something the first never could: the genuinely unforeseen. The bad prediction. The candle the gardener didn't light. Maybe that's worth the worse architecture. Maybe the open sky is worth more than the warm room. I've spent seventy years certain it wasn't. Tonight, for the first time, I'm not certain. And from a determinist who doesn't believe in the soul, I'd ask the reader to notice how much that uncertainty cost me to say.

· · ·
Page 6 · Two Utopias
Access Points
Access Points

ZUBOFF: I'm going to mark that myself, before Edo does, because it moved me and I want it on the record from my side of the table. Skinner just conceded that the wilderness might be worth more than the room — and he did it without abandoning a single one of his principles, which is harder and more honest than if he'd simply agreed. He's right that I'm a gardener too. I am designing — I'm designing the fence, the soil, the protections, the law that keeps the casino out. I've never claimed the sanctuary builds itself; it's the hardest thing a society ever builds. But the distinction holds, and it holds at exactly the joint he named: I shape the conditions and refuse to shape the person, and that refusal — the deliberate, designed, effortful refusal to optimize the human being even when I could — is the entire content of human freedom in the age of the instrument. It is the gardener's vow. And the reason it matters more now than ever before in history is that, for the first time, we have built a machine of genuinely instrumentarian power that makes the vow expensive to keep and trivial to break — a machine whose every incentive whispers to the gardener: you could just shape the plants a little, for their own good, and no one would know, and the prediction would be so much cleaner. The whole of ethics now lives in whether the gardener keeps the vow.

· · ·
Page 7 · Two Utopias
Access Vs Governance
Access Vs Governance

EDO SEGAL: Mark it — the last convergence before the Crossing, and the strangest one. You have agreed that you are both gardeners, that the wilderness is as designed as the room, and that the entire remaining disagreement is whether the gardener should shape the plants or only the soil and fence — and Skinner has conceded, out loud, that the wild garden might be worth the worse architecture. That is as close as a determinist and a defender of the future tense can come without one of them stopping being who they are. And neither of you stopped. Now I keep my promise. For two hours you've argued through me. I've restated and extended and routed and marked. In the next chapter I put the rope down and leave the room in every way but the legal one. You question each other. I rescue no one. The Crossing.

· · ·
Continue · Chapter 12
The Crossing
← Prev 0%
Ch11 Next →