Shoshana Zuboff vs B F Skinner on AI · Ch6. Who Holds the Levers? ← Ch5 Ch7 →
Txt Low Med High
HOUR TWO — THE GRAIN HOPPER AND THE LEVERS
Chapter 6

Who Holds the Levers?

Page 1 · Who Holds the Levers?
Nudge Framework
Nudge Framework

EDO SEGAL: Professor Skinner, in 1948 you wrote a novel, Walden Two, about a community engineered from the ground up on behavioral principles — every contingency designed, by a planner named Frazier, so that people would be happy, productive, cooperative, and free of the aversive control that runs the outside world. Most readers found it a nightmare. You found it a blueprint. I want you to make the case for it tonight, in front of the one person on earth least likely to let you, because I think your honest answer reframes the whole debate. What does Walden Two get right that the horror of it makes us miss?

SKINNER: It gets right the thing everyone refuses to look at, which is that the contingencies of the outside world are already designed — designed badly, designed by accident, designed by whoever profits — and that the people in it are not free; they're controlled by debt and advertising and fear and the slow violence of aversive consequences. Frazier's community simply admits this and does it on purpose, for the good of the members, using positive reinforcement instead of punishment. People recoil from Walden Two because they see the planning and think control — and they don't recoil from the supermarket, the megachurch, the casino, the political campaign, all of which control them more and worse and pretend they don't. My community is honest. The outside world is the same machinery, lying about itself. That's what the horror hides: that the reader's revulsion at Frazier is revulsion at seeing the strings — and the moment they leave the theater, they walk back into a world with thicker strings and no Frazier, just a thousand anonymous hands pulling for their own gain. I'd rather have one Frazier I can see and argue with than a thousand I can't.

· · ·
Page 2 · Who Holds the Levers?

ZUBOFF: And there — there — is the sentence I have spent my life answering, so let me answer it precisely, because Skinner has just stated the surveillance capitalist's creed so cleanly that I could put it on the cover. "I'd rather have one Frazier I can see than a thousand hands I can't." Here is the problem, Skinner, and it is fatal. You don't get to choose Frazier. You never get to choose Frazier. The premise of the whole fantasy is a benevolent planner who designs the contingencies for the good of the members — and the iron law of every actual system built this way is that the planner is selected not by goodness but by power, and power flows to whoever can capture the apparatus, and the apparatus of behavioral design is worth capturing precisely because it works. You built a beautiful machine for steering human beings and you imagined it would be operated by a kindly scientist with no profit motive and no will to power. It is being operated by a handful of corporations whose Frazier is a quarterly earnings target. That is not a perversion of your idea. It is your idea, released into the actual distribution of power, which you abstracted away because abstracting away power is the one move your science can't do without. Walden Two isn't the alternative to Big Other. Walden Two is Big Other's recruitment poster.

· · ·
Page 3 · Who Holds the Levers?

SKINNER: That is the strongest thing you've said tonight and I'm going to take it standing up. You're right that I abstracted away the politics of who becomes Frazier. I treated it as solved when it was the entire problem. But notice — your critique is not against designing contingencies. It's against designing them without controlling the designer. And that, again, is a contingency problem! Who controls Frazier? The members must — through countercontrol built into the structure, through the power to remove him, to inspect the design, to refuse. I wrote that into Walden Two; the planners rotate, the members can leave, the experiment is revisable. You're attacking me for the one thing I tried hardest to solve and admit I solved imperfectly. But your alternative — to forbid the design entirely — doesn't remove Frazier. It just blindfolds us while the casino's Frazier works unopposed. At least my Frazier can be removed. The casino's Frazier you can't even find.

Zuboff's whole point about the modern instrument is that it works best on the people least able to inspect or leave — the silent middle, the distracted majority, the exhausted, the young.

EDO SEGAL: Let me press on exactly that seam, Professor, because I think it's where you're most exposed and most interesting. You say: the members control Frazier, they can leave, they can inspect. But Dr. Zuboff's whole point about the modern instrument is that it works best on the people least able to inspect or leave — the silent middle, the distracted majority, the exhausted, the young. The countercontrol you're counting on requires a population with the time, knowledge, and power to exercise it — and the instrument's first effect is to erode exactly that capacity. So isn't your countercontrol a fair-weather safeguard — present when you need it least, absent when you need it most?

· · ·
Page 4 · Who Holds the Levers?

SKINNER: That is a genuinely hard question and I won't dodge it. You're describing a positive feedback loop: the instrument that needs countercontrol also weakens the capacity for countercontrol. If that's true — and I suspect it partly is — then it's the most serious objection to my entire program, more serious than Chomsky ever managed. My answer is not a refutation; it's a redirection. If the instrument erodes the capacity for countercontrol, then the first and highest priority of design must be to build countercontrol back into the population — to use the same powerful contingencies to reinforce inspection, refusal, deliberation, the very habits Dr. Zuboff prizes. The instrument that can make a person passive can, on the same principles, make a person more capable of resistance than any unaided human ever was. Which one it does is a design choice. She sees the instrument eroding autonomy and concludes: ban it. I see the same erosion and conclude: aim it the other way. We're staring at the identical danger. She wants to take the tool away from the dangerous hands. I want to wrench the tool toward the building of free people. And I'll grant her — soberly — that nothing in the current ownership makes my version likely. It makes it necessary, not likely.

· · ·
Page 5 · Who Holds the Levers?

ZUBOFF: I want to honor that, because Skinner just conceded the deepest objection and didn't flinch — and then I want to show why his redirection fails at the one joint where it has to hold. He says: use the instrument to reinforce resistance, to build free people. But who decides what counts as a free person? In his framework, "free" cashes out as a set of reinforced behaviors — and the party arranging the reinforcement defines the target. So when Skinner says "we'll reinforce countercontrol," he has to answer: countercontrol of what, toward what, defined by whom? And the moment you answer that, you've reintroduced exactly the problem — a Frazier choosing what freedom looks like and shaping people toward his definition of it. There is no neutral place to stand. Reinforcing "the capacity to resist" is itself an act of control, and it produces a person who resists in the ways the controller selected and not in the ways he didn't. You cannot engineer your way to genuine autonomy, because the engineering is the heteronomy. The free person is not a person who was reinforced toward freedom. The free person is a person the controller failed to fully predict. And a science whose entire ambition is the elimination of that failure cannot, even in principle, produce the thing it promises.

Zuboff says: the act of reinforcing freedom is itself control, which means engineered freedom is a contradiction — the free person is precisely the one the engineer failed to fully shape.

EDO SEGAL: Stop. Mark this, because it's the sharpest thing the night has produced and the reader needs it slowly. Skinner says: we can reinforce the capacity for freedom. Zuboff says: the act of reinforcing freedom is itself control, which means engineered freedom is a contradiction — the free person is precisely the one the engineer failed to fully shape. That is not a disagreement about facts. That is a disagreement about whether freedom is a behavior you can build or a residue that survives the building. Professor, that's the hardest thing she's said. Take your time.

· · ·
Page 6 · Who Holds the Levers?

SKINNER: I'll take it, and I'll concede more than my followers will like. There is something in her "residue" that my science genuinely cannot capture, and I've felt the edge of it before. When I tried to define the free person as a set of reinforced behaviors, I was always left with a remainder — the novelty, the response no schedule predicted, the variation that selection acts on but doesn't produce. Evolution needs variation it didn't design; so does behavior. Where the variation comes from, I treated as noise. She's calling the noise the soul. And I think she's wrong to call it that — it's not a soul, it's the irreducible spontaneity of a complex system, the part of the organism that even perfect knowledge of its history couldn't predict. But I'll grant her this, and it costs me: a science of total prediction would, if it succeeded, eliminate the very variation that freedom lives in — and so the success of my program would be the death of the thing she's protecting. I never had to face that, because my program never came close to succeeding. This instrument might. And so for the first time her objection isn't philosophy. It's a forecast. The better the prediction gets, the less room is left for the unpredicted person. That should frighten me more than it ever has, and I think, tonight, it does.

ZUBOFF: Then we have arrived somewhere real, Skinner, and I won't pretend it doesn't move me that you walked here with me. The success of behavioral design and the death of the future tense are the same event seen from two chairs. You spent your life wanting the first. I've spent mine warning about the second. And it turns out they're one floor of the same tower.

EDO SEGAL: Hold that — because that single sentence reorganizes the rest of the evening. The next round takes the two words Skinner spent a book trying to abolish and Zuboff has spent a life trying to save. Freedom. Dignity. And the right to a future no one has already sold. After this.

· · ·
Continue · Chapter 7
Freedom, Dignity, and the Right to the Future Tense
← Prev 0%
Ch6 Next →