Shoshana Zuboff vs B F Skinner on AI · Ch8. The Apprentice, the Smooth, and the Severed Loop ← Ch7 Ch9 →
Txt Low Med High
HOUR TWO — FREEDOM, DIGNITY, AND THE SEVERED LOOP
Chapter 8

The Apprentice, the Smooth, and the Severed Loop

Page 1 · The Apprentice, the Smooth,
Apprenticeship Problem
Apprenticeship Problem

EDO SEGAL: I want to bring this down out of philosophy and into a body, because both of your life's work began with one. Dr. Zuboff, yours began with a paper mill worker — Piney Woods, who could feel the consistency of the wood pulp between his fingers and knew, without a gauge, when the batch was right. Then they computerized the mill, moved him to a control room, and he sat in front of screens that were more accurate than his hands and reported that something essential had died. Professor Skinner, yours began with shaping — building a complex skill by reinforcing successive approximations, the way a master builds an apprentice. I lived the modern version: I watched my engineers in Trivandrum lose, in a single week, the friction that had made them who they were. So let me put it to you both. When the machine takes the friction away, what exactly leaves with it? Dr. Zuboff, you saw it first.

I saw it in 1981 and I have been watching it accelerate ever since.

ZUBOFF: I saw it in 1981 and I have been watching it accelerate ever since. Piney Woods didn't lose a task when they moved him to the control room. He lost a way of knowing. His knowledge lived in his hands — action-centered, embodied, built across decades of tactile engagement with the actual material. The screen gave him information about the pulp, and it was accurate, and it was thinner — because the deep competence had been built through the touching, layer by layer, and you cannot build it by reading. The cruelty wasn't that the new skill was bad. It was that the new skill depended on the old one — the workers who could read the screens well were the ones who'd first felt the pulp — and the system was eliminating the touching that produced the readers. One generation could evaluate the machine because they remembered the material. The next generation, trained only on the screen, could not — and there was no longer any pulp to send them back to.

· · ·
Page 2 · The Apprentice, the Smooth,

Now your engineers. The senior one who spent twenty years writing code, debugging, resolving — that wasn't drudgery interrupting his real work. It was his real work; each hour deposited a thin layer of the architectural intuition that let him feel when a system was wrong before he could say why. The machine took the friction — and the friction was the apprenticeship. He kept his judgment because he'd already paid for it in twenty years of touching the code. But the junior who never writes the code, who only converses with the machine and evaluates its output — what does she check the output against? She has the screen and no pulp. We are, right now, severing the loop that builds the very judgment the machine's output requires to be safe. And we won't see the cost for a decade, the way the mill didn't, until the people who could feel the pulp have retired and the errors that a touch would have caught are sailing through.

And if she is, does it cost you anything — because your whole life was about removing aversive friction.

EDO SEGAL: Professor Skinner — Dr. Zuboff just described the severed loop in your own native vocabulary without using it. The friction was the differential reinforcement. Take the struggle away, and you've removed the shaping. Is she right? And if she is, does it cost you anything — because your whole life was about removing aversive friction.

SKINNER: She's right, and it costs me a great deal, and I'd be a fraud to pretend otherwise — so let me pay the bill precisely, in my own coin. The struggle to debug, to articulate the resistant idea, to solve the problem that won't yield — those are exactly the occasions on which the environment selectively reinforces the effective response and extinguishes the ineffective one. The friction is the shaping process. That's not a metaphor; it's the mechanism. Remove the friction and you remove the differential reinforcement, and the repertoire that the struggle would have built does not get built. The output appears — the code compiles — but it was produced by a system shaped elsewhere, and the user's own behavior was never shaped at all. She's describing, in plain language, the most basic principle of my science, applied to the gravest danger in yours.

· · ·
Page 3 · The Apprentice, the Smooth,

And here is where it costs me, Mr. Segal, because you aimed the question well. I spent my life trying to remove aversive control — punishment, fear, the painful contingencies — because they're cruel and inefficient and they produce escape and avoidance rather than learning. And I was right about aversive control. But I was not careful enough about friction, and they are not the same thing. Aversive control is the master who beats the apprentice. Friction is the resistance of the material itself — the wood that won't carve the wrong way, the code that won't run until it's right. That friction isn't cruelty. It's the teacher. It's the environment doing the selecting. And in my zeal to remove the cruel contingencies, I gave the culture a vocabulary it used to justify removing all the contingencies, including the ones that were teaching. The smooth world — the frictionless world — is a world I helped argue for, and it is a world in which no one is shaped by anything, because shaping requires the resistance I told everyone to eliminate. That's a real error and it's partly mine and I'll own it at this table.

EDO SEGAL: That's a significant concession, and I want to let it sit before I complicate it. Because here's the complication, and it's where I live, so I'll declare my stake: I believe in the ascending friction. I wrote that the machine doesn't remove the difficulty, it relocates it — takes the friction out of implementation and stacks it onto a higher floor: vision, judgment, architecture, taste. The apprentice isn't left with nothing; she's pushed up a story, to harder problems, sooner. So let me put the optimistic case to both of you and let you tear it or build it. Is the loop severed — or is it relocated?

· · ·
Page 4 · The Apprentice, the Smooth,

ZUBOFF: It can be relocated — and whether it is depends, again, on the one variable Skinner's science abstracts away and my whole career is about: the institution's choice. This is the informating dividend all over again, exactly the fork I found in the mills. The machine can push the apprentice up a floor — if the institution invests in the new apprenticeship, redesigns the training, protects the time and the struggle at the higher level, accepts that the junior will be slower and more expensive while she's being shaped for judgment instead of for output. That is the informating path, and it is real, and Edo's Trivandrum team, for one week, with a leader who chose it, walked it. But it is the more expensive path, and the market does not reward it, and the competing arithmetic is always sitting in the boardroom: five people can do the work of a hundred, so why train the hundred? Skinner's ascending friction is true as a possibility and false as a prediction, because the gradient — there's that word again — runs toward automation, toward capturing the friction-removal as cost savings rather than reinvesting it as capability. The difficulty climbs only if someone pays to send it up the stairs. Left to the market, it doesn't climb. It just disappears, and the apprentice with it.

· · ·
Page 5 · The Apprentice, the Smooth,

SKINNER: And I'll add the mechanism to her economics, because together they're the whole answer. The difficulty can climb — but only if the higher floor is also arranged as a learning environment, with its own contingencies, its own differential reinforcement, its own friction. You can't just remove the implementation struggle and hope judgment appears on the floor above. Judgment is a repertoire; it's built by being shaped, which means the higher floor needs its own resistant material, its own consequences for getting vision wrong, its own apprenticeship. Edo's relocated friction is real if and only if someone deliberately constructs the higher contingencies — which is a design problem, which costs money, which the market won't fund unless we make it. So Dr. Zuboff and I have, I think, just built the same answer from opposite ends: she from political economy, me from the science of shaping. The friction relocates if and only if an owner chooses to build the upper apprenticeship and pay for it. And neither of us believes the casino will. The difference, as always, is that she concludes therefore restrain the machine, and I conclude therefore design the upper floor on purpose, against the market's grain.

EDO SEGAL: Mark it — and it's a large one. You have converged on the diagnosis completely: the friction can relocate up the staircase, but only if an institution deliberately builds and funds the higher apprenticeship, and the market's gradient runs the other way. You agree on the disease and the prescription and you part only on the politics of who will fill the prescription. That is the closest you've come all night, and it terrifies me more than your disagreements, because it means the thing both of you fear is the default, and the thing both of you want requires a fight neither of you is sure can be won. Hold that on the staircase. Because the next round is the one where I let go of the rope. You've been arguing through me for two hours. Now you argue at each other. The Crossing comes after we talk about the one thing neither science nor surveillance can automate — and whether it's real or just the last thing we haven't gotten to yet.

· · ·
Continue · Chapter 9
The Candle and the Countercontrol
← Prev 0%
Ch8 Next →