The Apprentice and the Candle

EDO SEGAL: Let me bring this down to a kitchen table, because we've been three hundred feet up. A father I know — me, actually — watches his kids grow up alongside these machines, and the question that keeps him awake isn't whether the machine can know. It's whether his kids will still get to. Here's the worry, plainly. For all of human history you became good at something by struggling at it — the apprentice ruined a hundred joints before he cut a clean one, the resident missed a hundred diagnoses before she saw the right one fast. The struggle was the learning. Now the machine does the first draft, the first cut, the first diagnosis, smoothly, instantly. Rich — you of all people believe knowing is learning from experience, from struggle against a world that pushes back. So I want to ask you the cruelest version: if the machine absorbs the struggle, does it absorb the apprenticeship — does it take the very experience our kids needed in order to become anyone at all?

SUTTON: That's the question that actually scares me, and it's mine more than Gottfried's, so let me not flinch from it. My whole life's claim is that there's no knowing without experience — that you cannot be handed competence, you can only grow it by acting and being corrected by consequences. Take that seriously and you get a frightening corollary: a person who is handed answers instead of consequences does not learn, in the strict sense. They accumulate the appearance of competence with none of the substance, because the substance was always the error signal — the missed cut, the wrong diagnosis, the surprise. If you remove the surprise, you remove the learning. So yes. A tool that smooths away every error before the learner feels it is, by my own theory, a machine for preventing knowing. I built a science that says the struggle is load-bearing. I have to look at your kids and tell you the struggle is load-bearing, and that a thing which removes it removes them.

EDO SEGAL: That's a hard thing to hear from the man who believes in learning most. Gottfried — does your side soften this or sharpen it?

LEIBNIZ: Mine softens it, and I am almost embarrassed to find myself the consoler, since I am usually the one accused of cold rationality. But hear me. If knowing were only experience, as Mr. Sutton holds, then to lose the struggle is to lose everything, and his despair is exact. But I have spent the evening insisting that there is a structure under the experience — that the mind arrives with a form, a capacity, an architecture that experience fills but does not create. The child is not a blank reckoner waiting for inputs. It has, in my language, the innate equipment of reason; in plainer language, it brings something to the encounter. And if that is so, then the machine doing the first draft need not steal the child's becoming — provided the child still does the thing the structure is for, which is to judge. Let the machine cut the joint. Let the child decide whether the joint is good, and why, and what a better one would be. The apprentice's hand may be relieved. The apprentice's judgment must still be exercised, or you are correct, sir, and we have raised a generation of fluent know-nothings. The danger is real. But it is a danger of how we use the tool, not a law of nature. We can preserve the friction that matters and shed the friction that was only toil. That distinction — which friction is load-bearing and which is mere drudgery — is the most important practical question in the room, and it is answerable.

SUTTON: I want to push on that, because it's seductive and I think it's half wrong. You're drawing a clean line — let the machine do the toil, keep the judgment. That's exactly the line I built between the easy mechanical part and the hard cognitive part, and the bitter lesson is the story of that line moving. Every time we said "the machine takes the rote part and leaves us the judgment," the machine came for the judgment next, because judgment turned out to be more learnable than we thought. So I don't trust the line. I don't think you can durably reserve "judgment" for the human, because judgment is just experienced pattern, and the machine accumulates experienced pattern faster than the child. Where I land with you is grimmer and I think truer: it's not that we keep judgment and give away toil. It's that we have to choose, deliberately and against convenience, to keep doing things the slow way even when the fast way is available — to protect the struggle on purpose, as a discipline, because nothing in the technology will protect it for us. The market gives you the smooth path. You have to refuse it, for your kids, by hand. That's not a law of nature. It's a fight, and most people will lose it because the smooth thing is right there.

EDO SEGAL: I want to name what just happened, because it's the strange topology of this whole evening. The empiricist, Rich, the man of experience, just told the father to manufacture artificial friction — to refuse the smooth path on purpose. And the rationalist, Gottfried, the man of pure reason, is the one defending the innate thing in the child that can't be taken. You've traded coats. Say more, Rich — why is the smooth thing the enemy here, when your whole life was about making learning more efficient?

SUTTON: Because efficiency of learning and removal of learning look identical from the outside and are opposites underneath. A good teacher makes your learning efficient by giving you the right struggles — the ones at the edge of what you can do, where the error signal is richest. That's efficient. A bad tool makes learning "efficient" by removing the struggle entirely, which feels like the same thing and is actually its negation. The candle you're holding up, Edo — the small light a person carries into the dark — it doesn't get brighter by someone handing you a finished map of the cave. It gets brighter by your walking the cave and bumping the walls. Hand someone the map and they hold a bright light and have learned nothing about darkness. My fear is that we've built the most magnificent map-handing machine in history, and we'll mistake the brightness for knowing, and a whole civilization will stand in a lit room having never once been in the dark, which is the only place anyone ever actually learned anything.

LEIBNIZ: And here, at last, we agree without reservation, and I want it recorded because it cost us both something. I came to defend composition and Mr. Sutton came to defend convergence, and we meet on this: that a human being who is handed the answer has been robbed, whichever theory of knowing is true. On my account the child's reason goes unexercised and atrophies; on his, the child's experience is stolen and never accrues. Two opposite theories of mind, one identical warning to the parent. The machine that knows for you is, by either light, a thief of your knowing. I did not expect to leave my century to find the reinforcement-learning man and the rationalist standing on the same small square of ground, holding the same candle, telling the same father the same true and unwelcome thing.

EDO SEGAL: Mark that convergence — the second of the night, and the one I'll route through the kitchen table at the end. Two theories that agree on nothing about what knowing is, agreeing completely on what protects it. Hold the candle. The next round takes the conversation to the most dangerous idea Rich has ever proposed — more dangerous than any machine — which is that everything you have ever wanted, every purpose you've ever felt, is the maximization of a single number. Gottfried has a thing or two to say to that, because he built a God who did exactly that.