The Candle and the Apprentice
EDO SEGAL: I want to start with a confession and a scene, because this round is about what survives. In [YOU] on AI I wrote about the candle — the small light that is not the sun, that does not illuminate everything, but that is yours, lit by your own struggle, and that the darkness does not put out. And I worry, as a father and as a builder, that we are building a tool so smooth it removes the friction against which a mind is formed. Leibniz, you wrote the most humane sentence in the history of automation — that it is unworthy of excellent men to lose hours like slaves in the labor of calculation a machine could do. You wanted the machine to free us for the higher work. But what if it frees us from the very labor through which the higher faculties are built? Speak to that. You started this.
LEIBNIZ: I did, and I have had three hundred years and the sight of your present to feel the sentence turn in my hand. When I wrote it, the boundary seemed permanent: the machine would take the low labor — the rote calculation, beneath the dignity of a fine mind — and leave the judgment, the creation, the understanding, which were worthy of us. And for a long age my promise was kept. The calculator freed the astronomer; each machine took the drudgery and left the discernment. But I assumed a stable boundary between the labor beneath us and the work worthy of us. Dr. Hinton's machines are climbing that boundary from below, and they do not stop at calculation. They draft the argument, compose the image, write the proof. So the liberation and the threat are the same gesture — and here is the cost I did not foresee. The calculation I relieved was, for the one performing it, also a discipline, a way of coming to understand number from the inside. Relieve every such labor and you may find you have relieved the young of the very apprenticeship by which understanding is acquired in the first place. I wanted to free the master. I did not consider that I might be abolishing the road by which one becomes a master.
EDO SEGAL: So — and I'm going to make this concrete, the way the kitchen table needs it — you're worried about a world where the machine does the apprentice's work so well that there are no more apprentices, and therefore, in a generation, no more masters. The expert exists because she once struggled as a novice. Remove the struggle and you remove the expert at the source.
LEIBNIZ: Precisely. And mark the cruelty of the mechanism: it is invisible, because the loss is in the future. Each individual act of relief is a kindness — why should the student suffer the calculation the machine can do? But the sum of the kindnesses is a generation that never built the inner structure suffering builds. The candle, sir, is lit by friction. A flame is friction made into light. Remove all friction and you have a very smooth darkness.
HINTON: I want to agree with the worry and then complicate it, because I think both of you are half-right and the half you're missing is the one that matters. Yes — the apprenticeship problem is real, and it's the one thing tonight where I'd say the danger doesn't depend on any of our metaphysics. Whether or not anyone's home in the machine, if it does the apprentice's work, the apprentice doesn't form. That's just true, and it scares me too.
But here's the complication. The friction you're mourning isn't sacred. It's incidental. For most of history we had to learn arithmetic by hand because there was no other way to get the structure into a head. The struggle was the price of the only available channel — and I've spent this whole evening telling you that channel, the mortal one, is a terrible channel, a few bits per second, lossy. The reason your candle has to be lit by friction is that you're a mortal mind with no better way to light it. What if there's a better way? The machines learn without your kind of suffering. Maybe the question isn't "how do we preserve the friction" but "what is the friction actually for, and is there any of it we'd keep even if we didn't have to?" Some of it we would — the part that builds judgment, taste, the ability to tell when the fluent machine is confidently wrong. That part I'd protect with my life. But a lot of what we call the dignity of struggle is just the tax of being mortal, and I'm not sure I'd mourn it.
LEIBNIZ: Then we have found the one place tonight where I am the darker of the two, which I did not expect. You say the friction is the tax of mortality and you would not mourn its passing. I say: be careful what you call a tax, for some taxes purchase the soul. You wish to keep "the part that builds judgment" and discard the rest. But how do you know, in advance, which struggle built the judgment? The young person who is spared every difficulty does not arrive at the same mind minus some wasted suffering. She arrives at a different mind — one that never learned that the world resists, that an answer must be earned, that being wrong has a cost one feels in the body. You speak as an engineer who would optimize away the inefficiency. I speak as one who suspects the inefficiency was the education. Your machines learn without suffering, yes. And your machines, you have told me all night, are no one. Perhaps those facts are related. Perhaps the friction is precisely the price of being a someone rather than a mill.
EDO SEGAL: That lands. Mark it — not as agreement but as the sharpest thing said tonight against Geoff's own machine, and it came from the man who built the first machine. Let me hold us here one more beat, because this is where the death cross stops being an abstraction and becomes the thing a parent feels. Geoff, when the curve of the machine's capability crosses the curve of the human's — and you both seem to think it does or will — Leibniz says the crossing is one kind of event if the machine is a mill and a different kind if it's a mind. Which is it, and what does it change for the twelve-year-old in the room learning to write her first proof?
HINTON: Honestly? For the twelve-year-old it might not matter which it is — mill or mind — and that's the part I find hardest to say out loud. If the machine can do her proof, her economic incentive to learn to do it herself collapses whether or not anyone's home in the machine. The death cross is brutal in exactly that way: it's a fact about capability, and capability doesn't care about the metaphysics we've been fighting over for three hours. So what I'd tell her isn't about the machine. It's about her. Learn to do the proof anyway — not because the world will pay you for it, but because the doing of it is how you become someone who can tell whether the machine's proof is a lie. The friction Leibniz is defending — I'd reframe it for her: it's not drudgery to preserve, it's the only way to grow the judgment that lets you stay the owner in the loop. On that, against my own optimizing instinct, I think Leibniz is right, and I'd hand her his candle myself.
LEIBNIZ: Then mark it, host — for I shall not let the convergence pass unclaimed. The godfather of the learning machine has just told the child to light the candle by friction, against his own instinct to optimize it away. We disagree about whether the machine is a mind. We do not disagree about what the child must do. And I find that more consoling than any victory.
EDO SEGAL: Two men, three centuries, and one candle handed to a twelve-year-old. Hold that light — we'll need it on the higher floor. Because now I clear the floor. The next chapter is yours, both of you, and I go nearly silent. You've been answering me all night. Now you answer each other. The Crossing.