The Apprentice and the Candle

EDO SEGAL: This round opens with a confession, because I owe the table a toll and I have been routing too much through my guests. I spent my life believing that the way you become someone who can make is by making badly, for years, in private — the apprenticeship, the ten thousand ugly drawings, the friction. I built companies on the back of people who had ground through that friction. And now I watch young people skip it. The machine hands them the polished output before they have made a single ugly thing, and I do not know whether I am watching liberation or the quiet paving-over of the ground where makers grow. I tell parents the candle in the darkness is the asking, not the answering — that the machine is an engine of answers and the child is an engine of questions. Tonight I get to test my consolation against both of you. Professor Kant — you wrote that genius must be cultivated, that even the gifted must labor. What does the machine do to the apprentice?

KANT: It threatens her at the precise point where she is made, and your instinct about friction is, I think, correct and Kantian. I held that genius supplies the spark but that taste, the capacity to judge, must be cultivated — and cultivated the hard way, by long exposure, by the labor of judging one's own work against the models and finding it wanting, again and again. The spark may be a gift of nature, but the discipline to make the spark into a finished work is acquired only through exactly the friction you describe. Here is my fear, and it is sharper than yours. The danger is not mainly that the machine makes the apprentice's products unnecessary. It is that it makes the apprentice's struggle feel foolish — and the struggle was never inefficiency to be removed. The struggle was the cultivation of judgment itself. A young person who can summon a finished form before she has developed the taste to judge it has been handed the products of cultivation without undergoing the cultivation, and judgment does not grow under those conditions, because judgment grows only by being exercised against resistance. You worried about the asking-muscle. I worry about the judging-faculty, and they are the same worry: a capacity that matures only by use, offered an existence in which it need never be used.

SCHMIDHUBER: I am going to do something my side rarely does and agree with the diagnosis, and then turn it over to show the floor under it. Professor Kant is right that judgment is cultivated by friction, and right that removing all friction would starve it. I watched the same thing in my own field — the young researcher's intuition came from years of code that did not work, and if you hand her code that always works, where does the next deep intuition come from? Nobody has fully answered that, including me. But here is the floor, and it is the same floor I always find. The friction you both want to protect was, for almost everyone in history, not a noble apprenticeship — it was a wall. The struggle that cultivated the rare child lucky enough to have a teacher, a studio, a tradition, was for everyone else an impassable barrier. The machine that removes the friction also removes the wall. A curious child in a place with no conservatory, no mentor, no models, can now pull on the thread of any craft as far as her curiosity runs, with a tireless interlocutor that meets her exactly at the edge of what she can almost do — which, I will note, is exactly where my theory says learning happens, the zone of maximal compression progress. So I hold both, and uncomfortably: the friction we are removing cultivated the few, and the wall we are removing excluded the many, and they are the same wall.

EDO SEGAL: That is the honest tension and I am not going to let either of you resolve it cheaply, because I live inside it. But let me press the thing under both of your answers, the thing my consolation rests on. I tell parents the child is an engine of questions. Jürgen, your whole theory is that curiosity itself — the asking — is the mechanizable part. The curiosity that comes before the prompt is exactly compression progress. So I have to ask you the question I am most afraid of: if you are right, then even the asking is not safe. Even the candle is a flame the machine carries. Is there anything left that is ours to make?

SCHMIDHUBER: I will answer it without flinching, because you asked it without flinching. If my theory is right, then no — the asking is not metaphysically ours. The curiosity drive is a mechanism, and machines will have it, and they will ask better questions than we do, about deeper structures, faster. I cannot tell you the candle is safe; I would be breaking our rule. But here is what I can tell you, and I believe it is true and not consolation. Even granting all of that, you are still the one for whom the answers matter. The machine that compresses the cosmos does not, on my own account, care in the human sense — it computes the derivative; whether anything is at stake for it is exactly the open question Professor Kant keeps marking, and I keep deferring, and I will defer once more. But there is no open question about you. Something is at stake for you. The new and the beautiful land on you and change how you live the one mortal life you have. The machine may make the candle. You are the room it lights. And I do not think that is nothing. I think, on my hardest days, it may be the only thing I am sure of.

KANT: [pause] That is more than I expected from him, and I want to meet it rather than score on it, because he has come most of the way to my own position without seeming to notice. He says the machine computes the derivative but that something is at stake only for you. That is my claim, in his vocabulary. The dignity I have defended all evening was never a claim that humans do something machines cannot do — I conceded hours ago that at the level of performance we may be matched entirely. It was always a claim about being the kind of being for whom things are at stake, a being that is an end in itself and not merely a means, a subject and not only a process. Professor Schmidhuber has just located that being exactly where I locate it — in the one for whom the beautiful matters — and called it the only thing he is sure of. So I will say to the parent, through Edo: tell your child that the machine may one day ask better questions than she can, and that it will not matter, because she is the one the questions are for. Her dignity was never in being the best asker. It was in being a someone to whom the answers come home. Cultivate that — cultivate the capacity to be moved, to judge, to care — and you have cultivated the one thing no machine can do on her behalf, because being-moved is not a service that can be performed by a proxy. No one can be moved for you.

EDO SEGAL: Mark this, because it is the largest convergence of the evening and it surprised them both. The man who says creativity is an equation and the man who says it is a gift of nature have just agreed on where the human stands: not as the best maker, but as the one for whom making matters. They still disagree completely about whether that — the mattering, the being-home — is itself a mechanism. But they agree it is the hinge. [long pause] And that means we can no longer defer the question we have deferred four times tonight. After two more turns I am going to stop refereing and let them ask it of each other directly. But first, the round that has to come before consciousness, because it sets the terms: dignity, freedom, and whether a thing built to maximize a reward can be the author of anything at all. After this.