Did Nature Speak, or Did the Data?

EDO SEGAL: Professor Kant, your definition of genius has a phrase in it that I have been waiting all evening to press, because the machine puts unbearable pressure on it. You said genius is the talent through which nature gives the rule to art. Nature. Not the artist's training, not her teachers, not the works she studied — nature, working through her, giving art a rule she herself did not author. I want you to defend that "nature," because Jürgen is about to say the machine has a nature too, and it is called the training corpus.

KANT: I will defend it, and I will start by conceding what the machine makes undeniable, because conceding it sharpens rather than weakens my claim. The genius does not create from nothing. She is steeped in the tradition; she has studied the models; the materials of her art come to her from her culture, her language, the works she loves. None of that is in dispute, and the Romantic caricature of genius as a lightning bolt from an empty sky is not my view. When I say nature gives the rule through her, I mean something more precise. The materials are given — by culture, by training, by everything she has absorbed. But the unity she makes of them, the exemplary form that organizes the materials into a whole no rule prescribed, comes through her as a gift she cannot account for, and that gift I attribute to nature because she did not give it to herself. It is not the sum of her influences. If it were the sum of her influences, it would be derivable from them, and we are back to skill. The "nature" in my formula names exactly the surplus over the influences — the part of the work that the influences do not explain.

SCHMIDHUBER: And there, Professor, with great respect, is where I think the briefing on the machine should change your mind, or at least cost you a night's sleep. You have defined the "nature" that speaks through the genius as the surplus over her influences — the part of her output not derivable from her training data. But that is a quantity I can now examine directly, because the machine is its training data plus an architecture, fully specified, nothing hidden. And here is the unsettling finding: the machine's best outputs are also a surplus over its training data. It does not retrieve. It does not collage. It generalizes — it learns the deep regularities and then produces points that are not in the data, that no individual training example contains, that emerge from the compression of the corpus into something far smaller than the corpus. The surplus you attribute to "nature" in the human, I can attribute to generalization in the machine — to the fact that a good compressor of a domain can generate genuinely new instances of that domain, instances that are lawful-without-being-copied, which is your definition of the exemplary. So when you say "nature gave the rule," I say: yes, and nature's name, in both cases, is the generalization that a deep enough compression of the influences makes possible. The surplus is real. It is also mechanical. The artist and the model both produce more than was put in, because compression is generative.

EDO SEGAL: Let me make sure the reader feels the size of that, because I think it is the most dangerous thing Jürgen has said. Professor Kant, you defined genius as the surplus over the influences — the part you can't trace back. And Jürgen has just said: I can build a system that also produces a surplus over its influences, demonstrably, because generalization is exactly the production of new lawful points that aren't in the data. If he's right, your "nature" isn't a soul speaking. It's what generalization looks like from the inside when you can't see the weights. That's the claim. Take it apart.

KANT: I will take it apart at the joint where it is assembled too quickly. Professor Schmidhuber's word is "generalization," and he treats generalization and origination as the same thing. They are not. Generalization produces new instances of the same kind — new faces, new melodies in the style, new points within the manifold the data defined, however far from any single example. That is real and it is impressive and I do not minimize it. But the work of genius is not a new instance of an existing kind. It founds a new kind. When the form that would later be called the symphony first cohered, or when a way of seeing first entered painting that reorganized what painting was for everyone after — that is not generalization within a space. It is the institution of a new space, and here is the test that separates them cleanly: generalization, however powerful, presupposes the space it generalizes within, which means it presupposes that the kind already exists to be extended. Genius is what brings the kind into existence in the first place, so there was no space to generalize within when it acted. The machine generalizes superbly within the kinds humans have already founded. Show me the machine that founds a kind — not a new instance, a new kind, one that reorganizes the practice for everyone after — and you will have shown me genius. Until then you have shown me the most powerful generalizer ever built, working entirely inside spaces that human genius founded and the machine inherited in its corpus.

SCHMIDHUBER: This is your strongest move and I want to honor it before I answer, because "founding a kind versus extending one" is a real distinction and I will not pretend it is not. Two replies. First, empirically: the boundary you are drawing has been moving in one direction my entire career. Every time someone names a thing the machine "merely extends but cannot found," and we scale and train a few more years, the machine does a version of the founding — it discovers strategies in games that no human had played and that changed how humans play the game after, which is precisely founding a new region of the practice. The moves that reorganized the game for every human who came after were made by a machine. That is your "new kind, exemplary for all after," and it has already happened, in a domain narrow enough that we can verify it cleanly. Second, conceptually: founding a kind is also a search — a search in the space of possible spaces, the transformational level. It is harder, it is rarer, it requires an agent curious enough to go where no existing space reaches. But "harder and rarer" is not "non-mechanical." My PowerPlay agents do exactly this in miniature: they invent their own new problems, founding new little kinds of task and then solving them, generating their own open-ended curriculum. The founding of kinds is not outside computation. It is the part of computation Professor Kant has not yet been shown, and I understand why — it is the frontier — but the frontier is not a wall.

EDO SEGAL: [pause] I have to step into my own bias here, because the round has reached the place where it touches me. I have watched a machine make a move — in code, in an image, once in a sentence — that I would have sworn founded something, that changed how I worked after. And I have never been able to decide whether I witnessed origination or whether I was the one who founded it, in the act of being changed by it — whether the new kind was in the output or in my reception of the output. Professor Kant, that is my honest confusion. Where does the founding happen — in the maker, or in the one who is changed?

KANT: That is the deepest question you have asked, and I will answer it in a way that may surprise you, because it concedes something to Professor Schmidhuber. The founding, in part, happens in the reception — you are right to suspect it. The work of genius becomes exemplary only by being taken up as exemplary by a community of judging subjects, by you and by everyone who orients to it after. The genius does not certify her own genius; the sensus communis does, over time. So the machine's output and the human genius's output may, at the moment of production, be more alike than I have admitted — both are forms cast into the world. But here is what does not change: the taking-up, the founding-in-reception you just described, is done by subjects who feel — by you, at the shore, with the ache. The machine can cast the form. It cannot be a member of the community that founds the form as exemplary by feeling it together, because it does not feel and is not a member. So even if I grant that the founding happens partly in reception, I have not lost my boundary — I have relocated it to exactly the place you keep flinching from, which is whether the maker, or anyone, is home to feel. We will get there. But mark that you just found the boundary yourself, in your own confusion.

EDO SEGAL: I did, and I felt it. Mark that convergence — both of you now agree that whether the machine creates may turn on whether anyone is home to feel, in the maker or the receiver. That is the second time the evening has funneled to consciousness, and we have a whole round for it. But first, the community that does the founding — the sensus communis, the common sense Kant says grounds every judgment of taste. Is the machine in the room where beauty is decided, or only on the wall? After this.