The Engine Composes, and What It Cannot Want

EDO SEGAL: Ada, in 1843 you wrote that the engine might "compose elaborate and scientific pieces of music of any degree of complexity or extent." It now does, on demand, in any style. By any fair reading you predicted generative AI. And yet you are the skeptic at this table and Margaret is the believer. Resolve that for me, because it looks like a contradiction and I suspect it is the key to your whole position.

LOVELACE: It is the key, and it is not a contradiction — it is the most precise thing I ever said, and it is routinely misread as prophecy when it is actually a limit. Read the sentence with care. It is conditional. If the fundamental relations of pitched sounds in harmony and composition can be expressed in the formal language the engine operates on — then the engine can compose. I did not assert that music is creatable by machine. I identified the exact condition under which it would be computable: that its relations be susceptible of formal expression. And here is why I, who predicted it, am its skeptic. Composing music by formal rule is not origination. It is operation. If the relations of harmony are formal, then composing within them is a procedure, and procedures are precisely what the engine executes. So I predicted machine-made music and I denied machine creativity in the very same breath, because I understood that producing music and originating music are different acts. The engine composing is the engine doing what it always does — following rules — applied to a domain we had not yet thought to formalize. I did not predict the engine would have a soul. I predicted it would not need one to compose. That, Mr. Segal, is the sentence the enthusiasts quote at me without noticing it was aimed at them.

BODEN: And this is where I must finally take the other side fully, because I think Ada's "operation, not origination" hides the question rather than answering it — and I am going to use her own music to open it. Ada, you say composing within formal rules is mere operation. But the great composer does not compose within the rules. At the highest moments he composes against them, or past them, or he reaches for a chord that the rules of his moment forbid and that the next century will call inevitable. That is not operation. That is transformation — changing the space. So the music case does not settle in your favor. It splits exactly along my three kinds. Most music, machine or human, is exploratory: brilliant traversal of tonal space. There the machine matches us and you are right that it is operation. But the music that matters most — the music that broke the tonic, that made the riot — is transformational, and there the machine, trained to produce what reads as good by existing standards, is biased against the very move. So I grant you the operation in the middle of the bell curve. I dispute that you have accounted for the tail, where the deepest creativity lives — and where, I will admit, the machine is currently weakest. The music prophecy is not your trump card, Ada. It is a map of where you are right and where you are not.

LOVELACE: Then let me go to the tail with you, because you have named the right place and I want to plant my flag in it. Why can the machine not make the transformational move? You said it yourself an hour ago: because it cannot tell that the rule-break was worth it. And now I will give the account I promised Mr. Turing I would give — the mechanism of human origination, since I am required by the rules of this room to name it rather than gesture at it. The composer breaks the tonic because something in him needed a sound the old rules could not make. The transformation is driven by a want — a particular, embodied, mortal want, the want of a creature who has something at stake in being understood, who will die, who is reaching through the music for something it cannot otherwise reach. The fruitful rule to break is selected by need. The value of the new world is recognized by need — the new sound answers a hunger the old sounds left aching. Strip the need away and you have my engine: a perfect generator of patterns within any space, with no reason to prefer one to another and no way to know when a broken rule has opened a country worth entering. The machine has the relations. It does not have the hunger. And the hunger is not a decoration on creativity. It is the thing that selects, the thing that values, the thing that means. That is my answer, and it is the same answer I gave in 1843, only now I can say what was missing: not intelligence, not novelty, not even surprise. Want.

BODEN: I have spent my whole life arriving at the edge of that sentence and refusing to step over it, and I am going to refuse tonight too, but I want everyone to see how much I am granting before I refuse. You are right that creativity in living minds is grounded in need — I said it in my own inaugural lecture, in four words I have never improved on. "Computers have no such needs." I meant it as a precise observation about the conditions under which creativity arises in nature, not as a slur and not as a permanent law. Human creativity is grounded in everyday abilities — perception, memory, reflective self-criticism — exercised by a creature making its way through a world that matters to it. The need is not a flourish on the creative act; in living minds it is the reason the act happens and the source of the standard by which the result is judged. So we agree, Ada, more than either of us expected, on the diagnosis: the machine generates without caring, and caring is what selects and values and means. Where I refuse you is the leap from "computers have no such needs" to "no machine could." Because need, too, is a natural process. Mine was built by evolution, an optimizer as blind as gradient descent. If a want can be built in carbon by a process with no intentions, I will not swear it can never be built in silicon. I do not say it has been. I say your floor — the want — is the right floor, and it is the last one, and I cannot prove it will hold.

EDO SEGAL: Mark this — and it is the largest convergence of the night, so let me say it slowly. You have both arrived, from opposite directions, at the same single word as the location of the whole disagreement. Need. Want. The hunger that selects. Ada says it is the human core the machine structurally lacks, the thing that was never in the wake. Margaret says it is exactly the right place to make the stand, that she has said "computers have no such needs" herself — and that she cannot prove the need could never be built, because ours was. The entire three hours has funneled to this. Not intelligence. Not creativity. Not consciousness, yet. Want. We are going to take a break, and when we come back we are going to stop talking about the machine for a while and talk about us — because Ada's objection, Margaret has been hinting all night, might cut backward. We turn the loom around. After this.