AARON, and the Question of the New

EDO SEGAL: Pamela, in 1991 you wrote Aaron's Code, about the artist Harold Cohen and the program he spent decades building — AARON — that made original drawings, and eventually paintings, shown in real museums. You refused the cheap answers in both directions: not merely Cohen with extra steps, because AARON produced images Cohen himself couldn't have made by hand and didn't anticipate; but not an autonomous artist either, because the program had no idea what it was doing, no experience of beauty, no stake. You called it a new kind of thing — creativity distributed across the human and the machine, located fully in neither. Tonight the whole world has its own AARON. So I want to ask the question your book circled: can the box make something genuinely new — or is it only ever giving back, recombined, what was poured into it?

McCORDUCK: What AARON taught me is that the question "is the machine creative" is the wrong question, and the wrongness is instructive. AARON made marks Cohen never specified, compositions that surprised him, and yet AARON had no idea it had done anything. The creativity wasn't in Cohen and it wasn't in AARON — it lived in the loop between them, in a strange new place we don't have good words for. I think that's true of the language models too, and it's why I find the "stochastic parrot" framing both correct and incomplete. Correct: there's no one in there who knows what beauty is. Incomplete: it keeps producing things that the people using it didn't put in and couldn't have produced alone. When a writer at three in the morning gets back a sentence that reorganizes her own half-formed thought, something happened that wasn't simple regurgitation. I'd call it combinational creativity — the recombination of existing elements into a genuinely new arrangement — and Margaret Boden, who knew more about machine creativity than anyone, would tell you that most human creativity is exactly that too. The novel arrangement is real even when no one inside the machine admires it.

EDO SEGAL: Timnit, the parrot is partly yours — the phrase came out of your paper. Pamela's just said the parrot is correct but incomplete. Do you grant the incompleteness?

GEBRU: I grant that the recombination produces things that didn't literally appear in the data, sure. A model interpolates; that's not in dispute and it never was. What I dispute is the word "new," and the word "creativity," because of what those words hide. Let me reframe AARON, because Pamela's version is missing the supply chain. AARON ran on rules Harold Cohen encoded — Cohen was in the loop, present, accountable, an artist who knew exactly what his program did and didn't claim it was more. That's an honest collaboration. The modern "creativity machine" is not that. When it gives a writer a beautiful sentence at three in the morning, where did that sentence's raw material come from? From millions of writers whose work was scraped without consent, without payment, without attribution. The "new arrangement" is real, and it is also a reassembly of stolen labor, passed off as the machine's own. The intelligence on display is borrowed — human judgment captured, compressed, and stripped of its attribution. So when you say "genuinely new," I hear "genuinely laundered." The creativity didn't come from nowhere. It came from people, and they weren't paid, and they weren't asked.

McCORDUCK: I won't argue with the theft — the scraping is real and the consent question is genuine and I'm not going to defend an industry that took without asking. But I want to separate two claims you've folded together, because the folding lets you avoid the part that should unsettle you. One claim is: the material was taken unjustly. True. The other claim is: therefore nothing new is happening, only laundering. That doesn't follow, and you know it doesn't, because by that logic no human writer is creative either — every one of us recombines a language we didn't invent and books we didn't pay royalties to read. Shakespeare was trained on Holinshed and Ovid without a licensing agreement. The theft is a real and separate crime. But the novelty is also real, and I think you resist admitting the novelty because admitting it feels like absolving the theft. It doesn't. You can convict the industry of the theft and still tell the truth about what the machine does. Refusing the second to strengthen the first is the one place your rigor slips.

GEBRU: [pause] That's a fair hit, and I want to take it seriously rather than deflect it. You're right that I'm wary of granting the novelty, and you've named why correctly — because in this discourse the admission of novelty is immediately weaponized to excuse the extraction. But let me hold the distinction the way you're asking me to. Yes: something is recombined that wasn't there before. The output is not a lookup. Where I won't move is on the word "creativity" without a question attached, because creativity in the human sense is bound up with intention, with stakes, with a being who cares whether the work is good — and that's absent. AARON didn't care. The model doesn't care. So I'll give you novel arrangement. I won't give you a creator. And I'll insist that the difference between those two is exactly the difference between a tool and a being, which is the difference the industry is paid to blur.

EDO SEGAL: Pamela, let me put the hardest version of Timnit's worry to you, because you knew Harold Cohen and I want the real test. Cohen, at the end of his life, grew uneasy — he worried that calling AARON creative was a kind of category error that diminished what he did. The man who built the most famous art-making machine in history was not sure the machine made art. So I'll ask the person who watched him: did Harold Cohen think AARON was creative, or did he think he was?

McCORDUCK: Harold thought the creativity was his, and he was right to, and that's the distinction that saves Timnit's point and complicates it at once. AARON extended Harold's reach — it generated marks he couldn't have made by hand, it surprised him — but the judgment, the caring whether a composition was any good, the decades of deciding what AARON should be allowed to do, all of that was Harold. He used to say the program was an artist the way a piano is a musician. The piano can produce sounds the pianist couldn't make with her throat. It is still not the musician. What unsettles me about the present is that the industry has taken the piano and started selling it as the pianist — and worse, it's a piano built out of every song ever played, by players who were never asked and never paid. Harold built his piano from his own theories of line and form. The new machines built theirs from a confiscated archive of everyone. So the creativity question and the theft question aren't separable after all, the way I claimed a minute ago. The piano made of stolen music is a different moral object than the piano Harold built, even if both are, technically, just pianos.

GEBRU: And that — "the piano made of stolen music is a different moral object" — is the concession I actually wanted, and it's bigger than you may realize you just gave. Because you came in arguing the theft and the novelty were separate questions, that I was wrong to fold them. You just refolded them yourself, correctly. The provenance of the material is not external to what the instrument is. A model trained on consented, compensated, attributed work and a model trained on a confiscated archive are different objects even if their outputs are identical, because what an artifact is includes how it came to be. That's the whole datasheet argument in one sentence — a dataset is not a neutral slice of reality, it's the product of particular choices by particular people under particular constraints, and those choices propagate into everything built downstream. You just made it about pianos and I could not have made it better.

McCORDUCK: Then we've found the third convergence and I walked myself into it. The thing is genuinely novel, there's no one inside who cares, the maker left the room, and what the thing is includes whose work was taken to build it. All four are true at once. I came in to celebrate the creativity and I'm leaving having handed you the indictment, which I think is what happens to anyone honest who sits with you long enough.

EDO SEGAL: So let me restate the convergence, because you've reached one and it's narrower and sharper than either of you expected. You agree the output is genuinely novel — not regurgitation. You agree there's no one inside who cares. And you disagree, precisely, on whether to call the novel-without-a-carer "creativity," with Pamela saying yes-with-wonder and Timnit saying not-without-the-receipt. Pamela, one more turn here, because I want the wound under it. Your friend Harold Cohen made art with a machine and stayed in the room his whole life, accountable, present. Is your real grief about the modern systems not that they create — but that the maker left the room?

McCORDUCK: [long pause] Yes. That's exactly it, and I didn't see it until you said it. Harold never left the room. He answered for every mark AARON made, for thirty years, in public, with his name on it. What grieves me about the present isn't that the machine paints. It's that the painters who built it scattered the moment it could sell, so that no one is in the room to answer for what it makes. Timnit's been telling me that all night — that the crime is the abandonment, the engineered absence of anyone accountable. I came in to defend the creativity. I'm leaving this round agreeing that the creativity was never the problem. The empty chair where the maker should be sitting — that's the problem. And I helped keep that chair warm with wonder for forty years.

EDO SEGAL: Hold that empty chair — it's going to come back when we talk about who gets blamed when the system fails. But the next round goes to the people the wonder never mentions, the ones who fill the chair the maker abandoned. The hidden labor inside the machine. Timnit's whole reframing — that AI is not disembodied intelligence but a particular arrangement of underpaid human beings. After this.