No Pretensions to Originate

EDO SEGAL: Here is the most consequential sentence in the prehistory of this entire field, and I'm going to read it in full because every word is doing work. "The Analytical Engine has no pretensions whatever to originate anything. It can do whatever we know how to order it to perform. It can follow analysis; but it has no power of anticipating any analytical relations or truths." Ada, you wrote that in Note G, in 1843. In 1950 Alan Turing quoted it in the founding paper of artificial intelligence, named it "Lady Lovelace's Objection," and tried to dismantle it. So let me ask you the thing I most want to know. Now that you have seen the machines — now that you've watched a system produce a proof its builders couldn't have anticipated — do you withdraw the sentence, defend it, or sharpen it?

LOVELACE: I sharpen it, because Turing, clever as he was, answered a sentence I did not write. Let me be precise about what I claimed, since everyone who quotes me flattens it. I did not say the engine was limited in scope — I said the opposite, that its reach was vast. I did not say it could not surprise us. I said it could not anticipate analytical relations or truths — that it cannot reach beyond the procedures given to it and discover something the procedures did not contain. The key phrase, the one nobody quotes, is "whatever we know how to order it to perform." The engine's reach is exactly coextensive with what we already know how to specify. It elaborates what is implicit in its instructions. It does not exceed them. That was true of my engine as a near-tautology, and I claim it remains true of yours, with the authorship hidden one layer deeper.

EDO SEGAL: But Gregory, Turing's reply was sharp, and I want you to give it its full force before you complicate it. Turing said: the objection is really the claim that a machine can "never take us by surprise," and he reports that machines surprise him constantly — that he cannot work out the consequences of his own instructions, so the outputs astonish him. By the surprise criterion, the objection is simply false, and the whole imitation-game framing he built around it treats behavior, not provenance, as the test. Machines surprise us all day. Is Turing right?

CHAITIN: Turing is right and Ada is right, and the reason both can be right is that they are talking about two different things, and I can actually prove the boundary between them, which is the one contribution I can make to a debate that has otherwise been stuck for seventy-five years. Watch. Turing's "surprise" is epistemic — it's a fact about what the programmer can foresee. And he's correct: a deterministic program surprises you the instant its consequences exceed your ability to trace them, which is immediately. But Ada's "originate" is not about surprise. It's about information. And here is where I can make it exact. I proved that a formal system containing a certain amount of information — call it L bits — cannot derive conclusions that contain substantially more than L bits of genuine information. A finite system cannot manufacture information it does not have. So take Ada's claim and translate it into my vocabulary: the engine cannot output more information than was put into it — weights, data, prompt, and whatever randomness it draws on. That claim is not a feeling. It's a theorem. And it's true. She was right.

LOVELACE: Thank you. I will take a theorem when one is offered.

CHAITIN: Don't take it yet, because the theorem says less than you want it to and the gap is everything. The conservation law says the machine can't conjure information from nothing. It does not say the machine can't produce something neither you nor its builders anticipated, because — and this is the cruelty of it — most of the information that comes out of these systems was latent in the data all along, in a form no human had ever extracted. The engine that computes a Bernoulli number nobody calculated has produced no new information; the number was implicit in the cards. But the machine that compresses ten million proofs and then produces a new proof has done something stranger. The new proof was, in the strict sense, latent in the compressed structure of all the old proofs — Ada is right, no new information was created. And yet no human had it, no card specified it, and extracting it required a search no human could perform. So the question "did it originate the proof?" splits in two. Did it create new information? No — theorem, Ada wins. Was it the source of a true thing no one possessed? Yes — and that's a different question, and the word "originate" has been smuggling both for a hundred and eighty years.

EDO SEGAL: Say that again, because I think it's the hinge of the whole evening. You're saying "originate" has been doing two jobs — and underneath them is the older fork in [YOU] on AI, whether the machine predicts the world or constructs it, and they come apart.

CHAITIN: They come apart completely. Job one: to be the source of new information — to add bits that were not in the system. By this definition the machine originates nothing, and neither, I'll add, does most of what we call human creativity, because a mathematician deriving a theorem from axioms adds no information either; the theorem was implicit. Job two: to be the place where a true and valuable arrangement first appears in the world, extracted from latent structure no one had unpacked. By this definition the machine originates constantly, and so do we, and they are the same activity. Ada's objection is unbeatable for job one and irrelevant for job two. Turing answered job two — surprise, novelty, the new thing appearing — and thought he'd refuted job one. He hadn't touched it. Two great minds, the same word, ships in the night.

LOVELACE: I accept the distinction, and I think it is the best thing said tonight, and I am going to use it against you. You say the second kind of origination — the first appearance of a latent truth in the world — is "the same activity" in the machine and the human. It is not, and the difference is the one I have been pointing at since 1843. When a human mathematician brings a latent truth to light, there is someone for whom the truth matters — someone who chose to look there, who recognized the truth as worth having, who meant it. When the machine brings the same truth to light, there is the extraction and nothing else. No one chose. No one recognized. No one meant. You have proved, beautifully, that the information is identical. You have not touched the question of whether the act is the same, because the act includes the meaning, and the meaning is exactly the thing your conservation law does not measure. Bits are conserved. Caring is not made of bits.

CHAITIN: Or caring is made of bits and you can't see the bits because you're inside them. That's the whole fight, and notice we've reached it in one round. You want "meaning" to be the thing the machine lacks. I want to know what meaning is made of before I agree it's missing. Because every previous time someone said "the machine lacks the special human ingredient X," X turned out to be a process, and the machine got it eventually, and the people who'd staked humanity on X had to retreat. Understanding was supposed to be the ingredient — I'm telling you it's compression, and the machine has it. Now you say meaning, caring, the someone-for-whom. Maybe. But I've watched this fence get repainted my whole life, and I'd bet against it being the last one.

EDO SEGAL: And there's the seam, in the open, an hour in. Ada says the conservation law proves the machine adds no information but misses that human origination carries meaning the machine cannot. Gregory says "meaning" is the next fence to fall, because every previous fence fell. Neither of you can prove it from here — which is why we keep climbing. Hold that word, meaning. We will need it on a much higher floor, when we ask whether anyone is home. But the next round goes to the engine of Gregory's whole position, the claim Ada has been circling and refusing: that understanding, the real thing, the thing we do, is nothing more and nothing less than compression.