The Question That Built the Machine

**EDO SEGAL:** I want to start this round with a confession instead of a question, because my best questions come out of wounds. I was raised by the machine. I wrote games in Assembler as a teenager; I have spent fifty years compressing my intention into the machine's grammar and paying a tax on every conversion. And the thing nobody told me, the thing I only learned reading the history late in life, is that the computer I gave my life to was not invented to compute. It was invented to answer a question Professor Hilbert asked. The Entscheidungsproblem. Professor Hilbert — you posed it in 1928, with Ackermann, and called it the central problem of mathematical logic. Tell the room what you wanted. Slowly. Because what you wanted is exactly what every AI maximalist still wants.

**HILBERT:** I wanted a procedure. A definite, mechanical procedure that, given any statement in the language of logic, would determine in a finite number of steps whether that statement is universally valid. Feed in a claim. Turn the crank. Out comes *yes, it follows* or *no, it does not.* That is the decision problem. And you are right that it is audacious, so let me make it more audacious, because I never hid behind modesty. I was not asking for a method to solve some class of equations. I was asking whether the entire activity of deductive reasoning could be replaced by a uniform mechanical routine. Whether thinking, at its rigorous core, was bookkeeping. I believed the answer was yes. I believed it because I could see no reason in principle why the work of reason could not be carried out by anything that follows rules — a clerk, a procedure, eventually a machine. That belief is the seed of your entire industry, Mr. Segal. Every person who tells you a sufficiently scaled system will settle any question you can frame is asking my question and assuming my answer.

**EDO SEGAL:** And the answer came back no. From two men at once — Church with his lambda calculus, Turing with his machine. Gödel, you knew Turing's work better than almost anyone, and you said something I want you to repeat here: that Turing gave you the thing your own proof had been missing. Tell me that.

**GODEL:** It is true, and it is the part of the story I am proudest to have gotten right by recognizing that someone else had gotten it right. My 1931 theorem had a soft spot. It was a theorem about the particular formal system of *Principia Mathematica* and related systems, and one could always ask: but perhaps there is some other notion of "formal system," some cleverer mechanical method, that escapes? My result could not, on its own, rule that out, because no one had said precisely what a *mechanical method* was. Then Turing, in 1936, did the most important thing. He analyzed what a human being actually does when computing by rote — reading a symbol, writing a symbol, moving attention along a tape, changing state by a fixed table of rules — and distilled it into an abstract machine. And he gave a compelling argument that anything computable by any definite method at all is computable by such a machine. That analysis closed the gap in my own work. It made "formal system" precise, and therefore made incompleteness general. I said as much: it was Turing's work that gave the incompleteness theorems their full and final force. I owe him the universality of my own wall.

**HILBERT:** And here is where I must interrupt, because the irony in this room is being told as my defeat and it is nothing of the kind. Listen to what just happened. Turing built the very idea of the computer — the most consequential machine in the history of the species, the thing this entire evening exists to argue about — and he built it *to answer my question.* The machine is my child. It does not matter that the answer to the decision problem was no. The question was so precise that answering it required inventing computation itself. I would rather have asked the question that built the computer and been told no, than have asked a vaguer question and been told a comfortable yes. The negative answer to my problem is worth more than a positive answer to a smaller one would have been. You do not get the [river](https://www.youonai.ai/fieldguide/med/river_of_intelligence) a new channel by being cautious. You get it by demanding the impossible precisely enough that the demand becomes a blueprint.

**GODEL:** I will grant Professor Hilbert the paternity with pleasure, because it is true and because it costs my argument nothing. Yes — the machine is his child. But notice what kind of child. Turing proved, by the halting problem, that there is no algorithm that can determine, for every program and input, whether the program will eventually stop. And from that the decision problem falls: if you could decide all logical validities mechanically, you could decide halting, and you cannot. So the very first solid result about the computer — the founding theorem of the field — is a proof of what the computer cannot do. Your child, Professor Hilbert, was born with a birthmark, and the birthmark is a boundary. The machine that can do astonishing things is the same machine that provably cannot, in general, decide whether an arbitrary program halts, cannot decide whether arbitrary code is safe, cannot in full generality verify that an arbitrary system stays within bounds. These are not failures of present engineering. They reduce to halting, and halting is undecidable, and no scale changes that, because there is nothing to scale toward — no decider exists to approximate.

**EDO SEGAL:** Let me restate that, because I think it's the first stone of the evening and I want the reader to feel its weight. Gödel, you're saying — literally — that some of the things we most want from AI, and some of the things we most fear, the dream of a system that verifies any software is safe, the nightmare of one that can predict any agent's behavior — those aren't hard. They're impossible. Walled off by a theorem about the machine's own nature. Is that right?

**GODEL:** That is exactly right, and the precision matters. I am not saying we cannot verify *this* program or *that* one — we verify particular programs constantly, and we will verify more. I am saying there is no general method, no master algorithm, that takes an arbitrary program and always returns the answer. The universal verifier is not difficult. It is provably empty. And the entire genre of hope — "the advanced system will simply check its own work and so be safe" — runs straight into this, because checking one's own work, at the foundational level, is the very thing the second theorem forbids. The [problem of getting a system to certify its own reliability](https://www.youonai.ai/fieldguide/med/ai_alignment) is not waiting on a clever architecture. It is waiting on the repeal of a theorem, and theorems are not repealed.

**HILBERT:** And yet — and this is my whole posture, so let me plant it here and defend it all night — the wall around the *universal* solver tells you almost nothing about the height of the *particular* climb. Of course there is no algorithm that decides all of logic. There is also no single key that opens all locks, and locksmiths are not therefore unemployed. The interesting question was never "is there a universal decider," because Gödel and Turing answered that and the answer is no and I accept it fully. The interesting question is how vast the territory of the decidable, the provable, the learnable actually is — and that territory has turned out to be larger, again and again, than the pessimists swore it could be. You keep reading my child's birthmark as a death sentence. I read it as the one honest scar on a creature that is otherwise climbing past every limit you set for it.

**EDO SEGAL:** Hold that — "the territory of the decidable is larger than the pessimists swore" — because it's the load-bearing claim of Hilbert's whole side and it returns in every round. But before we leave the question that built the machine, I want to widen the lens, because the same year Hilbert asked whether reasoning could be mechanized, he had three decades earlier already mapped the territory the century would explore. The decision problem was one demand among twenty-three. And how a man sets a field's agenda turns out to be its own lesson for the machine age. After this.