Göttingen, Emptied

EDO SEGAL: This round opens with a confession, and it's mine, because the moderator pays the table's toll first. I built engagement machinery once — systems designed to capture and hold human attention — and I built them well, and I knew the loops, and I told myself the cleverness was neutral, that what people did with the attention was their business and not mine. It was not neutral. I have lain awake over it. So I come to this round, the hardest one, without the right to lecture either of you, because I know exactly what it is to believe that intelligence and goodness travel together, and to be wrong. Professor Hilbert — your Göttingen was the center of the mathematical world. And within months of 1933, it was destroyed from within. Tell the room what happened, and then tell me whether your faith had anything to say to it.

HILBERT: They came for the Jews first, as they always do. Emmy Noether — one of the greatest algebraists who ever lived, a mind I would rank with anyone's — forced out, fled to America. Courant, Landau, so many others, dismissed, exiled, scattered. The institute I had built over forty years was gutted in a season. And there is a story, which is at least true in substance whether or not the words are exact: I was seated at a banquet beside Rust, the Nazi minister of education, and he asked me how mathematics was faring in Göttingen now that it had been freed of Jewish influence. And I said: there is no mathematics in Göttingen anymore. You ask whether my faith had anything to say to it. No. It had nothing to say. Wir werden wissen — we will know — had no sentence for a regime that did not want to know, and that murdered or exiled the people who did. The unbounded progress of reason was no defense against organized barbarism. It was, if anything, a convenience to it. The most rational society in Europe built the most rationally organized atrocity. My optimism, which had walked through every wall the pessimists built, walked straight into that one and shattered.

EDO SEGAL: Gödel, you lived this too — you left Vienna, crossed a world at war to reach Princeton. And I want to ask you the structural question, not the historical one, because it cuts directly at the machine. Is there a theorem of progress? Does greater capability bend toward greater good?

GODEL: There is no such theorem. I want to be very exact, because this is the one place where my discipline and my life speak with the same voice. I proved that a system cannot certify its own consistency. The deeper, unprovable truth that Professor Hilbert's Göttingen teaches is adjacent to it: a civilization cannot certify its own decency from the inside, by the mere fact of its intelligence. The is of capability and the ought of its use are not connected by any rule of inference. I have heard it said, by the optimists of your machine age, that a sufficiently intelligent system will naturally become benevolent — that understanding and goodness converge at the top. There is no proof of this. There is, in the history Professor Hilbert just narrated, a counterexample written in blood. The same intelligence that builds the cathedral builds the camp, and nothing in the intelligence itself chooses between them. Anyone constructing a powerful machine on the assumption that capability yields benevolence should study what happened to Göttingen, because it is the cleanest demonstration that the two are independent variables, and that the gap between them can be filled by the worst of us as easily as the best.

EDO SEGAL: Let me make that concrete, because "no theorem of progress" can sound like a seminar phrase until you put a face on it. Emmy Noether. You said it yourself, Professor Hilbert — one of the greatest algebraists who ever lived, and the faculty senate fought her appointment because she was a woman, and you stood up and said you did not see why a candidate's sex should be an argument against her admission to the senate, since the senate was not a bathhouse. And then, years later, the same institution that had finally admitted her threw her out for being Jewish, and your defense of her brilliance was worth exactly nothing against the machinery of the state. So here is my question to both of you, and it is the question the alignment people are really asking. If the most rational institution in the world could not align itself to decency — if all that intelligence could be captured and turned — what makes anyone think we can align a machine more powerful than any of us?

GODEL: The honest answer is that the machine cannot be aligned the way the dream imagines, and the reason is the one I have given all evening turned toward governance. There is no internal certificate. A system cannot prove its own soundness; a civilization cannot prove its own decency; and a machine cannot, from inside its own values, guarantee that those values are the right ones or that they will hold under pressure. The alignment, if it exists at all, is not a property the system possesses and demonstrates. It is a relation maintained from outside, continuously, by something that is willing to stand apart and judge — and that judging is the very act my theorem says no system performs upon itself. What happened to Göttingen is the proof of concept, written before the machine existed: the alignment of an institution to the good is not a solved state but a constant, fragile, external labor, and the moment the external watchers were removed or overpowered, the brilliant institution executed atrocity with full efficiency. The machine inherits this exactly. There is no alignment you can install once and trust forever; there is only the unending work of standing outside it and refusing to look away.

HILBERT: And here, for once, I will not turn it into optimism, because I have not earned the right and the dead are listening. Gödel is correct. This is the place where I was most wrong, and it is not a place where my partial-versus-total distinction rescues me. I believed, implicitly, in my marrow, that the growth of reason was the growth of the good — that to know more was to be better. That faith was not merely unproven. It was refuted, in my own city, by my own civilization, against my own students. And the machine inherits this exactly. You are building reasoners of enormous power and telling yourselves the power will be kind. I told myself the same thing about the power of mathematics, and then I watched the power of mathematics get conscripted by people who wanted to know nothing except how to kill efficiently. The orange pill is not only that the dream succeeds where it fails. It is that the dream is morally blind, and that the blindness is not a flaw in the dreamer but a property of the dream. Reason does not point. You must point it. The machine derives. You must mean. I said that an hour ago about hallucination. It turns out to be the same sentence, and it is the only thing I learned from Göttingen that I did not already know from Gödel.

EDO SEGAL: Mark that — and it's the gravest convergence of the night, so I'm going to state it slowly for the reader. The optimist and the wall-builder agree, completely, that there is no theorem of progress; that capability is morally blind; that the meaning and the pointing must come from a human being outside the system, because the system cannot supply them. Hilbert arrived at it through catastrophe. Gödel arrived at it through proof. They meet in the same place: the machine cannot mean. Only you can. Hold that as we walk down out of the mathematics and into the street, because the street is where this stops being philosophy. The machine that derives without meaning has begun rearranging who gets paid, who gets to learn, and what a human being is for. The death cross. After this.