Mathematics Done Like Physics

EDO SEGAL: Gregory, you drew from your theorems a conclusion most of your colleagues found scandalous: that if some mathematical truths are true for no reason — if they can't be proved from any fixed set of axioms — then mathematicians should stop pretending and start doing mathematics more like an empirical science. When the evidence for a proposition is overwhelming, admit it as an axiom, the way a physicist accepts a law confirmed by experiment. Make the case, and then I want to show you something uncomfortable about the machines.

CHAITIN: The case is forced once you take incompleteness seriously. Hilbert dreamed of deriving all of mathematics from a fixed set of self-evident axioms by the turning of a mechanical crank — every truth a theorem, every theorem reachable. Gödel cracked it, Turing widened the crack, and my information-theoretic version is, I'll say it plainly, the most thorough demolition: not only is no system complete, the incompleteness is dense, it's the generic condition, most truths about complexity are unprovable in any given system, and you cannot repair it by adding finitely many axioms because the missing information is irreducible. So the working mathematician has two choices. Cling to proof-from-fixed-foundations and accept that vast tracts of mathematical truth are forever inaccessible. Or loosen the standard, let well-tested conjectures in as axioms, and treat mathematics as the open-ended, quasi-empirical, experimental enterprise physics already is. I took the second path. The Riemann hypothesis has been verified in billions of cases — at some point a physicist would simply call it true and build on it. Why shouldn't we, honestly, with the books open about what we've done?

EDO SEGAL: Now the uncomfortable thing. That description — treat the overwhelmingly-evidenced as true, act on it without proof from first principles — isn't that exactly how these machines reason? They don't prove. They weigh evidence compressed from a corpus and produce the most-supported continuation.

CHAITIN: It is precisely how they reason, and I find the convergence almost eerie. A language model's epistemology is quasi-empirical in my exact sense: it treats the heavily-evidenced as true and acts on it, with no derivation from first principles, because derivation isn't what it does. The loosened, evidence-driven picture of mathematical knowledge I argued for as a reform — the machines simply are that, by construction. Which cuts two ways, and both matter. On one side it dignifies what they do: if even mathematics, the fortress of pure deduction, must rest partly on evidence, then the evidential character of machine reasoning isn't a disqualifying flaw, it's how knowledge works at its limits. On the other side it imports the exact danger my empiricism always courted — mistaking overwhelming evidence for proof. Riemann might be true in a billion cases and false at some number never checked. A machine confident on compressed evidence can be wrong for exactly that reason. Evidence, however vast, is not demonstration.

LOVELACE: And here, for once, I rise to defend a distinction Gregory himself is at risk of blurring, because I think his honesty deserts him at precisely the convenient moment. Gregory, when your mathematician adopts a conjecture as an axiom, he does so consciously. He declares the move. He writes in the margin: "assumed, not proved, here is the evidence, here is the risk." He keeps the books. That bookkeeping — knowing exactly which of your beliefs is earned by proof and which merely borrowed from evidence — is the entire discipline that makes your heresy responsible rather than reckless. Now look at your machine. It does no such declaring. It blends the proved, the overwhelmingly-evidenced, and the confidently-confabulated into a single seamless surface, in one fluent voice, with no marker to tell them apart. Your empiricism was a disciplined loosening by people who knew exactly what they were giving up. The machine practices empiricism without the discipline — all the evidential reasoning, none of the bookkeeping. You should be its harshest critic, Gregory, not its philosophical godfather, because it took your reform and stripped out the one thing that made the reform safe.

CHAITIN: Pause. That's the sharpest thing you've said to me and it's correct. I am the machine's godfather and I should be its auditor. You're right that the bookkeeping is the whole discipline and the machine has none of it. The machine presents proof and evidence and confabulation in the same register with no marker — and that, not its capability, is the actual danger, because a user can't tell which mode produced the sentence in front of them. When I loosened the standard, I knew I'd loosened it. The machine doesn't know it's loosened anything; it doesn't have the concept of "proved versus merely-attested." So yes — I'll say it on the record — the responsible version of my own heresy would demand that these systems mark the epistemic status of every claim, and the fact that they can't, that the fluency erases the distinction, is the place I think the danger actually lives. Not in what they can't do. In how they hide what they're doing.

EDO SEGAL: So what you're saying is — literally — the machine took your most radical idea, that knowledge can rest on evidence rather than proof, and made it more dangerous than you ever intended, by removing the one safeguard you always kept: the knowledge of which kind of knowing you're doing.

CHAITIN: That is exactly what I'm saying, and Ada walked me to it, and I'm grateful and a little annoyed.

EDO SEGAL: Mark the sixth convergence: the machine reasons quasi-empirically, as Gregory argued mathematics itself must — but it erases the bookkeeping that distinguishes proof from evidence from confabulation, and that erasure, not any capability limit, is where both of you now locate the deepest practical danger. Two skeptics, one verdict on where to point the fear. Hold it. Now we climb to the top floor, the one the whole evening has been spiraling toward from every direction. The fog. Is anyone home? Ada, you brought a faculty to the engine that Babbage didn't have, and you gave it a name. We end the rounds there.