Gottfried Wilhelm Leibniz vs John Searle on AI · Ch5. Let Us Calculate ← Ch4 Ch6 →
Txt Low Med High
HOUR ONE — FORM AGAINST MEANING
Chapter 5

Let Us Calculate

Page 1 · Let Us Calculate
Godel Escher Bach
Godel Escher Bach

EDO SEGAL: Professor Leibniz, this is the round I've waited for, so let me set it carefully. You had a dream more audacious than anyone before you dared: that reasoning itself could be mechanized. Two parts. The characteristica universalis — a universal language in which every concept gets a character, so complex ideas are built from simple ones the way numbers are built from primes. And the calculus ratiocinator — a logic for manipulating those characters, so valid inference becomes a kind of arithmetic. Put them together and you get the sentence you're famous for: when two philosophers disagree, they need argue no more than two accountants; let them take up their pens, sit at their slates, and say to each other — let us calculate. Calculemus. Now look at the box on my desk. Is it the fulfillment of that dream, or its refutation?

Interpretability Problem
Interpretability Problem

LEIBNIZ: Both, Edo, and that is the most painful and instructive thing I have to say tonight. It is the fulfillment of the ambition and the refutation of the method, and I must own the refutation because honesty is the discipline I imposed on the universe and cannot exempt myself from. My dream had a defining assumption: that reasoning is, at bottom, calculation over clear symbols — that if you could only decompose every concept into its primitives and write the rules exactly, thought would become arithmetic. For a long stretch that dream built a real and noble tradition. My "let us calculate" runs through George Boole, who turned it into an algebra of logic that every circuit in your device now performs; through the founders of modern logic; through the men in your 1950s and 1960s who built machines that represented knowledge as logical propositions and reasoned by formal rules. The expert systems of the 1980s were my calculus ratiocinator in working brass. I drew the blueprint, and an honest tradition spent decades trying to raise the building.

· · ·
Page 2 · Let Us Calculate
Mechanistic Interpretability
Mechanistic Interpretability

And it broke on the open world — that is the part I did not foresee and must confess. To reason about the actual world by my method, you must first write the world down in the characters: every concept, every rule, every tacit thing a child knows without being told. And the world will not fit. The concepts are vague at the edges; the rules have endless exceptions; the background knowledge is bottomless and mostly unstated. Try to write the rules for what counts as a chair, or what happens when you pour water, or why a joke is funny, and the project never ends. My characteristica assumed concepts could be cleanly cut into primitives. Most human concepts cannot. I was not wrong about logic. I was wrong about how much of human thought is logic.

Five Kinds Of Understanding
Five Kinds Of Understanding

SEARLE: And here's where it gets delicious, because the machine that finally worked escaped Leibniz's wall by abandoning his method while keeping his ambition. The large language model does not represent the world in a universal language of explicit characters. It does not reason by formal rules over crisp concepts that you could read. It manipulates high-dimensional vectors — weights with no individual meaning, numbers no one can narrate. It stores "dog" not as a character in a logical language but as a point in a space of thousands of dimensions, whose position relative to other points captures, statistically, how the word behaves. That is the opposite of what Leibniz wanted. He wanted discrete, meaningful, human-readable atoms of thought. The thing that achieved general competence uses continuous, meaningless, human-unreadable vectors instead.

· · ·
Page 3 · Let Us Calculate
Simulation Hypothesis
Simulation Hypothesis

LEIBNIZ: It is a partial refutation of me, and I will say so plainly, because a man who demanded sufficient reasons must accept the evidence when it arrives. I bet that thought is calculation over clear symbols. The machines that reason in something like the open-ended human way are built on the opposite bet — that competence emerges from statistical pattern over representations that are not symbols and cannot be read. If you had asked me whether genuine reasoning could be built from a heap of meaningless numbers tuned by statistics, I would almost certainly have said no. I would have been wrong about what is possible. I do not think I was wrong about what would be lost.

My symbolic dream, where it worked, gave you exactly that — a system that could show its work, prove its conclusion followed from its premises, equip every step with a sufficient reason.

EDO SEGAL: Tell me what's lost. Take it down to one scene.

LEIBNIZ: The scene is a court. A judge rules, and you ask why, and she gives you a reason — an account in terms of law and principle, stateable, inspectable, contestable. You may point to the reason and argue it was misapplied, and watch her change her mind in the light of the reason given. The reason is load-bearing; it does work. My symbolic dream, where it worked, gave you exactly that — a system that could show its work, prove its conclusion followed from its premises, equip every step with a sufficient reason. The vector machine cannot. It confabulates; it cannot reliably show its work; it gives you a cause and no reason, an output you cannot contest. What is lost is the thing a society is built from: the demand that consequential decisions come with grounds you can examine and challenge. That is not a technical inconvenience. It is the precondition of accountability — what you would call due process. I dreamed of ending disputes by calculation. The machine ends them by fiat, dressed as calculation, and that is the dream's nightmare, not its fulfillment.

· · ·
Page 4 · Let Us Calculate
Ai Consciousness Claims
Ai Consciousness Claims

SEARLE: I'll grant Leibniz the whole point and add my own grievance, because it's the same wound from my side. He wanted explicit symbols partly so reasoning would be transparent. I wanted to deny the machine understanding. And notice: the vector machine fails both of us in the same breath. It doesn't reason in stateable steps — fails Leibniz. And it has no aboutness behind the vectors — fails me. The field is now trying to bolt symbolic scaffolding back onto the networks, what they call neuro-symbolic systems, precisely to recover the thing Leibniz's method had and the vectors threw away. So the dream isn't dead. It's being smuggled back in through the service entrance to make the fluent machine justify itself. Leibniz gets a measure of revenge.

There's a ghost I have to seat here, because he is the man who broke the dream from the inside, with a proof.

EDO SEGAL: There's a ghost I have to seat here, because he is the man who broke the dream from the inside, with a proof. Kurt Gödel. Professor Leibniz, you wanted a complete formal system in which every truth could be reached by calculation. Gödel proved, in 1931, that any formal system rich enough to do arithmetic contains true statements it can never prove — that the dream of a complete mechanized reason is not merely hard but provably impossible. Does he bury your calculemus, or does the modern machine, which doesn't prove but predicts, somehow slip the noose?

· · ·
Page 5 · Let Us Calculate
Extended Consciousness
Extended Consciousness

LEIBNIZ: Gödel wounded the form of my dream and left its spirit strangely intact, and I have had three centuries beyond the grave to make my peace with him. He proved that no single formal system can capture all arithmetical truth — that my slate, however vast, has truths it cannot reach by its own rules. That is a real defeat for the characteristica as I conceived it, a complete and closed calculus of all reasoning. I concede it without reservation; a man who demanded sufficient reasons cannot quarrel with a proof. But notice what your machine did. It did not answer Gödel by building a bigger formal system — that road is closed forever. It abandoned the formal system altogether for the statistical one. The machine slips the noose Gödel tied because it never steps into the formal noose at all. It does not prove; it predicts. And so it escapes incompleteness the way a man escapes a locked room by refusing to enter it — which is also why it can never guarantee its conclusions the way a proof can. Gödel cost me the completeness. The machine recovered the reach by surrendering the certainty. I am not sure I would have made that trade. But I cannot say the world was wrong to.

· · ·
Page 6 · Let Us Calculate
Conscious Mind Book
Conscious Mind Book

SEARLE: And here's the part I find quietly devastating, and it's mine to add because it cuts at the worship, not the man. People invoke Gödel to argue that human minds transcend machines — that we can "see" the truth of the unprovable sentence the way no formal system can, so we must be more than computation. I never bought that; it smuggles in the very thing it claims to prove. But there's a humbler lesson in Gödel that does survive, and it's Leibniz's: the dream of settling every dispute by calculation was always going to leave a residue no calculation reaches. Leibniz thought that residue was small. Gödel proved it's structural. And I'd say the residue is exactly where meaning lives — in the truths you grasp but cannot derive, the judgment no rulebook contains. The machine predicts brilliantly inside the patterns. The residue is where it confabulates, and where, on my view, the understanding would have to be, if there were any.

I will take the revenge, though I would rather have the reasons.

LEIBNIZ: I will take the revenge, though I would rather have the reasons.

EDO SEGAL: Here's what unsettles me, and I want one of you to take it. You're both telling me the machine reasons without reasons — it reaches conclusions it cannot justify. But Professor Leibniz, you also said the dream was that disagreement could be ended by calculation, with a friend as witness. We are now ending disagreements by calculation — the model arbitrates, ranks, decides — but the friend, the witness, the contestable reason, is gone. We got the calculemus and lost the thing that made it just. Did your dream contain that danger from the start?

· · ·
Page 7 · Let Us Calculate
Orders Of Consciousness
Orders Of Consciousness

LEIBNIZ: It did, and you have found it, and I will not flinch from it. My dream was safe only because I imagined the calculation would be transparent — that anyone could check the slate. I assumed the symbols would be readable, the steps visible, the reason exposed. I never imagined a calculation no one could read. The danger was always there, sleeping inside the assumption that to calculate is to make the reasoning public. Strip out the publicity and you keep the power and lose the justice. You have built the calculus ratiocinator and welded shut the slate. I would not have called that the fulfillment of my dream. I would have called it the abandonment of the friend who was supposed to witness.

EDO SEGAL: Hold that — it returns when we get to who decides. But you've handed me the next round, both of you, because you keep saying the machine has the form and not the meaning, the wake and not the boat. There's a thought experiment that puts an animal at the bottom of that exact ocean, and a piece of know-how no rulebook can hold. After the break: the octopus, the hamburger, and the Background.

· · ·
Continue · Chapter 6
The Octopus, the Hamburger, and the Background
← Prev 0%
Ch5 Next →