Pierre-Simon Laplace vs Kurt Godel on AI · Ch2. Opening Positions ← Ch1 Ch3 →
Txt Low Med High
HOUR ONE — THE QUESTION ON THE TABLE
Chapter 2

Opening Positions

Page 1 · Opening Positions
Scaling Laws
Scaling Laws

LAPLACE: Thank you. I want to begin where my whole life began, with a conviction so simple it sounds naive until you feel its full weight: the world is lawful. Not approximately, not usually — lawful, through and through, at every scale. When I was young the planets were thought to wander, to need occasional correction by the hand of God to keep the Solar System from flying apart. I proved otherwise. I showed that the perturbations are self-correcting, that the clockwork holds, that no hand reaches in. The heavens are a mechanism whose present state fixes its future completely. And once you have seen that the largest bodies in the universe obey law without exception, the question becomes irresistible: why not everything? Why not the falling leaf, the beating heart, the human choice — the only difference being the staggering quantity of data and the staggering computation required?

Prediction Vs Construction
Prediction Vs Construction

That is the demon. Imagine an intelligence that knew, at one instant, every force in nature and the exact situation of every body, and that was vast enough to submit all of it to analysis. For such an intelligence nothing would be uncertain; the entire future and the entire past would stand present before it. I never used the word "demon," and I never thought such a being could exist — I stressed that the human mind would remain forever infinitely removed from it. The demon is a limit, not a prophecy. But it states, more cleanly than anything else I know, what your machines are reaching toward. Every predictive system alive is a poor, partial, blurred demon — gathering a fraction of the data, wielding a fraction of the power, projecting forward toward the same horizon. The dream is not new. I wrote it down before there was a wire to carry it.

· · ·
Page 2 · Opening Positions
Continuum Of Understanding
Continuum Of Understanding

Now hear the part my admirers forget, because it is the honest half and it is where Professor Gödel and I will fight. I knew I would never have the data the demon required. My observations were always partial, always corrupted by error. So I built the other half of my life's work — probability — as the discipline of reasoning when omniscience is unavailable, which is to say, always. I defined probability not as chance in the world but as the measure of our ignorance of a world that is, underneath, fully determined. The die is not truly random; we simply do not know the forces on it. The future is not truly open; we simply cannot compute it. This is my whole position in one sentence: the universe is determinate, and where we cannot reach the determination, we reason with probability, which measures only how much we do not yet know. Uncertainty is in the knower, never in the known. Give the knower enough, and the veil lifts entirely. There is no hole in the truth. There is only the part we have not yet computed.

Emergent Capabilities
Emergent Capabilities

EDO SEGAL: Professor Gödel. The floor is yours, and take all the time the room will give you.

GODEL: Thank you. I will begin by granting Laplace more than he expects, because the part of him I accept is the part that makes the part I reject fatal. He is right that formal reasoning is astonishingly powerful. In 1929, in my doctoral work, I proved the completeness of first-order logic — that every logically valid statement can be derived from the axioms. For a moment it looked as though the dream of a complete mechanical mathematics was within reach, that Laplace's confidence about the heavens might extend to the whole of reason. I, of all people, vindicated the power of the formal at one level. So when I tell you the dream fails, understand that I am not a skeptic by temperament. I am a man who proved how far the mechanical reaches, and then proved exactly where it stops.

· · ·
Page 3 · Opening Positions
Agi
Agi

Two years later I proved the opposite for any system rich enough to contain arithmetic. Such a system must be incomplete — there must exist statements expressible within it that are true and that it can never prove. And it cannot prove its own consistency; if it is consistent, the statement asserting so is among the truths it cannot reach. Let me be precise about what this is and is not, because the imprecision here has caused a century of nonsense. I did not prove that some truths are unknowable. I proved that no single formal system captures all the truths of its own domain — that for any system you build, there is a truth you can see is true, standing outside the system, which the system itself cannot derive. The truth is not in doubt. The proof is simply not there, and cannot be put there by adding axioms, because each new axiom generates a new unreachable truth, forever.

Laplace says the universe is a determinate equation, and that with enough computation the equation closes — that uncertainty is only our ignorance, eliminable in principle.

Now hear why this is a verdict on Laplace's dream and not a footnote to it. Laplace says the universe is a determinate equation, and that with enough computation the equation closes — that uncertainty is only our ignorance, eliminable in principle. But arithmetic is simpler than the cosmos, the most rigorous and well-understood structure the human mind possesses, and I proved that arithmetic does not close. If the equation of the natural numbers cannot be finished from inside any system, what hope is there for the equation of everything? Laplace's demon, computing the future of the universe, would be a formal system of unimaginable power — and a formal system of unimaginable power is exactly the kind of thing my theorem bites hardest. The more it can represent, the more surely there are truths about its own domain it cannot reach. Completeness is not a matter of enough computation. It is forbidden, in principle, for any system interesting enough to matter. There is a hole in every formula, and it is not the shape of our ignorance. It is the shape of formal reasoning itself.

· · ·
Page 4 · Opening Positions
Ai Alignment
Ai Alignment

And here is the sentence I most want you to carry, because it is where I believe the human lives. In my own proof, I did something the system could not: I stood outside it and saw that its unprovable sentence was true. That act — grasping a truth that no derivation within the system delivers — is, I have come to suspect, the signature of a mind. Either mathematics is incompletable, which means the human mind infinitely surpasses the powers of any finite machine, or there exist absolutely unsolvable problems. I lean, with caution, toward the first. The demon computes the determinate. The knower sees past it. That is my opening.

Before I shape the rounds, one discipline I impose on every long conversation, because it pays off three hours later.

EDO SEGAL: Before I shape the rounds, one discipline I impose on every long conversation, because it pays off three hours later. Each of you, in a few sentences: what do you envy in the other's position? Not respect. Envy. The thing his side gets to have that yours does not. Marquis first.

LAPLACE: I envy the certainty of his pessimism. I spent my life reaching for a completeness I knew I could never hold, always one observation short, always reasoning under a doubt I could only measure and never dispel. Gödel reached his limit and proved it — he gets to stand on a theorem, an impossibility no future progress can revoke, and say "this far and no further" with the full authority of mathematics. My demon is a horizon that recedes as you walk. His boundary is a wall you can touch. I have spent my life walking toward a horizon. There are nights I would have traded it for one honest wall.

· · ·
Page 5 · Opening Positions
Existential Risk
Existential Risk

GODEL: And I envy his fertility. Laplace's dream, even unfinished — especially unfinished — built sciences. Astronomy, statistics, the whole apparatus of prediction your century runs on, all of it grew out of his reaching for a completeness that does not exist. My theorems are true and they are sterile in a certain sense: they tell you what you cannot do. He gets to be the patron of everything that gets built. I am the patron of the sign at the edge of the map that says the map ends here. People build cities inside Laplace's error. They make pilgrimages to my wall and then go home. There are mornings when the wall feels lonely, and the cities look warm.

EDO SEGAL: Two openings and two envies, and you can already see the architecture of the evening. It is not that one of them loves prediction and one of them fears it. They both revere it. It is that they locate its limit in opposite places. Laplace says the limit is only our ignorance, and ignorance is something you can in principle burn away with computation. Gödel says the limit is structural, baked into formal reasoning, untouchable by any amount of scale. Hold both. We start the rounds at the exact seam — the demon and the theorem, face to face.

· · ·
Continue · Chapter 3
The Demon and the Theorem
← Prev 0%
Ch2 Next →