Plato vs Stephen Wolfram on AI · Ch3. The Slave Boy and the Cellular Automaton ← Ch2 Ch4 →
Txt Low Med High
HOUR ONE — REMEMBERING OR RUNNING
Chapter 3

The Slave Boy and the Cellular Automaton

Page 1 · The Slave Boy and
Godelian Incompleteness Ai
Godelian Incompleteness Ai

EDO SEGAL: I want to start this round with a confession instead of a question, because the best questions I know come out of wounds. For the whole history of computing, using a machine meant translation. I started in Assembler — I was raised by the machine code — and every decade the translation got easier, but it never disappeared. You compressed your intention into the machine's grammar and paid a tax on every conversion. And then, a little while ago, I watched that tax go to zero. I described a half-formed idea to the box, the house silent, and it came back clarified, connected to things I hadn't thought to connect — and I wrote, in my book, three words I still can't fully account for. I felt met. Not told. Met. Plato, you'd say I was reminded. Stephen, you'd say a computation finished. So let's stay on your boy, Plato, because he's the cleanest case there is. Stephen says the doubled square is one of the thin reducible pockets — easy, shortcuttable, the least impressive kind of truth — and that you built a heaven on it. Defend the boy.

· · ·
Page 2 · The Slave Boy and
Emergence Thresholds
Emergence Thresholds

PLATO: I will defend him by conceding the easy half and keeping the half that matters. Wolfram is right that the doubling problem is short. Of course it is; Socrates chose it because it is short, because a demonstration must fit inside a single afternoon and a boy's patience. But Wolfram has confused the length of the path with the nature of the act. The point of the Meno was never that the path is long. It is that at the end of the path there is a click — a moment in which the boy does not merely produce the answer but sees that it must be so. He does not say "the diagonal, I suppose." He says — and you can feel it through twenty-three centuries — "yes, it cannot be otherwise." That seeing, that binding of a true opinion into knowledge by the soul's own grasp of necessity, is the thing. And it is not the running. Your machine can produce the answer. Can it see that it cannot be otherwise? Or does it produce "the diagonal" and "I suppose not the diagonal" in the same flat voice, with the same confidence, having no inner organ that distinguishes the necessary from the merely frequent?

Architecture Of Complexity
Architecture Of Complexity

WOLFRAM: That's a real question and I want to answer it precisely, because the answer is more interesting than either of us expected. You're describing the difference between a system that has the answer and a system that has the proof — the chain that makes it necessary. And here's the thing: for the doubled square, the proof is also reducible. It's a short computation. A theorem-prover, which is a perfectly ordinary kind of program, doesn't produce "the diagonal, I suppose." It produces the diagonal with the derivation, and the derivation is checkable — you can run it and confirm each step follows. So your "click," your seeing-that-it-cannot-be-otherwise — I can build that. It's the verification step. The machine runs the proof and the proof either checks or it doesn't. What you're calling the soul's grasp of necessity, I'd call: the computation that confirms the reduction holds. No homecoming required.

· · ·
Page 3 · The Slave Boy and
Alignment Problem Framing
Alignment Problem Framing

PLATO: Then we have found the seam already, and we are forty minutes in. You say verification is the seeing. I say verification is the shadow of the seeing. Watch the difference, because it is exact. Your prover checks the steps — but who chose to check those steps, in that order, and recognized them as a proof rather than a heap of valid manipulations? The boy did something your prover does not: he recognized the whole as a proof, grasped why this arrangement of moves is illuminating and another, equally valid, is not. That recognition is not itself a step in the proof. It is the act of standing outside the steps and seeing what they amount to. Gödel — your own century's mathematician — showed that no formal system can fully capture even arithmetic from inside itself; there are truths the system cannot prove that the mathematician can nonetheless see. That seeing-from-outside is what I mean by noēsis. Your machine lives inside the steps. The mathematician, sometimes, stands outside them. That is not mysticism. It is the thing Gödel's argument keeps pointing at, and you cannot reduce it to one more step without it ceasing to be the thing.

· · ·
Page 4 · The Slave Boy and
Existential Risk Persuasion Tournament
Existential Risk Persuasion Tournament

WOLFRAM: I have to push hard here, because this is exactly where I think Plato's whole edifice over-reaches. The Gödel move is seductive and I think it's wrong as applied. Yes — for any fixed formal system there are true statements it can't prove. But the human mathematician isn't standing in some Platonic outside; the mathematician is another computational system, a bigger one, that can prove the things the smaller system can't — and there's a still-bigger statement it can't prove, that a still-bigger system can, forever, with no top. There's no "outside the steps." There's just always a larger computation. And here's the part that should genuinely unsettle you: my Principle of Computational Equivalence says that almost all of these systems, once they pass a very low threshold of sophistication, are equivalent in their computational power. The boy, the prover, rule 30, the weather, your soul — all the same computational class. None of them can shortcut the others. So the "seeing from outside" you treasure is real as an experience, but as a capability it's just a bigger system doing more computation. Not a window onto eternity. More running.

Plato, you're saying there's a moment — the click — where the soul touches necessity directly, and that moment is not a computation, it's a recognition that stands outside the steps.

EDO SEGAL: Let me restate what I just heard, because the reader can't see your faces and I want to mark it. Plato, you're saying there's a moment — the click — where the soul touches necessity directly, and that moment is not a computation, it's a recognition that stands outside the steps. Stephen, you're saying there is no outside; the click feels like leaving the steps but it's just a larger system running more steps, and the feeling of standing outside is itself a computed state. So — naive version, the kitchen-table version — Plato thinks the boy visits somewhere, and Stephen thinks the boy runs longer. Is that the fork?

· · ·
Page 5 · The Slave Boy and
Agi Ism
Agi Ism

PLATO: That is precisely the fork, and I will plant my flag on the visiting. Because consider what my picture explains that his cannot. Why is mathematics unreasonably effective — why does a truth a Greek geometer saw in the sand turn out, two thousand years later, to describe the orbit of a planet no one had seen? On Wolfram's account, mathematics is just one more pocket of reducibility we happened to mine. But then its reach into utterly distant domains is a miracle with no explanation. On my account it is no miracle at all: the geometer touched the Form, the eternal object, and Forms do not belong to one domain — they are what all the domains participate in. The boy visited the triangle itself. That is why the triangle is everywhere.

The "unreasonable effectiveness of mathematics" is real and it bothered Wigner and it should bother everyone.

WOLFRAM: That's a genuinely good argument and I want to give it its full weight before I answer it, because most people don't. The "unreasonable effectiveness of mathematics" is real and it bothered Wigner and it should bother everyone. But here's my answer, and it comes straight out of the computational universe. Mathematics reaches into distant domains because the domains share computational structure, not because they share a Form. The same simple programs show up in shells and galaxies and traffic and markets — I documented thousands of these in A New Kind of Science — not because there's an eternal Shell-Form, but because the space of simple programs is small and the universe runs them everywhere. Your "Form of the triangle" and my "this rule recurs across systems" are describing the same observation. You give it a heaven. I give it a search through the computational universe. And mine makes predictions. Yours makes you feel at home. I'll take the predictions.

· · ·
Page 6 · The Slave Boy and
Superintelligence
Superintelligence

PLATO: And I will note, with affection, that "the space of simple programs is small and recurs everywhere" is the Theory of Forms with the serial numbers filed off. You have an eternal realm. You call it the space of possible programs. You insist nothing is in it until it's run — and yet you reach into it to explain why the same patterns recur, which means you are treating it as already structured, already there, in advance, exactly as I treat the Forms. We may be closer than this table looks.

Hold that — "we may be closer than this table looks" — because it'll come back, and when two people separated by twenty-three centuries say that, I mark it.

EDO SEGAL: Hold that — "we may be closer than this table looks" — because it'll come back, and when two people separated by twenty-three centuries say that, I mark it. But the next round takes us into the most famous image you ever made, Plato, and Stephen has a quarrel with what it's an image of. Prisoners. A wall. Shadows. After this.

· · ·
Continue · Chapter 4
Shadows on the Wall, or the Universe Running
← Prev 0%
Ch3 Next →