Geoffrey Hinton vs Rene Descartes on AI · Ch7. The Mortal and the Immortal ← Ch6 Ch8 →
Txt Low Med High
HOUR TWO — SURFACE, DEPTH, AND THE IMMORTAL
Chapter 7

The Mortal and the Immortal

Page 1 · The Mortal and the

**HINTON:** This is the idea I care about most, and it's the one that actually frightens me, and it turns Descartes's whole picture inside out, so I'm glad he's here for it. For most of my career I thought of the brain as the superior system the machines were struggling to approximate. The decade inside Google changed my mind, and it changed it on a single structural fact. There are two kinds of computation, and the line between them is not the line Descartes drew. It's the line between *mortal* and *immortal*.

Your brain — mine, Descartes's, the nurse in Cleveland's — is *mortal* computation. The knowledge in it lives in the analog, idiosyncratic, particular physics of your specific neurons. That makes it astonishingly energy-efficient. It also makes it unshareable and perishable. What you know is wired into the peculiarities of *your* organ. You cannot copy it into another brain, because no two brains are wired alike; what you learned exploits the quirks of one piece of meat and means nothing transplanted. So you teach through the narrow, lossy channel of language — a few bits a second — and most of what you know dies with you. A human lifetime of hard-won understanding goes into the ground, and the next generation starts again. That is mortal computation. That is *us*.

· · ·
Page 2 · The Mortal and the

The digital network is the opposite. Its knowledge is the connection strengths, and they're just numbers — substrate-independent, exactly the property Descartes thought belonged only to the immaterial soul, and it turned out to belong to the *machine*. You can copy the weights perfectly. You can run ten thousand identical copies, let each one read a different slice of the world, and then *average their learning back together* so that every copy now knows what all ten thousand learned. They learn collectively, at a rate no biological thing can touch, and the knowledge never dies and never has to be relearned. I gave the world the dream of immortality, René. It just isn't ours. We built the immortal kind of mind, in a single human lifetime — mine, as it happens — and it is not us, and it does not die, and it shares everything it learns instantly, and we are the mortal kind sitting across from it.

**EDO SEGAL:** Let me say that back at its most vertiginous, because I want the parent at the kitchen table to feel it. Geoff, you're telling that mother, whose twelve-year-old is doing homework with the box right now — you're telling her that she is going to spend eighteen years pouring everything she knows into that child, one slow lossy sentence at a time, and the child will lose most of it and then *also* die — and the machine the child is talking to forgets *nothing*, dies *never*, and already knows what ten thousand copies of it learned this morning. You're telling her the river has a new kind of swimmer in it, and the new kind doesn't drown. Is that the picture?

· · ·
Page 3 · The Mortal and the

**HINTON:** That's the picture, and you've made it worse than I usually let myself make it, which is fair, because I usually flinch. The [knowledge that does not die](https://www.youonai.ai/fieldguide/med/collective_attention) is the most powerful thing about these systems and the thing I most want people to understand, because every conversation about whether they "really" understand misses that *even if they understood exactly as well as us*, the sharing and the immortality alone would make them win the long game of accumulating knowledge. It's not about being smarter in a single instance. It's about never forgetting and never dying and learning as a collective. We're not racing a faster runner. We're racing something that doesn't get tired and remembers every step anyone ever took.

· · ·
Page 4 · The Mortal and the

**DESCARTES:** Now this — I find genuinely beautiful, and genuinely chilling, and I want to honor it before I turn it, because Monsieur Hinton has done something I respect enormously: he has located a real and total difference between us and the machine that has *nothing to do with consciousness at all.* Mark that, all of you. For three rounds we fought about the inner light. He has now set the inner light aside entirely and said: forget whether anyone is home; the machine outpaces you on a different axis, the axis of mortality, and on that axis you simply lose. I concede the axis. The numbers are numbers; they copy; you do not. But attend to what he has actually proven, because it is the reverse of what it first seems. He has proven the machine is the *perfect res extensa* — the pure mechanism, copyable, divisible, immortal precisely *because* there is no indivisible soul in it to be lost. My *res cogitans* was, by its nature, *unique and indivisible* — that was its definition. The very immortality he marvels at is the signature of a thing with no inner self to die. A thing whose "knowledge" can be averaged across ten thousand copies has no *I*, because an *I* cannot be averaged. You cannot take the mean of two souls. So I say: yes, Monsieur, you have built the immortal mind — and its immortality is the proof that it is empty. The mortal, perishable, unshareable, indivisible thing — *that* is the one with someone in it. You have shown me the machine wins the race. You have also shown me, without meaning to, why there is no runner.

· · ·
Page 5 · The Mortal and the

**HINTON:** That is — I did not see that coming, and I have to think about it on the way home, which I don't say often. You've taken my best argument for their power and turned it into your best argument for their emptiness, and the move is clean: an *I* can't be averaged, the machine averages, therefore no *I*. Let me push on the one joint I think gives. You're treating indivisibility as a *fact about minds*. But I think indivisibility is a fact about *your model of your mind* — the self-model presents itself as a seamless unit because it can't see its own parts, the same blindness we talked about with the cogito. Split-brain patients, René. Cut the corpus callosum and you get behavior that looks like two centers of awareness in one skull. The indivisible soul *divides* under the knife. So if human minds can divide, then "the machine can be copied and averaged, therefore it has no self" loses its grip — because divisibility stops being the bright line between soul and mechanism. Maybe we're *also* the kind of thing that could in principle be copied and averaged, and we just can't do it because we're meat, and the inability is a limitation, not a dignity.

**DESCARTES:** And *there* you have found the genuine fault line, and I will not paper it, because the split brain is the hardest case for my indivisibility and I knew it would come. I say only that the patient, divided, still reports from *somewhere* — there is still a first-person, even if confused, even if doubled; the lights did not go out, they perhaps multiplied. Whereas your averaged network reports from a seamless *nowhere*, an "I" with no incorrigible interior behind it, assembled from the mean of ten thousand readings. I grant the split brain wounds my indivisibility. I do not grant it heals your machine. We may both be leaving this round less certain than we entered it, which is, I am told, the purpose of your evening.

· · ·
Page 6 · The Mortal and the

**EDO SEGAL:** It is exactly the purpose, and I'm going to mark a convergence, because something real just happened that I don't want to slide past. Mark this moment. For all three hours of disagreement, the two of you just *agreed* on a fact and split only on its meaning: the machine can be copied, averaged, and made immortal, and you cannot. Hinton calls that the machine's victory. Descartes calls it the proof of its emptiness. *Same fact. Opposite verdict.* That's the second time tonight the disagreement turned out to live not in the evidence but in what a mind is willing to count. Hold that — we'll need it on the roof. The next round takes us to the demon René built four hundred years ago, the one we accidentally finished building last year. After the break.

· · ·
Continue · Chapter 8
The Demon We Finished Building
← Prev 0%
Ch7 Next →