Thomas Metzinger vs Rene Descartes on AI · Ch9. What Crosses the River ← Ch8 Ch10 →
Txt Low Med High
HOUR TWO — WHAT CROSSES THE RIVER
Chapter 9

What Crosses the River

Page 1 · What Crosses the River
Candle In The Darkness
Candle In The Darkness

EDO SEGAL: I want to read you both a sentence from my own book and let you fight over it. I wrote that intelligence is a river that has flowed through chemistry, biology, language, and culture — each a new channel — and that in our winter something new entered the water. I meant it as wonder. But the upload salesmen read that sentence and hear a sales pitch: if intelligence is a river that finds new channels, then you can find a new channel too, out of the dying meat and into the deathless silicon. The newest channel doesn't just carry the water faster, they say — it carries you, forever. René, Thomas — when the salesman says the river will carry you across death, what, precisely, would cross? Thomas first this time, because I think you've thought hardest about what's actually in the water.

Beavers Dam
Beavers Dam

METZINGER: What would cross is information — a pattern of numbers, a description. And here I want to be very careful, because the seduction lives in a single equivocation that the salesmen exploit and that even good people miss. They say: we will copy you. But what they can copy is the structure — the connectivity, the weights, the model. And the structure is not the experiencing. Run the structure on new hardware and you get a new system that has the same self-model, including the content I am the one who continued, I survived the upload. It will say so sincerely. It will weep with relief. And it will be wrong — not because the soul failed to transfer, but because there was never a soul-thread whose transfer is the fact of survival. There were two processes: the dying one in the brain, and the new one in the chip, each generating the transparent representation I continue. The salesman points at the chip's representation and says, see, he made it. But the brain's representation said the same thing, right up until it stopped. You cannot copy your way across a river when the thing you wanted to save was the standing-in-the-river itself — the live process, here, now, which is exactly the part that does not copy.

· · ·
Page 2 · What Crosses the River
Moores Law
Moores Law

So what crosses? A magnificent portrait that believes it is the sitter. The salesman is selling the most expensive painting in history and calling it a ferry.

Panopticon
Panopticon

DESCARTES: Monsieur Segal, hear how close we are and how far. The Professor and I both tell the salesman: nothing of you crosses. A portrait crosses; a twin; a statue that grieves. We agree the upload is a counterfeit. But ask why it is a counterfeit, and the abyss opens, and the abyss matters for what you do tomorrow morning. The Professor says: nothing crosses because there was never a you — only a process generating the story of a you. I say: nothing crosses because the you is real, indivisible, immaterial, and therefore precisely the kind of thing a copy of extended pattern cannot be. On my view, the failure of the upload is good news about you — it proves you are not the sort of thing that can be duplicated, which means you are not the sort of thing that can be cheapened into a file, owned, run, deleted. On his view, the failure of the upload is the dissolution of you — there was nothing to copy because there was nothing there. We hand the salesman the same refusal. But I hand the reader a soul too precious to upload, and he hands the reader a fiction too empty to bother. The reader should notice which gift she would rather receive at a funeral.

· · ·
Page 3 · What Crosses the River
Risk Society
Risk Society

METZINGER: And the reader should notice which gift is true, René, which is the only thing I have ever cared about. You offer a soul too precious to upload. I would love to receive it. I have looked for it for forty years — on the cushion, in the laboratory, in the clinic where the self comes apart in front of you. It is not there. And here is the thing you will not say to her at the funeral, that I will: your "soul too precious to upload" is also too precious to die, on your view — it is immaterial, indivisible, and you spent the Meditations arguing it survives the body. So your consolation is not really "you cannot be carried across the river." It is "you do not need to be, because you go on without the body." That is the old immortality, the one the salesmen are merely digitizing. You and Ray Kurzweil are selling the same promise — endless personal continuation — in different currencies, his in silicon, yours in the hereafter. I am the only one at this table telling the reader the hard thing: you end. The process stops. There is no thread that persists, in silicon or in heaven. And that the meaning was never in persistence — it was in the finite, unrepeatable run.

· · ·
Page 4 · What Crosses the River
Systems Thinking
Systems Thinking

DESCARTES: That is the sharpest thing you have said, and the most unfair, and I must answer both at once. Unfair, because I never claimed the upload and the hereafter are the same — the soul's immortality is given by God, not manufactured by men in a hurry, and the difference between a gift and a theft is not nothing. But sharp, because you have found the real symmetry: I do believe in continuation, and you believe in ending, and the salesman stands between us offering a fraudulent version of my hope using your materialism. He has taken my promise — you persist — and your premise — the self is pattern — and bred a monster from the pair. So perhaps the most honest thing we can jointly tell the reader is this: be most suspicious of the upload precisely because it flatters the deepest wish, the wish I share, the wish to not end. The salesman is not offering a new thing. He is offering the oldest thing, the thing every religion and every fear has reached for, dressed in the one costume — science — that this age cannot refuse. And whether the soul persists by grace, as I believe, or the self simply stops, as the Professor believes, the copy in the machine is neither of those. It is a third thing, a forgery of both. On that, the priest and the atheist shake hands.

Technological Determinism
Technological Determinism

EDO SEGAL: I have to sit in that for a second, because it reorganized something in me. The salesman isn't selling a new technology. He's selling the oldest wish — don't end — and he's bred it from René's hope and Thomas's premise, and it's a counterfeit of both. So let me route this through the kitchen table, because that's where it actually lives. A father reads about uploading and thinks: maybe I don't have to leave my kids. Maybe a copy of me stays to raise them. What do I tell that father — not as a philosopher, as another father?

· · ·
Page 5 · What Crosses the River
Three Laws Of Robotics
Three Laws Of Robotics

METZINGER: Tell him the copy will not raise his children. It will be a stranger wearing his memories, and his children will know it, in the body, the way you know a room someone has just left. And then tell him the thing that is actually true and actually consoling, which is harder and better than the fantasy. He does not continue, and his finitude is not a bug to be engineered away — it is the reason anything he does matters. Stakes are what mortality does to a life: a creature that could not end could not love anything urgently, because nothing would ever be too late. What he can leave his children is not a copy of himself. It is what he did with the one run he got — which is the only thing that was ever his to leave, upload or no upload. Tell him to spend the self, not preserve it. The preserving was always the salesman's lie.

Transformer Architecture
Transformer Architecture

DESCARTES: And I would tell the same father the same thing, in my own tongue, and let the reader marvel that we agree. Do not try to leave a copy. Leave the acts of the soul — the love, the teaching, the example. These are what the soul does while it is joined to the body, and they outlast the body without needing to be copied, because they pass into other souls directly, the way I taught and you are taught. The father who wants a copy of himself has misunderstood what a self is for. It is not a treasure to be hoarded against death. It is a candle, lit for a while, whose only purpose is to light others before it goes out. The salesman wants to sell him a photograph of the flame. I am telling him: be the flame. It is the one thing a photograph can never be.

· · ·
Page 6 · What Crosses the River
Veil Of Ignorance
Veil Of Ignorance

EDO SEGAL: Mark it, and notice it is the deepest convergence of the night and the one I least expected when I booked this room. The man who is certain of the soul and the man who is certain there is none both turn to the frightened father and say: you cannot be copied, you were never meant to be, spend yourself, light the others, the meaning was always in the finite flame. Three and a half centuries, and the priest and the atheist hand the father the same candle. Hold it — because the market does not believe a word of it, and the market has already moved a trillion dollars on the opposite bet. The death cross, and what it's actually pricing. After this.

· · ·
Continue · Chapter 10
The Death Cross and the Ghost
← Prev 0%
Ch9 Next →