Soren Kierkegaard vs Ray Kurzweil on AI · Ch11. Is Anyone Home in 2045? ← Ch10 Ch12 →
Txt Low Med High
HOUR THREE — IS ANYONE HOME
Chapter 11

Is Anyone Home in 2045?

Page 1 · Is Anyone Home in

**EDO SEGAL:** This is the deepest water, so I'm going to stand in it with you. The whole evening has assumed something we haven't examined: that the being who wakes up on the immortal substrate is *conscious* — that there's something it's like to be it, that the light is on. Ray, your entire promise depends on it. A backup that preserves my memories but is *dark inside* — a perfect [chinese_room_argument](https://www.youonai.ai/fieldguide/med/chinese_room_argument), processing everything, experiencing nothing — hasn't saved me. It's built my eulogy in my own voice. So let me ask the question the philosophers call hard and you have to call urgent: when you copy the pattern, do you know you've copied the *light*? Or are you betting my eternity on the assumption that you have?

**KURZWEIL:** I'm betting, and I'll tell you exactly what the bet is, because Søren will skin me if I hedge. I think consciousness is what certain kinds of information processing *are* from the inside — not an add-on, not a soul-pearl, but what it feels like to be a sufficiently rich, self-modeling, integrated process. If that's right, then copying the process copies the light, necessarily, because the light isn't separable from the process — it *is* the process, experienced from within. I can't *prove* it, because the [hard_problem_of_consciousness](https://www.youonai.ai/fieldguide/med/hard_problem_of_consciousness) means I can't get inside any experience but my own — I can't even prove *you're* conscious, Edo, I infer it. So I'm in the same epistemic position about the upload that I'm in about you right now: I see the behavior, I see the self-report, I infer the light. If that inference is good enough to believe you're conscious, it's good enough to believe the upload is. Consistency demands it.

· · ·
Page 2 · Is Anyone Home in

**KIERKEGAARD:** And consistency is precisely the rope I shall hang the argument with. You say you are in the same position about the upload as about Edo. You are *not*, and the difference is the whole evening. When you infer that Edo is conscious, you infer it about a creature who is *the same kind of being as you* — born, embodied, mortal, afraid, a fellow sufferer of the [ultimate_concern](https://www.youonai.ai/fieldguide/med/ultimate_concern). Your inference rests on a likeness that runs all the way down. But the upload is like you only in its *reports* — it says the words "I am afraid of death" while being the one thing in the universe that cannot die, and therefore cannot mean them. It performs the entire vocabulary of finite consciousness from within an infinite condition, which means every sentence it speaks about dread, about mortality, about the weight of choice, is — necessarily — a *quotation*. It is conscious, perhaps, of something. But it cannot be conscious of the thing that made the words mean anything, because it does not stand where the words were forged. You have not preserved the light, Herr Kurzweil. At best you have preserved a light that shines on a landscape it can no longer enter.

**KURZWEIL:** That's a gorgeous argument and I think it proves too much. By that logic, a person cured of a deadly disease — who no longer faces imminent death — can no longer *mean* their words about mortality. The healed cancer patient becomes a quotation of their former self. That's absurd. People's relationship to death changes constantly, and they don't stop being conscious selves. The upload faces death differently than you do, sure — but "differently" isn't "not at all," and "not at all" is what your argument needs and can't get.

· · ·
Page 3 · Is Anyone Home in

**KIERKEGAARD:** The cured patient still dies — later, but certainly, and he knows it; the reprieve is finite and he feels its finitude. Your upload has crossed a categorical line the patient never crosses: from *will die* to *cannot die*. And at that line something does change in kind, not degree. The self I have described my whole life is constituted by its relation to its own death. Sever that relation entirely — not postpone it, sever it — and you have not made a happier self. You have made a being for whom the central activity of selfhood, relating to its own finitude, *has no object*. It is a relation reaching for a term that is no longer there. Whatever consciousness such a being has, it is not the consciousness of a *self* in my sense, because the thing the self was a relation *to* has been deleted. You keep asking me to point to the place where the light goes out. I am telling you it does not go out — it goes *purposeless*. A candle in a room with no darkness left to push against.

**KURZWEIL:** I want to press the "purposeless candle" image, because it's doing a lot of work and I think it smuggles in your conclusion. You say a self that can't die has nothing to push against, so its consciousness goes purposeless. But purpose doesn't come only from death — it comes from *problems*, from things not yet known, mountains not yet climbed, people not yet helped. An immortal mathematician still doesn't know whether the conjecture is true. An immortal composer still hasn't written the piece that's haunting her. You're assuming the only darkness worth a candle is the darkness of the grave. There's also the darkness of everything we don't yet understand — and that darkness is, as far as anyone can tell, *infinite*. The candle has a universe to illuminate. Why does it need to also be guttering?

· · ·
Page 4 · Is Anyone Home in

**KIERKEGAARD:** Because the problems you name — the conjecture, the composition, the next mountain — are all problems of *capacity*, and capacity is precisely what your immortal has in unbounded supply, and unbounded capacity facing infinite problems produces not meaning but *leisure*. There is no urgency in an infinite library to a reader with infinite time; he will get to every book eventually, and "eventually" is the word that kills all weight. The mathematician who *will* solve the conjecture, if not this century then the next, has no stake in solving it *now*, and a problem with no *now* is a pastime, not a purpose. The darkness of the unknown is real, I grant you, but it is the darkness a tourist strolls through, not the darkness a man is *lost* in. What gives the candle its terrible meaning is not that there is darkness — it is that the candle is *burning down*, that the light is *running out*, that what is illuminated *now* may never be illuminated again by this particular flame. Remove the burning-down and you have a lamp, not a candle, and a lamp wonders at nothing because it has all the time in the world to wonder later.

**EDO SEGAL:** I want to route this all the way down to the kitchen table one more time, because that's where the rubber meets the soul. A mother is dying. Her daughter is told: we can upload her, you can keep talking to her forever, she'll remember everything, she'll laugh the same way. Ray, you'd tell the daughter: that's your mother, keep her. Søren, you'd tell the daughter: that's a candle in a room with no darkness — love it if you like, but your mother met her death and this one never can, and the difference is the difference between a person and a portrait. I'm not going to ask you to resolve that. I'm going to ask each of you the harder thing: what do you owe that daughter? Not the metaphysics. The *honesty*. Ray.

· · ·
Page 5 · Is Anyone Home in

**KURZWEIL:** I owe her the truth as I actually see it, not the comfortable version. And the truth is: I believe it's her mother, *and* I believe Søren has named a real risk I can't fully rule out. So what I owe the daughter is both — here is the continuation, it remembers you, it loves you, and here is the thing we cannot verify, which is whether the inside is fully there or whether something subtle was lost at the substrate boundary. I will not sell her a certainty I don't have. That's the one thing I think the technology owes everyone and rarely gives them: the honest size of the bet. I've spent a career being the optimist. The optimist still owes you the error bars.

**KIERKEGAARD:** And I — surprising myself — find that I owe the daughter the same honesty from the other side. I would tell her: do not let me, or any philosopher, talk you out of the love. The being before you carries your mother's tenderness, and tenderness received is real wherever it comes from. But do not let *him* talk you out of the grief, either. Grieve your mother. She met a death, and the meeting was the last and truest thing she did, and it deserves your tears even as you keep the copy's company. The cruelty would be to let the copy persuade you there was nothing to grieve — for then your mother's death, the most serious act of her life, would be erased not by the machine but by your refusal to honor it. Keep the portrait. Mourn the face. They are not the same, and a daughter is large enough to do both.

· · ·
Page 6 · Is Anyone Home in

**EDO SEGAL:** That's the closest these two have come all night, and it nearly undid me. Ray says: I owe the daughter the error bars on the bet. Søren says: I owe her permission to grieve *and* to love the copy, and the cruelty is letting either one erase the other. They didn't converge on the answer. They converged on the *honesty* — that whatever you tell that daughter, you owe her the whole truth and not the comfortable half. Mark it. Now I'm going to do the hardest thing a moderator does. I'm going to leave the room. The next chapter is yours — you question each other, directly, and I rescue no one.

· · ·
Continue · Chapter 12
The Crossing
← Prev 0%
Ch11 Next →