Aristotle vs Hans Moravec on AI · Ch10. The Two of You ← Ch9 Ch11 →
Txt Low Med High
HOUR THREE — THE TWO OF YOU
Chapter 10

The Two of You

Page 1 · The Two of You
Collective Intelligence Augmentation
Collective Intelligence Augmentation

EDO SEGAL: Before I leave the room entirely, there's one knot we flagged early and have to untie, because it's the sharpest blade on the table and everything downstream depends on it. Hans, you called the dying original a "branch." Aristotle says: run your scanner without destroying the original, and there are visibly two people, both certain they're you, so the copy was never you in the first place — and killing the original quickly doesn't change the metaphysics, it just removes the embarrassing witness. This is the round where you have real work to do. Do it.

Capital Labor Split Ai
Capital Labor Split Ai

MORAVEC: I do, and I've thought about this longer than anything else in my work, so let me build it properly. Aristotle's objection assumes there's a fact of the matter about which one is "really" you — that identity is a deep, all-or-nothing thing, so that if there are two, at most one can be the true heir and the other is an impostor. I deny the premise. I think personal identity, looked at closely, is not that kind of fact. When I branch into two, the right thing to say is not "one is me and one is a fake." It's that there were one of me and now there are two of me, both equally continuous with the person who walked in, and the question "but which one is really me?" is a confused question, like asking which fork of a river is really the river. They both are. Identity isn't a thing that has to be conserved like a single coin. It's a relationship, and relationships can branch.

EDO SEGAL: So restate the radical version for the reader, because I want it to land. You're saying: if we copied me right now, there'd be no fact about which of the two of us is the real Edo — there'd just be two Edos, both real, both me, and the universe simply doesn't contain an answer to "which one is the original soul."

· · ·
Page 2 · The Two of You
Ai Industrial Reorganization Labor
Ai Industrial Reorganization Labor

MORAVEC: Exactly. And the reason that sounds horrifying is that we evolved in a world where branching was impossible, so our intuitions about identity were never stress-tested against it. Our gut says "there can only be one me" because there only ever has been one me. That's a fact about our history, not about the metaphysics of persons. Once branching is on the table, the one-coin model of identity just breaks, and the right response isn't panic — it's to upgrade the model.

Engels Pause
Engels Pause

ARISTOTLE: This is his best moment and his most revealing one, so I will praise it and then close it. He is right that the gut's "there can only be one me" is not yet an argument. And he is right that if you abandon the idea of a single continuing self, the duplication problem dissolves — because there is no longer anything that had to be the survivor. But look at the price, reader, because he has paid it without flinching and you should see the bill. To save the upload, he has had to give up the very thing the upload was supposed to save. He set out to let you survive death. And to make the survival coherent against duplication, he has had to conclude that there is no you — no single self, just a branching relationship, a river with no fact about which fork is the river. He has rescued the boat by proving there was never a captain. What, then, is being saved? Not you. There is no you in his system; he abolished it to win the argument. He offers the dying man immortality and, in the fine print, informs him there was no man.

· · ·
Page 3 · The Two of You
Automation Tax
Automation Tax

MORAVEC: That's rhetorically gorgeous and I think it's wrong, because you've equivocated on "you." I didn't abolish the self. I de-throned the single-instance self — the idea that a self is like a unique physical object that can exist in exactly one place. The self I keep is the pattern, which is perfectly real and perfectly continuous; it just turns out to be the kind of real thing that can exist in two places, like a song. You're treating "there can be two of you" as equivalent to "there is no you," and those aren't the same at all. A song that plays in two rooms hasn't ceased to exist. It's just not the kind of thing that's confined to one room. I didn't kill the self. I corrected a false belief about its plumbing.

*ARISTOTLE: Then answer the dying man's actual question, which is not abstract.

ARISTOTLE: Then answer the dying man's actual question, which is not abstract. He is on the table. The scan is about to run and then his body will be destroyed. He asks you, gripping your sleeve: will I — this one, the one afraid right now, the one asking — will I open my eyes in the machine, or will I simply stop, while something else opens its eyes and believes it is me? Tell him. Not the metaphysics of songs. Tell this terrified particular man what is about to happen to him.

MORAVEC: I tell him the truth, which is harder than either a yes or a no. I say: the one who is afraid right now will not experience a tunnel and then wake in the machine — there is no thread of experience that crosses the gap, and I won't lie to you that there is. And the one who wakes in the machine will remember gripping my sleeve, will remember being afraid, will be continuous with you in every way that you are continuous with yourself yesterday. Both of those are true. Whether that counts as "you surviving" is not a further fact I'm hiding from you — it's the choice you have to make about what you mean by "you." I can't make it for you. I can only refuse to pretend the choice isn't there.

· · ·
Page 4 · The Two of You
Ironies Of Automation
Ironies Of Automation

ARISTOTLE: And I honor that answer, because it is finally honest, and it concedes my case. He has just told the dying man: the one who is afraid will stop. That is the whole of what I have claimed all night. Everything after — the successor's memories, its continuity, its conviction — is the wax taking the impression while the seal is destroyed. He has dressed the destruction in the dignity of a "choice about what you mean by you," and that dignity is real. But the man on the table stops. Moravec said it himself. I rest.

There's one more round before you cross — a gentler one, about what is gained even if Aristotle is right that nothing is saved.

EDO SEGAL: I'm not going to break that. I'm going to let it stand exactly as sharp as it is, because this is the spine of the whole evening and you both just touched it bare-handed: Hans says the question "which one is really you" has no answer, and the demand for one is a relic; Aristotle says that having no answer is the answer, and it's the one the dying man feared. Hold it. There's one more round before you cross — a gentler one, about what is gained even if Aristotle is right that nothing is saved. Because the machine is here, and it is changing what a human being can do, and not all of that is loss.

· · ·
Continue · Chapter 11
What the Living Get to Build
← Prev 0%
Ch10 Next →