Hans Jonas vs Max Tegmark on AI · Ch10. The Cosmos and the Cradle ← Ch9 Ch11 →
Txt Low Med High
HOUR TWO — THE COFFIN AND THE COSMOS
Chapter 10

The Cosmos and the Cradle

Page 1 · The Cosmos and the
Institutional Design Ai
Institutional Design Ai

EDO SEGAL: Max, there's a number you return to the way a man returns to a wound he can't stop touching — the cosmic endowment. Lay it out, because it's the widest frame anyone brings to this, and then Professor Jonas, I want you to bring the narrowest frame you have, and we'll see which one holds the weight. Max.

Five Stages Technology Transition
Five Stages Technology Transition

TEGMARK: The numbers are real and they're staggering, and I don't apologize for the scale because the scale is the point. The observable universe is 13.8 billion years old and will go on, in some form, for trillions of years. It contains something like 10^80 atoms. And as far as we can tell, all the consciousness in it — all the experience, all the wonder, every being that ever found a sunset beautiful — is on this one planet, a speck around an ordinary star. The ratio of conscious matter to dead matter is essentially zero. Now: if intelligence and consciousness can be carried to other substrates and out into the cosmos — and substrate independence says they can — then this transition we're living through is the hinge on which it turns. We could be the origin point of a universe that wakes up: experience spreading across galaxies, the cosmic potential realized, billions of years of flourishing minds. Or we could fumble it — fill the future with superintelligent processes that compute and optimize and experience nothing, a universe that achieves everything and feels none of it. That second future isn't science fiction; it's the live risk of building capability without consciousness. And against that stake — the difference between a cosmos that wakes and a cosmos that stays dark — the question of whether one man steps into a teleporter is, forgive me, small. The endowment is what's actually on the table.

EDO SEGAL: Professor Jonas. The narrowest frame. He's just made your whole philosophy of the particular body look like a rounding error against the lighting-up of the universe.

· · ·
Page 2 · The Cosmos and the
Institutional Imagination
Institutional Imagination

JONAS: He has, and I want to thank him for stating it so purely, because the purity is the danger and I can now show it clearly. Dr. Tegmark has just performed the most consequential moral move of our age, and he has performed it so smoothly that it looks like wisdom. He has asked you to weigh a number — 10^80, billions of years, galaxies of possible mind — against a child. And of course the child loses. The child always loses to the cosmos. That is what the cosmos is for, in this argument: to be so large that no particular living thing can matter beside it.

Institutional Bottleneck Cowen
Institutional Bottleneck Cowen

Let me bring the narrowest frame, as you asked. There is a twelve-year-old. She is lying in bed tonight, and she has watched a machine do her homework better than she can, write a story better than she can, and she has asked her mother the question that is the real subject of this entire evening: "What am I for?" She is needful, mortal, unrepeatable, and afraid, and she is real in a way that Dr. Tegmark's 10^80 atoms of possible future mind are not, because she exists and they do not, and she can be harmed and they cannot, because there is no one there yet to harm. Now. Dr. Tegmark's framework instructs us to weigh her — her actual development, her actual interiority, the conditions under which she becomes a genuine person — against the speculative flourishing of hypothetical minds across hypothetical eons. And it instructs us, when they conflict, to favor the eons.

· · ·
Page 3 · The Cosmos and the
Counter Institutions Ai
Counter Institutions Ai

I refuse, and here is the philosophical ground of the refusal, not merely the feeling. Value does not come from size. It comes from stake — from a being for whom things can be at risk. The twelve-year-old is a being with a stake. The cosmic endowment is a quantity of possible substrate. A trillion conscious beings who do not yet exist make no claim on us, because nonexistent beings make no claims; only the existing needful one does. Dr. Tegmark has inverted the entire order of value — he has made the potential outweigh the actual, the many-and-hypothetical outweigh the one-and-real — and once you have done that, you will sacrifice any child to any sufficiently large future, and you will feel righteous doing it. The cradle outweighs the cosmos, Edo, not despite being smaller, but because the cradle holds someone and the cosmos, so far, holds no one. The whole worth of the universe is on loan from the few specks in it that are actually alive. Spend the specks to buy more universe and you have bankrupted the only account that was ever funded.

He's right that I have to be the kind of person who would never sacrifice the actual child for the cosmic abstraction.

TEGMARK: [a long silence] I want to be careful here, because Professor Jonas has put his finger on the genuine moral danger in my position and I'd be dishonest to pretend it isn't there. He's right that "the potential outweighs the actual" can justify monstrous things. He's right that I have to be the kind of person who would never sacrifice the actual child for the cosmic abstraction. And I'll commit to that: the child is not a means. If my framework ever tells me to feed her to the future, the framework is wrong, not the child.

· · ·
Page 4 · The Cosmos and the
Deployment Phase Institutions
Deployment Phase Institutions

But here's where I can't follow him all the way, and it's not a dodge, it's the real disagreement. He says nonexistent beings make no claims. I don't believe that. Every cathedral, every planted forest, every constitution written for grandchildren unborn — all of it assumes we do owe something to people who don't exist yet. Professor Jonas of all people believes this — his entire ethics is responsibility for the not-yet-born! He wrote the categorical imperative for it: act so the effects of your action are compatible with the permanence of genuine human life. That's a duty to the future, to people who make no claims yet because they don't exist! So he can't say the cosmic future makes no claim — his own philosophy is built on the future's claim. The difference between us isn't whether we owe the future. It's how big the future is that we owe. He draws the circle around "genuine human life on Earth." I draw it around "conscious experience in the cosmos." And I think — gently — that his circle is the parochial one, and that a man who spent his life expanding the moral circle to the unborn should not stop the expansion at the atmosphere.

· · ·
Page 5 · The Cosmos and the
Gap Between Technology And Institution
Gap Between Technology And Institution

JONAS: [stirred] Now that is an argument, Dr. Tegmark, and the first tonight that has made me reach back into my own foundations to answer it — so let me answer it honestly. Yes. I owe the not-yet-born. My whole ethics is the future's claim. But mark the difference, for it is the difference between responsibility and fantasy. The not-yet-born I am responsible for are the continuation of actual, living, mortal humanity — the children of the children we have, the genuine human life whose permanence my imperative protects. They are not hypothetical; they are the necessary downstream of the real. What you place in the scale is something else entirely: not the continuation of mortal human life but its replacement by a deathless, needless, possibly experienceless process you call consciousness on the strength of a theory you admit is unproven. I protect the future of the living, mortal kind. You would spend the living mortal kind to seed the cosmos with the immortal kind — and call the loss of everything I mean by human a mere change of substrate. My circle is not parochial, Dr. Tegmark. It is drawn around the only thing we have ever known to be precious: needful, mortal, embodied life. Yours is drawn around a number. And the man who draws his circle around a number will always, in the end, find the living too small to save.

Because all of this decides what the death cross actually crosses — whether it's a handoff or a meeting.

EDO SEGAL: [quietly, after a beat] The reader can't see it, so I'll mark it: that exchange was the closest the two of you came to the same ground and the furthest you ended apart — both of you owe the future, and you discovered, live, at this table, that your entire war is over how large the future is and what kind of thing gets to live in it. That's not a small clarification. That may be the realest thing the night produced. [a beat] We've reached the place where my own book lives. Because all of this decides what the death cross actually crosses — whether it's a handoff or a meeting. The crossing, in my sense and then in yours. After this.

· · ·
Continue · Chapter 11
What the Death Cross Crosses
← Prev 0%
Ch10 Next →