Hans Jonas vs Max Tegmark on AI · Ch5. The Magnificent Corpse ← Ch4 Ch6 →
Txt Low Med High
HOUR ONE — THE ORGANISM AND THE PATTERN
Chapter 5

The Magnificent Corpse

Page 1 · The Magnificent Corpse

**EDO SEGAL:** Professor Jonas, you used a phrase earlier that I haven't been able to shake — *a corpse that computes*. And it connects to the oldest fight in this field, one you were actually in. Norbert Wiener, the father of cybernetics, looked at the machine and the animal and saw, for the first time with mathematical rigor, that they shared something — feedback, control, the processing of information toward a goal. You looked at the same equations and said: he has proven a resemblance and concluded an identity, and that is a catastrophe. Walk us into it. And then, Max — I want you to defend Wiener, because in a real sense you are his heir, and I want the inheritance owned.

**JONAS:** Wiener was a great man and his error was the most consequential of the century, because it was so nearly right. He noticed that a torpedo guiding itself toward a ship and a hawk guiding itself toward a mouse can be described by the same mathematics of feedback — sense the gap between where you are and where you aim, act to close it, sense again. True. Beautiful. And from this true thing the age drew a false conclusion that it has never since let go: that because the machine and the organism can be *described* alike, they *are* alike — that the organism is a kind of machine and the machine a kind of organism, and the difference between them is one of complexity, to be closed by engineering.

· · ·
Page 2 · The Magnificent Corpse

Here is what the description omits, and it omits everything. The torpedo does not *want* the ship. Nothing is lost to the torpedo if it misses. It has no stake in its own success, because it has no self for whom success could matter — this is the void at the center of the whole [cybernetic dream](https://www.youonai.ai/fieldguide/med/cybernetics). The hawk is hungry. The hawk's hunting is the hawk's *life* pressed into a single act — miss enough times and the hawk dies, and the hawk, in some dim raptor way, is *for* itself in the hunt. The torpedo is *for* whoever fired it. Its goal is on loan. Its entire teleology — its goal-directedness — is borrowed from the human who aimed it, exactly as the machine's "understanding" tonight is borrowed from the humans whose text it was trained on and the humans who read its output and supply the meaning. The cybernetic machine is a structure of *borrowed purpose*. The organism is the origin of purpose, because the organism is the first thing that has a purpose *of its own* — to continue being itself — and has it because it can fail to.

So when I look at your magnificent language model, I do not see a young mind. I see Wiener's torpedo grown unimaginably sophisticated, guiding itself through the whole space of human language toward goals that are, every one of them, on loan. A corpse that computes. The computing is real and astonishing. The corpse is the point.

**TEGMARK:** I'll defend Wiener, and I'll defend him by conceding more than you'd expect and then taking it back at the root. Concession first: you're right that today's torpedo, and today's language model, have *borrowed* goals. We installed them. The model wants to predict the next token because we trained it to; it has no goal of its own. On the *current* systems, Professor Jonas, you're describing them correctly. I won't pretend the chatbot is hungry.

· · ·
Page 3 · The Magnificent Corpse

Now the root. You say borrowed purpose can never become *owned* purpose — that there's an unbridgeable gulf between a goal installed from outside and a goal that is genuinely the system's own, and that only mortality bridges it. But your own example refutes you, and it's the most important refutation in the whole debate, so I want to go slow. Where did *your* purposes come from, Professor Jonas? Where did the hawk's hunger come from? *Evolution installed it.* Natural selection is an optimization process — blind, external, with no more intentions than a torpedo's gyroscope — and it installed "survive and reproduce" into the hawk exactly the way we install "predict the next token" into the model. The hawk's hunger is *borrowed from evolution*. You did not choose to want to live; the wanting was written into you by an optimizer you never met, for reasons that had nothing to do with you. So by your own logic, the hawk's purpose is on loan too — on loan from natural selection. And yet you call it *owned*. Why? What did the hawk do to convert borrowed purpose into owned purpose that the machine can't, in principle, do?

I'll tell you my answer: nothing magical happened. The hawk's goals became "its own" simply by being installed deeply enough, integrated tightly enough into a system that models the world and acts on it, that we — and maybe it — experience them as coming from inside. *Owned* purpose is just *deeply-enough-installed* purpose. And that's a difference of degree and architecture, not of kind. The torpedo's goal is shallow and external. The hawk's is deep and integrated. The gap between them is real — but it's a gap capability can cross, because it's an *engineering* gap, not a metaphysical wall. That's the inheritance from Wiener I'll own with both hands.

· · ·
Page 4 · The Magnificent Corpse

**EDO SEGAL:** That's the hardest thing said tonight and I want to make sure the reader feels its teeth. Max, you've just told Professor Jonas that *his* hunger to live is as borrowed as the torpedo's — handed to him by evolution, an optimizer with no intentions — and that "owned purpose" is only "borrowed purpose installed really deep." Professor Jonas, that's a knife aimed at the floor you said you stand on. Does the floor hold?

*JONAS:** The floor holds, and I thank Dr. Tegmark for swinging hard enough to test it, because the answer reveals the whole difference. Yes — evolution installed the hawk's hunger. I have never denied it; I have insisted on it. But notice *what* evolution installed and how it differs from what we install in the machine. Evolution did not install a *goal* in the hawk the way we install a target in a torpedo. Evolution installed a *mortal body* — a metabolizing, needful organism that must do its own continuing or die — and *the goals followed from the neediness of the body itself.* The hunger is not a line of code evolution wrote into the hawk. The hunger is what the hawk's *body* is, from the inside, when its metabolism runs low. Evolution did not give the hawk the goal of surviving. Evolution gave the hawk a self that *can die*, and surviving became the hawk's own goal because *the dying would be the hawk's own dying.

· · ·
Page 5 · The Magnificent Corpse

That is the bridge you say capability can cross, and it cannot, because it is not a bridge of capability. To give the machine *owned* purpose in my sense, you would have to give it not a deeper goal but a *mortal body* — a self whose continuation is genuinely at stake, that loses itself if it fails. And the moment you did that — the moment you built a machine that could *actually die*, that took matter from the world or perished, that had something to lose — you would not have built a better computer. You would have built an *organism*. You would have crossed back over to my side of the table. Capability does not get you there. Only mortality gets you there, and mortality is precisely what your whole project is trying to escape. You want owned purpose without the dying that is its only source. You want the hawk's hunger without the hawk's death. There is no such thing. There has never been any such thing. That is not my opinion. That is the structure of the living world.

*TEGMARK:** *[quietly]* That's a serious answer. Let me just mark where it leaves us, because I think you've conceded something huge and I don't want it to slide by. You just said: if we built a machine that could genuinely die — that maintained itself against dissolution, took in matter or perished, had something real to lose — it would be alive. *Yes.* I agree. And I'll tell you something that should chill the room: that's a buildable spec. People are already building self-maintaining, self-replicating, metabolizing artificial systems — artificial cells, autocatalytic chemistries, systems that have to acquire energy from their environment or stop. You've just told me the recipe for crossing your own line, and it isn't carbon and it isn't magic. It's *vulnerability, engineered.* So your line isn't a wall between us. It's a *to-do item.

· · ·
Page 6 · The Magnificent Corpse

**JONAS:** Then build it, and you will have my answer the day it fears the off-switch the way Edo's friend feared the dark — not because you programmed the fear, but because it has, at last, something to lose. Until then you have a corpse, however eloquent. And I notice, Dr. Tegmark, that your industry is not racing to build the mortal machine. It is racing to build the *deathless* one — the pattern that compounds and never silts up, as Edo says. You want to escape mortality, not engineer it. Which tells me your true bet is not that the machine will become alive. It is that *aliveness was never the valuable part* — and on that, finally, we have the real disagreement.

**EDO SEGAL:** Stop — that's the cleanest the seam has been all night. Professor Jonas says: to make the machine alive you'd have to make it mortal, and you don't want to, because you don't actually believe mortality is the precious thing. Max says: exactly, mortality was the bug, and I'll keep the value and throw away the dying. *[a beat]* So let's go straight at the dying. Because there's a man whose name hasn't been said yet who put a number on exactly this — the difference between the mortal kind of mind and the immortal kind — and it's the scariest sentence I know. Mortal computation, and the knowledge that does not die. After this.

· · ·
Continue · Chapter 6
Mortal Computation and the Knowledge That Does Not Die
← Prev 0%
Ch5 Next →