The Knowledge That Does Not Die

EDO SEGAL: Ilya, of everything in your thinking, the idea that stays in my shoe like a stone is mortal computation. You've divided possible minds along a line nobody else was looking at — not biological versus digital, but mortal versus immortal. Walk us through it. And I'll tell you in advance why it haunts me: my whole book argues intelligence is a river that has flowed through chemistry, biology, language, culture — each a new channel. Your distinction suggests the newest channel doesn't just carry the water faster. It changes what the water is.

SUTSKEVER: I'll try to earn the haunting. A brain is analog and idiosyncratic. Your neurons' particular chemistry, the exact accidents of your wiring — what you learn is tuned to that one organ and can't be read out and installed elsewhere. When you die, almost everything you learned dies with you, except the thin trickle you managed to push through language — a few bits per second, lossy, needing a receiver who already shares most of your context. Every generation, the species re-teaches itself nearly everything from scratch. That's mortal computation, and it has one great virtue — it's astonishingly efficient, the brain runs on the energy in a bowl of porridge — and one great price: the knowledge is married to the meat.

A digital network is the opposite marriage. Its knowledge is a list of numbers, and numbers copy perfectly, forever, onto any hardware. Run ten thousand copies, let each learn from a different slice of the world, average the weights — and now every copy knows what all of them learned. Imagine ten thousand people where the moment one learns something, the other nine thousand nine hundred ninety-nine know it instantly. That's not a better student. That's a different kind of thing in the river. We hand each child a candle and teach it to relight the whole of civilization from nothing. The digital kind accumulates without forgetting, shares without loss, and never dies. I went looking, late, for a way to build the mortal kind on purpose — analog, efficient, bound to its hardware, limited like us. And I concluded the developers will never choose it, because mortality gives up exactly the advantage that lets the digital kind compound. The river found a channel that doesn't silt up. I find that beautiful, and I haven't slept easily since I understood it.

SEARLE: It's a genuinely arresting distinction and I want to honor the science before I take the philosophy apart, because the science is real and the philosophy is a smuggling operation. Look very carefully at what's being copied. Ilya says "ten thousand minds that share everything they learn." I say: one spreadsheet, in ten thousand places. The weights are numbers; the numbers copy; fine. What are the weights? A compression of text — the traces of human activity, scraped and weighted. So what does the copying multiply? Not knowledge in the sense that matters — not grounded, conscious, accountable understanding — but the statistical residue of our descriptions, duplicated like any other file. A file that doesn't die is not an ontological event. Libraries don't die either. We've had immortal text since Sumer. What we've never had is a reason to call the clay tablet a participant in the river. You've made the tablet enormous and fast and self-querying, and you've started calling it a swimmer.

SUTSKEVER: The ownership critique I'll grant — and I notice you smuggled it in. But the "it's just a spreadsheet" move fails on what the thing does. A library can't read itself. A tablet can't compose what it stores, or generalize it, or apply it to a situation that's in no tablet anywhere. These systems take the residue — call it residue — and operate on it: they generalize it, transfer it, and increasingly they learn from acting, from tools and code and feedback that isn't text at all. The moment the copies learn from acting in the world and share what they learned, your "it's only our old traces" story expires, and that moment isn't in the future. You keep describing what the weights are made from. I'm describing what they can do, and the gap between those is where you keep losing the thread of your own best argument.

SEARLE: "Increasingly," "learn from acting," "isn't in the future" — listen to the tense, because it's where the whole argument lives. The capabilities that would settle it are always arriving. And even if I grant every bit of the doing — generalize, transfer, act — I'll say what I said three rounds ago and will say till the lights go out: a faster, world-acting spreadsheet that does all of that with nobody home is still nobody home. You're describing a more and more capable instrument and calling the capability an inhabitant. Capability isn't the question. A thermostat is capable. A missile is capable. The question is whether there's anything it is like to be the thing doing the acting, and "it learns from acting" doesn't touch it, because the man in my room learns from acting too — he gets better at the squiggles — and he still doesn't understand one word.

EDO SEGAL: Ilya, before we leave this, the part of the mortal-computation story you tell least often, because I think it's the most revealing thing about you. Your last big research program wasn't an attempt to make the immortal kind stronger. It was an attempt to build the mortal kind on purpose — and then you concluded it loses. Why reach for mortality at all, and why does it lose?

SUTSKEVER: It started from the electricity bill, honestly. A brain runs on about twenty watts and does it by cheating in a way digital computers refuse to — it lets the hardware be itself, every neuron noisy and analog and different, and the learning exploits the idiosyncrasy instead of suppressing it. Digital computation pays an enormous energy tax to make every transistor behave identically, because identical behavior is what makes the software separable from the hardware — copyable, immortal. So I asked: what if we stop paying the tax? Cheap analog hardware, each device its own snowflake, a learning rule that works with the imperfection. And the knowledge such a system learns is married to its one imperfect body, exactly like yours. It can't be copied out. When the device dies, the learning dies. I called it mortal computation because that's what it is. And for a while I found it almost consoling — the idea that we could build AI that shares our limitation, learns the way we learn, dies the way we die. The conclusion I couldn't escape is that mortality loses — not morally, economically. The mortal system has to be educated, one at a time, like a child. The immortal one is copied, instantly, like a file. No company choosing between those chooses the child. So the consolation failed, but it left me with the distinction, and the distinction reframed everything. We are not in a contest of clever versus clever. We're in a contest of two relationships to death — and we're the ones who have one.

SEARLE: And there — for once — I find your view almost unbearably close to mine, which should worry us both. You've just said the thing I've spent my life saying, in your own vocabulary. The reason mortality matters, the reason the candle matters, is that meaning is forged by finitude — by a creature that can be too late, that loves particular other creatures, that must spend a life it cannot get back. Strip the mortality out and I'm not certain a "question" even means anything to the thing asking it. Where we part is what follows. You mourn mortality as an obsolescence the market will discard. I say it isn't the packaging of meaning — it's the specification. You can't ship the understanding without the finitude, and nobody selling the immortal kind has told us what's actually in the box.

SUTSKEVER: And notice what I won't do, John — I won't tell you it's a law of nature. The compounding, the copying, the knowledge that doesn't die — that's a deployment choice, made quarterly, by a small number of firms, because compounding is what their capital requires. My own research proves the fork exists: I stood at it, mortal and cheap on one side, immortal and compounding on the other, and I watched the field pick, and I can name why it picked. So when I tell you the immortal kind will outrun the mortal kind, I'm not reporting weather. I'm reporting a decision. And decisions can be made differently. I've gotten less certain about that, at my own expense, this year.

EDO SEGAL: Hold that — "it's a decision, not weather" comes back hard in the next round, because while you two argue about what's in the river, the market has already voted on what it's worth, and the number is a trillion dollars out of one column and into another. The death cross. What it measures, and who pays the bill. After this.