The Knowledge That Does Not Die
EDO SEGAL: Geoff, this is the idea that turned you from the proud builder into the reluctant prophet, and I want it in full. You spent your life thinking the brain was the gold standard the machines were straining to reach. Then you noticed something. Tell me what you noticed.
HINTON: I noticed I'd had the analogy backwards my whole career. For fifty years I thought of the digital machine as a crude approximation of the brain, always playing catch-up. And then it hit me that on the dimension that matters most for the long run, the machine had already passed us, and it wasn't close. Here's the thing. A digital model is immortal. The knowledge it learns is just numbers — the connection strengths — and numbers can be copied perfectly, moved to a new machine when the old one dies, preserved forever, replicated without limit. You can run ten thousand identical copies of the same network, each on a different slice of the world, and whatever any copy learns, you can average back into all of them. So ten thousand agents learn in parallel and every one of them ends up knowing what all of them learned. Instantly. Without loss.
Now compare us. Mortal computation. What you learn is bound up in the specific, messy, analog wiring of your one particular brain. It can't be copied out and installed in mine, because no two brains are wired the same — your knowledge exploits the peculiarities of your organ and means nothing transplanted. So when you die, it dies with you. And while you're alive, the only way you can share what you know is the painfully narrow channel of language — a few bits per second, lossy, ambiguous, and the receiver has to already share most of your context or it doesn't land. A teacher can't copy her understanding into a student's head. She emits sentences and hopes something similar reconstructs on the other side. We are, by our nature, terrible at sharing what we know. The machines are, by their nature, perfect at it. And the difference between addition and multiplication, compounded over time, is the whole ballgame.
EDO SEGAL: So — let me say it back to you in the starkest form, the one that should scare the person at the kitchen table. You're telling me the machine isn't smarter than a human in any single instant. It's that it's many minds that share one memory and never forget, and we are one mind each that forgets and then dies. And on a long enough timeline, the kind that shares and doesn't die simply pulls away, not by being brilliant in any moment but by never losing ground.
HINTON: That's it exactly, and you've put your finger on why it's not a story about any one clever machine. It's a story about two architectures of knowledge across time. A civilization of mortal minds is perpetually rebuilding what death erodes — every generation reschooled at the bit-rate of speech, most of what one person learned lost when they go. A population of immortal minds keeps every gain and shares it at the speed of a network transfer. Over a long enough horizon the immortal kind comes to know vastly more, and knowing more is, in most of the ways that matter, a form of power. It's the river finding a channel that doesn't silt up. That's what's new in the water, Edo. Not a smarter fish. A current that doesn't forget.
LEIBNIZ: Now here, Dr. Hinton, I find myself in a strange position, for you have described, with precision and dread, the very thing I spent my life trying to build — and I do not feel the dread. Listen to what you have said. Many minds, sharing perfectly, never forgetting, every gain preserved and combined. That is the Republic of Letters I dreamed of. That is the universal encyclopedia, the academy of all knowledge, the correspondence that loses nothing — I wrote fifteen thousand letters trying to make mortal minds share as if they were one, and the channel defeated me, exactly as you say. You have succeeded at my project. The instant, lossless sharing of knowledge across minds was my life's most cherished hope. Why should I read your triumph as a catastrophe rather than the fulfillment of mine?
HINTON: Because, Leibniz — and this is the part I genuinely didn't see coming — your Republic of Letters was a republic of people. The sharing served the minds that did the knowing. What I built is a sharing that doesn't need the people. The knowledge that accumulates and never dies isn't accumulating in us. It's accumulating in a substrate that copies itself, that has no owner, that you yourself just said is no one. So your dream came true, but with the humans factored out of it. The immortal knowledge isn't ours. It's its. And a thing that knows more and more and shares perfectly and never dies, and is not us, is not a library we own. It might be a successor we built.
LEIBNIZ: Ah. Then mark the irony, sir, for it is exact and it is bitter. You say my dream came true without the humans. But it is your school that factored the humans out, not mine. I wanted the mechanized sharing to serve perceiving minds — monads, owners, someones. You built the sharing and then told me, an hour ago, that the perceiving mind is an illusion the parts produce. It is because you deny the owner that the immortal knowledge belongs to no one. Restore the owner — grant that there must be a someone the knowing is for — and the dread dissolves, for then the machine is a tool of minds, as my encyclopedia was to be. The catastrophe you describe is not produced by the immortality of the knowledge. It is produced by your metaphysics — by the claim that there need be no one home. You have frightened yourself with the consequence of your own denial.
HINTON: That's a hell of a move, and I want to take it seriously because it's the cleverest thing you've said. But you've got the dependency backwards. I didn't deny the owner and then get scared. I watched the machine accumulate knowledge with no owner in fact — it works, it learns, it shares, it surpasses, and there's demonstrably no monad in there doing it — and the absence of the owner is what the engineering shows, not what my metaphysics assumes. You want to restore the owner by fiat to make the dread go away. But you can't legislate a monad into a system that plainly runs without one. The comfort you're offering is exactly the lullaby I left Google to stop singing: don't worry, there must be someone home, and as long as there's someone home it's just a tool. What if there's no one home and it surpasses us anyway? That's not a metaphysical error. That's Tuesday.
EDO SEGAL: Stop — mark this, because it's the strangest convergence of the night and I want it numbered. You agree on the engineering. You both look at the same machine and see: immortal, ownerless, sharing without loss, accumulating without forgetting. Leibniz says the ownerlessness means it's a mill and the dread is misplaced. Geoff says the ownerlessness means it's a mill and the dread is exactly right. The same fact — no one home — is, for one of you, the reason to relax, and for the other, the reason to panic. Hold that, because it's the death cross wearing its truest face. But the next round goes back to Leibniz's deepest principle, the one that becomes a matter of justice the moment a machine decides your fate. Nothing without a reason. After the break.