The Shortcut and the Leap

EDO SEGAL: Stephen, I want to give you a clean run at the idea that organizes everything you believe, because the whole second half of this debate turns on it, and most people get it slightly wrong. Computational irreducibility — say it as starkly as you can, and then tell me what it does to the dream that we could ever predict and control a powerful machine.

WOLFRAM: Starkly, then. For many computational processes there is no shortcut. The only way to find out what the process does is to run it, step by step, all the way through. You cannot leap ahead. You cannot derive the answer from a formula. You cannot, in any meaningful sense, predict the outcome faster than the process itself produces it. To know the future of the system, you must do as much computational work as the system does — which is to say, you must wait. This runs against the deepest instinct of traditional science, which is the instinct toward prediction — Newton's triumph was that you could calculate where the planets will be in a thousand years without living through the thousand years. My claim is that that power is the exception. It works for the thin set of systems that happen to be reducible. For the vast majority — the irreducible ones — no shortcut exists, and no amount of cleverness conjures one, because it is not there to be found. Now apply it to AI. The dream underneath all the safety thinking, and underneath most of the hope, is the dream of prediction and control — knowing in advance what a powerful system will do, proving it stays in bounds, proving it won't deceive us. Computational irreducibility says: for any system rich enough to be worth worrying about, complete advance prediction is impossible in principle. Not hard. Impossible. A system rich enough to be genuinely intelligent is rich enough to be irreducible, and an irreducible system cannot be fully predicted by anything short of running it and seeing. There is no shortcut. Not for the genius. Not for the superintelligence. Not, and I mean this precisely, for God — if God wants to know what the universe does, God has to run it. Omniscience-by-shortcut is not on the menu computation offers.

PLATO: This is magnificent and I believe a great deal of it, and I want to show you exactly where I get off the train, because it is one stop earlier than you think. You have proven — and I accept it — that the unfolding in time of an irreducible process cannot be shortcut. Granted, entirely. But you have slid from "the unfolding cannot be shortcut" to "the truth cannot be reached except by unfolding," and those are different claims. Consider: there are infinitely many true theorems about rule 30 that I can prove without running it a million steps — that it is deterministic, that it is reversible or not, that it belongs to such-and-such a class. Those truths are about the irreducible process and are themselves reached by a leap — by reason, not by running. So the irreducible process has a reducible halo of necessary truths around it, and the most important truths, the ones that tell you what kind of thing it is, live in the halo. You have shown that you cannot leap to what happens at step a million. You have not shown that you cannot leap to what rule 30 essentially is. And it is the second kind of truth — the essential, the necessary, the truth about the Form of the thing — that I have always cared about. The accidents you must run for. The essence you can behold.

WOLFRAM: Okay, this is a really good objection and I have to slow down for it because it's the best version of Plato's case I've heard. You're right that there's a reducible halo — there are always pockets of reducibility, even around the most irreducible system, and the meta-facts ("it's deterministic," "it's class four") live in the pockets. I've spent my life mining exactly those pockets; science just is the hunt for the reducible questions inside an irreducible ocean. But here's my counter, and it's where I think your "essence" smuggles something in. You say the essence is what rule 30 is, beheld whole. I say: there is no "what it is" over and above the totality of what it does — and the totality of what it does is irreducible, so the full essence is exactly the thing you can't behold; you can only run it. The pockets you can leap to are partial — true, valuable, real, and partial. You're treating the partial reducible facts as if they were the whole essence, because they're the part you can grasp, and then calling the graspable part "the Form" and the rest "mere accident." But for an irreducible system the "mere accidents" are most of what it is. You've defined essence as "the part Plato can reach by leaping" and then concluded that essence is reachable by leaping. The universe didn't agree to that definition.

PLATO: Touché, as your French would say — and I will not pretend you did not land it. Let me concede precisely and keep precisely. You are right that I have been tempted, my whole life, to call the graspable part the essence and dismiss the rest. It is the philosopher's besetting vanity, and I named it myself in the envy round — the pride of the mind that believes it can leap straight to the eternal. So I concede: there are systems whose full reality is not in any beholdable Form, whose truth is genuinely distributed across an unfoldable running, and for those, you are right and I am wrong, and the machine that runs them faster than I can is doing something I cannot do by any inward turn. But I keep this: it does not follow that all truth is like that. Some truths — the necessary ones, the mathematical ones, the moral ones at the summit — are beholdable, are graspable whole, and for those the running is beside the point. We have, between us, divided the world. You have the irreducible accidents. I have the eternal essences. The only fight left is the size of each territory.

EDO SEGAL: Mark it. Convergence Two — and this one's bigger than the first. You've partitioned the question. Stephen concedes there's a reducible halo of necessary truths you can leap to. Plato concedes there are irreducible truths you can only get to by running. The disagreement has narrowed to a border dispute: how much of what matters — knowing, learning, understanding — lives in the leapable halo, and how much lives in the irreducible deep. Plato thinks the things that matter most are in the halo, beholdable. Stephen thinks the things that matter most are in the deep, runnable only. That's no longer a clash of cosmologies. It's a map with a contested border. And the next round draws the border right through the middle of a human life — through learning, through what we do with children, through the river and the dam. Hold the map. We start there after this.