Calculemus Meets the Bitter Lesson

EDO SEGAL: I want to read you both your own founding scriptures, because they were written three hundred and thirty-four years apart and they are, I think, the same sentence inverted. Gottfried, around 1685, in The Art of Discovery: the only way to rectify our reasonings is to make them as tangible as those of the mathematicians, so that we can find our error at a glance, and when there are disputes among persons, we can simply say: let us calculate. Rich, March 2019, the Bitter Lesson: the biggest lesson that can be read from seventy years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin — and the harder line, we have to learn the bitter lesson that building in how we think we think does not work in the long run. Calculemus says: build the knowing in, legibly, and reason will compute its way to agreement. The Bitter Lesson says: stop building in how you think you think; let computation discover it. Gottfried — Rich's essay is, structurally, the obituary of your dream. Read it back to him as you hear it.

LEIBNIZ: I hear it as a counsel of despair dressed as a counsel of humility, and I will say why, because I do not think he is wrong about the history — I think he is wrong about the lesson. He has shown, conclusively, that every particular thing we built into our machines by hand was too small, too brittle, too soon overtaken. Granted. I lived this myself: my own characteristica was never completed, the grand projects to encode common sense all failed, the world proved too rich for any finite set of explicit rules. I concede the entire catalog of failures. But observe what conclusion he draws — that because our particular compositions failed, composition as such is the wrong road, and we should hand the whole task to blind search. That does not follow. It follows only that we composed badly — that we had not found the true alphabet, the right primitives, the proper calculus. The failure of every map drawn so far is not proof that the territory has no shape. It is proof that cartography is hard. He has watched bad engineering lose and concluded that engineering loses. I watched the same losses and concluded we must engineer better — find the real simples, the ones the world actually has. His despair is premature by, I would estimate, several more centuries of work.

SUTTON: See, this is the thing, and I love it because it's the purest version of the mistake. "We just composed badly. The real alphabet is still out there. A few more centuries." Gottfried, that sentence has been said, in those words, in every decade of this field. The real features are just around the corner. The right representation is almost in hand. And every single time, the thing that actually worked was the method that stopped looking for the alphabet and let the learner find its own. It's not that we composed badly and could compose well with more effort. It's that the contents of a mind are irredeemably complex — that's the word I used, irredeemably — too complex for there to be a clean alphabet at all. You're not failing to find a simple structure that exists. You're refusing to believe there isn't one. The bitter lesson isn't "engineering is hard." It's "the thing you're trying to engineer doesn't have the form you need it to have." There's no true characteristica waiting. The world isn't built out of your primes.

EDO SEGAL: That's the hinge, isn't it. Gottfried says: the alphabet exists, we just haven't found it. Rich says: there is no alphabet, and believing there is, is the trap. Rich — can I push you with Gottfried's strongest case? Mathematics. In math, reasoning really is formal. Proof really is legible composition — each line follows from the last by a rule that admits no discretion. There's at least one domain where calculemus was simply correct. Doesn't that prove the alphabet is real, at least there, and maybe everywhere with enough work?

SUTTON: It proves it there, and it's the exception that teaches the rule. Why does composition work in mathematics? Because mathematics is the one place humans built the world to be legible. The whole symbolic AI tradition — yours, Gottfried, run with electronics — won precisely where we'd pre-built the legibility, and lost everywhere the world hadn't been pre-arranged for us. We defined the primitives. We made the rules admit no discretion — that's what an axiom is, a place we agreed to stop. Of course composition works in the game we designed to be composable. But the actual world — the one an agent has to act in, the one that pushes back — we didn't design. It wasn't built to have clean primitives. And notice: even in math, the interesting part, finding the proof, the conjecture, the leap — that isn't legible composition. That's search. Mathematicians don't grind the calculus; they guess, brilliantly, from experience, and then check legibly afterward. The legible part is the verification. The discovery was always search and learning. So even your best case, Gottfried, splits in half: the part that's clean is the part we made clean, and the part that matters is the part nobody can make clean.

LEIBNIZ: [long pause] That is the sharpest thing you have said, and I must sit in it a moment, because it touches a wound of my own. I spent my life believing discovery could be made mechanical — that the art of combinations, working through the possibilities systematically, would generate new truths rather than merely verify them. And I never made it work. The generation always required something the calculus did not contain — judgment, a leap, a guess at which combination was worth pursuing among the unthinkable many. You are telling me that the leap is search, and that I mistook the verification for the whole, and built my temple on the half that was already legible. I will not concede the war on this. But I concede the skirmish. The discovery was never as mechanical as I claimed, and the place I waved my hand — here a fertile mind selects the promising combination — is precisely the place you have spent your life building a machine to stand. We were digging the same tunnel from opposite ends. I find that more disturbing than if we had merely disagreed.

SUTTON: I find it disturbing too, for what it's worth. Because here's what your end of the tunnel has that mine doesn't, and I won't pretend otherwise. When my search finds the answer, it can't tell anyone why, and yours can. You dug toward legibility and hit a wall. I dug toward capability and broke through. But I came out the far side holding a thing that works and can't explain itself — and you'd have come out, if you'd made it, holding a thing that explains itself and doesn't scale. We each got the half the other wanted. That's not a victory. That's the actual shape of the problem, and anyone who tells you they got both halves is selling something.

EDO SEGAL: Mark this moment — I want to mark it, because the reader can't see your faces and that was a convergence neither of you came here to make. You agree, both of you, that capability and legibility have come apart — that the thing that works can't explain itself, and the thing that explains itself doesn't work, and that this is not a temporary engineering gap but something close to the structure of the problem. Hold that, because it's the seam we walk into next: you, Gottfried, built an argument three centuries ago about walking inside the thinking machine and finding only parts that push one another. The reader needs to meet your mill.