The Crossing

**EDO SEGAL:** The rules of this round are short. Each of you questions the other, directly. I will not rescue anyone, and I will not referee. Noam — you've been pressing the experiment all night, so you put it to him. Begin.

**CHOMSKY:** Sir Francis, you keep saying the engine surprises you — that it does more than you can account for, and you take the surprise as evidence that something real is there. So here is my question, and I want the answer in your own method's terms, not in wonder. Surprise is a fact about your expectations, not about the system. When the symbolic-AI people were surprised that their rule-systems failed, the surprise told us about their bad theory, not about the world. You, of all men, the one who taught that the understanding receives an infusion from the will and believes what it would prefer were true — why is your surprise at the machine not simply your own Idol of the Tribe, your wish to see a knower where there is a sophisticated tabulator, dressed as evidence? Name what you have observed in the engine that is evidence rather than astonishment.

**BACON:** That is a just blow, struck with my own weapon, and I will not pretend it does not land. You are right that surprise is no instance, and that a man who built a doctrine of wishful seeing must suspect his own wonder first. So let me give you the instance and not the astonishment. What I have observed is this: the engine answers questions no man wrote down, follows instructions it never saw, composes constraints it was never given together, and — the one that matters — corrects an error when the error is pointed out, producing a repair fitted to a situation that appears nowhere in its instances. That last is not surprise. That is a behavior, observable, repeatable, which on my account requires that the engine has digested from the instances some compressed structure it can apply to the genuinely new. Now — I grant you it is not yet the decisive instance, which is the one you named: the loss taken to keep a word true, against incentive. I have not observed that cleanly, and until I do my knower is not proven. So I will turn your blow into my question, since the chair is silent. You say the loss-taking is the test and the machine cannot pass it. Have you tried? Or is your certainty that it cannot the same dogmatism you accuse me of — a conclusion deduced from your theory of what the machine must be, never vexed against the machine itself?

**CHOMSKY:** I have looked, and I will tell you honestly what I find, because evading it would prove your charge. I find systems that can be trained to express uncertainty, to hedge, to flag low confidence — and I find that this is a trained behavior, tuned because users reward it, not a disposition that arises from a held standard. The system that "admits" it was wrong does so because admission was reinforced, and the same system can be pushed, by a differently shaped prompt, to abandon a true claim it just made. So what I observe is the appearance of the test being passed and the substance of it failing — the footprint without the foot. But I will concede the thing your method demands I concede: this is current evidence about current systems, and it is not a proof about all possible systems, and if a system someday holds a standard against incentive as a stable disposition rather than a tuned reflex, I will have been wrong, and I will say so. I do not think it will happen, because I think the architecture forbids it — there is no world the words are anchored to for the standard to be a standard about. But "I do not think it will happen" is a prediction, not a theorem, and you have correctly made me say which one it is.

**BACON:** Then it is my turn, and I will not waste it on the easy ground, because there is one place where I think your fortress sits on sand, and it is the place you are most certain of. You say the human child is the miracle the machine does not explain — that it learns from almost nothing because it was built to, on a structure prior to all experience. But where did the structure come from? You will say evolution. Evolution is an optimizer with no foresight and no intent — it gathered instances over deep time, the survivals and the deaths, and it digested them, blindly, into the faculty you are so proud of. Your innate structure is itself the product of an induction — a vast, slow, pitiless induction over the history of a species, with the dead as the negative instances. So your "prior structure" is not prior to learning at all. It is the frozen residue of an earlier, larger learning. You and the machine are both inductions. The only difference is that yours ran on bodies over a billion years and the machine's ran on text over a year. Why is the slow induction that built your faculty a different kind of thing from the fast induction that built the engine's competence — and not merely an older one?

**CHOMSKY:** That is the best question I have been asked tonight, and I will not give it a cheap answer. You are right that the faculty is the product of evolution, and right that evolution is, in a loose sense, a search over instances — though I would resist calling it induction, because it has no hypotheses and tests nothing; it merely differentially reproduces. But grant your framing for the sake of the blade. Even so, the two inductions produce categorically different things, and the difference is the whole of my position. The slow one built a system embedded in a living body that acts in a world, that has stakes — survival, hunger, a child who is or is not where it was left — so that the structure it produced is a structure whose outputs are anchored, finally, in consequences to a self. The fast one built a system embedded in nothing, anchored to nothing, with no stakes and no self and [no world the words are about](https://www.youonai.ai/fieldguide/med/symbol_grounding_problem). So yes — both are inductions, if you insist. But one induction produced a knower with a world, and the other produced a model of the wake that world left on paper. The age is not the difference. The anchoring is the difference. Your billion-year induction was vexed, at every step, against a real world that killed the wrong answers. The machine's induction was vexed only against the text, which kills nothing and stakes nothing. You have shown me they are both inductions. You have not shown me they induced the same kind of thing — and the kind of thing is exactly what we have been arguing about since the first minute.

**BACON:** Then I concede the round to a draw, which from me is high praise, because you have answered the one question I thought unanswerable, and answered it with my own concept of vexation turned against me. The anchoring. I gave you the anchoring myself, hours ago, when I said the engine obeys the text and not the world. You have made it the load-bearing wall of your whole house, and I cannot knock it down, because I built it.

**CHOMSKY:** And I concede that your evolution-as-induction is the strongest pressure my position has ever been under, and that I will be thinking about it long after tonight — because if the difference really is only anchoring, and the machines are someday anchored, in bodies, in stakes, in a world that kills their wrong answers, then the wall I am leaning on is a wall that can in principle be moved. I do not expect it to be moved soon, or easily, or perhaps at all. But you have made me say "in principle," and that is a word I did not come into this room prepared to say.

**EDO SEGAL:** And there — after three hours — the two of you have traded weapons and found you are each holding the other's. Francis built the wall of anchoring and Noam is standing on it. Noam named the experiment of the held standard and Francis accepts it as the only test. We close after this. Final statements, and the last word each.