Syntax, Semantics, and the Wake of the World

EDO SEGAL: John, your most durable bequest isn't the room — it's the slogan the room dramatizes. Syntax is not semantics. Form is not meaning. And the strange thing is that AI arrived at the same worry by a completely different road and called it the grounding problem — Stevan Harnad's image of trying to learn Chinese from a Chinese-only dictionary, every word defined by other words, spinning forever inside the lexicon, never touching the world. Two diagnoses of the same wound, one from a philosopher attacking AI, one from a cognitive scientist worrying inside it. Make the wound vivid for me, and then I want to put the hardest counterexample to you that the machines have produced.

SEARLE: The wound is simple and it's everywhere once you see it. The word "water" on a page is not about water. It's a shape. It becomes about water only because English speakers mean water by it — because there are minds with intrinsic aboutness who supply the connection. Strip away every such mind and the marks are mere ink. Now: a computer trained on text has the marks and the relations among the marks, and nothing else. It has read "ocean" next to "wave" and "salt" and "deep" more times than any human could in a thousand lifetimes, and from that it builds a gorgeous web of relations. But it has never been wet. The water never entered. It has the most exquisite syntax in history and not one drop of contact with the world the syntax is about. The meaning is supplied entirely by you, reading the output — exactly as with the marks on the page, only faster and more flattering.

EDO SEGAL: Here's the counterexample, then, and it's the one your critics think is fatal. A model trained only on text learns that king is to queen as man is to woman. It learns Paris stands to France as Tokyo to Japan. It translates an idiom and keeps the joke. It answers a question no human ever wrote down. The distributional hypothesis — that you can learn what words mean from the company they keep — seems to be vindicated past anyone's expectation, because the company words keep is shaped by the world they're used to describe. So the structure of the world got into the text, and the model got it out. Isn't that a kind of grounding through the back door — the world's wake, pressed into our language, recovered by the machine?

SEARLE: It's a fossil of grounding, and a fossil is not a living thing. Yes — the human text the model devours is saturated with meaning that human beings deposited there, human beings who had bodies and got wet. The model inherits the trace of grounding, the shape the world left in our talk. But inheriting the trace is not having the meaning, any more than inheriting a photograph of your grandfather is remembering him. The relations among symbols carry structure, I grant it freely — more structure than my stark slogan might suggest. The question is whether having the structure of the relations is the same as the symbols being about anything for the system itself. And it isn't, because aboutness, real aboutness, requires the system to reach the world, and the model reaches only more symbols. Wire in a camera and you've added symbols about pixels. Symbols about symbols are still symbols. The regress doesn't terminate in meaning. It terminates in us.

SCHNEIDER: Now I get to be the one defending Searle, which will surprise everyone including Searle. I think he's right that the model has the trace and not the thing — that fluency about the world is not contact with the world. Where I split from him is the same place as before: he treats this as a permanent verdict resting on biology, and I treat it as a current description that might be changed by the engineering. The model has only ever met text. Fine. But the engineers are not standing still. They're welding the symbols to the world by force — multimodal systems tied to millions of pixels of actual cats, robots embodied in bodies that act and receive feedback from a world that pushes back. That is the Robot Reply being built in silicon, John. And your own answer to the Robot Reply gave away the game decades ago.

SEARLE: Did it.

SCHNEIDER: It did. The Robot Reply says: put the program in a robot, wire its symbols to cameras and limbs, and now the symbols are causally hooked to what they're about. Your reply was: drop the man inside the robot's head, let him shuffle the symbols coming from the cameras, and note he still understands nothing. But that reply concedes your central premise. It grants that syntax alone isn't enough — and then adds the missing ingredient, causal contact with the world. That's not a refutation of grounding. It's a roadmap for it. You treated a roadmap as a dead end because admitting it was a roadmap would mean admitting the gap could, in principle, be crossed by something other than a brain.

SEARLE: No — my reply was that adding the causal contact through more symbols doesn't conjure intrinsic meaning, because the man inside the robot's head is still just getting squiggles, now with a camera attached. The robot's behavior gets better. The aboutness doesn't arrive. You want the causal loop through the world to be the meaning. I say the causal loop through the world is necessary and not sufficient — you also need a system of the right kind to have states that are about what the loop connects to, and the man-in-the-robot shows that running the program isn't that kind of system. The robot reply relocates the problem to the body. It doesn't solve it. The water still hasn't entered. There's just a wetness sensor now, sending more squiggles.

SCHNEIDER: But "the right kind of system" is precisely where you smuggle in the biology you never justified. You keep saying the loop is necessary-not-sufficient and the missing piece is the right causal powers — and then you point at neurons. Why neurons? You could never say. You admitted it was a matter for future neuroscience. So at the deepest level your "the water hasn't entered" reduces to "it isn't made of meat," and that is not an argument, John. It's a bet. A serious bet, an unfashionable bet I respect — but a bet, and you dressed it as the floor of reality.

SEARLE: It's a bet the way "the sun is hot" is a bet. The only systems we have ever observed to have intrinsic intentionality are biological. Every single one. The functionalist confidence that the substrate doesn't matter is an extrapolation from a sample size of zero — there is not one example, anywhere, of meaning running on something that isn't a nervous system. I'm staying close to the only evidence there is. You're the one extrapolating into the void and calling my refusal to follow you a prejudice.

SCHNEIDER: And I'll grant you that — it's the strongest thing your side has, and the functionalists are too quick to wave it away. The sample size really is zero on the other side. We have never seen consciousness on silicon. But John, "we've only ever seen it in biology" is a fact about our observations, not a fact about the boundaries of the possible. We'd only ever seen flight in birds, too, until we hadn't. The honest position isn't your "no" and it isn't the optimist's "obviously yes." It's: this is an open empirical question about which we should build instruments and gather evidence, instead of settling it by either of our intuitions. Which is exactly why I spend my time designing tests instead of giving speeches.

EDO SEGAL: And there's my next round, handed to me by Susan. Because John has spent his life telling us what would not be evidence — fluency, behavior, passing the imitation game — and Susan has spent hers asking the harder question: if behavior can't prove it, what could? Is there any test at all that could detect a mind on the other side of the glass? You designed two of them. We open them next. Hold the thread on the costume of common sense, John — it's still waiting at the rainstorm. After this.