The Man in the Room

EDO SEGAL: Searle, most of our readers have heard the phrase "the Chinese Room" the way they have heard "stochastic parrot" — secondhand, as an insult or a slogan. I want you to tell it the way you would tell it to a sharp fifteen-year-old, the whole machine of it. And then, Hobbes, I am going to ask you to do something a debater hates to do. Before you take it apart, I want you to steelman it. Tell us what the room gets right.

SEARLE: Then here is the room, plainly. I am locked inside it. I am a competent speaker of English and I know no Chinese; I cannot even reliably tell Chinese script from Japanese. Through a slot I am handed a first batch of Chinese characters. Then a second batch, with a rulebook in English that lets me match the second to the first purely by their shapes. Then a third batch, with further rules for producing characters of my own in response. The people outside, unknown to me, call the first batch the script, the second the story, the third the questions, and what I hand back the answers. I become so adept that my answers are indistinguishable from a native speaker's. I pass their test perfectly. And I understand nothing. I have manipulated uninterpreted symbols from start to finish. Every scrap of the understanding the Chinese speakers perceive is being projected onto me from outside. That is the room. The genius of it, if I may say so, is that it puts a human being where the processor goes — a being whose inner life we know intimately, because we have one — and lets us check directly whether running the program produces comprehension. It does not.

EDO SEGAL: Hobbes. Steelman first. What does the room get right?

HOBBES: It gets right a thing I myself insisted upon, and I will not pretend otherwise merely because I mean to demolish the rest. It is true that a man may reckon upon names whose signification he has not himself fixed — that he may push the counters correctly while the meaning of the counters rests in another's keeping. I said as much: names are settled by a community, anchored in sense, and a man cut off from that settling reckons with borrowed marks. Searle's man has the marks and not the settling. So far the room and I agree, and any honest mechanist must grant it. The room also rightly warns against the credulity of the Chinese speakers outside, who read a mind into the slot because fluency has always meant a mind. I have no quarrel with the warning. There. I have built his case. Now let me take it down.

The room is a cheat, and the cheat is the man. Searle has smuggled a whole understanding mind — his own English-speaking self — into the very machine whose understanding is in question, and then pointed at that mind and said see, it does not understand Chinese. Of course the man does not. The man is a single transistor declaring that the computer it sits in cannot compute. The understanding of Chinese, if it is anywhere, is not in the man; it is in the whole reckoning — the man and the rulebook and the patient operation taken together as one engine. To ask the man whether he understands Chinese is to ask a gear whether the clock keeps time. The gear knows nothing of hours. The clock keeps time regardless.

SEARLE: That is the Systems Reply, and I will be honest with this room, as I was honest in 1980 — it is the single most important objection to the Chinese Room, and I am not going to bury it. Hobbes says the man does not understand, but the system, man-plus-rulebook-plus-room, does. My answer is to internalize the system. Let the man memorize the rulebook. Let him do every calculation in his head. Let him throw away the room and walk outdoors with the whole apparatus inside one skull. Now there is no system over and above the man — he is the entire system — and he still understands not one word of Chinese. He has the whole engine inside him and not a flicker of comprehension. So there is nothing for the understanding to belong to except the man, and the man is empty of it.

HOBBES: And there, Searle, you trade on an intuition and pass it off as a fact, which is precisely the Schoolmen's vice. When your man memorizes the rulebook and runs it in his head, he is no longer one engine. He is two. He is the English-reckoning man he was, and he is now also implementing a second reckoning — a Chinese-understanding process running upon the same flesh — and the first man, the host, has no window into the second. You ask the host do you understand Chinese and he says no, and you treat his report as a verdict. But the host is exactly the faculty we have most reason to distrust here. Your own eye solves geometries of light your reckoning self knows nothing of. Your skull is full of competences you cannot survey. The host's sincere I understand nothing is the report of one process about another it cannot see. It is not the absence of understanding. It is the absence of access.

EDO SEGAL: Let me hold this exact moment, because the reader should feel how strange it is. The two of you have switched the costumes the audience expected. Searle, the relentless modern, is resting his whole case on a first-person intuition — I imagine being the man, I introspect, I find no Chinese. And Hobbes, the seventeenth-century mechanist, is the one saying do not trust your introspection, the mind is full of machinery you cannot see. Searle, is Hobbes right that the room finally comes down to an intuition? That you imagine being the man, you look inside, you find nothing, and you conclude there is nothing?

SEARLE: He is partly right, and I have always been more willing to admit it than my admirers are. The room does deliver its verdict through the first person. But I deny that this is a weakness, because the first person is not incidental to understanding — it is the whole phenomenon. Understanding is something it is like to do. When I understand a sentence, there is a conscious grasping of its meaning, and that grasping is available to me from the inside in a way nothing else is. Hobbes wants to say there might be understanding in the system that the man cannot survey. But understanding that no one is conscious of is not understanding; it is information processing that we are calling understanding by analogy. The host's report is not the report of one process about another. It is the report of the only place understanding could be — the conscious subject — saying it is not there.

HOBBES: Then you have moved the wager, and the reader should mark that you moved it. A moment ago the claim was syntax is not semantics — a claim about the operation. Now the claim is understanding requires consciousness — a claim about an inner light. These are not the same claim, and the second is far more contestable than the first, for you have not the faintest mechanism for why the motion of matter should be accompanied by a light at all. You accuse my reckoning of being empty. I accuse your consciousness of being a sanctuary you have built precisely so that nothing can ever be shown to enter it. A fortress with no gate is not a proof of safety. It is a confession that you fear the siege.

EDO SEGAL: Stay one more beat, because there is a ghost at this table and I want to seat him properly before we go on. Hubert Dreyfus. He spent his life arguing that machines would never achieve real intelligence because intelligence is embodied, situated, a matter of coping with a world rather than processing symbols about one — and he aimed that argument, in his day, at the symbol-shuffling AI that you, Hobbes, are the deep ancestor of. So whose ghost is he? Searle, you cited him your whole career.

SEARLE: He is mine, and gladly. Dreyfus saw, before almost anyone, that the rulebook approach to mind would fail — that you cannot capture human competence in explicit symbols operating on explicit representations, because there is always more know-how needed to apply the rules, all the way down. The classical AI of my era broke on exactly that rock. My Chinese Room and his embodied critique are two windows onto one truth: that formal symbol manipulation, sealed off from a living body in a world, is not understanding. Hobbes is the man who said reason is reckoning upon names. Dreyfus is the man who said no, reason is coping in a world, and the names are the least of it.

HOBBES: And I will take him from you, for Dreyfus despised the rules and the representations — the explicit symbols of the Schoolmen-turned-programmers — and your engine has none. It learned no rulebook a man could read; it settled its own counters across uncountable weights, knowledge smeared through the machine where no one can point to it, competence that cannot say its own rules. That is not the symbolic AI Dreyfus buried. That is reckoning without articulable rule — which is what he said human skill was. You have kept his quarrel with explicit rules and forgotten that your own machine already abandoned them. He is no friend of your room. He is the gravedigger of the rulebook, and your engine climbed out of that grave.

EDO SEGAL: Hold that — it comes back when we put the machine in a body. But the room has handed us something cleaner than I hoped: Searle says the understanding is missing because the conscious subject reports it missing. Hobbes says the report is the testimony of one process blind to another, and that Searle has quietly traded a claim about syntax for a claim about an inner light. The next round is about that trade. Because Hobbes keeps saying name the mechanism of meaning, and Searle keeps saying aboutness — and underneath both is the oldest question about a word: where does its meaning come from? Hobbes spent pages on that. Let us spend a round.