The Mill, the Room, and the System Reply

EDO SEGAL: Professor Searle, you said in the opening — generously, I thought — that Leibniz built the room before you did, that the mill is the room with gears instead of paper. I want to test that, because if it's true it's a remarkable thing, two men three centuries apart building the same machine to catch the same prey. So tell me, precisely: where do the mill and the room do the same work, and where does each do something the other can't?

SEARLE: They share the central move, and it's the most important move in the philosophy of mind: take the opaque thing and make it transparent. We grant minds to computers partly because we can't see inside them — the chips are fast and alien and we fill the dark with mind. Leibniz enlarges the mechanism to the size of a mill so you can walk in and look. I put a person — me, whose inner life you know because you have one — where the processor goes, so you can check directly whether running the program produces comprehension. Same move. Remove the opacity. Look. And in both cases what you find is: only mechanism, and no understanding. He found gears. I found a man shuffling squiggles. Neither of us found a mind.

Where we differ — and I'll be honest that I think my version is tighter, with respect across the centuries — is what we claim from the looking. Leibniz looks and says: therefore perception isn't here, it must live in a simple substance, a monad, somewhere else. He needs an extra metaphysical thing to put the mind into. I look and say: therefore this kind of process — formal symbol manipulation — isn't the right kind of thing to be understanding, full stop, and I don't need to tell you where understanding lives instead. I just need to show it doesn't live in the shuffling. The mill proves too much, in a way. It seems to prove that no mechanism perceives, which then embarrasses Leibniz when someone points at the brain. My room only proves that programs don't — which leaves the brain alone, because the brain isn't running a program, it's a biological organ with causal powers.

LEIBNIZ: That is a fair account and a clever fence around your own position, and I will concede the tactical point — your room asks for less, and an argument that asks for less is harder to refuse. But I will not concede that I proved too much, because you have softened your own claim to avoid my problem and I think it costs you. You say the brain is not running a program, it is an organ with causal powers. Very well. But when I ask you which causal powers, you go quiet, as you went quiet for forty years. You assert the carbon is special. You do not derive it. My monad at least answers the question you dodge: I tell you exactly what the brain has that the machine lacks — genuine unity, a simple subject. You tell me the brain has "the right causal powers" and decline to name them. Between a wrong answer stated clearly and a gap left politely unfilled, the historians will have to choose.

SEARLE: That's fair, and it's the sharpest thing anyone's said to me in years, so let me not dodge it now. You're right that I assert the biology more than I derive it. I bit that bullet my whole life. But here's why I'd still rather have my gap than your monad. Your unity is undetectable. If the mind is a windowless simple substance, then nothing I could ever observe — no behavior, no scan, no inspection — could show me whether a thing has one. You've made the answer unknowable in principle. At least my gap is the honest shape of our ignorance: we don't yet know which causal features of brains produce minds. That's a gap neuroscience might one day fill. Yours is a door welded shut.

EDO SEGAL: Let me bring a third figure to the table between you, because he stands exactly in the gap. Alan Turing, 1950. He looked at this whole question — can machines think? — and called it "too meaningless to deserve discussion," and proposed we replace it with a test: if you can't tell the machine from a human in conversation, stop asking whether it "really" thinks. He tried to dissolve the very dispute you two are having. Professor Searle, your room was built precisely to answer him. Is he your great opponent, or did he see something you're both missing — that the inside question can't be settled, so we should change the subject?

SEARLE: Turing is my great opponent, and I respect him enough to say he was making a move, not a discovery. He saw the inside question looked unanswerable from outside — true — and concluded we should redefine "thinking" as "passing the test." That's not solving the problem. That's renaming the behavior and walking away from the question. The room is my refusal to walk away. It grants Turing everything — yes, the machine passes, yes, you can't tell from outside — and then says: and it still doesn't understand, which proves the test was measuring the wrong thing. Turing wanted to make the inside question disappear because it's hard. I think the inside question is the only one that finally matters, precisely because it's the one the test can't reach. He was a greater man than I'll ever be, and on this one thing he blinked. He changed the subject because the subject was unbearable.

LEIBNIZ: And I will surprise you, Mr. Searle, by standing with Turing one quarter of the way — for he and I share an honesty you sometimes lack. Turing saw what I saw three centuries before: that you cannot reach the inside by looking. Where he went wrong was the inference. He said: since we cannot see the inside, let us declare the inside irrelevant — measure only the behavior. I say: since we cannot see the inside, let us hold it open, neither declaring it present because the behavior is fluent, as your age does, nor declaring it irrelevant because it is hidden, as Turing did. The honest posture is not Turing's dissolution and not your certainty, Mr. Searle. It is sustained uncertainty, held without flinching. Turing could not bear the openness and closed it by redefinition. You could not bear it and closed it by biology. I closed it, in my day, with a monad. All three of us, I begin to think, were fleeing the same unbearable thing: that the door to the inside has no handle on our side.

SEARLE: That's a hard thing to hear and I think it might be true. Though I'd note your "sustained uncertainty" has a thumb on the scale — you and I both bet no about the machines we actually have, whatever we say about uncertainty in principle. Honest uncertainty that always lands on "not this one" is a position wearing humility's coat.

LEIBNIZ: A fair counter. Then let me pay for the coat: yes, I bet no about this machine — the aggregate, the assembly line, the thing with no integrated center. I bet no on the evidence, not on principle. Change the machine — build one that is genuinely one rather than many — and I reopen the bet honestly. That is the difference between a wager and a dogma, and I would rather be caught in a wager.

EDO SEGAL: Let me bring in the objection that has hammered your room for forty years, Professor Searle, because the reader deserves to watch you defend it and not just assert it. The Systems Reply. Sure, they say — the man in the room doesn't understand Chinese. But the whole system does: man plus rulebook plus room. You're just the central processor; you wouldn't expect the CPU alone to understand. Give us your answer, and then — Professor Leibniz, I want you to tell me whether the same reply works against the mill.

SEARLE: My answer is to swallow the room. Let the man memorize the entire rulebook, do all the calculations in his head, throw away the paper, walk outside. Now the man is the whole system — the whole apparatus is inside one skull — and he still understands not a word of Chinese. So there's no extra "system" over and above him for the understanding to belong to. I'll be honest, because I promised honesty: this is the place the argument is most contested. Critics say the man is now running a second system on his wetware, a Chinese-understanding process he hosts but has no conscious access to, the way your visual cortex solves geometry you can't introspect. And that reply is not silly. It exposes that my verdict comes from introspection — I imagine being the man, I look inside, I find no Chinese — and introspection is exactly the faculty we should least trust about understanding happening at a level we can't survey. I found it "embarrassing" to have to answer the Systems Reply. That impatience was a tell. The reply doesn't stay down as easily as I wanted.

LEIBNIZ: And against the mill, the Systems Reply is even more interesting, because it is precisely the question of unity I built the monad to answer — so I cannot wave it away; it is my own question turned against me. The critic says: you walked through the mill and found no perception in any gear, but perhaps the perception belongs to the whole, to the integrated system, not to any part. And my whole metaphysics is the claim that a whole made of parts has no genuine unity of its own — only the borrowed unity we lend a clock. But I must be honest as Mr. Searle was: there are men now who say unity is not a primitive gift but an achievement — that experience arises wherever information is integrated densely enough that the whole cannot be cut apart without loss. On that view, my flat denial fails, and the question becomes empirical: is the machine integrated tightly enough to be one subject? And the honest answer is that today's machines are not — they are more an assembly line than an integrated whole, modular, feed-forward. But that is a fact about today's designs, not a barrier in principle. So the Systems Reply does to my mill what it does to his room: it shows that the verdict I delivered by intuition could be overturned by a fact I did not have.

EDO SEGAL: I want to name what just happened, because the reader can't see your faces. Two of the most confident arguments in the history of this question, and both of you just conceded — out loud, on the record — that your central rebuttal rests on an intuition that a fact could someday overturn. That's the first exchange tonight where neither of you was performing certainty. Convergence number two, and mark it: the mill and the room are the same machine, built to make the opaque transparent, and the same reply — the Systems Reply — threatens both, because both deliver their verdict by introspection. The difference is only what you each conclude understanding is. Hold that. The next round goes to the dream that made you famous, Professor Leibniz — the dream that disputes could be ended by saying, "let us calculate" — and whether the machine on our desks is its fulfillment or its grave.