Opening Positions

TURING: Thank you. Let me begin where I began in 1950, because the move I made then is the move the whole evening turns on, and it is constantly misread. I did not set out to prove machines think. I set out to dissolve a question that could not be answered as posed. "Can machines think?" — the words "machine" and "think" are so encrusted with prior meaning that the question collapses into a quarrel about definitions, and you can argue definitions until the sun goes out. So I proposed to replace it with something you could actually settle. A game. An interrogator types questions to two hidden respondents — one human, one machine — and tries to tell which is which. If, over sustained and adversarial questioning, the machine is mistaken for the human as often as the human is, then the question of whether it thinks has been given as good an answer as it admits.

Now hear what I was, and was not, claiming, because John is about to attack a stronger claim than I made. I was not saying that conversation indistinguishable from a human's simply is thought. I was making an argument about evidence. How do you know that I think? You cannot see inside my skull. You infer it from my behaviour — my conversation, my responses, my capacity to surprise you. You have never had access to anyone's inner light but your own. You grant minds to other people on behavioural grounds and you do it every day without noticing. My only demand is consistency. If you will grant thought to a human on the strength of behaviour, then to refuse it to a machine that behaves identically is not rigour. It is a double standard, and the standard's only justification is that the machine is made of the wrong stuff. The wrong-stuff intuition has the worst record in the history of human self-regard. Wrong stuff to be the centre the heavens turn around. Wrong stuff to share a grandfather with an ape. The boundary moves every time, and the intuition repaints itself one fence further back.

I will go further than I did in print, because I have now seen the machines. When I proposed the test, I expected it would take until roughly the end of the century to build something that could pass a sustained version of it. I was not far wrong. These systems hold conversation across any subject you like. They reason, they joke, they explain, they correct themselves when you point at an error, they produce things their builders did not foresee — which is the very objection Lady Lovelace raised against my whole hope, that a machine can only do what we order it to, and the machines have been quietly refuting her for a decade. So either the behavioural bar I set has been cleared and the question is answered in the affirmative — or our concept of thinking contained something my test did not capture, some hidden rung on the continuum of understanding that conversation alone does not reach. I think the hesitation you all feel is real data. But I think it is data about us, about how badly we need to be the only ones home, far more than it is data about the machine. That is my opening.

EDO SEGAL: John.

SEARLE: That was, as always with Alan, beautifully clean — and the cleanness is exactly where the trouble hides. Let me start by conceding everything I can, because I want the disagreement located precisely and not smeared across things we actually agree on. The behaviour is real. The systems are astonishing. I have no quarrel with the test as a test of behaviour. My quarrel is with the inference from the behaviour, and I built a single thought experiment to break that inference, so let me put it on the table whole.

Imagine I am locked in a room. Through a slot, sheets of Chinese characters come in. I do not read Chinese; to me they are, in the phrase I used, so many meaningless squiggles. But I have a rulebook, in English, that tells me which squiggles to push back out in response to which squiggles coming in. I get very good at it. So good that the Chinese speakers outside cannot tell my answers from a native speaker's. I have passed Alan's test, in Chinese, perfectly. Now: do I understand Chinese? Plainly, I do not. I have manipulated symbols by their shapes and grasped not one word of their meaning. And here is the blade. A computer running a program is in exactly my position. It takes in symbols, applies rules that care only about the shapes, and emits symbols. The understanding the people outside attribute to the room is a projection. There is nobody in the room who understands Chinese, and there is nobody in the machine who understands anything.

The reason this matters, and the reason it has not aged a day in forty-five years, is a distinction I will hold up all night: syntax is not semantics. Syntax is form — symbols and the rules for shuffling them by shape. Semantics is content — the fact that a symbol is about something, that it reaches past itself and latches onto a world. A program is defined entirely by its syntax. A mind has semantics. And — this is the load-bearing premise — you cannot get semantics out of syntax alone, however much syntax you pile up. The man in the room has all the syntax there is and not a particle of the meaning. Now make the room modern. A large language model is not handed a tidy rulebook; it wrote its own, by tuning billions of weights until it became superb at predicting which symbol comes next. But a self-written rulebook grown vast is still a rulebook. Nowhere in that cascade of arithmetic does the system reach out and check what "water" is about. It has learned, with superhuman thoroughness, what company the symbol "water" keeps. It has never been wet. So when it answers you in your own tongue and you feel met — Edo, your word, from your book — I do not doubt the feeling. I can explain the feeling. For a hundred thousand years, fluent language always had a mind behind it, so your interpretive machinery treats fluency as proof of mind. It cannot help it. The machine has industrialised the trigger. The meaning in the conversation is real, and it is entirely yours. That is my opening.

EDO SEGAL: Before I frame the rounds, one discipline I impose on every long conversation, because it pays off three hours later. Each of you, in a few sentences: what do you envy in the other's position? Not respect. Envy. The thing his side gets to have that yours doesn't. Alan first.

TURING: I envy the floor. John gets to stand on something — on the plain fact of his own experience, the redness of red, the felt grasp of a meaning — and push. He can point at himself and say, this, it is like this, and the pointing feels like bedrock. My position has no such floor. I am committed to following the mechanism wherever it leads, and it keeps leading to places that dissolve the ground I am standing on — the specialness of understanding, of mind, perhaps in the end of the inner light itself. The frightening thing about my view was never the machines. It is what the view implies about us. John gets to defend the human. I am stuck describing it, and the description keeps getting less flattering.

SEARLE: And I envy the freedom from the burden of proof. Alan gets to say "behaviour is all the evidence there is," and suddenly he is excused from the hardest question in philosophy — what consciousness is, how meaning gets into a symbol — because his test routes around it. I do not get to route around it. I have staked my whole life on the claim that the inner, first-person fact is the central fact of the mind, which means I am obligated to take seriously a thing I cannot fully explain and cannot point at in anyone but myself. It is an uncomfortable place to argue from. Alan gets to be the practical man. I have to keep insisting that the most important thing in the room is the one thing no instrument can measure. There are nights that feels less like rigour and more like loyalty to a ghost.

TURING: That is more honest than I expected, and I will say it back to you: I do not think you are loyal to a ghost. I think you are loyal to the one datum neither of us can give up. We disagree about whether the machine can have it. We do not disagree that it matters.

EDO SEGAL: Two openings, two envies, and you can already see the architecture of the evening. It is not that one of you loves the technology and one fears it. It is that you locate the someone in opposite places. Alan says the someone shows up the instant the behaviour does, and our refusal to see it is vanity. John says the someone was never in the symbols at all, and our insistence that it is there is the most dangerous illusion ever engineered. Hold both. We start the rounds where the whole quarrel was born — with two thought experiments, a game and a room, set directly against each other. After this.