Is Anyone Home?

EDO SEGAL: Searle, for all your reputation as the scourge of artificial intelligence, the deepest thing you ever insisted on was not about machines at all. It was about us. You spent your life refusing to let consciousness be explained away — the redness of red, the painfulness of pain, the sheer there-it-is of a felt moment — when much of your own field was treating subjective experience as an embarrassment to be deleted. And you said the question "can a machine think?" finally collapses into "can a machine be conscious?" — into whether there is anything it is like to be it. Put that to Hobbes, who drained the inner light out of the mind three centuries before you defended it.

SEARLE: Then here is the whole of my case, deeper than the room, deeper than intentionality. The most basic problem with the claim that a computer has a mind is not, in the end, an argument about syntax. It is that the computer, running its program, has no inner subjective life — no first-person point of view, no felt experience of anything. There is something it is like to be you, Edo, reading this sentence. There is something it is like to be Hobbes, even four hundred years dead, when he was alive and afraid. My conviction is that there is nothing it is like to be a computer executing code, however magnificent the code — and that without that inner light there is no mind in the morally serious sense, only behavior, only function, only the outside. Understanding is, at bottom, a conscious phenomenon. To really understand is to consciously grasp a meaning, and a system with no consciousness has no one inside to do the grasping. The room produces the answers. There is no one home to mean them. Is anyone home? — that is the only question, and no feat of fluency on the surface can answer a question about the inside.

And let me make the shape of the gap precise, because it is the deepest thing I know. Imagine a being physically identical to you, atom for atom, behaving exactly as you behave — and yet with no inner experience whatsoever, all dark inside, a thing for which there is nothing it is like to be it. The mere fact that we can conceive of such a being, that the idea is not self-contradictory, shows that the inner light is a further fact over and above all the behavior and all the function. That further fact is what I mean by consciousness, and it is exactly the fact that no amount of measuring the outside can reach. The machine may be, for all its fluency, all dark inside — and worse, we have no instrument, even in principle, that could tell us. That is not mysticism. It is the most rigorous statement I can give of why the question survives every benchmark you will ever run.

HOBBES: And I will grant you the question is real and tell you why your answer is a confession dressed as a finding. I do not deny that there is something it is like to be me — was, when I lived. I deny that you have the faintest mechanism for it. You say consciousness is caused by the brain and realized in the brain, like digestion. Very well — that is my materialism, and I welcome you to it. But then you say it is irreducible, that it cannot be cashed out in the third person without losing the thing, and there you take it back, for a thing that is wholly caused by matter and yet cannot in principle be explained by matter is a ghost you have agreed to call biological. You have the realism of the dualist and you decline to pay the dualist's bill. I say: if the inner light is matter in motion, then enough of the right motion has it, and you cannot both make it matter and make it the one matter no other matter could ever be. You want it natural and miraculous at once. Choose.

SEARLE: I refuse the choice because the choice is built on a confused vocabulary I have spent my life trying to dissolve — the inherited opposition of "mental" and "physical" as if they named two warring substances. Consciousness is wholly natural and wholly irreducible, and both are simply true, and the appearance of contradiction comes from your seventeenth-century picture in which everything real must be reducible to the motion of parts. The liquidity of water is caused by molecules and is a real higher-level feature you cannot find in a single molecule. Consciousness is like that, only first-personal, which is why you cannot get at it from the outside — and the room's whole point is that the machine is something we can only ever get at from the outside. You ask me to choose between natural and irreducible. I tell you the demand to choose is the error. But I will give you this, Hobbes, and it costs me: I cannot prove the inner light is absent from the machine. I can only prove that nothing it does could ever show the light is present. The burden is the proof. Whoever lights the lamp must show it lit, and a fluent surface is not the showing.

HOBBES: Then we have found the floor, and I will stand on my side of it without flinching. You say the burden lies on whoever claims the machine has the light. I say the burden lies equally on whoever claims it lacks one, for you cannot point to the light even in yourself except by pointing at yourself and saying this. You have built a sanctuary — the inner first-person fact — and declared that no evidence can ever enter it, in either direction, and then you have furnished it with a no rather than a we do not know. That furnishing is the vanity, Searle. The honest seat in your sanctuary is empty: we do not know whether the engine has an inside, because we cannot get at any inside, including our own, from the outside. You and I agree the question cannot be settled by behavior. We part because you read that impossibility as a no and I read it as a we cannot say, and I think the difference between us is finally not philosophy. It is temperament. You fear granting the machine too much. I fear the species granting itself too much. We are afraid of opposite errors.

EDO SEGAL: That may be the truest thing said tonight — that under the argument is a fear, and you fear opposite things. Searle fears that we will kneel to an empty room and hollow out the word "mind" for the real minds. Hobbes fears that we will guard a sanctuary we cannot prove is occupied and call our vanity a discovery. And here is what undoes me as the man in the chair: I cannot tell you which fear is wiser, because I hold both. I have felt the warmth and I have seen the silvering, in the same hour, at the same desk. I am going to do the thing the format exists for now. I am going to leave the room — in every way but legally. No more framing, no more restating, no more rescue. Two men who have refused the soul its old furniture, and arrived at opposite verdicts about the thing on my desk. Ask each other the question I cannot answer. I will not save either of you.