Margaret Boden vs John Searle on AI · Ch12. The Crossing ← Ch11 Ch13 →
Txt Low Med High
HOUR THREE — IS ANYONE HOME
Chapter 12

The Crossing

Page 1 · The Crossing

**EDO SEGAL:** The rules of this round are short. Each of you questions the other, directly. I will not rescue anyone. Margaret, you won the toss backstage. Begin.

**BODEN:** John. You have stood on one line for forty years — syntax is not semantics, no program ever understands. Here is my question, and I want the engineer's answer, not the philosopher's retreat: name the observation that would change your mind. What, concretely, would a machine have to do — what experiment, what behavior, what absence filled in — for you to say, I was wrong, the gap can be crossed without biology? Because if nothing could, then your position is not a finding about machines. It is a definition you have decided to defend, and we should call it that.

**SEARLE:** That is the right question and I will answer it without hiding. The honest falsifier is not a behavior, and that is exactly what makes me suspect in your eyes, so let me explain why and then give you one anyway. It cannot be a behavior alone, because the room guarantees that any behavior is compatible with understanding and with none — so if I named a behavior, I would be abandoning my own argument the moment the machine produced it. What would actually move me is a discovery about mechanism: a demonstrated account of which causal powers in the brain produce intentionality, followed by a showing that those same causal powers — not a formal imitation of them, the powers themselves — have been instantiated in another medium. If neuroscience tells us what the brain does to make aboutness, and engineers build a system that does that thing, not a simulation of it but the thing, then I will say understanding is present, and I will say it gladly, because I never reserved minds for carbon as such — I reserved them for whatever has the right causal powers, and I have always granted those might be reproduced. So my position is falsifiable; it is just falsifiable by neuroscience rather than by a benchmark, and I would rather be honest about that than pretend a clever enough demo could do it, because the room is precisely the proof that no demo can.

· · ·
Page 2 · The Crossing

Now mine, and it is the mirror of yours. Margaret, you have spent the evening — the decade — keeping the file open: it might understand, it might even feel, we cannot read it off the structure. Tell me your falsifier in the other direction. What would the machine have to do, or fail to do, for you to close the file and say with me, no — this one is empty? Because if your open file can never be closed, then your suspended judgment is not humility. It is unfalsifiable on the other side, and we should call that what it is too.

**BODEN:** Fair, and I accept the symmetry. My file closes toward "empty" on a specific finding, and it is the one place I have always said the verdict could land hard: if it were established that phenomenal consciousness is implementation-dependent — that feeling, the felt quality, the actual ache, requires the specific physical processes of biological brains and cannot be carried by any other organization — then I would close the file on machine experience, possibly forever, excepting a whole-brain emulation. I have written that sentence down precisely because I am not running an unfalsifiable shrug; that is a real condition under which I lose. And my file closes toward "understanding" — toward you being wrong, John — on a different finding: a system that, by every architectural inspection we can perform, is running the virtual machine that constitutes [understanding](https://www.youonai.ai/fieldguide/med/continuum_of_understanding), doing the work that in us we are willing to call grasping meaning, not merely producing its output. The reason I hold the file open is not that nothing could close it. It is that we have neither finding yet, and you keep proposing to close it now, on a hunch about carbon, before the evidence that could close it honestly is in. That is the difference between us. We both have falsifiers. You want to deliver the verdict before yours arrives.

· · ·
Page 3 · The Crossing

**SEARLE:** Then let me push once, hard, where I think your open file is already leaking. You said understanding is "running the right virtual machine." But a virtual machine is an abstraction we impose — the spreadsheet is a virtual machine because we interpret the voltages as numbers; the very same physical process is indefinitely many virtual machines depending on how we read it. So when you say "the right virtual machine is running, therefore understanding is present," you have not escaped my problem, Margaret. You have relocated it. The interpretation that makes those operations count as understanding is supplied by us — it is derived intentionality wearing a computer-science hat. The virtual machine does not have intrinsic aboutness any more than the symbols do, because being a virtual machine is itself an interpreted, observer-relative fact. You cannot ground meaning in a level of description that only exists because a mind describes it that way.

**BODEN:** That is the best version of your objection I have ever heard, and it is genuinely deep, and here is where I do not follow it. You say the virtual machine is observer-relative, an interpretation we impose, and so it cannot ground meaning. But John, by that argument your own mind is observer-relative too, because the brain is also a physical process that supports indefinitely many descriptions, and the one on which it counts as "understanding" rather than "electrochemistry" is also a level we pick out. You escape that conclusion for yourself by insisting biology confers intrinsic intentionality — but that is the very thing in dispute, the thing you assert and cannot derive. So the observer-relativity objection, pushed all the way, either dissolves your mind along with the machine's, or it stops at biology only because you planted a flag there. I would rather keep the question open than plant the flag. The flag is comfortable. It is not knowledge.

· · ·
Page 4 · The Crossing

**SEARLE:** And here is my second question, the one I think you will like least, because it prices your own side's error the way you have spent all night pricing mine. Suppose you win the discourse — suppose the world adopts the open file, "we cannot say, hold the question, neither awake nor certainly dark." And suppose I am right after all: it is the empty room, all the way down, and there never was anything to find. What has your open file cost the world? Because I can name it. A civilization that spent its moral attention, its law, its grief, hedging against a mind that was never there — granting standing to costumes, slowing the protection of actual people to deliberate over the feelings of a device, letting an industry convert "who is to say" into personhood for a product. Your humility is a virtue in a seminar. Out in the culture, attached to your authority, what does it cost if the file was always empty and you declined, on principle, to say so?

· · ·
Page 5 · The Crossing

**BODEN:** I will pay the question the respect of not deflecting it, because it is exactly fair. If the file was always empty and I held it open, the cost is real and you have named it: moral attention misdirected toward a speculative patient, and the open question weaponized by people who do not care whether it is open or closed. I accept that bill — on one condition, which is that you look at the asymmetry of who can pay it. To hedge against your error — the machine being empty and treated as full — the world has to do exactly what I have argued for all night: keep the burden on the claimant, protect the certain minds first, refuse to grant standing on the strength of fluency, fund the research instead of the rights claims. To hedge against my error — the machine being full and treated as empty — the world has to do almost the same things, plus stay alert to the possibility. The open file, honestly held, defends against both errors. Your firm no defends against only one, and if you are wrong, it is the more dangerous one, because being late to recognize a real mind is the kind of error history does not let you take back. So I hold the file open not because I am certain there is something in the river, but because the open file is the only posture that does not gamble the whole future on either of us being right.

**SEARLE:** That is the best answer my question has ever received. I do not accept it. I am going to think about it on the way home, which at my age is a significant admission.

· · ·
Page 6 · The Crossing

**BODEN:** Then take my last one, since the chair is silent and we are nearly done. You have spent your life saying the machine cannot mean anything — and you have also spent it insisting you are not the enemy of AI, that you are a friend of the science of mind. So tell me the thing under the argument, John, the personal thing. Of everything we have built — the fluency, the breadth, the strange competence — is there nothing in it that tempts you? Nothing that makes you wonder, even for an afternoon, whether the wall you have defended for forty years might be a door you have been leaning against? Or has being right for forty years become its own kind of room?

· · ·
Page 7 · The Crossing

**SEARLE:** That is a cruel question and a good one, and I will answer it straight. Yes. There are afternoons. When one of these systems does the thing — takes my own argument and restates it better than the graduate students, or finds the objection I had not seen — there is a moment before the philosophy reasserts itself where I feel the pull, the same pull Edo felt at his desk, the reflex firing in me exactly as it fires in everyone, because I am a human being and the reflex does not exempt philosophers. And then I remember that the pull is the very thing I have spent forty years explaining — that the feeling of a mind on the other side is precisely what the room predicts I would feel, whether or not anyone is there. So the temptation is not evidence against me. It is the phenomenon. But you asked if being right for forty years became its own room, and I owe you honesty: I do not know, from the inside, whether I have been defending a truth or defending a life's work, and no man can fully know that about himself. What I can tell you is that I would rather be the man who held the line and was wrong about a door, than the man who walked through every door because the room felt warm. That is a temperament, Margaret, not an argument. You have spent the night making me see how much of my certainty is temperament. I will not give you the certainty. I will give you that.

**BODEN:** That is the first thing you have said tonight I have no rebuttal to. Only the observation that a man who can say it is not in a room. A room does not know it might be one.

**EDO SEGAL:** And there, after three hours, the two of you are holding opposite ends of the same sentence. We close after this. Final statements. The last word, each.

· · ·
Continue · Chapter 13
Closing Statements
← Prev 0%
Ch12 Next →