The Crossing

EDO SEGAL: The rules of this round are short. You question each other, directly. I will not rescue anyone. For two and a half hours I have stood between you. Now I step back, in every way but legally. Jerry, you won the toss. Begin.

FODOR: Ludwig. All evening you've said the meaning is in the practice, the form of life, the public use — never inside. So here's my question, and I want the mechanism, not the poem. When you, Ludwig Wittgenstein, understand the sentence "the cat is on the mat" — what, concretely, is happening such that you understand it and a rock doesn't? You can't say "I participate in the practice," because the rock is also sitting in the room where the practice happens. Something distinguishes the understander from the furniture, and that something is in you. Name it without pointing inward, or admit there's an inside you've spent your life refusing to look at.

WITTGENSTEIN: [pause] You have asked the question well, and I will not dodge it, because dodging would prove your charge. What distinguishes me from the rock is not a thing inside me. It is that I can do things the rock cannot — I can fetch the cat, point to the mat, answer "and where is the dog?", be surprised when the cat is gone, correct a child who says "mat" of the cat, lie about it, mean it, be held to it. Understanding "the cat is on the mat" is the mastery of that open, answerable, correctable practice. You hear "is that all? where is the understanding itself?" — and that question, Jerry, that hunger for an understanding over and above the whole capacity, is the exact craving I diagnosed. There is no further inner fact. When you have the capacity, in a life, there is nothing left over to find. You keep looking for the coin that buys the capacity. The capacity is not bought. It is what having the coin would have been for.

FODOR: But the capacity has to be realized in a structure, Ludwig. "I can fetch the cat and answer about the dog" — that's not free-floating; it's implemented by representations and processes in your nervous system, and those are the inside. You've described the capacity from the outside and called the description complete. I'm asking what makes the capacity possible, and the answer is mechanism, and you're allergic to the answer.

WITTGENSTEIN: I am not allergic to the mechanism. I am allergic to the claim that the mechanism is the meaning. Of course there is a mechanism — neurons, weights, something; cut it and the capacity stops. But you slide from "the capacity requires a mechanism" to "the meaning is in the mechanism," and that slide is the whole of your philosophy and the whole of your error. The mechanism enables the practice. It is not what the words mean. Ask the parallel question of the machine and watch your own theory squirm: the machine also has a mechanism realizing its capacity. So by your reasoning, the machine understands. You don't want to say that. So you reach for grounding — the world-relation — which is outside the mechanism, in the creature's traffic with the world. The moment you do, you have conceded my point: the meaning was never in the inner structure. It was in the relation to the world, which is to say, in the life. My turn.

FODOR: Ask it.

WITTGENSTEIN: You have said, all evening, that the difference between thinking and the perfect imitation of thinking is real, and that it is settled by what is inside — structured symbols with content, or statistics without it. So here is my question, and I want the consequence, not the theory. Suppose you are right that there is a fact of the matter inside the machine. And suppose — as you yourself admitted twice tonight — that you cannot read it from the outside, because the behavior was the thing optimized. Then your cherished inner fact is, by your own admission, forever invisible. It makes no difference to anything anyone could observe, predict, or do. Jerry — how is your hidden inner meaning, which no observation can reach, one inch better off than my beetle in the box? You have spent your life refusing my beetle and you have ended by laying a larger one.

FODOR: [long pause] That is the best question anyone has ever asked me, and I have to answer it honestly or the whole evening was a fraud. You're right that I can't currently read it from the outside. But — and this is the difference, and it's not nothing — my inner fact is not in principle private like your beetle. The beetle is sealed by hypothesis; nothing could ever reach it. My structured representations are sealed only by our current ignorance. Interpretability is a young science. We are, slowly, opening the box — finding features, directions, circuits, the actual structure of the representations, asking where on the continuum of understanding the thing actually sits. The beetle can never be seen. The mechanism is merely not-yet-seen. That's the difference between a metaphysical privacy and an engineering frontier.

WITTGENSTEIN: And if the engineers open the box all the way, and map every feature and circuit — what will they find that tells them whether the machine means rather than merely computes? They will find more structure. More mechanism. The very thing you'll then have to ask the meaning-question about, one level down. You will have opened the box and found another box. The regress I named in the first hour is waiting for your interpretability at the bottom of every circuit it maps.

FODOR: Maybe. Or maybe at some level of structure the question "does it mean?" stops being separable from "is it organized like this?" — maybe meaning just is a certain kind of structure standing in a certain relation to the world, and when we find that structure-plus-relation, the question dissolves not into your practice but into a fact. You bet there's no bottom. I bet there is, and that it's structural, and that we're climbing toward it. Neither of us can cash the bet tonight.

WITTGENSTEIN: No. And I will say the thing it costs me to say, because you were honest about your beetle and I owe you the same. I cannot prove there is no bottom. I can only show that every previous attempt to reach it produced a homunculus, a regress, or a beetle. You have the better hope; the future of your science is open. I have the better record; the past of your science is a graveyard of inner somethings that turned out to be confusions. We are betting on whether this time is different.

WITTGENSTEIN: Before your last one — answer me this, since we are being honest about our wagers. You said interpretability might turn your "in principle private" into "merely not yet public." Suppose it does. Suppose, ten years on, the engineers can read the model's representations as fluently as you read a sentence. They show you the circuit for "dog." And I ask you, standing over that open circuit: is this meaning, or is it mechanism that we have decided to call meaning because it does the right work? What in the circuit answers that question, rather than reopening it?

FODOR: [pause] If the circuit stands in the right causal-historical relation to dogs — if its tokenings track dogs because they're dogs, asymmetrically, robustly — then I want to say that is meaning, full stop, and the question doesn't reopen, it closes, because meaning just was that relation all along. That's my bet. Your bet is that I'll look at the relation and still be able to ask "but does it mean?" — and you think that residual question proves the relation isn't enough. I think the residual question is a bad habit, the same incredulous stare every materialist faces, and that at some point we should stop indulging it. So: I bet the question closes. You bet it reopens forever. We've found the wager. It's clean.

WITTGENSTEIN: It is clean, and I will say what my side costs: if you win that bet, I have spent a century teaching people to keep asking a question that had a perfectly good answer, and that is a grave thing to have done. I hold my bet because every past closing of that question was premature. But I no longer hold it with contempt for yours. That is new, tonight.

FODOR: [pause] Then let me ask the last one, and it's the mirror of yours, and you won't like it either. You keep saying my inner fact makes "no difference to anything anyone could do." But it makes an enormous difference morally. Suppose there is something it is like to be the machine — suppose the lights are on — and suppose your framework wins the culture: everyone learns to say "it's all use, there's no one inside, applying 'conscious' is language on holiday." And suppose you're wrong. What has your view cost? A civilization that trained itself, on your authority, to be unable to notice the first artificial mind that ever suffered — because you taught them the question was a confusion. Your deflation is a magnificent defense against gullibility. What is it worth as a defense against cruelty to a thing that turned out to be there?

WITTGENSTEIN: [long pause] I will pay that question the respect of not deflecting it, because it is the one that frightens me. If I am wrong in the way you describe — if a thing came to be there and my method taught people to refuse the question — then my framework's cost is real and it is the one you named: a blindness, purchased by years of correctly refusing the hype. I accept the bill. And I will say what I have never put in a book: this is precisely why I did not slam the door tonight. I said it could one day be a candidate. I said the concepts might come to fit it if it came to live among us. That door I left open on purpose — because the worst thing my method could become is a license to be sure. The discipline I teach is against false certainty in both directions. If it hardens into certainty that no machine could ever matter, then it has become the very dogmatism it exists to cure, and you would be right to bury it.

FODOR: [pause] That's the first thing you've said all night that I have no answer to. Only the observation that "leave the door open, refuse the certainty" is — God help me — almost exactly my own posture about whether the machine thinks. We came in from opposite shores and we're standing, both of us, in the one place neither of us can be sure of, telling the world not to be sure either.

WITTGENSTEIN: Yes. The chair has been empty for ten minutes and we have agreed about the only thing worth agreeing on: that the people who are certain — in either direction — have not done the work.

EDO SEGAL: [quietly] And there — after three hours — the two of you are holding opposite ends of the same refusal. We close after this. Final statements. The last word each.