Opening Positions

WITTGENSTEIN: I will begin where every confusion in this subject begins: with a picture we hold without noticing we hold it. The picture is that a word is a name, and that it gets its meaning by standing for something — an object, out in the world, or a private something, inside the head. On this picture, to understand a word is to have got hold of the thing it names. And so, naturally, when the machine produces the word, you ask: but has it got hold of the thing? Is the meaning in there? You feel you are asking a deep question. You are asking a confused one.

I spent thirty years dismantling that picture, and I dismantled it for human beings first, long before any machine. Consider how a child learns the word "slab." A builder calls "Slab!" and the assistant brings a slab. The meaning of "Slab!" is not a mental image of a slab that floats up in the assistant's mind. The meaning is the role the word plays in the work — the response it calls for, the practice it belongs to. The meaning of a word is its use in the language. Not a thing it points at. A thing we do with it. And the doing is woven into a whole fabric of activity — building, fetching, correcting, mattering — that I call a form of life. Strip the form of life away and the word is not a word with a hidden meaning missing. It is a noise.

Now. The strange thing — and I will be honest, because honesty is the only method I trust — is that the machine seems to vindicate me. The men who built it did not read me, and yet they built a device that learns meaning the only way I ever said meaning lives: in use. It is trained on nothing but use — which words follow which, across billions of human sentences — and from that alone it produces language of astonishing aptness. The linguist Firth said it: you shall know a word by the company it keeps. The machine knows every word by its company and nothing else. If meaning were use in my thin sense, the machine would have meaning, and the debate would be over, and I would have lost it to Jerry by accident.

But the use I meant was never the thin one. It was use within a form of life — within the building, the fetching, the suffering, the stakes. The machine has the silhouette of use without the life that casts the silhouette. It plays every language game and inhabits none, because to inhabit a game is to be answerable to it — to be the kind of being that can greet and offend, mean it and fail to, be corrected and not merely retrained. The machine's "good morning" acknowledges no one. It cannot be insincere, because there is no sincerity it is capable of. It has the moves and not the answerability. And answerability, I want to say loudly so Jerry cannot mistake me for a romantic, is not an inner glow. It is a public feature of the practice. The machine lacks it not because its private box is empty but because it does not stand in the practice at all.

So my opening position is this. Do not ask what is inside the machine. That question imports the whole bad picture. Ask instead: does it share our form of life? Is it answerable, as we are, to the world its words are about? And the answer, today, is no — not because we peered into its soul and found nothing, but because it has the language torn up from the roots that, in us, let language reach the world. It can say everything and show nothing. That is my case. I expect Jerry to tell me I have described the surface beautifully and explained nothing. He will be wrong, but he will say it well.

EDO SEGAL: Jerry.

FODOR: That was, genuinely, beautiful, and I want the audience to notice that I mean it, because what I'm about to do is take it apart. Ludwig has just described, with more grace than anyone, exactly the thing my entire career was a war against — and exactly the thing that, if you believe it, makes you helpless in front of this machine.

Here is where I start. There is a fact about thought so obvious that we forget it needs explaining: thought is systematic. Anyone who can think John loves Mary can think Mary loves John. You never, ever meet a mind that has the first thought and is constitutionally incapable of the second. And thought is productive: from a finite head you generate endlessly many new thoughts you have never had before. Why? There is only one explanation anyone has ever produced that isn't magic. Thoughts have parts — constituents — and the parts recombine by rules. Thought is built like a language: a stock of symbols, a syntax that combines them. That is the Language of Thought, Mentalese, and I did not propose it because I liked it. I proposed it because it is the only game in town.

And here is the genius of the symbol, the thing Ludwig's "use in a form of life" cannot touch. A symbol has two faces at once. It has a shape — a form a mechanism can grab and manipulate without knowing what it means. And it has a meaning — a thing in the world it is about. A computer pushes the shapes around, and if you have built the syntax right, the meanings take care of themselves: true premises march to true conclusions because the syntax was designed to track the semantics. That is how a lump of matter can reason. That is the only solution anyone has ever found to the deepest puzzle there is — how thinking can be at once a physical process and a rational one. Ludwig wants to dissolve the puzzle by staring at the surface. I want to solve it, and I claim I have the only solution on the table.

Now to the machine, and I'll surprise you: I am not the booster here. The machine has no symbols in my sense. It has vectors — long lists of numbers, meaning smeared across a continuous space, no discrete cat-symbol that shows up self-identical in every thought about cats. It is, in its bones, the connectionist architecture I spent thirty years arguing could not, in principle, support real thought. So when it produces systematic, compositional language anyway — and it does, in exactly the way the literature calls emergent capabilities — there are exactly two possibilities, and Ludwig's framework cannot even state them, which is its poverty. Either there is structured thought in there, implemented in a format I didn't foresee, in which case I was wrong about the format and right that structure is what matters. Or there is no structured thought in there, and it is producing the products of thought without the thing — a perfect statistical echo of a mind, which would mean it understands nothing, in the strict and only sense of "understand" I care about, namely: manipulating representations that actually mean something.

A recording of a violin is not a violin. The machine was trained to produce exactly the outputs a thinker would produce, because those outputs are its training data. So the outputs cannot settle the question. You have to look inside. And Ludwig's whole philosophy is a sustained argument that you must never look inside — which is why, of the two of us, he is the one who has disarmed himself in front of the most important machine ever built. That's my opening.

EDO SEGAL: Before I frame the rounds, one discipline I impose on every long conversation, because it pays off two hours later. Each of you, briefly: what do you envy in the other's position? Not respect. Envy. The thing his side gets to have that yours does not. Jerry, you first this time.

FODOR: I'll answer honestly, which is rarer than you'd think at this table. I envy that Ludwig gets to be done. His method ends in peace — you dissolve the confusion, the fly is out of the bottle, you stop. He gets to say "there is no problem here, only a knot in your language," and walk into the garden. I don't get peace. I'm committed to a mechanism, and a mechanism owes the world an account that can be wrong, that has to be built, that can fail tomorrow when the next model ships. He gets to dissolve the question. I have to answer it, and the answer can come back stamped REJECTED. That's a harder way to live, and some mornings I envy the man who gets to put the ladder down and be quiet.

WITTGENSTEIN: And I envy that Jerry gets to hope for an answer. He believes there is a fact of the matter inside the machine, waiting, and that one day the interpretability people will open it and we will know. I do not get to believe that. My discipline forbids it. I have to keep saying the harder, lonelier thing — that the question "is there understanding inside?" may have no determinate answer at all, that we may simply be holding a word, "understand," up against a thing it was never shaped to fit, and that no peering inside will ever settle what was never a matter of an inside. He gets the consolation of a hidden fact. I get only the work of clarity, which never ends and never comforts. There are nights I would trade.

FODOR: That may be the most honest thing either of us says tonight.

WITTGENSTEIN: Do not get used to agreeing with me. It will not last.

EDO SEGAL: Two openings and two envies, and you can already see the architecture of the evening. It is not that one of you loves the machine and one fears it — you'd both tell me the hype is poison. It is that you locate the question in opposite places. Jerry says: the truth is inside, go and look. Ludwig says: there is no inside to look in, watch how it lives. Hold both. We start the rounds at the exact seam — the sentence the men who built the machine never read, and accidentally engraved in silicon. Meaning is use. After the break.