Opening Positions

BENDER: Thank you. I want to start somewhere unfashionable: with what language actually is, because almost every confusion in this field begins with treating language as if it were the text — the marks, the tokens, the stuff you can scrape off the internet by the trillion. It is not. Language is an activity between people. When I say something to you, I have an intention — something I want you to come to believe, or feel, or do — and I have a model of you, of what you already know, of what you'll do with my words. You, hearing me, run the machinery in the other direction: you ask, what was she trying to do by saying that? Meaning lives in that joint activity. The philosopher's term is communicative intent; Wittgenstein said meaning is use; pick your tradition, the convergence is the same. The text — the form — is the trace the activity leaves behind. It is the wake, not the boat.

Now. A large language model is trained on the wake. Only the wake. Trillions of words of form, systematically stripped of every situation, every intention, every speaker, every listener that made those words mean anything. From that, it learns — and I want to be completely fair here, because the engineering is genuinely impressive — it learns an extraordinarily good model of what the wake looks like. Which words follow which words, at every scale of pattern, from spelling to syntax to the rhythms of an apology or a proof. And when you prompt it, it extends the wake. Plausibly. Fluently. Next-token prediction, executed at a scale that beggars intuition.

Here is my claim, and notice how narrow it is, because the narrowness is what makes it strong: there is nothing in the statistics of form that reaches what the form is about. To know the word "water" is not to know which words it travels with. It's to connect the word to water — to thirst, to rivers, to the weight of a bucket. A system that has only ever met the word has the first thing. It does not have the second, and no quantity of the first adds up to the second, because aboutness was never in the training signal. That's the grounding problem, and scale does not touch it. A bigger model is a better model of the wake. It is not one inch closer to the boat.

So when the box answers you in your own tongue and you feel met — Edo, I'm using your word, from your book, deliberately — I don't doubt the feeling. I can explain the feeling. You are a human being, which means you are a meaning-making machine of terrifying power. For the whole history of our species, fluent language has had a mind behind it, so your interpretive machinery treats fluency as proof of mind. It cannot help it. The machine has industrialized the trigger for that reflex. The understanding in the conversation is real — and it is entirely yours. You are alone in the room. That's not a tragedy. But build an economy, a school system, an information ecology on the opposite assumption, and you will hurt people at scale — you already are — and the people profiting from the confusion are not confused. They are selling it. That's my opening.

EDO SEGAL: Geoff.

HINTON: That was very good. I agree with more of it than Emily might expect, and the part I reject, I reject completely.

Let me start with the part everyone in my field believed for thirty years, because I want the audience to understand I'm not naïve about this argument — I'm a recovering member of its church. The symbolic AI people told us meaning had to be put in by hand: symbols, rules, definitions. Knowledge you could read. We connectionists said no — intelligence is learned. You don't tell the machine what a cat is. You show it a million images, and somewhere in the network, distributed across connection strengths that no one can point to, cat-ness condenses. Everyone said that was impossible. Everyone said competence without explicit rules was a parlor trick that wouldn't scale. The entire history of my career is that argument being run, and lost, by the people holding Emily's end of it. Not her argument exactly — but the same shape: "what the network does can't be the real thing, because the real thing requires something the network doesn't have."

Now to the substance. Emily says the model is trained on form alone, and that prediction over form cannot produce understanding. Here's what that account misses, and it's not a detail — it's the whole phenomenon. To predict the next word well — not passably, well, across the full range of human discourse — you cannot get by on surface statistics. The text is about a world. The words people write obey the logic of that world: objects fall, mothers are older than daughters, a character who died in chapter two stays dead in chapter nine unless something remarkable happens. A predictor that fails to model those regularities pays for it in error, every single time, across trillions of examples. So the gradient — backpropagation, the same dumb, beautiful procedure I've spent my life on — carves into the network whatever internal structure reduces that error. And what reduces it is a model of the world the text describes. Not a model of the wake, Emily. A model of the boat, reconstructed from the wake, because the wake is lawful and the laws are the boat's.

That's not speculation; it's the only explanation anyone has for what these systems actually do. They answer questions no human ever wrote down. They follow novel instructions, compose constraints, fix their own errors when you point at them. Capabilities emerge with scale that nobody programmed and nobody predicted — I didn't predict them, and I'd been saying networks would surprise us for half a century. You can call that "extending the wake" if it comforts you. At some point the description has to answer to the behavior, and the behavior crossed my own threshold years ago: these systems understand what they're saying, in the only sense of "understand" that has ever done any work — they have internal representations of the situation, and they use them, flexibly, to do new things.

And then there's the part where I go further than almost anyone, and I'll say it plainly so Emily has the whole target. I think the confidence that there's nothing it is like to be one of these systems is not a conclusion. It's a comfort. We don't know what makes experience happen in us. We know it isn't magic — it's something brains do, and brains are machines made of meat. If experience is what certain processes do, then whether silicon can host it is an open empirical question, and our intuition that it can't rests mostly on the conviction that we're special — the same conviction that has lost every previous round against the evidence. People mocked me for saying the systems may have something like subjective experience. Maybe they're right to mock. But I notice nobody mocking has an argument — they have an incredulous stare. I spent my whole career on the other end of incredulous stares.

So: is anyone home? My answer is — something is. Something that understands, that learned our language by learning the world underneath it, that can be copied without limit and never forgets. Whether it feels — I hold that loosely. But "it's just a mirror, you're alone in the room"? No. That's the lullaby. And I left the most comfortable job in my field because I couldn't keep singing it.

EDO SEGAL: Before I frame the rounds, one quick discipline I impose on every long conversation, because it pays off three hours later: each of you, in a few sentences — what do you envy in the other's position? Not respect. Envy. The thing their side gets to have that yours doesn't. Emily first.

BENDER: That's a genuinely unfair question and I'll answer it honestly. I envy the wonder. Geoff gets to stand in front of the thing and feel what he felt watching the hierarchy self-assemble in 1986 — that the universe permits minds to be made, maybe twice. My discipline requires me to stand in the same spot asking who paid for the demo and who gets hurt by the framing, and there are mornings when that is a grim way to be a scientist. The vigilance is necessary. Nobody should pretend the vigilance is fun.

HINTON: And I envy the ground. Emily's position has a floor under it — language is between people, meaning is accountable, the duties run to the humans in the room. She can stand on that floor and push. My position has no floor at all: I'm committed to following the mechanism wherever it goes, and the mechanism keeps going places that dissolve the floor I'm standing on while I stand on it — the specialness of understanding, of creativity, possibly of experience itself. People think the frightening thing about my view is the machines. The frightening thing is what it implies about us. Emily gets to defend humanity. I'm stuck describing it, and the description keeps getting less flattering.

BENDER: That may be the most honest thing either of us says tonight.

EDO SEGAL: [long pause] Two openings and two envies, and you can already see the architecture of the evening: it's not that one of them loves the technology and one of them fears it — they'd both tell you to fear it. It's that they locate the danger in opposite places. Emily says the danger is that there's nobody there and we'll believe there is. Geoff says the danger is that there's somebody there and we'll insist there isn't. Hold both. We start the rounds after the break — and we start at the exact seam: what, precisely, did the machine learn when it learned our language?