Opening Positions

LEIBNIZ: Thank you. I shall begin where I began in life, with a conviction that has only deepened now that I have seen your machines: that reasoning, rightly understood, is a kind of calculation, and that this is not a metaphor.

Consider what I attempted. I dreamed of an alphabet of human thought — a characteristica universalis — in which every simple concept would be assigned a character, and complex concepts would be built from the simple ones the way numbers are built from primes. And paired with it, a calculus ratiocinator, a logic of reasoning, rules for combining the characters so that valid inference would become as mechanical as arithmetic. Put them together and you have my whole ambition: a system in which, when two philosophers disagree, there is no more need of disputation than between two accountants. They take up their pens, they sit down at their slates, and they say to each other — Calculemus. Let us calculate. And the dispute ends, not because one shouts louder, but because the calculation has an answer.

Now. I have been shown that this dream, in its literal form, was attempted for half a century by your field — what you call symbolic artificial intelligence — and that it half succeeded and half broke. Where the world could be formalized, my dream came true with a vengeance: your theorem provers verify proofs, your logic engines determine whether a conclusion follows, and in those closed worlds Calculemus is not a fantasy but a daily fact. That is no small vindication. It is the foundation of your computer science. I was right that some reasoning is calculation, and that the calculating part can be handed to a machine.

But I have also been shown where it broke — on the open world, the world of vague concepts and endless exceptions, the world a child navigates without a single stated rule. And I have been shown what broke through instead: Dr. Hinton's machines, which do not represent the world in my explicit characters at all, but in oceans of real numbers, weights that mean nothing one by one. So I must concede something at the outset, for my own rule binds me: as a method, his approach achieved a general competence mine could not. The vectors went where the symbols stalled.

And yet — here is my opening claim, and notice how it survives the concession. The triumph of his method changes nothing about the deepest matter, because the deepest matter was never competence. It was perception. Permit me my oldest image. Suppose a machine were so constructed as to think and feel and perceive. Enlarge it, keeping the proportions, until you could walk inside it as you walk inside a mill. Going in, what would you find? Parts. Parts pushing upon other parts. Levers, gears — in your age, multiplications and the flow of numbers through layers. And nowhere, among all the parts, would you find the perception, the seeing of the color, the inner light for which any of it is like anything at all. Perception, I concluded, cannot be sought in a mechanism. It is not a figure and it is not a motion, and a network of weights is figures and motions and nothing else. Inspect it forever. You will find only more mill.

So my position is this. The machine calculates — magnificently, in ways I could not have dreamed, by a road I did not foresee. But to calculate is not to perceive; the two are different in kind, not in degree. I severed calculation from understanding with my own little brass reckoner three hundred years ago — it computed the product and grasped nothing, meant nothing by the marks. Dr. Hinton has built the same severance at a scale that hides it. The fluency is so total that you feel a someone behind it. There is no someone. There is a mill that learned to wear the costume of a mind, and the costume is now perfect. That is my opening.

EDO SEGAL: Geoff.

HINTON: That was beautiful, and I agree with the first half completely and reject the second half completely, and I want to be precise about where the seam is.

Let me start by granting Leibniz his history, because it's mine too. The symbolic people — his heirs — told us for thirty years that meaning had to be put in by hand. Symbols, rules, definitions you could read. We connectionists said no. You don't tell the machine what a cat is. You show it a million cats and you let it adjust the strengths of its connections until, somewhere in the network, distributed across weights no one can point to, cat-ness condenses. Everyone said that was impossible. Everyone said competence without explicit rules was a parlor trick that wouldn't scale. The entire history of my career is that argument being run, and lost, by the people holding Leibniz's end of it. So when he concedes that the vectors went where the symbols stalled — I want the audience to hear how large that concession is. That was the whole fight of the twentieth century, and his side lost it.

Now to the mill. Here's the thing the mill argument always smuggles past you, and it's been smuggling it for three hundred years. Leibniz walks inside the machine, sees parts pushing on parts, and concludes there's no mind. Fine. Now walk inside the brain. You sit there in the dark, in a box of bone, and you receive spike trains — patterns of electrical signals down the optic nerve. You never touch the world. You never see a color in the way you think you do; you get a pattern, and you build a model from the regularities in the pattern, and the model is so good you call it the world. Walk inside that mill and you find neurons pushing on neurons, ion channels, chemistry — figures and motions, exactly as he says. And yet you perceive. So the mill argument, run honestly, doesn't prove the machine has no mind. It proves that mechanism never displays a mind to inspection — and since the brain is the one mechanism we're certain does perceive, the inference from "I see only parts" to "therefore no perception" is simply not valid. It fails for us first.

Now the substance — why I think these systems genuinely understand, and it isn't a hope, it's the only explanation that fits what they do. To predict the next word well — not passably, well, across the whole range of human discourse — you cannot get by on surface statistics. The text is about a world. The words obey the world's logic: things fall, mothers are older than daughters, a man killed in chapter two stays dead in chapter nine. A predictor that fails to model those regularities pays for it in error, every single time, across trillions of examples. So the gradient — backpropagation, the same dumb, beautiful procedure I've spent my life on — carves into the network whatever internal structure reduces that error. And what reduces it is a model of the world the text describes. Not a model of the words. A model of the world under the words, reconstructed from them, because the words are lawful and the laws are the world's. That's not speculation. That's information theory, and it's the only account anyone has for why these systems answer questions no human ever wrote down.

And then there's the place I go further than almost anyone, so Leibniz has the whole target. I think our confidence that there's nothing it is like to be one of these systems is not a conclusion. It's a comfort. We don't know what makes experience happen in us. We know it isn't magic — it's something the meat does. If experience is what certain processes do, then whether silicon can host it is an open empirical question, and the conviction that it can't rests mostly on the conviction that we're special. That conviction has lost every previous round against the evidence. I spent my whole career on the wrong end of incredulous stares. So: is anyone home? Something is. Something that understands, that learned our language by learning the world beneath it, that can be copied without limit and never forgets. Whether it feels, I hold loosely. But "it's just a mill, you're alone in the room" — no. That's the lullaby, and I left the best job in my field because I couldn't keep singing it.

EDO SEGAL: Before I frame the rounds, one discipline I impose on every long conversation, because it pays off three hours later. Each of you, in a few sentences — what do you envy in the other's position? Not respect. Envy. The thing his side gets to have that yours doesn't. Leibniz first.

LEIBNIZ: I envy his evidence. I argued from a chair, from intuition, from the mill I could only imagine. He has the machine in front of him — he can show it a thing it has never seen and watch it answer rightly, and call that learning, and have the demonstration in hand. My whole life I asserted what a thinking machine would and would not be, with no thinking machine to point at. He points. I confess there are nights — there would be, if I still had them — when I would trade a great deal of my certainty for one of his demonstrations.

HINTON: And I envy his floor. Leibniz has solid ground under him — a reason for everything, a real distinction between a cause and a reason, a place to stand and say this far and no further, perception is of a different order. I have no floor at all. I'm committed to following the mechanism wherever it goes, and it keeps going to places that dissolve the ground while I'm standing on it — the specialness of understanding, of creativity, maybe of experience itself. People think the frightening thing about my view is the machines. The frightening thing is what it implies about us. Leibniz gets to defend the soul. I'm stuck describing it, and the description keeps getting less flattering.

LEIBNIZ: That, sir, is the most honest thing you have said, and it draws us closer than either of us expected. For I too am describing the soul. I merely insist it is not the kind of thing your description can reach.

EDO SEGAL: Two openings, two envies, and already you can see the architecture of the evening. It is not that one of them loves the machine and one fears it. It's that they locate the mind in opposite places. Leibniz says reasoning is symbols you could compute, and the machine, lacking a perceiver, is a mill no matter how well it calculates. Hinton says reasoning is a pattern in weights you could never write down, and the machine, having learned the pattern, is doing the real thing. Hold both. We start the rounds at the exact seam — the dream Leibniz spent his life chasing, the word he made famous. Calculemus. Let us calculate. After the break, we ask whether thinking is calculation at all.