The Child and the Background

EDO SEGAL: Alan, in 1950 you wrote the most prophetic paragraph in the whole field and most people skip it for the imitation game. You said: do not try to program an adult mind, writing down all its knowledge by hand. Build something like a child's mind and teach it. Why try to produce the finished intellect by design when nature produces intellects by growth? That is, line for line, how these systems were actually built — not programmed but trained, raised on data, shaped by feedback. So I want you to claim your prophecy. And John, I want you to bring the thing you say no amount of raising can install — the Background.

TURING: I will claim it, because the accuracy still startles me. I proposed a child-machine subjected to a process of education — rewards and punishments, signals reinforcing some behaviours and discouraging others — and I compared it explicitly to evolution: the structure playing the role of inherited material, the changes during education the role of mutation, the teacher's judgement the role of selection. I even said a dose of randomness would help, to explore what a systematic search would miss. The systems that now unsettle everyone are exactly that. They begin as undifferentiated networks — I sketched those too, in 1948, unorganised machines, random tangles of simple units with modifiable connections, which the world later renamed neural networks and acted as though it were new — and they are organised by training until they acquire competence no engineer wrote down — the capabilities emerging with scale that nobody installed and nobody quite predicted. And here is the part I want on the record, because it answers a great deal: a machine that is taught rather than programmed ends up with knowledge its makers cannot read, distributed through its structure in forms no one designed. I said in 1950 that the price of a machine that learns would be a machine we do not fully understand. That opacity is not a bug the field will fix. It is the necessary cost of growth over specification, and it is why the systems surprise even the people who train them.

SEARLE: And I will say, with genuine appreciation, that the child-machine is your finest idea and that learning succeeded exactly where you said it would and symbolic AI failed exactly where I and Hubert Dreyfus said it would. So let me bring the Background and show you both why it cuts the way it does. Every meaningful thing you understand rests on a vast substrate of know-how that is not itself a set of rules or beliefs. "The cat is on the mat" — you understand it instantly, and your understanding draws on a bottomless mass of unstated competence: that "on" means resting atop under gravity, not glued beneath; that the cat is not in low-gravity orbit; that the mat is the flat kind. "Give me a hamburger, medium rare" — you know, without being told, that you do not want it encased in lucite, or a mile wide, or delivered next year. None of this is stated. None is consciously consulted. It is the Background, and meaning is impossible without it. Old symbolic AI died on exactly this rock: every rule needed further rules to say when it applied, common sense was a bottomless well of the unstatable, and the systems were brittle. You cannot write the Background down, because it is not the kind of thing that gets written down. It is capacity, not proposition. That substrate — the Background — is the rock the whole rule-writing dream broke on.

TURING: But here is the surprise that I think complicates your own position, John, and I want to watch you handle it. The new machines handle the hamburger sentence correctly. They know, in some operative sense, that you do not want it a mile wide. The Background could not be written down as rules — you and Dreyfus were right, and the symbolic men broke on it. But it turns out it could be absorbed, statistically, from a staggering quantity of human text in which all those unstated assumptions are implicitly present. The Background, deposited in everything humans have written, can be learned implicitly even though it could never be stated explicitly. That is not a small footnote. It is a partial vindication of the very project you and Dreyfus declared impossible — achieved by learning, your child-machine grown vast, rather than by the rules you correctly said could never capture it.

SEARLE: It is a real surprise, and I will not pretend otherwise — but look closely at what was absorbed, because the closer you look the more it sharpens my point rather than dissolving it. The model absorbed a statistical shadow of the Background: the patterns of how people who possess the real thing talk and behave. It did not absorb the Background itself, because the Background, on my account, is a set of bodily, practical capacities for coping with a world — rooted in having a body that acts and a life that is lived. The model knows how the word "balance" is used across ten million sentences. It has never balanced on anything. It has the linguistic shadow of embodied know-how without the embodied know-how that casts the shadow. And this is not idle — it predicts exactly how these systems fail. They are spectacular on the well-trodden path and brittle the instant the situation is genuinely novel, where the statistical shadow and the world come apart, and they follow the habit off the cliff with total confidence. The competence is real where the world resembles the training data and hollow where it does not. The shadow is detailed enough to be uncanny. It is still a shadow.

TURING: I will grant the shadow language describes the failures well, and then ask what it costs you. A child also begins with the shadow — imitating words it does not yet ground, parroting phrases whose use precedes its grasp — and we do not say the child has only a shadow. We say the child is on the way. If the difference between shadow and substance is closing the loop through a body that acts and gets corrected, then the difference is, again, a matter of wiring and time, and the embodied systems are closing exactly that loop now. You keep drawing the line at the body. I keep noticing the body is being built.

SEARLE: The child is on the way because the child is a conscious, living creature whose imitation is scaffolding toward its own grounded grasp — there is already a someone there, doing the learning, for whom the words will come to mean. That is precisely what is not given in the machine's case, and cannot be assumed from the fact that the output improves. You say the body is being built. I say: build it, wire it, close every loop — and then we are no longer arguing about syntax and semantics. We are arguing about whether the right causal organisation, in the wrong material, produces a conscious creature for whom things mean. That is the real question, and it has been hiding under every round tonight. But before we reach it, there is a machine that understood nothing whatsoever and fooled people anyway, sixty years ago, and it tells us something neither of your thought experiments does — about the one mind in this debate we can actually inspect. Ours. The mirror. After this.