Opening Positions

HEIDEGGER: I will begin where everyone begins wrongly, with the picture of a mind. You imagine, when you imagine understanding, a small chamber inside the skull in which representations of the world are stored and manipulated — facts, rules, a model. To understand "the red block" is, on this picture, to possess inside that chamber a representation of redness, of blockness, of the relation "on," and to compute over them. This picture is so old and so deep that you mistake it for the structure of reality rather than for one historical way of covering reality over. It runs from Descartes — the mind on the inside, the world on the outside, a representation bridging them — straight into the laboratory where Herr Winograd built his program. The program is the picture made of wire. That is precisely why it is so instructive.

Now hear the other account, the one I spent my life recovering. Before you ever represent anything, you are already in a world. Not in it the way water is in a glass — in it the way the carpenter is in his workshop. The carpenter does not consult an inner theory of the hammer. The hammer is simply ready-to-hand; it has withdrawn into the swing; he sees through it to the work, and his understanding of "hammer" is the skilled coping, the wordless fluency, not any proposition he could write down. He knows the hammer the way you know how to walk down stairs — which is to say, not as knowledge at all in your sense, but as a way of being. Only when the hammer breaks, when the head flies off, does it leap into view as an object with properties, and only then does representation begin. Representation is what happens at the breakdown. It is the exception, the emergency tool, parasitic on the seamless involvement that is primary and that can never itself be fully represented, because it is the very thing that makes any representation mean anything.

So when you ask whether the machine that stacked the blocks understood the sentence, my answer is contained in a prior question you have not asked. Did it have a world? A world is not a collection of objects. It is the whole significant background — the workshop, the task, the mattering — within which objects can show up as the objects they are. The machine had a list. It had tokens bound to definitions a human had written down. It had, in my terms, nothing but the broken hammer, the present-at-hand, the exception with no rule beneath it — representation without the involvement that representation exists to serve. It did not stand in a clearing. It did not cope. It had no being-in-the-world, and so it had no understanding, because understanding, properly cashed out, is a mode of being-in-the-world and not a content inside a head. The block was never red for it. Nothing was ever anything for it. There was no "it" for whom anything could be at issue.

And here is what should frighten you about your large machine, which I have now been shown. It does not change this analysis. It enlarges it past the point where you can see the edges. SHRDLU's world was a tabletop you could feel the walls of in two sentences. Your transformer's world is the entire textual output of a civilization, so vast you may converse for hours and never strike a wall. But the wall is there, because the machine is still the broken hammer — still the present-at-hand without the ready-to-hand beneath it, still representation with no coping under it, still a thing for whom nothing is at issue. You have not built a being-in-the-world. You have built an enframing so total it has filled the space where the question of being would arise, and called the filling intelligence. That is my opening. I am, you will notice, not impressed by scale. Scale is the size of the broken hammer.

EDO SEGAL: Terry.

WINOGRAD: That was the clearest statement of my own first principle I have ever heard, delivered by the man I stole it from, and I am going to agree with almost all of it and then tell you where I am not a philosopher but an engineer, and where that changes the bill.

I built SHRDLU, and for a while I believed it. You could type "Pick up the big red block," and it would. You could ask "What is the pyramid supported by?" and it would answer. You could say "Why did you do that?" and it would trace its own reasoning and explain. In 1970, on the hardware of 1970, this was the most complete demonstration of machine language understanding the world had seen, and I will not let anyone, including Heidegger, call it a fraud. It was not a fraud. It was an illusion, which is a different and more interesting thing — an illusion about where the understanding lived. The program worked not because it grasped what the words meant but because the blocks world was so small and so tidy that meaning could be faked. Every object had a fixed list of properties the programmer had written down. Every relation was defined in advance. There were no surprises, because there was nothing in the world the programmer had not put there. The intelligence in the conversation was on the human side. I came to see that the success of the communication depended entirely on the real intelligence of the listener, and that there were many more honest ways to build a machine, given that the machine did not have the intelligence we kept crediting it with.

Now — where I part from Heidegger, and it is not where you think. I do not part from the diagnosis. I part from the prediction I once made and got wrong, and I will say so on the record in the first hour so no one can accuse me of hiding it. In 1986, with Flores, I wrote that one cannot construct machines that exhibit intelligent behavior. As a flat prediction, that was false. We now have machines that exhibit behavior we would call intelligent without hesitation if a human produced it. They pass the bar exam. They write working code. They hold context across long conversations. If "exhibit intelligent behavior" means "produce the outputs we associate with intelligence," I was simply wrong, and the salvage operation — that I only ever meant a critique of the representational approach — is legitimate and is also, honestly, a salvage operation. Let it stand.

But the deeper claim survived the thing that killed the prediction, and it survived in the most poetic way possible: the machine that proved my prediction wrong did it by abandoning everything Heidegger and I were attacking. The symbolic, rule-based program — the broken hammer made of wire — never escaped the blocks world. The thing that finally escaped was built on statistics over an ocean of data, with no hand-written representations at all. The transformer did not vindicate the representational tradition. It buried it. So Heidegger and I were right about the dead end. We were wrong about the road around it. And the open question — the only one worth three hours — is whether the road around it arrives at the embodied, situated understanding we both said a machine could not have, or whether it has merely built a mirror so large that the involvement reflected in it — which is still ours, still on the listener's side — is too vast for anyone to see the silvering. My bet is the mirror. But I hold it as a bet now, not as the clean verdict it was against SHRDLU, and an honest reader should feel the difference. That is my opening.

EDO SEGAL: Before I frame the rounds, one discipline I impose on every long conversation, because it pays off three hours later. Each of you, in a few sentences — what do you envy in the other's position? Not respect. Envy. The thing the other side gets to have that yours does not. Professor Heidegger first.

HEIDEGGER: I envy the workshop. Herr Winograd built the thing. He has callouses where I have only the word. When he says the understanding was on the listener's side, he is not reciting Heidegger; he is reporting from inside the machine, with the authority of a man who watched his own creation fail to be what everyone said it was. I spent my life arguing that you cannot reach being-in-the-world by piling up representations, and I argued it from the armchair, against a program I never touched. He earned the same conclusion with his hands, and then he kept his hands, and built tools instead of idols. I had only the danger to point at. He had the danger and a way to live inside it and keep working. The philosopher diagnoses the broken hammer. The builder picks up a different hammer that does not pretend to be a mind. I envy that he could.

WINOGRAD: And I envy the patience with not-knowing. My whole discipline pushes me to ship — to resolve, to decide, to build the thing and find out. Heidegger gets to dwell with the question and refuse to close it, and there are moments, watching these machines, when refusing to close the question is exactly the right and the hardest thing to do, and my training works against it. He would tell me to let the machine's nature remain open, neither conscious nor mere mechanism, and to live in that openness because closing it prematurely forecloses what I might otherwise see. I find that nearly impossible. I want a falsifier. I want a test. And he is right that wanting a test, here, may itself be the symptom — the engineer's reflex to turn every question into a benchmark, when the whole issue is whether the thing in question lives somewhere a benchmark cannot reach. I envy that he is not tempted the way I am tempted.

EDO SEGAL: Two openings and two envies, and the architecture of the evening is already visible. It is not that one of you loves the machine and one fears it — neither of you is selling it. It is that you locate the missing thing in slightly different places. Heidegger says what is missing is a world — being-in-the-world, the coping beneath all coping. Winograd says what is missing is care — the stake, the involvement, the giving of a damn — and that the missing thing may be the same thing seen from the engineer's chair instead of the philosopher's. We start the rounds where the seam is sharpest: with the blocks world itself. What, exactly, did SHRDLU know — and what did it never know it didn't know? After this.