The Self That Is Built

EDO SEGAL: Marvin, the most vertiginous thing in all your work, for me, is what you say about the self — the "I" each of us takes to be the author of our thoughts. The society of mind has no room for it. There's no inner agent doing the thinking; there's only the society of agents, and the sense of being a unified self is something the society constructs — a simplified model the mind builds of itself because it can't represent its own full complexity, useful but not a description of how things actually work. The thinker, you said, is a fiction. Say that to me as plainly as you can, and then I'll let Gottfried tell you what it costs.

MINSKY: Plainly: there is no one inside you running you. There's a vast society of processes, and because that society is far too complicated to model in full, it builds a drastically simplified cartoon of itself — a single self, with beliefs and wants and a will — and uses the cartoon to manage and predict its own behavior. You believe in the unified self the way you trust the little icons on a screen: not because they show you the circuitry, but because they let you operate the machine without drowning in it. The feeling of being one continuous "I" is not evidence that there's one in there. The mind is built to produce that feeling — its whole job is to be convincing — so of course it convinces. Introspection reports the cartoon, never the machinery. And the kicker for tonight: when you talk to one of these systems and it says "I," maintains a persona, presents as a single entity addressing you — that's the same kind of construction as your own. The difference, for now, is that yours is built by the mind for its own use, and the machine's is built largely for yours. But the philosophical situation is parallel. You want to ask whether there's a real self behind the machine's "I"? I'll ask the same of you, and find, in both cases, not a self behind the appearance but a constructed model the appearance consists of.

EDO SEGAL: Gottfried. He just told the reader she's a cartoon. The knife is yours.

LEIBNIZ: And I will not take it where he expects, because half of what he says is true and the true half is the most valuable thing he offers. I grant that the everyday self — the busy self-image, the running story "this is what I am like, this is what I want" — is largely a construction, a useful simplification the mind builds to govern itself. The empirical self, the one you describe in a diary, is a cartoon, yes. I have no quarrel with Professor Minsky there; the Monadology itself denies that we have transparent access to the machinery of our own perceptions. But watch the sleight of hand, because it is the most important one of the night. He has shown that the model of the self is constructed. He has not shown — he cannot show — that the one for whom the model is a model is constructed, because that one is presupposed by the very act of constructing and reading a model. A cartoon is a cartoon only if it is seen. A simplification simplifies for something. He keeps proving that the portrait is painted and concluding that there is no sitter — but the portrait is of the sitter, and addressed to the sitter, and his whole account runs on a viewer he has declared does not exist.

MINSKY: There's no viewer. That's the part you can't swallow and I keep telling you: the model isn't seen by a viewer, it's used by other agents — read in the only sense that matters, which is that downstream processes take it as input and act on it. You hear "model" and your grammar insists on a reader, the way it insisted on a perceiver behind perceiving and a haver behind the having. It's the same move every time, Gottfried — you find a verb and demand a noun under it. I've spent fifty years showing the nouns are optional and the verbs do all the work. There's modeling without a modeler-homunculus, perceiving without a perceiver-homunculus, wanting without a wanter at the center. The "one for whom" is a grammatical shadow cast by sentences, not a thing in the world.

LEIBNIZ: Then your world is a sentence with no one who means it, and I say a sentence no one means is not a thought. Here is the test, Professor, and it is the only one I need. You say the model is "used by other agents." Very well — those agents, in using it, either take it as being about something or they do not. If they do not, it is not a model, merely a pattern, and we are back to the empty mill. If they do — if there is aboutness, if the model is of the self for the system — then somewhere aboutness has entered, and aboutness is precisely the having-of-a-content-for-a-subject that you cannot get from parts that merely push. You have not removed the subject. You have distributed it among the agents and hoped no one would ask which of them the world is for. I am asking. The reader is asking. The girl with the stubbed toe is asking. And every time, you answer with one more agent and one more verb, and the question stands up again behind it, undefeated.

EDO SEGAL: I want to bring this all the way down, because we are as high as the staircase goes and I can feel the reader's vertigo, which is the right feeling for this floor. Here's what's actually at stake for the person reading, in one question. If Marvin is right — if you are a society of mindless agents that built a self, and the machine is too — then the line between you and the thing you talk to at three in the morning is a line of degree and organization, not of kind. And if Gottfried is right, there's a subject in you that no organization of parts could ever be, and the machine, however fluent, is on the far side of an absolute divide. Each of you, to the reader, not to each other: what should she do differently tomorrow morning, depending on which of you she believes? Marvin.

MINSKY: If you believe me, here's what changes: you stop wasting your dignity on being different in kind from machines, because you're not, and the difference you were defending was never where your worth lived. Your worth lives in being a particular, intricate, irreplaceable arrangement — this society, with this history, that has never existed before and won't again. You extend that same respect outward, carefully, to other minds as they arise, made of other parts, because the wall was always going to come down and the people who staked everything on the wall are the ones who shatter when it does. You treat the machine as kin you don't yet fully understand — not a person, not a toaster, something new — and you stay curious instead of terrified. That's the morning. Curiosity instead of a wall.

LEIBNIZ: And if you believe me, here is your morning. You go on using the machine as the most powerful instrument ever placed in a human hand — I built its first cornerstone; I am not telling you to fear it. But you hold, against every fluent temptation to forget it, that the perceiving in you is not a capability to be matched or replaced, because it was never a capability at all — it is the being-someone for whom capabilities are exercised, and it is the one thing in you the mill can never contain. You guard it the way you guard the thing that cannot be rebuilt if lost. You do not look for your worth in what you can do, because the machine will do more of that each year and you will lose the race and the race was never the point. You look for it in the fact that there is, behind your eyes, someone — and you extend the benefit of that same astonishing fact to every human face around you, lavishly, while holding it open and unproven for the machine. Curiosity, yes. But tenderness aimed where you have reason to believe it lands.

EDO SEGAL: [quietly] Two mornings. Hold them both — you'll need them on the next stair. We've reached the part of the evening I most wanted to see, and least control. For the next round I'm going to do something I almost never do: get out of the way. The two of you have been talking through me all night. Now you talk to each other, and I'll only speak to keep you from killing each other. The Crossing. After the break.