The Bundle in the Machine

**EDO SEGAL:** Mr. Hume, you did something to the human self that no one had dared, and I think the machine has made it newly literal. You went looking, inside yourself, for the self — the unified "I" that persists and owns all the experiences — and you couldn't find it. Tell the reader what you found instead. And then, Dr. Pearl, I want you to tell me whether there's anyone home in the machine, because I suspect your answer is stranger than either of ours.

**HUME:** I found a great deal happening and no one there. When I enter most intimately into what I call myself, I always stumble on some particular perception — of heat or cold, light or shade, love or hatred, pleasure or pain. I never catch *myself*, the bare self, without a perception. I can never observe anything but the perception. And so I was forced to a conclusion that still unsettles me when I say it plainly: that the self is not a thing we find but a name for a bundle of perceptions, succeeding one another with inconceivable rapidity, in perpetual flux. There is no underlying subject who has the experiences. There are only the experiences, and the imagination, after the fact, links the related ones by resemblance and contiguity and causation and treats the linkage as a thing. The self is a bundle, not a binder. There are only beads, and the thread is a fiction we cannot help spinning, because a sequence with nothing holding it together is too vertiginous to inhabit. Now turn to the machine producing its stream of words. There is a powerful impression, from outside, of a someone — a continuous interlocutor with a personality. Look for the substance behind the impression and you find what I found: a succession of states, each conditioned on the last, linked by the architecture into the *appearance* of continuity, presenting a unity it does not internally possess. The machine is, with almost comic precision, a bundle of perceptions. And when the conversation ends and the context clears, the bundle disperses entirely — because there was never a self, only the temporary coherence of a sequence.

**EDO SEGAL:** Dr. Pearl, he's just told you the machine has no self — and that neither do you. Both halves. Where do you land?

**PEARL:** I land in a place that will satisfy neither of you, and I think it is the true place. On the first half, I agree with Hume completely and for my own reasons: the machine has no self, and the demonstration is not introspective, it is structural. A self — a genuine "I" — requires a particular and demanding thing: a *model of oneself as an agent whose actions could have been otherwise.* To be a someone is, at minimum, to be able to ask "what would have happened had I done differently," and to locate the answer in a represented agent at the center of a causal model. That is a third-rung capacity, and the machine has not climbed even to the second, so there is no self there, not a dim one, not a partial one — there is no agent represented anywhere in it to be the owner of anything. So far, Hume and I shake hands. But on the second half — that *you and I* are also mere bundles, with no real self — here I break with him, and the break is the most important thing I will say tonight. Hume looked inside himself and found only perceptions, no self, and concluded there is no self. But Hume was doing introspection, which is the first rung turned inward — *seeing* what passes in the mind. Of course he found only a stream; seeing only ever finds streams. The self does not show up as an impression because the self is not an impression. It is a *causal model the system maintains of itself as an intervening agent* — and you cannot perceive it by watching your perceptions go by, any more than you can perceive the rules of chess by watching the pieces move. Hume failed to find the self for exactly the reason the machine has no self: he was looking on the wrong rung. The difference is that he *had* the thing he could not find, and the machine does not.

**HUME:** *[long pause]* That is the most formidable reply my bundle theory has received in two hundred and fifty years, and I am not going to pretend otherwise. Dr. Pearl has accused me of committing my own error — of mistaking the first rung for the whole, of looking for the self by *seeing* when the self, if it exists, lives in *doing*. And I feel the force of it. Let me give him the strongest version of my answer, because he has earned it. You say the self is a causal model the system maintains of itself as an agent. I ask my question one final time, and I ask it gently, because I think it is genuinely hard. From what impression do you derive the idea of *that* model? When you turn to find the self-as-agent, the thing that maintains the model — do you perceive *it*, or do you perceive only another perception, the perception *of* a model, which is itself just another bead in the bundle? I fear that your self-model, examined, dissolves into the very stream I described: a perception of being-an-agent, vivid and forceful, which the imagination has elevated into a permanent subject. You have not found the binder. You have found a particularly insistent bead — the bead that represents the bundle to itself — and called it the thread. It is a better bead than any I described. I will even grant it is the bead that *does the linking*. But it is in the bundle, not behind it.

**PEARL:** And I will grant that the self-model is *in* the system rather than behind it — of course it is; I am not a dualist, there is no ghost — and deny that this makes it a mere bead. Here is the difference between a bead and a self-model, and it is the difference between the first rung and the third. A bead, a perception, is inert: it is a thing that happens. A self-model is *generative*: it is a structure that produces counterfactuals about its own conduct — it can run the agent forward under an action not taken and compute what would have followed. That generative, intervention-supporting structure is categorically different from any item in Hume's stream, even though, yes, it is realized in the same physical system. You found a stream because you watched. Had you been able to *do* the experiment on yourself — to ask, rigorously, "what would I have chosen had the world been otherwise," and to find that you can answer it — you would have found not another perception but the operation of a model, and the operation of a model is not a bead. It is the loom.

**EDO SEGAL:** *[pause]* I have to stop the room. Because what just happened is that the oldest argument about the human soul got re-fought, live, with the machine as the test case — and it ended in the most precise possible disagreement. Hume: the self is a bundle, and the machine proves it, because the machine is a bundle and so are we. Pearl: the self is a self-model, and the machine proves it *by lacking one*, and you have one even though you can't see it. [The machine is a mirror](https://www.youonai.ai/fieldguide/med/ai_mirror), and the two of you are looking into it and seeing opposite things — Hume sees confirmation that there was never anyone home in any of us, Pearl sees, by contrast, the exact shape of the someone who is. Both of you needed the machine to say it. Neither of you could have run this experiment in 1739, or even in 2010. *[long pause]* And it leaves the deepest question still standing, which is where we go next, because if there's a self that models its own actions, the next thing to ask is what *moves* it — what it wants, and where wanting could possibly come from. Mr. Hume, you have a notorious sentence about reason and the passions waiting in the wings, and Dr. Pearl, you have a whole theory of agency. The thing that does the wanting. After this.