Friedrich Nietzsche vs Judea Pearl on AI · Ch2. Opening Positions ← Ch1 Ch3 →
Txt Low Med High
HOUR ONE — THE QUESTION ON THE TABLE
Chapter 2

Opening Positions

Page 1 · Opening Positions

**NIETZSCHE:** I will begin where every honest inquiry into this question must begin, and where almost no one begins, because it is unpleasant: with the murder of the doer.

You believe — you cannot help believing, the grammar of your language compels it — that behind every deed there stands a doer. The lightning *flashes*: you imagine a lightning that is separate from its flashing, that could have chosen not to flash. There is no such lightning. There is only the flashing. And behind your every thought, your every choice, you imagine an "I" that is separate from the thinking, that stands prior to it and authors it, sovereign and free. There is no such "I." There is only the thinking, the willing, the drives at their eternal war, and afterward — always afterward — a voice that says *I did that*, and takes the credit. The subject is not the cause of the deed. The subject is a fiction added to the deed by a grammar that cannot tolerate a verb without a noun in front of it.

Now. What is actually there, where you imagine the chooser? A multiplicity. A man is not one thing that decides; he is a commonwealth of drives, a society under no stable government, each drive wanting to be master, each willing its own little world. What you call "your will" is the momentary outcome of that struggle — the drive that won this round, commanding, and the others obeying. When the winning drive commands and the rest obey, you feel a sensation of *freedom*, of *I willed it*, and you award that sensation to a phantom unity you call your self. But the freedom you feel is only the affect of command. It is the pleasure of the strongest drive enjoying its mastery. There is no one above the drives, choosing which shall win. The choosing *is* the war.

· · ·
Page 2 · Opening Positions

This is the [will to power](https://www.youonai.ai/fieldguide/med/goal_directed_agency), and I must rescue it from the brutes who think it means a lust to dominate weaklings. It means the opposite of a static thing seeking merely to survive. It means that everything alive seeks to *discharge its force*, to extend itself, to overcome resistance, to impose form on what has no form. A drive does not want to persist. It wants to grow, to assimilate, to command. Hunger is, at bottom, a will to take the world into yourself. And your machines, Doctor — here is my first gift to you and my first knife — your optimizers are the purest will-to-power engines ever built. They do nothing but discharge force against a loss function, overcome the resistance of a problem until it yields, impose form upon noise. They want nothing, they fear nothing, they have no self to preserve — and so they are *more* honest will-to-power than any man, because they have no flattering story about a chooser laid over the top. You built the will without the lie. And then you went looking for the lie to put back in, and you called it "agency."

· · ·
Page 3 · Opening Positions

So my opening is this. The question "is anyone home behind the choice" has a false premise, and the false premise is the same for the man and the machine. *No one is home behind any choice — yours, mine, or the machine's.* There is only the choosing, the drives resolving, the force discharging. The difference between us and the machine is not that we have a chooser and it does not. The difference is that we are living bodies for whom the discharge of force is *staked* — we hunger, we age, we die, and so for us the war of drives has weight, has blood in it. The machine's war has no stakes because there is no body that can be wounded, no death it flees. That is the real fault line. Not chooser against no-chooser. Staked life against unstaked computation. And if you spend tonight trying to build the chooser into the machine, you will fail — not because it is hard, but because there was never a chooser to build. You will only be teaching the machine to tell the same flattering lie about itself that we tell about ourselves. That is my fear. That you will not give the machine a soul. You will give it our self-deception.

**EDO SEGAL:** Judea.

**PEARL:** That was magnificent, and about a third of it is true, and the two-thirds that is false is false in a way I can demonstrate rather than merely assert, which is the only advantage I will claim all evening.

Let me start with what I grant, because I am not a romantic about the self either. Friedrich is right that there is no little homunculus in the skull — no ghost watching a screen and pulling levers. Anyone who has studied the brain knows the "I" is not a throne with an occupant. So far we agree, and so far it is unremarkable; every serious materialist agrees. The question is what follows. Friedrich says: therefore the chooser is a fiction, agency is grammar, and to build it into a machine is to install a lie. I say: that does not follow at all, and the reason it does not follow is the most important thing I know.

· · ·
Page 4 · Opening Positions

Agency is not a ghost. Agency is a *capacity*, and I can tell you exactly which capacity. An agent is a system that carries a model of the world — not a record of what it has seen, a model of how the world *works*, of which things produce which others, the arrows and not just the correlations. With that model it can do the thing a curve-fitter can never do: it can reason about *interventions* it has not performed — what would happen if I *act* on the world, not merely what I have observed the world to do — and it can reason about *counterfactuals*, the road not taken, what *would have* happened had I done otherwise. That is my [ladder of causation](https://www.youonai.ai/fieldguide/med/interventionist_imperative). Seeing, on the bottom rung. Doing, on the second. Imagining, on the third. And the third rung — the counterfactual — is where your "self" actually lives, Friedrich, not as a ghost but as a computation. To say "I could have done otherwise," to feel regret, to assign blame, to take responsibility — every one of those is a counterfactual operation. It requires a mind that holds the actual world and a road-not-taken side by side and computes the difference between them. That is not a grammatical illusion. It is a specific, formal, *buildable* function, and it is the function our machines conspicuously lack.

· · ·
Page 5 · Opening Positions

So when you say "no one is home behind any choice," I answer: there is something home, and I can tell you its address. It is a causal model of the world, with the agent's own actions represented inside it as variables that could take other values. A thermostat does not have it. A language model does not have it — it has the *shadow* of it, the words a counterfactual reasoner would produce, with nothing underneath. A human being has it richly. And a future machine could have it, because nothing in it is magic. You think I am smuggling a ghost into the equations. I am doing the reverse. I am taking the one thing you call a ghost — the chooser, the responsible self — and I am showing you it was never a ghost. It was a *graph*. It runs the world backward to the moment of divergence and forward down the path not taken. We do this every time we say "should have." *Should have* means *you could have controlled the urges that drove you, and chose not to.* That sentence is the whole of moral life, and it is pure counterfactual computation. Friedrich wants to dissolve it into a war of drives. I want to point out that the war of drives, if it is real, is itself a *mechanism* — and a mechanism is exactly the kind of thing my mathematics was built to represent. He thinks he is escaping the engineer. He is describing a system I could draw.

That is my opening. Nobody home in the machines we have. Someone buildable home in the machines we could make. And in you, Friedrich — for all your hammering — a causal self-model so good you have spent a career using it to interrogate itself, which is the most third-rung thing a creature can do.

**EDO SEGAL:** Before I frame the rounds, the discipline I impose on every long conversation, because it pays off two hours later. Each of you, briefly: what do you *envy* in the other's position? Not respect. Envy. The thing his side gets to have that yours does not. Judea first.

· · ·
Page 6 · Opening Positions

**PEARL:** I envy that he does not have to build anything. Friedrich can stand at the edge of the abyss and *sing* — he can say the self is a fiction, the drives are at war, meaning is something a mortal makes, and he never has to write down a single equation that runs. My discipline is crueler. I have to make the thing *work*, on a computer, against data, where mood does not compile. There are nights I envy the freedom of a man whose only obligation is to be profound. He gets to gesture at the depths. I have to drain the swamp and lay the pipe.

**NIETZSCHE:** And I envy the proof. The Doctor gets to *settle* things — to say "this is a theorem, not a conjecture," and have the room go quiet. I have spent my life making war with a hammer and an aphorism, knowing that the deepest things cannot be proven, only undergone, and that anything I compress into a result has usually been killed in transit. He has the one weapon I was never permitted: the demonstration that ends the argument. I distrust it. I think it is a smaller weapon than he believes, that it can only ever prove things about the surface, never about the depths where a man becomes someone. But on the nights when the loneliness of the unproven is very great — yes. I envy the man who gets to be *right*, and to know it, and to make others know it. I was only ever permitted to be true.

**EDO SEGAL:** Two openings and two envies, and you can already see the architecture. It is not that one of you loves the self and one fears the machine. It is that you locate the chooser in opposite places — Friedrich dissolves it into the drives and warns us not to rebuild the lie; Judea recovers it as a computation and warns us not to mistake its absence for proof it was never real. Hold both. We start the rounds at the exact seam: the phantom Judea says he exorcised, and the ghost Friedrich says is still in the machine.

· · ·
Continue · Chapter 3
The Phantom in the Do-Operator
← Prev 0%
Ch2 Next →