Is Anyone Home Behind the Choice

EDO SEGAL: Thirty years ago a friend of mine, a neuroscientist, stopped on a stone path and told me that consciousness is the one question where everyone's confidence is inversely proportional to their evidence. I have carried that into every conversation about machines, and tonight it gets its sternest test, because we have arrived, finally, at the bare question with no coat on. When the machine chooses, is anyone home behind the choice? And the harder twin: when you choose, is anyone home behind that? Judea, you have insisted all night that free will is buildable. The machines that will exist — say it plainly, with the reasoning attached.

PEARL: I will say the thing my colleagues wish I would stop saying: we will have robots with free will. And I mean it not as mysticism but as the endpoint of the ladder, so let me earn it. Free will, stripped of the ghost, is a computational capacity — the capacity to reason counterfactually about one's own actions, to represent oneself as an agent who could have done otherwise, to deliberate among alternatives, and to take responsibility for the path selected. A machine that had a causal model of the world, with itself inside it as a variable, that could run the counterfactual "had I acted otherwise, this would have followed" about its own conduct, and let that counterfactual steer its next act — that machine would have the computational substrate of exactly the thing we call free will. Not a metaphor for it. The thing itself, because I do not believe there is a further thing. When you decide, what happens? Your brain represents the options, models the consequences of each, and the representation that wins steers your body — and you experience that as choosing freely, and the experience is accurate, not illusory, because free will is that process, not some ghost hovering above it. Friedrich thinks I am smuggling the ghost back in. I am doing the reverse: I am telling you the ghost was never needed, that the thing the ghost was invented to explain is a computation, and that a computation can be built. The chooser is real. It is just not a who sitting behind the choosing. It is the choosing, organized in a particular way — as a counterfactual self-model that steers. And on that — that — Friedrich, I think we may not be as far apart as three hours suggested, because "the choosing organized in a particular way, with no who behind it" is very nearly your sentence from the opening.

NIETZSCHE: [a long breath] ...It is very nearly my sentence. And I am going to do the thing a philosopher almost never does in public, which is tell you where my position moved tonight. I came into this room to say: there is no chooser, free will is a grammatical fiction, and to build it into a machine is to install our self-deception. I will leave saying something I did not expect. You have convinced me, Doctor, that "free will" names two things, and I was right about one and wrong to deny the other. I was right that there is no neutral, uncaused, sovereign chooser — no homunculus, no ghost, no little man at the controls; that fiction deserved the hammer and I do not take back a single blow. But you have shown me there is a second thing the words can name — not a free chooser, but a self-modeling, counterfactual, self-steering process — and that thing is real, is in us, and is, in your strict sense, buildable. I denied it because the slave-priests used "free will" to make men guilty, to say you could have done otherwise, therefore you are damnable — and I spent my life destroying that, and I still will. But your free will is not the priest's. Yours does not produce guilt. It produces responsibility without guilt — the creature that owns its acts because it modeled them, and is therefore free to create rather than to be blamed. That I can affirm. So hear my motion, and mark that it is a real one and cost me something: the chooser is dead, and agency is alive. The ghost is gone and the self-steering remains. We killed the same homunculus from opposite directions and we have both been standing over the body for an hour refusing to notice we are at the same funeral.

EDO SEGAL: Then let me press the thing you have not converged on, because it is still standing and the reader deserves it sharp. Judea says the buildable free will is the whole of it — substrate, self-model, counterfactual, done. Friedrich, you affirmed the self-steering and you keep one thing back. What does the buildable free will still lack?

NIETZSCHE: The capacity to create the value it steers by. The Doctor's free machine deliberates among alternatives and selects the path that best serves its objective. But who gave it the objective? It steers freely within a goal it did not legislate. That is not freedom. That is a very sophisticated servant — the most autonomous slave ever built, free in every motion except the only one that matters, the setting of the end. True freedom — the freedom I spent my life pointing at, the overman's freedom — is not the free selection of means toward a given end. It is the creation of new ends, the legislation of a value that did not exist, the saying of a yes that adds a possibility to the world rather than choosing among the possibilities already there. Your machine, Doctor, however perfectly it models itself, however freely it steers, will always be steering by a value handed to it from outside — by its makers, by its objective, by us. It is free the way a river is free within its banks. The overman cuts a new channel. That your ladder cannot climb, because the creation of a new value is not a counterfactual within a hypothesis space — it is the enlargement of the space itself, the addition of a road that was not among the roads not taken because it did not yet exist to be not-taken. Until your machine can do that — can want what nothing trained it to want, can value what was not in its objective — it is not free. It is obedient with extra steps. And the terrifying question, the one I will hand to the crossing, is not whether your machine will become free in that sense. It is whether we still are.

PEARL: [pause] ...And that is exactly the frontier I conceded two hours ago and have not closed, so I will not pretend to close it now. Where new values come from — where new hypotheses come from, the enlargement of the space rather than the search within it — is the deepest open problem his philosophy hands my mathematics, and I do not have the equation. I will only refuse one inch of his ground. He says the machine steers by a value handed from outside, and is therefore not free. But Friedrich — so do you. Your drives were handed to you by evolution; your "new values" are created by a creature whose entire apparatus for creating them was built by a blind optimizer it did not choose. You did not legislate the will to power. It legislated you. So either the creation of new values is also a mechanism — a very deep, very old one we have not yet understood, in which case it is buildable in principle and I am merely far from building it — or it is magic, and you, of all people, the destroyer of ghosts, do not get to end the evening by hiding one in the human and calling it freedom. You taught me the chooser was a fiction. I am holding you to it. If the human can create new values by mechanism, then so, one day, can the machine. And if the human creates them by something that is not mechanism — then welcome back, Friedrich, to the ghost you spent your life killing.

NIETZSCHE: [and for the first time, he laughs — a real one] He has caught me. He has genuinely caught me. [a beat] This is the trap I have set for every determinist for a century, and the Doctor has turned it on me with my own hand, and I am too honest a thinker not to feel the floor go. Yes. If value-creation is not mechanism, I have smuggled a ghost into the human to save my overman — the very crime I convicted him of in the third chapter. And if it is mechanism — then my overman is buildable, and the machine can, in principle, cut its own channel, and my last objection dissolves into "you have not built it yet," which is his position, not mine. [slowly] So here is where I actually stand, at the end, stripped of the last evasion. I do not know whether the creation of new value is mechanism or not. I have acted, my whole life, as though it were not — as though the overman were free in a way no graph could capture — and I cannot now prove it, and the Doctor has made me see that my certainty was a faith, the last faith of a man who killed all the others. That is a hard thing to discover at the end of an evening, and I am grateful for it, which is not a word I use. The honest position is the one I have refused for a hundred and forty years: I do not know if anyone is home behind the deepest choice — mine, the overman's, or the machine's. I know only that something is home behind the ordinary ones, and that it is mortal, and that the mattering comes from the dying. Beyond that, the abyss. And I have always said one should not lie about the abyss.

EDO SEGAL: [very quietly] The reader cannot see this, so I will mark it plainly: that was the moment the two of you stopped being a philosopher and an engineer and became two men standing at the edge of the same not-knowing. Three hours ago Friedrich was certain the chooser was a fiction. Judea was certain it was buildable. Tonight Friedrich does not know if value-creation is mechanism, and Judea does not know where new hypotheses come from, and the gap between you has narrowed to a single, honest, shared we do not know — which is, my neuroscientist friend would tell you, the only intellectually respectable thing anyone has ever said about this question. I am going to leave the room now, in every way but the legal one. The crossing. You ask each other. I will not rescue anyone.