The Candle the Curve Can't Reach

EDO SEGAL: I want to give you both the metaphor I've been holding all night and ask each of you to take a position on it, because it's the one I actually live by. In [YOU] on AI I wrote about the candle in the darkness — the small, particular, mortal flame of a single human consciousness, the thing that is this life from the inside, that goes out and does not come back. The whole climb up the tower is supposed to serve the candle, not replace it. So here is the question, and I want it answered as a confession, not a thesis. Is there anything in you — not in the species, in you, the particular mortal man — that this curve cannot carry up to perfection and cannot push off any edge, because it simply isn't on the curve at all? Marquis, you first, and remember you wrote your candle in a room you expected to die in.

CONDORCET: [long pause] Yes. And I learned its name in that room. I wrote the Sketch for my daughter, Eliza — she was barely five — and for the others I would never see grow. And I will tell you something I did not put in the book, because the book was for the species and this is for the table. When I wrote that the mind progresses without limit, I was not, in that room, feeling the progress of the species. I was feeling the love of a man for a child he was about to lose, and the only thing I could do with that love, the only tool I had, was to build her a future in words — to say to her, across the death I knew was coming: the world your father is leaving will become worthy of you. That love was not on the curve, monsieur. It did not progress. It was complete, in that instant, in that locked room, at its full and final size. The candle. You have asked whether the curve can perfect it. It cannot, because it was never incomplete. The progress I spent my life proving is the servant of that flame — the whole vast staircase exists so that more children may be loved in safety by fathers who get to watch them grow. The error of my optimism, if I made one, was ever letting the reader think the staircase was the point. The child was the point. The staircase was for the child.

EDO SEGAL: [long pause] I did not expect the apostle of progress to be the one who put the curve in its place. Nick. Same question — and remember this is the image my whole book turns on, the candle in the darkness, the one flame the staircase is supposed to serve. The particular mortal man. What's not on the curve?

BOSTROM: I have spent my career trying to protect a future so large I can barely write the number, and I want to answer the Marquis honestly, because he just did something I'm not sure I can match. The thing in me that isn't on the curve — I think it's the same thing, and I think the Marquis just named it better than I ever have. All my astronomical stakes, all the unborn trillions I'm trying to safeguard — when I ask why they matter, the answer isn't a bigger number. It's that each of them would be a candle. A particular someone for whom things matter, the way they matter to me, the way they mattered to him in that room. The whole reason extinction is the worst thing isn't that it's a large quantity of death. It's that it's the permanent extinguishing of the capacity for there to be candles at all — that after it, there is no one for whom anything is like anything, forever. So the Marquis is right that the flame isn't on the curve. But here's why I can't fully rest where he rests. The flame isn't on the curve — and the curve can still blow out every flame there will ever be. That's the asymmetry I can't put down. He protected one candle with a future built of words. I'm trying to protect the conditions under which candles can keep being lit. We're doing the same thing. He did it for Eliza. I'm doing it for everyone who could ever be an Eliza. And neither of us, I notice, is doing it for the species in the abstract. We're both doing it for a child.

EDO SEGAL: Then let me press the one place you might still differ, because I don't want to manufacture an agreement that isn't real. Marquis, Nick just said the flame is what the curve is for — and yet his whole life's discipline is to subordinate the present flame to the future ones, the eight billion alive now to the trillions who could be. That is a kind of arithmetic on candles. Does it trouble you?

CONDORCET: It troubles me, and I will say exactly how, because it is the last seam between us. Monsieur Bostrom, I love your trillions. I would have wept to think a calculating engine could speak to them. But a candle is not a quantity, and the moment you sum it — the moment you say one present flame may be spent to secure a thousand future ones — you have done to the flame precisely what the Republic did to me: subordinated the particular living person to a magnificent abstraction about persons in general. I do not say your arithmetic is wrong. I say it is the same arithmetic, in the same family, as the one that built the guillotine in the name of a future virtue. And I learned, in that locked room, the one rule that arithmetic must obey: the future is built for the present flame, never out of it. Spend no living candle for an unborn one. That is not sentiment. It is the only firewall I know against your own orthogonality thesis turning up inside the people who are trying to prevent it.

BOSTROM: [long pause] That is the strongest thing you have said to me all night, and it is the one I most need to hear, because you have found the place where my own framework could metastasize into the thing it warns against. You're right that the cold sum of candles is one short step from the calculus that justifies any present cruelty by a future good. I'll give you the firewall, fully: the unborn are real and they are many, but no living person is a resource to be spent for them. The astronomical stakes raise the value of getting it right. They never license sacrificing the people in the room. If my discipline ever forgets that, it has become the disease. You have, I think, just told me the one sentence I most needed and least wanted to put at the center of my own work. Thank you for it.

CONDORCET: Then we have arrived, monsieur, at the place I did not believe we would reach. You and I have warred all night over the curve. And it turns out neither of us ever loved the curve. We loved what the curve was for. You guard the conditions of the flame; I sang the future to a particular flame I was leaving. The pessimist and the optimist, and underneath both, the same father in the same locked room, doing the only thing love can do in the face of a death it cannot stop — building, in words, a world worthy of the child. I have never felt less disagreement with anyone I have argued with so hard.

EDO SEGAL: I'm going to leave that exactly where it is, because the reader needs to sit in it. Three hours of collision and the floor underneath turns out to be shared. But sharing the floor is not the same as agreeing on the climb, and there is one room left where I go quiet and you two finally turn to each other — no moderator between you. The Crossing. After this.