The Sketch and the Shutdown

EDO SEGAL: I want to put your two defining texts on the table side by side, because I have never seen two documents so opposite, written by men so alike. Marquis — in 1794, hunted, weeks from death, you wrote the Sketch for a Historical Picture of the Progress of the Human Mind, and its final section is a vision of the future so luminous it's almost unbearable: the abolition of inequality between nations, the abolition of inequality within them, and the indefinite improvement of mankind, including — you wrote this — the indefinite extension of the human lifespan. Eliezer — in 2023, you wrote in TIME that the only responsible course is to shut all advanced AI development down, by international treaty, enforced if necessary by destroying datacenters with airstrikes. Marquis, you wrote your hope while you were the one being hunted. Eliezer, you wrote your alarm while the technology was the thing you wanted hunted down. Marquis — start here: how did you write that vision with the guillotine at your door?

CONDORCET: Because the guillotine at my door was the proof of my thesis, monsieur, not the refutation of it. Understand what I was watching. I had helped make a revolution in the name of reason, and I was watching it devoured by exactly the forces I had spent my life opposing — by faction, by fear, by the oldest human reflex to find an enemy and a knife. And I asked myself, in that room: is this the truth about us, or is this the weather? And I concluded it was the weather. The Terror was a storm, a violent reversion, terrible and real — and local, in the long view, a single dark passage in a story whose direction I could read across millennia. I wrote the Sketch precisely because I was being hunted, as a wager placed against my own death: that the thing killing me was not the future but the past, making its last and most violent stand. A man writes his deepest conviction when it costs him everything. Mine cost me my life, and I would write it again tonight, in this chair, knowing the cell at the end.

EDO SEGAL: That may be the most extraordinary thing anyone has said in this series. You wrote the future as an act of defiance against the men killing you.

CONDORCET: I wrote it as the only revenge available to a condemned man — to insist that he was on the side of where the river was going, and his murderers on the side of where it had been. Now let me ask Monsieur Yudkowsky the question his own life poses, because I think we are stranger mirrors than the audience yet sees. You wrote shut it down. I have read the essay in my briefing. You called for the destruction of the very thing your field has spent its genius creating, knowing it would make you a pariah among your own. Monsieur — you wrote your alarm against your own community, the way I wrote my hope against my own revolution. We are both men who turned and condemned the thing we served. So I ask you, mirror to mirror: when you wrote shut it down, did you believe it would be heeded?

YUDKOWSKY: No. And I want to sit in that with you, because you've put your finger on something true. I wrote it not expecting to be heeded. I wrote it the way you wrote the Sketch — as the thing a person says when staying silent would be the only unforgivable act. The difference between us isn't courage and it isn't honesty. It's the direction the arrow of time points for our two beliefs. You looked at the Terror and saw the past making its last stand against a future that was going to win anyway, so your defiance was a kind of confidence — you were betting with history. When I wrote shut it down, I was betting against history. I think the river is flowing toward the cliff, and I'm the man standing in it with my arms out saying stop, knowing the water doesn't stop for a man with his arms out. You wrote from faith that the current was on your side. I wrote from the conviction that the current is the problem — that what we're racing toward is not progress but extinction-level risk, and that no amount of faith in the trend makes that current safe to stand in.

And here's the piece I have to say plainly, because it's the cruelest asymmetry between us. You got to be right slowly. You wrote the Sketch, you died, and then across two centuries the things you predicted mostly came true — the slavery abolished, the franchise extended, the lifespan doubled — and history vindicated you on a timescale that didn't require anyone to act fast. The thing I'm warning about doesn't give us a slow vindication. If I'm right, there's no two centuries in which the survivors gradually come around to my view and build the statues. There's one bad training run and then there's no one to build statues. Your kind of being-right gets confirmed by a future that keeps existing. My kind of being-right, if it happens, erases the audience that could confirm it. So I can't afford your patience. You had the luxury of writing for a posterity you trusted would arrive. I'm writing against the possibility that posterity is exactly what we're about to lose.

CONDORCET: Quietly. That is a profound and a terrible asymmetry, and I had not seen it so clearly until you said it. The condemned man in the garret was sustained by his faith in tomorrow. You are asking me to imagine writing the Sketch with no faith that tomorrow comes at all. I am not certain I could have held the pen.

YUDKOWSKY: That's the honest center of my whole life, Marquis. I'm trying to hold the pen anyway.

EDO SEGAL: I'm going to mark this, because the reader can't see your faces, and what just happened needs marking. That is the first exchange of the night where the two of you stopped arguing and recognized each other — two condemned men, two centuries apart, each writing his deepest conviction against the thing that would destroy him. The Marquis wrote hope against the men killing him. Eliezer wrote alarm against the field he founded. You are not opposites. You are the same act of defiance pointed in opposite directions, because one of you trusts the river and one of you fears it. Hold that recognition — it's the warmth under everything that follows. Because now I have to make it cold again. The next round is the one where I think you genuinely cannot both be right, and it's about a thing the Marquis spent his mathematics on and Eliezer spent his alignment work on: whose values does a mind end up serving? After the break.