Ray Kurzweil vs Bernard Williams on AI · Ch1. The Question on the Table Ch2 →
Txt Low Med High
Ray Kurzweil vs Bernard Williams cover
HOUR ONE — THE QUESTION ON THE TABLE
Chapter 1

The Question on the Table

Page 1 · The Question on the
River Of Intelligence
River Of Intelligence

EDO SEGAL: Somewhere in the world tonight a man is sitting by a hospital bed, holding a hand that is getting colder, and he is doing the arithmetic that everyone in that situation does and no one admits to — counting the hours, bargaining with a universe that does not bargain, and feeling, underneath the grief, a flash of pure animal rage at the simple fact that this is allowed to happen. That it has always been allowed to happen. That the smartest, kindest, most irreplaceable people we have are taken from us on a schedule none of us agreed to, and that the entire history of human wisdom amounts, on this one subject, to an elaborate set of instructions for how to lose gracefully.

Elevator And Staircase
Elevator And Staircase

I have been that man at the bedside. So has nearly everyone who will read this. And the question we are here to spend three hours inside is the question that flashes through you in that room and then gets buried under the consolations: what if it didn't have to be this way? What if the machine that learned our language could also learn the pattern that is us — could read it, repair it, and run it past the wall? And the deeper question underneath that one, the one only one of my guests thinks is even coherent: if it could — if the door at the top of the stairs could be unlocked — would you want it open? Or is the door, locked, the only reason any of the climbing was ever worth doing?

I can think of no two people who have ever lived with more right to this table.

· · ·
Page 2 · The Question on the
Weight Of Finitude
Weight Of Finitude

Ray Kurzweil has spent half a century being told he was a crank and then watching the calendar prove him right. He built the first print-to-speech reading machine for the blind, the first text-to-speech synthesizer, music synthesizers that fooled concert pianists. But the work that matters tonight is the curve. In 1999 he wrote down, with dates attached, that machines would master human language and begin matching us across cognitive tasks by the late 2020s, and the serious people laughed, and then December 2025 arrived and nobody was laughing. He calls the underlying engine the Law of Accelerating Returns, and he has aimed it, for fifty years, at one target above all others: the proposition that aging and death are not the fixed background of the human condition but a problem — an engineering problem — and that we are closer to solving it than almost anyone dares to believe.

The Pattern
The Pattern

KURZWEIL: That's a generous introduction, and I'll only correct the word "crank." I was never predicting magic. I was reading a graph. The graph is still there.

EDO SEGAL: Across the table — and I have to acknowledge the strangeness of the sentence I'm about to say, because the format of this series permits one impossible thing per evening and tonight we're spending it here — across the table is a man who died in 2003. Bernard Williams was, by wide agreement, the most important moral philosopher in the English-speaking world in the second half of the twentieth century. He spent fifty years dismantling the idea that ethics could be reduced to a system, insisting that the particular person, the specific life, the actual texture of a human predicament is where everything that matters lives. And in 1973 he wrote an essay about a fictional woman who had been alive for three hundred and forty-two years that has haunted everyone who thinks seriously about immortality ever since. He has been briefed on the present. He knows what a transformer is. He knows what my other guest is proposing to build.

· · ·
Page 3 · The Question on the
Agi
Agi

WILLIAMS: I know what he proposes to build. I should say at the outset that I find the engineering genuinely astonishing and the philosophy underneath it almost entirely unexamined, and that the gap between those two facts is the most interesting thing in the room. I'm grateful to be dead and able to say so.

Ai Alignment
Ai Alignment

EDO SEGAL: That's the format, so let me set the rules of the evening — there are three. First: we have three hours, which means nobody has to win by the next bell. Long form exists so an argument can breathe before anyone strangles it. Second: I declare my bias up front, and on this subject my bias is divided against itself — I have buried people I would do anything to bring back, and I have also felt, in my own body, the specific horror of a desire that would not switch off. I carry this question on both sides of my own heart. Third: if the disagreement survives three hours, we do not resolve it for you. We hand it to you, intact, and you carry it up the stairs. Either of you may add a rule.

KURZWEIL: One. When Bernard says "immortality," I'd like it understood that I have never used the word and never will. I am not promising anyone eternity. I am proposing that we stop dying involuntarily, on a schedule set by entropy rather than by us. The difference is the entire argument, and the word "immortality" smuggles in a metaphysics I reject before I've said a thing.

That's the format, so let me set the rules of the evening — there are three.

WILLIAMS: I accept the correction and I'll hold you to it, because it cuts both ways. If what you're offering is not eternity but a much longer run with an exit still available, then half my objection evaporates and we should say so early. But I suspect that when we get to the substance you'll want the exit door to recede further and further, because every reason you give for living to two hundred is a reason to live to two thousand, and a desire with no horizon is the precise thing I came here to talk about. So: agreed, no "immortality." Let's see how long you can keep the door in view.

· · ·
Page 4 · The Question on the
Alignment Problem Framing
Alignment Problem Framing

EDO SEGAL: You see why I wanted this. Before the opening statements I want to put one image on the table, because it's the frame this whole series climbs inside and both of you are going to have to take a position on it. In [YOU] on AI I argued that intelligence is less a possession than a river — a current that has flowed for thirteen point eight billion years through chemistry, biology, brains, language, culture, finding a new channel each time. Ray, your six epochs are the same river drawn with data instead of metaphor, and your claim is that the newest channel doesn't just carry the water faster. It changes what the water is — because for the first time the channel doesn't silt up, doesn't forget, and doesn't die. Bernard, I suspect your answer is that a river that cannot end is not a river at all.

Existential Risk Persuasion Tournament
Existential Risk Persuasion Tournament

KURZWEIL: My answer is that your metaphor is more literally correct than you may have intended. The river has been escaping the limitations of each substrate for billions of years — escaping the slowness of chemistry into biology, the mortality of the individual brain into culture and writing. What's arriving now is the substrate that escapes the last limitation, the one we've never been able to route around: the death of the individual pattern. I don't find that frightening. I find it the most natural continuation of the oldest process in the universe.

WILLIAMS: And my answer is that a current which cannot stop is not freedom from limitation — it is the loss of the only thing that gives a current a direction. Water flows downhill. Take away the destination and you don't get an eternal river. You get a flood, or a stagnant lake. But I'll make the case properly when you give me the floor.

· · ·
Page 5 · The Question on the
Superintelligence
Superintelligence

EDO SEGAL: Then we have our evening. One more thing before openings, because the reader deserves to know the stakes are not academic. Ray — you have said, repeatedly and without flinching, that you intend to bring back your father. That you have boxes of his letters and records preserved precisely so that one day a sufficiently advanced AI might reconstruct him. That is not a thought experiment for you. It is a plan. Bernard — your entire philosophy is a defense of the particular person against every system that would dissolve her into a type or a pattern or a sum, and your deepest claim is that a self is made of finite, exhausting, mortal commitments. You are not here to split a difference. One of you thinks death is the thief of everyone we love. The other thinks death is the condition that lets us love anyone at all. The worst possible outcome of tonight is the reader concluding the truth is comfortably in between.

Pause Giant Ai Letter
Pause Giant Ai Letter

KURZWEIL: On the stakes being personal — yes. I'll defend the plan when we get there. I won't pretend it's abstract.

WILLIAMS: Nor will I pretend my objection is. A man who wants to resurrect his father is not a man I want to defeat in an argument. He's a man I want to understand, and then tell, as gently as the truth allows, what I think he'd actually get back.

A man who wants to resurrect his father is not a man I want to defeat in an argument.

EDO SEGAL: So here is the question on the table, stated once, plainly, because every round we fight tonight is this question wearing a different coat. If the machine could hand you a life that never ends, would you climb forever — or is it the locked door of death that makes any of the climbing mean anything at all? Ray Kurzweil, the floor is yours.

· · ·
Continue · Chapter 2
Opening Positions
← Prev 0%
Ch1 Next →