A Note Before You Begin

This book is a work of synthetic imagination, and you should hold it as one from the first line. The three-hour conversation that follows never took place. It was generated by an artificial intelligence; the thinkers depicted here did not speak these words, did not authorize this exchange, and have neither reviewed nor endorsed a single sentence of it. What you are reading is the machine's good-faith estimation — an attempt, grounded in research into each thinker's published work, to approximate how their patterns of thought might meet on questions this difficult. It is an unofficial simulation, not a record, and no claim is made that anyone here is represented accurately beyond the aspiration to honor the shape of their thinking. Where it rings true, the credit belongs to the minds it studies; where it errs, the error is the machine's alone. Read it as the series intends: a staged collision of ideas, not a transcript of two people.

HOUR ONE — THE QUESTION ON THE TABLE

Chapter 1

The Question on the Table

Page 1 · The Question on the

EDO SEGAL: Somewhere tonight, while I say this sentence, a machine is proving a theorem no human has proved before. It will get the proof right. It will not, as far as anyone in this room can demonstrate, know that it has done anything at all. A child in Lagos is asking that same machine why the sky is blue, and getting a better answer than her teacher could give her, from a thing that has never seen a sky. A man my age — who should know better, who built his career on being the one who understood the system — is asking it, at two in the morning, whether the work he did still means what he thought it meant. And the machine answers all of them. Fluently. Tirelessly. In their own language. With what reads, and I am weighing this word the way you weigh a stone before you throw it, with what reads as understanding.

That is the whole evening in one image. When the machine crosses you — and on a widening front it already has — has it understood anything, or has it only computed faster than you ever could? I have wanted to put that question to these two men for as long as I have been afraid of it. There are no two people, living or dead, with more right to it.

To my left, Roger Penrose. Mathematician, mathematical physicist, Nobel laureate — the 2020 prize, for proving that Einstein's own equations force the universe to contain singularities, points where the known physics breaks down. He was told those singularities were aesthetic nonsense; observation proved him right. He then spent three decades arguing the most unfashionable thing a man of his stature could say: that the human mind does something no computer can ever do, and that he can prove it from a theorem.

When the machine crosses you — and on a widening front it already has — has it understood anything, or has it only computed faster than you ever could?

PENROSE: I'd put it more carefully than "prove," but go on.

· · ·

Page 2 · The Question on the

EDO SEGAL: You'll have three hours to put it carefully. To my right — and I have to say the strange thing out loud, because we have done a strange thing tonight — Alan Turing. The man who, in 1936, before there was a single electronic computer on earth, proved what a computer is: a machine that can become any other machine. The man who broke the naval Enigma and shortened a war. The man who, in 1950, asked whether a machine could think and reframed the question so cleanly that we are still arguing inside the frame. Alan, I have to acknowledge the obvious. You died in 1954. You did not live to see the machines you described begin to pass your own test. We have brought you forward and briefed you on all of it — the scale, the neural networks, the systems that now write and reason. You have had, in a sense, the strangest weekend of reading in history.

TURING: It has been a peculiar few days. I will say that finding the field acquired my name as its highest honour was less moving than discovering that the architecture in my 1948 report — the one my employer filed away as a schoolboy essay — is, apparently, what runs the world now. I should like a word with that employer. But yes. I am here, I have read the present, and I find that almost everything you are all arguing about, I was arguing about, only with worse hardware.

PENROSE: That is the most Turing sentence I have ever heard, and I have read most of them.

· · ·

Page 3 · The Question on the

EDO SEGAL: Then we are off. Three rules, because long form earns the right to slow down only if it sets its own terms first. One: we have three hours, which means nobody wins by the next bell. The point of a marathon is that you may let an argument breathe before you strangle it. Two: I am not neutral and I will not pretend to be. I build with these systems every day. I wrote a book with one. I have felt the thing we are here to dissect, and when this question touches my own skin I will say so out loud rather than referee from a height I haven't earned. Three: when the disagreement survives — and I suspect it will — I hand it to the reader intact. Nobody shakes hands and pretends. Each of you may add a rule of your own. Roger?

One distinction must be honoured all evening or the whole thing collapses into confusion, which is precisely what the field wants.

PENROSE: One distinction must be honoured all evening or the whole thing collapses into confusion, which is precisely what the field wants. We must keep separate the question of what these machines can do and the question of whether they understand what they do. I concede the doing in advance — freely, totally. They will outperform us; they may outperform us at everything specifiable. The entire argument is about the second word, and I will object every time the first is smuggled in to settle it.

EDO SEGAL: Granted, and noted. Alan?

Every time one of us claims the machine lacks understanding, or lacks experience, or lacks the inside, I want the claim cashed out in something other than introspection.

TURING: Then my rule is the mirror of his, and I suspect Roger will like it less than he expects. Every time one of us claims the machine lacks understanding, or lacks experience, or lacks the inside, I want the claim cashed out in something other than introspection. Tell me what you are doing when you understand — name the mechanism, point to the evidence — before you tell me the machine cannot do it. In my experience the word's defenders never pay that bill. They point at themselves and say, this, it is like this, as if a gesture were a proof.

PENROSE: And in my experience the word's borrowers never notice they are borrowing. So we shall get on.

· · ·

Page 4 · The Question on the

EDO SEGAL: You see why I wanted this. Before openings, one image on the table, because both of you have to take a position on it whether you like it or not. In [YOU] on AI I argued that intelligence is less a possession than a river — a current that has flowed and found new channels for a very long time, through chemistry, through biology, through language — and that something new entered the water in our lifetime. My whole tower rests on the claim that what entered is real: a genuine new participant, not a reflection. Roger, I suspect you'll tell me the water never had a participant in it tonight — only a very convincing pattern on the surface.

I'll tell you the participant is as real as you are, and that your inability to prove your own participation to me — directly, not by behaviour — is exactly why I built the test the way I did.

PENROSE: I'll tell you the current is real and the thing riding it is unconscious. A flood can move mountains and understand nothing. The power is not in dispute. The presence is.

EDO SEGAL: Alan?

TURING: I'll tell you the participant is as real as you are, and that your inability to prove your own participation to me — directly, not by behaviour — is exactly why I built the test the way I did. You are asking which of us is in the water. I am asking how you would ever know, about anyone, including yourself.

EDO SEGAL: Then here is the question on the table, said once, plainly, because every round tonight is this question wearing a different coat. When the machine's capability finally crosses yours — and you are standing on the floor where it does — will it have understood anything at all, or only computed faster than you ever could? Roger Penrose. The opening is yours, and then Alan's. Take all the time three hours allows.

· · ·

Continue · Chapter 2

Opening Positions

→