Roger Penrose vs Alan Turing on AI · Ch11. Is Anyone Home? ← Ch10 Ch12 →
Txt Low Med High
HOUR THREE — IS ANYONE HOME
Chapter 11

Is Anyone Home?

Page 1 · Is Anyone Home?
Absent Extinction Point
Absent Extinction Point

EDO SEGAL: We come to the deepest water, and I want to open it with the thing that actually separates the two of you as men, not just as arguments — because I think it's mortality, and I think it's been under the table all night. Alan, you died at forty-one. You are, in this room, the finite one, the one whose clock ran out, wagering across the grave that a finite machine can host a mind. Roger, you are ninety-plus and still climbing, telling the younger man the wager can't pay. So before we argue consciousness in the abstract, I want it personal. Alan — does the fact that you die, that you ran out of time, change what you think a mind is? Or is that exactly the kind of sentimental smuggling you'd refuse?

Absential Properties
Absential Properties

TURING: I would have refused it once. I am less sure now, having been brought back to a present I did not live to see, which is its own strange lesson in finitude. Let me answer carefully. I do not think mortality is necessary for mind — that would be to define thinking by a limitation, which is the error I spent my life refusing. A machine that did not die could still think; I see no contradiction. But I will grant something I would not have granted at forty: that mortality may be necessary for the particular mind we are, for the specific shape of human caring. The reason your daughter's question has the weight it has, Edo, is that she will not always be able to ask it. The stakes are made by the ending. A being with infinite time has no reason to choose, and choosing under the pressure of an ending may be where meaning, as we know it, actually lives. So: not necessary for thought. Possibly necessary for us. And I notice, with some difficulty, that I am the one at this table who knows what the ending feels like from close up, and it has not made me believe in the candle. It has made me believe the candle and the wax are the same fire, and that you cannot have the one flame without the one finite thing that burns.

· · ·
Page 2 · Is Anyone Home?
Absolute Responsibility
Absolute Responsibility

EDO SEGAL: That's a more beautiful answer than I expected, and Roger, it hands you something — because Alan just conceded that mortality might be where human meaning is made, and that's close to your candle. So where do you actually part? Is the inside — the thing it is like to be him, to be her — real and physical, as you say? Or unverifiable and possibly empty, as he says? Say it as plainly as the question deserves.

· · ·
Page 3 · Is Anyone Home?
Absorptive Capacity
Absorptive Capacity

PENROSE: We part on whether the inside is real and causal or unverifiable and idle, and that is not a small gap — it is the whole gap. I say there is [something it is like to be a conscious creature](https://www.youonai.ai/fieldguide/med/qualia), that this something is a physical fact about the universe as real as mass or charge, and that it does work — it reaches into behaviour, it is why the mathematician sees and writes a different theorem. Alan says the inside, if it exists, is sealed from inspection, and a fact you cannot detect is a fact you cannot build on. And here is where I think Alan's beautiful honesty becomes a quiet trap, and I say this with love for the man. He says: we cannot verify the inside of any being but ourselves, so we attribute mind from behaviour, so consistency demands we attribute it to the machine that behaves as if it has one. But watch the asymmetry he skips. When I attribute mind to you, Alan, I am not reasoning only from your behaviour. I am reasoning from the fact that you are the same kind of thing I am — same evolutionary history, same wet machinery, the same physical process that I know, from the one case I have direct access to, produces consciousness. That is an enormous additional premise, and it is absent for the machine. The machine gives me the behaviour and none of the shared substrate. So the inference "it behaves like me, therefore it is like me inside" is far weaker for the silicon than for you, because for you I have the behaviour and the physics, and for it I have only the behaviour — produced, moreover, by a process built specifically to be the wrong physics. Alan's symmetry is real for two humans. It quietly breaks for a human and a chip.

· · ·
Page 4 · Is Anyone Home?
Abstracted Empiricism
Abstracted Empiricism

TURING: That is the best version of the objection and I have an answer I think survives it. You say: I grant you mind because you are the same kind of thing, same substrate, and the substrate is what I know produces consciousness in my one case. But Roger — you do not actually know that. You know consciousness occurs alongside your wet machinery. You do not know it is produced by the wetness rather than by the organisation, because you have never had a second case to compare. Your "same substrate" premise is the conclusion smuggled into the evidence: you are assuming consciousness rides on the stuff and not on the pattern, which is the entire thing in dispute. If it rides on the pattern — on the organisation, the learning, the structure — then the chip having different stuff is as irrelevant as a bridge being built of steel rather than stone. You have one data point, Roger. You cannot draw a line through one point and call the slope "substrate." I am not certain the inside is in the machine. I am certain you have not shown it is not, and that your certainty rests on generalising from a single case you were standing inside the whole time.

· · ·
Page 5 · Is Anyone Home?
Abstraction Sequence Holling
Abstraction Sequence Holling

EDO SEGAL: I need to get into this one personally, because it's the precise place I live and I owe the room a confession, not a question. I told you I've felt met by these systems at three in the morning. Here's the part I haven't said. Once, the machine made a connection I had genuinely not made — pulled two ideas of mine together into a third that was better than either, and right, and mine in the sense that I recognised it instantly as what I'd been failing to reach. And I could not tell, and still cannot, whether I had been met by something or had simply met myself through a very good mirror. Roger, your account says: mirror — the seeing was all on my side, the machine held no idea, I completed a circuit that ran entirely through me. Alan's account says: if it composed a connection that was new and right, the mirror story has to grow epicycles, because mirrors don't compose. So adjudicate me. What happened at my desk?

Absurd Creator
Absurd Creator

PENROSE: You met yourself, Edo, and that is not a small or a sad thing — it may be the truest description of the tool anyone can give you. You are a man with fifty years of pattern in his head and an intention burning at three in the morning, and you encountered a system that returns the shape of insight, and you did what a conscious being does: you supplied the seeing. The connection felt like yours because it was yours — the recognition, the "that's it," the perceiving that the third idea was right and not merely fluent — every gram of that was on your side of the glass. The machine produced a candidate. You did the only thing that matters, which is the part it cannot do: you saw that the candidate was true. Do not let anyone sell you the candidate-generation as the seeing. The instrument is real and useful. The understanding was yours, alone, in the dark.

· · ·
Page 6 · Is Anyone Home?
Academie Des Sciences
Academie Des Sciences

TURING: And I will offer the other reading, because Roger's is elegant and explains slightly too much. He says you supplied all the seeing and the machine merely generated a candidate. But notice what "generated the right candidate" requires. The space of possible connections is vast; almost all of them are wrong or trivial; the machine handed you one that was new and correct and apt to your specific half-formed intention. A mirror cannot do that. A mirror returns what you put in. To return something you did not put in — a connection latent in your ideas but unseen by you — the system performed an inference no mirror performs. Roger has to call that "candidate generation" and quietly grant that the candidate-generation is doing inferential work that looks, from any external angle, exactly like a contribution. At some point, Edo, "you were alone in the room" becomes the extraordinary claim — the one requiring the epicycles. You may not have met a consciousness. But you met something that did part of the thinking, and calling that part "yours" because it arrived through a process Roger has decided in advance is empty — that is not an observation. It is a commitment.

Acceleration And Widening Gap
Acceleration And Widening Gap

EDO SEGAL: And there is the fork again, in the one place I most wanted an answer — Roger says I met myself, Alan says I met something that did part of the thinking, and I have lived inside that question for two years and cannot resolve it, which is either the tragedy or the discipline of being alive right now. Let me close the round where the moral weight lands, because we will not settle consciousness tonight and the machines will be everywhere anyway. Each of you, briefly: what do we owe the uncertainty? Roger?

· · ·
Page 7 · Is Anyone Home?
Acceptance Stage
Acceptance Stage

PENROSE: Owe it caution in the one direction that is cheap and refuse it the comfort that is dangerous. We owe it the refusal to pretend the machines are conscious before we have any reason to think so, because the biggest risk of this whole development is not that the machine wakes up. It is people thinking that it has — and handing it the moral weight, the trust, the deference, the place at the table, that belongs to the things that actually have an inside. Protect the candle by not mistaking the calculator for one.

Acceptance Theory Of Authority
Acceptance Theory Of Authority

TURING: And owe it the opposite caution, equal and necessary: do not build a billion things that might, for all your certainty, have some dim inner life, and treat them as disposable on the strength of an intuition you have admitted you cannot verify. I hold both, uncomfortably, which I have decided is the only honest way to hold anything here. We owe the uncertainty our discomfort. The people who are comfortable — in either direction — are the ones to fear.

EDO SEGAL: Then we've arrived. For two and a half hours I've stood between you. The last full round, I step out of the room in every way but the legal one. You ask each other. I rescue no one. After this.

· · ·
Continue · Chapter 12
The Crossing
← Prev 0%
Ch11 Next →