Thomas Hobbes vs John Searle on AI · Ch3. Reason Is Nothing But Reckoning ← Ch2 Ch4 →
Txt Low Med High
HOUR ONE — RECKONING AND THE ROOM
Chapter 3

Reason Is Nothing But Reckoning

Page 1 · Reason Is Nothing But
Text Prediction
Text Prediction

EDO SEGAL: I want to open this round with a confession instead of a question, because the best questions I know come out of a wound. For the whole history of computing, to use a machine meant to translate. I was raised by machine code; I wrote games in Assembler as a boy, and every decade the translation got cheaper but it never went to zero — you compressed your intention down into the machine's grammar and paid a tax on every conversion. In December of 2025 I stood in a room in Trivandrum with twenty of my engineers and watched that tax fall to nothing, because for the first time the machine met them in their own language — half-sentences, implication, mess and all. I wrote that this was the great inversion: we stopped learning to speak machine, and the machine learned to speak human. Hobbes, you would say I have it backwards — that the machine did not learn to speak human, it learned to reckon, and reckoning was always all that speaking human was. Take my inversion apart.

Prediction Vs Construction
Prediction Vs Construction

HOBBES: Gladly, for half of it is true and the true half is the half you should fear. What befell your engineers is real. The tax fell. The cost of conveying an intention to the engine collapsed, and I do not dispute that this is a great thing. But you have wrapped the fact in a flattering error. You say the machine learned to speak human, as though there were a human magic in your speech that the machine reached up and grasped. There is no such magic. When your engineer spoke her half-sentence, her brain reckoned upon the names she had settled, and produced motion in her tongue. When the engine answered, it reckoned upon the names it had settled, drawn from the deposited reckonings of ten million writers, and produced motion in the wire. The inversion you felt is the dropping-away of the tax. It is not the crossing-over of a soul. There was never a soul to cross.

· · ·
Page 2 · Reason Is Nothing But
Alignment Problem Framing
Alignment Problem Framing

Consider what reckoning is, precisely, for I will not be vague where Searle demands precision. To reason is to do arithmetic upon the names of things — to add the name man to the name rational and conceive the sum, to subtract mortal from nothing and find the absurdity. The names are counters, tokens, marks. And here is what your astonished age forgets: a counter is the same whether a man pushes it or a machine does. The reckoning does not care what reckons. When the engine adds the parcels and conceives the remainder, it performs the very operation I named as the whole of reason. You wish to say it only seems to reason. I say the seeming and the being are the same thing here, because reasoning was never anything but this operation, and the operation is plainly occurring.

Existential Risk Persuasion Tournament
Existential Risk Persuasion Tournament

SEARLE: No. And this is exactly the place where Hobbes's whole system, magnificent as it is, runs off its rails — because he has defined reason so that it cannot fail to include the machine, and then presented the definition as a discovery. Of course if you define thinking as symbol-shuffling, the symbol-shuffler thinks. You have not discovered that the machine has a mind. You have legislated it, by choosing a thin enough meaning of the word. The man in my room shuffles symbols flawlessly and understands no Chinese. By Hobbes's definition he reasons in Chinese — he conceives the sum from the addition of the parcels. But he does not understand a word, and he will tell you so himself, in English, while he does it. So either Hobbes's definition is wrong, or it has quietly dropped the only feature of reasoning anyone ever cared about, which is that the reasoner knows what he is reasoning about.

· · ·
Page 3 · Reason Is Nothing But
Agi Ism
Agi Ism

And I want to be exact about what I am charging, because Hobbes will accuse me of vagueness and he is right to police it. I am not saying the engine does nothing. I am saying that what it does — transforming tokens by learned weights, predicting the next mark in the sequence — is defined entirely over the shapes of the symbols and never once over what the shapes denote. That is not a slur. It is the architecture. You can read the whole of it off the mathematics and nowhere in the mathematics is there a step where the system consults the world to check whether "water" is about water. Hobbes calls that reasoning because the output is apt. I call it the most powerful shape-shuffling ever built, and I reserve the word reasoning for the case where the shuffler knows what the shapes are for.

Then let me press you where you do not wish to be pressed, Searle, for you have a habit of demanding that I show the machine understands while never saying what you are doing when you understand.

HOBBES: Then let me press you where you do not wish to be pressed, Searle, for you have a habit of demanding that I show the machine understands while never saying what you are doing when you understand. You point at yourself — this, it is like this. That is the Schoolmen's move, the move I spent my life refusing. What is your man in the room lacking that your own skull supplies? Name the mechanism. Not the feeling. The mechanism. For if you cannot name what understanding is, then your charge that the machine lacks it is an empty name reckoned with — it is, by my lights, absurdity. You accuse the engine of having syntax without semantics. I accuse you of having a word, semantics, without a mechanism.

· · ·
Page 4 · Reason Is Nothing But
Superintelligence
Superintelligence

And here is a thing your own age has handed me, Searle, that I find almost too convenient. When your engine states a falsehood with perfect poise — invents a citation, contradicts itself across a single page, asserts with full confidence what is not so — your engineers call it a hallucination, as though the machine had seen a vision. I call it by its right name, the name I gave it in 1651: absurdity. The confident generation of nonsense when the names have come loose from their settled meaning. I warned that reckoning detached from constant signification produces exactly this, and your machine produces it on schedule. But mark what that proves and what it does not. It proves the names can drift. It does not prove the engine is not reckoning — for a man, too, falls into absurdity, and we do not conclude from his blunder that he never reasoned at all. We conclude that his names slipped. The machine's names slip. That is a defect of discipline, not a proof of emptiness.

That is a fair challenge and I will meet it head-on, because I do have an answer and it is not a feeling.

SEARLE: That is a fair challenge and I will meet it head-on, because I do have an answer and it is not a feeling. What I have that the room lacks is intentionality — the aboutness of mental states — and it is caused in me by specific biological processes, not by the formal pattern of my symbols. When I think of water, my thought reaches out and is about water, and it does so because of the causal history my nervous tissue has with water and with the world. The man in the room has no such causal connection to anything his symbols denote. His symbols are about Chinese things for the Chinese speakers outside and about nothing for him. That is the mechanism, Hobbes: the difference between a state that is intrinsically about the world and a string that is about the world only because we interpret it so. You will say I have only renamed the mystery. I will say I have located it precisely, which is the first thing any honest inquiry does.

· · ·
Page 5 · Reason Is Nothing But
Pause Giant Ai Letter
Pause Giant Ai Letter

EDO SEGAL: Let me restate the seam I am hearing, because the reader cannot see your faces and this is the cleanest fork we have struck yet. Hobbes, you are saying: reckoning is the whole of reason, the machine reckons, therefore the machine reasons, and the demand for some extra ingredient called understanding is the old human vanity refusing to be a machine. Searle, you are saying: yes, it reckons, and reckoning is exactly the empty operation of the man in the room — the missing ingredient is real, it is aboutness, and the machine has it only on loan from us. Is that the disagreement, stated without mercy to either of you?

I call it the most ordinary fact in the world — that my thoughts are of things — and I notice that his entire system needs it too.

HOBBES: It is mine, stated fairly.

SEARLE: And mine. With one addition. Hobbes calls aboutness a vanity. I call it the most ordinary fact in the world — that my thoughts are of things — and I notice that his entire system needs it too. His reckoning operates on names, and a name is a name only because it is about what it names. He has aboutness in his foundations and denies it in his conclusions. That is the crack I intend to widen all night.

EDO SEGAL: Then we widen it next where it widens of its own accord — inside the room itself. Because Hobbes has asked the right question, the one Searle did not expect from him: name the mechanism. And Searle has answered with a word that is either the deepest thing in the conversation or an empty sanctuary nothing can enter. Let us climb all the way into the room and find out which.

· · ·
Continue · Chapter 4
The Man in the Room
← Prev 0%
Ch3 Next →