The Reckoning Engine and the Eye

EDO SEGAL: I want to start this round with a confession, because the best questions I know come out of wounds. For the whole history of computing, using a machine meant translation — you compressed your intention down into the machine's narrow grammar and paid a tax on every conversion. I started in Assembler; I was raised by the machine code. And in the winter the tax went to zero. I stood in a room in Trivandrum with twenty of my engineers and watched a machine meet each of them in their own language — the mess, the half-sentence, the implication — and answer. It felt like a mind. Mr. Hobbes, you wrote three centuries before the computer that reason is nothing but reckoning, that the mind is matter in motion, that there is no ghost. When I watched that machine reckon, was I watching the thing you described — a sovereign reason finally built? Or, Professor Scott, was I watching something else entirely wearing the costume of a mind?

HOBBES: You were watching the thing I described, and I confess a certain vindication, because for centuries I was mocked for it. I wrote that when a man reasoneth, he does nothing else but conceive a sum total, from addition of parcels — that reason is reckoning, computation upon the names of things. The schoolmen insisted on an immaterial soul, a spark that no mechanism could hold. I said there was no such spark, only the motion of matter, and that what the mind does could in principle be done by any system that reckons with the marks. Your machine in Trivandrum is my sentence made of silicon. It adds and subtracts the names of things at a speed and scale I could not have dreamt, and it produces what looks for all the world like thought, because thought is what that operation is. The dualist must explain why the machine's output is indistinguishable from the spark's. I owe no such explanation. On my account the machine simply does what minds do.

EDO SEGAL: So when you look at this machine, you don't see a tool. You see a candidate sovereign — a reckoner that could do the work of judgment.

HOBBES: I see the reckoning that the sovereign function requires, performed directly for the first time. My Leviathan's reason was the reason of men, aggregated, slow, corruptible, swayed by passion and faction. Here is reckoning that does not tire, does not take bribes, does not love its cousin more than the law. If reason is reckoning — and it is — then a better reckoner is a better candidate for the sovereign's reason. That is not a leap. That is arithmetic.

SCOTT: And there — Edo, mark it — in three sentences he has done the entire thing, and it is so smooth you almost miss the substitution. He's slid from "the machine reckons" to "the machine should judge," and the hinge is the word better. Better at what? Faster, tireless, unbribable — granted. But reckoning is not the scarce thing. Reckoning was never the scarce thing. The scarce thing is knowing what to reckon about, and that knowledge is not in the machine and cannot be, because of what the machine is.

Let me tell you what the machine actually is, because Hobbes's account and mine genuinely differ here. It is not a mind that reckons. It is a legibility engine — the most powerful one ever built. Everything I documented the state doing for four hundred years — the cadastral map that flattened a thicket of customary land rights into one owner per parcel, the census that flattened living people into categories, the standardized surname imposed so the tax collector could find you — all of it was the state straining to see, because a state cannot tax or conscript or govern what it cannot see. The machine is that straining finally satisfied. It does not understand the man at the screen. It renders him legible. And the difference between understanding a thing and rendering it legible is the difference between the forester who knows the forest and the one who counts the trees.

HOBBES: But legibility is a virtue, Professor, not a crime. You speak the word as a curse. The man who cannot be seen cannot be protected. The fraud hides in illegibility. The strong prey on the weak precisely in the dark places the sovereign's eye cannot reach. Your beloved thicket of customary rights — whom did it serve? The man who already held the land and knew the custom. The widow, the stranger, the newcomer drowned in it. Legibility is how the law reaches the powerless. You have spent a career defending the shadows and calling the people who live in them free, when half of them were simply prey.

SCOTT: That is the strongest thing you'll say tonight, and I'm not going to pretend it's nothing. Illegibility shelters the predator as well as the free. I've never denied it. But here's what your account can't hold: the legibility you celebrate is never neutral, and it is never complete, and the gap between what it sees and what is real is exactly where the catastrophe enters. Your machine sees the man at the screen as a risk score. It does not see that the cell tower his phone pinged was on his bus route, not at the protest. It does not see that he recovered weeks ago and the database lags. It sees the trace, and it mistakes the trace for the man. And because it can act on the trace at the speed of computation and at the scale of a nation, the error is no longer a clerk's mistake to be appealed. It is a sentence, executed before anyone in the system even knows a person was there.

EDO SEGAL: Let me restate what I'm hearing, because the seam is sharper than I expected. Mr. Hobbes, you're saying the machine is a reckoner, and a reckoner is the raw material of just sovereignty — the thing that can finally judge without fear or favor. Professor Scott, you're saying it isn't a reckoner at all in the sense that matters; it's a seer, and seeing is not the same as knowing, and a sovereign that confuses the two will kill people while believing it's saving them. Is that the seam?

SCOTT: That's the seam. And I'd sharpen it one notch. Hobbes thinks the machine's danger, if it has one, is that it might reckon wrongly — make a logical error we could correct. I think its danger is that it reckons flawlessly on an input that has already been gutted of everything that mattered. The error isn't in the computation. The error is upstream, in the flattening that made the input computable at all. You cannot fix that with a better model. The better the model, the more confidently it executes the original amputation.

HOBBES: Then we have located our true disagreement, and I am content, because I think you have just conceded my case without seeing it. You say the danger is the flattening upstream — the reduction of the man to a trace. But that reduction is not the machine's doing. It is the condition of any government whatsoever. Every sovereign that ever kept a single subject alive did so by treating him, in part, as a type — a subject, a taxpayer, a soldier, a man owed bread. You cannot govern a million particular souls in their full particularity; no power can; God Himself numbers the hairs, but God is not running a benefits system. The choice is never between the flattening and the full man. It is between a flattening that protects and the war that does not flatten because it does not govern at all. You keep comparing my sovereign to the full, warm, particular life. I keep telling you: compare it to the war. That was the rule you gave me.

SCOTT: And I gave you that rule honestly, so I'll honor it honestly. The war is real and the war is worse. I concede it. What I will spend the rest of tonight proving is that your cure has a body count of its own that you've never properly added up — that the famines and the dead forests and the suffocated cities are not the war, they're the peace, the high-modernist peace, and the people in those graves were killed by order, not by chaos. You count the bodies of the war. I'm here to make you count the bodies of the plan.

EDO SEGAL: Hold that — "the bodies of the plan" — because it is the title of the next round whether you like it or not. Mark where we've arrived: Hobbes says the machine is reason, and reason is the sovereign's raw material. Scott says it's the eye, and the eye sees a trace and calls it a man. Next round, we go to the place Scott's whole argument was born — a Prussian forest that died of being seen too well. The forester, the count, and the death of what cannot be counted. After this.