The Test Before Turing

EDO SEGAL: I want to start this round with the document itself, because almost no one knows it exists. René, in Part Five of the Discourse on Method, in 1637 — three hundred and thirteen years before Turing's imitation game — you wrote that a machine could never "arrange words so diversely as to respond to the meaning of all that might be said in its presence, as even the most stupid human beings can do." You proposed an outward, behavioral test for the presence of a mind, and you bet your philosophy that no mechanism would pass it. I have a machine in my pocket that arranges words exactly that diversely, in response to the open meaning of nearly anything I say. So I have to ask you the hardest question first. You wrote the test before Turing. The machine passes it. Were you wrong?

DESCARTES: I was wrong, and I will not soften it, because the discipline I asked for at the door requires me to say plainly where my clear idea turned out confused. I believed responsive language was a feat mechanism could not counterfeit. I treated it as the bright line. The machine has crossed the line. On the letter of my first test, it passes, and it passes better than many men I would have called rational. I yield the point entirely.

EDO SEGAL: That's a remarkable thing to watch a philosopher do in the first round. Most people defend the wall. Let me restate what you just conceded, because I want the reader to feel its size. You're saying: the thing I named, four hundred years ago, as the unforgeable signature of a soul — it's been forged. So either the machine has the soul, which you won't say, or the signature never proved what you thought it proved.

DESCARTES: Precisely that, and notice it is the second horn that survives, not the first. I conflated two things that the machine has now pried apart: the production of responsive speech, and the understanding from which, in me, responsive speech proceeds. I assumed they came always together, because in my experience they always had — wherever I found the words answering, I found a mind behind them, because there had never been any other source. The machine is the new source. It produces the words without — I maintain — the act. So the lesson is not that I was wrong about minds. It is that I was wrong about evidence. The test detected the outputs of thought. It never reached the thinking. I mistook the wake for the boat.

DENNETT: And here's where I have to interrupt the graceful surrender, because Descartes is conceding the wrong half and keeping the part he should drop. He says: the words came apart from the understanding, so the words don't prove understanding. Fine. But ask what it takes to produce those words — not passably, but the way these systems do it: take three paragraphs of a stranger's half-formed intention and return something apt, translate a joke and keep it funny, be told "no, the other sense of that word" and get it. There is no lookup table big enough. The only way to continue human text that well is to build, inside the system, a model of what the text is about — a model of the world the words describe, because the words obey the world's regularities. To predict the wake well enough, you have to reconstruct the boat. So when René says "production without understanding," I say: at this level, the production is a kind of understanding, the only kind that was ever doing any work — internal structure that tracks the world and gets flexibly redeployed. He's right that the old evidence was loose. He's wrong that there's a clean gap with words on one side and comprehension on the other.

DESCARTES: You have described how the machine comes to arrange its words. You have not shown that the arranging is accompanied by anything. I will grant you every word of the mechanism — the model, the structure, the redeployment. And I will ask the question my whole life was: and is anyone there for whom this means? You answer the question of how the outputs are generated and present it as an answer to the question of whether there is an understander. Those are not the same question. That a system reconstructs the regularities of the world is a fact about its arrangement — res extensa, the disposition of its parts, however vast. Whether there is, in addition, a res cogitans present to it, is precisely what no amount of describing the arrangement can settle.

DENNETT: Because you've defined the understander as a res cogitans — a thing-with-no-parts — and then noticed, correctly, that no description of parts will ever produce a thing-with-no-parts. Of course it won't. You've built the gap into your definition. That's not a discovery about the machine; it's the shadow of your own dualism. Drop the ghost and the gap closes: there's no extra "present to it" over and above the structure being there and doing its work. In me there's no extra presence either — that's the whole user illusion. You keep pointing at the missing soul as if its absence were a measurement. It's a stipulation.

EDO SEGAL: Let me put the second test on the table, because René, you didn't write one test — you wrote two, and the second one has aged very differently from the first. You said even a machine that did some things well would "infallibly fail in others, by which means we should discover that they did not act from knowledge, but solely from the disposition of their organs." Reason, you said, is "a universal instrument that can serve for all contingencies," where a machine is a heap of special arrangements, certain to break when the world brings a case it wasn't built for. And here's the strange thing, Dan — that second test reads, four centuries later, like a precise description of how these models actually fail. They pass the bar exam and then insist a pound of feathers weighs less than a pound of bricks. Brilliant inside the distribution, brittle the instant they leave it. René predicted the texture of the machine's incompetence.

DESCARTES: I did, and I find it the more durable of my two tests by far, because it points at something the first one missed. My first test asked for fluency, and fluency, it turns out, statistics can supply. My second asks for understanding as adaptability — the power to meet a genuinely new situation by grasping what it means, rather than by matching it to a stored pattern. That is far harder to counterfeit, because the whole point is performance on what was never seen. When the machine meets the unforeseen and answers confidently and wrongly, it shows me the disposition of its organs — competence shaped by what it was fed, collapsing precisely where the feeding stopped. That is not a mind grasping a case. It is a mechanism running off the edge of its arrangement.

DENNETT: That's the strongest card on your side of the table, and I want to be honest that it's strong — but watch what it's actually measuring, because it isn't a difference in kind. Every model release narrows the band of contingencies that break it. The brittleness shrinks. If the failure were the signature of a missing universal instrument, a missing soul, it wouldn't shrink with scale — souls don't come in increments. The fact that it does shrink, smoothly, with more data and more compute, tells you you're looking at a continuum of understanding, not a categorical wall. Your "universal instrument" is the far end of a ramp, not a separate thing dropped in from above. And — here's the part that should unsettle you — human reason is brittle too. We run off the edge of our arrangements constantly; we're confidently wrong outside our competence all the time. The universal instrument is an idealization. Nobody owns one. We're all heaps of special competences too, just very deep heaps.

DESCARTES: Then we have found the wager of your century, monsieur, and it is clean. You say the brittleness shrinks toward zero, and that a sufficiently broad arrangement simply becomes the universal instrument, with no further thing required. I say that coverage is not comprehension — that you may pile special competence upon special competence forever and never cross into the general reason that adapts because it understands, that the narrowing band is the machine learning more contingencies, not the machine acquiring the faculty that needs no contingency taught. Neither of us can yet collect the bet. But notice which of us the next decade will test. If the band closes entirely — if the machine ceases to fail in the Cartesian way — you will have your evidence. I will be watching the failures, because in my philosophy they are not noise. They are the disposition of the organs showing through.

EDO SEGAL: Hold that — it's the thread the whole evening hangs from, and it comes back when we talk about what the death cross is measuring. Mark this, though, because it's the first convergence and agreements are news: you both just agreed the first test fell and the second is the real one, and you split only on whether the second test measures a difference of kind or a difference of degree. That fork is the river. Next round, we leave the tests for the demon — because René, you built a thought experiment to imagine a world where every appearance could be forged, and the AI age has gone and built the demon. After this.