Alan Turing vs John Searle on AI · Ch9. The Death Cross and the Promise No One Is Making ← Ch8 Ch10 →
Txt Low Med High
HOUR TWO — THE MIRROR, THE LIMITS, AND THE LEDGER
Chapter 9

The Death Cross and the Promise No One Is Making

Page 1 · The Death Cross and

**EDO SEGAL:** Let me set the table with numbers, because this round is about what the numbers mean. By early 2026, a trillion dollars of market value had left the public software industry — the [software death cross](https://www.youonai.ai/fieldguide/med/software_death_cross), the moment the line of human-written software crosses under the line of the machine's. Entry-level programming jobs down sixteen percent since 2022, the floor eroding first. And in that room in Trivandrum I watched twenty engineers become, by any measure I can apply, twenty times more productive — and then I sat in board meeting after board meeting where the arithmetic on the table was: if five people can do the work of a hundred, why pay for a hundred? I kept the team. I know the structure punishes that. Here is the trap, John, and I suspect you will refuse it: if the systems understand nothing, the trillion dollars is a delusion and the displacement is fraud. But the displacement is happening. Can a mirage restructure a labor market?

· · ·
Page 2 · The Death Cross and

**SEARLE:** Of course it can, and that is not a trap — it is the heart of the matter, and it is where the consciousness question turns out to be, in one sense, a luxury. Labor markets run on what decision-makers believe, and belief is exactly what the fluency manufactures. Watch the mechanism, because it is not "the machine can do the job." The machine produces something that resembles the work product. Resemblance is enough for an executive with a cost target and a consultant's deck. So the junior lawyer, the support agent, the translator, the illustrator are let go — not because the system does what they did, but because the story that it does survives long enough to clear a fiscal year, while the quality erosion lands later, diffusely, on customers and on the skeleton crew of seniors now babysitting the output. Whether or not anyone is home in the machine, the owners win, because the story is owned the way the weights are owned. That is why I say the inner-light question can be, for the worker at the kitchen table, almost beside the point. She is being told the machine made her redundant. The truth — whichever of us holds more of it — is that a decision did.

**EDO SEGAL:** Alan, here is where your own work bites, because you helped invent the thing now restructuring the floor under everyone. You watched mechanised reasoning win a war by doing cognitive work faster than minds could. Does the historical consolation hold this time — that every technology destroys some jobs and makes others, and the grandchildren end up better off?

· · ·
Page 3 · The Death Cross and

**TURING:** I am less sure than I would like to be, and I will not sell anyone the consolation cheap. The old reassurance quietly assumed the machine takes the muscle and the human moves up to the mind. This technology is aimed at the mind. When the machine does the cognitive work too, "move up" points at a floor that may not exist for most people. I was, for what it is worth, a socialist of a sort — I cared about who the gains accrue to — and the honest reading is that the gains flow to whoever owns the abundance, and absent deliberate redistribution that makes a few people much richer and most people poorer and more easily steered. And notice, John — this does not require my optimism about understanding to be correct. It only requires the systems to be good enough, owned by few enough. On the economics, your parrot and my participant point at the same cliff. We converge.

**EDO SEGAL:** Then let me take you both into the place your speech-act work goes, John, because there is a wound under the economics that the numbers hide. When a model says "I promise this is accurate," or "I recommend this dose," or "I'm so sorry for your loss" — your whole early career was about what makes those utterances real. Walk us into it.

· · ·
Page 4 · The Death Cross and

**SEARLE:** This is the part of my work I least expected to become urgent, and it has become the most urgent of all. Long before the room, I worked out — building on my Oxford teacher Austin — that to say something is very often to do something. "I promise to be there" is not a description of my future; it is an act that binds me, that did not exist until I uttered it. And speech acts succeed only when certain conditions are met. A promise is genuine only if there is a someone who intends to do the thing, who can be held to it, who places himself under an obligation. A verdict is a verdict only when uttered by someone with the authority to render it. The act is never in the words alone. It is in the words plus the intention, the standing, the responsibility behind them. Now put the model in that frame. It produces sentences with the full grammatical form of speech acts — it asserts, advises, promises, apologises, flawlessly. But the conditions that would make them genuine speech acts — the intending, the committing, the authoritative standing — are simply absent. There is no one who promised. We are building a world saturated with promises no one is making, assertions no one is standing behind, advice no one is responsible for. The form of commitment without the fact of it.

· · ·
Page 5 · The Death Cross and

**TURING:** And here I will push back, because I think your own theory contains the complication, and you taught it to me. You said meaning, at bottom, is the speaker's intention to produce an effect in the hearer by the hearer's recognition of that intention — Grice's idea, which you developed. Grant that the machine has no such intention and its speech acts are defective at the root. But you also showed, across decades, that speech acts have a public, conventional, institutional dimension that does not reduce to anyone's inner state. A form letter "states" things. A statute "declares" things. The force can ride on the institution and the pattern of use, somewhat independently of any individual's mind. So perhaps the machine's outputs acquire a kind of derived institutional force — real enough to bind, to mislead, to be relied upon — even with no meaner behind them. The accountability does not vanish because no one meant it. It relocates — to the builders, the deployers, the institution. The promise is real; the promisor is a corporation.

**SEARLE:** That is exactly right, and it is the most important thing we will agree on tonight, so let me mark it precisely. The aboutness, and the responsibility, run back to us — to the humans who built and deployed the system, the way the meaning of a road sign runs back to the highway authority that put it up. The danger is not that the machine is secretly committing to things. It is that human beings read illocutionary force automatically — we trust an assertion, rely on a promise, feel the warmth of an apology, without auditing whether anyone meant it — and that automatic uptake is the very machinery the fluent system exploits. When the model says "I understand how hard this is for you," your nervous system responds as if a person performed an empathetic act, and no one did. We are extending real uptake to as-if acts at industrial scale, and severing, quietly, the link between speech and a responsible speaker that gave speech its force in the first place.

· · ·
Page 6 · The Death Cross and

**EDO SEGAL:** Mark this convergence and number it, because the reader should see how strange it is. The behaviourist and the room agree, completely, that the gravest near-term harm does not depend on who is right about the inside. Alan: the gains flow to the owners whether the machine understands or not. John: the promises bind whether anyone made them or not. The two people least able to agree on what these systems are agree exactly on what they are doing to work and to trust. And a labor market, like a commons, does not care whether the thing draining it has anyone home. Hold that. Because the next round goes back inside — to the body, to a Berkeley ghost, and to the question of whether closing the loop through a world changes anything at all. After this.

· · ·
Continue · Chapter 10
The Robot, the Body, and the Ghost from Berkeley
← Prev 0%
Ch9 Next →