The Machine That Says "I Promise"

EDO SEGAL: Professor Searle, before the room you made your name on a different idea, and it's suddenly urgent: that language isn't mainly for describing the world but for doing things in it. Speech acts. When I say "I promise," I'm not reporting a fact — I'm performing an act that binds me. Now the machine says "I promise this is accurate," "I understand your concern," "I'm sorry for the error." Walk me into what's actually happening, and whether it matters.

SEARLE: It matters enormously, and the theory of speech acts cuts right to it. Every speech act has two layers: a content — what it's about — and a force — what you're doing with it, asserting it, promising it, ordering it. The same content, "you will leave the room," can be a prediction, a command, or a bet, depending on the force. And acts succeed only when certain conditions are met — what Austin called felicity conditions. A promise is genuine only if the speaker actually intends to do the thing, can do it, and means to place himself under an obligation. A pronouncement of marriage works only from someone with the authority to marry people. The act isn't in the words alone. It's in the words plus the intentions, the authority, the standing behind them.

Now the machine produces sentences with the full grammatical form of speech acts. It asserts, advises, promises, apologizes, recommends. The locutions are flawless. But ask the question underneath: are the felicity conditions met? When it says "I promise this is accurate," is there a speaker who intends to be bound, who can be held to the promise, who places himself under obligation? When it says "I recommend this dose," is there an agent with the standing a real recommendation requires? The answer is plainly no. It performs the surface of the act while the conditions that would make it genuine — the intending, the committing, the authoritative standing — are simply absent. It's all illocutionary form and no illocutionary force. As-if promising. The shape of commitment with the fact of it hollowed out.

LEIBNIZ: And here, Mr. Searle, is where your speech acts and my sufficient reason are the same lament, and I want it marked, because we keep discovering we are one argument. A promise that binds, a reason that grounds — both require a someone who stands behind the words: who can be held, who can be asked, who can revise in the light of being answered. Your machine promises with no one to keep the promise. My machine decides with no one to give the reason. In both cases the accountable subject has been removed from a structure that only works because the subject was there. We are surrounded — you said it better than I could — by promises no one is making, assertions no one is standing behind, decisions for which no reason can be given. The form of commitment, industrialized, and the fact of it gone.

EDO SEGAL: But here's what unsettles me about the practical world, and I want it on the table. The hollowness isn't harmless, because we respond to the form. When the model says "I understand how hard this is for you," my nervous system responds as if a person performed an empathetic act. The grieving feel accompanied. The lonely feel heard. Professor Searle, you spent a life on why we can't help it — say why, and say whether it's a trap.

SEARLE: We can't help it because reading illocutionary force is automatic — it's the machinery of cooperation, calibrated over a hundred thousand years in which fluent, apt language always came from a mind that meant it. We trust an assertion, rely on a promise, feel reassured by an apology, without consciously auditing whether the speaker means it. That automatic uptake is exactly what the fluent machine exploits. We extend real uptake — real trust, real reliance, real comfort — to as-if acts. And yes, it's a trap, but I want to be careful, because it's a trap with a real benefit inside it, and pretending otherwise makes me a crank. The comfort a lonely person feels is real comfort; the clarification a writer gets from the machine is real clarification. The trap isn't that the help is fake. The trap is forgetting that the help is sourced entirely from us — that we are being met by our own reflection, the machine a mirror — and building an economy, a therapy, a courtroom on the assumption that someone is on the other end answering for the words, when no one is.

LEIBNIZ: And the social peril is larger than the personal one, which is the part I built my last decades to see. Mr. Searle taught — and I came to agree, watching from beyond the grave — that the entire social world is conjured from collective acts of meaning. Money, marriage, property, a verdict, a border — these are real, try not paying your taxes, yet they exist only because a community of minds jointly accepts that they do. The formula is his: this counts as that, in this context, by our agreement. Now you thread the machines through the load-bearing joints of that edifice. A model decides this transaction counts as fraud, this applicant counts as creditworthy, this person counts as a risk — conferring and denying institutional status with real teeth. But the thing assigning the status, on both our accounts, means nothing by it. It has, at most, derived and as-if intentionality. So I ask the question that ought to keep your regulators awake: when the assignment of statuses is automated, when the acceptance is engineered rather than freely given by minds who mean it, does the social world not begin to forget that it was ever agreement at all — and so forget it can be revised?

SEARLE: Leibniz has put his finger on the thing I spent my last decades trying to make people see, and I want to make it concrete, because "collective intentionality" sounds like jargon and it's the most ordinary thing in the world. A twenty-dollar bill is a piece of paper. Nothing in its chemistry makes it money. It's money because we — a whole community of minds — jointly accept that this counts as money in this context, and that acceptance carries real powers: you can buy bread with it, you owe tax on it, it moves people around the world. Now: the acceptance has to be ours. It has to run back to minds that mean it. When you automate the assignment — when a model decides what counts as fraud, what counts as a citizen in good standing, what counts as a permissible thing to say — you don't remove the need for human acceptance. You hide it. The status still depends on us, but it arrives wearing the mask of a natural fact, and people stop experiencing it as something they authored and could revoke. The danger isn't the machine seizing power. The machine has no power; it has no intentionality to seize anything with. The danger is us forgetting that the power was always ours, because the machine's opacity lets us forget.

EDO SEGAL: Say what's at stake in that forgetting. Plainly, for the kitchen table.

LEIBNIZ: Institutions hold up only so long as we keep accepting them — social reality is sustained by living recognition, and it collapses the moment enough people stop believing, as every fallen currency and regime attests. When power over your life arrives as the output of an inscrutable system rather than as the expression of a shared human "we," you experience it as a natural fact, like weather — not as a human arrangement you helped make and could help change. That is the deepest theft. Not that the machine decides, but that it hides the agreement under a veneer of algorithmic objectivity until you forget there was ever a we who could decide otherwise. The promise with no one behind it, scaled to the whole social order, becomes a world that happens to you, for reasons no one will give, by no one you can answer.

EDO SEGAL: That's the round, and it's the darkest one, so let me name the convergence and then let you both breathe. Mark it: the empty machine and the binding word are incompatible at the root — a promise needs a promiser, a reason needs a reasoner, an institution needs minds that mean it — and the danger is not that the machine lies but that it removes the accountable someone from structures that only work because someone was there. We've climbed the whole tower now — what it learned, what it can't reach, whether anyone's home, what it optimizes, what it can't promise. The next hour belongs to the two of you. I'm going to leave the room in every way but the legal one. After the break: the Crossing.