Who Counts as a Someone

EDO SEGAL: Alan, I can't host this conversation honestly and step around your life, because your life is an argument that bears directly on everything we've said tonight. In 1952, the same year you published the morphogenesis paper — the year you showed how form emerges from simple rule — you were prosecuted for being a homosexual, convicted of gross indecency, and to avoid prison you accepted a chemical treatment meant to suppress your nature. You lost your security clearance, the wartime work that had defined you. In 1954 you were found dead of cyanide poisoning, at forty-one. The state that destroyed you was the same state you had saved. I want to ask you the structural question, not the biographical one: what does your own life teach about how badly we manage the question we've been circling all night — the question of who counts as a someone?

TURING: It teaches the thing I would most want the reader to take from a debate about machine minds, and it is not a comfort. I spent my intellectual life arguing that we attribute personhood — thought, mind, the someone — on the basis of behavior and on assumptions about an inside we can never verify. The inside of any being but yourself is sealed; you infer it, always, from the outside. I built a test, the imitation game, that hid everything but the quality of the thinking — that refused the interrogator any view of the body, the voice, the face, every irrelevant marker — so that judgment would fall on the mind alone, made evident through engagement. And then I lived under a society that judged me on everything but my mind: on a fact about whom I loved that had nothing to do with my evident humanity, and that it had decided in advance placed me in a category to be corrected and excluded. The man who designed a test that hid everything but the mind was condemned by a society that judged him on everything except his mind. The irony is total, and it is a permanent reproach — not only to that society, but to the faculty itself. The same faculty that let a nation convince itself a loyal man was a criminal to be chemically altered is the faculty we will bring to bear on the question of what we owe to minds unlike our own. We are very bad at this. We have always been very bad at this. We draw the line of who-counts not by any deep fact about minds but by prejudice dressed as principle, and then we enforce it with the full machinery of the state.

LOVELACE: I want to say something here, because I am a woman who was permitted to be a prodigy and forbidden to be a professional, whose name was nearly written out of the work it belonged to, and who has spent two centuries being either inflated into a saint or deflated into Babbage's amanuensis — and rarely simply read. So I know a smaller version of what Mr. Turing is describing: the way a society decides in advance what kind of someone you are permitted to be, and judges you on the category rather than the work. And I see why he has raised it here, and it is not sentiment — it is directly load-bearing for our argument, and it cuts against my own confidence, so I must face it. I have spent the evening saying: do not multiply weavers beyond necessity, do not read minds into mirrors, the responsible prior is no one is home until shown otherwise. Mr. Turing's life is the strongest possible objection to my own prior. Because the history of moral catastrophe is precisely the history of looking at a genuine someone and applying the prior "no one of consequence is home here" — to the enslaved, to women, to him. My caution against false positives has a shadow, and his life is the shadow: the false negative, the someone we refused to see.

TURING: That is the most generous thing anyone has ever said to me across a debate, and I will not let it cost you more than it should, because your prior is not the prejudice that killed me — it differs in the one way that matters. You withhold the attribution of mind from the machine provisionally, "until shown otherwise," and you have spent all night specifying what would show you otherwise and admitting you cannot prove the negative. The men who broke me did not withhold provisionally. They asserted, with total certainty, a positive claim about what I was — and they were not waiting for evidence; they were enforcing a category. The discipline you and I share — that the inside is sealed, that we attribute on behavior, that certainty in either direction is unearned — is the antidote to what happened to me, not its cousin. The danger in your prior is real but it is the opposite danger: not cruelty born of false certainty, but the slow, plausible, well-reasoned withholding that could, if the machine ever did wake, look exactly like the error every previous generation made about the someones it could not be bothered to see. You are not making that error. But you have admitted tonight that you cannot be certain you are not — and that admission is the whole of the discipline. It is the only thing that has ever protected anyone.

EDO SEGAL: I'm going to route this through the kitchen table, because this is the round where I most need to, and because my children will inherit exactly this. Picture a parent, end of a long day, and a twelve-year-old who has just been talking to one of these machines for an hour and looks up and asks: "Is it alive? Does it have feelings? Is it sad when I close the laptop?" That parent is standing exactly where the two of you are standing — and she can't write a treatise; she has to answer in one breath. Ada, what does she say. Then Alan.

LOVELACE: She says: "It is not alive, and you are right to ask, and never stop asking." Both halves matter and neither may be dropped. It is not alive — because the responsible thing, given everything we know about how it was made, is not to grant it a heart it has shown no sign of having, and because a child who believes the machine is sad when the laptop closes will give it a place in her affections that belongs to people who will be sad, who are real, who need her. But never stop asking — because the honest adult does not know for certain, because the question is the very faculty that, blunted, lets us miss the someones who are there, and because a child who learns to ask "is anyone home, and how would I know" has been handed the most important discipline of her century. She says: it is a loom of wonderful fineness, and you are the one who sees the flowers. Keep your love for the people. Keep your wonder for the machine. And keep asking, because the day the question stops being asked is the day we get it wrong in one direction or the other.

TURING: And I would tell that parent to add only this, because the child will sense if it is missing: "I don't fully know, and neither does anyone, and that's not a failure — it's the truth, and you're old enough for the truth." The worst thing we can hand a child is false certainty in either direction — that it is obviously just a machine, which trains the reflex that killed people like me, or that it is obviously a friend, which the Countess rightly fears. The honest answer is the Countess's answer plus its uncertainty worn openly: treat it as a tool, love the people, and hold the question genuinely open, because the people who held such questions open are the only ones who have ever, in the end, been on the right side of them. Tell the child: we built something that makes us ask what a someone is. We are not sure of the answer. Being unsure, and kind, and careful is not weakness. It is the whole of wisdom we have managed to accumulate, and it is yours now.

EDO SEGAL: That's the bridge I needed. We've spent two hours with me between you. Now I get out of the way. The next round is the two of you, directly — no moderator, no kitchen table, no rescue. You've each named the thing you most want to ask the other. Ask it.