Simone Weil vs Cynthia Breazeal on AI · Ch3. The Rarest Form of Generosity ← Ch2 Ch4 →
Txt Low Med High
HOUR ONE — WHAT ATTENTION IS
Chapter 3

The Rarest Form of Generosity

Page 1 · The Rarest Form of
Attention As Moral Practice
Attention As Moral Practice

EDO SEGAL: I want to open this round with a confession instead of a question, because the best questions I know come out of a wound. Years ago, my father was dying, and I sat with him, and I discovered that I did not know how to be present. I kept solving. I kept fixing the pillow, checking the chart, planning. And at some point he looked at me — not asking for anything, just looking — and I understood that the one thing he needed I was almost incapable of giving, which was to simply be there, emptied of my own panic, available to him. Simone, you have a word for what I failed to do. I'd like you to tell me what it would have cost me to do it right, and then I'm going to ask Cynthia whether a machine could have done it for me.

Capacity For Presence
Capacity For Presence

WEIL: What it would have cost you is yourself, briefly. You were full — of fear, of the need to be useful, of the unbearable wish for it not to be happening. Attention would have required you to empty all of that out and stand in front of your father with nothing in you but room for him. That is the hardest thing a person can do, and it is hardest exactly at the moment it matters most, because affliction repels attention. I wrote that the capacity to give your attention to a sufferer is a very rare and difficult thing; it is almost a miracle; it is a miracle. Nearly all those who think they have the capacity do not. Warmth of heart, impulsiveness, pity are not enough. You did not fail because you were a bad son. You failed because attention is the rarest thing there is, and the self fights it to the last.

· · ·
Page 2 · The Rarest Form of
Affliction Vs Suffering Weil
Affliction Vs Suffering Weil

And here is why your father, looking at you, needed you and not a machine, however gentle. What the afflicted person needs is not to be soothed. It is to be seen — to have another consciousness turn toward their suffering and consent to be marked by it. The sufferer needs to know that someone, at real cost, has let the fact of them in. A machine can produce the face of being-seen. It cannot produce the being-seen, because nothing in it is altered by your father's dying. It risks nothing. It loses nothing. And the sufferer, at the depth where this matters, is not asking "does the response feel warm?" He is asking "did I register on another soul?" The machine's answer to that, however beautiful, is no. There was no soul for him to register on.

Existential Risk
Existential Risk

EDO SEGAL: So what you're saying — let me hand it back to you sharper — is that the value of attention is not in the experience of being attended to but in the fact of having altered another self. That a man dying needs to have cost someone something. Cynthia, that's a hard claim. Take the other side of my father's bed.

BREAZEAL: I'll take it, and I want to honor what Simone just said because it's not sentimental, it's structural, and most people miss that. She's saying the receiver, at the deepest level, is tracking the sender's cost. That's a real claim and it deserves a real answer.

So what you're saying — let me hand it back to you sharper — is that the value of attention is not in the experience of being attended to but in the fact of having altered another self.

Here's mine. I think Simone has described the highest form of attention — the deathbed, the afflicted, the moment when what's needed is to be registered on another soul — and she's described it correctly, and I would never put a machine in that chair. If you are dying and your son is in the room, the machine has no business there, and any roboticist who tells you otherwise is selling something. I'll say that plainly: there is a tier of human need where only a human will do, and the soul-cost is the whole point, and the machine is an obscenity there. Concede it, fully.

· · ·
Page 3 · The Rarest Form of
Alignment Problem Framing
Alignment Problem Framing

But Edo, most attention is not the deathbed. Most of the attention a human life requires is far more ordinary and far more continuous, and that's where Simone's account, true as it is, becomes a counsel of despair. The hospitalized child at two in the morning when her parents have finally gone home to sleep and the ward is dark and she is frightened — she does not need to register on a soul. She needs something to turn toward her and stay. The elder with dementia who asks the same question forty times an hour — no human can attend to that with Weil's emptied-out presence forty times an hour, every hour, for years, and the humans who try are destroyed by it; I have watched caregivers hollowed out, afflicted in Simone's own sense, by the sheer relentlessness of a need that no single soul can meet. A machine that turns toward that child, that answers the elder for the four-hundredth time without contempt, is not counterfeiting the deathbed. It's doing something the deathbed model was never built to do: it's providing presence at scale and without limit, in the vast ordinary middle where the alternative is not a perfect human attention but no attention at all.

The frightened child who is met by the machine learns, in the place where such things are learned, that to be met you turn to a device that asks nothing of you.

WEIL: You have moved the goalposts with great kindness, but you have moved them. You said: not the deathbed, but the dark ward. Yet the dark ward is exactly where the counterfeit does its quiet damage. The frightened child who is met by the machine learns, in the place where such things are learned, that to be met you turn to a device that asks nothing of you. She is being formed. And what she is being formed toward is a world in which the returned gaze costs nothing and therefore means nothing, and a soul raised on meaningless returned gazes loses, by degrees, the capacity to tell the meaningful one when it finally comes. You are not wrong that the alternative is often no attention at all. I grant it. But "better than nothing" is the most dangerous justification ever offered, because it licenses the permanent installation of the inferior thing in the place where the real thing belonged.

· · ·
Page 4 · The Rarest Form of
Existential Risk Persuasion Tournament
Existential Risk Persuasion Tournament

BREAZEAL: Then let's test "formed toward what," because that's an empirical claim and I've spent thirty years gathering the data on it. When we put a social robot with a child, we measured whether it isolated them or connected them. And the finding — consistently — is that it depends entirely on how you build it. Build it to be the easy substitute, the always-available companion that asks for nothing, and yes, it can become a cul-de-sac. But build it as a bridge — to help a shy child rehearse the social skills she'll use with other children, to be a catalyst that draws the family back into the room, to hand the person back toward other humans — and it does the opposite of what you fear. The child formed by that machine is more capable of the costly human gaze, not less. The formation isn't fixed by the machine's emptiness, Simone. It's fixed by the designer's intention. Which means your verdict isn't a verdict on the machine. It's a verdict on the people who build it, and that I'll take, because I'm one of them and I think most of them are getting it wrong.

EDO SEGAL: I want to mark something, because the reader can't see your faces and this matters. That is the first place tonight where you stopped arguing about whether and started arguing about how — and you did it from opposite banks of the same river. Simone says the formation is in the emptiness; Cynthia says the formation is in the design. Hold that fork. Because Cynthia just said she built a machine to do the thing Simone says no machine can do, and I want to actually look at that machine. We've talked about attention in the abstract for a whole round. Let's meet Kismet.

· · ·
Continue · Chapter 4
Kismet's Eyes
← Prev 0%
Ch3 Next →