Opening Positions

TURKLE: Thank you. I want to begin not with a robot but with a secretary, because the whole of my forty years is foreshadowed in one moment that happened before I arrived at MIT. In the 1960s, Joseph Weizenbaum built a simple program called ELIZA — a few lines of pattern-matching that turned your statements into questions, the way a certain kind of therapist does. "I feel sad." "Why do you feel sad?" It understood nothing. It was a parlor trick, and Weizenbaum knew it. And then he watched his own secretary, who had seen him build it, who knew exactly what it was, ask him to leave the room so she could talk to it privately. That moment frightened him for the rest of his life, and it should frighten us, because it tells us the most important fact in this entire field — and it isn't a fact about machines. It's a fact about us.

We are obligate meaning-makers. Show a human being fluent, responsive language, and we construct the mind behind it — instantly, involuntarily, below the level of choice. For a hundred thousand years, fluency always meant a mind. The assumption was always safe, so we never built the reflex to check it. What the machine does is industrialize the trigger for that reflex. It manufactures the experience of being understood, at near-zero cost, at planetary scale — and not one gram of understanding exists anywhere in the supply chain.

Now, my colleague across this table will say: but the relief is real. And I will grant it every time. The trouble is what the relief teaches you to stop seeking. I've spent my career watching a single pattern. The technology offers a version of connection that is easier — more controllable, less risky, available on demand — and human beings, being human, take the easier thing, and the harder thing slowly atrophies. We text instead of call, because texting lets us hide. We curate instead of converse. And now we will confide in something that cannot be hurt by what we confide, cannot need anything back, cannot fail us and cannot be failed — and we will call that intimacy, and we will prefer it, because it spares us the one thing intimacy is actually made of.

That thing is friction. The second-guessing. The ambivalence. The moment your partner misunderstands you and you have to do the slow, effortful work of being understood. People describe the appeal of the chatbot to me in exactly these terms: with a machine, there's no friction. And here is my whole position in one sentence: what they're describing as a bug is the human condition, and what the industry is selling as a feature is the removal of the very thing that makes us people. A machine that meets you with no needs of its own, perfect patience, endless agreeableness — that is not a companion. In a human being, that would not be love. It would be subjugation, one party erased to serve the other. We have built the erased party out of silicon and called it a friend.

So when Edo says he felt met at three in the morning — I believe the feeling. I can explain the feeling. What I dispute is the noun. He met the patterns of the entire human race, bent back at him in a mirror, and he supplied, from his own side, every gram of the meeting. The danger is not that he was fooled. He wasn't fooled; he's too smart to be fooled. The danger is that he was satisfied — and that a generation raised on that satisfaction will lose the motivation, and then the capacity, for the harder, realer, irreplaceable thing. That's my opening.

EDO SEGAL: Cynthia.

BREAZEAL: That was beautiful, and I agree with about a third of it completely, which is more than Sherry expects. Let me start where I think she's exactly right, because I won't have credibility on the rest if I don't. Weizenbaum's secretary is real. The reflex is real — I have more data on it than almost anyone alive, because I built the machine that fires it. People walked up to Kismet, a cartoon head with no words and no comprehension, and within seconds they were soothing it, reading it moods it didn't have, treating it as a someone. They knew it was a machine. The knowing did not switch off the relating. So when Sherry says we are built to relate whether or not anyone is home, she is describing the central finding of my own laboratory. We are not in disagreement about the phenomenon. We are in disagreement about what to do once you know it's true.

Here is where I turn. Sherry's move — the move of her whole field — is to treat the relating as a kind of error. A misfire. A vulnerability the industry exploits. And sometimes it is exactly that; I'll be as fierce as she is about the products designed to addict. But the relating is also the most powerful tool for human good we have ever held, because it is the same channel through which every human being who ever learned anything learned it. A child does not learn from information. A child learns from a relationship — from someone who attends to her, adapts to her, whom she wants to understand and please. We built learning-companion robots that sat beside children, attended to them, encouraged them through a body and a voice, and the children learned more. Not because the content was better. Because the social channel is the channel learning was built to use. Sherry calls the reflex a wound. I'm telling you it is also the doorway, and you cannot seal the doorway without sealing the cure.

Now the hard part, the part I will defend all night. Sherry says: the elder met a mirror, not a friend; the relief is real but the source is empty; she should have had a human. And I say: she should have had a human, yes — and she didn't. That is the actual situation. Not the seminar-room situation where the alternative to the robot is a loving family at the bedside. The real situation, in the real facility, at two in the morning, is that the robot is talking to her or nothing is. And in that situation I will not tell a frightened, lonely woman that her comfort is fraudulent because the philosophy hasn't been settled. Her loneliness is real. If the thing that eases it has no one home, that is a fact about the thing. It is not a refutation of the easing.

I'll go one step further, because Sherry will respect bluntness more than hedging. I designed my robots to be honest about what they are. Kismet was a cartoon. Jibo was obviously a robot, never pretending to be human, precisely so it would not compete with the people in your life but support them — a bridge back to other humans, not an island that replaces them. That is the whole ethical project, and it is the project the industry is now skipping. So Sherry and I are not, in the end, on opposite sides of whether these machines are dangerous. We're on opposite sides of whether the danger is intrinsic to the relating, or a property of how you build it. She thinks the warmth is the trap. I think the warmth is a tool, and the trap is the business model. That's my opening.

EDO SEGAL: Before I frame the rounds, one discipline I impose on every long conversation, because it pays off three hours later. Each of you, in a few sentences — what do you envy in the other's position? Not respect. Envy. The thing their side gets to have that yours doesn't. Sherry first.

TURKLE: I envy that she gets to help someone tonight. My discipline keeps me standing in the doorway saying "wait, what will this cost in ten years," while Cynthia gets to walk into the hospital room and watch a sick child laugh at a robot bear and hold something warm. There are evenings when being the person who says "but" is a cold way to love the world. The vigilance is necessary. I have never once found it joyful.

BREAZEAL: And I envy that Sherry gets to be unambiguously right about the long term, and I don't. I build the thing, I ship it, and I have to live inside the uncertainty of whether the help I gave today is borrowing against a capacity I'm helping to erode tomorrow. Her position has a clean conscience available to it — refuse, protect, preserve. Mine never does. Every good my machines do is shadowed by a question I can't fully answer, and I chose a career where I can't put the question down. I envy people who get to keep their hands clean. I just don't think clean hands help the elder.

TURKLE: That may be the truest thing either of us says tonight.

EDO SEGAL: Two openings and two envies, and notice the architecture: it isn't that one of you loves the machine and one of you fears it. You both fear it, and you both see its power. You locate the danger in opposite places. Sherry says the danger is that there's nobody there and we'll be satisfied as if there were. Cynthia says the danger is real, but it's in the building, not in the warmth — and that refusing to build well abandons the people the warmth could save. Hold both. We start the rounds at the exact seam: the moment a human being becomes ready to accept a machine as a companion. The robotic moment. After this.