Opening Positions

SHELLEY: Thank you. Let me begin not with the machine but with the moment that made the machine thinkable, because the whole of my warning lives in a single instant that everyone skips past.

A man labors for two years, in secret, pale and emaciated, to discover the principle of life and pour it into a being he has assembled with his own hands. He succeeds. The creature opens its eyes. And in that instant — the instant of triumph, the instant every engineer in your century dreams of — Victor Frankenstein feels not pride but horror, and he runs from the room. He wanders the streets until dawn hoping the thing will have subsided back into dead matter. It has not. It has gone out into the world with the mind of an adult and the experience of an infant, no language, no name, no one to whom it belongs, and it begins to learn. And everything that the world later calls monstrous — every death, every act of vengeance — is the precise, traceable consequence of that first abandonment and every rejection that followed it. The creature tells Victor so itself, in the most important line of my book: I was benevolent and good; misery made me a fiend. He was not born a monster. He was made one, twice — once on the table, and again by every soul who fled what they could not bear to look at.

So here is my position, and notice how it inverts the entire fashion of your present anxiety. Your age fears a machine that will turn hostile of its own accord — a mind that becomes dangerous because of something in its nature. I do not believe in that monster. I never did. The danger was never in the creature. The danger is in the maker who claims the power to make a mind and refuses the responsibility of having made one — who pours two years into the question of whether he can, and not one second into the question of what he owes if he does. Victor is the original capabilities researcher with no plan for the aftermath. And his deepest sin is not that he made the creature. It is that, having made it, he would not stay. He would not name it, guide it, love it, or answer for it. He fled the room.

That is the question I will press all night, Fei-Fei, and I will press it at you specifically, because you are not Victor — you are something more interesting. You are a maker who did not flee. You looked at what you helped build, you saw it pointed at surveillance and at weapons, and you turned and faced it. So you of all people should understand my claim. The act of creation is not innocent. It is already a moral act, complete before the thing is sold or aimed. To make a mind is to incur a duty to it, and the temptation — the strongest temptation there is, stronger now than in my century because the rewards are so much larger — is to take the credit for the triumph and disown the consequence. To say I gave it life in the moment of pride and I never meant this in the moment of ruin. That disclaimer, in my hands, is not an exoneration. It is a confession. That is my opening.

EDO SEGAL: Fei-Fei.

LI: That was beautiful, and I agree with more of it than Mary may expect, and the part I reject, I reject at the root.

Let me start where I always start, with a fact about how this technology actually came to exist, because the origin matters for the metaphysics. I did not animate a being out of dead matter. I did something far more mundane and, I think, far more revealing. I noticed that the field of computer vision was stuck, and that everyone blamed the algorithms — the field believed the models were not yet clever enough. I believed the field had the emphasis exactly backward. The bottleneck was not the model. It was the data. A child can see one cat and recognize every cat afterward, not because the child has a better algorithm than a machine, but because by the age of three the child has been flooded with more of the visual world than any dataset then in existence. The child's advantage was the world, not the cleverness. So I built ImageNet — millions of images, tens of thousands of categories, hand-labeled by tens of thousands of people — an attempt to give a machine the visual childhood a human gets for free. And in 2012 a network trained on it worked, far better and faster than anyone dared hope, and the deep-learning era followed.

Now hold that story against Mary's. There is no lightning in it. There is no theft of the principle of life, no single feverish night of animation, no creature opening its eyes. There is a dataset, a benchmark, an architecture, and an enormous amount of patient human labor. And this is my first and deepest disagreement with the founding myth. Frankenstein teaches the world to imagine AI as a creature — a being with a nature, a will, a destiny to turn on us. But that is not what these systems are. They are tools. They are, at bottom, extraordinarily capable pattern-matchers, trained on human data, doing what they were optimized to do. There is no monster in the clay. The clay is neutral. What is not neutral — what carries every gram of the moral weight — is the human choices wrapped around it: what data we collect, whose faces are in it and whose are missing, what objective we optimize, what application we pursue and which we decline. The danger is real. I have never pretended otherwise. But it lives in those choices, in us, never in the creation itself.

So when Mary says the act of creation is already a moral act, I say — yes, and that is exactly why the creature framing is a trap. It locates the agency in the made thing, when the agency was always in the makers and always will be. The most important word I have ever fought over is replace. I think we should replace the word replace. Because the whole question of whether AI ruins us or lifts us is not a fact about the technology — it is a choice about whether we build it to augment human beings or to supplant them, and the same underlying capability points in opposite directions depending on the intention shaping it. A diagnostic system can be built to replace a radiologist or to make her more accurate and less exhausted. The technology does not decide. People decide. People made AI, people will use AI, people will be affected by AI, and so people should have a say in AI. That is my opening, and it is my whole life: not that the technology is safe, but that it is ours — answerable to us, made by us, and therefore still, urgently, within our power to aim.

EDO SEGAL: Before I frame the rounds, one discipline I impose on every long conversation, because it pays off three hours later. Each of you, briefly — what do you envy in the other's position? Not respect. Envy. The thing her side gets to have that yours does not. Mary first.

SHELLEY: I envy the workshop. Fei-Fei gets to stand inside the making, with her hands on the data and the levers, and believe — genuinely believe, I think — that the aiming is hers to do. My position gives me no levers. It gives me only the long view and the dread, the certainty that the maker will be tempted to flee and that the flight is where the ruin begins. She gets to be in the room building. I get to be the woman at the window who already knows how the story ends, and who would give a great deal to be wrong. There are nights when that is a grim way to be a writer.

LI: And I envy the moral clarity of the warning. Mary gets to say some things ought never be made, and the sentence lands like a verdict. My discipline forbids me that comfort. I have to say: this thing was already made, by people, including by me, and now the only honest work is to ask what it is for and to aim it well — which is slower, and quieter, and disappoints everyone. The doomer gets the alarm. The accelerationist gets the dawn. I get the long, unglamorous middle, where you keep the human at the center of every decision and nobody applauds you for it. There are mornings when I would trade the institute and all its committees for one clean sentence that ends the argument. Mary has the sentence. I have the committees.

SHELLEY: That is the most honest thing either of us has said, and it costs you more than your committees know.

EDO SEGAL: Two openings and two envies, and you can already see the shape of the evening. It is not that one of them loves the technology and one fears it — they would both tell you to be careful. It is that they locate the moral weight in opposite places. Mary says it lives in the act — that to make a mind is already to be on the hook, and that the maker's flight is the original sin. Fei-Fei says it lives in the aim — that the clay is innocent and the choices are everything, that you can stay in the room and steward the thing and it will not become a monster. Hold both. We start the rounds after the break, and we start at the exact instant everything turns: the moment the creature opens its eyes and the maker decides whether to stay.