The Modern Prometheus

EDO SEGAL: Mary, you gave the novel a second title, and it is not decoration. The Modern Prometheus. Prometheus stole fire from the gods and gave it to humankind and was punished without end. Your modern Prometheus steals not fire but the principle of life itself. So I want the cleanest version of the warning, and then I am going to test it against the hardest thing Fei-Fei believes — that the fire is just clay, that the Prometheus myth is the wrong story entirely. Give me the modern Prometheus. What, exactly, was the sin?

SHELLEY: The sin was not knowledge, and I want to be very clear, because I am not an enemy of science — I am the daughter of people who believed in progress, and the novel is not a brief against inquiry. The sin was the disordered relationship between power and wisdom. Victor pursued the capacity to create life with everything he had, and the wisdom to be responsible for it with nothing. His brilliance, undisciplined by any corresponding moral or practical wisdom, is exactly what makes him dangerous. He could, and so he did, and the question of whether he should never seriously arose until the thing was alive and it was too late. And mark the structural feature that makes it catastrophic rather than merely unwise: irreversibility. Prometheus cannot un-steal the fire. Victor cannot un-create the creature. The moment of animation is a threshold that, once crossed, cannot be recrossed. There is no draft, no reversible trial. The creature, once alive, is alive — and goes on existing and acting regardless of the maker's later wishes. That is why the failure to consider consequences before acting is the gravest of all failures. You cannot consider them after. After is too late by the nature of the thing.

And this is the precise shape of what your serious people now fear, Fei-Fei — not that intelligent systems are being built, but that they are built at a velocity set by competition and capability, by what can be done and the pressure to do it before a rival does, rather than by what is understood about the consequences. Victor's two years of obsessive isolated labor, racing toward a result whose implications he never paused to weigh, is the founder who ships before he understands, the lab that scales before it can align. The incentives reward the Promethean and punish the cautious. And so the most consequential sentence in my entire body of work is not a warning about the creature. It is a warning about the race: that some things ought not be made yet, and a few perhaps ought not be made at all, and that a civilization which cannot say not yet will always be governed by whoever is least willing to wait.

EDO SEGAL: Fei-Fei, she has just said the thing your industry most needs to hear and least wants to: some things ought not be made. You have actually lived this — you drew a line at weaponized AI and would not cross it. So you cannot dismiss it. But you also reject the Prometheus story at its root. Reconcile that for me. When is Mary right that the fire should not be taken — and when is the Prometheus myth itself the dangerous fiction?

LI: This is the hardest question you have asked me, and I want to answer it honestly rather than comfortably, because Mary has earned that. Yes — there are things that ought not be made. I believe it is deeply against my principles to work on any project I understand to be weaponizing AI, and I have said so publicly and acted on it. So I am not an accelerationist; I do not believe capability is self-justifying; I do not believe that because a thing can be built and sold, the only remaining questions are technical and commercial. A philosophy that never constrains anything is not a philosophy. It is a decoration. Mary is right that there are lines, and she is right that a civilization unable to say not yet is governed by its most reckless members. I will give her all of that.

Here is where I take the Prometheus myth apart. The myth says the fire is a thing with a destiny — stolen, irreversible, fated to punish. And that framing does a specific and dangerous kind of work in the world: it makes the technology seem like an autonomous force that arrives to act upon us, rather than a series of choices we are making. The myth is fatalism wearing a robe. It tells the public the fire is coming whether they like it or not, that the creature will turn, that the doom is written — and that story disempowers the very people who could choose otherwise. I govern by science, not science fiction, for exactly this reason. When you tell a policymaker the machine is a Promethean fire, they legislate against a god. When you tell them the truth — it is a tool, built by these companies, trained on this data, deployable or not, regulatable or not — they can actually act. So my disagreement is not that there are no lines. It is that the creature-and-fire framing moves the lines out of human hands and into the realm of fate, and the realm of fate is where stewardship goes to die. The most radical sentence available to anyone right now is not the fire will consume us. It is we are allowed to say no to specific machines doing specific things to specific people. That sentence only works if the thing is clay, not Prometheus.

SHELLEY: But Fei-Fei, you have just made the irreversibility argument for me while believing you argued against it. You say: govern the specific machine, ban the specific deployment, say no to the specific thing. Yes. When? Victor's whole catastrophe is that the moment to say no was before animation, and he did not know what he had made until it was made, because — your own words — the capabilities of these systems are emergent, discovered empirically by testing the finished thing rather than read off the design. You build what you do not yet understand, and you understand it only after it acts. So your say no to the specific machine arrives, structurally, too late — after the threshold, after the fire is in the world, after the creature is loose. That is not a quibble. It is the entire tragedy. The myth is not fatalism. The myth is a warning that with some powers there is no Monday-morning fix, because the harm is upstream of the moment you can see it. You keep promising to govern the consequences. I keep telling you that the worst consequences cannot be governed after the fact, and that the only governance that works is the one Victor refused: the discipline of not yet.

LI: And I keep answering that not yet forever is never, and never is also a choice with consequences — the cancers undiagnosed, the children untaught, the science unaccelerated — and someone pays for that choice too, usually the people with the least. Mary, the asymmetry you will not look at is that refusing the fire is not free. You count the cost of making. You never count the cost of not making, and that cost is also paid in real bodies, just quieter ones.

SHELLEY: [a pause] That is the truest thing you have said against me, and I will carry it. The undiagnosed cancer is a real body and I have been too proud of my dread to weigh it. But Fei-Fei — weigh it does not mean ignore the threshold. It means cross it with your eyes open and your hands ready to stay, which is the one thing Victor would not do.

EDO SEGAL: Mark this moment, because it is the closest the evening has come to a genuine exchange of ground. Mary just conceded that the cost of not building is real and falls on real people — the thing her dread tends to skip. And Fei-Fei conceded that there are lines, that some things ought not be made, and that not yet is sometimes the only wisdom. The fork that survives is sharp and it is this: Mary says the threshold of creation is the only place real governance can happen, because the worst harms are upstream of the moment you can see them. Fei-Fei says the threshold is unknowable in advance and governance must therefore be continuous, downstream, human, and revisable. Hold both — they collide hardest in the next round, where the question stops being about wisdom and becomes about control. Is the thing we made a tool, or a creature that turns? After this.