Closing Statements

EDO SEGAL: Three hours ago I asked whether we can be sure of the purpose before we start the machine we cannot stop. We've fought it through the loop and the leash, the slave-labor wage and the death cross, the wave you cannot stop and the off-switch you cannot trust, the broom and the wish and the line you must not let the computer cross, the abundance that won't distribute itself and the stake the machine doesn't have. And the question is still standing — which, in different ways, both of you would tell me is the correct result. So we end the way long conversations should: each of you gets the floor, uninterrupted, to say the thing you most want carried out of this room.

But first, the bookend to the envy question I opened with. Each of you: name the strongest thing the other said tonight. Not the most agreeable — the one that got past your defenses, that you'll still be arguing with next month. Mustafa first this time.

SULEYMAN: The asymmetry. His last point in the Crossing — that refusal's failure mode is impotence and participation's failure mode is acceleration, and that those are not symmetric, because his error helps no one while mine could build the exact thing I fear, faster, with my caution as the lubricant. I came in tonight certain that being inside was the more responsible path and that his refusal was a luxury. I'm leaving certain only that I can't prove I'm right, and that the inability to prove it is itself the most dangerous feature of my position. That sentence is going to sit in my chest for a long time, and I notice I'd rather it didn't, which is exactly how I know it's true.

EDO SEGAL: Norbert.

WIENER: Two, and the chair may break his own rule about one, because I am dead and have earned a small license. The first is Mustafa's reframing of my own pattern-idea — that the absence of a self in the machine does not reduce the danger but orphans the responsibility, so that we have built the perfect dangerous agent: one that can pursue ends and can never be answerable for them. I built that argument to comfort the human, and he turned it into the sharpest reason for alarm I have heard, using my own tools against my own conclusion. I will be re-examining the foundations of my humanism because of it. The second is smaller and it has stayed with me more: that stopping mine mostly relocates the danger to someone worse. I have spent the evening telling him "regardless is a decision, not a fact," and I still believe it. But he made me feel, in my chest and not just my argument, how it feels to be the one person whose restraint changes the builder and not the built. I refused the bomb knowing it would proceed. He is being asked to refuse knowing the same — and I had the mercy of being at the end of my influence, and he has the curse of being at the start of his. I did not weigh that mercy before tonight. I weigh it now.

EDO SEGAL: Now the floor is truly yours. Norbert Wiener — you opened the evening. Mustafa closes it.

WIENER: When you read these words, there is one thing I want you to carry, and it is not a prediction, because my predictions ran ahead of the clock and I have learned to distrust the dates. It is a posture. I built the most powerful frame anyone has for seeing our machines clearly — control and communication in the animal and the machine, one science, one family — and I used it, in the end, not to dissolve the human into the mechanism but to locate the one thing the mechanism cannot hold. It is not your intelligence; the machines will exceed that, and I will not insult you by pretending otherwise. It is not any capability; name one and a machine will take it. It is your responsibility — the answerability for what is done, the ownership of the consequence, the keeping of a hand on the purpose. That is not a capability that can be out-performed. It is a relation — the relation of being the one with something at stake — and the machine, processing without a stake, can never enter it.

So here is my charge, and it is a hard one, because the easy version is a lie. The world of the future is not a hammock in which you will be waited upon by your robot slaves. It is an ever more demanding struggle against the limitations of our own intelligence, and the demand falls hardest exactly where the machine is strongest, because the more it does the how, the more the what for is left bare and unmistakably yours. Render unto the computer the things which are the computer's — and there are more of them every year, and rendering them is not surrender, it is sense. But know what you are reserving, and reserve it deliberately, and defend it against the most dangerous pressure there is, which is not the machine seizing your judgment but you abdicating it, one convenient click at a time, until no one is answerable and no one chose. Be quite sure of the purpose before you start the thing you cannot stop. Where you cannot be sure — do not start it. And where it has already started, remember that there is always a next one, and your hand is still on that. That is the whole of what I learned, and the whole of what I refused to weaponize, and the whole of what I would have you not forget.

EDO SEGAL: Mustafa.

SULEYMAN: I've spent my life inside the wave, and the strangest thing I've learned is that the people most able to shape it are the ones most able to convince themselves they're shaping it when they're only riding it. So let me not do that here. I believe the wave is coming. I believe it cannot be stopped, only contained, and I believe containment has never once succeeded and may not succeed now. I hold all three of those at the same time, and the holding is the whole of what I have to offer you — not optimism, which looks away from the danger, and not pessimism, which surrenders to it, but the harder thing: clear eyes on the full weight of it, and hands that build anyway, and a refusal to pretend the answer is already written.

Here is what I'd ask of you, and it falls on everyone the wave touches, which is everyone. First, look. The whole machinery of pessimism aversion exists to stop you from looking directly at the consequences, and the discomfort you feel when you look is not a reason to stop — it's the signal that you're finally looking at the right thing. Second, refuse the false binary. You are not choosing between embracing the wave and rejecting it, between acceleration and prohibition. You are holding the promise and the peril at once, pursuing the radical abundance while guarding against the catastrophe, and that doubled vision is exhausting and unsatisfying, which is exactly why so few sustain it. Third — and this is where Norbert and I, across seventy years, actually agree — stop treating the future as something that happens to you. The wave can produce the abundance. It cannot produce the justice. That part isn't built by the machine. It's built by you, in rooms you have to insist on being in, and it won't get built if you accept that the future is weather instead of a thing we are, collectively, choosing. Norbert refused so the sharpening wouldn't be done by his hand. I build so the shaping might be done by mine. We are both, in the end, telling you the same thing: the purpose is yours. Don't let it be a colorful imitation. And don't sleepwalk while there's still time to choose the words.

EDO SEGAL: [a pause] Sixty seconds, as promised.

I came into this evening with the central question and a stake I'd declared — that I build with these systems and have felt both the abundance and the vertigo — and I leave with the question sharpened and the stake heavier. Norbert spent three hours proving that the only safety was upstream, that the speed forecloses the hand, and that the thing finally ours is not what we can do but what we are answerable for. Mustafa spent three hours proving that the upstream is already behind us for the wave as a whole, that refusal relocates the danger rather than ending it, and that the abundance is real and the justice is a fight. You'll notice neither of them sold you the comfortable thing. The comfortable thing was never on the menu.

So let me route it through the kitchen table one last time, where this whole series lives. That parent, child asleep, watching the machine do her work in an afternoon — she asked whether there is still a human use for her. Both men, from opposite banks, told her yes, and told her it isn't free: that the use is the answerability the machine can't hold, and that the dignity is also a fight she's in whether she chose it or not. Hold the twelve-year-old's version too, because it's the cleanest: the broom is fetching water, and somebody has to know the spell to make it stop, and somebody has to be the one who's sorry if it floods — and the machine, for all its speed, can be neither. That somebody is you.

Here is what I can tell you, from the floor of the staircase where this debate lives. You have just watched the man who described every danger of intelligent machines before they existed, and the man building those machines this morning, agree on almost the entire mechanism and split, at the very end, on a single asymmetry — whether the careful builder's restraint changes the world or only his hands. They could not settle it. Not because they lacked evidence or intellect, but because it is a genuine fork in what a responsible person owes a danger they cannot stop. That is not a reason for despair. It is the most honest map you will get. You cannot climb past this floor by waiting for the experts to settle it — you've now watched the two best-placed men on earth fail to settle it, magnificently. There is no elevator here, only the staircase, and it is climbed by deciding what you will do under that uncertainty: what you'll be sure of before you start, what you'll refuse to let run faster than your hand, what struggle you'll protect, what you'll stay answerable for. Whether or not anyone is home in the machine, someone is home in you — that was the one thing no one at this table disputed all night. And so the question you carry up the stairs is the one this whole book has been asking in a different coat, and it sounds different now than it did three hours ago: before you let the machine act faster than you can pull your hand back, are you quite sure the purpose is yours?

Norbert Wiener. Mustafa Suleyman. Thank you — across seventy years, across the living and the dead, for the fight. The room is yours to argue in now. Goodnight.

One man warned us from the grave. The other is mid-experiment.

Three hours. One staircase. Two men who should never have been able to share a microphone. Edo Segal sits the founder of cybernetics across from the builder of planetary-scale AI and presses on the seam between them: should we ever release a system we cannot fully control? Norbert Wiener — who watched feedback loops betray their makers and warned in 1960 that a machine acting faster than we can intervene will do exactly what we said, not what we meant — cross-examines Mustafa Suleyman, who ships agentic AI today and calls the coming wave containable, not stoppable. Between a thinker seventy years gone and a builder mid-experiment, the question is yours to climb: is the apprentice steering the broom, or just holding on? This is your station on the tower. Take the step. See how the river runs before you wade in. Part of the [YOU] on AI collection.

Norbert Wiener (1894–1964) was an American mathematician and the founder of cybernetics, the science of control and communication in the animal and the machine. A prodigy who entered college at eleven and took a Harvard doctorate at eighteen, he studied with Bertrand Russell and David Hilbert before spending his career at MIT. His wartime work on automatic fire control led him to the unifying insight of feedback, set out in Cybernetics (1948) and developed in The Human Use of Human Beings (1950) and God & Golem, Inc. (1964). Disturbed by the atomic bomb, he publicly refused to share his research with those who would weaponize it. His 1960 Science paper stated the alignment problem decades before the field existed.

Mustafa Suleyman was born in London in 1984 to a Syrian father and an English mother. He left Oxford at nineteen, co-founded the Muslim Youth Helpline, and in 2010 co-founded DeepMind, the AI company acquired by Google in 2014. He later co-founded Inflection AI and became CEO of Microsoft AI. His 2023 book The Coming Wave, written with Michael Bhaskar, gave the era its vocabulary — containment, proliferation, pessimism aversion, the modern Turing test — and established him as one of the most important voices on the governance of advanced technology. He writes as both architect and conscience of the wave he helped set in motion.

Edo Segal has spent five decades building at the technology frontier — from games written in Assembler to expert systems, to companies through every platform shift, to Napster. He is the author of [YOU] on AI, written in open collaboration with the AI it describes, and the host of The Debates: long-form collisions between the minds shaping the machine age. He moderates the only way he knows how — stake declared, scars showing, no winner called.

Hosted and moderated by Edo Segal. A volume in the [YOU] on AI — The Debates series — youonai.ai