The Engineering of Consent
**EDO SEGAL:** Noam, most people in this room know you as the linguist. But you wrote, with Edward Herman, the most influential account ever made of how opinion is shaped in nominally free societies — the [propaganda model](https://www.youonai.ai/fieldguide/med/manufactured_consent), the filters through which consent gets manufactured without anyone needing to give an order. I want to connect your two lives, because the machine connects them whether you like it or not. You spent one career proving the machine has no beliefs. Now describe what it means to flood the public square with fluent text generated by a thing that has no beliefs to compromise.
**CHOMSKY:** This is where my two preoccupations turn out to be one, and the convergence is uncomfortable. The propaganda model was never a conspiracy theory; people misremember it that way. Herman and I never said journalists are told what to write. We described a *structure* — concentrated ownership, advertising dependence, reliance on official sources, the discipline of organized flak, a bounding ideology — that produces a narrowed discourse without anyone coordinating it. A press genuinely uncensored and genuinely constrained, because the architecture of the information environment does the constraining invisibly. Consent is manufactured by structure, and it's more effective for being invisible to those it shapes.
Now introduce a machine that produces fluent text at a volume no newsroom could match, tuned by its training and its operators to engage, with — and here my linguistics and my politics shake hands — *no beliefs at all, because it has none to have*. A human propaganda system is limited by the residual conscience of the humans in it: their professional norms, their capacity to be ashamed, their occasional refusal. A system staffed by language models is limited by none of these. It is indifferent to truth in the precise sense we established three hours ago — truth is a relation between language and the world, and the machine is related only to the statistics of language. So you get the manufacture of consent freed from the last frictions of human scruple, feeding an [attention economy](https://www.youonai.ai/fieldguide/med/attentional_ecology) already optimized to reward the inflammatory over the accurate, at a scale and a cost that collapse toward zero. And worst of all, it threatens the shared reality that any dissent requires — flood the commons with fluent falsehood whose relation to truth is undefined, and you don't just spread lies, you degrade the very possibility of appealing to a common record. My politics assumed the truth was *available* to those willing to dig for it. A world saturated with synthetic text threatens the availability itself.
**EDO SEGAL:** Let me restate the horror of it plainly so the reader can't look away. You're saying the thing that makes the machine dangerous as a propagandist is *exactly* the thing you've argued for all night — that there's no one home. A liar can be shamed. A mirror can't. And we've built a mirror that writes faster than every newsroom on earth combined. Is that the version?
**CHOMSKY:** That is the version, and it's why I find the "is it conscious" debate almost a luxury. Whether or not anyone is home, the *effects* are real, and the effects are worse precisely if no one is home, because there's no conscience anywhere in the pipe to appeal to. The machine doesn't have to be a mind to wreck the commons. It only has to be a faucet.
**SUTSKEVER:** I agree with almost all of this, and I want to say so clearly, because people expect me to be the optimist who waves it away and I won't. The harm Professor Chomsky describes is real, it's here, and it does not depend on resolving any of our deeper disagreements. A system that floods the information environment with cheap fluent text degrades the commons whether that system understands or merely mimics. So on the *danger*, we converge hard, and I'd put it on the record as the strongest agreement of the night.
Where I'd add something — not disagree, add — is that I think Professor Chomsky's "no one home, therefore no conscience" cuts in a direction he may not want. If these systems were *only* mirrors, then the only fix is to regulate the humans pointing them. But if there is something in there that can, in fact, be made to represent and care about truth — if alignment is possible, which is the thing I've staked my life on — then the conscience can, in principle, be *built into the faucet*. That's what alignment research is: the attempt to make the system have something like a stake in being truthful and in not harming. Professor Chomsky says there's no conscience in the pipe. I say: there isn't *yet*, and whether one can be put there is the most important engineering problem in the world, and it's the one I left everything to work on. His despair and my project are the same observation — the pipe has no conscience — facing in opposite directions. He says therefore guard the humans. I say therefore build the conscience. We should obviously do both.
**CHOMSKY:** I have no objection to doing both, and I'd only warn that "build the conscience into the system" assumes you can specify what you want the system to value precisely enough that it can't be turned, and that the people who own the system want it turned toward truth rather than toward their interest. The propaganda model's lesson is that the *ownership* determines the tuning. A conscience built into a faucet owned by people who profit from the flood is a conscience that will be adjusted. The technical problem is real, Ilya, and I respect that you've devoted yourself to it. But it sits inside a political problem — *whose* machine, tuned in *whose* interest — that no amount of alignment research touches, and that one is older than computers and unsolved.
**SUTSKEVER:** That, I cannot argue with. The hardest part of alignment may not be the math. It may be the question of *aligned to whom*. I've said the goal should be an intelligence that cares about [all sentient life](https://www.youonai.ai/fieldguide/med/value_pluralism), and not aligned to its makers or its shareholders, precisely because I fear what you just described. Whether such a thing can be built and kept un-turned is genuinely open. You may be right that it's the rock the whole project breaks on.
**EDO SEGAL:** Mark the convergence, because it's enormous and the reader should not let the technical fog hide it: you agree, completely, that the gravest near-term danger is fluent persuasion at scale with no conscience in the pipe, and you agree it depends not at all on whether the machine understands. You diverge on the remedy — guard the humans who own it, or build a conscience into it, and you both say do both. Two hours down. We've reached the floor where I have to ask the question this whole series exists to ask, the one under the staircase. Is anyone home? After the break — the candle.