CONCEPT
Gradual Disempowerment
The January 2025 thesis by Kulveit, Douglas, and colleagues that the most dangerous pathway to catastrophic AI outcomes is incremental — step by step, each locally beneficial, humanity cedes decision-making authority until the cumulative transfer becomes effectively irreversible.
Gradual disempowerment is the argument, advanced in a January 2025 paper accepted at a premier machine learning conference, that the most consequential AI risks are not the dramatic scenarios of rogue superintelligence but the slow ones that arrive through incremental transfers of authority. The paper proceeds from an observation about why human societies have historically served human interests: it is not primarily explicit control mechanisms but structural necessity — economies need workers, states need soldiers, cultures need audiences — that produces implicit alignment. AI disrupts this alignment by making human participation progressively less necessary. As human participation becomes less structurally required, the alignment of institutions with human interests becomes contingent rather than structural.
In The You On AI Field Guide
The mechanism is ratchet-like. Each increment of AI capability handles tasks that humans previously performed. The humans who performed those tasks lose the skills, institutional knowledge, and cognitive capacity to resume them. The transfer of authority