The Recursive Trap

In the AI Story

The trap operates across governance levels and mechanisms. At the electoral level, Clogger and its real-world approximations threaten the integrity of processes through which regulators are selected. At the regulatory level, synthetic comments can swamp public participation channels. At the media level, recommendation algorithms shape the public discourse through which governance questions are deliberated. Each of these mechanisms was designed to serve democratic governance; each is being compromised by AI.

The trap's most important feature is self-reinforcement. Degradation of democratic mechanisms reduces governance capacity, which allows further degradation, which further reduces capacity. Left unchecked, the loop progressively erodes both the quality of governance and citizens' capacity to engage in it. The erosion occurs not through dramatic assault but through the quiet degradation of institutional capacity, making it difficult to mobilize response until the damage is substantial.

The trap's implications extend to the attentional ecology on which democratic participation depends. Deliberative democracy requires citizens capable of sustained engagement with complex information, tolerance of ambiguity, consideration of multiple perspectives. These capacities are precisely what AI-shaped information environments undermine. Algorithms optimized for engagement reward emotional reaction over reflective judgment, producing citizens systematically less capable of the cognitive work democratic participation requires.

The trap has particular significance in authoritarian contexts, where AI concentrates surveillance and censorship capabilities in states that already possess them, further widening the asymmetry between governments and civil society. Fung's December 2024 workshop at the Ash Center documented this dynamic: democracy movements have experienced historic decline in their ability to challenge autocratic governments, due in part to the changing technology landscape that has allowed autocratic governments to monopolize the advantages of breakthrough technologies.

Minipublics and other deliberative mechanisms are Fung's answer to the trap because they operate outside the algorithmic informational environment. Face-to-face deliberation with balanced information and skilled facilitation gives algorithmic recommendation engines no purchase, persuasion technologies no target, synthetic comments no channel. Deliberation becomes structurally resilient to the specific threats AI poses to democratic governance — not because it is inherently superior but because it is resistant to the mechanisms through which AI degrades other democratic forms.

Origin

The concept emerged from Fung's extension of the Clogger analysis beyond the electoral context. The recognition that AI-shaped electoral outcomes would produce AI-shaped governance institutions generalized into the broader observation that AI shapes every informational environment within which governance occurs, creating the recursive dynamic the trap describes.

The conceptual debt to cybernetic theory — particularly Norbert Wiener's work on feedback loops — is implicit. The trap is a specific instance of the general problem of governing systems that reshape the conditions under which they are governed, a problem Wiener identified in his analysis of automation and that Fung has adapted to the specific features of contemporary AI.

Key Ideas

AI reshapes the conditions of its own governance. Unlike previous technologies, AI does not merely require governance — it actively shapes the institutional environments within which governance decisions are made.

The feedback loop is self-reinforcing. Degradation of democratic mechanisms reduces governance capacity, enabling further degradation in a cycle that accelerates absent structural intervention.

Conventional regulation assumes stable governance. Most regulatory frameworks presuppose that the institutions doing the regulating will continue to function as designed — an assumption the recursive trap invalidates.

Deliberation is structurally resilient. Face-to-face participatory processes operate outside the algorithmic environment, giving AI's degradation mechanisms no foothold.