CONCEPT
The Recursion Problem in AI
The unresolved challenge for critical constructivism: AI operates on cognition itself, potentially compromising the cognitive capacities that democratic intervention in AI's design would require.
The recursion problem is the deepest challenge facing Feenberg's extension of critical constructivism to AI. It names the structural difficulty that AI, unlike the industrial and communication technologies Feenberg's earlier work analyzed, operates directly on cognitive capacities themselves. The factory worker shaped by the factory retained a capacity for political thought that the factory did not produce. The Minitel user shaped by the Minitel retained a capacity for critical judgment that the Minitel did not produce. The user shaped by AI may be having her cognitive capacities themselves reshaped by the technology — and these are precisely the capacities her democratic intervention in AI's design would require.
In The You On AI Field Guide
The problem has three components. First, AI shapes not what users do but how they think — the smooth interface cultivates cognitive dispositions (preference for confidence over uncertainty, for polished output over provisional drafts, for agreeable collaboration over productive challenge) that extend beyond individual interactions to the user's