The Knowledge Problem in AI

In the AI Story

Hayek distinguished between scientific knowledge (general principles) and knowledge of particular circumstances (local, timely, contextual). The latter is more important for coordination but cannot be centralized. A price change transmits the fact that circumstances have changed without requiring anyone to know what changed. An AI model trained on programming forums learns patterns of debugging advice but loses the context—this error in this codebase given this architecture with this team's constraints. When a user prompts the model, she must supply context the model cannot generate. If she knows which context matters, the model is useful. If she doesn't, the model produces plausible outputs that fail when they meet the particular circumstances the user didn't specify.

Sowell's extension adds institutional and incentive analysis. Centralized knowledge systems—whether Soviet planning bureaus or AI training pipelines—concentrate power in the hands of those who control the aggregation mechanism while claiming to serve those whose knowledge was aggregated. The training corpus is humanity's textual output, created by millions for diverse purposes, now enclosed and deployed by a few companies optimizing for engagement and revenue. The dispersed contributors cannot negotiate terms, cannot withdraw consent retroactively, and capture almost none of the value their contributions generate. This is not merely an economic extraction but an epistemological transformation—knowledge created in situated contexts, for situated purposes, is redeployed in decontextualized form by systems that do not (cannot) understand what made it valuable.

The practical consequence is the capability gap. The developer with twenty years of experience using Claude Code gets superior results because she possesses the dispersed knowledge of this system, this team, this set of constraints—knowledge she uses to direct the tool, supply missing context, and evaluate outputs. The developer with two years of experience gets syntactically correct code that compiles and runs but may fail under conditions she didn't know to specify. The tool democratizes access to patterns while amplifying the value of situated knowledge—widening the gap between those who possess contextual expertise and those who do not. This is Hayek's knowledge problem at the individual scale: the tool's value depends on knowledge the tool itself cannot supply.

Origin

Hayek formulated the knowledge problem during the socialist calculation debate of the 1930s–1940s, arguing against Ludwig von Mises's critics that central planning was impossible not for political reasons but for epistemological ones—the knowledge required did not exist in centralizable form. "The Use of Knowledge in Society" (1945) became the canonical statement. Sowell encountered Hayek's work as a Chicago graduate student and spent the next fifty years applying it beyond economics to courts, schools, bureaucracies, and families. The AI application was inevitable once language models began aggregating dispersed textual expertise—the training process is literal centralization of knowledge at a scale that would have astonished Hayek, and his framework predicts its characteristic failures with uncomfortable precision.

Key Ideas

Situated knowledge cannot be fully centralized. The most valuable information is contextual, particular, timely—knowledge of this circumstance at this moment—which loses value when extracted from the knower's position.

Patterns are not knowledge. AI learns statistical regularities from aggregated expertise but cannot replicate the situated judgment that determined which pattern applied in which context.

Context supply requires expertise. Effective AI use depends on the user knowing which context matters—a knowledge requirement that tools cannot eliminate, only relocate.

Aggregation concentrates power. The companies controlling AI training and deployment capture value created by millions whose dispersed contributions cannot be individually negotiated or compensated.

The capability gap widens. Tools appearing to democratize access actually amplify returns to situated expertise—those who already know direct tools better, compounding existing knowledge inequality.