CONCEPT
The Substitution Fallacy
The assumption that because an AI system can produce the same output as a human expert, the machine has replicated the expert's knowledge — a category error that confuses the product of expertise with the developmental process that produced it.
The substitution fallacy is Flyvbjerg's name for the dominant cognitive error in contemporary AI discourse: the assumption that functional indistinguishability between machine output and human output entails substantive equivalence between machine capability and human capability. The AI system that writes code, drafts legal briefs, or generates medical diagnoses is performing operations previously performed by human experts. The output may be functionally identical to what the expert produced. But the process by which it was produced is fundamentally different — pattern-matching across training data versus embodied knowledge built through years of practice — and the difference matters because the developmental process, not the output it produces, is the substrate on which phronesis is built.
In The You On AI Field Guide
The fallacy operates at two levels. At the individual level, it treats the production of expert-like output as evidence that the machine has replicated expert knowledge — ignoring the fact that expert