CONCEPT
Smuggled Expertise
Klein's term for the human judgment embedded in AI training data that the system then appears to have generated itself — the structural reason AI-versus-expert comparisons are methodologically unfair.
Smuggled expertise names the structural feature of any AI system trained on expert-generated data: the system's performance incorporates the judgment of the humans whose work constitutes the training corpus. Medical records written by physicians, legal briefs drafted by lawyers, code written by engineers — in each case, the data is not raw observation but the product of human cognition. Every data point reflects a clinical decision, an engineering judgment, a lawyer's assessment of relevance. When the AI is then evaluated against human experts, the comparison is structurally unfair: the system is being measured against the people whose judgment it has already absorbed. Klein identifies this problem as one of three methodological failures that make claims of AI superiority over experts systematically unreliable, the other two being learning confounds and big-data intimidation.
In The You On AI Field Guide
Klein's February 2024 essay dissecting an emergency department prediction study made the concept operational. The algorithm was trained on electronic health records — records that