Learning Confound
The learning confound is the second of Klein's three diagnostic categories for evaluating claims of AI superiority over experts. The confound operates at the structural level of study design: the AI system is designed to learn from the data, given access to large datasets and computational resources to identify patterns no human could detect through unaided cognition. The human experts, by contrast, are typically evaluated on their existing knowledge without comparable learning opportunities. They are not shown the data the algorithm learned from. They are not given time to study the patterns the algorithm identified. They are tested cold, on their clinical judgment as it stands at the moment of evaluation, against a system optimized specifically for the task at hand. The comparison's unfairness is so stark Klein argues it should disqualify the conclusions drawn from it — but such studies are regularly published, cited, and used to justify organizational decisions about AI deployment and reduction of human expert involvement.
In the AI Story
The confound's methodological significance extends beyond individual study designs. It reflects a deeper asymmetry in how AI and human capability are typically compared: AI is evaluated at its performance peak following optimization on task-specific data, while humans are evaluated on existing capabilities without task-specific preparation. The asymmetry creates a systematic bias toward AI superiority findings that does not reflect the relative capabilities of humans and AI under comparable conditions.
The concept connects to Klein's broader critique of how the AI discourse evaluates human expertise. The discourse systematically compares AI at its best against human performance under constraints AI does not face, while simultaneously obscuring the degree to which AI performance depends on the human expertise it is being used to replace. The learning confound is one specific mechanism through which this comparison asymmetry operates.
Klein's prescription for addressing the confound is to design evaluation studies that give humans comparable learning opportunities — access to the data the algorithm learned from, time to study the patterns the algorithm identified, preparation specific to the task being evaluated. Studies designed this way typically produce results that differ substantially from those of standard designs, often showing that properly prepared humans match or exceed AI performance on tasks where the asymmetric comparison had shown AI superiority.
The confound has practical implications for organizational decisions about AI deployment. Organizations evaluating whether to replace or augment human experts with AI systems should be skeptical of comparison studies that did not give humans comparable learning opportunities to those given the AI. The question is not whether AI can outperform unprepared humans — that is often true — but whether AI outperforms humans given the preparation and data access AI takes for granted.
Origin
Klein identified the concept in his February 2024 essay on exaggerated claims for AI superiority, alongside smuggled expertise and big-data intimidation. The framework built on his decades of experience evaluating studies of human performance under conditions where methodological design heavily influences apparent conclusions.
The concept connects to broader literatures on the ecology of comparison in cognitive science — the recognition that evaluative comparisons presuppose choices about what conditions the compared entities face, and that these choices shape apparent results in ways that are often invisible to readers.
Key Ideas
Structural asymmetry. AI is given task-specific learning opportunities humans are denied.
Performance-peak comparison. AI is evaluated at its optimized peak; humans at their existing baseline.
Disqualifying methodology. The asymmetry is severe enough to undermine conclusions drawn from such studies.
Comparable-conditions prescription. Proper evaluation requires giving humans learning opportunities comparable to those given the AI.
Organizational decision implications. Deployment decisions should account for the asymmetry in evaluation studies.
Appears in the Orange Pill Cycle
Further reading
- Klein, G. (2024). Spotting exaggerated claims for AI superiority over experts. Psychology Today, February.
- Rajpurkar, P., et al. (2022). AI in health and medicine. Nature Medicine, 28, 31–38.
- Topol, E. (2019). Deep Medicine: How Artificial Intelligence Can Make Healthcare Human Again. Basic Books.