The Paint Color Anecdote

In The You On AI Field Guide

The anecdote's methodological power lies in the asymmetry it exposes. Paint-color matching is the kind of task that should be easy for AI — a narrow, specifiable operation with an empirically testable outcome. The system's failure at this task, while it continues to project confident authority on harder tasks, reveals the structural problem: AI systems do not distinguish between domains where they are competent and domains where they are not. They produce the same fluent, confident output regardless of underlying accuracy.

The contrast with human expertise is instructive. A painter asked to match a color will typically acknowledge uncertainty if the sample is degraded, the lighting is unusual, or the target is ambiguous. The acknowledgment of uncertainty is itself expert behavior — a phronetic signal that the expert calibrates to context. AI systems do not produce such signals because they do not possess the internal epistemic states that would ground them. The absence of uncertainty signals is not a feature to be added later but a structural consequence of the generation architecture.

The anecdote complements the Big Dig test by covering the opposite end of the stakes spectrum. The Big Dig test exposed the problem at the professional knowledge level — a question a scholar might plausibly outsource to an AI. The paint-color test exposes the problem at the consumer level — the millions of small daily decisions that AI tools are marketed to assist. The failures are structurally identical. The lesson is that the failure mode is general, not specific to any particular domain or stakes level.

The industrial parallel Flyvbjerg draws — Mercedes' product liability concerns about ChatGPT integration into vehicles — demonstrates that the problem is visible to organizations whose deployment decisions carry direct legal and safety consequences. The automotive industry has proceeded with a caution that the broader culture has not matched, precisely because the industry cannot afford to treat AI confident wrongness as an acceptable operational cost.

Origin

Flyvbjerg recounted the anecdote in his 2025 AI writings and subsequent social media posts accompanying the release of 'AI as Artificial Ignorance.'

Key Ideas

Narrow task failure. AI systems fail at tasks that should be easy for them while continuing to project confident authority on harder tasks.

Uniform confidence. The systems do not distinguish between domains where they are competent and domains where they are not — the confidence signal is decoupled from the accuracy signal.

Absence of uncertainty. Expert behavior includes calibrated uncertainty signals; AI systems do not produce such signals because they lack the internal epistemic states that would ground them.

Consumer-scale diagnosis. The paint-color failure demonstrates that the problem operates at the scale of ordinary daily decisions, not only in high-stakes professional contexts.

Industrial recognition. Sectors with direct liability exposure — automotive, medical, aviation — have recognized the problem and proceeded with corresponding caution.