CONCEPT
Predictive Harm Invisibility
The structural property of predictive AI harms that makes them undetectable by the standard instruments of accountability: because the thing being predicted has not yet happened and may never happen, the model's errors are hidden indefinitely behind the irreducible uncertainty of the future.
When a hiring algorithm rejects a candidate, no one ever learns whether that candidate would have excelled, because the candidate was never hired. When a recidivism-prediction tool labels a defendant high-risk and the defendant is incarcerated longer, the counterfactual in which she was released and did not reoffend cannot be observed. When a healthcare model predicts that a patient needs fewer days of care and the insurer acts on the prediction, the patient's true recovery trajectory is foreclosed by the intervention. In each case, the predictive system's error is not merely hidden from external observers but structurally unfalsifiable in practice: the error would only become visible in a world that the prediction itself has prevented from occurring. Arvind Narayanan's analysis of predictive AI identifies this property—which the cycle calls predictive harm invisibility—as the mechanism that makes the snake oil of
predictive AI so durable. Snake oil in general survives because the buyer