CONCEPT
Robust Statistics in AI
John Tukey's program of methods that survive bad data—estimators that degrade gracefully when assumptions are violated rather than collapsing on first contact with outliers—translated into the urgent engineering challenge of building AI systems that are not destroyed by the messy, contaminated data they actually train on.
Robust statistics is the systematic study of methods that continue to perform well even when their assumptions are violated—when the data contains outliers, errors, contamination, or departures from the idealized model that standard procedures require.
John Tukey was a founding force in this program, developing through the 1960s and beyond estimators that bound the influence of any single observation, so that a wild value contributes less and less rather than more and more, and that degrade gracefully under contamination rather than collapsing. The central intuition was realist rather than idealist: any method that assumes its data is clean has already failed, because data is never clean. Classical statistics had been built on convenient fictions—normally distributed errors, correctly specified models, data that is what it claims to be—and Tukey's robust program began from the observation that all of these are routinely false, and asked how to do good work