CONCEPT
Intersectional Disaggregation
Timnit Gebru and Joy Buolamwini’s methodological principle that the design of an AI evaluation is itself a political act—and that a benchmark reporting a single aggregate accuracy score is structurally designed to hide the worst failures, which concentrate at the intersection of marginalized identities.
Aggregate accuracy is a way of hiding. A commercial face-recognition system that misclassifies darker-skinned women at thirty-four percent and lighter-skinned men at less than one percent can report an overall accuracy of ninety-eight percent if the test set is composed mostly of the latter group—and for decades, it was. The methodological contribution of
Timnit Gebru and Joy Buolamwini’s 2018 “Gender Shades” study was not merely the finding of disparate performance but the insistence on a different way of looking: not by overall accuracy but by the intersection of skin tone and gender, the axes along which harms actually compound. This is intersectional disaggregation—the practice of breaking aggregate AI performance metrics down by the crossing of multiple identity dimensions simultaneously, because harms that are invisible at every single axis often become clearly visible at their intersection. A system audited only for gender bias or only for racial bias will systematically miss the darker-skinned