The Designer's Blind Spot

The designer’s blind spot is Lisanne Bainbridge’s name for the most consequential error in the design of automated systems. The error is not technical; it is temporal. The designer of an automated system observes the current expert: her speed, accuracy, judgment, and ability to detect anomalies and respond to exceptions. The designer then creates a system that handles the routine work and assigns the human a supervisory role requiring those capabilities. The design is rational. The human possesses the required capabilities at the moment of deployment. The system should work. But the design does not account for what happens next: the capabilities the designer observed were built through practice, the automation removes the practice, and the human who exists after twelve months of automated operation is not the human the system was designed for. She is a different human, one whose capabilities have been systematically altered by the very system the designer built. The designer has modeled