By Edo Segal
The accident nobody caused is the one that keeps me up now.
Not the dramatic failure — the server crash, the botched deployment, the product that ships broken and gets fixed by Monday. Those have authors. You can trace them back to a decision, a person, a moment where someone chose wrong. They hurt, but they make sense. You learn from them the way you learn from touching a hot stove.
The failure I cannot stop thinking about is the one that emerges from the system working exactly as designed. Every person doing their job well. Every tool performing as specified. Every process followed correctly. And the catastrophe arriving anyway, not because anyone failed but because the architecture of the system — its complexity, its speed, its tight coupling
A reading-companion catalog of the 45 Orange Pill Wiki entries linked from this book — the people, ideas, works, and events that Charles Perrow — On AI uses as stepping stones for thinking through the AI revolution.
Open the Wiki Companion →