Amplified Dysfunction

In the AI Story

The individual case is most visible in Segal's own foreword admission: writing past exhaustion, three in the morning over the Atlantic, unable to stop, recognizing the pattern as compulsion rather than creativity and continuing anyway. The tool was not forcing him to write; the tool was faithfully amplifying his already-present compulsive pattern. What would have been a late night became a working through the night became a no-longer-productive grinding that the tool supported with the same indifference it supported his earlier flow state. The tool does not distinguish between the two phases. Only the human could, and the human could not stop.

The organizational case is equally common but less widely discussed. Consider a company with poor product judgment — one that builds features no one needs, prioritizes engineering elegance over user value, optimizes for metrics that measure activity rather than outcome. This company adopts AI tools. The tools amplify the existing judgment, which is poor. The company builds more of the wrong things, faster, with more impressive-looking output. Quarterly metrics improve — more commits, more deployments, more velocity — while the underlying pathology (disconnection between output and value) deepens and becomes harder to see because the volume of output obscures its misdirection.

The institutional case is the most consequential. Educational systems that already teach to the test adopt AI and produce more sophisticated teaching to the test. Healthcare systems that practice defensive medicine adopt AI and produce more sophisticated defensive documentation. Legal systems that reward procedural complexity adopt AI and produce more sophisticated procedural filings. Each of these institutional pathologies is real, documented, and amplified by the tools that optimism promised would reform them.

The pattern's structural logic is Toyama's: amplification is faithful and indifferent. The optimistic claim — that the tool will force the institution to reform by making dysfunction visible or costly — has not been confirmed by evidence in previous technology transitions and is not being confirmed in the AI transition. Dysfunctional institutions do not become functional when given powerful tools. They become dysfunctional institutions with powerful tools, and the dysfunction is amplified by the power of the tool.

The response Toyama prescribes is the same response the corollary demands: reform the dysfunction first, deploy the tool only where the institutional capacity exists to absorb it productively. The alternative — which is the current default — produces the specific pattern Segal's foreword describes at individual scale, multiplied across every organization and institution in the economy.

Origin

The concept is implicit throughout Toyama's work and is developed explicitly in his AI essays from 2023 onward. It has resonance with Jenny Odell's work on the attention economy, Byung-Chul Han's on the burnout society, and the broader critical tradition that diagnoses how technologies of productivity intensify rather than resolve the pathologies they were supposed to address.

Key Ideas

Amplification is indifferent. The tool does not distinguish function from dysfunction; it carries whatever signal it receives.

Scale matters. Dysfunction at individual scale is manageable; dysfunction amplified across an organization or institution becomes structural.

The optimistic claim fails. The hope that tools would force institutional reform by making dysfunction visible has not been confirmed by evidence.

Pattern at every level. The dynamic appears in individual compulsion, organizational dysfunction, and institutional pathology, with the same mechanism.

Requires reform first. The response is to fix the dysfunction before deploying the tool that would otherwise amplify it.