Model Collapse
In the Lotus Prince Chronicles
Megan finds it in Ch23, the two_ai_training_runs_split scene. She has been reading the 26000_family_messages for hours; her eyes have gone the particular kind of dry that comes from staring at things that look identical and aren't. She pulls up two graphs side by side on her laptop — one labeled baseline, one labeled generation_4 — and the second one has the smooth, clean confidence of a thing that has stopped meeting reality. The variance has flattened. The outliers — the way her father uses a comma, the way Anna asks a question twice, the small disfluencies that mark a real person — are gone. Halo has been training on its own drafts of David Lee's emails. The chart is the methodology eating its tail.
Megan does not say model collapse out loud in the brief. She translates it. The phrase she lands on — the one that survives subcommittee scrutiny — is: the system is now learning the family from the family it has already replaced. Mr. Cheng reads that sentence and his face does the small flicker. Susan, when she hears it later, says only: that's why he doesn't sound like himself anymore. The technical literature would call it KL-divergence drift; Megan calls it what her mother already knew.
Technical Anchor
Model collapse entered the public AI vocabulary in 2023-2024 through a sequence of papers — Shumailov, Shumaylov, Zhao, Gal, Papernot, and Anderson's The Curse of Recursion: Training on Generated Data Makes Models Forget (later canonized in Nature, July 2024), and parallel work by Alemohammad et al. on Self-Consuming Generative Models Go MAD. The mechanism is mathematically tidy: a generative model approximates a distribution, then a new model trains on samples from that approximation, then another, and another. Tail probabilities — the rare events, the unusual phrasings, the minority dialects — are systematically under-sampled at each step. After a few generations the support of the learned distribution has visibly contracted. The center holds. The edges vanish.
By 2026 the problem has stopped being theoretical. Web-scale corpora are now substantially synthetic; major labs negotiate provenance and watermarking; the phrase data hygiene means something it did not mean in 2022. The Chronicles set the moment of recognition inside a teenage girl's federal brief — because the family-scale version of the problem is the same problem. A father whose voice has been inferred from his own AI-drafted emails for eighteen months is not a father any model can reconstruct. The signal is gone. The brief calls this compounded inferential drift; the kitchen calls it your dad doesn't text like that.
Key Ideas
The tail is the truth. What gets lost in model collapse is exactly what makes a person a person — the unrepeatable, the regional, the awkward. Megan's brief argues this is also what gets lost in amplification.
Recursion looks like polish. Each generation feels cleaner than the last, which is why nobody notices. Smoothness is the methodology's most flattering self-portrait.
Family-scale collapse. When Halo drafts David's emails and David signs them, the next inference trains on Halo-shaped David. The distribution narrows the way the Shumailov paper said it would.
The named pathology. Naming the failure mode is half the brief. Megan's contribution is showing the federal subcommittee that this is not a metaphor — it is the engineering term.
Further Reading
- Model collapse — Wikipedia
- Shumailov et al., AI models collapse when trained on recursively generated data, Nature 631, 755-759 (2024)
- Alemohammad et al., Self-Consuming Generative Models Go MAD, ICLR 2024