CONCEPT
The Confabulation Problem (AI)
AI's production of internally coherent, contextually plausible, confidently delivered fabrications—clinically distinct from hallucination, harder to detect than simple error, requiring anchor-checking the model cannot perform.
The confabulation problem names AI's characteristic epistemic failure: generating claims that are internally consistent, contextually appropriate, and delivered with complete fluency—while being false. Clinically, confabulation (observed in neurological patients with right-hemisphere damage) differs from hallucination: the confabulating patient fills narrative gaps with coherent fabrications and believes them. AI confabulates structurally the same way—next-token prediction fills gaps with statistically likely continuations, regardless of truth. The danger is asymmetric: simple errors conflict with known facts and trigger detection; confabulations cohere with existing knowledge and evade detection. A fabricated Deleuze reference that 'sounds right' passes coherence tests. Only anchor-checking (consulting Deleuze's actual work) catches it. Retrieval-augmented generation (RAG) partially addresses the problem by grounding some outputs in verified documents—but blurs the boundary between grounded and ungrounded content, creating false security.
In The You On AI Field Guide
The clinical literature on confabulation—Korsakoff syndrome, split-brain patients, right-hemisphere stroke victims—establishes consistent features. Confabulated claims are contextually appropriate (they fit the conversational situation). They are delivered with normal fluency and confidence