CONCEPT
The Testimony Paradox of AI Collaboration
Machine output is most reliable in domains where the human can already evaluate it and least reliable where she most needs help—the structural inversion at the heart of human-AI epistemic trust.
Paul Ricoeur analyzed testimony as a speech act requiring the witness’s attestation—the staking of personal credibility on a claim. The witness who testifies puts something at risk: a history of reliability that can be damaged by error. Machine output lacks this attestation structurally. When a large language model produces a confident summary, it risks nothing, because it possesses no credibility that can be eroded by being wrong. The testimony paradox follows from this asymmetry: the builder who asks the machine for help in a domain she already understands can evaluate the output critically, catch errors, and benefit from the collaboration. The builder who asks for help in a domain she does not understand—which is frequently the entire point of asking—cannot evaluate the output, cannot catch the errors, and is maximally vulnerable to what [YOU] on AI calls “confident wrongness dressed in good prose.” The paradox is not a defect to be engineered away but a structural feature of any
Keep reading with YOU ON AI
Unlock the full book, 10,000+ field-guide entries, and a 1000+ thinker library. If you have a book code, register now — it takes a minute.