WORK
Collins and Thorne (2026)
Collins and Simon Thorne's 2026 arXiv paper testing large language models against the specific discourse practices of gravitational wave physicists — the empirical demonstration that LLMs cannot reproduce reasoning dependent on collective tacit knowledge acquired through social participation.
The 2026 paper represents Collins's most direct empirical engagement with the AI moment. Working with Simon Thorne, Collins tested whether large language models could reproduce the reasoning that gravitational wave physicists use when evaluating fringe science claims — specifically, the community's practice of deciding to ignore a particular unconventional paper without extensive engagement. The physicists could articulate reasoning that drew on the community's collective tacit knowledge: reputation assessments, historical pattern-matching against similar claims, implicit standards for what evidence would justify serious engagement. The language models could not reproduce this reasoning. They produced plausible-sounding arguments that lacked the specific social grounding that made the physicists' judgments authoritative.
In The You On AI Field Guide
The paper's methodological innovation is its specificity. Rather than asking whether LLMs can 'reason about physics' in general — a question that invites fuzzy answers — Collins and Thorne asked whether LLMs can reproduce a specific form of social reasoning