Confirmation Holism

In the [YOU] on AI Field Guide

Confirmation holism frames one of the most practically urgent questions in AI deployment: how much can we trust a model’s outputs on individual claims, given that those outputs are generated by a holistic system in which no belief is individually certified? The cycle’s answer, grounded in Quine’s framework, is that a model output is evidence about a process—a process that is generally reliable in certain domains and unreliable in others—not a window onto a single belief the system “holds.” The appropriate response to a model claim is not to ask whether this belief was individually confirmed but to ask whether the process that generated it is reliable for this kind of claim in this kind of domain, which is an empirical question about the training data and the distribution.

Holism also explains the phenomenon the cycle calls the “fluent error”—a model that is confidently wrong about a specific fact in a domain where it is generally reliable. In a holistic system, a correct general framework can generate locally incorrect outputs when the relevant evidence was sparse or absent in the training data, and no internal signal distinguishes the well-grounded from the poorly-grounded claim. The model cannot feel the difference, because there is no single belief with a credence attached. There is only the whole system, generating with equal fluency from areas of high evidential support and areas of near-void.

Origin

Quine developed confirmation holism in “Two Dogmas of Empiricism” (1951) as the consequence of rejecting both the analytic-synthetic distinction and the dogma of reductionism. The logical positivists had held that each meaningful sentence has its own empirical content—its own test, its own confirmation conditions. Quine showed that this picture was incoherent: any sentence can be held true in the face of recalcitrant experience if we are willing to make adjustments elsewhere in the web, and any sentence can be revised if the cost of keeping it is sufficiently high. The pragmatist point is that revision decisions are a matter of economy and strategy, not logic. The sentence that seems most obviously empirical can be insulated from experience by adjustments to auxiliary hypotheses; the sentence that seems most obviously necessary can be given up if it costs too little elsewhere.

The holism is not total or unconstrained. Quine held that pragmatic virtues—simplicity, conservatism, breadth, fecundity—discipline the choice among revision strategies. We prefer to preserve core beliefs when peripheral revision suffices; we prefer simpler webs when revision is forced; we prefer to keep what is most widely connected because it is most costly to change. These virtues are what make holism a discipline rather than a license for arbitrariness.

Key Ideas

The web and its edges. Knowledge is a vast web of mutually supporting commitments in which experience enters only at the periphery, through observation and experiment. In a trained neural network, training data plays the role of experience at the edges; the deep layers of broadly useful representations correspond to the interior commitments that are most expensive to revise; and the fine-tuned upper layers correspond to the peripheral beliefs most responsive to new data.

No immunity, no foundation. Quine’s most radical claim was that no statement is immune to revision—not the laws of logic, not the axioms of arithmetic, not the most entrenched empirical generalizations. In a network, everything was learned from data; nothing is held as sacrosanct; the system has no internal bedrock it can appeal to when its commitments conflict. When a model contradicts itself or asserts something false with total fluency, there is no analytic core inside it that “knows better” and could in principle override the error.

Catastrophic forgetting as Quinean prediction. When a neural network is fine-tuned aggressively on new data, it can lose old competences in domains unrelated to the new task. This is catastrophic forgetting, and it is the specific disaster Quine’s picture predicts when revision is allowed to penetrate the deep interior: the widely-shared representations that underpin many distant outputs are disrupted, and the distant outputs fail. Regularization techniques that prevent catastrophic forgetting—elastic weight consolidation, rehearsal, freeze-then-fine-tune—are, in Quine’s terms, attempts to confine revision to the periphery rather than letting it tear through the core.