Testing Is Not Verification

The difference between testing and verification is the difference between evidence and proof, and the industry runs almost entirely on evidence. A program that has passed a thousand tests may pass the thousand-and-first; it may also fail catastrophically on it. The number is irrelevant. Testing is induction — generalizing from observed cases to unobserved cases — and induction can fail. Verification is deduction — deriving conclusions from premises with logical necessity — and deduction cannot fail, provided the premises are true and the reasoning is valid. Dijkstra stated the distinction in a sentence so quotable that it has become a platitude, which is the worst thing that can happen to a theorem. Platitudes are nodded at. Theorems are acted on. The sentence is a theorem.