CONCEPT
Distributed Systems
Systems of communicating processes with no shared memory and no shared clock, whose correctness cannot be verified by inspecting any single node—the infrastructure substrate on which every large-scale AI is built and the domain whose hardness Lamport's logical clocks first made precise.
A distributed system is a collection of independent processes that communicate by passing messages, appear to users as a single coherent system, and must remain correct even when individual components fail or messages are delayed. The defining challenge is that there is no global shared state: each process sees only what it has received, and events that are simultaneous on a wall clock may be causally unordered, while events that are causally ordered may appear simultaneous.
Leslie Lamport formalized this in 1978 with logical clocks and the happens-before relation, giving distributed systems their mathematical foundation. The CAP theorem (Brewer 2000, Lynch and Gilbert 2002) proved that no distributed system can simultaneously guarantee consistency, availability, and partition tolerance—a fundamental impossibility result that governs every design decision in cloud infrastructure. Every
large language model is served across distributed infrastructure governed by these constraints: inference requests are routed to machines that may fail, state must be synchronized across