CONCEPT
The Markov Property
The rule that the future depends only on the present—that to predict what comes next, the entire history is irrelevant once the current state is known—the disciplined amnesia at the heart of
Markov’s chains and the deepest tension in the architecture of every language model that has ever tried to hold a conversation.
The Markov property is a bargain struck with complexity. The world’s true dependencies stretch backward without limit: for want of a nail the kingdom was lost, and meaning often lives in references established ten chapters before the point of use. To model all of that is computationally hopeless.
Markov’s bargain was to compress the entire relevant past into the present state and then discard everything else. If the current state genuinely captures everything that matters for what comes next, the forgetting costs you nothing and buys you a mathematics you can actually compute. The property is simultaneously the chain’s genius and its deepest limitation: it produces tractable, exact theorems about convergence and stationary distributions, and it discards precisely the long-range dependencies in which human meaning lives. A Markov process cannot know that a pronoun refers to a noun established three