The Five Tribes of Machine Learning

Origin

Domingos introduced the five-tribe framework in The Master Algorithm (2015), where it served as the organizing structure for a book aimed at readers without technical backgrounds. The framework was not invented to simplify; it crystallized an observation Domingos had been developing across decades of research: that the apparently competing schools within machine learning were each capturing something real, and that the competition was therefore a division of labor rather than a race to be won by one side.

The framework draws on the genuine intellectual lineages of the field. The symbolist tradition runs from the logic programming and expert systems of the 1950s through 1980s through modern inductive logic programming. The connectionist tradition runs from the perceptron through the backpropagation revolution of 1986 through modern deep learning. The evolutionary tradition runs from John Holland’s genetic algorithms through John Koza’s genetic programming. The Bayesian tradition runs through Thomas Bayes’s theorem, Pierre-Simon Laplace, and the modern probabilistic graphical model lineage through Judea Pearl. The analogizer tradition runs through nearest-neighbor methods through Vladimir Vapnik’s support vector machines.

Pedro Domingos

The practical consequence of the framework was to reframe a field defined by tribal rivalry as one seeking complementary contributions. Where the symbolists see connectionist opacity as a fundamental flaw, and the connectionists see symbolist brittleness as a fundamental flaw, Domingos sees both as partial visions of a deeper structure—and the master algorithm as the structure that reconciles them.

Key Ideas

Symbolists: Logic as Learning. The symbolist insight is that knowledge is compositional and representable as structures that can be combined and recombined. Learning is inverse deduction: given observations, find the rules that would have produced them. The symbolist virtue is transparency—a learned rule can be read, audited, and reasoned about further. The symbolist weakness is brittleness: the real world refuses to be clean, exceptions multiply faster than rules can absorb them, and the crisp logic cracks under the messiness of real data.

Connectionists: The Triumph of the Network. Neural networks learn by adjusting weights in response to error, propagating credit and blame backward through layers via backpropagation. Their strength is the handling of raw, noisy, high-dimensional data where symbolist rules would fail. Their weakness is opacity: the knowledge is distributed across millions of weights, the system cannot explain its conclusions, and the brittleness that results is hidden until failure. All of today’s most celebrated systems are connectionist triumphs—and, Domingos insists, still pattern-matchers without comprehension.

Evolutionaries, Bayesians, Analogizers. The evolutionaries are unmatched at discovering structure but undirected and slow. The Bayesians handle uncertainty with rigor but are burdened by computational cost; their modern lineage includes Bayesian networks and the entire probabilistic graphical model tradition. The analogizers generalize gracefully but lack a theory of representation. Each strength is shadowed by a corresponding weakness that is precisely the strength of another tribe.

The Complementarity Argument. The key claim is not merely that the tribes have different strengths but that the strengths and weaknesses are structured complements. The evolutionaries’ undirectedness is cured by Bayesian inference; Bayesian intractability is eased by geometric kernel methods; the analogizers’ lack of structure is supplied by symbolist logic. This nested pattern of mutual complement suggests that the five are components of one answer rather than five competing answers—and that a learner combining all five might have no walls at all.

John Holland