Law of Small Numbers

Definition

People hold an intuitive but false belief that small samples are highly representative of their population. They expect small samples to exhibit the same statistical properties as large samples, and expect any two samples from the same population to be more similar to each other and to the population than statistical theory predicts. This leads to excessive confidence in research findings based on small N and to the gambler's fallacy: the mistaken belief that deviations from expected proportions will self-correct.

In the Book

Tversky and Kahneman document this through the lens of research practice. When they asked psychologists "If you ran a 20-subject study that achieved a significant result (t=2.23, p<.05), what's the probability a replication with 10 new subjects will also be significant?", the median answer was .85. The correct answer is approximately .48. Most scientists dramatically underestimate sampling variability because they intuitively apply the law of large numbers to small samples.

The authors show this manifests in multiple ways: when asked to generate a random coin-flip sequence, people produce sequences where heads and tails are far more evenly distributed than chance would predict (each short segment should "look fair"). When the first child in a sample of 50 has IQ 150 and the population mean is 100, many people still expect the sample mean to be 100—believing the process self-corrects. This belief has pernicious consequences: studies are designed with far too few subjects, power is dismally low, and researchers demand that replications be independently significant rather than evaluating cumulative evidence.

Why It Matters

This concept exposes how normative ideals (the law of large numbers is true) get misapplied to contexts where they don't hold. It explains why evidence thresholds feel unreasonably high to intuition yet are statistically appropriate, and why researchers trained in statistics still cling to intuitions about sample representativeness that formal theory contradicts. It grounds a fundamental gap between intuitive and statistical reasoning that appears across expertise levels.