Behavioral Science Dictionary

Insensitivity to sample size

Heuristics & Biases

Ignoring that small samples swing far more wildly than large ones.

What it means

Insensitivity to sample size is the failure to take the number of observations into account when judging the reliability or likelihood of a statistical result, so people treat outcomes from small and large samples as roughly equally informative. The mechanism is again representativeness: judgments are anchored on how well a result matches an intuitive prototype (such as a 50–50 split of boys and girls) while the crucial fact — that small samples produce extreme deviations from that prototype much more often — is neglected. This is a close cousin of the law of small numbers, but framed around the specific blindness to how variance shrinks as n grows. A boundary worth noting is that the error softens when sample size is made vivid or the question is reframed in frequencies rather than abstract probabilities, suggesting the problem is partly one of presentation. It matters because the same blindness underlies overreaction to short-term data, misjudged medical and quality-control statistics, and the seductive but false belief that a small but 'clean-looking' result is trustworthy.

Examples

Asked which hospital records more days when over 60% of newborns are boys, most people say 'about the same' — yet the smaller hospital does, by a wide margin, because small samples fluctuate more.

A team declares a new checkout button the winner after forty visitors, when a gap that size turns up routinely at such a sample and evaporates once thousands arrive.

Small schools crowd the top of exam league tables — and the bottom of them too. The lesson people draw is about small schools, not about small samples.

First described in Kahneman & Tversky (1972).

← All 1001 terms