Simpson's paradox
Also known as: Yule-simpson effect, Reversal paradox
A trend in every subgroup can vanish or flip when the groups are combined.
What it means
Simpson's paradox is the phenomenon in which an association observed within each of several subgroups reverses or disappears when the subgroups are pooled. It arises because a lurking variable is unevenly distributed across the groups and is related to both variables of interest. The 'paradox' is really a warning that the correct answer depends on the causal structure: which version — aggregated or disaggregated — is right is determined by what is confounding what, not by the numbers alone. Causal graphs resolve it by indicating whether the grouping variable should be conditioned on.
Examples
A drug looks worse than placebo overall yet better in both men and women separately, because more severe cases happened to receive it.
The classic case: Berkeley's 1973 graduate admissions looked biased against women overall, yet most departments admitted women at a slightly higher rate. Women applied more often to the hardest-to-enter departments.
A country's median wage can rise in a recession while every education group's median wage falls, simply because the lowest-paid workers are the ones losing their jobs and dropping out of the count.
First described in Edward Simpson (1951); anticipated by Yule (1903) and Pearson.