Bonferroni correction
Divide your significance threshold by the number of tests you run.
What it means
The Bonferroni correction is a simple method for controlling the family-wise error rate by dividing the desired significance level by the number of comparisons, so each test must clear a stricter bar. It guarantees that the probability of even one false positive across the whole family stays below the target level, regardless of dependence between tests. Its virtue is conservatism and ease; its cost is a sharp loss of power when the tests are many or correlated, which can bury real effects. For that reason it is often replaced by less stringent procedures such as Holm's step-down method or false-discovery-rate control.
How it works
The guarantee rests on Boole's inequality: the chance that at least one of m tests yields a false positive can never exceed the sum of their individual false-positive rates. If every test runs at level alpha/m, that sum is exactly alpha, so the family-wise error rate is capped no matter how the tests relate to one another. This is why the method needs no assumption about independence — the union bound holds even for perfectly correlated tests. In practice you either shrink each threshold to alpha/m or, equivalently, multiply every raw p-value by m and compare it against the original alpha. The arithmetic is trivial, which is much of its appeal: any reader can check the adjustment by hand.
The price in power
Conservatism is the flip side of that guarantee. Because the union bound is loose whenever tests overlap, correlated comparisons drive the true error rate well below alpha, so the threshold sacrifices more power than the target requires. As the number of tests climbs, alpha/m shrinks toward zero: a scan of thousands of markers pushes the bar to the vanishing point, and even a modest battery of twenty outcomes can bury a genuine medium-sized effect. The cost lands as Type II errors — real differences ruled non-significant — often in exactly the thorough studies that run the most tests. Sidak's variant, 1-(1-alpha)^(1/m), recovers a sliver of power under independence, but the gain is usually negligible.
Sharper procedures
Two refinements dominate modern practice, but they improve on plain Bonferroni in different ways: Holm gives up nothing Bonferroni guarantees, while Benjamini and Hochberg's method instead trades that family-wise guarantee for a weaker but far more powerful target. Holm's 1979 step-down method sorts the p-values, tests the smallest against alpha/m, the next against alpha/(m-1), and so on until one fails; it controls the same family-wise rate under any dependence yet rejects at least as much, so there is little reason to prefer plain Bonferroni. Benjamini and Hochberg's 1995 procedure changes the target itself, bounding the false-discovery rate — the expected share of mistakes among the results you call significant — rather than the chance of any single error. That trade is far more powerful when tests number in the thousands, which is why it, not Bonferroni, is standard in genomics, neuroimaging, and other large-scale screens.
Whether to correct at all
A quieter debate questions the whole exercise. Perneger (1998) and Rothman (1990) argued that the 'family' of tests is arbitrary — nothing in the data tells you where it begins or ends — so a single result's meaning should not hinge on how many other questions the analyst happened to ask. Blanket correction also penalizes thoroughness, turning careful, wide-ranging work into a thicket of non-significant findings. The usual resolution is to separate roles: pre-specified, confirmatory hypotheses warrant strict family-wise control, while exploratory analyses are better reported transparently — every test disclosed, effect sizes and intervals shown — and treated as hypothesis-generating rather than adjusted into silence. Bonferroni answers a precise question well; the harder task is deciding when that question is the one you actually have.
Examples
Running ten tests and wanting overall alpha = 0.05, each result is judged significant only if its p-value falls below 0.005.
A team runs one landing-page test but reads twenty metrics off it. Bonferroni says nothing counts as a win unless its p-value beats 0.0025 — otherwise roughly one 'win' is pure noise.
A drug shows no overall effect, so the sponsor tests six subgroups. Each now needs a p-value under 0.008, a bar the promising result in older left-handed men does not clear.
A genome-wide association study screens a million genetic variants against a disease. Bonferroni logic sets the bar near 5e-8, so only variants far beyond the usual 0.05 count as real hits.
An fMRI study tests 100,000 brain voxels for activation. At 0.05 many would light up by chance; a Bonferroni threshold near 0.0000005 removes almost all false blips, and some weak real ones too.
First described in Named for Carlo Emilio Bonferroni; popularized by Dunn (1961).
Key references
- Nakagawa, S. (2004). A farewell to Bonferroni: the problems of low statistical power and publication bias. Behavioral Ecology, 15(6), 1044-1045. doi.org/10.1093/beheco/arh107
- Perneger, T. V. (1998). What's wrong with Bonferroni adjustments. BMJ, 316(7139), 1236-1238. doi.org/10.1136/bmj.316.7139.1236
- Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society: Series B (Methodological), 57(1), 289-300. doi.org/10.1111/j.2517-6161.1995.tb02031.x
- Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6(2), 65-70. www.jstor.org/stable/4615733
- Dunn, O. J. (1961). Multiple comparisons among means. Journal of the American Statistical Association, 56(293), 52-64. doi.org/10.1080/01621459.1961.10482090