Bayes' theorem
Also known as: Bayes' rule
The formula for updating a belief in light of new evidence.
What it means
Bayes' theorem expresses how to revise the probability of a hypothesis after observing data: the posterior probability is proportional to the prior probability times the likelihood of the data under that hypothesis. It is a logical consequence of the axioms of probability, yet it furnishes the entire engine of Bayesian inference and rational belief updating. The theorem makes vivid why base rates matter: a positive result from an accurate test can still leave a rare condition unlikely, because the small prior anchors the posterior. Its terms — prior, likelihood, posterior, and the normalizing evidence — are the vocabulary of coherent reasoning under uncertainty.
The parts, and the odds form
The theorem has four moving parts: a prior, the belief held before the data; a likelihood, how probable the data are if the hypothesis holds; the evidence, how probable the data are overall; and the posterior that results. In practice the tidiest way to run it is the odds form: posterior odds equal prior odds multiplied by the likelihood ratio, meaning how much more consistent the data are with one hypothesis than with its rival. That single multiplication turns updating into arithmetic, and because the output is again odds, evidence can be fed in piece by piece, each new fact scaling the running total. The normalizing denominator, which trips people up in the textbook fraction, cancels out entirely in this form.
Why people find it hard
The theorem is arithmetic, yet humans apply it badly. Casscells, Schoenberger and Graboys (1978) asked staff at Harvard teaching hospitals about a test with a five percent false-positive rate for a disease affecting one in a thousand; the most common answer was ninety-five percent, when folding in the base rate makes the true figure closer to two percent. The error is treating the likelihood as if it were the posterior, ignoring how few people actually have the condition. Gigerenzer and Hoffrage (1995) showed the failure is largely one of presentation: restate the same problem in natural frequencies, ten out of a thousand rather than percentages, and nearly half of respondents recover the Bayesian answer. The mathematics does not change; only the format that lets people see it does.
Where it shows up
Bayes' rule is the working core of a great deal of modern machinery. Spam filters score each message by combining a prior over junk with the likelihood of its words. Medical screening programs use it to convert a test's sensitivity and specificity into the only number a patient cares about, the chance of actually being ill. Courts weigh DNA and forensic matches through likelihood ratios that are Bayesian in form. Search-and-rescue teams have used Bayesian updating to locate lost ships and downed aircraft, revising a probability map of the seabed as each sweep comes back empty. In machine learning it underwrites classifiers, Bayesian networks, and the whole practice of updating model parameters as data arrive.
The part that stays contested
For a theorem, Bayes' rule has had an unusually turbulent history. Efron (2013) notes it has twice soared to scientific celebrity and twice crashed, and remains eyed warily by many statisticians. The sticking point is the prior. When genuine prior information exists, the update is uncontroversial; when it does not, the analyst must still choose a prior, and so-called uninformative priors are never truly neutral, since different default choices can move the posterior, especially when data are sparse. Frequentists object that this smuggles subjectivity into inference. Bayesians reply that the alternative merely hides its assumptions. The pragmatic modern stance, Efron's included, is to use Bayes where priors are real and to check its answers against other methods where they are not.
Examples
A test 99% accurate for a disease affecting 1 in 1,000 still yields a positive that is more likely false than true, once the base rate is folded in.
A spam filter multiplies your prior that a message is junk by how often words like 'invoice' and 'urgent' appear in junk versus real mail, then files it on the resulting posterior.
A smoke alarm going off in a block of flats rarely means fire, because kitchens burn toast constantly and buildings burn rarely; the prior does most of the work in your reaction.
A DNA match quoted at one in a million barely convicts a suspect pulled from a database of millions; the size of the searched pool, not the match alone, sets the odds of guilt.
A product team holding a prior that most tweaks do nothing treats one strong day of an A/B test as weak evidence, waiting for the likelihood to build before letting the result move the call.
First described in Thomas Bayes (1763); generalized by Laplace.
Key references
- Efron, B. (2013). Bayes' theorem in the 21st century. Science, 340(6137), 1177-1178. doi.org/10.1126/science.1236536
- Gigerenzer, G., & Hoffrage, U. (1995). How to improve Bayesian reasoning without instruction: Frequency formats. Psychological Review, 102(4), 684-704. doi.org/10.1037/0033-295X.102.4.684
- Casscells, W., Schoenberger, A., & Graboys, T. B. (1978). Interpretation by physicians of clinical laboratory results. New England Journal of Medicine, 299(18), 999-1001. doi.org/10.1056/NEJM197811022991808
- Bayes, T. (1763). An essay towards solving a problem in the doctrine of chances. Philosophical Transactions of the Royal Society of London, 53, 370-418. doi.org/10.1098/rstl.1763.0053