Type I error
Also known as: False positive, Alpha error
Crying wolf — declaring an effect that isn't really there.
What it means
A Type I error is the mistake of rejecting a true null hypothesis, concluding that an effect exists when in reality it does not. Its long-run rate is set by the significance threshold alpha, conventionally 0.05, meaning a researcher accepts a 5% chance of a false alarm per test. Type I errors proliferate when many comparisons are run, when analyses are flexibly chosen, or when results are selectively reported, which is why multiplicity corrections and pre-registration exist. It trades off against Type II error: making the criterion stricter to avoid false positives raises the risk of missing real effects.
Examples
A trial concludes a useless supplement 'works' because the data happened to look extreme by chance.
A marketing team tests twenty button colours at once and one comes out 'significantly' better. At a 5% threshold across twenty tests, roughly one false alarm is what chance alone would deliver.
An analyst checks a live A/B test every morning and stops it the moment it crosses significance. A fluctuating metric will eventually wander over the line by itself, so peeking manufactures winners.
First described in Jerzy Neyman & Egon Pearson (1928, 1933).