Attrition bias
Also known as: Dropout bias
When the people who quit a study differ from those who stay, results skew.
What it means
Attrition bias is the distortion that arises when the participants who leave a study before it ends differ systematically from those who remain, so the result describes the survivors rather than the group that was enrolled. It appears in clinical trials, long panel surveys, and online experiments alike, and it bites hardest when dropout is differential — when one arm loses more people, or different people, than another, undoing the balance randomization created. It is a form of selection bias, and while dropout accumulates over a long follow-up, the damage depends on why people left rather than how many: a large loss unrelated to the outcome may cost only precision, while a small outcome-related one can reverse a result. Retention efforts and sensitivity analyses are the standard defenses; intention-to-treat analysis is often listed alongside them, but it protects randomization against crossover and non-adherence and cannot recover an outcome that was never measured.
Why the reason matters more than the rate
The reported figure — 8% lost to follow-up — says little on its own. What matters is why people left. If they left for reasons unrelated to the outcome, like moving house, the sample shrinks and precision falls, but the estimate stays honest. If they left because of the outcome — the drug caused nausea, the training did not land a job — the survivors are a selected group, and the estimate is wrong in a direction the data cannot reveal. Differential attrition, where one arm loses more or different people, is sharpest: randomization balanced the groups at baseline, and dropout unbalances them again.
What the evidence shows
Small dropout rates carry large consequences. Akl and colleagues (2012) re-analyzed 235 trials reporting significant results in five leading medical journals. Median loss to follow-up was just 6% among the trials that reported it — 31 of the 235 never said whether any occurred at all — yet under plausible assumptions about how the departed fared, between none and a third of trials lost significance depending on the assumption, and under a worst case, 58% did. Nüesch and colleagues (2009) measured the analytic cousin rather than dropout itself — patients excluded from the analysis rather than lost from the study — and across 167 trials of osteoarthritis treatments found those excluding patients reported more favorable effects, though the difference (-0.13, 95% CI -0.29 to 0.04) was imprecise; their conclusion was that the direction of bias is unpredictable, not that it always flatters.
Defenses, and what they cannot do
Intention-to-treat analysis keeps everyone in their assigned arm, protecting randomization against crossover and non-adherence. It does not conjure outcomes for people who vanished; when an outcome is unobserved, something must be assumed. Multiple imputation and inverse-probability weighting work if dropout is explainable by data you collected — an assumption the data cannot verify, because the people who would falsify it are the ones missing. So sensitivity analysis is not decoration: vary the assumption about the departed, and see whether the conclusion survives. Bounding approaches such as Lee's (2009) trimming procedure report a range rather than a single point estimate, though they too buy that range with an assumption — that treatment pushed dropout in only one direction.
Beyond the clinic
Attrition bias is not a clinical-trials curiosity. Zhou and Fishbach (2016) recorded 30% to 50% attrition in online experiments, varying by condition, and showed it manufacturing conclusions such as imagining applying eyeliner causing weight loss. Product analytics inherits the flaw: retention curves and satisfaction scores are computed on whoever stayed. Long panel surveys face it structurally — the PSID had lost roughly half its original 1968 sample by 1989 — yet Fitzgerald and colleagues (1998) found attrition there was selective on observables while cross-sectional representativeness held up well. Selective dropout is a threat to test, not an automatic verdict.
Related but distinct
Survivorship bias is the broader family: reasoning from whoever remains, whether or not anyone enrolled them. Attrition bias is its within-study form, and the difference is practical rather than semantic — a known baseline sample makes the loss measurable. You can see who left, compare them with those who stayed, and bound the damage. Non-response bias strikes earlier, at recruitment, before any measurement has happened, so there is no baseline to compare against at all. Cochrane's RoB 2 tool treats missing outcome data as its own domain for exactly this reason.
Examples
In a weight-loss trial, if discouraged participants in the diet arm quit, the survivors make the diet look more effective than it is.
A language app reports that learners who stay a year reach fluency — but everyone who found it useless deleted it in week two and was never counted.
A graduate scheme's glowing five-year satisfaction survey only reaches people still employed there; everyone who hated it left, and took their answers with them.
A tutoring trial randomizes half a year group to after-school sessions, but the strugglers assigned to tutoring drift away by March while the control group sits the final test intact, so the arms being compared are no longer the balanced groups randomization created.
A study following released prisoners for three years loses contact with those who move often or land back in custody elsewhere, so the reoffending rate it publishes is quietly understated.
First described in Clinical-trials methodology; Campbell & Stanley (1963) on experimental mortality.
Key references
- Sterne, J. A. C., Savović, J., Page, M. J., et al. (2019). RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ, 366, l4898. doi.org/10.1136/bmj.l4898
- Zhou, H., & Fishbach, A. (2016). The pitfall of experimenting on the web: How unattended selective attrition leads to surprising (yet false) research conclusions. Journal of Personality and Social Psychology, 111(4), 493-504. doi.org/10.1037/pspa0000056
- Akl, E. A., Briel, M., You, J. J., et al. (2012). Potential impact on estimated treatment effects of information lost to follow-up in randomised controlled trials (LOST-IT): systematic review. BMJ, 344, e2809. doi.org/10.1136/bmj.e2809
- Nüesch, E., Trelle, S., Reichenbach, S., et al. (2009). The effects of excluding patients from the analysis in randomised controlled trials: meta-epidemiological study. BMJ, 339, b3244. doi.org/10.1136/bmj.b3244
- Lee, D. S. (2009). Training, wages, and sample selection: Estimating sharp bounds on treatment effects. Review of Economic Studies, 76(3), 1071-1102. doi.org/10.1111/j.1467-937X.2009.00536.x
- Fitzgerald, J., Gottschalk, P., & Moffitt, R. (1998). An analysis of sample attrition in panel data: The Michigan Panel Study of Income Dynamics. Journal of Human Resources, 33(2), 251-299. doi.org/10.2307/146433