Risk of bias
Also known as: Study quality assessment
The chance that flaws in a study's design or conduct have skewed its results.
What it means
Risk of bias is the likelihood that systematic flaws in the design, conduct, analysis, or reporting of a study have distorted its findings away from the truth, and it is the central criterion for appraising the credibility of evidence in systematic reviews. It is assessed not by judging whether a result happens to be correct but by examining the methods for known threats — inadequate randomization or concealment of allocation, lack of blinding, selective attrition, and selective reporting of favorable outcomes — each mapping to a recognized type of bias such as selection, performance, attrition, and reporting bias. Structured tools guide reviewers to rate each domain as low, high, or unclear risk, because the magnitude and even direction of distortion are usually unknown. Crucially, low risk of bias is about internal validity and is distinct from a study's precision or its external generalizability. It matters because meta-analyses and guidelines that pool high-risk studies produce confident but potentially misleading conclusions, the 'garbage in, garbage out' problem.
Examples
A trial reporting a large drug benefit is downgraded when reviewers find that patients and doctors knew who got the drug and that a quarter of participants dropped out unevenly — high risk of performance and attrition bias.
A review downgrades a study of a new teaching method after finding that head teachers chose which classes got it, so the keenest schools opted in — high risk of selection bias.
A nutrition trial reports benefits on one of many measured outcomes; reviewers rate it high risk of reporting bias when the registered protocol named a different primary outcome.
First described in Evidence-based medicine; Cochrane risk-of-bias framework.