Practical significance
Also known as: Clinical significance, Substantive significance
Whether an effect is big enough to matter in the real world, not just statistically detectable.
What it means
Practical significance is the judgment of whether a finding is large and consequential enough to matter for real-world decisions, as opposed to statistical significance, which only indicates that an effect is unlikely to be zero given the data. The two come apart in both directions: with a large sample, a trivially small effect can be highly statistically significant, while a substantively important effect can fail to reach significance in an underpowered study. Assessing practical significance requires looking beyond the p-value to the effect size, its confidence interval, and the cost-benefit context — what magnitude would justify the expense, side effects, or effort of acting. It is inherently a domain judgment rather than a purely statistical one, often anchored to a pre-specified smallest effect size of interest. It matters because conflating statistical with practical significance leads to hyping inconsequential results and dismissing meaningful ones, a chronic problem in applied and policy research.
Examples
A weight-loss app shows a 'statistically significant' average loss of half a pound over a year in a huge trial — real, but far too small to be practically worth the subscription.
A bank's new savings prompt reliably lifts deposits by about two pounds a month across millions of customers — statistically rock solid, and still not worth the redesign it would take to roll out.
A tutoring trial in forty classrooms misses significance, yet the estimated gain is half a school year — too important to file away as 'no effect' without a bigger test.