Contents
On this page
- Distinction between absence of evidence and evidence of absence
- Requirements for demonstrating the absence of a meaningful effect
- Interpretation of an underpowered null result
- Reporting guidelines for equivalence and non-inferiority claims
- A worked example of the distinction
- Application of the same logic to a null meta-analysis result
- Recommended and inappropriate terminology
Academy · Research Data, Analytics & AI · CIs and Proving Ineffectiveness
Confidence Intervals and the Conclusion That an Intervention Is Ineffective: Equivalence and Non-Inferiority
A confidence interval that includes the null value indicates that the data are inconclusive, not that an intervention has no effect. Demonstrating the absence of a meaningful effect requires equivalence or non-inferiority margin methods, which constitute an entirely distinct statistical approach.
A confidence interval that includes the null value of no effect demonstrates only that the available data cannot rule out the possibility of no effect; it does not, by itself, demonstrate that an intervention is ineffective. Concluding that an intervention lacks a clinically meaningful effect is a different, more specific statistical claim, one that formally requires an equivalence or non-inferiority design with a pre-specified margin, not simply a failure to reject the null hypothesis in a standard superiority analysis1. Conflating "we did not find evidence of an effect" with "we found evidence of no effect" is one of the most consequential statistical errors identified across the methodological literature on confidence intervals.
Learning objectives
- Why an interval including the null value shows inconclusiveness, not proof of ineffectiveness
- What a formal equivalence or non-inferiority margin requires to demonstrate the absence of a meaningful effect
- Why a narrow interval lying entirely within a pre-specified margin constitutes the evidence required for conclusions of ineffectiveness or equivalence
Common misconceptions
- “If a confidence interval includes zero (or one, for a ratio measure), the intervention has been shown to not work.” An interval that includes the null value means the data are compatible with no effect, but they remain equally compatible with a range of real effects on either side of the null, including potentially important ones, unless the interval is also narrow; demonstrating that an intervention lacks a clinically meaningful effect requires a pre-specified equivalence or non-inferiority margin and an interval that falls entirely within it, which is a fundamentally different analysis from simply failing to reject a superiority null hypothesis.
Distinction between absence of evidence and evidence of absence
When a confidence interval for a treatment effect includes the null value, the correct statistical conclusion is that the trial or meta-analysis did not find sufficient evidence to reject the null hypothesis at the chosen confidence level, not that the null hypothesis, zero effect, has itself been positively demonstrated1. Greenland and colleagues' widely referenced guide to statistical misinterpretations lists exactly this error, treating a non-significant result as proof of no effect, as among the most common and consequential misreadings of confidence intervals in the published literature, and emphasizes that a wide interval spanning the null value is frequently a sign of an underpowered study rather than genuine evidence against an effect existing2.
Requirements for demonstrating the absence of a meaningful effect
Formally demonstrating that an intervention lacks a clinically meaningful effect, or is no worse than a comparator by more than a trivial amount, requires a study specifically designed for that purpose, using an equivalence or non-inferiority margin defined before data collection, the largest difference that would still be considered clinically unimportant3. ICH's E9 guideline on statistical principles for clinical trials describes this as a distinct analytical framework from a standard superiority trial: in a non-inferiority analysis, the conclusion that an intervention is not meaningfully worse than a comparator is supported specifically when the confidence interval for the difference falls entirely within the pre-specified margin, not merely when it happens to include the null value3. A standard superiority trial that simply fails to find a significant difference was never designed to, and cannot, support this kind of equivalence conclusion, regardless of the appearance of the resulting confidence interval.
Interpretation of an underpowered null result
A wide confidence interval spanning the null value is a strong indication that the study’s statistical power and sample size should be examined before any conclusion is drawn about the intervention’s effectiveness, because a study with too few participants or events will routinely produce an inconclusive interval even when a real, clinically important effect exists2. The appropriate, defensible conclusion from such a result is that the evidence remains inconclusive and that a larger or differently designed study is needed to resolve the question, not that the intervention has been shown not to work; reviewers and readers should examine whether the width of the interval, and not merely whether it crosses the null, is consistent with the study having had an adequate chance of detecting a meaningful effect if one existed.
Reporting guidelines for equivalence and non-inferiority claims
Because the distinction between an inconclusive superiority result and a demonstrated equivalence result is so easily blurred in practice, the CONSORT reporting guideline, the standard checklist for reporting randomised trials, has a dedicated extension specifically for non-inferiority and equivalence trials4. Piaggio and colleagues' extension requires authors to state the pre-specified margin explicitly, to justify how that margin was chosen, and to report the confidence interval for the treatment difference specifically in relation to that margin, rather than simply reporting whether a conventional superiority P value reached significance4. A trial report that fails to state its margin, or that reports a non-inferiority conclusion based only on a superiority-style P value, does not follow this guidance and should be read with corresponding caution as to whether its “no meaningful difference” claim is supported by its own design.
A worked example of the distinction
In a hypothetical trial comparing a new drug against an existing one on a continuous outcome, a difference of fewer than 2 points is pre-specified as clinically unimportant. If the trial reports a mean difference of 0.5 points with a 95% confidence interval of -1.8 to 2.8, the interval includes the null value and also extends beyond the 2-point margin on the upper side, so neither a superiority nor a non-inferiority conclusion is supported; the accurate description is “inconclusive, with more data needed.” If instead the same trial reports a mean difference of 0.5 points with a narrower interval of -1.5 to 1.6, falling entirely within the pre-specified margin, a non-inferiority conclusion is supported, specifically because the interval, not merely the point estimate or an unrelated superiority P value, demonstrates that the drug is not meaningfully worse within the agreed margin4. The two scenarios share an almost identical point estimate; only the interval distinguishes a genuine equivalence finding from an inconclusive one.
Application of the same logic to a null meta-analysis result
The same reasoning applies, with an additional consideration, when a meta-analysis pooling several trials produces a confidence interval that includes the null value: because a meta-analysis increases the effective sample size behind the estimate, a null-including pooled interval generally carries more weight toward genuine inconclusiveness, or toward a genuinely small effect, than a null-including interval from any one small trial would3. Even so, the same equivalence-versus-superiority distinction applies at the pooled level: a meta-analysis of superiority trials that produces a null-including pooled interval has not thereby demonstrated equivalence, because none of its component trials, and therefore the pooled analysis built from them, were designed around a pre-specified equivalence margin1. A true equivalence conclusion from pooled data would require either pooling trials that were themselves designed as equivalence or non-inferiority studies with a consistent margin, or a post-hoc analysis that explicitly states and justifies the margin being applied retrospectively.
Recommended and inappropriate terminology
A short vocabulary guide helps to keep this distinction explicit in writing. Phrases to avoid include “no effect was found,” “the intervention does not work,” “proven ineffective,” and “equivalent” when no equivalence margin was pre-specified. Preferable alternatives are “the data did not provide evidence of a difference,” “the result was inconclusive given the available sample size,” “a confirmatory, adequately powered study is needed,” and, only when a pre-specified margin and design support it, “shown to be non-inferior (or equivalent) within the stated margin”4. Applying this vocabulary consistently across the abstract, results and discussion sections of a manuscript is a simple, verifiable practice that prevents authors from overstating a null result as proof of ineffectiveness, a pattern that otherwise recurs across the clinical and health-sciences literature regardless of what the underlying statistics show. Editors and statistical reviewers increasingly check for this vocabulary during peer review, so adopting it during drafting both improves the accuracy of a manuscript’s claims and reduces the number of review cycles needed before a cautious, defensible version of the results is accepted for publication.
A further worked illustration shows how this distinction in vocabulary alters a manuscript’s abstract. A draft abstract stating “the intervention was not effective (p=0.42)” makes a claim that the underlying superiority trial, with no pre-specified equivalence margin, was never designed to support; a revised abstract stating “the trial did not find evidence of a difference between groups (mean difference 0.8, 95% CI -1.2 to 2.8, p=0.42), and the available sample size may have been insufficient to detect a clinically important effect if one exists” makes a claim that the same data do support1. The second version is longer and less concise, which helps to explain why the shorter, overstated version appears so frequently in practice despite being the less accurate of the two; reviewers and journal editors increasingly flag the first kind of phrasing during peer review, and incorporating the more cautious and accurate phrasing into a first draft, rather than relying on a reviewer to identify and correct it later, both improves the scientific accuracy of the manuscript and reduces the number of revision cycles typically needed before acceptance2. The same principle applies to subgroup analyses in a results section, where the null-including interval of an individually underpowered subgroup is especially prone to being misreported as showing that the intervention “does not work” in that subgroup, when the more defensible conclusion is almost always that the subgroup-specific evidence remains inconclusive given the smaller numbers involved3.
Key points
Key points
A confidence interval that includes the null value indicates inconclusive evidence, not proof that an intervention is ineffective; demonstrating a genuine absence of meaningful effect requires a dedicated equivalence or non-inferiority design with a pre-specified margin, reported in accordance with the CONSORT extension, and an interval falling entirely within that margin.
Practical application
Practical application
- An interval that includes the null value should not be reported as showing that the intervention does not work. It should be described as inconclusive evidence given the available data, with an explicit statement of whether the study was adequately powered to detect a clinically important effect if one existed.
- Any claim of no difference should be checked to confirm that it was tested with an equivalence or non-inferiority design. A non-significant result from a standard superiority trial cannot support an equivalence conclusion; the pre-specified design and margin of the trial should be verified before such a claim is accepted.
- Statistical power should be reported alongside any null result. A wide interval including the null value from an underpowered study should be identified as such and distinguished from a narrow interval including the null value from a well-powered study, which carries different evidentiary weight.
Frequently asked questions
It shifts the interpretation of inconclusiveness toward ‘a true effect, if it exists, is probably small,’ since a narrow interval around the null indicates that the data had good power to detect even a modest effect and did not find one outside the null range; however, this is still not formally the same as a pre-specified equivalence or non-inferiority conclusion unless that framework was used.
Because a superiority trial is designed and powered to detect a difference, not to rule out a meaningful one within a pre-specified margin; without that margin and the corresponding sample-size planning, a non-significant superiority result simply reflects insufficient evidence of difference, not evidence of equivalence.
References
- Greenland S, Senn SJ, Rothman KJ, Carlin JB, Poole C, Goodman SN, Altman DG. Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. European Journal of Epidemiology. 2016;31(4):337-350. read 10 October 2026. View on European Journal of Epidemiology
- International Council for Harmonisation. E9: Statistical Principles for Clinical Trials. read 10 October 2026. View on ICH
- Gardner MJ, Altman DG. Confidence intervals rather than P values: estimation rather than hypothesis testing. BMJ. 1986;292(6522):746-750. read 10 October 2026. View on The BMJ
- Piaggio G, Elbourne DR, Pocock SJ, Evans SJW, Altman DG; CONSORT Group. Reporting of noninferiority and equivalence randomized trials: extension of the CONSORT 2010 statement. JAMA. 2012;308(24):2594-2604. read 10 October 2026. View on JAMA
- Deeks JJ, Higgins JPT, Altman DG (editors). Chapter 10: Analysing data and undertaking meta-analyses. In: Cochrane Handbook for Systematic Reviews of Interventions. Cochrane, 2024. read 10 October 2026. View on Cochrane Training
- Schulz KF, Grimes DA. Sample size calculations in randomised trials: mandatory and mystical. The Lancet. 2005;365(9467):1348-1353. read 10 October 2026. View on The Lancet
- Altman DG, Bland JM. Statistics notes: Absence of evidence is not evidence of absence. BMJ. 1995;311(7003):485. read 10 October 2026. View on The BMJ
- DerSimonian R, Laird N. Meta-analysis in clinical trials. Controlled Clinical Trials. 1986;7(3):177-188. read 10 October 2026. View on ScienceDirect
- Riley RD, Higgins JPT, Deeks JJ. Interpretation of random effects meta-analyses. BMJ. 2011;342:d549. read 10 October 2026. View on The BMJ
- Higgins JPT, Thompson SG. Quantifying heterogeneity in a meta-analysis. Statistics in Medicine. 2002;21(11):1539-1558. read 10 October 2026. View on PubMed
Every source above is a peer-reviewed article, a publisher’s record of one, or a standard issued by the body that owns it. Each carries the date it was last read. Where a standard moves, this article is revised rather than left up with a note.
How to cite this article
Pick your referencing style and copy it. Your university or target journal may vary the punctuation — check its guide if it has one.
Pubrica Academy. (2026, October 10). Confidence Intervals and the Conclusion That an Intervention Is Ineffective: Equivalence and Non-Inferiority. Pubrica. https://pubrica.com/academy/confidence-interval-in-the-estimation-of-intervention-effect/
Pubrica Academy. “Confidence Intervals and the Conclusion That an Intervention Is Ineffective: Equivalence and Non-Inferiority.” Pubrica, 10 Oct. 2026, pubrica.com/academy/confidence-interval-in-the-estimation-of-intervention-effect/.
Pubrica Academy (2026) Confidence Intervals and the Conclusion That an Intervention Is Ineffective: Equivalence and Non-Inferiority. Pubrica. Available at: https://pubrica.com/academy/confidence-interval-in-the-estimation-of-intervention-effect/ (Accessed: 15 October 2026).
Pubrica Academy. “Confidence Intervals and the Conclusion That an Intervention Is Ineffective: Equivalence and Non-Inferiority.” Pubrica. October 10, 2026. https://pubrica.com/academy/confidence-interval-in-the-estimation-of-intervention-effect/
Pubrica Academy, ‘Confidence Intervals and the Conclusion That an Intervention Is Ineffective: Equivalence and Non-Inferiority’ (Pubrica, 10 October 2026) <https://pubrica.com/academy/confidence-interval-in-the-estimation-of-intervention-effect/> accessed 15 October 2026.
Pubrica Academy. Confidence Intervals and the Conclusion That an Intervention Is Ineffective: Equivalence and Non-Inferiority. [Internet]. Pubrica; 2026 Oct 10 [cited 2026 Oct 15]. Available from: https://pubrica.com/academy/confidence-interval-in-the-estimation-of-intervention-effect/
Pubrica Academy, “Confidence Intervals and the Conclusion That an Intervention Is Ineffective: Equivalence and Non-Inferiority,” Pubrica, Oct. 10, 2026. [Online]. Available: https://pubrica.com/academy/confidence-interval-in-the-estimation-of-intervention-effect/
@misc{/la_confidence_intervals_intervention_ineffective_2026,
author = {{Pubrica Academy}},
title = {Confidence Intervals and the Conclusion That an Intervention Is Ineffective: Equivalence and Non-Inferiority},
howpublished = {Pubrica},
year = {2026},
month = {oct},
url = {https://pubrica.com/academy/confidence-interval-in-the-estimation-of-intervention-effect/},
note = {Accessed: <span data-cite-acc="iso">2026-10-15</span>}
}This article is written to be learned from. Cite it if you are referring to what it says; if you are using the evidence behind it, cite the sources listed above instead.
Null-Result Interpretation Checklist
This checklist should be used before concluding from a confidence interval that an intervention ‘does not work’.
[ ] Confirmed the interval includes the null value before considering an 'ineffective' claim [ ] Checked the study's statistical power and sample size relative to a clinically meaningful effect size [ ] Verified whether the study was designed as an equivalence or non-inferiority trial with a pre-specified margin [ ] If no such design was used, described the result as inconclusive rather than negative [ ] If an equivalence/non-inferiority design was used, checked that the interval falls entirely within the pre-specified margin before concluding equivalence
Pubrica AcademyWritten and maintained by Pubrica’s subject matter experts. Articles are revised when the underlying standards change, and each article states the date on which it was last reviewed.
How Pubrica can help
Pubrica’s clinical trial statistics team assesses whether a conclusion of no effect is supported by an equivalence or non-inferiority design and margin, or whether it reflects an underpowered and inconclusive result.