Contents
Academy · Research Data, Analytics & AI · P Values in Cochrane Reviews
Reporting P Values in Cochrane Reviews: Confidence Intervals, Heterogeneity and Recommended Wording
Cochrane methods guidance instructs review authors to report an effect estimate with its confidence interval as the primary result and to treat the P value as a secondary, supporting figure, rather than the reverse.
A P value in a Cochrane review is the probability of observing an effect at least as extreme as the one calculated, if the null hypothesis of no difference were actually true. Cochrane's own methodological guidance is explicit that this number should not be the headline result of a meta-analysis. The Cochrane Handbook for Systematic Reviews of Interventions instructs review authors to present the point estimate of effect together with a confidence interval as the primary statistical output, and to report the associated P value as a secondary piece of information that supports, but does not replace, that estimate1. The practical reporting conventions that follow from this (the wording of a results sentence, the handling of a borderline P value, and the avoidance of reporting a P value as though it proved the absence of an effect) distinguish the statistical reporting of a Cochrane review from a loosely worded summary of a single test.
Learning objectives
- The position of Cochrane methods guidance on reporting P values alongside confidence intervals
- The reasons a non-significant P value cannot be reported as proof that an intervention has no effect
- The wording expected in Cochrane reviews when a result is inconclusive rather than negative
Common misconceptions
- “A Cochrane review reports a P value first and a confidence interval as a formality.” Cochrane Handbook guidance specifies the opposite order: the effect estimate and its confidence interval are the primary result and the P value is secondary, because a confidence interval conveys the precision and plausible range of an effect in a way that a single P value cannot.
Instructions in Cochrane methods guidance
The Cochrane Handbook for Systematic Reviews of Interventions, in its chapter on analysing data and undertaking meta-analyses, frames confidence intervals as the primary way a Cochrane review should communicate the precision of a pooled estimate, with the associated P value serving a supporting role rather than standing as the headline statistic1. This reflects a broader methodological point made across the statistical literature: a P value answers only whether an observed effect is compatible with a null hypothesis of exactly no difference, while a confidence interval shows the full range of effect sizes compatible with the data, information a single dichotomous test result cannot carry2. For a review author, following this guidance in practice means a results paragraph should state the pooled risk ratio, mean difference, or odds ratio with its 95% confidence interval first, and may then add the P value as a secondary descriptor, for example reporting a risk ratio with its interval and noting the associated test result afterward rather than leading with a bare "p=0.03."
The misstatement that Cochrane reviews are designed to avoid
A P value above a conventional threshold does not mean an intervention has no effect; it means the data did not provide statistically significant evidence against the null hypothesis at that threshold, which can happen either because there truly is no effect or because the review was underpowered to detect one2. Gardner and Altman's influential 1986 paper in the British Medical Journal made this exact argument in the context of clinical research: a confidence interval around a non-significant result can still show that both a clinically important benefit and a clinically important harm remain plausible, a distinction a P value alone erases3. A Cochrane review that reports a wide confidence interval straddling the line of no effect, together with its P value, correctly signals "inconclusive," whereas a review that reports only "not significant" risks readers wrongly concluding "proven ineffective."
Recommended wording for a results section
Applying Cochrane's guidance consistently requires a specific convention for constructing results sentences: the effect measure is named, the point estimate is given, the 95% confidence interval is given, and only then, if relevant, are the P value and the significance threshold used noted1. The American Statistical Association's 2016 statement on statistical significance reinforces the same discipline from the wider statistics community, warning explicitly against treating "statistically significant" as equivalent to "true" or "important," and against treating "not statistically significant" as equivalent to "no effect" or "equivalent"4. Review authors preparing a Cochrane summary of findings table should carry this same discipline into the GRADE certainty assessment, where the width of the confidence interval, not the P value, is one of the factors used to judge whether the evidence is imprecise5.
Position of heterogeneity statistics relative to the P value
A Cochrane meta-analysis typically reports a second P value alongside the main result, the P value from the chi-squared test for heterogeneity, together with the I-squared statistic that quantifies what proportion of the variability across study results reflects genuine heterogeneity rather than chance6. This heterogeneity P value answers a different question than the main-effect P value, whether the included studies' results are more different from one another than sampling error alone would explain, and Cochrane guidance treats a low heterogeneity P value (or a high I-squared) as a reason to consider a random-effects model and to interpret the pooled confidence interval more cautiously, not as a reason to drop the heterogeneity statistic from the report6. Review authors sometimes conflate the two P values in a results sentence, reporting only one without specifying which hypothesis it tests; Cochrane's own reporting conventions call for both to be clearly labelled, since a reader cannot distinguish "evidence of an overall effect" from "evidence of disagreement between studies" unless each P value is tied explicitly to the hypothesis it addresses. Riley, Higgins, and Deeks note that when heterogeneity is substantial, reporting a prediction interval alongside the standard confidence interval gives readers a more complete picture than either P value can convey about how much an individual future study's result might differ from the pooled estimate7.
A pre-submission check of a review draft
A practical self-check for a draft Cochrane-style results section is to read each results sentence and identify which statistic appears first: if a P value is the first number a reader encounters for a given comparison, the sentence probably requires reordering so that it leads with the point estimate and its confidence interval1. A second check is to scan the discussion section specifically for the phrases “no effect,” “no difference,” or “ineffective” and to verify, for each occurrence, that it is supported by a narrow interval close to the null value rather than merely by a P value above the significance threshold; Altman and Bland's short, frequently cited note on this point remains one of the clearest explanations of why the distinction matters in practice, and serves as a useful one-page reference during manuscript revision8. A third check is to confirm that any heterogeneity P value in the manuscript is clearly labelled as testing for heterogeneity, not for the main effect, since an unlabelled second P value next to the main result is one of the more common sources of reviewer confusion in Cochrane-style reporting. A fourth, often omitted check is to examine the abstract in isolation, since abstracts are frequently written or shortened after the main results section and can revert to P-value-led phrasing even when the full text was written correctly; because many readers, and many automated indexing and summarization tools, rely on the abstract alone, an abstract that leads with “p=0.03” rather than with the pooled estimate and its interval undermines much of the rigor built into the rest of the review1. Running this same four-point check on the plain-language summary required for Cochrane reviews is equally worthwhile, since that summary is often the only part a non-specialist reader, including a journalist, a guideline panel member, or a patient, will ever see.
A further, concrete way to apply this checklist is to compare a short numerical example with a draft manuscript. If a draft reports a pooled risk ratio of 0.78 with a 95% confidence interval of 0.58 to 1.05 and a P value of 0.09 for the primary outcome, the wording “the intervention showed no significant effect (p=0.09),” omits the point estimate and interval entirely and risks being read as proof that the intervention does not work. Rewritten in the order that Cochrane guidance recommends, as “the intervention was associated with a risk ratio of 0.78 (95% CI 0.58 to 1.05; p=0.09), indicating a potential reduction in risk that the available data could not confirm with statistical confidence,” the same numbers convey the genuine uncertainty rather than a false impression of a negative result1. A related check for a review with several outcomes is whether the same ordering (estimate first, interval second, P value third) has been applied consistently to every reported outcome, not only the primary one; reviews not infrequently lead with the point estimate and interval for a favorable primary outcome while reverting to P-value-led phrasing for secondary or safety outcomes, which reintroduces, for whichever outcome receives less careful treatment, the misstatement risk that Cochrane guidance is designed to avoid2. Building a single, reusable template sentence structure, as the checklist above suggests, and applying it mechanically to every outcome in a results table, rather than relying on memory to apply the same discipline freshly to each new sentence as it is drafted, is a practical way to keep this consistency across a full-length review with many comparisons.
Key points
Key points
Cochrane Handbook guidance treats the effect estimate and its confidence interval as the primary result of a meta-analysis and the P value as secondary support. A results sentence should therefore lead with the estimate and interval, identify the hypothesis addressed by any P value, and never present a non-significant P value as proof of no effect.
Practical application
Practical application
- Results sentences should present the estimate first, followed by the interval and then the P value. The pooled effect measure and its 95% confidence interval should be stated first, with the P value added afterward as supporting detail rather than as the principal figure.
- Wide intervals should be described as inconclusive rather than negative. When a confidence interval is wide and crosses the line of no effect, the result should be described as inconclusive given the available data, not as evidence that the intervention does not work.
- The same principle should be applied in GRADE assessments. The width of the confidence interval, not only the P value, should inform the judgment of imprecision in a summary of findings table.
Frequently asked questions
No. Cochrane Handbook guidance keeps the P value in the report, but as a secondary figure alongside the primary effect estimate and confidence interval, not as the sole or lead statistic.
A non-significant result means the data were compatible with the null hypothesis at the chosen threshold; it does not rule out a real effect the review was underpowered to detect, which is exactly why Cochrane guidance pairs the P value with a confidence interval that shows the plausible range of effects.
They do not disagree when calculated consistently from the same model: a 95% confidence interval excluding the null value corresponds to a two-sided P value below 0.05 for the same test, so an apparent mismatch usually indicates a reporting or rounding error that should be rechecked.
References
- Deeks JJ, Higgins JPT, Altman DG (editors). Chapter 10: Analysing data and undertaking meta-analyses. In: Cochrane Handbook for Systematic Reviews of Interventions. Cochrane, 2024. read 10 October 2026. View on Cochrane Training
- Greenland S, Senn SJ, Rothman KJ, Carlin JB, Poole C, Goodman SN, Altman DG. Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. European Journal of Epidemiology. 2016;31(4):337-350. read 10 October 2026. View on European Journal of Epidemiology
- Gardner MJ, Altman DG. Confidence intervals rather than P values: estimation rather than hypothesis testing. BMJ. 1986;292(6522):746-750. read 10 October 2026. View on The BMJ
- Wasserstein RL, Lazar NA. The ASA Statement on p-Values: Context, Process, and Purpose. The American Statistician. 2016;70(2):129-133. read 10 October 2026. View on Taylor & Francis Online
- Altman DG, Bland JM. Statistics notes: Absence of evidence is not evidence of absence. BMJ. 1995;311(7003):485. read 10 October 2026. View on The BMJ
- Higgins JPT, Thompson SG. Quantifying heterogeneity in a meta-analysis. Statistics in Medicine. 2002;21(11):1539-1558. read 10 October 2026. View on PubMed
- Riley RD, Higgins JPT, Deeks JJ. Interpretation of random effects meta-analyses. BMJ. 2011;342:d549. read 10 October 2026. View on The BMJ
- DerSimonian R, Laird N. Meta-analysis in clinical trials. Controlled Clinical Trials. 1986;7(3):177-188. read 10 October 2026. View on ScienceDirect
- Altman DG, Bland JM. How to obtain the confidence interval from a P value. BMJ. 2011;343:d2090. read 10 October 2026. View on The BMJ
- International Council for Harmonisation. E9: Statistical Principles for Clinical Trials. read 10 October 2026. View on ICH
- du Prel JB, Hommel G, Rohrig B, Blettner M. Confidence interval or p-value? Part 4 of a series on evaluation of scientific publications. Deutsches Arzteblatt International. 2009;106(19):335-339. read 10 October 2026. View on Deutsches Arzteblatt International
Every source above is a peer-reviewed article, a publisher’s record of one, or a standard issued by the body that owns it. Each carries the date it was last read. Where a standard moves, this article is revised rather than left up with a note.
How to cite this article
Pick your referencing style and copy it. Your university or target journal may vary the punctuation — check its guide if it has one.
Pubrica Academy. (2026, October 10). Reporting P Values in Cochrane Reviews: Confidence Intervals, Heterogeneity and Recommended Wording. Pubrica. https://pubrica.com/academy/p-values-reporting-in-cochrane-reviews/
Pubrica Academy. “Reporting P Values in Cochrane Reviews: Confidence Intervals, Heterogeneity and Recommended Wording.” Pubrica, 10 Oct. 2026, pubrica.com/academy/p-values-reporting-in-cochrane-reviews/.
Pubrica Academy (2026) Reporting P Values in Cochrane Reviews: Confidence Intervals, Heterogeneity and Recommended Wording. Pubrica. Available at: https://pubrica.com/academy/p-values-reporting-in-cochrane-reviews/ (Accessed: 15 October 2026).
Pubrica Academy. “Reporting P Values in Cochrane Reviews: Confidence Intervals, Heterogeneity and Recommended Wording.” Pubrica. October 10, 2026. https://pubrica.com/academy/p-values-reporting-in-cochrane-reviews/
Pubrica Academy, ‘Reporting P Values in Cochrane Reviews: Confidence Intervals, Heterogeneity and Recommended Wording’ (Pubrica, 10 October 2026) <https://pubrica.com/academy/p-values-reporting-in-cochrane-reviews/> accessed 15 October 2026.
Pubrica Academy. Reporting P Values in Cochrane Reviews: Confidence Intervals, Heterogeneity and Recommended Wording. [Internet]. Pubrica; 2026 Oct 10 [cited 2026 Oct 15]. Available from: https://pubrica.com/academy/p-values-reporting-in-cochrane-reviews/
Pubrica Academy, “Reporting P Values in Cochrane Reviews: Confidence Intervals, Heterogeneity and Recommended Wording,” Pubrica, Oct. 10, 2026. [Online]. Available: https://pubrica.com/academy/p-values-reporting-in-cochrane-reviews/
@misc{/la_p_values_reporting_cochrane_reviews_2026,
author = {{Pubrica Academy}},
title = {Reporting P Values in Cochrane Reviews: Confidence Intervals, Heterogeneity and Recommended Wording},
howpublished = {Pubrica},
year = {2026},
month = {oct},
url = {https://pubrica.com/academy/p-values-reporting-in-cochrane-reviews/},
note = {Accessed: <span data-cite-acc="iso">2026-10-15</span>}
}This article is written to be learned from. Cite it if you are referring to what it says; if you are using the evidence behind it, cite the sources listed above instead.
Results-Sentence Checklist for P Values
This checklist is intended for use when writing a results paragraph for a meta-analysis or Cochrane-style review.
[ ] Named the effect measure (risk ratio, mean difference, odds ratio, etc.) [ ] Stated the point estimate before any P value [ ] Reported the 95% confidence interval immediately after the point estimate [ ] Added the P value as a secondary, supporting figure only [ ] Avoided describing a non-significant P value as evidence of 'no effect' [ ] Carried the confidence interval's width into any GRADE imprecision judgment
Pubrica AcademyWritten and maintained by Pubrica’s subject matter experts. Articles are revised when the underlying standards change, and each article states the date on which it was last reviewed.
How Pubrica can help
Pubrica’s biostatistics review team checks meta-analysis results sections against Cochrane Handbook reporting conventions and identifies phrasing led by P values, and misstatements of “no effect”, before submission.