Free scoping read. A written assessment of your manuscript in two business days, at no cost.

Start one
Contents

Academy · Research Data, Analytics & AI · RevMan P-Value Reporting

Reporting of P Values in RevMan Meta-Analyses: The Test for Overall Effect and the Test for Heterogeneity

The forest plot output of Review Manager (RevMan) reports two separate statistical tests: a “Test for overall effect” Z statistic and p-value for the pooled estimate, and a separate chi-squared-based heterogeneity or “Test for subgroup differences” p-value. Conflating the two is a common misreading of a RevMan results panel.

Pubrica AcademyWritten by a meta-analysis methodologist Reviewed bya biostatistics reviewer Published10 October 2026 Last checked10 October 2026 Reading time7 minutes Sources10 peer-reviewed

Review Manager (RevMan), the software that Cochrane review authors use to conduct and display a meta-analysis, prints two distinct statistical tests on a standard forest plot, and correct interpretation requires knowing which test addresses which question. The “Test for overall effect” line reports a Z-statistic and an associated p-value testing the null hypothesis that the pooled effect across all included studies is zero (no difference). A separate line, typically labeled “Test for heterogeneity” or, when subgroups are specified, “Test for subgroup differences,” reports a chi-squared statistic, its degrees of freedom, an I-squared percentage and its own p-value, testing an entirely different null hypothesis: that the individual studies' effects all estimate the same true underlying effect. The two p-values are calculated from different statistics and test different hypotheses, and a result on one says nothing directly about the other.

Learning objectives

  • The calculation represented by RevMan’s “Test for overall effect” Z and P values
  • How the separate chi-squared-based heterogeneity p-value differs in both test statistic and null hypothesis
  • Why a statistically significant overall-effect p-value and a statistically significant heterogeneity p-value can appear on the same forest plot without contradiction

Common misconceptions

  • “The single p-value on a RevMan forest plot summarizes whether the meta-analysis found a significant result overall.” A RevMan forest plot carries at least two separately calculated p-values, one for the pooled effect (the Test for overall effect) and one for heterogeneity between studies (the Test for heterogeneity or subgroup differences). Each tests a different null hypothesis using a different test statistic, and reading only one on the assumption that it represents the whole analysis is a documented and avoidable misreading of standard meta-analysis output.

The Test for overall effect: the hypothesis tested by the Z statistic and P value

RevMan's statistical algorithms, documented in the Cochrane Handbook's technical appendix on the software's calculations, compute the pooled effect estimate using the inverse-variance method by default, weighting each study's contribution by the inverse of its variance, and then calculate a Z-statistic by dividing that pooled estimate by its standard error1. The associated p-value, labeled on the forest plot as part of the “Test for overall effect” line, tests the null hypothesis that the true pooled effect across all included studies is exactly zero (no difference between the compared groups) against the two-sided alternative that it is not zero1. This is the p-value to which most readers look first on a forest plot, and it summarizes only the distance of the pooled effect from the null value; it conveys nothing about how consistent the individual contributing studies were with one another.

The separate test for heterogeneity and subgroup differences

A separate calculation, printed on the same forest plot, addresses an entirely different question: whether the variation observed between the individual studies' effect estimates exceeds that expected from chance (sampling error) alone if every study were estimating the same underlying effect2. RevMan computes this using Cochran's Q, a chi-squared statistic, with degrees of freedom equal to the number of studies minus one, and reports its own p-value alongside the I-squared statistic, which expresses the proportion of total variation across studies attributable to heterogeneity rather than chance21. Higgins, Thompson, Deeks, and Altman's widely cited paper on measuring inconsistency in meta-analyses explains why this test is reported separately from the overall-effect test: the chi-squared test for heterogeneity has low statistical power when a meta-analysis includes few studies, so a non-significant heterogeneity p-value, particularly with a small number of studies, cannot be taken as confident evidence that no real heterogeneity exists. For this reason, I-squared is reported alongside it as a descriptive measure less dependent on power3. When a review specifies subgroups, RevMan prints a closely related but distinct “Test for subgroup differences,” which tests whether the pooled effect differs across the predefined subgroups rather than testing heterogeneity among individual studies in general2.

Joint interpretation of the two tests

Because the two p-values test different hypotheses using different statistics, a forest plot can legitimately show a highly significant overall-effect p-value alongside a highly significant heterogeneity p-value. Such a combination indicates strong evidence both of a real pooled effect and of real inconsistency between the contributing studies, and neither result on its own invalidates or contradicts the other3. The Cochrane Handbook's chapter on analysing data recommends that review authors report both results distinctly in the text accompanying a forest plot, explaining what each test found and, importantly, what each finding implies for the confidence with which the pooled estimate can be interpreted, rather than leaving readers to infer this from the numbers on the forest plot1. A reviewer who reports only the overall-effect p-value and omits any discussion of substantial heterogeneity found in the same analysis gives readers an incomplete, and potentially misleading, account of the confidence that the pooled estimate merits.

Variation in the displayed statistics by outcome type

RevMan's forest plot layout varies slightly with the type of outcome being pooled, which determines the numbers that accompany the Test for overall effect line. For continuous outcomes pooled as a mean difference (MD) or standardized mean difference (SMD), the software displays each study's mean, standard deviation and sample size per arm, together with the pooled MD or SMD, its 95% confidence interval, and the overall-effect Z and P values1. For dichotomous outcomes pooled as an odds ratio, risk ratio or risk difference, the display instead shows each study's event count and total sample size per arm; ratio measures are plotted on a logarithmic scale because odds ratios and risk ratios are multiplicative and are symmetric around 1.0 on a log scale rather than a linear one1. The overall-effect Z and P values are calculated on this same underlying scale, so that for a ratio measure the Z-test assesses whether the log of the pooled ratio differs from zero (equivalent to the ratio differing from 1.0). This detail explains why RevMan's underlying calculation for a ratio measure differs mechanically from that for a mean difference, even though both report the result in the same “Test for overall effect” format2.

Common errors in interpreting RevMan output

A frequent error is to interpret a low heterogeneity chi-squared p-value, like a low p-value for the overall effect, as a favorable result, when a significant heterogeneity test indicates the opposite: the individual studies' results differ from one another more than chance alone would explain, which should prompt caution about the pooled estimate rather than confidence in it3. A second common error is to quote the Z-statistic as though it carried interpretive meaning independent of the p-value it produces; the Z-statistic is an intermediate calculation, and reporting the p-value and confidence interval communicates the same result more directly to a reader unfamiliar with the construction of a Z-test. A third is to overlook that RevMan computes separate P values for the “Test for subgroup differences” of a subgroup analysis and for the pooled overall effect within each individual subgroup; these are three distinct numbers that are easily conflated if a results table does not label each one by name exactly as RevMan does1. Labeling every reported statistic by its exact RevMan name (Test for overall effect, Test for heterogeneity or Test for subgroup differences) when transcribing results into a manuscript is the simplest safeguard against all three errors. A further check is to reopen the RevMan analysis file when writing up results months after the analysis was run, rather than working from a screenshot or a previously copied number, since rerunning the calculation confirms that the transcribed figures still match the underlying data and detects any transcription error introduced between the original analysis and the final draft. This practice is important because a review that undergoes several rounds of peer-review revision, each adding or removing a study, can readily produce a manuscript whose narrative text still describes an earlier version of the pooled statistics if the RevMan file is not rerun and the write-up rechecked after every substantive change to the list of included studies. The risk increases when writing and analysis are divided between co-authors, since the person drafting the narrative results may not be the person who last updated the RevMan file; an explicit, shared final cross-check between the two is therefore a necessary part of completing a collaborative review rather than an optional step. Assigning this cross-check as a named task on the review's project timeline, rather than assuming that it will occur informally before submission, closes a gap that teams repeatedly report discovering only after a reader or reviewer identifies a mismatch.

Key points

Key points

A RevMan forest plot reports at least two distinct p-values, one from a Z-test for the pooled overall effect and one from a chi-squared test for heterogeneity between studies. Both should be read and reported together, since neither alone conveys all of the information that the forest plot displays.

Practical application

Practical application

  1. The test to which a given p-value belongs should be identified before it is interpreted. The label should be checked to establish whether the value belongs to the “Test for overall effect” line or to the “Test for heterogeneity” or “Test for subgroup differences” line before any conclusion is drawn.
  2. Both the overall-effect and the heterogeneity results should be reported in the text, not only in the forest plot. The text should state explicitly what each test found and the implications of that combination for the confidence with which the pooled estimate can be interpreted.
  3. A non-significant heterogeneity p-value should not be treated as evidence of the absence of heterogeneity. The test has low power when few studies are included; I-squared should be reported alongside it rather than consistency across studies being judged on the p-value alone.

Frequently asked questions

No. They use different test statistics (a Z-statistic for the pooled effect, a chi-squared statistic for heterogeneity) and test different null hypotheses, so they must be interpreted separately.

Yes, and this is not a contradiction; it means there is evidence both of a real average effect and of real inconsistency between the individual studies contributing to that average.

The chi-squared heterogeneity test has low statistical power when few studies are included, so a non-significant p-value cannot confidently rule out real heterogeneity; I-squared gives a descriptive measure less dependent on that power limitation.

Whether the pooled effect differs across predefined subgroups of studies, which is a distinct question from whether individual studies within the overall analysis are heterogeneous.

References

  1. Deeks JJ, Higgins JPT, Altman DG. Statistical algorithms in Review Manager 5 / Chapter 10: Analysing data and undertaking meta-analyses. Cochrane Handbook for Systematic Reviews of Interventions. read 10 October 2026. View on training.cochrane.org
  2. Deeks JJ, Higgins JPT, Altman DG. Chapter 10: Analysing data and undertaking meta-analyses. Cochrane Handbook for Systematic Reviews of Interventions (current version). read 10 October 2026. View on training.cochrane.org
  3. Higgins JPT, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ. 2003;327(7414):557-560. read 10 October 2026. View on BMJ
  4. DerSimonian R, Laird N. Meta-analysis in clinical trials. Controlled Clinical Trials. 1986;7(3):177-188. read 10 October 2026. View on Controlled Clinical Trials
  5. Higgins JPT, Li T, Deeks JJ. Chapter 6: Choosing effect measures and computing estimates of effect. Cochrane Handbook for Systematic Reviews of Interventions. read 10 October 2026. View on training.cochrane.org
  6. Schünemann HJ, Higgins JPT, Vist GE, et al. Chapter 15: Interpreting results and drawing conclusions. Cochrane Handbook for Systematic Reviews of Interventions. read 10 October 2026. View on training.cochrane.org
  7. Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. Introduction to Meta-Analysis. John Wiley & Sons; 2009. read 10 October 2026. View on wiley.com
  8. Greenland S, Senn SJ, Rothman KJ, Carlin JB, Poole C, Goodman SN, Altman DG. Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. European Journal of Epidemiology. 2016;31(4):337-350. read 10 October 2026. View on Springer
  9. Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71. read 10 October 2026. View on BMJ
  10. Guyatt GH, Oxman AD, Vist GE, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336(7650):924-926. read 10 October 2026. View on BMJ

Every source above is a peer-reviewed article, a publisher’s record of one, or a standard issued by the body that owns it. Each carries the date it was last read. Where a standard moves, this article is revised rather than left up with a note.

How to cite this article

Pick your referencing style and copy it. Your university or target journal may vary the punctuation — check its guide if it has one.

Pubrica Academy. (2026, October 10). Reporting of P Values in RevMan Meta-Analyses: The Test for Overall Effect and the Test for Heterogeneity. Pubrica. https://pubrica.com/academy/revman-report-of-p-values-in-meta-anaysis/

This article is written to be learned from. Cite it if you are referring to what it says; if you are using the evidence behind it, cite the sources listed above instead.

Forest Plot Reading Note

This checklist may be used to distinguish RevMan's two statistical tests when reporting results.

[ ] Located the 'Test for overall effect' Z and P values and noted what they test (pooled effect vs zero)
[ ] Located the 'Test for heterogeneity' chi-squared, df, I-squared, and P values separately
[ ] Checked whether a 'Test for subgroup differences' line is also present and what it tests
[ ] Reported both the overall-effect and heterogeneity results explicitly in the accompanying text
[ ] Avoided treating a non-significant heterogeneity P value as proof heterogeneity is absent

Pubrica AcademyWritten and maintained by Pubrica’s subject matter experts. Articles are revised when the underlying standards change, and each article states the date on which it was last reviewed.

How Pubrica can help

Pubrica’s biostatistics team prepares and interprets RevMan forest-plot output for systematic reviews, reporting both the overall-effect and heterogeneity results clearly rather than leaving readers to interpret the plot unassisted.

Find something else

Search the Pubrica Academy

Access is free and requires no account. Enter a topic, or open the full library to filter by category, sub-topic or resource type.

Pubrica can review results text to ensure that both p-values reported by RevMan are explained.

A scoping assessment is free and takes two business days. It provides a written assessment of what the work requires and which specialist would undertake it.

Get a free quote

Regulations

Check your project against this rule

  1. 1Your work
  2. 2Your details
  3. 3Send
What do you need?

Material you send is held under signed confidentiality, with access limited to the assigned team and encrypted transfer. It is not used to train anything and is not passed to third-party tools. Ask for the agreement before you send anything you would rather cover first.