Contents
Academy · Research Data, Analytics & AI · Reporting Effect-Size Labels
Effect Size Rules of Thumb in Meta-Analysis Reports: Arguments For and Against Their Inclusion
The inclusion of effect-size rules of thumb remains a subject of methodological disagreement rather than a settled rule. The Cochrane Handbook cautions that generic small, medium and large labels may mislead when applied without clinical context, whereas other methodologists argue that a labeled benchmark still assists non-specialist readers who cannot otherwise interpret a raw effect-size value.
This is a question that review authors must answer deliberately rather than by habit, because methodological guidance does not give a single uniform answer. Including a conventional small/medium/large label alongside a reported effect size gives non-specialist readers, who make up a large share of the audience for a published systematic review, a rapid interpretive reference for an otherwise abstract number. Omitting the label and instead requiring readers to judge the number against field-specific context avoids the documented risk that a generic label implies more universal meaning than the number carries. Cochrane guidance and several methodologists have taken opposing positions on this specific trade-off, which means the decision belongs in a review's methods section as a stated, justified choice rather than a default either way.
Learning objectives
- The guidance of the Cochrane Handbook on the use of generic effect-size labels in the reporting of a review
- The arguments advanced by methodologists in favor of a labeled benchmark for non-specialist readers
- An intermediate approach adopted by several published meta-analyses to obtain both benefits
Common misconceptions
- “A small, medium or large label is a harmless and purely optional addition to a results section.” The Cochrane Handbook’s guidance on interpreting results explicitly cautions against presenting a generic magnitude label as though it reflects clinical or practical importance, because the same labeled magnitude may represent a trivial or a substantial difference depending on the outcome. The decision to include or omit the label is therefore a substantive editorial decision with a documented disadvantage in either case.
Arguments against the inclusion of generic labels
The Cochrane Handbook's chapter on interpreting results and drawing conclusions warns specifically against treating a standardized effect size's magnitude label as equivalent to a judgment of clinical importance, because the same numeric SMD can represent a negligible difference on one outcome scale and a substantial, patient-relevant difference on another1. A peer-reviewed methodological critique examining the practice of borrowing field-specific effect-size distributions as interpretive benchmarks goes further, arguing that the importation of a generic or even a field-average benchmark into a specific review's discussion risks anchoring readers to an interpretation that the review's own data do not support, since the benchmark was derived from an entirely different population of studies2. From this position, a review's discussion should interpret its pooled effect size against the clinical minimally important difference for that specific outcome measure where one has been established, or against directly comparable prior trials, rather than attaching an unqualified small/medium/large label that a reader might mistake for a substantive conclusion that the review itself has not drawn.
Arguments for the inclusion of a labeled benchmark
The opposing, and also methodologically grounded, position starts from a practical communication problem: a raw SMD or mean-difference number, reported without any interpretive anchor, conveys little meaning to the large share of a systematic review's readership who are not statisticians, including many clinicians, policymakers, and patients who read a review's abstract or plain-language summary. Sullivan and Feinn's effect-size paper, aimed at a clinical and educational readership rather than a purely statistical one, argues that some interpretive benchmark, even an imperfect generic one, is more useful to a reader trying to judge a result's magnitude than no benchmark at all, provided the benchmark is presented as a rough aid rather than a precise measure of importance3. On this view, the risk the Cochrane Handbook flags is real but manageable: it is a reason to qualify how a label is presented, not necessarily a reason to omit it altogether, particularly in a plain-language summary where some readers will have no other means of judging the magnitude of the number.
A practical intermediate approach
Several published meta-analyses resolve this tension by using both approaches in different places within the same report. The main results and discussion sections interpret the pooled effect size against the specific clinical context (a minimally important difference for that outcome measure where one exists, or a direct comparison with closely related prior trials), following the Cochrane Handbook's caution against relying on a generic label alone1. A plain-language summary or abstract, aimed at a broader audience who will not read the full methods, may still note the conventional small/medium/large label as a rough orientation, explicitly flagged as a general convention rather than a field-specific judgment3. This dual approach allows a review to satisfy the Handbook's caution where interpretive precision matters most, in the results and discussion, while still providing less specialized readers with a reference point for an otherwise abstract number; this choice should be stated explicitly in the methods section so that readers understand which interpretive standard is being applied where.
Emerging expectations of journals and funders
Irrespective of a review author's position in this debate, PRISMA 2020's reporting checklist requires authors to state the methods used to assess certainty in the body of evidence for each outcome, which favors explaining a pooled effect's importance through a stated method, such as GRADE certainty assessment or a cited minimally important difference, rather than through an unexplained generic label alone6. Journals that require PRISMA 2020 compliance at submission are, in effect, requiring authors to document which interpretive standard they used, which makes the choice described in this article a reporting requirement in many venues rather than a purely optional editorial preference. Funders commissioning systematic reviews for policy or guideline use have similarly moved toward expecting GRADE certainty ratings specifically because a certainty rating captures considerations (risk of bias, inconsistency and imprecision) that a bare effect-size label, generic or field-specific, does not address at all4.
A decision framework for review teams
A review team can resolve this question for a specific review by addressing three sequential questions. First, does a validated minimally important difference exist for this specific outcome measure and population? If so, it should be used as the primary interpretive reference in the discussion, and any generic label treated as redundant. Second, if no validated minimally important difference exists, is there a closely comparable prior trial or review whose effect size can serve as a direct, field-specific point of comparison? If so, that direct comparison should be used in the discussion rather than a generic statistical convention. Third, only if neither of the first two options is available should the review team consider whether a clearly identified generic label would help a non-specialist reader of the abstract or plain-language summary, recognizing that the Handbook's caution against relying on it for the main clinical interpretation still applies1. Addressing these three questions in order, rather than defaulting immediately to Cohen's convention, keeps the final reporting choice grounded in the best available context-specific evidence rather than in the most convenient shortcut. Recording the answer to each of these three questions directly in the review's protocol, before results are known, also prevents a more subtle problem: a review team that defers the labeling decision until after the pooled estimate is known risks, even unintentionally, selecting whichever interpretive framing presents a borderline result most favorably, which is the type of outcome-dependent reporting choice that a pre-specified protocol is designed to prevent. Review teams working on multiple outcomes within the same review should also apply the same decision framework consistently across every outcome, rather than using a minimally important difference for outcomes where it presents the result favorably and a generic label for outcomes where it would not, since inconsistent application of the framework across a single review's own outcomes is itself a subtle form of the selective framing the framework exists to prevent. A review's statistical analysis plan, drafted and registered before data extraction, is the natural place to record this outcome-by-outcome interpretive plan, and doing so provides a peer reviewer or reader with a concrete document against which to check the published report, which is a stronger guarantee of consistency than reliance on the final report alone. Where a review's protocol is itself registered publicly, as Cochrane protocols and PROSPERO-registered systematic reviews both are, this interpretive plan becomes independently checkable by any reader comparing the published report against the earlier registered protocol, which is a further, external safeguard against the specific risk of outcome-dependent reporting this section describes. A reader who observes that a published review's interpretive approach diverges from its registered protocol has a concrete basis for raising that discrepancy with the journal or the review's authors, which is a substantially stronger position than that of a reader who merely suspects, without a documented point of comparison, that a result has been framed more favorably than the data warrant. This is one of the more concrete practical benefits of protocol registration beyond its more commonly cited role in preventing selective outcome reporting: it also provides later readers with a fixed point of comparison for how a result's magnitude was to be judged, established before the result was known. Journals that ask authors to confirm, at submission, that the published interpretation matches the registered protocol are in effect formalizing this check, converting what would otherwise depend on an individual reader's initiative into a routine part of the editorial process.
Key points
Key points
The inclusion of a generic effect-size label is a methodological choice rather than a formatting default. The Cochrane Handbook cautions against relying on such labels for clinical interpretation, whereas other methodologists defend them as an approximate aid for non-specialist readers. The most defensible approach uses field-specific benchmarks in the discussion and reserves the generic label, clearly identified as such, for less technical summary sections.
Practical application
Practical application
- The labeling choice should be stated explicitly in the methods section. Whether generic small, medium and large labels are included or omitted, this decision and its rationale should be stated rather than left implicit.
- Judgments of clinical importance should be based on field-specific benchmarks. In the discussion, the pooled effect should be compared with a minimally important difference for the outcome or with directly comparable prior trials, rather than with a generic label.
- Where a generic label is included in a lay summary, it should be identified as an approximate convention. Non-specialist readers should be informed that the label is a general statistical convention rather than a field-specific judgment of importance.
Frequently asked questions
It does not forbid them outright, but it cautions against presenting a generic magnitude label as though it were equivalent to a judgment of clinical or practical importance, since that importance depends on the specific outcome and context.
Methodologists who focus on readability for non-specialist audiences argue that a clearly identified generic label can still help readers who have no other means of judging an abstract effect-size number, particularly in plain-language summaries.
A minimally important difference specific to the outcome measure, where one has been established, or a direct comparison with closely related prior trials, rather than an unqualified small/medium/large label.
Yes. Several published reviews reserve field-specific interpretation for the results and discussion while still noting a generic label, clearly identified as an approximate convention, in a plain-language summary aimed at broader readers.
References
- Schünemann HJ, Higgins JPT, Vist GE, et al. Chapter 15: Interpreting results and drawing conclusions. Cochrane Handbook for Systematic Reviews of Interventions. read 10 October 2026. View on training.cochrane.org
- Bakker M, et al. Denouncing the use of field-specific effect size distributions to inform magnitude. PeerJ. 2021;9:e11383. read 10 October 2026. View on PeerJ
- Sullivan GM, Feinn R. Using Effect Size-or Why the P Value Is Not Enough. Journal of Graduate Medical Education. 2012;4(3):279-282. read 10 October 2026. View on JGME
- Guyatt GH, Oxman AD, Vist GE, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336(7650):924-926. read 10 October 2026. View on BMJ
- Lakens D. Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs. Frontiers in Psychology. 2013;4:863. read 10 October 2026. View on Frontiers
- Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71. read 10 October 2026. View on BMJ
- Cohen J. Statistical Power Analysis for the Behavioral Sciences. 2nd ed. Lawrence Erlbaum Associates; 1988. read 10 October 2026. View on routledge.com
- Higgins JPT, Li T, Deeks JJ. Chapter 6: Choosing effect measures and computing estimates of effect. Cochrane Handbook for Systematic Reviews of Interventions. read 10 October 2026. View on training.cochrane.org
- Deeks JJ, Higgins JPT, Altman DG. Chapter 10: Analysing data and undertaking meta-analyses. Cochrane Handbook for Systematic Reviews of Interventions. read 10 October 2026. View on training.cochrane.org
- Sullivan GM, Feinn R. Using Effect Size-or Why the P Value Is Not Enough. Journal of Graduate Medical Education. 2012;4(3):279-282. (publisher record) read 10 October 2026. View on Ingenta Connect
Every source above is a peer-reviewed article, a publisher’s record of one, or a standard issued by the body that owns it. Each carries the date it was last read. Where a standard moves, this article is revised rather than left up with a note.
How to cite this article
Pick your referencing style and copy it. Your university or target journal may vary the punctuation — check its guide if it has one.
Pubrica Academy. (2026, October 10). Effect Size Rules of Thumb in Meta-Analysis Reports: Arguments For and Against Their Inclusion. Pubrica. https://pubrica.com/academy/include-effect-size-rules-of-thumb-meta-analysis-reports/
Pubrica Academy. “Effect Size Rules of Thumb in Meta-Analysis Reports: Arguments For and Against Their Inclusion.” Pubrica, 10 Oct. 2026, pubrica.com/academy/include-effect-size-rules-of-thumb-meta-analysis-reports/.
Pubrica Academy (2026) Effect Size Rules of Thumb in Meta-Analysis Reports: Arguments For and Against Their Inclusion. Pubrica. Available at: https://pubrica.com/academy/include-effect-size-rules-of-thumb-meta-analysis-reports/ (Accessed: 15 October 2026).
Pubrica Academy. “Effect Size Rules of Thumb in Meta-Analysis Reports: Arguments For and Against Their Inclusion.” Pubrica. October 10, 2026. https://pubrica.com/academy/include-effect-size-rules-of-thumb-meta-analysis-reports/
Pubrica Academy, ‘Effect Size Rules of Thumb in Meta-Analysis Reports: Arguments For and Against Their Inclusion’ (Pubrica, 10 October 2026) <https://pubrica.com/academy/include-effect-size-rules-of-thumb-meta-analysis-reports/> accessed 15 October 2026.
Pubrica Academy. Effect Size Rules of Thumb in Meta-Analysis Reports: Arguments For and Against Their Inclusion. [Internet]. Pubrica; 2026 Oct 10 [cited 2026 Oct 15]. Available from: https://pubrica.com/academy/include-effect-size-rules-of-thumb-meta-analysis-reports/
Pubrica Academy, “Effect Size Rules of Thumb in Meta-Analysis Reports: Arguments For and Against Their Inclusion,” Pubrica, Oct. 10, 2026. [Online]. Available: https://pubrica.com/academy/include-effect-size-rules-of-thumb-meta-analysis-reports/
@misc{/la_effect_size_rules_of_thumb_in_reports_2026,
author = {{Pubrica Academy}},
title = {Effect Size Rules of Thumb in Meta-Analysis Reports: Arguments For and Against Their Inclusion},
howpublished = {Pubrica},
year = {2026},
month = {oct},
url = {https://pubrica.com/academy/include-effect-size-rules-of-thumb-meta-analysis-reports/},
note = {Accessed: <span data-cite-acc="iso">2026-10-15</span>}
}This article is written to be learned from. Cite it if you are referring to what it says; if you are using the evidence behind it, cite the sources listed above instead.
Effect-Size Reporting Policy Note
The review’s approach to magnitude labeling should be decided and documented.
[ ] Decided whether to include generic small/medium/large labels anywhere in the report [ ] Stated this decision explicitly in the methods section [ ] Identified a minimally important difference or comparator trial for the main discussion's interpretation [ ] If using a generic label in a plain-language summary, flagged it as a rough convention [ ] Checked that results/discussion interpretation does not rely on the generic label alone
Pubrica AcademyWritten and maintained by Pubrica’s subject matter experts. Articles are revised when the underlying standards change, and each article states the date on which it was last reviewed.
How Pubrica can help
Pubrica’s evidence synthesis team assists review authors in selecting and documenting a defensible, field-appropriate approach to effect-size interpretation, rather than defaulting to an unexamined label.