Free scoping read. A written assessment of your manuscript in two business days, at no cost.

Start one
Contents

Academy · Research Data, Analytics & AI · Effect Size Basics

Effect Size in Meta-Analysis: Definition, Measures and What Constitutes a Good Effect Size

Effect size is the quantity that a meta-analysis pools, and it is distinct from the p-value. No fixed value qualifies as a good effect size, because the clinical or practical meaning of an identical effect-size value varies with the outcome, population and field it describes.

Pubrica AcademyWritten by a meta-analysis methodologist Reviewed bya biostatistics reviewer Published10 October 2026 Last checked10 October 2026 Reading time7 minutes Sources10 peer-reviewed

An effect size is a standardised or unstandardised numerical summary of the magnitude of a relationship or difference, calculated from a single study, which a meta-analysis then uses as its basic unit of synthesis. It is deliberately a different quantity from a p-value: a p-value indicates whether an observed result is unlikely under a null hypothesis of no effect, whereas an effect size indicates how large the observed difference or relationship is, in a metric that in principle does not depend on sample size in the way that a p-value does. The related question of which effect size counts as “good” cannot be answered by a lookup table: published guidance states explicitly that any fixed numerical threshold for a “good” or “meaningful” effect must be interpreted against the specific outcome, population and measurement context of a study, not applied as a universal rule.

Learning objectives

  • The definition of effect size and its precise distinction from the p-value as a measure of an effect
  • The common families of effect size used in meta-analysis (mean difference, standardized mean difference, odds ratio, risk ratio)
  • The reasons published methodological guidance explicitly cautions against treating any fixed effect-size value as universally good

Common misconceptions

  • “A larger effect size number is always a better, more clinically important result.” Sullivan and Feinn’s methodological paper on effect size states explicitly that statistical significance and practical importance are different questions, and that the same numerical effect size may represent a trivial change on one outcome measure and a substantial, clinically important change on another, depending entirely on what the scale and the population represent.

Definition of effect size and its distinction from the p-value

Sullivan and Feinn’s widely cited paper on effect size begins from a specific gap in common practice: a p-value indicates only whether an observed result is statistically unlikely to have occurred by chance under a null hypothesis, and that judgment is heavily influenced by sample size, since a sufficiently large sample can render even a trivially small difference statistically significant1. An effect size, by contrast, is constructed to quantify the magnitude of the observed difference or relationship largely independently of the sample size that produced it, which is why a meta-analysis pools effect sizes rather than p-values across studies1. The chapter of the Cochrane Handbook on choosing effect measures presents this as a deliberate methodological decision at the protocol stage: review authors are expected to decide in advance which effect measure best represents the outcome under study, because that decision determines what the eventual pooled estimate in the forest plot will mean to a reader2.

Common families of effect size in meta-analysis

For continuous outcomes, measured on a numerical scale such as blood pressure or a symptom score, the Cochrane Handbook specifies the mean difference where studies share a scale and the standardised mean difference where they do not2. For dichotomous outcomes, in which each participant either does or does not experience an event, the Handbook specifies the odds ratio, risk ratio and risk difference as the standard alternatives, each expressing the same underlying comparison differently: the risk ratio as a multiplicative comparison of event probabilities, the odds ratio as a comparison of odds rather than probabilities, and the risk difference as an arithmetic difference in event probability2. The meta-analysis text by Borenstein and colleagues notes that the choice between these dichotomous measures is not purely a matter of preference; odds ratios have mathematical properties, such as symmetry around 1.0 irrespective of which outcome is coded as the event, that facilitate their statistical combination across studies, which partly explains why they remain common in meta-analysis even though the risk ratio is often more directly interpretable for a clinical reader3.

Absence of a single numerical threshold for a good effect size

Cohen’s original benchmarks for standardised effect sizes (0.2 as small, 0.5 as medium and 0.8 as large for a standardised mean difference) were introduced in his 1988 text as an approximate, general-purpose convention for behavioural-science researchers who lacked any other benchmark, not as a universal scale of clinical or practical importance4. Lakens’s practical primer on calculating and reporting effect sizes states that these benchmarks should be treated with caution and that the importance of an effect size is better interpreted against effects previously observed in the same research area than against Cohen’s generic scale5. Sullivan and Feinn make the same point from a clinical perspective: a mean difference of a few points on a symptom scale may be clinically trivial for one condition and represent genuine, patient-relevant improvement for another, so whether an effect size is “good” cannot be determined without knowing what the outcome measure represents in the specific clinical context1. The practical implication for a meta-analysis is that reporting the pooled effect size with its confidence interval and a stated, field-specific benchmark for a meaningful difference on that outcome measure is more informative than reporting the number alone and leaving the reader to apply an unstated, generic rule of thumb.

The minimally important difference as an alternative to generic benchmarks

A minimally important difference (MID) is a benchmark established directly for a specific outcome measure and population, usually by methods such as asking patients how much change on a scale corresponds to a meaningful improvement in how they feel or function, which provides review authors with a context-specific standard independent of Cohen’s generic conventions1. Where a validated MID exists for the outcome being pooled, for example for a widely used pain scale or a disease-specific quality-of-life instrument, comparison of the pooled effect size with that MID addresses the question of practical importance far more directly than any generic or field-average statistical benchmark, because the MID was derived from the population and instrument that the review is using6. GRADE (Grading of Recommendations Assessment, Development and Evaluation) guidance on interpreting continuous outcomes explicitly recommends this approach where an MID is available, regarding it as a more defensible basis for judging clinical importance than a generic small, medium or large label applied without reference to the scale in question6.

Interpretation of magnitude, as well as direction, in a forest plot

A common and avoidable error in reading a meta-analysis forest plot is to consider only whether the pooled diamond crosses the line of no effect, which addresses a question of significance, while overlooking the position and width of the diamond, which convey the magnitude that an effect size is intended to express2. Two pooled estimates may both lie entirely on the same side of the null line yet support very different practical conclusions: a narrow diamond lying just beyond a clinically meaningful threshold conveys a different message from a wide diamond whose lower bound lies well beyond that threshold and whose upper bound suggests an implausibly large effect. Reading a forest plot for its effect size, and not only its statistical significance, requires the position of the diamond to be checked explicitly against a stated clinical benchmark (a generic convention, a field-specific comparison or, ideally, a validated minimally important difference) whenever a pooled result is reported, rather than treating the observation that the diamond does not cross the line as the end of interpretation. The same principle applies when forest plots from different reviews are compared: a narrower diamond in one review is not necessarily a stronger finding than a wider diamond in another unless both measure comparable outcomes on comparable scales in comparable populations, since a narrow interval may equally reflect a larger pooled sample as a more precise or more important underlying effect. For the same reason, the abstract of a review should state the effect size and its confidence interval rather than only a qualitative summary such as “the intervention was effective,” since readers assessing the relevance of a finding to their own setting require the magnitude itself, not a conclusion reached on their behalf without the supporting figure. A clinician considering whether to recommend a treatment to a specific patient, for instance, must judge whether the magnitude of benefit outweighs that patient’s particular risks and preferences, a judgment that the review cannot make on the clinician’s behalf and should not pre-empt by substituting a qualitative verdict for the figure the clinician requires. Guideline panels drawing on the pooled effect size of a review face the same requirement more formally, since the GRADE framework for moving from evidence to recommendations explicitly asks panel members to weigh the magnitude of benefit and harm rather than a qualitative label, which is possible only if the review reported the figure rather than a conclusion in its place.

Key points

Key points

An effect size quantifies the magnitude of a result independently of sample size, which is why meta-analysis pools effect sizes rather than p-values. No fixed numerical threshold, however, defines a good effect size across outcomes; that judgment depends on what the specific scale and population represent.

Practical application

Practical application

  1. The effect measure should be selected at the protocol stage rather than after the data have been examined. The choice among mean difference, standardized mean difference, odds ratio, risk ratio and risk difference should be based on the outcome type and on whether studies share a measurement scale, and should be made before extraction begins.
  2. Magnitude should be interpreted against the specific field rather than against Cohen’s generic benchmarks. The pooled effect size should be compared with previously observed effects for the same outcome measure in the same research area, rather than labelled small, medium or large out of context.
  3. The effect size should be reported with its confidence interval and a stated clinical benchmark. Authors should state explicitly what magnitude would be considered clinically meaningful for the specific outcome, rather than leaving readers to apply an unstated rule of thumb.

Frequently asked questions

No. A p-value indicates how unlikely a result is under a null hypothesis and is heavily influenced by sample size; an effect size quantifies the magnitude of the observed difference or relationship largely independently of sample size.

Not necessarily. Cohen’s benchmark of 0.8 was a general statistical convention, not a threshold of clinical importance, and the guidance of both Lakens and Sullivan and Feinn cautions against applying it outside the context for which it was intended.

The mean difference should be used if all included studies measured the outcome on the same scale and in the same units, and the standardised mean difference if they used different scales for a similar construct, in accordance with the guidance of the Cochrane Handbook.

Odds ratios have statistical properties, including symmetry irrespective of which outcome is coded as the event, that make them more convenient to combine mathematically across studies, which partly explains why they remain widely used alongside the more intuitively interpretable risk ratio.

References

  1. Sullivan GM, Feinn R. Using Effect Size-or Why the P Value Is Not Enough. Journal of Graduate Medical Education. 2012;4(3):279-282. read 10 October 2026. View on JGME
  2. Higgins JPT, Li T, Deeks JJ. Chapter 6: Choosing effect measures and computing estimates of effect. Cochrane Handbook for Systematic Reviews of Interventions. read 10 October 2026. View on training.cochrane.org
  3. Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. Introduction to Meta-Analysis. John Wiley & Sons; 2009. read 10 October 2026. View on wiley.com
  4. Cohen J. Statistical Power Analysis for the Behavioral Sciences. 2nd ed. Lawrence Erlbaum Associates; 1988. read 10 October 2026. View on routledge.com
  5. Lakens D. Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs. Frontiers in Psychology. 2013;4:863. read 10 October 2026. View on Frontiers
  6. Guyatt GH, Oxman AD, Vist GE, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336(7650):924-926. read 10 October 2026. View on BMJ
  7. Deeks JJ, Higgins JPT, Altman DG. Chapter 10: Analysing data and undertaking meta-analyses. Cochrane Handbook for Systematic Reviews of Interventions. read 10 October 2026. View on training.cochrane.org
  8. Schünemann HJ, Higgins JPT, Vist GE, et al. Chapter 15: Interpreting results and drawing conclusions. Cochrane Handbook for Systematic Reviews of Interventions. read 10 October 2026. View on training.cochrane.org
  9. Hedges LV, Olkin I. Statistical Methods for Meta-Analysis. Academic Press; 1985. read 10 October 2026. View on elsevier.com
  10. Bakker M, et al. Denouncing the use of field-specific effect size distributions to inform magnitude. PeerJ. 2021;9:e11383. read 10 October 2026. View on PeerJ

Every source above is a peer-reviewed article, a publisher’s record of one, or a standard issued by the body that owns it. Each carries the date it was last read. Where a standard moves, this article is revised rather than left up with a note.

How to cite this article

Pick your referencing style and copy it. Your university or target journal may vary the punctuation — check its guide if it has one.

Pubrica Academy. (2026, October 10). Effect Size in Meta-Analysis: Definition, Measures and What Constitutes a Good Effect Size. Pubrica. https://pubrica.com/academy/optimal-effect-size-in-meta-analysis/

This article is written to be learned from. Cite it if you are referring to what it says; if you are using the evidence behind it, cite the sources listed above instead.

Effect-Size Selection Note

The effect measure and reporting plan should be decided before data extraction.

[ ] Identified the outcome type (continuous or dichotomous) for each planned comparison
[ ] Chosen the effect measure (MD/SMD for continuous; OR/RR/RD for dichotomous) based on Handbook guidance
[ ] Confirmed this choice fits whether included studies share a measurement scale
[ ] Identified a field-specific benchmark for what magnitude would be clinically meaningful
[ ] Planned to report the effect size with its confidence interval, not as a bare number

Pubrica AcademyWritten and maintained by Pubrica’s subject matter experts. Articles are revised when the underlying standards change, and each article states the date on which it was last reviewed.

How Pubrica can help

Pubrica’s biostatistics team selects the effect measure appropriate to the outcome type and data structure and interprets the resulting pooled estimate against field-specific benchmarks rather than a generic small, medium or large label.

Find something else

Search the Pubrica Academy

Access is free and requires no account. Enter a topic, or open the full library to filter by category, sub-topic or resource type.

Suitability of the effect measure for the available data

A scoping assessment is free and takes two business days. It provides a written assessment of what the work requires and which specialist would undertake it.

Get a free quote

Regulations

Check your project against this rule

  1. 1Your work
  2. 2Your details
  3. 3Send
What do you need?

Material you send is held under signed confidentiality, with access limited to the assigned team and encrypted transfer. It is not used to train anything and is not passed to third-party tools. Ask for the agreement before you send anything you would rather cover first.