Free scoping read. A written assessment of your manuscript in two business days, at no cost.

Start one
Contents

Academy · Research Data, Analytics & AI · Interpreting Asymmetry

What an Asymmetrical Funnel Plot Indicates: Small-Study Effects and Their Causes

An asymmetrical funnel plot indicates a small-study effect, in which smaller studies in a meta-analysis systematically report different results from larger ones. The 2011 guidance of Sterne and colleagues lists publication bias as only one of several possible causes that must be weighed against one another.

Pubrica AcademyWritten by a meta-analysis methodologist Reviewed bya biostatistics reviewer Published10 October 2026 Last checked10 October 2026 Reading time7 minutes Sources10 peer-reviewed

An asymmetrical funnel plot shows a gap or skew in the expected inverted-funnel scatter, typically a missing cluster of small, imprecise studies on one side of the pooled effect estimate. Such asymmetry specifically indicates a small-study effect: a systematic difference between the results reported by smaller and larger studies in the meta-analysis. Sterne and colleagues' 2011 BMJ methods paper, the current standard reference on this question, is explicit that small-study effects can arise from several distinct causes, and that publication bias is only one of these, not a synonym for the finding itself1.

Learning objectives

  • The causes of funnel-plot asymmetry identified in the 2011 guidance of Sterne and colleagues
  • Why asymmetry is more accurately described as a ‘small-study effect’ than as evidence of publication bias specifically
  • Investigating the alternative explanations once asymmetry has been observed

Common misconceptions

  • “An asymmetrical funnel plot demonstrates that publication bias is present.” The 2011 BMJ guidance of Sterne and colleagues lists reporting biases, poor methodological quality in smaller studies, true heterogeneity by study size, statistical artefacts and chance as separate possible causes of asymmetry, and recommends investigating which of these applies rather than assuming publication bias by default.

Causes of asymmetry in a funnel plot

Sterne and colleagues' 2011 BMJ paper organises the possible causes of funnel-plot asymmetry into five categories. The first is reporting biases in the narrow sense: publication bias itself, together with related patterns such as time-lag bias (where smaller or less favourable studies are published later), language bias, citation bias, and the selective reporting of specific outcomes or analyses within an otherwise published study1. The second category is poor methodological quality in smaller studies specifically (inadequate randomisation or allocation concealment, for example), which the authors note can inflate observed effects in a way correlated with study size, producing the same visual signature as missing studies even when no studies are missing.

The third category is true heterogeneity: smaller trials sometimes differ genuinely from larger ones in the populations they enrolled, the intensity of the intervention tested, or the baseline risk of the patients involved, any of which can produce a real difference in effect size that happens to correlate with study size. The fourth category is a statistical artefact arising from the mathematical relationship between some effect measures and their own standard errors, particularly for odds ratios, which can produce a degree of asymmetry that is a mechanical property of the measure rather than a signal about the evidence itself. The fifth category is chance, which is more likely to produce a misleadingly asymmetric-looking plot when a meta-analysis includes relatively few studies1.

‘Small-study effect’ as the more accurate term

Because all five of these categories can produce the same visual signature, a systematic difference in results correlated with study size, Sterne and colleagues deliberately use the broader, cause-neutral term “small-study effect” to describe what an asymmetric funnel plot shows, reserving “publication bias” for the specific subset of cases where that particular cause has been separately confirmed1. Lau and colleagues' 2006 BMJ editorial underscores the practical importance of this distinction: their review of systematic reviews published in the BMJ found that a majority of the reviews using a funnel plot to assess bias drew a categorical conclusion directly from an asymmetry finding, collapsing the five-category distinction back down to a single, usually publication-bias-specific, claim2.

Tang and Liu's earlier 2000 analysis of funnel-plot misuse had already identified heterogeneity specifically as an underappreciated cause of apparent asymmetry, finding that the mathematical link between an outcome's underlying heterogeneity and the shape of its funnel plot was often overlooked in applied meta-analyses, with authors proceeding directly to a publication-bias interpretation without first checking whether their own heterogeneity statistics were elevated3. A heterogeneity statistic such as I-squared or tau-squared, calculated as a routine part of most meta-analyses, is therefore one of the first measures to check once asymmetry is observed, since high heterogeneity directly supports one of the five alternative categories without requiring any further investigation.

Investigating alternative explanations in practice

In practice, assessment of Sterne and colleagues' five categories begins with the heterogeneity check described above, and then proceeds to a direct comparison of the specific smaller and larger studies driving the asymmetry, to establish whether they differ in patient population, intervention intensity, or study design in a way that would independently explain a different effect size, separate from anything about publication status1. A risk-of-bias assessment, already a standard part of most systematic reviews under Cochrane methodology, can then be checked for a correlation with study size specifically, addressing the second category directly4.

Only once heterogeneity, methodological quality, and the specific effect measure's statistical properties have been checked and set aside does reporting bias become the leading explanation by elimination, at which point searching trial registries, grey literature, and conference abstracts for unpublished studies matching the apparent gap becomes the appropriate next step, rather than reporting publication bias as a conclusion based on the plot's shape alone2. Begg and Mazumdar's and Sterne, Gavaghan, and Egger's separate analyses of formal asymmetry tests both note that this elimination process is as important for a statistically significant test result as for a visually asymmetric plot, since the tests themselves cannot distinguish between the five categories any better than visual inspection can56. The Cochrane Handbook's guidance on reporting biases and PRISMA 2020's reporting requirements both converge on the same final instruction: report which of the five categories were considered and what was found, not merely whether the plot appeared symmetrical47.

Limits of what an asymmetry finding indicates

An asymmetry finding, on its own, does not mean the pooled effect estimate from the meta-analysis is wrong, that the underlying treatment is ineffective, or that any specific individual study was fraudulent or improperly conducted. It means the pattern of results across studies of different sizes departs from what would be expected if only ordinary sampling variation were operating, which is a narrower and more cautious claim than any of those three. Duval and Tweedie's trim-and-fill method offers one means of quantifying how much the pooled estimate might change under one specific assumption, that missing studies exist and resemble a mirror image of the observed ones, but the method's developers frame the adjusted estimate it produces as illustrative of a plausible scenario, not as a corrected replacement for the original pooled result10.

Precision is also required regarding what the absence of asymmetry does not exclude: a symmetrical plot is reassuring but does not guarantee the absence of bias, since Begg and Mazumdar's and Sterne, Gavaghan, and Egger's analyses of test power both show that a non-significant or non-asymmetric result can simply reflect low statistical power rather than a genuinely unbiased evidence base, particularly in a meta-analysis built from a modest number of studies56. Both aspects of this caution, the overinterpretation of asymmetry and excessive confidence in symmetry, reflect the same underlying principle: the funnel plot's shape is one input into a broader judgement about a body of evidence, not a verdict that can be read off the chart on its own.

An asymmetry finding also does not, by itself and without further analysis, indicate the direction in which the true effect is likely to be biased. A gap among smaller studies on one side of the pooled estimate suggests the true effect could be smaller, larger, or in rare cases even reversed in direction relative to the reported pooled result, depending on which studies are missing and why, which is a further reason why the specific investigation described above is more important than the asymmetry finding in isolation. A review that reports “the funnel plot was asymmetric” without this additional context leaves a reader unable to judge how much, or in which direction, that asymmetry should affect their confidence in the review's main conclusion, which is the gap that Duval and Tweedie's sensitivity-style adjustment and a clearly documented investigation of the five causes are both intended to close. For this reason, the funnel-plot section of a methods report should set out this reasoning in full, rather than allowing a single adjective, symmetric or asymmetric, to bear the weight of the entire bias assessment.

Key points

Key points

An asymmetrical funnel plot indicates a small-study effect. The 2011 guidance of Sterne and colleagues names five distinct possible causes (reporting bias, poor methodological quality correlated with study size, true heterogeneity, statistical artefacts and chance), and an asymmetry finding should therefore prompt investigation of all five before any one of them, including publication bias, is named as the explanation.

Practical application

Practical application

  1. Heterogeneity statistics should be checked before any asymmetry is interpreted. High I-squared or tau-squared values directly support the heterogeneity explanation, which should be confirmed or excluded first.
  2. The smaller and larger studies responsible for the asymmetry should be compared directly. Population, intervention intensity and risk-of-bias ratings should be examined for a correlation with study size before any studies are assumed to be missing.
  3. A search for unpublished studies should be undertaken only after the other four explanations have been assessed. This preserves the appropriate order of investigation and avoids reporting publication bias as a default explanation.

Frequently asked questions

This has not been established as a general rule; Sterne and colleagues' guidance treats all five categories as possible without ranking them, which is why investigation of the specific body of evidence is necessary rather than assuming any one cause by default.

Yes. If heterogeneity, methodological quality differences, or a statistical artefact tied to the effect measure fully account for the pattern, no unpublished or missing studies need to exist for the asymmetry to be explained.

No. A significant test result indicates that the asymmetry is unlikely to be due to chance alone, but it does not distinguish between reporting bias, heterogeneity, methodological quality, or a statistical artefact; that distinction requires separate investigation.

References

  1. Sterne JAC, Sutton AJ, Ioannidis JPA, et al. Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials. BMJ. 2011;342:d4002. read 10 October 2026. View on The BMJ
  2. Lau J, Ioannidis JPA, Terrin N, Schmid CH, Olkin I. The case of the misleading funnel plot. BMJ. 2006;333(7568):597-600. read 10 October 2026. View on The BMJ
  3. Tang JL, Liu JLY. Misleading funnel plot for detection of bias in meta-analysis. Journal of Clinical Epidemiology. 2000;53(5):477-484. read 10 October 2026. View on Journal of Clinical Epidemiology
  4. Sterne JAC, Egger M, Moher D (editors). Chapter 10: Addressing reporting biases. In: Higgins JPT, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions, Version 5.1.0. The Cochrane Collaboration; 2011. read 10 October 2026. View on Cochrane Training
  5. Begg CB, Mazumdar M. Operating characteristics of a rank correlation test for publication bias. Biometrics. 1994;50(4):1088-1101. read 10 October 2026. View on PubMed
  6. Sterne JAC, Gavaghan D, Egger M. Publication and related bias in meta-analysis: power of statistical tests and prevalence in the literature. Journal of Clinical Epidemiology. 2000;53(11):1119-1129. read 10 October 2026. View on Journal of Clinical Epidemiology
  7. Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71. read 10 October 2026. View on The BMJ
  8. Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ. 1997;315(7109):629-634. read 10 October 2026. View on The BMJ
  9. Peters JL, Sutton AJ, Jones DR, Abrams KR, Rushton L. Comparison of two methods to detect publication bias in meta-analysis. JAMA. 2006;295(6):676-680. read 10 October 2026. View on JAMA
  10. Duval S, Tweedie R. Trim and fill: a simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics. 2000;56(2):455-463. read 10 October 2026. View on Biometrics

Every source above is a peer-reviewed article, a publisher’s record of one, or a standard issued by the body that owns it. Each carries the date it was last read. Where a standard moves, this article is revised rather than left up with a note.

How to cite this article

Pick your referencing style and copy it. Your university or target journal may vary the punctuation — check its guide if it has one.

Pubrica Academy. (2026, October 10). What an Asymmetrical Funnel Plot Indicates: Small-Study Effects and Their Causes. Pubrica. https://pubrica.com/academy/indication-of-asymmetrical-funnel-plot/

This article is written to be learned from. Cite it if you are referring to what it says; if you are using the evidence behind it, cite the sources listed above instead.

Funnel Plot Asymmetry Investigation Order

Once asymmetry is observed, the causes should be assessed in this order, rather than publication bias being assumed first.

[ ] Checked heterogeneity statistics (I-squared, tau-squared) for the outcome
[ ] Compared smaller vs. larger studies on population, intervention intensity, and design
[ ] Checked risk-of-bias ratings for a correlation with study size
[ ] Considered whether the effect measure's own mathematics could produce an artefact
[ ] Searched trial registries and grey literature for missing studies only after the above
[ ] Documented which of the five causes were ruled in or out, and why

Pubrica AcademyWritten and maintained by Pubrica’s subject matter experts. Articles are revised when the underlying standards change, and each article states the date on which it was last reviewed.

How Pubrica can help

Pubrica’s meta-analysis team examines all five documented causes of funnel-plot asymmetry for every review it supports, so that a conclusion of bias is reported only after the alternatives have been excluded.

Find something else

Search the Pubrica Academy

Access is free and requires no account. Enter a topic, or open the full library to filter by category, sub-topic or resource type.

Excluding the four alternative explanations for funnel plot asymmetry

A scoping assessment is free and takes two business days. It provides a written assessment of what the work requires and which specialist would undertake it.

Get a free quote

Regulations

Check your project against this rule

  1. 1Your work
  2. 2Your details
  3. 3Send
What do you need?

Material you send is held under signed confidentiality, with access limited to the assigned team and encrypted transfer. It is not used to train anything and is not passed to third-party tools. Ask for the agreement before you send anything you would rather cover first.