Contents
Academy · Research Data, Analytics & AI · Funnel Plots and Publication Bias
The Funnel Plot in Meta-Analysis: Construction, Interpretation and Use
A funnel plot displays the effect size of each study against its precision in order to assess publication bias. A 2011 BMJ methods paper cautions, however, that asymmetry may reflect genuine heterogeneity or chance as often as missing studies; the plot therefore prompts further investigation rather than providing a conclusion in itself.
A funnel plot is a scatter plot used in meta-analysis that places each included study's estimated effect size on the horizontal axis against a measure of its precision, usually standard error or sample size, on the vertical axis, with precision typically plotted so that larger, more precise studies appear near the top. In the absence of publication bias or other causes of asymmetry, studies are expected to scatter symmetrically around the pooled effect estimate, with smaller, less precise studies spreading more widely near the bottom and larger studies clustering more tightly near the top, producing a shape resembling an inverted funnel1. A visibly asymmetric funnel, with studies missing from one side, is the classic visual signal that methodologists regard as a possible indication that smaller studies showing unfavorable or null results were less likely to be published or included.
Learning objectives
- The construction of a funnel plot and the expected appearance of a symmetric plot
- Why asymmetry is a signal for investigation rather than proof of publication bias
- Why formal statistical tests for funnel plot asymmetry require a minimum number of included studies
Common misconceptions
- “An asymmetric funnel plot proves that a meta-analysis is affected by publication bias.” The 2011 methods paper by Sterne and colleagues in the BMJ explicitly cautions that funnel plot asymmetry has multiple possible causes beyond publication bias, including genuine heterogeneity between study populations, differences in methodological quality across studies of different sizes, and chance. It recommends that these alternative explanations be investigated before asymmetry is attributed to publication bias alone.
Construction of a funnel plot
Each point on a funnel plot represents one included study, positioned by its estimated effect size on the horizontal axis and a measure of its precision, most commonly the standard error of the effect estimate, plotted on an inverted vertical axis so that more precise (typically larger) studies appear toward the top1. Under the assumption that all relevant studies, large and small, are equally likely to have been conducted, found, and included, the scatter of points is expected to form a roughly symmetric, funnel-like shape around the pooled effect estimate: small studies, having less precision, scatter more widely near the base, while larger, more precise studies cluster more tightly near the apex1. The plot is a visual diagnostic tool, examined by eye in the first instance, designed specifically to make immediately visible any systematic absence of studies in one region of the plot, commonly small studies with unfavorable results.
Asymmetry and its multiple causes
Egger and colleagues' original 1997 paper in the BMJ proposed a formal statistical test for funnel plot asymmetry, treating a significant asymmetry as a flag for possible publication bias or other small-study effects2. Sterne and colleagues' later 2011 methods paper, also in the BMJ, substantially refined this guidance, stating explicitly that asymmetry in a funnel plot can arise from several distinct sources beyond publication bias: genuine heterogeneity where smaller trials were conducted in different populations or under different protocols than larger ones, methodological differences correlated with study size (for example, smaller trials having systematically weaker methodology), and chance, especially when relatively few studies are included3. The paper's central recommendation is that an asymmetric funnel plot should prompt an investigation into which of these explanations fits the specific body of evidence, rather than being read as automatic confirmation of publication bias.
Formal tests and a documented case of extreme asymmetry
Sterne and colleagues also caution against relying on funnel plot asymmetry tests, including Egger's test, when a meta-analysis includes fewer than about ten studies, because the test has low power to distinguish genuine asymmetry from chance variation with so few data points, and recommend visual inspection alongside, rather than instead of, any formal test in that situation3. Methodological work on citation concentration offers a related, documented illustration of how skewed, non-random patterns in a body of literature can distort a metric's apparent shape: Arnold and Fowler's bibliometric analysis of the International Journal of Nonlinear Science and Numerical Simulation found that a small number of editor-linked publications supplied over 70% of the citations counted toward the journal's impact factor, a concentration pattern conceptually parallel to the kind of asymmetric, non-representative distribution a funnel plot is designed to surface in a body of trial evidence4. The broader lesson from both literatures is the same: a skewed distribution, whether of citations or of study results, is a prompt to investigate its specific cause, not a self-explanatory finding on its own.
Variations on the classic funnel plot
Several variations on the standard funnel plot address specific limitations of the basic version. A contour-enhanced funnel plot overlays shaded regions corresponding to conventional statistical significance thresholds, which helps distinguish asymmetry caused by the selective non-publication of small, non-significant studies from asymmetry caused by other sources, since publication bias specifically predicts missing studies clustering in the non-significant region3. Sterne and colleagues also note that the choice of precision measure on the vertical axis (standard error, sample size or the inverse of variance) can itself affect how symmetric or asymmetric a plot appears for the same underlying data, which is why they recommend standard error as the default choice for the vertical axis in most circumstances, specifically because it behaves most consistently across different effect measures and study designs3.
A practical workflow for the responsible use of funnel plots
A defensible workflow for using a funnel plot in a systematic review or meta-analysis starts with confirming that at least about ten studies are available before relying on any formal asymmetry test, follows with a visual inspection of the plot's overall shape, and then, if asymmetry is present, examines alternative explanations (heterogeneity, correlated study quality and chance) before considering publication bias as the explanation3. The Cochrane Handbook's guidance situates this workflow within the broader GRADE assessment of a body of evidence, where publication bias is one of five domains considered, and explicitly instructs review authors to document which alternative explanations were considered and ruled out, rather than reporting an asymmetry finding without that context1.
Sources of small-study effects other than missing studies
Sterne and colleagues use the broader term “small-study effects” specifically because several distinct mechanisms, not only missing unpublished studies, can make smaller trials in a meta-analysis report systematically different, usually larger, effects than bigger trials3. Smaller trials are sometimes conducted in more selected, higher-risk patient populations where the true effect is larger; smaller trials sometimes have less rigorous methodology, such as inadequate allocation concealment, which can inflate observed effects; and smaller trials, precisely because they are quicker and cheaper to run, are sometimes stopped early for apparent benefit at a point where the observed effect is, by chance, larger than the true underlying effect3. A funnel plot cannot, on its own, distinguish which of these mechanisms, including publication bias, is driving an observed asymmetry; that distinction requires direct examination of the characteristics of the specific smaller studies involved, not just at the shape of the plot itself. In practice, this entails retrieving the smaller, more extreme studies identified from the plot and comparing their populations, methodological quality and publication timing with those of the larger studies before drawing a conclusion about what, if anything, the asymmetry reflects; a funnel plot is valuable as a diagnostic precisely because it directs reviewers to the studies that require closer examination, not because it resolves the question of publication bias on its own. Reporting this follow-up check explicitly in a review's methods or discussion section, rather than only reporting the asymmetry test's result, is what makes a funnel plot substantive evidence, rather than a decorative figure, about the reliability of the pooled estimate it accompanies.
A worked illustration demonstrates how these mechanisms interact. In a hypothetical meta-analysis of twelve trials, the funnel plot shows the five smallest trials clustered noticeably to the favorable side of the pooled estimate, producing a visibly asymmetric funnel; before concluding that unpublished unfavorable small trials are missing, a reviewer following Sterne and colleagues' recommended workflow would check whether those five smallest trials also share a methodological feature (for example, the absence of blinded outcome assessment in all five), or whether they were conducted in a notably different patient population from that of the larger trials3. If the five smaller trials do share a specific methodological weakness correlated with their size, the more defensible conclusion is that methodological quality, not missing unpublished evidence, is driving the asymmetry, and the appropriate response is a sensitivity analysis excluding or downweighting those trials on quality grounds rather than a formal publication-bias adjustment such as trim-and-fill1. Only if no such alternative explanation holds up under this specific scrutiny does the asymmetry become genuine evidence pointing toward missing studies, which is the order of investigation the 2011 BMJ guidance recommends precisely because the visual pattern alone cannot distinguish between these competing explanations3.
Key points
Key points
A funnel plot displays study precision against effect size to assess whether there is a systematic gap in the evidence. A 2011 BMJ methods paper states explicitly that asymmetry can reflect genuine heterogeneity, correlated methodological quality or chance as readily as publication bias. Formal asymmetry tests require at least about ten studies to be reliable, and any finding of asymmetry should be documented together with the alternative explanations considered and ruled out.
Practical application
Practical application
- The funnel plot should be inspected visually before, or alongside, any formal asymmetry test. A formal test should not be relied upon alone, particularly in a meta-analysis containing fewer than approximately ten studies.
- Alternative explanations should be investigated before asymmetry is attributed to publication bias. It should be established whether smaller and larger studies differ systematically in population, protocol or methodological quality before missing studies are concluded to be the cause.
- The number of included studies should be reported alongside any funnel plot asymmetry test result. This allows readers to judge whether the test had adequate power to detect genuine asymmetry.
Frequently asked questions
No. A symmetric funnel plot is reassuring but not conclusive, since publication bias can in principle be present without producing a detectable asymmetry, particularly with few studies or specific patterns of missing evidence; a symmetric plot reduces but does not eliminate the possibility of bias.
Sterne and colleagues' 2011 BMJ guidance recommends against relying on formal asymmetry tests, including Egger's test, with fewer than about ten studies, since the test has low statistical power to distinguish real asymmetry from chance in smaller meta-analyses; visual inspection should still be used but interpreted cautiously in that situation.
References
- Deeks JJ, Higgins JPT, Altman DG (editors). Chapter 10: Analysing data and undertaking meta-analyses. In: Cochrane Handbook for Systematic Reviews of Interventions. Cochrane, 2024. read 10 October 2026. View on Cochrane Training
- Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ. 1997;315(7109):629-634. read 10 October 2026. View on The BMJ
- Sterne JAC, Sutton AJ, Ioannidis JPA, et al. Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials. BMJ. 2011;343:d4002. read 10 October 2026. View on The BMJ
- Arnold DN, Fowler KK. Nefarious Numbers. arXiv:1010.0278 [math.HO]. 2011. read 10 October 2026. View on arXiv
- Guyatt GH, Oxman AD, Vist G, et al. GRADE guidelines: 4. Rating the quality of evidence - study limitations (risk of bias). Journal of Clinical Epidemiology. 2011;64(4):407-415. read 10 October 2026. View on Journal of Clinical Epidemiology
- Higgins JPT, Thompson SG. Quantifying heterogeneity in a meta-analysis. Statistics in Medicine. 2002;21(11):1539-1558. read 10 October 2026. View on PubMed
- DerSimonian R, Laird N. Meta-analysis in clinical trials. Controlled Clinical Trials. 1986;7(3):177-188. read 10 October 2026. View on ScienceDirect
- Riley RD, Higgins JPT, Deeks JJ. Interpretation of random effects meta-analyses. BMJ. 2011;342:d549. read 10 October 2026. View on The BMJ
- Gardner MJ, Altman DG. Confidence intervals rather than P values: estimation rather than hypothesis testing. BMJ. 1986;292(6522):746-750. read 10 October 2026. View on The BMJ
- Greenland S, Senn SJ, Rothman KJ, Carlin JB, Poole C, Goodman SN, Altman DG. Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. European Journal of Epidemiology. 2016;31(4):337-350. read 10 October 2026. View on European Journal of Epidemiology
Every source above is a peer-reviewed article, a publisher’s record of one, or a standard issued by the body that owns it. Each carries the date it was last read. Where a standard moves, this article is revised rather than left up with a note.
How to cite this article
Pick your referencing style and copy it. Your university or target journal may vary the punctuation — check its guide if it has one.
Pubrica Academy. (2026, October 10). The Funnel Plot in Meta-Analysis: Construction, Interpretation and Use. Pubrica. https://pubrica.com/academy/funnel-plots-in-meta-analysis/
Pubrica Academy. “The Funnel Plot in Meta-Analysis: Construction, Interpretation and Use.” Pubrica, 10 Oct. 2026, pubrica.com/academy/funnel-plots-in-meta-analysis/.
Pubrica Academy (2026) The Funnel Plot in Meta-Analysis: Construction, Interpretation and Use. Pubrica. Available at: https://pubrica.com/academy/funnel-plots-in-meta-analysis/ (Accessed: 15 October 2026).
Pubrica Academy. “The Funnel Plot in Meta-Analysis: Construction, Interpretation and Use.” Pubrica. October 10, 2026. https://pubrica.com/academy/funnel-plots-in-meta-analysis/
Pubrica Academy, ‘The Funnel Plot in Meta-Analysis: Construction, Interpretation and Use’ (Pubrica, 10 October 2026) <https://pubrica.com/academy/funnel-plots-in-meta-analysis/> accessed 15 October 2026.
Pubrica Academy. The Funnel Plot in Meta-Analysis: Construction, Interpretation and Use. [Internet]. Pubrica; 2026 Oct 10 [cited 2026 Oct 15]. Available from: https://pubrica.com/academy/funnel-plots-in-meta-analysis/
Pubrica Academy, “The Funnel Plot in Meta-Analysis: Construction, Interpretation and Use,” Pubrica, Oct. 10, 2026. [Online]. Available: https://pubrica.com/academy/funnel-plots-in-meta-analysis/
@misc{/la_funnel_plots_meta_analysis_2026,
author = {{Pubrica Academy}},
title = {The Funnel Plot in Meta-Analysis: Construction, Interpretation and Use},
howpublished = {Pubrica},
year = {2026},
month = {oct},
url = {https://pubrica.com/academy/funnel-plots-in-meta-analysis/},
note = {Accessed: <span data-cite-acc="iso">2026-10-15</span>}
}This article is written to be learned from. Cite it if you are referring to what it says; if you are using the evidence behind it, cite the sources listed above instead.
Funnel Plot Interpretation Checklist
This checklist should be used when a funnel plot in a meta-analysis is built or interpreted.
[ ] Confirmed the plot uses a standard precision measure (standard error or sample size) on the vertical axis [ ] Visually inspected the plot for asymmetry before relying on a formal test [ ] Checked the number of included studies before trusting a formal asymmetry test (Egger's test needs roughly 10+) [ ] Investigated genuine heterogeneity and methodological-quality differences as alternative explanations for asymmetry [ ] Avoided concluding publication bias from asymmetry alone without this further investigation
Pubrica AcademyWritten and maintained by Pubrica’s subject matter experts. Articles are revised when the underlying standards change, and each article states the date on which it was last reviewed.
How Pubrica can help
Pubrica’s meta-analysis support team prepares funnel plots alongside formal asymmetry testing and investigates heterogeneity and methodological-quality explanations before attributing any asymmetry to publication bias.