Contents
On this page
- Inflation of the pooled estimate by missing negative studies
- Detection by funnel plots and Egger’s test
- Insufficiency of any single test according to the Cochrane Handbook
- A documented case: publication of antidepressant trials
- Trial registries as a countermeasure to publication bias
- Trim-and-fill and other sensitivity analyses
- Appropriate responses to a signal of publication bias
Academy · Research Data, Analytics & AI · Publication Bias
Publication Bias in Systematic Reviews: Mechanisms, Detection and Sensitivity Analysis
Egger’s 1997 test for funnel-plot asymmetry provided systematic reviewers with a verifiable statistical signal of publication bias. The Cochrane Handbook, however, is explicit that no single test can confirm or rule it out, and a credible review therefore reports multiple checks rather than a single plot.
Publication bias is the tendency for studies with statistically significant or otherwise “positive” findings to be published, and to be published faster, more often, and in more accessible venues, than studies with null or negative findings on the same question. When a systematic review pools only the published literature on a topic, and that published literature systematically over-represents positive results, the pooled effect estimate the review produces is inflated relative to the true effect, sometimes substantially. Detecting and addressing this is one of the core methodological tasks of conducting a credible systematic review or meta-analysis, and it has its own dedicated set of statistical tools, chief among them the funnel plot and Egger's regression test, alongside explicit guidance from the Cochrane Handbook on how to interpret what those tools can and cannot tell a reviewer.
Learning objectives
- The mechanism by which publication bias inflates a pooled effect estimate in a meta-analysis
- How a funnel plot and Egger’s test are used to detect it, and what asymmetry in a funnel plot indicates
- Why the Cochrane Handbook states that no single test can confirm or rule out publication bias
Common misconceptions
- “A symmetrical funnel plot proves a systematic review's underlying literature is free of publication bias.” Egger's original 1997 paper, and the Cochrane Handbook's current guidance, both treat funnel-plot asymmetry as one detectable signal among several possible causes, including genuine small-study effects unrelated to publication bias, and explicitly caution that a symmetrical plot does not rule bias out, particularly when the number of included studies is small, since funnel-plot methods lose statistical power with fewer than roughly ten studies.
Inflation of the pooled estimate by missing negative studies
The mechanism of publication bias is straightforward: if studies finding a treatment effective are more likely to be submitted, accepted, and published quickly than studies finding no effect on the same question, then the pool of published studies available for a systematic review to search and include is not a representative sample of all the research that was conducted. A meta-analysis that pools only what was published will therefore tend to produce an effect estimate biased toward the positive result, because the null and negative studies that would have pulled the average back toward the true effect were never available to be found and included1. This is distinct from, though related to, outcome reporting bias, where a study is published but selectively reports only its significant outcomes, and from the broader file-drawer problem, where entire studies with null results are never submitted for publication at all. A systematic review of the empirical evidence on both publication bias and outcome reporting bias by Dwan and colleagues found consistent evidence across multiple cohorts of registered studies that trials and outcomes with statistically significant, “positive” results were more likely to be published and more likely to be fully reported than those with null or unfavorable results, confirming that both mechanisms operate broadly across medical research rather than being confined to a handful of unusual fields9.
Detection by funnel plots and Egger’s test
A funnel plot graphs each included study's effect estimate against a measure of its precision, typically standard error, with smaller, less precise studies expected to scatter more widely around the pooled effect and larger, more precise studies expected to cluster more tightly near it, producing a roughly symmetrical, funnel-shaped pattern in the absence of bias. Egger and colleagues formalized the detection of asymmetry in this pattern into a statistical test, published in the BMJ in 1997, that regresses a study's standardized effect estimate against its precision and tests whether the regression intercept differs significantly from zero, a signal of the kind of asymmetry publication bias would be expected to produce2. The paper's own validation involved applying the test to 37 published meta-analyses, finding evidence of asymmetry consistent with publication bias in a meaningful subset of them, which helped establish the test as a standard reported alongside funnel plots in subsequent systematic reviews2.
Insufficiency of any single test according to the Cochrane Handbook
The current Cochrane Handbook for Systematic Reviews of Interventions is explicit that funnel-plot asymmetry has more than one possible cause, and publication bias is only one of them; genuine heterogeneity, where smaller studies were conducted in different populations or settings that happen to show larger effects, or an artifact of choice of effect measure, can also produce an asymmetric-looking plot without publication bias being present at all3. The Handbook also notes that Egger's test and related tests require a reasonable number of included studies, generally recommended as at least ten, to have adequate statistical power to detect asymmetry reliably, which means a review with a small number of included studies cannot meaningfully rule out publication bias just because its funnel plot looks symmetrical or its Egger's test is non-significant3. For this reason, the Handbook recommends interpreting funnel-plot and regression-test results alongside other evidence, such as a search of trial registries for unpublished or ongoing studies, rather than treating any single statistical test as conclusive on its own3.
A documented case: publication of antidepressant trials
One of the most concrete documented illustrations of publication bias in medicine comes from Turner and colleagues' 2008 study in the New England Journal of Medicine, which compared the published antidepressant trial literature against the complete set of trials the US Food and Drug Administration had on file, including trials that were never published5. The study found that of 74 FDA-registered trials, 37 of 38 trials with positive results were published, while only 14 of 36 trials with negative or questionable results were published, and several of the negative trials that were published were written up in a way that conveyed a more positive outcome than the FDA's own review of the data supported5. Based on the published literature alone, the apparent effect size of the antidepressants studied appeared substantially larger than the effect size calculated once the complete, FDA-held dataset, including unpublished trials, was included, a direct, quantified demonstration of the inflation mechanism publication bias is expected to produce5.
Trial registries as a countermeasure to publication bias
The case documented by Turner and colleagues is part of the reason mandatory trial registration has become standard practice in clinical research: if every trial must be registered before it begins, with its planned outcomes stated in advance, a systematic reviewer can at least identify that an unpublished or selectively reported trial exists, even if its full results remain unavailable. Zarin and colleagues, describing the development and function of the ClinicalTrials.gov registry, note that it was built specifically to create a public, searchable record of trials as they are registered, independent of whether or how their results are later published, which gives systematic reviewers a way to detect missing studies that a database search of the published literature alone cannot provide7. The ICMJE's own policy requiring trial registration as a condition of publication in its member journals, in place since 2005, was adopted specifically to strengthen this check, by making registration a prerequisite most trialists cannot avoid if they intend to seek publication in a major journal later8.
Trim-and-fill and other sensitivity analyses
Duval and Tweedie's trim-and-fill method, one of the most commonly reported sensitivity analyses once funnel-plot asymmetry is detected, works by iteratively removing the most extreme small studies on the side of the funnel plot causing the asymmetry, re-estimating the pooled effect from the trimmed set, and then adding back both the removed studies and their mirror-image counterparts on the opposite side of the plot to restore symmetry, producing an adjusted pooled estimate that reflects what the result might look like if the apparently missing studies existed and were included4. The method has known limitations the Cochrane Handbook flags directly: it assumes the asymmetry is caused specifically by publication bias rather than genuine heterogeneity, and simulation studies have found it can either under- or over-correct depending on how much heterogeneity is actually present among the included studies, which is why the Handbook describes it as an exploratory adjustment to report transparently alongside the original estimate, not a replacement for it2. McGauran and colleagues' broader review of reporting bias in medical research similarly cautions that no single sensitivity analysis can fully substitute for a search strategy and registry check that was comprehensive from the outset6.
Appropriate responses to a signal of publication bias
A systematic review that addresses publication bias rigorously builds in checks before the final analysis rather than only after seeing a result, including a comprehensive search strategy that extends beyond the most convenient databases, a check of clinical trial registries such as ClinicalTrials.gov for completed but unpublished trials on the same question, and explicit documentation of the funnel plot and any formal asymmetry test used, with a stated interpretation of what the result does and does not demonstrate given the number of studies available3. Where a meaningful asymmetry signal is present and cannot be explained by heterogeneity, sensitivity analyses such as the trim-and-fill method can estimate how the pooled effect might shift if the missing studies were accounted for, though the Handbook treats this as an exploratory adjustment to report transparently rather than a corrected, final answer3.
Key points
Key points
Publication bias inflates the pooled effect of a meta-analysis through the systematic exclusion of unpublished negative or null studies. Funnel plots and Egger’s test can detect a signal of such bias, but the Cochrane Handbook is explicit that neither tool alone confirms or rules it out; a credible review therefore combines them with registry searches and transparent reporting.
Practical application
Practical application
- Searches should extend beyond convenient databases and include trial registries. Searching only one or two familiar databases systematically misses unpublished studies; a registry search for completed but unpublished trials on the same question is a direct check for the file-drawer problem.
- Funnel plots and asymmetry tests should be reported with an explicit caveat regarding the number of studies. If a review includes fewer than roughly ten studies, it should state explicitly that the funnel plot and any formal test have limited power to detect asymmetry, rather than presenting a symmetrical plot as reassurance.
- An asymmetry signal should prompt sensitivity analysis rather than be treated as a final verdict. If asymmetry is found and heterogeneity does not explain it, a sensitivity analysis such as trim-and-fill should be reported transparently, together with its limitations, rather than its adjusted estimate being substituted silently for the primary result.
Frequently asked questions
No. The Cochrane Handbook notes that genuine heterogeneity between smaller and larger studies, or the specific choice of effect measure, can also produce an asymmetric-looking plot, so asymmetry is a signal to investigate further rather than direct proof of publication bias on its own.
The Cochrane Handbook generally recommends at least around ten included studies before relying on these tools, since they have limited statistical power to detect asymmetry with fewer studies than that.
Publication bias concerns whether an entire study gets published at all based on its overall findings; outcome reporting bias concerns whether a published study selectively reports only some of its originally planned outcomes, typically the significant ones, while omitting others.
Yes, in a related but distinct sense: pre-registering a review's protocol, for example through PROSPERO, commits the reviewers to a pre-specified search and analysis plan, which reduces the risk that the reviewers themselves selectively report only the synthesis approaches that produced a favored result.
References
- Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ. 1997;315(7109):629-634. read 10 October 2026. View on BMJ
- Higgins JPT, Thomas J, Chandler J, et al (editors). Cochrane Handbook for Systematic Reviews of Interventions, version 6.4. Cochrane, 2023, Chapter 13: Assessing risk of bias due to missing results. read 10 October 2026. View Cochrane Handbook
- Sterne JAC, Sutton AJ, Ioannidis JPA, et al. Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials. BMJ. 2011;343:d4002. read 10 October 2026. View on BMJ
- Duval S, Tweedie R. Trim and Fill: A Simple Funnel-Plot-Based Method of Testing and Adjusting for Publication Bias in Meta-Analysis. Biometrics. 2000;56(2):455-463. read 10 October 2026. View on Biometrics
- Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R. Selective Publication of Antidepressant Trials and Its Influence on Apparent Efficacy. N Engl J Med. 2008;358(3):252-260. read 10 October 2026. View on New England Journal of Medicine
- McGauran N, Wieseler B, Kreis J, Schüler YB, Kölsch H, Kaiser T. Reporting bias in medical research - a narrative review. Trials. 2010;11:37. read 10 October 2026. View on Trials
- Zarin DA, Tse T, Williams RJ, Califf RM, Ide NC. The ClinicalTrials.gov Results Database - Update and Key Issues. N Engl J Med. 2011;364(9):852-860. read 10 October 2026. View on New England Journal of Medicine
- De Angelis C, Drazen JM, Frizelle FA, et al; International Committee of Medical Journal Editors. Clinical Trial Registration: A Statement from the International Committee of Medical Journal Editors. N Engl J Med. 2004;351(12):1250-1251. read 10 October 2026. View on New England Journal of Medicine
- Dwan K, Altman DG, Arnaiz JA, et al. Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias. PLoS One. 2008;3(8):e3081. read 10 October 2026. View on PLOS ONE
- Song F, Parekh S, Hooper L, et al. Dissemination and publication of research findings: an updated review of related biases. Health Technol Assess. 2010;14(8):1-193. read 10 October 2026. View on Health Technology Assessment
Every source above is a peer-reviewed article, a publisher’s record of one, or a standard issued by the body that owns it. Each carries the date it was last read. Where a standard moves, this article is revised rather than left up with a note.
How to cite this article
Pick your referencing style and copy it. Your university or target journal may vary the punctuation — check its guide if it has one.
Pubrica Academy. (2026, October 10). Publication Bias in Systematic Reviews: Mechanisms, Detection and Sensitivity Analysis. Pubrica. https://pubrica.com/academy/concepts-definitions/what-is-publication-bias-in-a-systematic-review/
Pubrica Academy. “Publication Bias in Systematic Reviews: Mechanisms, Detection and Sensitivity Analysis.” Pubrica, 10 Oct. 2026, pubrica.com/academy/concepts-definitions/what-is-publication-bias-in-a-systematic-review/.
Pubrica Academy (2026) Publication Bias in Systematic Reviews: Mechanisms, Detection and Sensitivity Analysis. Pubrica. Available at: https://pubrica.com/academy/concepts-definitions/what-is-publication-bias-in-a-systematic-review/ (Accessed: 15 October 2026).
Pubrica Academy. “Publication Bias in Systematic Reviews: Mechanisms, Detection and Sensitivity Analysis.” Pubrica. October 10, 2026. https://pubrica.com/academy/concepts-definitions/what-is-publication-bias-in-a-systematic-review/
Pubrica Academy, ‘Publication Bias in Systematic Reviews: Mechanisms, Detection and Sensitivity Analysis’ (Pubrica, 10 October 2026) <https://pubrica.com/academy/concepts-definitions/what-is-publication-bias-in-a-systematic-review/> accessed 15 October 2026.
Pubrica Academy. Publication Bias in Systematic Reviews: Mechanisms, Detection and Sensitivity Analysis. [Internet]. Pubrica; 2026 Oct 10 [cited 2026 Oct 15]. Available from: https://pubrica.com/academy/concepts-definitions/what-is-publication-bias-in-a-systematic-review/
Pubrica Academy, “Publication Bias in Systematic Reviews: Mechanisms, Detection and Sensitivity Analysis,” Pubrica, Oct. 10, 2026. [Online]. Available: https://pubrica.com/academy/concepts-definitions/what-is-publication-bias-in-a-systematic-review/
@misc{/la_publication_bias_systematic_review_2026,
author = {{Pubrica Academy}},
title = {Publication Bias in Systematic Reviews: Mechanisms, Detection and Sensitivity Analysis},
howpublished = {Pubrica},
year = {2026},
month = {oct},
url = {https://pubrica.com/academy/concepts-definitions/what-is-publication-bias-in-a-systematic-review/},
note = {Accessed: <span data-cite-acc="iso">2026-10-15</span>}
}This article is written to be learned from. Cite it if you are referring to what it says; if you are using the evidence behind it, cite the sources listed above instead.
Publication-Bias Check Protocol
These checks should be built into the methods of a systematic review before the pooled estimate is finalized.
[ ] Search strategy extends beyond the most convenient 1-2 databases [ ] Checked a trial registry (e.g. ClinicalTrials.gov) for completed, unpublished studies on the same question [ ] Generated a funnel plot if 10+ studies are included [ ] Ran a formal asymmetry test (e.g. Egger's) if study count supports it, and reported its limitations if not [ ] Considered whether heterogeneity, rather than publication bias, could explain any asymmetry found [ ] If asymmetry is unexplained by heterogeneity, reported a sensitivity analysis (e.g. trim-and-fill) transparently
Pubrica AcademyWritten and maintained by Pubrica’s subject matter experts. Articles are revised when the underlying standards change, and each article states the date on which it was last reviewed.
How Pubrica can help
Pubrica’s systematic review and meta-analysis team incorporates publication-bias checks — registry searches, funnel plots and appropriate asymmetry testing — into the review protocol from the outset.