Free scoping read. A written assessment of your manuscript in two business days, at no cost.

Start one

Home/Insights/Evidence synthesis/Publication bias in a systematic review

Publication bias in a systematic review

Studies with positive results are more likely to be published, published sooner and published in English — so a review of the published literature over-estimates effects by construction.

Bias and critical appraisal

What it is

Publication bias is the tendency for the publication of a study to depend on the direction and significance of its results. It is one of several reporting biases, alongside time-lag bias, language bias, outcome reporting bias and selective analysis reporting.

The consequence for a review is systematic: the studies available to be found are a biased sample of the studies conducted, so a faithful synthesis of them is still biased.

Detecting it, and the limits of detection

Funnel plots show effect size against precision. Asymmetry is consistent with publication bias — but also with heterogeneity, with small-study effects arising from genuine differences in conduct, and with chance.

Tests for asymmetry such as Egger’s require an adequate number of studies; Cochrane guidance advises against them with fewer than about ten, since power is too low for a negative result to mean anything.

The honest statement is that funnel plot asymmetry raises the possibility of reporting bias and does not establish it. Papers that describe a symmetrical funnel plot as evidence that publication bias is absent are over-reading it.

Reducing it at source

Search trial registries, not just bibliographic databases, and compare registered outcomes with published ones — which also detects outcome reporting bias.

Search regulatory sources where they exist; drug regulators hold trials that were never published.

Include grey literature, theses and conference abstracts. Do not restrict by language: restricting to English is a decision that imports a known bias and must be justified, not assumed.

Contact investigators for unpublished data where the gap matters.

Reporting it

State what was done to look for unpublished work, what was found, and how the possibility of reporting bias affects the conclusion. GRADE includes publication bias as a domain and allows the certainty rating to be downgraded for it.

In practice

Fix, flag or query.

The distinction that keeps an edit defensible: what an editor may change silently, what must be shown to the author, and what only the author can answer.

  • Do not: claim absence of publication bias from a symmetrical funnel plot.
  • Do not: test for asymmetry with fewer than about ten studies.
  • Search: registries and regulatory sources, not only databases.

Sources

  • Cochrane Handbook, chapter on reporting biases
  • GRADE Working Group, on publication bias as a certainty domain

Sources are given so each claim can be checked at its origin. Where a tool’s current maintenance status could not be confirmed, the page says to check it rather than asserting it.

Get a free quote

Regulations

Check your project against this rule

  1. 1Your work
  2. 2Your details
  3. 3Send
What do you need?

Material you send is held under signed confidentiality, with access limited to the assigned team and encrypted transfer. It is not used to train anything and is not passed to third-party tools. Ask for the agreement before you send anything you would rather cover first.