Why the log scale makes ratio measures symmetrical
An odds ratio or risk ratio is bounded below by zero and unbounded above, so on a natural scale the effects are skewed: an odds ratio of 0.5 sits 0.5 units below the null value of 1, while its reciprocal, 2.0, sits a full unit above it. On the log scale, ln(0.5) = −0.69 and ln(2.0) = +0.69, which places the two effects equidistant from ln(1) = 0. Meta-analysis software therefore pools and plots ln(OR) or ln(RR) with its standard error, and labels the axis with the back-transformed ratios. A funnel plot drawn on the natural scale would look asymmetrical even with no bias at all, which is exactly the pattern the plot is meant to detect.
Choosing the vertical axis
The current Cochrane Handbook (chapter 13, on reporting biases) recommends the standard error of the effect estimate as the measure of study size on the vertical axis, plotted on a reversed scale so that the largest, most precise studies appear at the top. This follows Sterne and Egger's analysis of funnel plot axes, which showed that the standard error gives the expected funnel shape, with pseudo 95% confidence limits drawn as straight lines around the summary estimate. Plotting sample size instead produces a curved, less interpretable funnel.
Continuous outcomes complicate this. For a mean difference, the standard error is independent of the effect itself, so the usual plot works. For a standardised mean difference (SMD), however, the formula for the standard error includes the SMD, so studies with larger effects automatically have larger standard errors. That built-in correlation can create asymmetry where none exists. This is why, as the Q&A above notes, a sample-size-based measure such as 1/√n is often used on the vertical axis for SMDs, and why modified standard errors that remove the effect-size term have been proposed for the same purpose.
A similar artefact with binary outcomes
The standard error of a log odds ratio also depends on the event counts, so it is correlated with the estimate when events are few or the odds ratio is far from 1. Egger's regression test, which regresses the standardised effect on precision, can then give false-positive results. Alternatives designed for binary outcomes include Harbord's modified test and Peters' test, which uses the inverse of total sample size as the predictor. Choose the test before looking at the data and state it in the protocol.
Interpreting asymmetry with care
Funnel plot asymmetry is evidence of small-study effects, not proof of publication bias. Sterne and colleagues' recommendations in the BMJ (2011) list several explanations that should be considered:
- reporting biases, including non-publication and selective outcome reporting;
- poorer methodological quality in smaller studies, which can inflate their effects;
- true heterogeneity, for example when smaller trials enrolled higher-risk patients or delivered the intervention more intensively;
- artefacts arising from the choice of effect measure, as described above;
- chance.
The same recommendations advise against formal asymmetry tests when there are fewer than ten studies, or when the studies are of similar size, because the tests then have too little power to distinguish chance from real asymmetry. Contour-enhanced funnel plots, which shade regions of conventional statistical significance, help to judge whether missing studies would fall in areas of non-significance (pointing towards reporting bias) or elsewhere (pointing towards other causes). Adjustment methods such as trim-and-fill rest on strong assumptions and are best reported as sensitivity analyses rather than corrected estimates.
Reporting the funnel plot in your review
PRISMA 2020 asks authors to describe the methods used to assess risk of bias due to missing results and to present the results of that assessment. In practice this means stating the effect measure and its scale, the vertical axis used, the test chosen and its rationale, and the number of studies in each plot. Present one funnel plot per outcome with enough studies, and interpret it alongside risk-of-bias judgements and heterogeneity statistics rather than in isolation. For related questions see our notes on standard error in funnel plots, what an asymmetrical funnel plot indicates and the limitations of funnel plots. Our meta-analysis service covers the full analysis, including small-study effect assessment.
Common questions
Should I plot odds ratios or log odds ratios on a funnel plot?
Plot the log odds ratio, because that is the scale on which the estimates are symmetrical and on which the meta-analysis is performed. Most software, including RevMan and the R package metafor, does this automatically and labels the horizontal axis with odds ratio values on a logarithmic scale, so readers still see familiar numbers such as 0.5, 1 and 2 placed at equal distances.
How many studies do I need before testing funnel plot asymmetry?
Widely cited recommendations, reflected in the Cochrane Handbook, suggest at least ten studies. With fewer, tests such as Egger's have low power, so a non-significant result offers little reassurance and a significant one may reflect chance. Tests are also unhelpful when all studies are of similar size. In those situations, describe the plot qualitatively, if at all, and discuss the risk of missing results in other ways.
