Free scoping read. A written assessment of your manuscript in two business days, at no cost.

Start one
Contents

Academy · Research Data Analytics & AI · Meta-analysis methodology

Conducting a Meta-Analysis End to End: The Analytic Decisions and Their Order

A meta-analysis is a sequence of about a dozen decisions, most of which must be made before the data are visible and almost all of which are poorly reported. This article follows the sequence in the order in which the decisions arise, identifies the currently recommended method at each step, and notes where the methodological literature disagrees.

Pubrica AcademyWritten by the medical writing and biostatistics teams Reviewed byPubrica biostatistics team Published9 October 2026 Last checked9 October 2026 Reading time13 minutes Sources42 peer-reviewed

The statistical component of a meta-analysis is comparatively brief. What requires time, and what reviewers question, is the chain of decisions surrounding it: how studies were grouped, which effect measure was chosen, how multiple effect sizes per study were handled, which between-study variance estimator was used, how the confidence interval was calculated, and whether any of that was fixed in advance. PRISMA 2020 item 13d asks for most of it explicitly. Page and colleagues have shown that reviews commonly fail to supply it.

Learning objectives

  • The order in which the analytic decisions must be made, and which must precede data extraction
  • How to choose a between-study variance estimator and a confidence-interval method appropriate to the number of studies available
  • What PRISMA 2020 item 13d requires authors to report about the model
  • How to handle more than one effect size per study without overstating precision
  • When meta-regression is informative and when it is underpowered
  • The appropriate course when the included studies cannot be pooled at all

Common misconceptions

  • “Pick fixed or random effects based on the heterogeneity statistic.” PRISMA 2020 explicitly discourages choosing the model on the basis of observed I-squared. The choice follows from whether a single true effect is assumed.
  • “The DerSimonian-Laird method is the default, so it is the right default.” Veroniki and colleagues recommend the Hartung-Knapp-Sidik-Jonkman method as standard for meta-analyses of five or more studies, with sensitivity analysis across two or three methods when fewer than ten are pooled.
  • “Every outcome in a study can go into the same model.” López-López and colleagues set out why multiple correlated effect sizes from one study break the independence assumption and understate the standard error.
  • “Subgroup differences explain the heterogeneity.” Subgroup and meta-regression comparisons are observational comparisons between studies, however the individual studies were designed.
  • “If I cannot pool, there is no synthesis.” Campbell and colleagues published the SWiM reporting guideline in the BMJ precisely because structured synthesis without meta-analysis is a legitimate and reportable method.

The primacy of decision order over method

Almost every criticism a methods reviewer makes of a meta-analysis is in fact a criticism of sequence: a decision that should have been made in the protocol was evidently made after the results appeared. The model was selected once the heterogeneity was known. The subgroup was defined once the overall estimate was null. The effect measure was changed when the first one yielded a wide interval. None of these is detectable from the final numbers, which is precisely why reviewers examine them.

The first methodological commitment is therefore one of ordering. Eligibility criteria and the grouping of studies for synthesis come first, and McKenzie and colleagues devote a full Cochrane Handbook chapter to the principle that defining the criteria and defining the synthesis groups are the same exercise 2. The effect measure, the model, the variance estimator and the interval method follow, still before extraction is complete. Diagnostics and sensitivity analyses are specified at the same time, because a sensitivity analysis devised after the primary result is fundamentally different from one that was planned.

If the review question is not yet framed tightly enough to support these decisions, the review should return to the PICO framework before touching the analysis plan.

The decision sequence, and the latest point at which each can honestly be made

DecisionMust be fixed byPrimary reference
Eligibility criteria and synthesis groupsProtocol, before searchingMcKenzie and colleagues, Cochrane Handbook ch. 3
Effect measureProtocolDeeks, Higgins and Altman, Cochrane Handbook ch. 10
Rule for multiple effect sizes per studyProtocol, before extractionLópez-López and colleagues, Research Synthesis Methods
Fixed or random effects, with reasonProtocol — explicitly not from observed I-squaredPRISMA 2020 item 13d
Between-study variance estimatorProtocolVeroniki and colleagues; Langan and colleagues
Confidence-interval methodProtocolVeroniki and colleagues; Hartung and Knapp
Moderators for meta-regressionProtocol, before any model is runThompson and Higgins, Statistics in Medicine
Sensitivity and influence analysesProtocol, reported either wayViechtbauer and Cheung, Research Synthesis Methods
Conditions for not poolingProtocolMcKenzie and Brennan, Cochrane Handbook ch. 12; SWiM

Sequence compiled from the Cochrane Handbook chapters, PRISMA 2020 and the methodological sources cited on this page. “Must be fixed by” reflects the point beyond which the decision becomes contingent on observed results.

Grouping of studies into comparable comparisons

The synthesis is defined by its groups, and the groups must be specified at a level that is both clinically meaningful and populated by the available literature. Groups defined too narrowly yield a set of single-study comparisons; groups defined too broadly yield a summary of an intervention that no study delivered. McKenzie and colleagues’ Handbook chapter on defining inclusion criteria and synthesis groups is the practical reference, and the operative discipline is to record, before extraction, the intervention categories, comparator categories, outcome domains and measurement timings that will be treated as equivalent.

Higgins and colleagues, in BMJ Global Health, extend this to complex interventions, where the components differ between studies by design 9. Their argument is that the grouping decision there is a model of how the intervention is thought to work, and should be stated as such rather than left to the eligibility table.

This is also the stage at which the extraction form takes shape: every variable needed to assign a study to a group, or to serve later as a moderator, must appear on the form. Adding a variable afterwards requires re-reading every paper, which is why deciding what data to extract is a synthesis decision rather than an administrative one.

Effect measure and effect-size multiplicity

Deeks, Higgins and Altman’s Handbook chapter on analysing data covers the choice of effect measure in detail; in brief, the measure should be the one most likely to be consistent across studies, and that consistency is an empirical claim about the included set rather than a general property of risk ratios or odds ratios 1.

The more difficult problem in practice is multiplicity. Most included studies offer more than one eligible effect size: several time points, several scales measuring the same construct, or several intervention arms against one control. Treating these as independent observations inflates the apparent amount of information and narrows the interval spuriously. López-López, Page, Lipsey and Higgins, in Research Synthesis Methods, set out the options: select one per study by a pre-stated rule, average within study, or model the dependence 11. Where dependence is modelled, Hedges, Tipton and Johnson’s robust variance estimation is the standard approach, with the small-sample corrections developed by Tipton and Pustejovsky for tests of moderators and model fit 17,25. Mavridis and Salanti’s introduction to multivariate meta-analysis addresses the case in which several outcomes are deliberately modelled jointly 18.

The particular rule chosen matters less than its documentation in advance. A per-study selection rule applied after the effect sizes have been seen introduces selection bias.

Model, variance estimator and interval method

These are three decisions, not one, and the reporting literature treats them separately. The model choice (fixed or random effects) follows from whether a single true effect is assumed across the included studies. PRISMA 2020 states the requirement plainly: the model and the reason for choosing it must be reported, and the model must not be chosen on the basis of the observed I-squared 6.

The between-study variance estimator is a separate choice. Veroniki and colleagues, in Research Synthesis Methods, reviewed the available methods and their uncertainty, and Langan and colleagues compared them in simulated meta-analyses in the same journal; restricted maximum likelihood and the Paule-Mandel estimator generally perform better than the DerSimonian-Laird moment estimator, which remains the most widely used because it is the historical default 10,16,39. Viechtbauer’s earlier work in the Journal of Educational and Behavioral Statistics on the bias and efficiency of these estimators is the technical reference.

The confidence-interval method is the third decision, and the one most often left unreported. Veroniki and colleagues’ review recommends the Hartung-Knapp-Sidik-Jonkman method (originally proposed by Hartung and Knapp in Statistics in Medicine) as standard for meta-analyses of five or more studies, with sensitivity analysis across two or three methods when fewer than ten studies are pooled 34,42. Röver, Knapp and Friede, in BMC Medical Research Methodology, examined its behaviour and a modification for the few-studies case 15. The consequences of this choice are discussed on the page on what pooling buys you: it can move a result across the conventional significance threshold without any change in the data.

5+studies: Hartung-Knapp-Sidik-Jonkman recommended as standard
<10studies: sensitivity analysis across two or three interval methods
<10studies: no test for funnel plot asymmetry

Study-count thresholds for interval method and asymmetry testing. Interval-method recommendations: Veroniki AA, Jackson D, Bender R, Kuss O, Langan D, Higgins JPT, Knapp G, Salanti G. Methods to calculate uncertainty in the estimated overall effect size from a random-effects meta-analysis. Asymmetry-testing floor: Sterne JAC and colleagues, BMJ 2011;343:d4002.

Description of heterogeneity without over-interpretation

Higgins and Thompson, in Statistics in Medicine, proposed three statistics — H, I-squared and R — to describe heterogeneity independently of the effect metric and the number of studies, and illustrated them on five datasets 33. All three serve description, not decision. None of them tests whether pooling is permissible, and PRISMA 2020 requires authors to report how heterogeneity was assessed, including the between-study variance, I-squared and prediction intervals, rather than a single verdict.

Riley, Higgins and Deeks, in the BMJ, make the interpretive case for prediction intervals: a random-effects model estimates the mean of a distribution of effects, and the spread of that distribution is what a clinician needs 21. Where the heterogeneity itself is to be modelled rather than merely described, Viechtbauer and López-López’s location-scale models in Research Synthesis Methods allow the between-study variance to depend on study characteristics 5.

Subgroups and meta-regression

Thompson and Higgins’ paper in Statistics in Medicine on how meta-regression analyses should be undertaken and interpreted remains the clearest statement of the limits. Meta-regression relates a study-level characteristic to the study-level effect, so it is an observational analysis across studies even when every study is a randomised trial, and it has very little power with the number of studies most reviews contain 32. Van Houwelingen, Arends and Stijnen, and Kalaian and Raudenbush, provide the multivariate and mixed-model framework.

Two practical disciplines prevent most of these problems. The moderators should be pre-specified and the list kept short, because the number of candidate moderators in a typical review exceeds the number of studies. Where effect sizes within studies are dependent, robust variance estimation with the small-sample corrections should be used, rather than ignoring the dependence because the software will run in either case. Nakagawa and Santos’ review of methodological issues in biological meta-analysis is a useful corrective for those working outside clinical trials, where dependence structures are more complex still.

Small-study effects and diagnostics

Sterne and colleagues’ recommendations in the BMJ are the operative guidance on funnel plot asymmetry, and the operative number is 10: these tests should generally not be used when a meta-analysis contains fewer than 10 studies, and the minimum may be substantially higher when heterogeneity is substantial 22,42. Many published reviews nevertheless apply them at five or six studies and report the non-significant result as reassurance, which it is not.

Outlier and influence diagnostics are more informative at small numbers. Viechtbauer and Cheung, in Research Synthesis Methods, set out the standard set, and Viechtbauer’s Journal of Statistical Software paper on the metafor package documents their implementation 23,24. A leave-one-out analysis, reported whether or not it changes the conclusion, is the minimum.

Design-related bias is a separate question from small-study effects. Page and colleagues’ systematic review of meta-epidemiological studies in PLoS ONE summarises the empirical evidence on how design features relate to estimated effects, and Murad and Wang’s reporting guidance in Evidence-Based Medicine covers how such meta-epidemiological work should itself be reported 13,14.

Reporting the model to permit verification

Page and colleagues’ cross-sectional analysis in the Journal of Clinical Epidemiology found flaws in the application and interpretation of statistical analyses to be common in systematic reviews of therapeutic interventions 12. A large share of those flaws are invisible to a reader because the analysis was not described in enough detail to check. PRISMA 2020 item 13d is the checklist against which to write the paragraph: the model and the reason for it, the weighting method, how heterogeneity was assessed including between-study variance, I-squared and prediction intervals, the variance estimator and confidence-interval method for random-effects models, and the prior distributions for any Bayesian analysis 6.

A statistical analysis paragraph, checked against PRISMA 2020 item 13d

Not checkable

Pooled estimates were calculated using a random-effects model because heterogeneity was high. Statistical analyses were performed in R. Publication bias was assessed using a funnel plot and Egger’s test, which was not significant.

The model was chosen from the heterogeneity statistic. PRISMA 2020 explicitly discourages this, and the sentence states it as the reason.

No variance estimator named. The reader cannot tell whether DerSimonian-Laird, REML or Paule-Mandel produced the between-study variance.

No interval method named. Veroniki and colleagues found 45% versus 35% significance across 920 Cochrane meta-analyses on this choice alone.

An asymmetry test below the stated floor, reported as reassurance. Sterne and colleagues advise against these tests under 10 studies; a non-significant result here carries no information.

Checkable

A random-effects model was used throughout, specified in the protocol on the grounds that the included interventions were delivered in different care settings and a single common effect was not plausible; the model was not selected on the basis of observed heterogeneity. Studies were weighted by inverse variance, between-study variance was estimated by restricted maximum likelihood, and confidence intervals were calculated by the Hartung-Knapp-Sidik-Jonkman method with a DerSimonian-Laird interval reported alongside as a pre-specified sensitivity analysis. Heterogeneity is reported as the between-study variance estimate with I-squared and a 95% prediction interval. With six included studies we did not test for funnel plot asymmetry; instead we report leave-one-out estimates and standardised residuals for every study. Analyses used the metafor package in R, with code and data in the supplement.

The reason for the model is substantive and pre-specified, so it cannot have been chosen from the result.

Estimator and interval method are both named, with the comparison interval given as a planned sensitivity analysis rather than a footnote.

Heterogeneity is reported as a variance plus a prediction interval, which is what item 13d asks for and what a clinician can use.

The asymmetry test is explicitly declined with the reason, and replaced by diagnostics that work at six studies.

The second paragraph is longer, but every additional clause answers a question that a methods reviewer would otherwise ask, and each maps to a named element of PRISMA 2020 item 13d, so the checklist can serve as a drafting prompt rather than a submission formality.

Synthesis when pooling is not possible

If the studies measure different constructs, if there are too few to estimate between-study variance with any stability, or if risk of bias differs so greatly that an average would be meaningless, the appropriate output is a structured synthesis without meta-analysis. McKenzie and Brennan’s Handbook chapter on synthesising and presenting findings using other methods sets out the approach, and Campbell and colleagues’ SWiM guideline in the BMJ gives the reporting items — how studies were grouped, what standardised metric or direction-of-effect summary was used, how results were presented, and what limitations the approach carries 3,7.

Two adjacent cases merit mention because reviewers encounter them often. Prediction model studies have their own synthesis framework, set out by Debray and colleagues in Statistical Methods in Medical Research, and diagnostic accuracy studies require a bivariate or hierarchical model rather than a standard univariate one; these are addressed in the pages on pooling CNN radiology studies and cardiovascular imaging AI 8. For comparisons across more than two interventions, the network meta-analysis hub covers the additional assumptions.

Key points

Key points

A meta-analysis is a sequence of decisions, and their order constitutes the methodology. Groups and effect measure come first, followed by the rule for multiple effect sizes per study, then the model, variance estimator and interval method, all before the results are visible. The model should not be chosen from the observed I-squared; PRISMA 2020 explicitly discourages this. The Hartung-Knapp-Sidik-Jonkman interval is preferred at five or more studies, with sensitivity analysis across two or three methods below ten, and funnel plot asymmetry should not be tested at all below ten studies. Where the studies cannot be pooled, SWiM provides a reportable alternative.

Application to your study

Application to your study

  1. Draft the grouping table before searching. The table specifies which interventions, comparators, outcome domains and timings will be treated as equivalent, and thereby defines both eligibility and the synthesis.
  2. Decide the one-effect-size-per-study rule in the protocol. Select, average, or model the dependence with robust variance estimation; applying a rule after the effect sizes have been seen introduces bias.
  3. State the model’s justification in substantive terms. Justify it by differences in setting, population or delivery, never by the heterogeneity statistic.
  4. Name the between-study variance estimator and the interval method separately. Both belong in the protocol and in the methods paragraph.
  5. Count the included studies before planning bias checks. With fewer than 10 studies, omit the asymmetry test and report leave-one-out estimates and influence diagnostics instead.
  6. Keep the moderator list shorter than the number of studies, and fix it in advance. Meta-regression across a handful of studies is underpowered and observational.
  7. Draft the analysis paragraph directly from PRISMA 2020 item 13d. Model and reason, weighting, heterogeneity assessment including prediction intervals, variance estimator, interval method, and priors if Bayesian.
  8. Where studies are not pooled, report the synthesis according to SWiM. Grouping, the summary metric used, presentation method and limitations — a described synthesis, not an absence of one.

Frequently asked questions

The decision should rest on substantive rather than statistical grounds. A fixed-effect model assumes one true effect common to all included studies; a random-effects model assumes that the true effects vary and estimates the mean of their distribution. If the studies differ in population, setting or mode of delivery, a random-effects model is the appropriate choice, and that justification belongs in the protocol. PRISMA 2020 item 13d requires authors to report the model and the reason for choosing it, and explicitly discourages choosing it on the basis of the observed I-squared, which is nonetheless the most common reason given in published reviews.

The methodological literature has moved away from the DerSimonian-Laird moment estimator, which remains the most widely used because it was the historical default in most software. Veroniki and colleagues reviewed the available methods and their uncertainty in Research Synthesis Methods, and Langan and colleagues compared them across simulated meta-analyses in the same journal; restricted maximum likelihood and the Paule-Mandel estimator generally perform better. Whichever estimator is used should be named in the methods and treated as a decision separate from the model and from the confidence-interval method, because all three are reported together under PRISMA 2020 item 13d.

Veroniki and colleagues recommend the Hartung-Knapp-Sidik-Jonkman method, originally proposed by Hartung and Knapp in Statistics in Medicine, as standard for meta-analyses of five or more studies, and recommend a sensitivity analysis across two or three methods when fewer than ten studies are pooled. Röver, Knapp and Friede examined its behaviour and a modification for the few-studies case in BMC Medical Research Methodology. This choice is consequential rather than cosmetic: the same authors found 45% of 920 Cochrane meta-analyses statistically significant under a Wald-type interval against 35% under Hartung-Knapp-Sidik-Jonkman.

One of three approaches should be chosen and written into the protocol before extraction: selecting a single effect size per study by a pre-stated rule, averaging the eligible effect sizes within each study, or modelling the dependence explicitly. López-López, Page, Lipsey and Higgins set out the trade-offs in Research Synthesis Methods. If the dependence is modelled, Hedges, Tipton and Johnson’s robust variance estimation is the standard approach, with Tipton and Pustejovsky’s small-sample corrections for tests of moderators. Correlated effect sizes must not be entered as independent observations: the standard error, and therefore the interval, will be too narrow.

Rarely with much confidence. Thompson and Higgins, in Statistics in Medicine, set out both the method and its limits: meta-regression relates a study-level characteristic to a study-level effect, so it is an observational comparison between studies even when all the studies are randomised trials, and it has little power at the study counts typical of clinical reviews. The moderator list should be short and pre-specified (in most reviews the number of plausible moderators exceeds the number of studies), and any finding should be presented as a hypothesis about why effects differ, not as an explanation of them.

When the included studies measure different constructs, when there are too few to estimate between-study variance with any stability, or when risk of bias varies so greatly that an average would be uninterpretable. Declining to pool does not mean forgoing synthesis. McKenzie and Brennan’s Cochrane Handbook chapter covers synthesis using other methods, and Campbell and colleagues’ SWiM guideline in the BMJ gives the reporting items: how studies were grouped, what standardised metric or direction-of-effect summary was used, how results were presented and what limitations follow. A review that explains why pooling would mislead is more useful than one that pools regardless.

PRISMA 2020 item 13d should be used as a drafting prompt. It asks for the meta-analysis model and the reason for choosing it, the weighting method, how heterogeneity was assessed (including the between-study variance, I-squared and prediction intervals), the variance estimator and confidence-interval method for random-effects models, and the prior distributions for Bayesian analyses. Page and colleagues’ cross-sectional analysis in the Journal of Clinical Epidemiology found flaws in the application and interpretation of statistical analyses to be common in reviews of therapeutic interventions, and many of these flaws are undetectable by readers because the paragraph was too brief to check.

It depends on the diagnostic. DerSimonian and Laird developed the random-effects model on eight re-analysed reviews containing between 5 and 26 randomised trials each, so small sets have always been within scope for pooling itself. Funnel plot asymmetry testing has a stated floor: Sterne and colleagues advise against it below 10 studies, and state that the minimum may be substantially higher under substantial heterogeneity. Below that threshold, outlier and influence diagnostics should be used instead; Viechtbauer and Cheung set out the standard set in Research Synthesis Methods, and leave-one-out estimates should be reported for every study whether or not they change the conclusion.

References

  1. Deeks JJ, Higgins JPT, Altman DG. Analysing data and undertaking meta-analyses. Cochrane Handbook for Systematic Reviews of Interventions. 2019:241-284. doi:10.1002/9781119536604.ch10 read 9 October 2026. doi.org/10.1002/9781119536604.ch10
  2. McKenzie JE, Brennan SE, Ryan RE, Thomson HJ, Johnston RV, Thomas J. Defining the criteria for including studies and how they will be grouped for the synthesis. Cochrane Handbook for Systematic Reviews of Interventions. 2019:33-65. doi:10.1002/9781119536604.ch3 read 9 October 2026. doi.org/10.1002/9781119536604.ch3
  3. McKenzie JE, Brennan SE. Synthesizing and presenting findings using other methods. Cochrane Handbook for Systematic Reviews of Interventions. 2019:321-347. doi:10.1002/9781119536604.ch12 read 9 October 2026. doi.org/10.1002/9781119536604.ch12
  4. The Nordic Cochrane Centre. Review Manager (RevMan) version 4.2 for Windows. The Cochrane Collaboration (software). 2003. read 9 October 2026. link.springer.com
  5. Viechtbauer W, Lopez-Lopez JA. Location-scale models for meta-analysis. Research Synthesis Methods. 2022;13:697-715. read 9 October 2026. arxiv.org
  6. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, Shamseer L, Tetzlaff JM, Akl EA, Brennan SE, Chou R, Glanville J, Grimshaw JM, Hrobjartsson A, Lalu MM, Li T, Loder EW, Mayo-Wilson E, McDonald S, McGuinness LA, Stewart LA, Thomas J, Tricco AC, Welch VA, Whiting P, Moher D. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. PLOS Medicine. 2021;18(3):e1003583. doi:10.1371/journal.pmed.1003583 read 9 October 2026. doi.org/10.1371/journal.pmed.1003583
  7. Campbell M, McKenzie JE, Sowden A, Katikireddi SV, Brennan SE, Ellis S, Hartmann-Boyce J, Ryan R, Shepperd S, Thomas J, Welch V, Thomson H. Synthesis without meta-analysis (SWiM) in systematic reviews: reporting guideline. BMJ. 2020:l6890. doi:10.1136/bmj.l6890 read 9 October 2026. doi.org/10.1136/bmj.l6890
  8. Debray TPA, Damen JAAG, Snell KIE, Ensor J, Hooft L, Reitsma JB, Riley RD, Moons KGM. A framework for meta-analysis of prediction model studies with binary and time-to-event outcomes. Statistical Methods in Medical Research. 2019;28:2768-2786. read 9 October 2026. arxiv.org
  9. Higgins JPT, Lopez-Lopez JA, Becker BJ, Davies SR, Dawson S, Grimshaw JM, McGuinness LA, Moore THM, Rehfuess EA, Thomas J, Caldwell DM. Synthesising quantitative evidence in systematic reviews of complex health interventions. BMJ Global Health. 2019;4(Suppl 1):e000858. doi:10.1136/bmjgh-2018-000858 read 9 October 2026. doi.org/10.1136/bmjgh-2018-000858
  10. Langan D, Higgins JPT, Jackson D, Bowden J, Veroniki AA, Kontopantelis E, Viechtbauer W, Simmonds M. A comparison of heterogeneity variance estimators in simulated random-effects meta-analyses. Research Synthesis Methods. 2018;10(1):83-98. doi:10.1002/jrsm.1316 read 9 October 2026. doi.org/10.1002/jrsm.1316
  11. Lopez-Lopez JA, Page MJ, Lipsey MW, Higgins JPT. Dealing with effect size multiplicity in systematic reviews and meta-analyses. Research Synthesis Methods. 2018;9(3):336-351. doi:10.1002/jrsm.1310 read 9 October 2026. doi.org/10.1002/jrsm.1310
  12. Page MJ, Altman DG, McKenzie JE, Shamseer L, Ahmadzai N, Wolfe D, Yazdi F, Catala-Lopez F, Tricco AC, Moher D. Flaws in the application and interpretation of statistical analyses in systematic reviews of therapeutic interventions were common: a cross-sectional analysis. Journal of Clinical Epidemiology. 2018;95:7-18. doi:10.1016/j.jclinepi.2017.11.022 read 9 October 2026. doi.org/10.1016/j.jclinepi.2017.11.022
  13. Murad MH, Wang Z. Guidelines for reporting meta-epidemiological methodology research. Evidence-Based Medicine. 2017;22(4):139-142. read 9 October 2026. equator-network.org
  14. Page MJ, Higgins JPT, Clayton G, Sterne JAC, Hrobjartsson A, Savovic J. Empirical evidence of study design biases in randomized trials: systematic review of meta-epidemiological studies. PLoS ONE. 2016;11:e0159267. doi:10.1371/journal.pone.0159267 read 9 October 2026. doi.org/10.1371/journal.pone.0159267
  15. Rover C, Knapp G, Friede T. Hartung-Knapp-Sidik-Jonkman approach and its modification for random-effects meta-analysis with few studies. BMC Medical Research Methodology. 2015;15:99. doi:10.1186/s12874-015-0091-1 read 9 October 2026. doi.org/10.1186/s12874-015-0091-1
  16. Veroniki AA, Jackson D, Viechtbauer W, Bender R, Bowden J, Knapp G, Kuss O, Higgins JPT, Langan D, Salanti G. Methods to estimate the between-study variance and its uncertainty in meta-analysis. Research Synthesis Methods. 2015;7(1):55-79. doi:10.1002/jrsm.1164 read 9 October 2026. doi.org/10.1002/jrsm.1164
  17. Tipton E, Pustejovsky JE. Small-sample adjustments for tests of moderators and model fit using robust variance estimation in meta-regression. Journal of Educational and Behavioral Statistics. 2015;40:604-634. read 9 October 2026. arxiv.org
  18. Mavridis D, Salanti G. A practical introduction to multivariate meta-analysis. Statistical Methods in Medical Research. 2012;22(2):133-158. doi:10.1177/0962280211432219 read 9 October 2026. doi.org/10.1177/0962280211432219
  19. Savovic J, Jones HE, Altman DG, Harris RJ, Juni P, Pildal J, Als-Nielsen B, Balk EM, Gluud C, Gluud LL, Ioannidis JPA, Schulz KF, Beynon R, Welton NJ, Wood L, Moher D, Deeks JJ, Sterne JAC. Influence of reported study design characteristics on intervention effect estimates from randomized, controlled trials. Annals of Internal Medicine. 2012;157(6):429-438. doi:10.7326/0003-4819-157-6-201209180-00537 read 9 October 2026. doi.org/10.7326/0003-4819-157-6-201209180-00537
  20. Nakagawa S, Santos ESA. Methodological issues and advances in biological meta-analysis. Evolutionary Ecology. 2012;26:1253-1274. read 9 October 2026. arxiv.org
  21. Riley RD, Higgins JPT, Deeks JJ. Interpretation of random effects meta-analyses. BMJ. 2011;342:d549. doi:10.1136/bmj.d549 read 9 October 2026. doi.org/10.1136/bmj.d549
  22. Sterne JAC, Sutton AJ, Ioannidis JPA, Terrin N, Jones DR, Lau J, Carpenter J, Rucker G, Harbord RM, Schmid CH, Tetzlaff J, Deeks JJ, Peters J, Macaskill P, Schwarzer G, Duval S, Altman DG, Moher D, Higgins JPT. Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials. BMJ. 2011;343:d4002. doi:10.1136/bmj.d4002 read 9 October 2026. doi.org/10.1136/bmj.d4002
  23. Viechtbauer W. Conducting meta-analyses in R with the metafor package. Journal of Statistical Software. 2010;36(3):1-48. doi:10.18637/jss.v036.i03 read 9 October 2026. doi.org/10.18637/jss.v036.i03
  24. Viechtbauer W, Cheung MWL. Outlier and influence diagnostics for meta-analysis. Research Synthesis Methods. 2010;1(2):112-125. doi:10.1002/jrsm.11 read 9 October 2026. doi.org/10.1002/jrsm.11
  25. Hedges LV, Tipton E, Johnson MC. Robust variance estimation in meta-regression with dependent effect size estimates. Research Synthesis Methods. 2010;1(1):39-65. doi:10.1002/jrsm.5 read 9 October 2026. doi.org/10.1002/jrsm.5
  26. Simera I, Moher D, Hirst A, Hoey J, Schulz KF, Altman DG. Transparent and accurate reporting increases reliability, utility, and impact of your research: reporting guidelines and the EQUATOR Network. BMC Medicine. 2010;8:24. read 9 October 2026. trialsjournal.biomedcentral.com
  27. Bax L, Yu LM, Ikeda N, Tsuruta N, Moons KGM. A systematic comparison of software dedicated to meta-analysis of causal studies. BMC Medical Research Methodology. 2007;7:40. doi:10.1186/1471-2288-7-40 read 9 October 2026. doi.org/10.1186/1471-2288-7-40
  28. Bax L, Yu LM, Ikeda N, Tsuruta H, Moons KG. Development and validation of MIX: comprehensive free software for meta-analysis of causal research data. BMC Medical Research Methodology. 2006;6(1):50. doi:10.1186/1471-2288-6-50 read 9 October 2026. doi.org/10.1186/1471-2288-6-50
  29. Viechtbauer W. Bias and efficiency of meta-analytic variance estimators in the random-effects model. Journal of Educational and Behavioral Statistics. 2005;30(3):261-293. doi:10.3102/10769986030003261 read 9 October 2026. doi.org/10.3102/10769986030003261
  30. Leandro G. Meta-analysis in medical research. Blackwell Publishing / BMJ Books (book). 2005. read 9 October 2026. link.springer.com
  31. van Houwelingen HC, Arends LR, Stijnen T. Advanced methods in meta-analysis: multivariate approach and meta-regression. Statistics in Medicine. 2002;21:589-624. read 9 October 2026. arxiv.org
  32. Thompson SG, Higgins JPT. How should meta-regression analyses be undertaken and interpreted? Statistics in Medicine. 2002;21(11):1559-1573. doi:10.1002/sim.1187 read 9 October 2026. doi.org/10.1002/sim.1187
  33. Higgins JPT, Thompson SG. Quantifying heterogeneity in a meta-analysis. Statistics in Medicine. 2002;21(11):1539-1558. doi:10.1002/sim.1186 read 9 October 2026. doi.org/10.1002/sim.1186
  34. Hartung J, Knapp G. On tests of the overall treatment effect in meta-analysis with normally distributed responses. Statistics in Medicine. 2001;20:1771-1782. read 9 October 2026. arxiv.org
  35. Glasziou P, Irwig L, Bain C, Colditz G. Systematic reviews in health care: a practical guide. Cambridge University Press (book). 2001. read 9 October 2026. link.springer.com
  36. Egger M, Davey Smith G, Altman DG. Systematic reviews in health care: meta-analysis in context. BMJ Books (book). 2001. read 9 October 2026. link.springer.com
  37. Sutton AJ, Abrams KR, Jones DR, Sheldon TA, Song F. Methods for meta-analysis in medical research. Wiley (book). 2000. read 9 October 2026. link.springer.com
  38. Kalaian HA, Raudenbush SW. A multivariate mixed linear model for meta-analysis. Psychological Methods. 1996;1:227-235. read 9 October 2026. arxiv.org
  39. DerSimonian R, Laird N. Meta-analysis in clinical trials. Controlled Clinical Trials. 1986;7(3):177-188. doi:10.1016/0197-2456(86)90046-2 read 9 October 2026. doi.org/10.1016/0197-2456(86)90046-2
  40. Fleiss JL. Measuring nominal scale agreement among many raters. Psychological Bulletin. 1971;76:378-382. read 9 October 2026. trialsjournal.biomedcentral.com
  41. Cochran WG. The combination of estimates from different experiments. Biometrics. 1954;10(1):101-129. doi:10.2307/3001666 read 9 October 2026. doi.org/10.2307/3001666
  42. Veroniki AA, Jackson D, Bender R, Kuss O, Langan D, Higgins JPT, Knapp G, Salanti G. Methods to calculate uncertainty in the estimated overall effect size from a random-effects meta-analysis. doi:10.1002/jrsm.1319 read 9 October 2026. doi.org/10.1002/jrsm.1319

Every source above is a peer-reviewed article, a publisher’s record of one, or a standard issued by the body that owns it. Each carries the date it was last read. Where a standard moves, this article is revised rather than left up with a note.

How to cite this article

Pick your referencing style and copy it. Your university or target journal may vary the punctuation — check its guide if it has one.

Pubrica Academy. (2026, October 9). Conducting a Meta-Analysis End to End: The Analytic Decisions and Their Order. Pubrica. https://pubrica.com/academy/research/a-practical-guide-to-do-primary-research-on-meta-analysis-methodology/

This article is written to be learned from. Cite it if you are referring to what it says; if you are using the evidence behind it, cite the sources listed above instead.

Statistical analysis plan stub for a meta-analysis

This stub should be inserted into the protocol and every line completed before extraction is finished. The completed version closely approximates the required methods paragraph, and the lines map onto PRISMA 2020 item 13d.

META-ANALYSIS STATISTICAL ANALYSIS PLAN
Review: ______________________    Version / date: ______________

1. SYNTHESIS GROUPS
   Interventions treated as equivalent: ______________________
   Comparators treated as equivalent: ______________________
   Outcome domains, and the scales accepted for each: ______________
   Measurement timings accepted, or how timing is handled: __________

2. EFFECT MEASURE
   Primary outcome measure (e.g. RR, OR, MD, SMD): ________________
   Reason this measure is expected to be consistent across studies:
   ______________________________________________________________

3. MULTIPLE EFFECT SIZES PER STUDY   (choose ONE, before extraction)
   [ ] Select one per study. Selection rule: ______________________
   [ ] Average within study. Method: ______________________
   [ ] Model dependence (robust variance estimation, small-sample
       correction: ______________________)

4. MODEL
   Fixed / random: ____________
   Substantive reason (NOT the heterogeneity statistic):
   ______________________________________________________________
   Weighting method: ______________________

5. VARIANCE ESTIMATOR AND INTERVAL METHOD
   Between-study variance estimator: ______________________
   Confidence interval method: ______________________
   Planned comparison method(s) for sensitivity: ______________________
   (HKSJ recommended as standard at 5+ studies; compare 2-3 methods
    when fewer than 10 studies are pooled)
   Bayesian analysis? If yes, prior distributions: ______________________

6. HETEROGENEITY REPORTING
   [ ] Between-study variance estimate
   [ ] I-squared
   [ ] Prediction interval for the summary effect

7. MODERATORS   (pre-specified; keep shorter than the study count)
   1. ______________________  2. ______________________
   3. ______________________
   [ ] Stated in the report as observational between-study comparisons

8. SMALL-STUDY EFFECTS AND DIAGNOSTICS
   Expected number of studies: ____
   [ ] If fewer than 10: no funnel plot asymmetry test will be run
   [ ] Leave-one-out estimates reported for all studies, either way
   [ ] Influence diagnostics / standardised residuals reported

9. IF POOLING IS NOT APPROPRIATE
   Conditions triggering a synthesis without meta-analysis:
   ______________________________________________________________
   [ ] SWiM reporting items will be followed

10. COMPUTATION
   Software and package, with version: ______________________
   [ ] Analysis code and extracted data to be deposited

Pubrica AcademyWritten and maintained by Pubrica’s subject matter experts. Articles are revised when the underlying standards change, and each article states the date on which it was last reviewed.

How Pubrica can help

Pubrica’s biostatistics team writes the statistical analysis plan before extraction closes, so that the model, estimator and interval method are on record rather than chosen once the forest plot exists, and runs the analysis in a reproducible script that can be deposited with the paper. Where the included studies will not support pooling, the team constructs a structured, SWiM-compliant synthesis instead and states in the manuscript why that approach was appropriate.

Find something else

Search the Pubrica Academy

Access is free and requires no account. Enter a topic, or open the full library to filter by category, sub-topic or resource type.

Pubrica can review the protocol and inclusion list and draft the analysis plan before the data can influence it.

A scoping assessment is free and takes two business days. It provides a written assessment of what the work requires and which specialist would undertake it.

Get a free quote

Regulations

Check your project against this rule

  1. 1Your work
  2. 2Your details
  3. 3Send
What do you need?

Material you send is held under signed confidentiality, with access limited to the assigned team and encrypted transfer. It is not used to train anything and is not passed to third-party tools. Ask for the agreement before you send anything you would rather cover first.