Systematic Reviews and Meta-Analysis in the Medical Sciences: Best Practice Methods for Research Synthesis

Best practice in systematic review and meta-analysis methodology is not a single technique but a sequence, where each stage's output becomes the next stage's input: a well-built question defines the eligibility criteria, the eligibility criteria define the search, the search defines what gets risk-of-bias assessed, and the risk-of-bias assessment feeds directly into the certainty rating attached to the final pooled estimate. Treating any one of these stages as optional, or disconnected from the others, is the most common way an otherwise competently executed review still produces a weaker synthesis than its underlying evidence could support.

Formulating the review question: PICO and eligibility criteria

The 1995 framework of Richardson and colleagues for structuring a clinical question into Population, Intervention, Comparison and Outcome (PICO) remains the standard starting point for the eligibility criteria of a systematic review, because each element maps onto a specific, verifiable inclusion or exclusion rule1. The search strategy, data extraction form and planned statistical analysis of a review all follow from the precision with which the PICO elements were defined at the outset, which is why best-practice guidance regards a vague or shifting PICO as a problem that no degree of downstream search rigour can fully remedy2. A common indication of a weakly constructed question is a review team that cannot agree, partway through screening, on whether a specific borderline study should be included; such disagreement usually originates not in that study but in a Population, Intervention, Comparison or Outcome element that was never defined with sufficient precision. This dependency explains a pattern that editors and peer reviewers observe repeatedly: a review that is weak at the question-formulation stage rarely recovers, however carefully the subsequent stages are executed, because every later decision is made against a question that was not specified clearly enough to make those decisions unambiguous. For questions about diagnostic accuracy, prognosis or qualitative evidence, variants of the same framework (such as PICOTS or SPIDER) exist because a single rigid template does not suit every review type equally well, and the selection of the appropriate variant is itself a best-practice decision made before any searching begins2. The development by Cooke, Smith and Booth of the SPIDER framework for qualitative evidence, which among other changes replaces Population with Sample and Outcome with Evaluation and Research type, illustrates why forcing every review question into the standard PICO structure regardless of fit is a methodological failure: a framework designed for intervention trials asks inappropriate questions of a qualitative evidence base, and answering them nonetheless produces eligibility criteria that do not correspond to what the review seeks to establish2.

Protocol registration as an essential requirement

The PRISMA-P statement, published in 2015, standardised the content of a systematic review protocol, including the PICO-structured question, planned eligibility criteria, search strategy and planned analysis, so that a protocol could be registered publicly, on PROSPERO or an equivalent register, before data extraction begins3. Registration is a best practice because it creates a fixed, dated, public record of the planned methods of a review, which allows an editor, a peer reviewer or a reader to compare the final published methods with those originally planned and to identify any undisclosed deviation4. Once the PRISMA-P-structured protocol has been drafted, registration typically requires no more than a few hours, which makes the ratio of cost to benefit of this step unusually favourable compared with most other items in the sequence; this merits emphasis, given how often registration is the step omitted under time pressure. This step underlies much of the reason that systematic reviews are considered a higher tier of evidence than an unregistered narrative review: best-practice guidance treats the pre-commitment itself, not only the comprehensiveness of the search, as the protective element.

Matching the risk-of-bias tool to the study design

Best practice requires a risk-of-bias assessment tool appropriate to the design of the studies being synthesised: the Cochrane RoB 2 tool for randomised trials, revised in 2019 to assess bias across five specific domains including the randomisation process and selection of the reported result, and the ROBINS-I tool for non-randomised studies of interventions, which assesses bias from confounding and from selection into the study, concerns that randomised designs are specifically constructed to avoid56. Use of a tool designed for a different study type, for instance applying RoB 2 to observational studies, systematically fails to capture the biases to which that design is most prone, which is why methodological guidance treats tool selection as a best-practice decision in its own right rather than a detail to be settled after data extraction has begun6. A review that includes both randomised and non-randomised studies on the same research question should apply each tool to the corresponding subset and report the findings and certainty ratings of the two groups at least partly separately, since combining them into a single pooled estimate without acknowledging their different bias profiles obscures the information that a design-matched risk-of-bias assessment is intended to reveal.

GRADE certainty rating as the final stage of the methodological sequence

The GRADE approach is the final link in the methodological sequence because it formally combines the outputs of every earlier stage into a single certainty rating for each outcome: risk of bias from the selected assessment tool, inconsistency from the analysis of statistical heterogeneity, indirectness from the correspondence between the included studies and the original PICO question, imprecision from the width of the confidence interval, and publication bias from the funnel-plot or small-study-effects assessment7. A team that reaches this final stage having omitted or rushed an earlier one (an unregistered protocol, a search conducted against a loosely defined PICO, or a risk-of-bias tool selected without regard to study design) is applying a rigorous final step to inputs that were not produced with the same rigour, which tends to yield a certainty rating that appears more authoritative than the underlying process warrants. AMSTAR 2, a widely used tool for the critical appraisal of completed systematic reviews, scores reviews partly on whether this full sequence, from PICO through registration, appropriate selection of the risk-of-bias tool and a GRADE-style certainty rating, was followed and reported, which indicates that current best-practice guidance treats these stages as one connected sequence rather than a checklist of independent options8. The scoring structure of AMSTAR 2 assigns particular weight to a small number of “critical” items, including protocol registration and the comprehensiveness of the search, reflecting the judgement of the appraisal tool about which links in the sequence matter most to the confidence a reader should place in the conclusions of a review. A review that scores poorly on one of these critical items cannot be fully redeemed by strong performance elsewhere, which demonstrates that the earliest links carry disproportionate weight in how the finished review is judged. Applying AMSTAR 2 to the draft of a review before submission, rather than awaiting its application by peer reviewers, is an efficient use of time in the final stages of a project, since it identifies gaps that break the sequence (an unregistered protocol, a mismatched risk-of-bias tool, an unexplained departure from the planned analysis), each of which is inexpensive to remedy before submission and considerably more costly to remedy during revision. Teams that incorporate this self-appraisal step into their internal timeline, rather than regarding AMSTAR 2 as a tool applied only by journal reviewers, are likely to receive fewer major-revision requests related to such gaps, because the gaps are identified and closed while the authors still control the timeline of the manuscript rather than after an editor has set a revision deadline.

Key points

Practical application

Frequently asked questions

@misc{/la_systematic_review_meta_analysis_best_practice_2026,
  author       = {{Pubrica Academy}},
  title        = {Systematic Reviews and Meta-Analysis in the Medical Sciences: Best Practice Methods for Research Synthesis},
  howpublished = {Pubrica},
  year         = {2026},
  month        = {oct},
  url          = {https://pubrica.com/academy/systematic-review/systematic-reviews-and-meta-analyses-in-the-medical-sciences-best-practice-methods-for-research-synthesis/},
  note         = {Accessed: <span data-cite-acc="iso">2026-10-15</span>}
}

[ ] PICO (or appropriate variant) finalized and used to derive eligibility criteria
[ ] Protocol drafted per PRISMA-P and registered on PROSPERO before extraction began
[ ] Risk-of-bias tool selected to match study design (RoB 2 for RCTs, ROBINS-I for non-randomized studies)
[ ] Heterogeneity, indirectness, imprecision, and publication bias each formally assessed
[ ] GRADE (or equivalent) certainty rating applied per outcome, combining all the above
[ ] AMSTAR 2 run against the completed review before submission