Home/Insights/Evidence synthesis/Quantitative synthesis: an update
Quantitative synthesis: an update
The AHRQ methods chapter on when and how to combine results — including the circumstances in which the right answer is not to.
The decision to pool
The chapter’s central position is that pooling is a judgement about clinical and methodological comparability first, and a statistical procedure second. Studies differing substantially in population, intervention, comparator or outcome definition should not be combined into a single estimate however statistically tidy the result appears.
Where pooling is inappropriate, structured narrative synthesis is the correct output, not a failure. SWiM — Synthesis Without Meta-analysis — is the reporting guideline for it, and exists precisely because narrative synthesis was previously reported without structure.
Model choice and small numbers
With few studies, the between-study variance is estimated poorly and random-effects confidence intervals can be too narrow. Methods such as the Hartung–Knapp adjustment address this and are recommended where the number of studies is small.
Rare events raise separate problems: standard methods perform badly with zero cells, and continuity corrections introduce bias. Alternative approaches such as the Peto odds ratio or exact methods are preferred depending on the situation.
Beyond pairwise pooling
Network meta-analysis allows comparison of interventions never tested head-to-head, by combining direct and indirect evidence. It requires transitivity — that studies are similar enough for indirect comparison to be valid — and consistency between direct and indirect estimates, both of which must be assessed and reported rather than assumed.
Individual participant data meta-analysis permits consistent handling of covariates and genuine subgroup analysis, and is the stronger design where data can be obtained.
Reporting the synthesis
State the effect measure and why. State the model and why. Report heterogeneity with tau-squared and a prediction interval. Pre-specify subgroup analyses and report all of them, including those that showed nothing.
In practice
Fix, flag or query.
The distinction that keeps an edit defensible: what an editor may change silently, what must be shown to the author, and what only the author can answer.
- Where pooling is inappropriate, report to SWiM rather than pooling anyway.
- With few studies, consider the Hartung–Knapp adjustment.
- For network meta-analysis, assess transitivity and consistency explicitly.
Sources
- AHRQ EPC Methods Guide, quantitative synthesis chapter
- SWiM reporting guideline, synthesis without meta-analysis
- Cochrane Handbook, on network meta-analysis
Sources are given so each claim can be checked at its origin. Where a tool’s current maintenance status could not be confirmed, the page says to check it rather than asserting it.
Related