Contents
Academy · Research Data, Analytics & AI · Combining RCTs in Meta-Analysis
Combining Randomized Controlled Trials in a Meta-Analysis: Sequence, Heterogeneity and Model Selection
I², the statistic most meta-analyses use to describe inconsistency between trials, was introduced because the older chi-squared test for heterogeneity loses power when few studies are available. The Cochrane Handbook now treats I² values above 50% as a signal to question whether pooling the trials into a single estimate is appropriate.
Combining several randomized controlled trials into a meta-analysis is a defined statistical process with specific steps: a systematic search identifies every eligible trial, explicit inclusion and exclusion criteria determine which trials qualify, each trial’s effect estimate and its variance are extracted in a comparable form, a weighting scheme combines them, and a heterogeneity assessment determines whether a single pooled value is an appropriate summary of the trial findings. The order of these steps matters, and the statistical choices within them, particularly how heterogeneity between trials is measured and handled, directly determine whether the resulting pooled estimate is trustworthy or misleading12.
Combining RCTs at a glance
- What happens
- Search, screen, extract each trial’s effect and its precision, weight and pool, then judge heterogeneity before reporting12
- Heterogeneity measures
- Cochran’s Q (a test), I² (a percentage) and τ² (the between-trial variance on the outcome scale)36
- Two models
- Fixed-effect and random-effects models rest on fundamentally different assumptions about the data12
- Common flag
- I² above 50% is often called substantial, but the Handbook’s rough bands overlap and depend on context26
- Main caution
- I² rises towards 100% as trials get larger, even when the real spread of effects does not change6
- Reading time
- About 10 minutes, with 3 figures and 4 tables
Learning objectives
- The defined sequence of steps followed by a systematic review and meta-analysis, from literature search to pooled estimate
- The rationale for introducing the I² statistic and what it measures that the chi-squared test does not capture well
- The recommendations of the Cochrane Handbook once substantial heterogeneity between trials has been identified
Common misconceptions
- “A meta-analysis simply averages the results of every trial that measured the same outcome.” The Cochrane Handbook’s guidance treats a pooled estimate as appropriate only after heterogeneity between trials has been assessed and judged acceptable. Where I² is substantial (commonly flagged above 50%), the Handbook recommends investigating the source of the inconsistency, using a random-effects model or, in some cases, not pooling the trials into a single estimate, rather than defaulting to a simple average.
Sequence of steps from literature search to pooled estimate
A systematic review informing a meta-analysis begins with a comprehensive, reproducible literature search across multiple databases, guided by a pre-specified protocol that states the research question, the population, intervention, comparator and outcome (PICO) of interest, and explicit inclusion and exclusion criteria; this is precisely the kind of advance specification that the PRISMA 2020 reporting guideline requires authors to document and disclose1. Once eligible RCTs are identified, each trial’s result is extracted in a standardized form, typically an effect size (a risk ratio, odds ratio, mean difference or hazard ratio) together with a measure of its precision (a standard error or confidence interval), because the statistical pooling step requires both the magnitude of each trial’s effect and the uncertainty surrounding it.
The pooling step assigns each trial a weight, generally larger for trials with more participants and more precise estimates, and combines the weighted effects into a single pooled estimate with its own confidence interval, a weighting procedure first formalized for clinical trial meta-analysis in the foundational 1986 paper by DerSimonian and Laird4, and typically displayed as a forest plot, the graphical format that the Cochrane Handbook recommends for presenting this kind of pooled result5. Before that pooled value is reported as a meaningful summary, however, the trials being combined must be assessed for heterogeneity, that is, whether they estimate sufficiently similar underlying effects for an average to be meaningful rather than a combination of incommensurable results. This heterogeneity assessment is not an optional addition at the end of the process; the Cochrane Handbook treats it as an integral part of deciding how, or whether, to report a pooled estimate2.
What to extract from each trial before pooling
| Outcome type | Typical effect measure | Precision to extract | Pooled on |
|---|---|---|---|
| Binary (event or no event) | Risk ratio or odds ratio | 95% confidence interval or standard error, or the 2×2 counts | Log scale |
| Continuous (score, measurement) | Mean difference, or standardised mean difference when scales differ | Standard deviation and group sizes, or the standard error | Original or standardised scale |
| Time-to-event (survival, relapse) | Hazard ratio | 95% confidence interval or standard error of the log hazard ratio | Log scale |
General summary of common practice, written for this article. The pooling step needs both the size of each trial’s effect and its uncertainty.
Rationale for the introduction of the I² statistic
Before I² became standard, meta-analysts relied primarily on a chi-squared test (often called Cochran’s Q) to determine whether observed differences between trials exceeded those expected by chance alone. Higgins and Thompson’s 2002 paper in Statistics in Medicine introduced I² specifically to address a known weakness of the chi-squared approach: with a small number of trials, the test has low statistical power, so that genuine heterogeneity may go undetected simply because there are too few trials for the test to identify it reliably3. I² instead expresses the percentage of total variation across trials that is attributable to genuine heterogeneity rather than chance, a formulation that does not depend on the number of trials in the way that the power of the chi-squared test does, and that yields a readily interpretable percentage rather than a bare significance test result3.
This distinction is of practical importance in the situation that a meta-analysis of RCTs most often faces: a modest number of trials, sometimes only three or four, addressing the same clinical question. In that setting, a non-significant chi-squared test for heterogeneity should not be interpreted as proof that the trials are consistent; it may simply reflect the low power of the test with few studies. I² provides a more informative indication of how much of the variation between trial results reflects genuine inconsistency rather than sampling error, even when the number of trials is small3. Subsequent methodological work has nonetheless cautioned against interpreting the I² percentage in isolation, since undue reliance on the point estimate alone, without its confidence interval, can mislead a reviewer about the seriousness of the inconsistency6.
Measures of heterogeneity and how they behave
| Measure | Scale and range | Increases with | What it tells you |
|---|---|---|---|
| Cochran’s Q | Absolute, 0 to ∞ | More trials and larger trials | A test of whether differences exceed chance; low power with few trials |
| I² | Percentage, 0% to 100% | Larger (more precise) trials only | The share of total variability attributed to between-trial heterogeneity rather than sampling error |
| τ² (and τ) | Outcome scale, 0 to ∞ | Neither | The between-trial variance itself; τ is in the same units as the effect |
| H | Absolute, from 1 | Larger trials only | Observed Q relative to its expected value; 1 means perfect homogeneity |
| R | Absolute, from 1 | Larger trials only | How much wider the random-effects interval is than the fixed-effect interval |
Adapted from Table 1 of Rücker et al. (2008).6
Cochrane Reviews began reporting I² to help readers judge how consistent the results of the included studies are11. The table shows why it should not be read alone: unlike τ², it depends on how precise the trials are.
A rough guide to I², as quoted from the Cochrane Handbook (version 5.0.1)
| I² range | Rough interpretation |
|---|---|
| 0% to 40% | Might not be important |
| 30% to 60% | May represent moderate heterogeneity |
| 50% to 90% | May represent substantial heterogeneity |
| 75% to 100% | Considerable heterogeneity |
Ranges as reproduced in Table 3 of Rücker et al. (2008). The ranges overlap because, in the Handbook’s words as quoted there, “the importance of inconsistency depends on several factors”.62
Management of substantial heterogeneity between trials
The Cochrane Handbook provides explicit guidance once I² indicates substantial heterogeneity, commonly taken as a value above 50%, although the Handbook states clearly that this threshold should be interpreted alongside the confidence interval around I² and the clinical context rather than applied as a rigid cut-off2. The first recommended response is to investigate the source of the inconsistency, examining whether differences in trial population, intervention dose or duration, outcome definition or risk of bias explain the disagreement between trials, often through predefined subgroup analyses or meta-regression. The second is a modeling choice: changing from a fixed-effect model, which assumes that every trial estimates exactly the same true effect, to a random-effects model, which allows the true effect to vary across trials and incorporates that additional between-trial variation into a wider confidence interval around the pooled estimate2. The third, and most conservative, option is to conclude that the trials are too inconsistent to combine into a single meaningful value, in which case the Handbook recommends presenting the individual trial results narratively or in a forest plot without a pooled summary estimate, rather than imposing a figure that would misrepresent genuinely conflicting evidence2. The choice among these three responses is a matter of judgment that depends on clinical plausibility as much as on the statistic itself, which is why the Handbook presents I² as a prompt for investigation rather than a mechanical decision rule.
Fixed-effect and random-effects models compared
| Feature | Fixed-effect model | Random-effects model |
|---|---|---|
| Assumption | Every trial estimates one common true effect | True effects differ between trials and follow a distribution12 |
| Weight per trial | 1 / within-trial variance | 1 / (within-trial variance + τ²)6 |
| Influence of a very large trial | Can dominate the pooled estimate | Reduced, because τ² is added to every trial’s variance |
| Confidence interval | Narrower; reflects sampling error only | Wider when τ² > 0; reflects between-trial variation too2 |
| Worked example: 70 thrombolysis trials | OR 0.747 (95% CI 0.705 to 0.792), Mantel-Haenszel | OR 0.732 (95% CI 0.664 to 0.808), DerSimonian-Laird64 |
| Answers the question | What is the common effect in these trials? | What is the average effect across a range of settings like these? |
Weights as defined by Rücker et al. (2008); the worked example is the thrombolytic therapy meta-analysis they reanalysed. In that example the two models gave similar answers because heterogeneity was low.
The choice of model is not a test result
Rücker and colleagues quote the Cochrane Handbook directly: “The choice between a fixed-effect and a random-effects meta-analysis should never be made on the basis of a statistical test of heterogeneity.” They add that the yardstick for pooling should be the clinical relevance of any heterogeneity, and that τ², which is measured on the outcome scale, is the better measure for that purpose than I².6
The weighting step that precedes any heterogeneity assessment also warrants closer attention than it usually receives, because the choice between a fixed-effect and a random-effects model changes not only the width of the pooled confidence interval but also the weight each trial receives in the final estimate. Under a fixed-effect model, weighting is based solely on each trial’s statistical precision, so a single very large trial can dominate the pooled estimate almost entirely, reducing several smaller trials to a minor adjustment of the large trial’s result. Under a random-effects model, by contrast, an additional between-study variance term is added to every trial’s weight, which proportionally reduces the influence of the largest trials and increases the relative influence of smaller ones, because the model explicitly allows for the possibility that trials estimate somewhat different true effects rather than one shared true effect. Consequently, changing from a fixed-effect to a random-effects model in response to detected heterogeneity does not merely widen the confidence interval conservatively; it can also shift the point estimate itself, sometimes noticeably, if the smaller and larger trials in the pooled set disagree in direction or magnitude. An author reporting a meta-analysis should state explicitly which model was used and why, rather than treating the choice as a minor technical detail, since the Cochrane Handbook links this choice directly to the heterogeneity findings that determine how trustworthy the final pooled estimate is.
A related but distinct concept that the heterogeneity statistics described above do not capture should also be noted: clinical heterogeneity, meaning substantive differences between trials in patient population, dosing or outcome definition, as distinct from statistical heterogeneity, the numerical inconsistency in results that I² measures. Two trials can show low statistical heterogeneity (a small I²) purely by chance, while still differing in clinically important ways that make pooling them questionable on substantive rather than statistical grounds. Conversely, two trials can show high statistical heterogeneity while testing a very similar clinical question, simply because one had a smaller sample and therefore a less precise individual estimate. The Cochrane Handbook advises that clinical and methodological similarity across trials be assessed as a judgment made before pooling is attempted, independently of the I² value subsequently reported, since a statistically convenient I² value is not in itself sufficient justification for combining trials that a subject-matter expert would recognize as addressing meaningfully different clinical questions.
Sensitivity analysis is the practical tool that meta-analysts use to test how far these judgments (model choice, inclusion of a borderline trial, handling of a trial with missing outcome data) change the final result. Running the pooled analysis once with every eligible trial included and again after excluding trials judged to be at higher risk of bias, assessed using a tool such as RoB 28, or after changing from a fixed-effect to a random-effects model, shows directly whether the headline pooled estimate is a robust feature of the underlying evidence or an artifact of the inclusion of one particular trial or one particular modeling choice. A pooled estimate that shifts only modestly across these sensitivity analyses warrants considerably more confidence than one that changes substantially depending on which trials or which model were used, and reporting the sensitivity analysis explicitly, rather than presenting only the single final pooled figure, is part of what a transparent, PRISMA-compliant meta-analysis is expected to show. A small-study or publication-bias assessment, such as Egger’s regression-based funnel-plot test, is a further sensitivity step that should be conducted routinely, since an asymmetric funnel plot may indicate that smaller trials with less favorable results were not published or were not identified by the search7, and the overall certainty of the pooled estimate that remains after these checks should ultimately be stated using a formal framework such as GRADE9, applied to the kind of individual-trial risk-of-bias assessment that the methodology chapter of the Cochrane Handbook sets out in detail10.
Weak heterogeneity reporting
“Heterogeneity was low (I² = 19%), so a fixed-effect model was used.”
The model is chosen from a test result, I² is given without its interval, and τ² is missing.
Stronger heterogeneity reporting
“I² = 18.6% (95% CI 0% to 40.1%); τ² = 0.018; Q = 85, p = 0.095. A random-effects model was prespecified; the fixed-effect estimate is shown as a sensitivity analysis.”
Illustrative wording, using the figures from Rücker et al.’s reanalysis. The point estimate, its uncertainty, τ² and the reason for the model are all visible.
Before you report the pooled estimate
A checklist version of the sensitivity paragraph above.789
- Rerun the analysis without trials at high risk of bias (for example, judged with RoB 2) and compare the estimates.
- Rerun it with the other model (fixed-effect or random-effects) and report both if they differ.
- Check whether one borderline or very large trial drives the result.
- State how missing outcome data were handled and test the effect of that choice.
- Examine small-study effects, for example with a funnel plot and Egger’s test.
- Rate the certainty of the final estimate with GRADE and report every sensitivity analysis, not only the headline figure.
Key points
Key points
Combining randomized controlled trials (RCTs) in a meta-analysis follows a defined sequence of search, data extraction, weighting and assessment of heterogeneity. The I² statistic exists because older tests lose power when few trials are available. Substantial heterogeneity should prompt investigation, the use of a random-effects model or a decision not to pool, rather than a default average.
Practical application
Practical application
- The PICO question and the inclusion and exclusion criteria should be pre-registered before the search is conducted. This practice follows the documentation requirements of PRISMA 2020 and prevents criteria from being adjusted after it becomes apparent which trials would be included or excluded.
- I² should be reported alongside, rather than instead of, the chi-squared test for heterogeneity. I² was designed to remain informative when the number of trials is small, a situation in which the chi-squared test alone may be underpowered to detect true heterogeneity.
- Substantial heterogeneity should prompt investigation rather than a change of model alone. Predefined subgroup differences in population, dose or outcome definition should be examined before a choice is made between a random-effects model and not pooling the trials.
Frequently asked questions
No. I² measures genuine inconsistency between trials, which can reflect real clinical differences, such as different patient populations or doses, rather than a flaw in the meta-analysis itself; the Cochrane Handbook treats a high I² as a signal to investigate and adjust the analytic approach, not as evidence that the entire analysis is invalid.
A fixed-effect model assumes that every included trial estimates a single true effect, whereas a random-effects model allows the true effect to vary across trials and incorporates that additional variation into a wider confidence interval; the Cochrane Handbook recommends the latter once heterogeneity is substantial.
Yes; the Cochrane Handbook explicitly allows for presenting individual trial results narratively or in a forest plot without a single pooled estimate when trials are judged too inconsistent to combine meaningfully.
PRISMA 2020 requires the search strategy, eligibility criteria, and analytic methods to be documented transparently, which allows readers to judge whether the pooling process, including how heterogeneity was handled, was applied consistently rather than selectively.
References
- Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71. read 10 October 2026. View on The BMJ
- Deeks JJ, Higgins JPT, Altman DG (editors). Chapter 10: Analysing data and undertaking meta-analyses. In: Cochrane Handbook for Systematic Reviews of Interventions. read 10 October 2026. View on Cochrane Training
- Higgins JPT, Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat Med. 2002;21(11):1539-1558. read 10 October 2026. View on Statistics in Medicine
- DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials. 1986;7(3):177-188. read 10 October 2026. View paper (PDF)
- Cochrane Training. Graph recommendations (forest plots). Cochrane Handbook for Systematic Reviews of Interventions. read 10 October 2026. View on Cochrane Training
- Rücker G, Schwarzer G, Carpenter JR, Schumacher M. Undue reliance on I² in assessing heterogeneity may mislead. BMC Med Res Methodol. 2008;8:79. read 10 October 2026. View on BMC Medical Research Methodology
- Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ. 1997;315(7109):629-634. read 10 October 2026. View on The BMJ
- Sterne JAC, Savović J, Page MJ, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ. 2019;366:l4898. read 10 October 2026. View on The BMJ
- GRADE Working Group. GRADE Handbook: Introduction to GRADE Handbook. read 10 October 2026. View GRADE Handbook
- Cochrane Training. Chapter 8: Assessing risk of bias in a randomized trial. Cochrane Handbook for Systematic Reviews of Interventions (archived v5.1). read 10 October 2026. View on Cochrane Training
- Higgins JPT, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ. 2003;327(7414):557-560. read 13 November 2026. View on The BMJ
- Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. A basic introduction to fixed-effect and random-effects models for meta-analysis. Res Synth Methods. 2010;1(2):97-111. read 13 November 2026. View on Research Synthesis Methods
Every source above is a peer-reviewed article, a publisher’s record of one, or a standard issued by the body that owns it. Each carries the date it was last read. Where a standard moves, this article is revised rather than left up with a note.
How to cite this article
Pick your referencing style and copy it. Your university or target journal may vary the punctuation — check its guide if it has one.
Pubrica Academy. (2026, October 10). Combining Randomized Controlled Trials in a Meta-Analysis: Sequence, Heterogeneity and Model Selection. Pubrica. https://pubrica.com/academy/meta-analysis/how-randomized-controlled-trials-are-used-in-meta-analysis/
Pubrica Academy. “Combining Randomized Controlled Trials in a Meta-Analysis: Sequence, Heterogeneity and Model Selection.” Pubrica, 10 Oct. 2026, pubrica.com/academy/meta-analysis/how-randomized-controlled-trials-are-used-in-meta-analysis/.
Pubrica Academy (2026) Combining Randomized Controlled Trials in a Meta-Analysis: Sequence, Heterogeneity and Model Selection. Pubrica. Available at: https://pubrica.com/academy/meta-analysis/how-randomized-controlled-trials-are-used-in-meta-analysis/ (Accessed: 15 October 2026).
Pubrica Academy. “Combining Randomized Controlled Trials in a Meta-Analysis: Sequence, Heterogeneity and Model Selection.” Pubrica. October 10, 2026. https://pubrica.com/academy/meta-analysis/how-randomized-controlled-trials-are-used-in-meta-analysis/
Pubrica Academy, ‘Combining Randomized Controlled Trials in a Meta-Analysis: Sequence, Heterogeneity and Model Selection’ (Pubrica, 10 October 2026) <https://pubrica.com/academy/meta-analysis/how-randomized-controlled-trials-are-used-in-meta-analysis/> accessed 15 October 2026.
Pubrica Academy. Combining Randomized Controlled Trials in a Meta-Analysis: Sequence, Heterogeneity and Model Selection. [Internet]. Pubrica; 2026 Oct 10 [cited 2026 Oct 15]. Available from: https://pubrica.com/academy/meta-analysis/how-randomized-controlled-trials-are-used-in-meta-analysis/
Pubrica Academy, “Combining Randomized Controlled Trials in a Meta-Analysis: Sequence, Heterogeneity and Model Selection,” Pubrica, Oct. 10, 2026. [Online]. Available: https://pubrica.com/academy/meta-analysis/how-randomized-controlled-trials-are-used-in-meta-analysis/
@misc{/la_how_rcts_are_combined_in_meta_analysis_2026,
author = {{Pubrica Academy}},
title = {Combining Randomized Controlled Trials in a Meta-Analysis: Sequence, Heterogeneity and Model Selection},
howpublished = {Pubrica},
year = {2026},
month = {oct},
url = {https://pubrica.com/academy/meta-analysis/how-randomized-controlled-trials-are-used-in-meta-analysis/},
note = {Accessed: <span data-cite-acc="iso">2026-10-15</span>}
}This article is written to be learned from. Cite it if you are referring to what it says; if you are using the evidence behind it, cite the sources listed above instead.
RCT Meta-Analysis Pooling Checklist
This checklist should be completed before a pooled estimate from multiple RCTs is reported.
[ ] Pre-specified PICO question and inclusion/exclusion criteria documented (PRISMA 2020) [ ] Effect size and precision extracted in a standardized, comparable form from every trial [ ] Both chi-squared (Cochran's Q) test and I² reported for heterogeneity [ ] If I² is substantial, investigated population/dose/outcome-definition differences as possible sources [ ] Chose fixed-effect vs. random-effects model explicitly, with justification [ ] Considered, and stated, whether pooling into a single estimate is appropriate at all
Pubrica AcademyWritten and maintained by Pubrica’s subject matter experts. Articles are revised when the underlying standards change, and each article states the date on which it was last reviewed.
How Pubrica can help
Pubrica’s meta-analysis support team conducts the full pooling sequence, comprising search documentation, data extraction, heterogeneity testing and model selection, in accordance with PRISMA and Cochrane Handbook standards, so that the pooled estimate can withstand methodological scrutiny.