Free scoping read. A written assessment of your manuscript in two business days, at no cost.

Start one
Contents

Academy · Research Data, Analytics & AI · Meta-Analysis with Cohort Studies

How Meta-Analysis Works with Cohort Studies: Pooling, Confounding and Adjustment

A 2024 review of 132 published meta-analyses that pooled cohort studies alongside randomised trials found that most did not specify whether the cohort estimates used were adjusted for confounding, an omission the authors describe as a basic, remediable safeguard that most syntheses neglect.

Pubrica AcademyWritten by an evidence synthesis methodologist Reviewed bya biostatistics reviewer Published10 October 2026 Last checked10 October 2026 Reading time8 minutes Sources10 peer-reviewed

A meta-analysis combines results from multiple individual studies into a single, statistically weighted estimate, and cohort studies, which follow a group of people forward in time to observe who develops an outcome, are routinely included alongside or instead of randomized controlled trials (RCTs) when RCTs are scarce, unethical or not yet available for a given question. Conducting such an analysis correctly is more difficult than pooling RCTs alone, because cohort studies carry a risk, confounding, that randomization is specifically designed to remove and that a meta-analysis cannot correct after the fact. A 2024 meta-research study examined how well published meta-analyses handle this risk in practice and found that the basic safeguards required were frequently missing1.

Learning objectives

  • The findings of a 2024 review of 132 meta-analyses regarding how often cohort-study estimates were specified as adjusted or unadjusted
  • The weight typically carried by cohort studies in meta-analyses that combined them with RCTs
  • The specific requirements of the MOOSE reporting guideline when cohort studies are pooled

Common misconceptions

  • “If a meta-analysis includes cohort studies, the statistical pooling itself accounts for confounding.” A 2024 review of 132 published meta-analyses found that confounding was investigated in only 10% of them, and that the type of cohort-study estimate pooled, whether adjusted for confounders or not, was unspecified in 82% of cases. The pooling process does not correct for confounding that has not already been addressed within each individual cohort study.

The limitation of pooling in addressing confounding

The core vulnerability of a cohort study is that the groups exposed and not exposed to a risk factor or treatment were never randomly assigned; they differ because of whatever factors led individuals, in real-world settings, to become exposed or not. That difference, confounding, can produce an association between exposure and outcome that is unrelated to any causal effect. Within a single cohort study, researchers typically attempt to address this through statistical adjustment, including known confounders as covariates in a regression model, but such adjustment is only as complete as the set of confounders the researchers chose to measure. A meta-analysis that subsequently pools several such studies inherits whatever confounding was, or was not, addressed in each; the statistical combination of studies does not retroactively correct for a confounder that a particular cohort study never measured1. The ROBINS-I tool, developed specifically to assess this risk in non-randomized studies of interventions, guides a reviewer through seven distinct bias domains precisely because confounding is only one of several ways in which a cohort study’s result can be distorted3.

For this reason, guidance specific to the synthesis of observational studies, most notably the MOOSE (Meta-analysis Of Observational Studies in Epidemiology) checklist, asks authors to report explicitly how confounding was handled in each included study and in the synthesis itself, rather than treating pooling as a self-correcting process2. A meta-analysis of cohort studies is only as trustworthy as the weakest confounding-control practices among the studies it includes, a limitation that pooling cannot conceal and that the widely used Newcastle-Ottawa Scale attempts to quantify study by study before any pooling takes place4.

Findings of a review of 132 meta-analyses

Cheurfa and colleagues examined 132 meta-analyses, published between 2015 and 2019, that combined observational studies, prominently including cohort studies, with randomized trials when assessing healthcare interventions1. The findings describe a field in which the basic safeguards required by MOOSE are inconsistently applied. Confounding was investigated as a specific issue in only 10% of the meta-analyses reviewed1. Whether the observational-study estimates being pooled were adjusted for confounders or left unadjusted was not specified in 82% of the meta-analyses, and among the minority that did specify this, practices varied, with some analyses using only unadjusted estimates1. With respect to statistical weighting, observational studies contributed a mean of 57.3% of the total weight in these pooled analyses; cohort and other observational data therefore frequently accounted for more than half of the contribution to the final pooled estimate, rather than serving as a minor supplement to trial data1. Only 33% of the meta-analyses reported results stratified by study design (RCT versus observational), and within that minority, RCT and observational results agreed in direction in 70% of cases, leaving a substantial minority in which they diverged1.

The authors also found that study design was explored as a possible source of statistical heterogeneity in 79% of the meta-analyses that examined heterogeneity at all, with I² exceeding 50% in nearly half of those, indicating substantial inconsistency between the pooled studies in a large share of cases1, although methodological work on the I² statistic cautions that undue reliance on this single value, without examination of its confidence interval, can mislead a reviewer about the seriousness of the inconsistency9. Only 21% of the reviewed meta-analyses used the GRADE framework to appraise formally the overall certainty of the pooled evidence, leaving the majority without a standardized statement of the confidence that the pooled result warrants1, and small-study or publication-bias tests, such as Egger's regression-based funnel-plot test, were also inconsistently reported across the meta-analyses the review examined10.

Requirements for a methodologically sound cohort-inclusive meta-analysis

In light of the deficiencies identified by the 2024 review, a meta-analysis that includes cohort studies requires several specific safeguards to be trustworthy. It should state, for every included cohort study, whether the pooled effect estimate is adjusted for confounders and, if so, for which, rather than leaving this unspecified, as 82% of the reviewed meta-analyses did1. It should predefine a small set of key confounders expected to be relevant to the research question and check, study by study, whether each included cohort adjusted for at least those, which the authors explicitly recommend as a practical minimum safeguard1. Where both RCTs and cohort studies are included, results should be reported stratified by design, so that a reader can see directly whether the two types of evidence agree, rather than being presented with a single pooled figure that may mask a disagreement between study types. Finally, the overall certainty of the pooled evidence should be graded using a formal framework such as GRADE, whose handbook sets out how a body of observational evidence should be upgraded or downgraded for this kind of synthesis7, rather than left for the reader to infer from the statistics alone. None of these steps is unusual or costly to perform; the core finding of the 2024 review is that most published meta-analyses pooling cohort studies do not perform them1.

A further, often overlooked safeguard concerns how cohort studies define their exposed and comparison groups in the first place, a methodological choice that can introduce bias before any statistical adjustment is applied. Two cohort studies measuring, for example, the association of the same drug with a cardiovascular outcome may define “exposed” differently, one counting any prescription filled and another requiring a minimum duration of continuous use, producing effect estimates that are not directly comparable even before confounding is considered. The MOOSE reporting checklist asks authors pooling observational studies to extract and report these exposure and outcome definitions explicitly for every included study, specifically so that a reader can judge whether the studies being combined measure a sufficiently similar construct to be pooled meaningfully2, a requirement that closely parallels what the STROBE statement separately asks of each individual observational study’s reporting of exposure and outcome definitions8. Omitting this step risks a specific, subtle failure: a meta-analysis can report low statistical heterogeneity (a small I²) simply because the included studies agree numerically, while nonetheless pooling studies that do not address the same clinical question; no heterogeneity statistic can detect this problem on its own, because it is a conceptual rather than a statistical mismatch.

Finally, two different reasons for including cohort studies in a meta-analysis should be distinguished, because the appropriate rigor differs between them. In some cases, cohort studies are included because RCTs exist but are few, and the observational data are intended to add precision or external validity to a trial-based estimate; in other cases, cohort studies are the only available evidence because randomizing the exposure in question would be unethical or impractical, long-term smoking exposure and cancer risk being a classic example. In the first scenario, reporting results stratified by design, so that a reader can see the RCT-only estimate alongside the combined one, is especially important, since the purpose of adding observational data is to supplement, not replace, the more rigorous trial evidence, a distinction that the Cochrane Handbook’s chapter on including non-randomized studies sets out in detail6. In the second scenario, where no RCT evidence exists or is likely to exist, the burden instead falls entirely on rigorously assessing and reporting confounding control within and across the cohort studies being pooled, since no higher tier of evidence is available against which to cross-check the result.

Dose-response analysis is a further technique specific to cohort-based meta-analyses that merits attention, since it offers a partial, indirect safeguard against confounding that a simple binary exposed-versus-unexposed comparison does not. If increasing levels of an exposure (for example, higher cumulative smoking pack-years) are associated with a correspondingly graded increase in risk across several independent cohort studies, that dose-response pattern is more difficult to attribute to confounding alone, since a confounder would need to track the dose level of the exposure as closely as the true causal relationship would, in every included cohort, to account fully for the graded pattern. The Bradford Hill considerations5, long used in epidemiology to assess the overall plausibility of a causal claim from observational evidence, treat a consistent dose-response gradient as one of the more persuasive indirect indicators available for precisely this reason, and a meta-analysis that can demonstrate such a gradient across pooled cohort studies makes a stronger implicit case for causality than one relying on a single pooled hazard ratio alone.

Key points

Key points

Pooling cohort studies in a meta-analysis does not in itself correct for confounding, and a 2024 review of 132 such meta-analyses found that most did not specify whether the underlying cohort estimates were adjusted. This is a basic, verifiable gap that should be assessed in any pooled analysis.

Practical application

Practical application

  1. Determine, for every included cohort study, whether the pooled estimate is adjusted for confounders. This check, unspecified in 82% of the meta-analyses reviewed, is the clearest indicator of whether a cohort-inclusive meta-analysis has addressed its principal risk.
  2. Pre-specify the key confounders relevant to the research question before data extraction. This enables systematic assessment of whether each included cohort study adjusted for at least the most important confounders, rather than accepting whichever variables each study measured.
  3. Report RCT and observational results stratified by design, in addition to the pooled result. Only a third of the reviewed meta-analyses did so; stratification is the only means by which a reader can determine whether the two types of evidence agree.

Frequently asked questions

Yes, this is common practice, especially when RCTs are scarce; the 2024 review found that observational studies contributed a mean of 57.3% of the total weight in meta-analyses that pooled both types, but it also found that the safeguards required to do this well (reporting adjustment status and investigating confounding) were frequently missing.

It means the original cohort study used a statistical method, typically a regression model, to account for known confounding variables when calculating its effect estimate, rather than reporting the raw, unadjusted association between exposure and outcome.

Divergence indicates that confounding, a different population or another methodological difference between the designs may account for the difference, and the 2024 review found that this occurred in approximately 30% of the stratified comparisons it was able to examine.

GRADE is a widely used framework for rating the overall certainty of evidence underlying a pooled estimate; the 2024 review found that only 21% of the meta-analyses it examined used it, meaning that most readers of pooled cohort-inclusive evidence are not given a standardized statement of the confidence that the result warrants.

References

  1. Cheurfa C, Tsokani S, Kontouli KM, et al. Synthesis methods used to combine observational studies and randomised trials in published meta-analyses. Syst Rev. 2024;13:70. read 10 October 2026. View on Systematic Reviews
  2. Stroup DF, Berlin JA, Morton SC, et al. Meta-analysis of observational studies in epidemiology: a proposal for reporting. MOOSE Group. JAMA. 2000;283(15):2008-2012. read 10 October 2026. View on JAMA
  3. Sterne JAC, Hernán MA, Reeves BC, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ. 2016;355:i4919. read 10 October 2026. View on The BMJ
  4. Wells GA, et al. The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. read 10 October 2026. View on OHRI
  5. The missed lessons of Sir Austin Bradford Hill. read 10 October 2026. View on PMC
  6. Cochrane Handbook for Systematic Reviews of Interventions. Chapter 24: Including non-randomized studies on intervention effects. read 10 October 2026. View on Cochrane
  7. GRADE Working Group. GRADE Handbook: Introduction to GRADE Handbook. read 10 October 2026. View GRADE Handbook
  8. von Elm E, Altman DG, Egger M, et al. The STROBE statement: guidelines for reporting observational studies. BMJ. 2007;335(7624):806-808. read 10 October 2026. View on The BMJ
  9. Rücker G, Schwarzer G, Carpenter JR, Schumacher M. Undue reliance on I² in assessing heterogeneity may mislead. BMC Med Res Methodol. 2008;8:79. read 10 October 2026. View on BMC Medical Research Methodology
  10. Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ. 1997;315(7109):629-634. read 10 October 2026. View on The BMJ

Every source above is a peer-reviewed article, a publisher’s record of one, or a standard issued by the body that owns it. Each carries the date it was last read. Where a standard moves, this article is revised rather than left up with a note.

How to cite this article

Pick your referencing style and copy it. Your university or target journal may vary the punctuation — check its guide if it has one.

Pubrica Academy. (2026, October 10). How Meta-Analysis Works with Cohort Studies: Pooling, Confounding and Adjustment. Pubrica. https://pubrica.com/academy/meta-analysis/how-does-meta-analysis-work-with-cohort-study/

This article is written to be learned from. Cite it if you are referring to what it says; if you are using the evidence behind it, cite the sources listed above instead.

Cohort-Inclusive Meta-Analysis Checklist

This checklist should be completed before pooling cohort studies in a meta-analysis, alone or alongside RCTs.

[ ] Recorded, for each included cohort study, whether the pooled estimate is adjusted for confounders
[ ] Predefined a short list of key confounders relevant to the research question
[ ] Checked whether each cohort study adjusted for at least those predefined confounders
[ ] Reported results stratified by study design (RCT vs. observational) where both are included
[ ] Assessed and reported statistical heterogeneity (I²) and investigated study design as a possible source
[ ] Applied GRADE or an equivalent framework to state the overall certainty of the pooled evidence

Pubrica AcademyWritten and maintained by Pubrica’s subject matter experts. Articles are revised when the underlying standards change, and each article states the date on which it was last reviewed.

How Pubrica can help

Pubrica’s evidence synthesis team verifies the confounding-adjustment status and design-stratified agreement of every cohort study entering a meta-analysis, so that a pooled estimate reflects a verified safeguard rather than an assumed one.

Find something else

Search the Pubrica Academy

Access is free and requires no account. Enter a topic, or open the full library to filter by category, sub-topic or resource type.

Verifying the adjustment status of pooled cohort estimates

A scoping assessment is free and takes two business days. It provides a written assessment of what the work requires and which specialist would undertake it.

Get a free quote

Regulations

Check your project against this rule

  1. 1Your work
  2. 2Your details
  3. 3Send
What do you need?

Material you send is held under signed confidentiality, with access limited to the assigned team and encrypted transfer. It is not used to train anything and is not passed to third-party tools. Ask for the agreement before you send anything you would rather cover first.