Free scoping read. A written assessment of your manuscript in two business days, at no cost.

Start one
Contents

Academy · Research Data, Analytics & AI · NMA Assumptions & Risk of Bias

Key Assumptions in Network Meta-Analysis: Transitivity, Consistency and Their Relation to Risk of Bias

A meta-epidemiological study of 112 trial networks found that direct and indirect evidence disagreed significantly in 14% of them, nearly three times the rate expected by chance. This figure quantifies the consequences when the core assumptions of network meta-analysis do not hold. This article examines those assumptions and how they relate to risk of bias.

Pubrica AcademyWritten by an evidence synthesis statistician Reviewed bya systematic review methodologist Published10 October 2026 Last checked10 October 2026 Reading time9 minutes Sources10 peer-reviewed

Network meta-analysis rests on two linked assumptions that do not apply to a standard pairwise meta-analysis in the same way: transitivity, the premise that trials connected indirectly through a shared comparator are sufficiently similar in all relevant respects for the indirect comparison to be valid, and consistency, the premise that direct and indirect evidence for the same comparison agree once transitivity holds12. These assumptions are not merely theoretical. They can be checked statistically, and when they are checked across published networks, they fail with sufficient frequency that treating them as a formality rather than as conditions to be tested is a documented source of unreliable conclusions. The practical implications are specific: a network meta-analysis that reports a confident ranking of treatments based on an inconsistent or non-transitive network is not simply less precise than one based on a sound network; it can direct decision-makers toward an entirely incorrect treatment choice, which is why regulatory and health technology assessment bodies that rely on network meta-analysis evidence increasingly ask authors to document how both assumptions were checked, rather than merely asserting that they held.

Learning objectives

  • What transitivity and consistency mean precisely, and how they differ from each other
  • How frequently real networks show statistically significant inconsistency when formally tested
  • How study-level risk of bias informs network-level certainty frameworks such as CINeMA

Common misconceptions

  • “If a network meta-analysis runs without an error message, its assumptions have been satisfied.” A meta-epidemiological study by Song and colleagues examined 112 independent trial networks and found that direct and indirect comparisons were significantly inconsistent (P<0.05) in 16 of them, 14% (95% CI 9% to 22%), nearly three times the approximately 5% rate expected by chance, indicating that inconsistency is a measurable and non-trivial occurrence rather than a rare exception.

Requirements of the transitivity assumption

Transitivity requires that the trials connected through a common comparator are sufficiently similar in their populations, settings, co-interventions, and other effect modifiers that an indirect estimate, inferred rather than directly measured, is a valid substitute for a comparison that no trial directly tested1. Salanti's influential account of network meta-analysis assumptions frames transitivity as a clinical and epidemiological judgment, not a statistical test on its own, since it concerns whether the populations enrolled in different trials are sufficiently exchangeable for indirect comparison to be meaningful, which is assessed by comparing baseline characteristics and trial design features across the network before pooling begins1. This is a subtly different and stricter requirement than the similarity assumption underlying a standard pairwise meta-analysis, because transitivity must hold not only between the two arms of any single trial but across every trial connected anywhere in the network, which means that a network spanning many trials and many treatments offers correspondingly more opportunities for the assumption to be violated. Donegan and colleagues' systematic review of methods for assessing this and related assumptions found a wide range of approaches in use across published networks, from informal tabulation of effect-modifier distributions to formal meta-regression, with no single method universally adopted as the field's standard3. In practice, authors must actively decide, and justify, which effect modifiers matter most for their specific research question rather than checking an arbitrary generic list: a network comparing pain interventions, for example, would need to examine baseline pain severity and prior treatment history closely across trials, whereas a network comparing antihypertensives would focus more on baseline blood pressure and comorbidity burden, since these are the characteristics most plausibly capable of altering the apparent magnitude of a treatment effect in a given trial.

Consistency and the frequency of its violation

Consistency is the statistical counterpart to transitivity: once transitivity holds, direct evidence for a comparison (from trials that tested it head-to-head) and indirect evidence for the same comparison (inferred through the network) should agree within the bounds of sampling error2. Song and colleagues tested this directly across 112 independent trial networks drawn from the published literature and found statistically significant inconsistency in 16 of them, a rate of 14% (95% CI 9% to 22%), substantially above the roughly 5% false-positive rate that a significance threshold of P<0.05 would produce by chance alone if every network were truly consistent4. The study examined networks across a broad range of clinical topics rather than a single therapeutic area, which supports the interpretation that this inconsistency rate reflects a general methodological risk inherent in combining evidence across a network, rather than a problem concentrated in one unusually difficult field of medicine. Veroniki and colleagues' subsequent methodological review catalogued the statistical approaches available for detecting this kind of inconsistency, including loop-specific methods that test individual closed loops in a network and global methods that test the network as a whole, noting that different methods can disagree about whether a given network is inconsistent5. The 14% figure from Song and colleagues is also likely to be conservative with respect to the true underlying rate, since statistical tests for inconsistency generally have limited power in networks built from a modest number of trials; some genuinely inconsistent networks in their sample may therefore have gone undetected because the available evidence was too sparse to detect the problem with statistical confidence4. That possibility argues for treating a clean statistical test result as reassuring rather than conclusive, particularly in a network built from a modest number of trials per comparison.

Incorporation of study-level risk of bias into network-level assessment

Transitivity and consistency describe how trials relate to one another across a network, but each individual trial also carries its own risk of bias from the usual sources (randomization, blinding, and selective reporting) that a standard risk-of-bias tool such as RoB 2 assesses6. The CINeMA framework, developed specifically for rating confidence in network meta-analysis results, combines both layers explicitly: it rates each pairwise comparison in the network on within-study bias (aggregated from the risk-of-bias ratings of the contributing trials), indirectness, imprecision, heterogeneity, and incoherence (statistical inconsistency), producing a single certainty rating per comparison rather than one rating for the whole network7. CINeMA's structure effectively treats study-level risk of bias and network-level inconsistency as two separate, additive sources of uncertainty rather than conflating them into one undifferentiated quality score, which allows authors and readers to identify where a given comparison's uncertainty originates and therefore what, if anything, could realistically reduce it through more or better evidence. This domain-by-domain structure matters because a network can show excellent statistical consistency while still warranting a low certainty rating if the underlying trials carried a high risk of bias, and the reverse is equally possible. CINeMA's per-comparison structure also means that a single network meta-analysis can legitimately report high certainty for some comparisons and low certainty for others within the same published model, which provides a more granular and more transparent account than the single overall “quality” label that older, less structured appraisal approaches sometimes attached to an entire network.

Assessment of the assumptions in practice

In practice, authors assess transitivity by tabulating and comparing key effect modifiers, such as baseline disease severity or average follow-up duration, across every node in the network before synthesis, flagging any systematic imbalance that could bias indirect comparisons13. Consistency is then checked statistically using node-splitting, which compares the direct and indirect estimate for each comparison in turn, or a global design-by-treatment interaction test applied to the whole network89. Authors should plan and allocate resources for this checking work from the start of a network meta-analysis project rather than treating it as an optional step to be added, if time permits, after the main pooled results are available, since both the effect-modifier tabulation and the statistical consistency tests require the full extracted dataset and are most efficiently run as a defined stage in the analysis pipeline, not as an afterthought. The PRISMA extension for network meta-analysis requires both the transitivity assessment and the consistency-checking method to be reported explicitly, precisely because the measured 14% inconsistency rate found by Song and colleagues indicates that these are not formalities that can be waived without consequence104. Jansen and Naci's methodological analysis adds a final practical caveat relevant to this entire checking process: network meta-analysis is not inherently more or less valid than a standard pairwise meta-analysis, since both rest on the same underlying comparability assumptions, and the relevant question for any specific network is always how well those assumptions hold for the specific trials being combined, not which method was chosen in the abstract9. Framed in this way, the checking work described throughout this article is not a special burden that network meta-analysis imposes beyond ordinary meta-analytic practice; it is the same comparability scrutiny that any pooled analysis owes its readers, made more extensive because a network connects more trials, through more comparators, than a single pairwise comparison. Readers of a network meta-analysis are entitled to ask whether this scrutiny was performed, and a methods section that cannot answer that question clearly does not meet the standard the field now expects.

Key points

Key points

Transitivity and consistency should be assessed rather than assumed; a documented 14% rate of significant inconsistency across 112 trial networks indicates that omitting this assessment carries a measurable rather than theoretical risk.

Practical application

Practical application

  1. Effect modifiers should be tabulated across every node before the analysis is run. Baseline severity, length of follow-up and use of co-interventions should be compared across trials connected through each shared comparator, and systematic imbalance identified before synthesis.
  2. Both a loop-specific and a global inconsistency assessment should be performed. Node-splitting should be used for individual comparisons and a design-by-treatment interaction test for the network as a whole, since the two may disagree.
  3. Certainty should be rated for each comparison rather than for the network as a whole. A framework such as CINeMA, which combines within-study risk of bias, indirectness, imprecision, heterogeneity and incoherence, should be used to produce a separate rating for each pairwise comparison.

Frequently asked questions

No. Transitivity is a clinical and epidemiological judgment about whether trial populations are similar enough for indirect comparison, made by authors comparing effect-modifier distributions, not something a statistical package tests on its own.

Transitivity is the underlying assumption about trial similarity; consistency is the statistical, testable consequence that follows if transitivity holds, namely that direct and indirect evidence for the same comparison should agree.

No. It means that inconsistency is a real, non-trivial occurrence that should be actively checked for in any given network, not that most or even many specific networks are automatically compromised; each network requires its own assessment.

Veroniki and colleagues' review notes this can happen and recommends reporting results from more than one method rather than relying on a single test, since each has different statistical power to detect different patterns of inconsistency.

Yes. CINeMA also factors in indirectness, imprecision, heterogeneity, and incoherence, so a network built from low-risk-of-bias trials can still receive a low overall certainty rating if, for example, the direct and indirect evidence are statistically inconsistent.

References

  1. Salanti G. Indirect and mixed-treatment comparison, network, or multiple-treatments meta-analysis: many names, many benefits, many concerns for the next generation evidence synthesis tool. Research Synthesis Methods. 2012;3(2):80-97. read 10 October 2026. View on Research Synthesis Methods
  2. Higgins JPT, Jackson D, Barrett JK, Lu G, Ades AE, White IR. Consistency and inconsistency in network meta-analysis: concepts and models for multi-arm studies. Research Synthesis Methods. 2012;3(2):98-110. read 10 October 2026. View on Research Synthesis Methods
  3. Donegan S, Williamson P, D'Alessandro U, Smith CT. Assessing key assumptions of network meta-analysis: a review of methods. Research Synthesis Methods. 2013;4(4):291-323. read 10 October 2026. View on Research Synthesis Methods
  4. Song F, Xiong T, Parekh-Bhurke S, et al. Inconsistency between direct and indirect comparisons of competing interventions: meta-epidemiological study. BMJ. 2011;343:d4909. read 10 October 2026. View on The BMJ
  5. Veroniki AA, Vasiliadis HS, Higgins JPT, Salanti G. Evaluation of inconsistency in networks of interventions. International Journal of Epidemiology. 2013;42(1):332-345. read 10 October 2026. View on International Journal of Epidemiology
  6. Sterne JAC, Savović J, Page MJ, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ. 2019;366:l4898. read 10 October 2026. View on The BMJ
  7. Nikolakopoulou A, Higgins JPT, Papakonstantinou T, et al. CINeMA: An approach for assessing confidence in the results of a network meta-analysis. PLOS Medicine. 2020;17(4):e1003082. read 10 October 2026. View on PLOS Medicine
  8. Dias S, Welton NJ, Caldwell DM, Ades AE. Checking consistency in mixed treatment comparison meta-analysis. Statistics in Medicine. 2010;29(7-8):932-944. read 10 October 2026. View on Statistics in Medicine
  9. Jansen JP, Naci H. Is network meta-analysis as valid as standard pairwise meta-analysis? It all depends on the distribution of effect modifiers. BMC Medicine. 2013;11:159. read 10 October 2026. View on BMC Medicine
  10. Hutton B, Salanti G, Caldwell DM, et al. The PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions. Annals of Internal Medicine. 2015;162(11):777-784. read 10 October 2026. View on Annals of Internal Medicine

Every source above is a peer-reviewed article, a publisher’s record of one, or a standard issued by the body that owns it. Each carries the date it was last read. Where a standard moves, this article is revised rather than left up with a note.

How to cite this article

Pick your referencing style and copy it. Your university or target journal may vary the punctuation — check its guide if it has one.

Pubrica Academy. (2026, October 10). Key Assumptions in Network Meta-Analysis: Transitivity, Consistency and Their Relation to Risk of Bias. Pubrica. https://pubrica.com/academy/concepts-definitions/mention-some-assumptions-related-to-nma-and-the-risk-of-bias/

This article is written to be learned from. Cite it if you are referring to what it says; if you are using the evidence behind it, cite the sources listed above instead.

NMA Assumption-Checking Worksheet

This checklist should be completed before the results of a network meta-analysis are reported as final.

[ ] Tabulated key effect modifiers (severity, follow-up, co-interventions) across every network node
[ ] Flagged any systematic imbalance in effect modifiers between directly and indirectly connected trials
[ ] Run a loop-specific inconsistency check (e.g. node-splitting) on closed loops in the network
[ ] Run a global inconsistency check (e.g. design-by-treatment interaction model) on the whole network
[ ] Rated certainty per comparison using a framework like CINeMA, combining risk of bias, indirectness, imprecision, heterogeneity, and incoherence

Pubrica AcademyWritten and maintained by Pubrica’s subject matter experts. Articles are revised when the underlying standards change, and each article states the date on which it was last reviewed.

How Pubrica can help

Pubrica’s evidence synthesis statisticians conduct full transitivity and consistency assessments, including node-splitting and global inconsistency tests, before the results of a network meta-analysis are reported as final.

Find something else

Search the Pubrica Academy

Access is free and requires no account. Enter a topic, or open the full library to filter by category, sub-topic or resource type.

Assessment of assumptions in network meta-analysis

A scoping assessment is free and takes two business days. It provides a written assessment of what the work requires and which specialist would undertake it.

Get a free quote

Regulations

Check your project against this rule

  1. 1Your work
  2. 2Your details
  3. 3Send
What do you need?

Material you send is held under signed confidentiality, with access limited to the assigned team and encrypted transfer. It is not used to train anything and is not passed to third-party tools. Ask for the agreement before you send anything you would rather cover first.