Free scoping read. A written assessment of your manuscript in two business days, at no cost.

Start one
Contents

Academy · Research Data, Analytics & AI · Trial Efficacy & Safety Validity

Biostatistics in Clinical Trials: Analysis Populations, Intention-to-Treat and the Validity of Efficacy and Safety Evidence

The trustworthiness of a trial’s evidence on the benefits and harms of a therapy depends on specific, verifiable statistical choices, above all the definition of the analysis set.

Pubrica AcademyWritten by a clinical trial biostatistician Reviewed bya senior biostatistician Published10 October 2026 Last checked10 October 2026 Reading time7 minutes Sources10 peer-reviewed

A clinical trial can be well designed and still produce a misleading answer about efficacy or safety if the statistical analysis is handled inappropriately after the data have been collected. One of the clearest illustrations of this is the choice between intention-to-treat (ITT) analysis and its alternatives — a decision that can by itself determine whether the principal result of a trial is credible.

This article focuses specifically on how biostatistics protects the validity of efficacy and safety conclusions in trials evaluating prevention, detection, and therapy, using intention-to-treat analysis as the central worked example.

Learning objectives

  • What intention-to-treat (ITT) analysis means in practice, beyond the principle ‘once randomized, always analyzed’
  • Why per-protocol and as-treated analyses can produce the appearance of a treatment effect that does not exist
  • How nonadherence introduces bias that ITT is specifically designed to control
  • What trade-off ITT accepts in exchange for that protection, and why it is usually the right trade

Common misconceptions

  • “Excluding patients who did not follow the protocol makes a trial’s result more accurate.” A worked hypothetical in the literature shows a per-protocol analysis making a surgery with truly zero effect appear to cut mortality by 41%, purely because excluding nonadherent and early-dying patients broke the balance randomization had created; the correctly analyzed ITT result showed no difference.
  • “Intention-to-treat analysis only matters for efficacy, not safety.” ITT's core justification — preserving the balance in prognostic factors that randomization created — applies to any outcome being compared between groups; selectively excluding participants after randomization can distort a safety comparison by the same mechanism it distorts an efficacy comparison.
  • “If ITT gives a conservative (smaller) estimate, the drug does not work as well as hoped.” The ITT estimate reflects the effect at the level of adherence observed in that trial population; it is not a downward bias to be corrected but is often the more realistic effect size for a treatment that, in practice, will not be taken perfectly by every patient.

Analysis population choice as a question of validity

Randomization is designed to balance prognostic variables between treatment groups at baseline, so that any observed difference in outcome can be attributed to the treatment rather than to pre-existing differences between the groups1. That balance can be undone after the fact, however, if the statistical analysis silently excludes participants who were randomized but did not receive or complete their assigned treatment1.

Definition of intention-to-treat analysis

Intention-to-treat analysis requires that every randomized participant be analyzed in the group to which they were originally assigned, regardless of whether they received, completed or adhered to that treatment — summarized in the literature as “once randomized, always analyzed”1. This single rule is what preserves the baseline balance that randomization created1.

Spurious treatment effects in per-protocol analysis

A hypothetical worked example in the literature illustrates the risk: in a trial of a surgery with a true effect of zero, excluding patients who never received surgery or who died before treatment — a per-protocol or as-treated analysis — made the surgery appear to cut mortality by 41%1. The correctly conducted ITT analysis on the same data showed no difference between groups (relative risk of 1)1. The distortion was not a rounding error: it was large enough to turn a null result into an apparently dramatic benefit.

Non-random nature of nonadherence

Exclusion after randomization is hazardous because nonadherence is not random: patients who adhere to a treatment protocol tend to have better outcomes regardless of the group to which they were assigned, a pattern sometimes called the “healthy adherer” effect1. Dropping nonadherent participants therefore systematically removes sicker or less engaged patients from the analysis, which can create the appearance of benefit (or safety) that has nothing to do with the treatment itself1.

The trade-off accepted by intention-to-treat analysis

ITT is not free of trade-offs. If a treatment is genuinely effective but many trial participants did not adhere to it, ITT will tend to underestimate the effect size that would be observed under full adherence1. That conservatism, however, is unbiased for the adherence level actually observed in the trial1 — and for safety evidence in particular, a conservative estimate is a considerably smaller problem than one inflated by selectively excluding the patients who fared worse.

A trial in which per-protocol and intention-to-treat analyses gave opposite results

The distortion described above is not merely a theoretical risk. A 2016 methodology paper in Perspectives in Clinical Research examines a published trial to demonstrate the gap in practice: the LEAP study of early allergenic-food introduction in breastfed infants6. In the full, 1,162-participant intention-to-treat analysis, food allergy rates did not differ significantly between the early-introduction and standard-introduction arms (5.6% vs. 7.1%)6. In a per-protocol analysis restricted to the 732 participants who followed their assigned regimen, the early-introduction arm showed a significantly lower allergy rate than control (2.4% vs. 7.3%, P = 0.01)6. Only 32% of the intervention arm adhered to the demanding early-introduction protocol, compared with 88% of the control arm, and the analysis authors conclude that the correct, pre-specified ITT reading is that the trial did not demonstrate efficacy, despite the far more dramatic per-protocol number6. The episode serves as a caution against reporting whichever population happens to produce the more impressive headline result.

Per-protocol analysis is not, however, universally inappropriate: it has one well-recognized, narrowly defined legitimate role. In a non-inferiority trial, where the goal is to show a new treatment is not meaningfully worse than an existing one, ITT’s tendency to dilute real differences between arms can bias the result toward a false finding of non-inferiority, which is why FDA’s own 2016 guidance on non-inferiority trial design recommends reporting both ITT and per-protocol analyses and treating consistency between them as an indication of a reliable result4. This is the one trial context in which a per-protocol-type analysis is expected practice rather than a warning sign, the opposite of its role in a standard superiority trial.

A separate, more technical alternative to the ITT/per-protocol binary is the complier average causal effect (CACE), grounded in the instrumental-variables framework formalized by Angrist, Imbens, and Rubin in 19965. Rather than discarding noncompliant participants, CACE methods use randomization itself as an instrument to estimate the treatment effect specifically among the subgroup of participants who would have complied under either assignment, preserving unbiasedness in a way naive per-protocol analysis cannot. The method answers a different question from ITT (the effect among compliers, not the effect of offering the treatment to everyone), and investigators who want evidence closer to a biological “true effect” sometimes report it as a secondary, clearly labeled analysis alongside the primary ITT result rather than as a substitute for it.

The estimand framework

"Once randomized, always analyzed" is a useful shorthand, but current international guidance treats the choice of analysis population as only one part of a larger, more precise specification called an estimand. The ICH E9(R1) addendum, adopted by FDA and other regulators, requires trial sponsors to define, before unblinding, exactly what clinical question a trial's primary analysis is meant to answer: the target population, the outcome variable, how specific post-randomization events (a rescue medication, a treatment switch, a discontinuation) will be handled, and the population-level summary measure2. The guidance names five explicit strategies for handling such intercurrent events, including a "treatment policy" strategy — in which the occurrence of the intercurrent event is treated as irrelevant to the question being asked, which is close to the spirit of classic ITT — and a "hypothetical" strategy, which asks what the outcome would have been had the intercurrent event not occurred at all2. The practical consequence is that modern regulatory science no longer treats “ITT versus per-protocol” as the whole question; it requires investigators to state in advance which of several distinct scientific questions a given analysis is intended to answer.

Magnitude of the healthy-adherer effect

The claim that nonadherence is not random, and that adherent patients do better regardless of what they are adhering to, is itself backed by a specific, replicated evidence base rather than a single textbook example. The original demonstration comes from the Coronary Drug Project, a 1980 placebo-controlled trial in which patients who took their assigned placebo faithfully (at least 80% of doses) had substantially lower five-year mortality than those who did not, despite both groups receiving an inert pill with no pharmacological effect7. A 2006 meta-analysis in the BMJ, pooling adherence-mortality data across multiple placebo-controlled trials, confirmed this was not a one-off finding: better-adherent patients, even in the placebo arm, consistently showed lower mortality than poorer adherers8. A more recent 2011 study extended the pattern to real-world statin use, finding that adherent statin users had fewer adverse events entirely unrelated to statins — such as fall-related injuries — than nonadherent users (rate ratio 0.93), which the authors attribute to adherent patients being systematically healthier and more engaged with their own care, not to any effect of the statin itself9. Together, this evidence base explains why per-protocol comparisons are considered structurally biased rather than merely “less conservative”: the bias runs in a consistent, well-documented direction across unrelated drug classes and unrelated trials.

Missing outcome data

Preserving the randomized population on paper does not fully solve the validity problem if a meaningful fraction of those participants have no recorded outcome at all. A 2010 report commissioned by the FDA from a National Research Council panel on missing data in clinical trials distinguished this problem carefully from protocol nonadherence: a patient can discontinue assigned treatment yet still have their outcome measured and analyzed under ITT, while a different patient's outcome may simply never be observed (what the panel terms analysis dropout, as distinct from treatment dropout)3. The panel's central recommendation is that trials should minimize missing data at the design stage wherever possible, and that simple single-imputation fixes such as last-observation-carried-forward should not be relied upon as a default, because they can introduce their own systematic bias rather than neutrally filling a gap3. Reporting guidance built on this evidence base, including the CONSORT 2010 statement's participant flow diagram requirements, now expects trials to account for every randomized participant explicitly, specifically so that readers can judge for themselves whether missing data patterns threaten the ITT estimate's validity10.

Key points

Key points

The analysis population is among the most influential statistical determinants of whether a trial’s efficacy or safety conclusion is trustworthy; intention-to-treat analysis exists specifically to prevent the balance created by randomization from being broken after the fact.

Practical application

Practical application

  1. Confirm the analysis population before relying on the headline result. Whether the primary analysis is ITT, per-protocol or as-treated should be established; this may indicate more about the reliability of the reported effect than the p-value does.
  2. Treat large discrepancies between per-protocol and ITT results as a warning sign. Where a trial reports a substantially larger effect in a per-protocol subset than in its ITT analysis, the reasons for exclusion, and whether exclusion was related to outcome, warrant scrutiny.
  3. Apply the same scrutiny to safety analyses as efficacy analyses. The mechanism that distorts efficacy estimates through selective exclusion works identically on adverse-event comparisons.

Frequently asked questions

Yes. ITT analyzes every randomized participant according to the assigned group, regardless of whether they received or completed treatment, which is the essential principle of the method.

It can be reported as a supportive or exploratory analysis alongside ITT, but relying on it as the primary analysis risks the kind of bias illustrated by the per-protocol mortality example, where a genuinely null effect appeared beneficial.

Because it is unbiased for the adherence level actually observed in the trial population, rather than an inflated number produced by excluding the patients who did worse.

References

  1. McCoy CE. Understanding the intention-to-treat principle in randomized controlled trials. Western Journal of Emergency Medicine. 2017;18(6):1075-1078. read 10 October 2026. View on PubMed Central
  2. International Council for Harmonisation. ICH E9(R1): Statistical Principles for Clinical Trials: Addendum: Estimands and Sensitivity Analysis in Clinical Trials. read 10 October 2026. View source
  3. National Research Council. The Prevention and Treatment of Missing Data in Clinical Trials. Panel on Handling Missing Data in Clinical Trials. Washington, DC: The National Academies Press; 2010. read 10 October 2026. View source
  4. U.S. Food and Drug Administration. Non-Inferiority Clinical Trials to Establish Effectiveness: Guidance for Industry. read 10 October 2026. View source
  5. Angrist JD, Imbens GW, Rubin DB. Identification of causal effects using instrumental variables. Journal of the American Statistical Association. 1996;91(434):444-455. read 10 October 2026. View source
  6. Ranganathan P, Pramesh CS, Aggarwal R. Common pitfalls in statistical analysis: intention-to-treat versus per-protocol analysis. Perspectives in Clinical Research. 2016;7(3):144-146. read 10 October 2026. View on PubMed Central
  7. Coronary Drug Project Research Group. Influence of adherence to treatment and response of cholesterol on mortality in the coronary drug project. New England Journal of Medicine. 1980;303(18):1038-1041. read 10 October 2026. View on PubMed Central
  8. Simpson SH, Eurich DT, Majumdar SR, Padwal RS, Tsuyuki RT, Varney J, Johnson JA. A meta-analysis of the association between adherence to drug therapy and mortality. BMJ. 2006;333(7557):15. read 10 October 2026. View on PubMed Central
  9. Patrick AR, Shrank WH, Glynn RJ, et al. The association between statin use and outcomes potentially attributable to an unhealthy lifestyle in older adults. Value in Health. 2011;14(4):513-520. read 10 October 2026. View on PubMed Central
  10. Schulz KF, Altman DG, Moher D; CONSORT Group. CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c332. read 10 October 2026. View on PubMed Central

Every source above is a peer-reviewed article, a publisher’s record of one, or a standard issued by the body that owns it. Each carries the date it was last read. Where a standard moves, this article is revised rather than left up with a note.

How to cite this article

Pick your referencing style and copy it. Your university or target journal may vary the punctuation — check its guide if it has one.

Pubrica Academy. (2026, October 10). Biostatistics in Clinical Trials: Analysis Populations, Intention-to-Treat and the Validity of Efficacy and Safety Evidence. Pubrica. https://pubrica.com/academy/concepts-definitions/biostatistics-importance-in-clinical-trial-prevention-detection-and-therapy/

This article is written to be learned from. Cite it if you are referring to what it says; if you are using the evidence behind it, cite the sources listed above instead.

Trial validity screening checklist

Use this when reviewing a clinical trial's efficacy or safety claims.

[ ] Confirm the stated primary analysis population (ITT, per-protocol, or as-treated)
[ ] Check whether all randomized participants are accounted for
[ ] Compare ITT and per-protocol results if both are reported
[ ] Verify the same analysis logic is applied to safety outcomes, not only efficacy
[ ] Flag any unexplained post-randomization exclusions

Pubrica AcademyWritten and maintained by Pubrica’s subject matter experts. Articles are revised when the underlying standards change, and each article states the date on which it was last reviewed.

How Pubrica can help

Pubrica’s clinical trial biostatisticians specify and defend intention-to-treat and supportive analysis populations so efficacy and safety conclusions withstand scrutiny.

Find something else

Search the Pubrica Academy

Access is free and requires no account. Enter a topic, or open the full library to filter by category, sub-topic or resource type.

Review of analysis populations for efficacy and safety outcomes

A scoping assessment is free and takes two business days. It provides a written assessment of what the work requires and which specialist would undertake it.

Get a free quote

Regulations

Check your project against this rule

  1. 1Your work
  2. 2Your details
  3. 3Send
What do you need?

Material you send is held under signed confidentiality, with access limited to the assigned team and encrypted transfer. It is not used to train anything and is not passed to third-party tools. Ask for the agreement before you send anything you would rather cover first.