Free scoping read. A written assessment of your manuscript in two business days, at no cost.

Start one
Contents

Academy · Research Data Analytics & AI · Causal inference and regulation

Why Prediction Is Not Causation in Machine Learning for Health Care

A model can predict an outcome extremely well and still be useless, or actively harmful, as the basis for a decision. The two tasks answer different questions — who will experience this outcome under current care, versus what will happen if we act — and they require different data, different designs and different evidence. Regulators are increasingly explicit about the distinction; most published models are not.

Pubrica AcademyWritten by the medical writing and biostatistics teams Reviewed byPubrica biostatistics team Published9 October 2026 Last checked9 October 2026 Reading time20 minutes Sources63 peer-reviewed

The sentence that should stop a reviewer is “patients identified as high risk by the model should receive the intervention.” Nothing in a predictive model licenses that step. The model was fitted to data generated under existing practice, which means the clinicians’ own decisions are part of what it learned, and a feature can predict an outcome precisely because it already triggers treatment. This page sets out why the two questions come apart, what it takes to answer the causal one, and how regulatory frameworks for machine-learning tools treat the difference.

What you will learn

  • The three distinct questions a clinical model can be asked, and which design each one needs
  • Why a feature can predict an outcome because it triggers treatment — and what that does to a deployed model
  • What a shortcut feature is in structural terms, and why it is a causal problem rather than a data-cleaning one
  • How target trial emulation turns an observational dataset into an answerable causal question
  • The difference between generalisability and transportability, and why a validated model can still fail in a new population
  • How regulators approach machine-learning-based devices, including the evidence base behind clearances and the problem of models that update
  • How to write a prediction-model paper so the causal claim matches the design

What to unlearn

  • “If the model predicts well, it will support decisions well.” Predictive accuracy under current care says nothing about what happens under a different care pattern. A model deployed to change practice invalidates the data it was fitted to.
  • “We adjusted for confounders, so the estimate is causal.” Adjustment is only valid for a set identified from an assumed causal structure. Putting every available variable into a model can introduce bias rather than remove it, including through colliders and mediators.
  • “Machine learning avoids the assumptions that regression needs.” It avoids some functional-form assumptions and none of the identification assumptions. No algorithm substitutes for knowing what adjusts away confounding.
  • “An explanation tells us the model is safe.” Post-hoc explanation describes the model’s behaviour, not the world. It cannot establish that an association is causal, and it is not a substitute for prospective evidence.
  • “Regulatory clearance means the tool was shown to improve outcomes.” Clearance pathways turn substantially on equivalence and on the adequacy of specified evidence, which is frequently retrospective. Clearance is not an outcome claim.

Machine learning, regulation and causation at a glance

Subject of the original article
Regulatory affairs for machine-learning tools in health care, and the causal inference needed to show they are safe and effective
Main US route to market
85.9% of 64 AI/ML devices in one analysis, and 95.0% of 343 in another, were cleared through the 510(k) pathway4025
Models that change
Predetermined change control plans pre-specify permitted updates163
Core distinction
Prediction under current care, prediction under intervention and causal effect need different designs2657
Commonest error
Recommending an action on the strength of predictive accuracy alone
Reading time
About 20 minutes, with 4 new figures and 3 new tables

Regulatory affairs for machine learning in health care: the FDA’s approach

The title of this page refers to regulatory affairs for machine learning in health care: how tools that use machine learning to support diagnosis or treatment are authorised, and what evidence regulators ask for. In the United States most such tools reach the market as medical devices. Benjamens, Dhunnoo and Meskó cross-checked FDA approvals and identified 64 AI or machine-learning-based devices, of which 85.9% were cleared through the 510(k) pathway, 12.5% through the de novo pathway and one through premarket approval; only 45% of them mentioned AI or machine learning in the official FDA announcement40. Zhu and colleagues reviewed the approval letters of 343 AI/ML-enabled devices authorised by June 2021 and found that 95.0% were cleared under 510(k), that 69.4% were software as a medical device, and that 70.3% were reviewed by the radiology panel25.

How AI and machine-learning devices have reached the US market

510(k) clearance64 AI/ML devices (Benjamens et al. 2020)85.9%
De novo pathwaysame 64 devices12.5%
Premarket approval (PMA)same 64 devices1.6%
510(k) clearance343 AI/ML-enabled devices to June 2021 (Zhu et al. 2022)95.0%
Software as a medical devicesame 343 devices69.4%
Reviewed by the radiology panelsame 343 devices70.3%

First colour: Benjamens et al. (2020). Second colour: Zhu et al. (2022). Bar length is the percentage of devices in each analysis.

Figure 1. Regulatory pathways and characteristics of FDA-authorised AI and machine-learning devices in two published analyses. In the earlier analysis, only 29 of the 64 devices (45%) mentioned any AI or machine-learning expression in the official FDA announcement.4025

Two further themes run through the regulatory literature. The first is the gap between what is cleared and what is claimed: Clark, Kim and Aphinyanaphongs compared 119 510(k) summaries with the devices’ marketing materials and classed 15 devices as discrepant and 8 as contentious, concluding that discrepancies were present in about one-fifth of the devices surveyed62. The second is the model that keeps learning. Lee and colleagues describe the United States as advancing mechanisms for predictable adaptation, including predetermined change control plans, while the European Union and South Korea have taken different routes63; the FDA, Health Canada and the MHRA have published guiding principles for such plans1.

2019The lockdown dilemmaBabic et al., Science: freezing a model guarantees what was evaluated is what is deployed, at the cost of forbidding updates42
2020First open databaseBenjamens et al., npj Digital Medicine: 64 FDA-approved AI/ML devices identified and catalogued40
2022Landscape to June 2021Zhu et al., International Journal of Medical Informatics: 343 AI/ML-enabled devices analysed25
2023Predicate networksMuehlematter et al., The Lancet Digital Health: more than a third traced to predicates that were not AI/ML-based18
2023Marketing versus clearanceClark et al., JAMA Network Open: 15 of 119 cleared devices marketed in ways discrepant with their 510(k) summaries62
2025Change control principlesFDA, Health Canada and MHRA: guiding principles for predetermined change control plans1
2026Lifecycle governanceLee et al., Journal of Medical Internet Research: continuous oversight proposed for learning medical AI63
Figure 2. Selected peer-reviewed analyses and regulatory documents on machine-learning devices, from the lockdown dilemma to lifecycle governance. A selection, not a complete list.

Three regulatory orientations to learning medical AI, as described by Lee et al. (2026)

JurisdictionOrientationMechanisms named
United StatesPredictable adaptationPredetermined change control plans, real-world evidence frameworks and lifecycle-oriented quality management reforms
European UnionPrecautionary, rights-based governanceThe Artificial Intelligence Act and modernised liability rules
South KoreaStatutory framework for learning medical AIThe Digital Medical Products Act, described as one of the first comprehensive statutory frameworks of its kind

Summarised from the abstract of Lee et al.63, who propose four pillars for oversight: continuous validation, agile change management, proactive performance surveillance and distributed accountability.

Why the rest of this article focuses on prediction versus causation

Each regulatory question above (whether a device performs as specified in its intended population, whether an update is safe, whether marketing claims match what was cleared) turns on what a model’s output licenses a clinician to do. That is a causal question, and the commonest weakness in published clinical models is to answer it with predictive evidence. The rest of this article therefore concentrates on the difference between prediction and causation, and returns to regulators in a later section.

Three questions, routinely confused

Almost every argument about machine learning in health care becomes clearer once you separate three questions that a model might be asked.

The first is prediction under current care: given what I know now, who is likely to experience this outcome if nothing changes? This is what a conventional predictive model estimates, and it is a legitimate and useful quantity — for resource planning, for triage, for prognostic information.

The second is prediction under a hypothetical intervention: what is this patient’s risk if we treat, and what is it if we do not? Lin and colleagues, in Diagnostic and Prognostic Research, carried out a scoping review of the causal methods that make this kind of prediction possible, and the point of their review is that it requires methods a standard predictive pipeline does not contain 26.

The third is a causal effect: does this intervention change the outcome, and by how much, in this population? Pearl’s overview in Statistics Surveys gives the formal machinery for why these are different objects and why no amount of data on the first identifies the third 57.

The confusion is not academic. A model answering the first question is routinely deployed as if it answered the second, which is the single most consequential methodological error in clinical machine learning. Piccininni and colleagues, in BMC Medical Research Methodology, show how directed acyclic graphs make the distinction visible in clinical risk prediction modelling specifically — drawing the graph is often enough to reveal that the intended use requires a quantity the model does not estimate 35.

Three questions a clinical model can be asked, and what each one requires

QuestionWhat is estimatedWhat the design must provideThe error that follows from confusing it
Prediction under current care
who will have this outcome?
Risk given observed features, in a population managed as the training population wasRepresentative sampling, a stated prediction time point, calibration in the target populationDeployed as a treatment trigger. The model flags patients already being treated and misses those not yet acted on.
Prediction under a hypothetical intervention
what is the risk if we treat, and if we do not?
Risk under each of two or more specified treatment strategiesAn identified causal structure, treatment strategies defined well enough to be assigned, methods for predictions under interventionReported as if it were ordinary prediction, hiding the assumptions the estimate depends on.
Causal effect
does acting change the outcome?
Contrast between outcomes under treatment and under the comparator in a defined populationRandomisation, or an emulated target trial with a stated protocol and no post-baseline eligibilityInferred from predictive performance, which identifies no causal quantity at all.

Framing follows Pearl in Statistics Surveys on the formal distinction, Lin and colleagues in Diagnostic and Prognostic Research on prediction under hypothetical interventions, Piccininni and colleagues in BMC Medical Research Methodology on causal thinking in clinical risk prediction, and Hernan and Robins in the American Journal of Epidemiology on target trial emulation. No performance figures are implied.

1Will the output only describe who is likely to have the outcome if care continues as now?Prediction under current care
2Will it be used to compare a patient’s risk if treated and if not treated?Prediction under intervention
3Will it be used to claim that acting on it changes outcomes?Causal effect: trial or emulated trial
Figure 3. Work down the questions and stop at the first “yes”; the answer names the estimand the design must support (see the table above).

How a well-predicting model becomes a bad decision rule

Consider a model that predicts in-hospital deterioration and performs excellently. Among its strongest features is whether the patient is receiving a particular escalation of care. The feature predicts deterioration well because clinicians give it to patients they judge to be deteriorating. The model has learned the clinicians’ judgement, encoded as a treatment variable, and it is an excellent predictor precisely because of that.

Now deploy it as a decision rule. The model flags patients already receiving the escalation — the ones who least need flagging — and misses the patients whose clinicians have not yet acted, who are the entire reason for building it. If the deployment then changes who receives the escalation, the relationship the model learned no longer holds, and performance degrades in a way that monitoring the output alone will not explain.

This is not a contrived example; it is the structure of a whole class of failures, and it generalises. Any variable that records a clinical response to the outcome will predict the outcome, and will invert or collapse the moment the model changes practice. The implication for authors is specific: when you list candidate features, flag every one that could be a response to the condition rather than a sign of it, and either exclude it or state that the model is prognostic under current care only and must not be read as a treatment trigger. The implication for readers is to look for that sentence and distrust the paper when it is absent.

Early signs ofdeteriorationClinician judgementEscalation recordedmodel featureDeteriorationmodel outcomeAssociation themodel learns
Figure 4. Conceptual diagram (not to scale, no real values) of the deterioration example. Solid arrows are causal; the dashed line is the association a predictive model exploits. Escalation predicts deterioration because both follow from the same early signs, so using the model to trigger escalation changes the relationship it relies on.

Shortcut features are a causal problem, not a data-quality one

Bellamy, Hernán and Beam, in the European Journal of Epidemiology, give a structural characterisation of shortcut features: a shortcut is a feature associated with the outcome through a path that is specific to the dataset rather than stable across settings 22. Characterising it this way matters because it tells you what will and will not fix the problem. More data from the same source will not; a larger model will not; regularisation will not. What helps is identifying, from the assumed causal structure, which associations should be stable and which are artefacts of how this dataset came to exist.

Subbaswamy and Saria, in Biostatistics, take this from diagnosis to construction, showing how causal reasoning about dataset shift supports building models that are deliberately stable across the shifts you anticipate — accepting lower apparent performance in the development data in exchange for performance that survives deployment 37. Kyono and van der Schaar, in IEEE Transactions on Artificial Intelligence, address the related practical problem of selecting between models when you have no labels in the target domain, exploiting causal structure to do it.

The appraisal question that follows is short and rarely asked: which of this model’s inputs would you expect to retain their relationship with the outcome at a different hospital, in a different year, under a different care pattern — and which would not? A paper that cannot answer it has not established that it has a model rather than a description of one dataset.

What “we adjusted for confounders” has to mean

Confounding has a precise definition and the loose usage causes real errors. VanderWeele and Shpitser, in the Annals of Statistics, work through what it takes for a variable to be a confounder, and Greenland, Robins and Pearl, in Statistical Science, set out the relationship between confounding and collapsibility — the reason a measure of association can change on adjustment without any confounding being removed 56,60. Greenland and Morgenstern’s treatment in the Annual Review of Public Health remains the clearest practical statement for health researchers.

The operational consequence is that a valid adjustment set cannot be chosen from the data. It follows from an assumed causal structure, which has to be stated. “We included all available covariates” is not a conservative choice: conditioning on a collider induces association where none existed, and conditioning on a mediator removes part of the effect you were trying to estimate. A high-capacity machine learning model with every variable in the feature set does this enthusiastically and silently.

Missing data sits in the same category. Rubin’s paper in Biometrika established that the handling of missing data depends on assumptions about why it is missing, and in clinical data missingness is very often informative — a test was not ordered because the clinician did not suspect the condition 61. A model that treats missingness as a feature is then learning the clinician’s suspicion, with all the consequences described in the section above. Glymour, Zhang and Spirtes, in Frontiers in Genetics, review causal discovery methods that attempt to learn structure from data rather than assume it; these are useful tools and they do not remove the need to state what you assumed.

Weak adjustment statement

“We adjusted for all available covariates to control for confounding.”

No causal structure is stated, so a reader cannot tell whether colliders or mediators were conditioned on, and the claim that confounding was controlled cannot be checked.

Stronger adjustment statement

“The adjustment set was chosen from the causal diagram in the supplement. Variables measured after treatment initiation were excluded as potential mediators, and laboratory-test missingness was treated as informative rather than as a predictor.”

States where the set came from, why some variables were left out, and how missingness was handled, so each assumption can be challenged.

Illustrative wording written for this article; not quoted from a published paper.

Asking a causal question of observational data: the target trial

Where a randomised trial is not available, the most disciplined available approach is to specify the trial you would have run and then emulate it. Hernán and Robins, in the American Journal of Epidemiology, set out the protocol: write down the eligibility criteria, the treatment strategies being compared, the assignment procedure, the start and end of follow-up, the outcome, the causal contrast of interest and the analysis plan — and only then go to the data 54.

The value of the exercise is that it forces the ambiguities into the open before any estimate exists. Most of the irreparable problems in observational treatment-effect studies are visible at the protocol stage: there is no clear time zero, the treatment strategy is not well defined, eligibility is determined using information from after baseline, or the comparison group is composed of people who were never candidates for the treatment. Each of these is fatal, each is cheap to find on paper, and each is expensive to discover after the analysis.

For machine learning specifically, the target trial framing also clarifies where flexible models genuinely help. They are well suited to estimating the nuisance components — the probability of treatment given covariates, the outcome regression — within an estimator whose identification rests on a stated causal structure. They are not a route to skipping that structure. If you are planning this kind of analysis, the design work belongs with the statistical team from the start: see the biostatistics role hub and our statistical analysis plan service.

Target trial protocol components and the failure each one exposes

ComponentWhat to specifyFailure it exposes if left vague
Eligibility criteriaWho could have been enrolled, using information available at baseline onlyEligibility determined from information after baseline
Treatment strategiesEach strategy defined well enough to be assignedA treatment strategy that is not well defined
Assignment procedureHow assignment is emulated, and the adjustment set that justifies itA comparison group of people who were never candidates for treatment
Start and end of follow-upTime zero aligned with eligibility and assignmentNo clear time zero
Outcome and causal contrastThe outcome and the contrast of interest, stated before analysisAn estimand chosen after seeing the data
Analysis planEstimator, nuisance models (where flexible machine learning can help) and sensitivity analysesFlexible models used as a substitute for an identified structure

Components follow Hernán and Robins54; the failures are those listed in the section above.

Generalisability and transportability are different problems

A model is fitted to a population, not to a disease, and the question of whether it carries to another population has two distinct forms. Degtiar and Rose, in the Annual Review of Statistics and Its Application, separate them: generalisability concerns extending inference from a study sample to the population it was drawn from, transportability concerns extending it to a different population altogether 17. Justice and colleagues, in the Annals of Internal Medicine, framed the same issue for prognostic information decades ago, and the framing has not improved on.

The methodological literature here is mature and under-used by model developers. Steingrimsson and colleagues, in the American Journal of Epidemiology, address transporting a prediction model to a new target population directly 21. Dahabreh and colleagues, in Statistics in Medicine, extend inferences from a randomised trial to a new target population; Westreich and colleagues, in the American Journal of Epidemiology, give the inverse odds of sampling weights approach; Rudolph and van der Laan, in the Journal of the Royal Statistical Society Series B, treat the transport of encouragement-design effects across sites; and Colnet and colleagues, in the Journal of Causal Inference, provide a sensitivity analysis for the common situation where the target population is missing covariates you would need 23,36,51,52. Bareinboim and Pearl, in the Proceedings of the National Academy of Sciences, give the general data-fusion framework in which all of these sit 53.

The practical point for a prediction-model paper is that “externally validated” is not one claim. Validated in a second hospital in the same health system, in a different country, or in a population with materially different prevalence and case mix are three different findings. Say which one you have, describe the target population you are claiming transportability to, and state what you assumed makes the transport valid.

How regulators actually approach machine-learning tools

Two features of the regulatory landscape surprise researchers coming from the methods literature. The first is how much of it rests on equivalence rather than on fresh clinical evidence. Muehlematter, Bluethgen and Vokinger, in The Lancet Digital Health, analysed the predicate networks behind cleared AI and machine-learning-based medical devices and found that more than a third of them originated from predicate devices that were not themselves AI or machine-learning-based. Zhu and colleagues, in the International Journal of Medical Informatics, catalogued the characteristics and intended uses of the devices approved to that point, and Wu and colleagues, in Nature Medicine, examined how medical AI devices are evaluated and documented the limitations: evaluations frequently retrospective, test data often from few sites, and subgroup performance rarely reported 25,31.

The second is the problem of a model that changes. Babic and colleagues, in Science, described the dilemma of algorithms held on regulatory lockdown: freezing a model guarantees that what was evaluated is what is deployed, at the cost of forbidding the updating that is the method’s chief advantage 42. Gilbert and colleagues, in the Journal of Medical Internet Research, discuss algorithm change protocols as the mechanism for pre-specifying permitted changes, and Gerke and colleagues, in npj Digital Medicine, argue for a system-level view in which the regulated object is not the algorithm alone but the algorithm in its clinical workflow 27,39. Larson and colleagues, in the Journal of the American College of Radiology, set out concrete recommendations for imaging algorithms along these lines.

The implication for anyone writing a submission or a clinical evaluation is that the causal and transportability questions above are not academic extras. They are the content of the argument that the device will perform as specified in the population it will be used on. Our regulatory writing and clinical evaluation report pages cover the documents; what belongs in them is decided here.

Regulatory-readiness checklist for a machine-learning tool

  • State the intended use and which of the three questions the model answers, in the same words in the paper, the submission and the marketing material.62
  • Identify the likely pathway and predicate device, and say where the predicate differs from the proposed device.18
  • Describe the evaluation data: retrospective or prospective, number of sites, and performance by subgroup.
  • Declare whether the model is locked; if not, pre-specify permitted changes and how each will be evaluated.1
  • Name the target population and the argument that performance will transport to it.
  • Plan post-market performance monitoring rather than treating clearance as the end of evaluation.63
More than 1 in 3cleared AI/ML-based devices whose predicate device was not itself AI/ML-based

From an analysis of the 510(k) predicate networks behind cleared AI and machine-learning-based medical devices. Source: Muehlematter UJ, Bluethgen C, Vokinger KN. FDA-cleared artificial intelligence and machine learning-based medical devices and their 510(k) predicate networks. The Lancet Digital Health.

Explainability does not substitute for evidence

Babic, Gerke, Evgeniou and Cohen, in Science, argue against treating explainability as a safety requirement for medical AI, and the argument is worth understanding rather than dismissing 28. Post-hoc explanations describe how a model behaves. They do not establish that the relationships the model exploits are causal, they can be unstable across methods and runs, and they offer a false sense of assurance that displaces the evidence that would actually warrant trust — prospective evaluation, external validation and monitoring.

Moraffah and colleagues, in the ACM SIGKDD Explorations Newsletter, survey causal interpretability as a more demanding alternative: explanations framed in terms of interventions rather than input sensitivity 34. That is the right direction, and it is also harder, because it inherits all the identification problems in the sections above. The honest position for a paper is to report what explanation methods revealed as a debugging finding, state plainly what they cannot establish, and not offer interpretability as a reason the model is fit for use.

Writing it up so the claim matches the design

Reporting guidance for this field has converged quickly. Collins and colleagues’ TRIPOD+AI statement in BMJ is now the reference point for reporting clinical prediction models developed with regression or machine learning, and it is explicit about the items this page has been circling: the source of data and the eligibility criteria, the predictors and when they were measured relative to the outcome, how the model was developed and validated, calibration as well as discrimination, performance in relevant subgroups, and a statement of intended use 14. The earlier protocol by Collins and colleagues in BMJ Open also describes PROBAST-AI, the companion risk-of-bias tool, which is what a methods reviewer will reach for 32.

For diagnostic accuracy studies specifically, Sounderajah and colleagues, in BMJ Open, set out the development of STARD-AI. Trial-stage guidance for interventions involving AI — the SPIRIT-AI, CONSORT-AI and DECIDE-AI family — is covered on our page on AI and machine learning in clinical development, and the general checklist landscape on reporting guidelines 29. Where a guideline itself was produced by a consensus process, Gattrell and colleagues’ ACCORD statement in PLOS Medicine now sets the standard for reporting how that consensus was reached.

What none of these guidelines can do is fix a mismatch between claim and design after the fact. The commonest version of that mismatch is a prediction model whose abstract recommends an action, and it is visible in a single sentence.

A prediction model’s conclusion, written two ways

Claims more than the design supports

Our machine learning model identified patients at high risk of deterioration with excellent discrimination, outperforming the existing clinical score. Patients flagged by the model should be prioritised for early escalation of care, and the model could be integrated into the electronic record to reduce adverse events.

“Should be prioritised” is a causal claim derived from a predictive estimate, which identifies no causal quantity.

“Reduce adverse events” was not measured. No outcome under the proposed intervention exists anywhere in the study.

Treatment-response predictors are unexamined. If the model learned the clinicians’ judgement, deployment inverts the relationship it relies on.

“Outperforming the existing score” compares discrimination, not decisions — and discrimination is prevalence-dependent and threshold-free.

No target population named, so the transportability claim implicit in “integrated into the electronic record” cannot be assessed.

Matches claim to design

The model discriminates between patients who did and did not deteriorate under the care patterns prevailing in the development cohort, with discrimination and calibration reported for the external cohort and by subgroup. Because several predictors record clinical responses to suspected deterioration rather than signs of it, the model is prognostic under current care and must not be used as a trigger for escalation: it cannot estimate risk under an intervention, and we have not evaluated what happens to its predictors once escalation decisions change. Whether acting on the model reduces adverse events is an untested causal question that would require a prospective comparison against current practice.

The estimand is stated: discrimination under the care patterns of the development cohort, not risk under intervention.

The treatment-response problem is named rather than left for a reviewer to find, with the consequence spelled out.

Calibration and subgroup performance appear, which is what TRIPOD+AI asks for and what decisions at a threshold depend on.

The intended use is restricted explicitly, so the paper cannot be cited for a use it does not support.

The causal question is identified as open, with the design that would answer it named.

The second version makes a smaller claim and is far harder to refute, which is why it ages better. More importantly, it is the version that can be used: a reader can tell what the model is for, a regulator can tell what has been evidenced, and a subsequent trial has a stated hypothesis to test. The first version will be cited for a conclusion the study never reached.

Take-home message

Take-home message

Prediction and causation answer different questions and require different designs. A model fitted to data generated under current practice has learned that practice, including the clinicians’ own treatment decisions, so a feature can predict an outcome precisely because it triggers treatment — which makes the model a poor and sometimes harmful decision rule. Valid causal claims need an identified structure, stated assumptions and, where no trial exists, an explicitly emulated target trial. Transportability to a new population is a further claim requiring its own argument. Regulatory frameworks rest substantially on equivalence rather than fresh outcome evidence, so clearance is not a demonstration of benefit. Report under TRIPOD+AI, state the intended use, and make the claim no larger than the design.

How to apply this to your study

How to apply this to your own study

  1. Write down which of the three questions your model answers — prediction under current care, prediction under intervention, or causal effect — and put the answer in the methods. Most papers never state it, and most confusion follows from that.
  2. Go through your feature list and flag every treatment-response variable. Anything that records a clinical reaction to the suspected condition will predict it and will invert on deployment. Exclude it, or restrict the intended use in writing.
  3. Draw the causal diagram before choosing the adjustment set. If you cannot draw it, you cannot justify the adjustment, and “we included all covariates” is a choice with its own bias, not a neutral default.
  4. If the question is causal and there is no trial, write the target trial protocol first — eligibility, strategies, assignment, time zero, follow-up, outcome, contrast, analysis — and only then open the dataset.
  5. Name the target population you claim transportability to, and state what makes the transport valid. “Externally validated” in a second hospital of the same system is a much weaker claim than it sounds.
  6. Report calibration and subgroup performance, not discrimination alone. Decisions happen at thresholds, and TRIPOD+AI treats both as required items.
  7. Remove every action recommendation from the abstract that the design cannot support. Replace it with the study that would support it.
  8. Treat explanation output as a debugging finding. Report what it revealed; do not offer it as evidence of fitness for use.
  9. If a regulatory submission is in view, specify the change-control position early — whether the model is locked, and if not, what changes are pre-specified and how they will be evaluated.

Common questions

Prediction estimates who will experience an outcome given observed features, in a population managed as the training population was. Causal inference estimates what would happen if we acted. They are different mathematical objects, as Pearl sets out in Statistics Surveys, and no quantity of predictive data identifies a causal effect. The practical consequence is that a model can discriminate excellently and still be a poor decision rule, because the features driving the prediction may be consequences of clinical decisions rather than causes of the outcome. Lin and colleagues, in Diagnostic and Prognostic Research, review the methods needed for the intermediate case: prediction under a hypothetical intervention, which requires causal machinery a standard pipeline lacks.

Through a specific and common mechanism. Suppose a strong predictor of deterioration is receipt of an escalation of care — strong precisely because clinicians give it to patients they judge to be deteriorating. The model has learned their judgement. Deployed as a trigger, it flags patients already being treated and misses those whose clinicians have not yet acted, which is the entire use case. Worse, if deployment changes who receives the escalation, the learned relationship no longer holds and performance degrades for reasons that output monitoring will not reveal. Any variable recording a clinical response to the condition behaves this way, which is why feature lists need auditing for it explicitly.

No. Adjustment is only valid for a set of variables identified from an assumed causal structure, and that structure has to be stated rather than inferred from the data. VanderWeele and Shpitser, in the Annals of Statistics, work through what it takes for a variable to be a confounder; Greenland, Robins and Pearl, in Statistical Science, show why an association measure can change on adjustment without confounding being removed. Including every available covariate is not conservative: conditioning on a collider creates bias that was not there, and conditioning on a mediator removes part of the effect you wanted. A flexible model with a large feature set does both silently.

It is the practice of specifying the randomised trial you would have run, then emulating it in observational data. Hernán and Robins, in the American Journal of Epidemiology, set out the components: eligibility criteria, the treatment strategies compared, the assignment procedure, time zero, start and end of follow-up, the outcome, the causal contrast and the analysis plan. Use it whenever the question is causal and a trial is unavailable or not yet done. Its main benefit arrives before any analysis: it exposes the fatal problems — no clear time zero, an ill-defined treatment strategy, eligibility determined using post-baseline information, a comparison group that was never treatment-eligible — while they are still cheap to find.

Because validation and transportability are different claims. Degtiar and Rose, in the Annual Review of Statistics and Its Application, separate generalisability — extending from a sample to the population it came from — from transportability, extending to a different population entirely. A new population can differ in prevalence, case mix, measurement practice, treatment patterns and the stability of the associations the model relies on. Methods exist for this: Steingrimsson and colleagues on transporting a prediction model, Dahabreh and colleagues and Westreich and colleagues on extending trial inferences, Colnet and colleagues on sensitivity analysis when target covariates are missing. What is usually absent from papers is any statement of which target population is being claimed.

Substantially through equivalence to existing devices, plus evidence specified for the particular claim. Muehlematter, Bluethgen and Vokinger, in The Lancet Digital Health, found that more than a third of cleared AI and machine-learning-based devices traced to predicate devices that were not themselves AI or machine-learning-based. Wu and colleagues, in Nature Medicine, documented the resulting limitations: evaluations commonly retrospective, test data often from few sites, subgroup performance rarely reported. Zhu and colleagues catalogued the devices and their intended uses. The upshot is that clearance establishes a regulatory status, not a demonstrated improvement in patient outcomes, and a clinical evaluation argument still has to be made on its own merits.

This is an unresolved tension rather than a settled rule. Babic and colleagues, in Science, described the lockdown dilemma: freezing the model guarantees that what was evaluated is what is deployed, but forbids the continuous learning that is the method’s main advantage. Gilbert and colleagues, in the Journal of Medical Internet Research, discuss algorithm change protocols, which pre-specify the permitted changes and how they will be evaluated so that updating does not require a fresh submission each time. Gerke and colleagues, in npj Digital Medicine, argue the regulated object should be the whole system — algorithm plus workflow plus the people using it — rather than the model alone. Expect requirements in this area to keep moving.

Babic, Gerke, Evgeniou and Cohen argue in Science that it should not be treated as a safety requirement, and the reasoning is strong. Post-hoc explanations describe the model’s behaviour, not the world; they can be unstable across methods; and they cannot establish that an exploited association is causal. Treated as assurance, they displace the evidence that would actually warrant trust: external validation, prospective evaluation, subgroup performance and monitoring. Moraffah and colleagues survey causal interpretability as a more demanding alternative framed in terms of interventions, which inherits all the identification problems involved. Report explanation findings as debugging, and make the trust case from validation design.

Fewer of one kind, none fewer of the kind that matters here. Flexible models relax assumptions about functional form — linearity, additivity, specific interaction terms — and that is a genuine advantage. They relax no identification assumptions: exchangeability given the adjustment set, positivity, consistent treatment definition, and correct assumptions about why data are missing, which Rubin addressed in Biometrika. Those come from substantive knowledge of the causal structure, not from the algorithm. The productive role for machine learning in causal analysis is estimating nuisance components within an estimator whose identification is argued separately, which is where most of the current methodological work sits.

References and sources

  1. U.S. Food and Drug Administration, Health Canada, Medicines and Healthcare products Regulatory Agency. Predetermined Change Control Plans for Machine Learning-Enabled Medical Devices: Guiding Principles. U.S. Food and Drug Administration. 2025. read 9 October 2026. fda.gov
  2. Artificial Intelligence and Machine Learning in Software as a Medical Device. U.S. Food and Drug Administration (Digital Health Center of Excellence). 2024. read 9 October 2026. fda.gov
  3. European Medicines Agency, Committee for Medicinal Products for Human Use and Committee for Medicinal Products for Veterinary Use. Reflection paper on the use of Artificial Intelligence (AI) in the medicinal product lifecycle (EMA/CHMP/CVMP/83833/2023). European Medicines Agency. 2024. read 9 October 2026. ema.europa.eu
  4. Lee H, Qiu S, Haupert S, Innes GK, Naumann T, Alemayehu D, van der Laan M. Methodological and regulatory considerations for causal AI in drug development. npj Digital Medicine. 2026;9(1):295. doi:10.1038/s41746-026-02477-w read 9 October 2026. doi.org/10.1038/s41746-026-02477-w
  5. Teodoro D, Naderi N, Yazdani A, Zhang B, et al. , Bornet A, et al. A scoping review of artificial intelligence applications in clinical trial risk assessment. npj Digital Medicine. 2025;8(1):486. doi:10.1038/s41746-025-01886-7 read 9 October 2026. doi.org/10.1038/s41746-025-01886-7
  6. Rademakers FE, Biasin E, Bruining N, Caiani EG, Davies RH, Gilbert SH, Kamenjasevic E, McGauran G, O'Connor G, Rouffet JB, Vasey B, et al. , Fraser AG, et al. CORE-MD clinical risk score for regulatory evaluation of artificial intelligence-based medical device software. npj Digital Medicine. 2025;8(1):90. doi:10.1038/s41746-025-01459-8 read 9 October 2026. doi.org/10.1038/s41746-025-01459-8
  7. Zhang J, Li H, Zhang Y, Huang J, Ren L, Zhang C, Zou Q, Zhang Y. Computational toxicology in drug discovery: applications of artificial intelligence in ADMET and toxicity prediction. Briefings in Bioinformatics. 2025;26(5):bbaf533. doi:10.1093/bib/bbaf533 read 9 October 2026. doi.org/10.1093/bib/bbaf533
  8. Gattrell WT, Logullo P, van Zuuren EJ, Price A, Hughes EL, Blazey P, Winchester CC, Tovey D, Goldman K, Hungin AP, Harrison N. ACCORD (ACcurate COnsensus Reporting Document): a reporting guideline for consensus methods in biomedicine developed via a modified Delphi. PLOS Medicine. 2024;21:e1004326. doi:10.1371/journal.pmed.1004326 read 9 October 2026. doi.org/10.1371/journal.pmed.1004326
  9. Johnson D, Del Fiol G, Kawamoto K, Romagnoli KM, Sanders N, Isaacson G, Jenkins E, Williams MS. Genetically guided precision medicine clinical decision support tools: a systematic review. Journal of the American Medical Informatics Association. 2024;31(5):1183-1194. doi:10.1093/jamia/ocae033 read 9 October 2026. doi.org/10.1093/jamia/ocae033
  10. Li YH, Li YL, Wei MY, Li GY, et al, et al. Innovation and challenges of artificial intelligence technology in personalized healthcare. Scientific Reports. 2024;14(1):18994. doi:10.1038/s41598-024-70073-7 read 9 October 2026. doi.org/10.1038/s41598-024-70073-7
  11. Becker R, Chokoshvili D, Dove ES. Legal bases for effective secondary use of health and genetic data in the EU: time for new legislative solutions to better harmonize data for cross-border sharing? International Data Privacy Law. 2024;14(3):223-246. doi:10.1093/idpl/ipae014 read 9 October 2026. doi.org/10.1093/idpl/ipae014
  12. European Parliament and Council of the European Union. Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act). Official Journal of the European Union (EUR-Lex, CELEX 32024R1689). 2024. read 9 October 2026. eur-lex.europa.eu
  13. Liu Y, Yu W, Dillon T. Regulatory responses and approval status of artificial intelligence medical devices with a focus on China. npj Digital Medicine. 2024;7(1):255. doi:10.1038/s41746-024-01254-x read 9 October 2026. doi.org/10.1038/s41746-024-01254-x
  14. Collins GS, Moons KGM, Dhiman P, Riley RD, Beam AL, Van Calster B, Ghassemi M, Liu X, Reitsma JB, van Smeden M, Boulesteix AL, Camaradou JC, Celi LA, Denaxas S, Denniston AK, Glocker B, Golub RM, Harvey H, Heinze G, Hoffman MM, Kengne AP, Lam E, Lee N, Loder EW, Maier-Hein L, Mateen BA, McCradden MD, Oakden-Rayner L, Ordish J, Parnell R, Rose S, Singh K, Wynants L, Logullo P. TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods. BMJ. 2024;385:e078378. doi:10.1136/bmj-2023-078378 read 9 October 2026. doi.org/10.1136/bmj-2023-078378
  15. Van Spall HGC, Bastien A, Gersh B, Greenberg B, Mohebi R, Min J, Strauss K, Thirstrup S, Zannad F. The role of early-phase trials and real-world evidence in drug development. Nature Cardiovascular Research. 2024;3(2):110-117. doi:10.1038/s44161-024-00420-4 read 9 October 2026. doi.org/10.1038/s44161-024-00420-4
  16. Gerdes A. The role of explainability in AI-supported medical decision-making. Discover Artificial Intelligence. 2024;4(1):29. doi:10.1007/s44163-024-00119-2 read 9 October 2026. doi.org/10.1007/s44163-024-00119-2
  17. Degtiar I, Rose S. A review of generalizability and transportability. Annual Review of Statistics and Its Application. 2023;10:501-24. read 9 October 2026.
  18. Muehlematter UJ, Bluethgen C, Vokinger KN. FDA-cleared artificial intelligence and machine learning-based medical devices and their 510(k) predicate networks. The Lancet Digital Health. 2023;5(9):e618-e626. doi:10.1016/S2589-7500(23)00126-7 read 9 October 2026. doi.org/10.1016/S2589-7500(23)00126-7
  19. Chou WC, Lin Z. Machine learning and artificial intelligence in physiologically based pharmacokinetic modeling. Toxicological Sciences. 2023;191(1):1-14. doi:10.1093/toxsci/kfac101 read 9 October 2026. doi.org/10.1093/toxsci/kfac101
  20. Arora A, Alderman JE, Palmer J, Ganapathi S, Laws E, McCradden MD, Oakden-Rayner L, Pfohl SR, Ghassemi M, McKay F, et al. , Liu X, et al. The value of standards for health datasets in artificial intelligence-based applications. Nature Medicine. 2023;29(11):2929-2938. doi:10.1038/s41591-023-02608-w read 9 October 2026. doi.org/10.1038/s41591-023-02608-w
  21. Steingrimsson JA, Gatsonis C, Li B, Dahabreh IJ. Transporting a prediction model for use in a new target population. American Journal of Epidemiology. 2023;192:296-304. read 9 October 2026. View via DOI
  22. Bellamy D, Hernan MA, Beam A. A structural characterization of shortcut features for prediction. European Journal of Epidemiology. 2022;37:563-8. read 9 October 2026. View via DOI
  23. Colnet B, Josse J, Varoquaux G, Scornet E. Causal effect on a target population: a sensitivity analysis to handle missing covariates. Journal of Causal Inference. 2022;10:372-414. read 9 October 2026.
  24. Lin Z, Chou WC. Machine learning and artificial intelligence in toxicological sciences. Toxicological Sciences. 2022;189(1):7-19. doi:10.1093/toxsci/kfac075 read 9 October 2026. doi.org/10.1093/toxsci/kfac075
  25. Zhu S, Gilbert M, Chetty I, Siddiqui F. The 2021 landscape of FDA-approved artificial intelligence/machine learning-enabled medical devices: an analysis of the characteristics and intended use. International Journal of Medical Informatics. 2022;165:104828. read 13 November 2026. doi.org/10.1016/j.ijmedinf.2022.104828
  26. Lin L, Sperrin M, Jenkins DA, Martin GP, Peek N. A scoping review of causal methods enabling predictions under hypothetical interventions. Diagnostic and Prognostic Research. 2021;5:3. read 9 October 2026. View via DOI
  27. Gilbert S, Fenech M, Hirsch M, Upadhyay S, Biasiucci A, Starlinger J. Algorithm change protocols in the regulation of adaptive machine learning-based medical devices. Journal of Medical Internet Research. 2021;23(10):e30545. read 9 October 2026. View via DOI
  28. Babic B, Gerke S, Evgeniou T, Cohen IG. Beware explanations from AI in health care. Science. 2021;373:284-6. doi:10.1126/science.abg1834 read 9 October 2026. doi.org/10.1126/science.abg1834
  29. Sounderajah V, Ashrafian H, Golub RM, Shetty S, De Fauw J, Hooft L, Moons KGM, Collins GS, Moher D, Bossuyt PM, Darzi A. Developing a reporting guideline for artificial intelligence-centred diagnostic test accuracy studies: the STARD-AI protocol. BMJ Open. 2021;11:e047709. doi:10.1136/bmjopen-2020-047709 read 9 October 2026. doi.org/10.1136/bmjopen-2020-047709
  30. Kyono T, van der Schaar M. Exploiting causal structure for robust model selection in unsupervised domain adaptation. IEEE Transactions on Artificial Intelligence. 2021;2:494-507. read 9 October 2026.
  31. Wu E, Wu K, Daneshjou R, Ouyang D, Ho DE, Zou J. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine. 2021;27(4):582-4. read 9 October 2026. View via DOI
  32. Collins GS, Dhiman P, Andaur Navarro CL, Ma J, Hooft L, Reitsma JB, Logullo P, Beam AL, Peng L, Van Calster B, van Smeden M, Riley RD, Moons KGM. Protocol for development of a reporting guideline (TRIPOD-AI) and risk of bias tool (PROBAST-AI) for diagnostic and prognostic prediction model studies based on artificial intelligence. BMJ Open. 2021;11:e048008. doi:10.1136/bmjopen-2020-048008 read 9 October 2026. doi.org/10.1136/bmjopen-2020-048008
  33. Larson DB, Harvey H, Rubin DL, Irani N, Tse JR, Langlotz CP. Regulatory frameworks for development and evaluation of artificial intelligence-based diagnostic imaging algorithms: summary and recommendations. Journal of the American College of Radiology. 2021;18(3 Pt A):413-24. read 9 October 2026. View via DOI
  34. Moraffah R, Karami M, Guo R, Raglin A, Liu H. Causal interpretability for machine learning: problems, methods and evaluation. ACM SIGKDD Explorations Newsletter. 2020;22(1):18-33. read 9 October 2026.
  35. Piccininni M, Konigorski S, Rohmann JL, Kurth T. Directed acyclic graphs and causal thinking in clinical risk prediction modeling. BMC Medical Research Methodology. 2020;20:179. read 9 October 2026. View via DOI
  36. Dahabreh IJ, Robertson SE, Steingrimsson JA, Stuart EA, Hernan MA. Extending inferences from a randomized trial to a new target population. Statistics in Medicine. 2020;39:1999-2014. read 9 October 2026. View via DOI
  37. Subbaswamy A, Saria S. From development to deployment: dataset shift, causality, and shift-stable models in health AI. Biostatistics. 2020;21:345-52. read 9 October 2026. View via DOI
  38. Raphael MJ, Gyawali B, Booth CM. Real-world evidence and regulatory drug approval. Nature Reviews Clinical Oncology. 2020;17(5):271-272. doi:10.1038/s41571-020-0345-7 read 9 October 2026. doi.org/10.1038/s41571-020-0345-7
  39. Gerke S, Babic B, Evgeniou T, Cohen IG. The need for a system view to regulate artificial intelligence/machine learning-based software as medical device. npj Digital Medicine. 2020;3:53. read 9 October 2026. View via DOI
  40. Benjamens S, Dhunnoo P, Mesko B. The state of artificial intelligence-based FDA-approved medical devices and algorithms: an online database. npj Digital Medicine. 2020;3(1):118. doi:10.1038/s41746-020-00324-0 read 9 October 2026. doi.org/10.1038/s41746-020-00324-0
  41. Liu X, Faes L, Kale AU, Wagner SK, Fu DJ, Bruynseels A, Mahendiran T, Moraes G, Shamdas M, Kern C, Ledsam JR, Schmid MK, Balaskas K, Topol EJ, Bachmann LM, Keane PA, Denniston AK. A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis. The Lancet Digital Health. 2019;1(6):e271-e297. doi:10.1016/S2589-7500(19)30123-2 read 9 October 2026. doi.org/10.1016/S2589-7500(19)30123-2
  42. Babic B, Gerke S, Evgeniou T, Cohen IG. Algorithms on regulatory lockdown in medicine. Science. 2019;366(6470):1202-4. read 9 October 2026. View via DOI
  43. Shah P, Kendall F, Khozin S, Goosen R, Hu J, Laramie J, Ringel M, Schork N. Artificial intelligence and machine learning in clinical development: a translational perspective. npj Digital Medicine. 2019;2(1):69. doi:10.1038/s41746-019-0148-3 read 9 October 2026. doi.org/10.1038/s41746-019-0148-3
  44. Topol EJ. High-performance medicine: the convergence of human and artificial intelligence. Nature Medicine. 2019;25(1):44-56. doi:10.1038/s41591-018-0300-7 read 9 October 2026. doi.org/10.1038/s41591-018-0300-7
  45. Bourassa Forcier M, Gallois H, Mullan S, Joly Y. Integrating artificial intelligence into health care through data access: can the GDPR act as a beacon for policymakers? Journal of Law and the Biosciences. 2019;6(1):317-335. doi:10.1093/jlb/lsz013 read 9 October 2026. doi.org/10.1093/jlb/lsz013
  46. Kelly CJ, Karthikesalingam A, Suleyman M, Corrado G, King D. Key challenges for delivering clinical impact with artificial intelligence. BMC Medicine. 2019. doi:10.1186/s12916-019-1426-2 read 9 October 2026. doi.org/10.1186/s12916-019-1426-2
  47. Glymour C, Zhang K, Spirtes P. Review of causal discovery methods based on graphical models. Frontiers in Genetics. 2019;10:524. read 9 October 2026. View via DOI
  48. Ioannidis JPA, Khoury MJ. Evidence-based medicine and big genomic data. Human Molecular Genetics. 2018;27(R1):R2-R7. doi:10.1093/hmg/ddy065 read 9 October 2026. doi.org/10.1093/hmg/ddy065
  49. Ballantyne A. Where is the human in the data? A guide to ethical data use. GigaScience. 2018;7(7):giy076. doi:10.1093/gigascience/giy076 read 9 October 2026. doi.org/10.1093/gigascience/giy076
  50. Burton PR, Banner N, Elliot MJ, Knoppers BM, Banks J. Policies and strategies to facilitate secondary use of research data in the health sciences. International Journal of Epidemiology. 2017;46(6):1729-1733. doi:10.1093/ije/dyx195 read 9 October 2026. doi.org/10.1093/ije/dyx195
  51. Rudolph KE, van der Laan MJ. Robust estimation of encouragement design intervention effects transported across sites. Journal of the Royal Statistical Society Series B: Statistical Methodology. 2017;79:1509-25. read 9 October 2026. View via DOI
  52. Westreich D, Edwards JK, Lesko CR, Stuart E, Cole SR. Transportability of trial results using inverse odds of sampling weights. American Journal of Epidemiology. 2017;186:1010-4. read 9 October 2026. View via DOI
  53. Bareinboim E, Pearl J. Causal inference and the data-fusion problem. Proceedings of the National Academy of Sciences. 2016;113(27):7345-52. read 9 October 2026. View via DOI
  54. Hernan MA, Robins JM. Using big data to emulate a target trial when a randomized trial is not available. American Journal of Epidemiology. 2016;183:758-64. read 9 October 2026. View via DOI
  55. Dahabreh IJ, Hayward R, Kent DM. Using group data to treat individuals: understanding heterogeneous treatment effects in the age of precision medicine and patient-centred evidence. International Journal of Epidemiology. 2016;45(6):2184-2193. doi:10.1093/ije/dyw125 read 9 October 2026. doi.org/10.1093/ije/dyw125
  56. VanderWeele TJ, Shpitser I. On the definition of a confounder. Annals of Statistics. 2013;41(1):196-220. read 9 October 2026. View via DOI
  57. Pearl J. Causal inference in statistics: an overview. Statistics Surveys. 2009;3:96-146. read 9 October 2026.
  58. Greenland S, Morgenstern H. Confounding in health research. Annual Review of Public Health. 2001;22(1):189-212. read 9 October 2026. View via DOI
  59. Justice AC, Covinsky KE, Berlin JA. Assessing the generalizability of prognostic information. Annals of Internal Medicine. 1999;130:515-24. read 9 October 2026. View via DOI
  60. Greenland S, Robins JM, Pearl J. Confounding and collapsibility in causal inference. Statistical Science. 1999:29-46. read 9 October 2026.
  61. Rubin DB. Inference and missing data. Biometrika. 1976;63(3):581-92. read 9 October 2026.
  62. Clark P, Kim J, Aphinyanaphongs Y. Marketing and US Food and Drug Administration clearance of artificial intelligence and machine learning enabled software in and as medical devices: a systematic review. JAMA Network Open. 2023;6(7):e2321792. read 13 November 2026. doi.org/10.1001/jamanetworkopen.2023.21792
  63. Lee JH, Choi B, Jeong K, Suh SW, Kim JH, Son DS. Co-lifecycle governance for learning medical AI: a hybrid convergence framework for adaptive regulatory oversight. Journal of Medical Internet Research. 2026;28:e90654. read 13 November 2026. doi.org/10.2196/90654

Every source above is a peer-reviewed article, a publisher’s record of one, or a standard issued by the body that owns it. Each carries the date it was last read. Where a standard moves, this article is revised rather than left up with a note.

How to cite this article

Pick your referencing style and copy it. Your university or target journal may vary the punctuation — check its guide if it has one.

Pubrica Academy. (2026, October 9). Why Prediction Is Not Causation in Machine Learning for Health Care. Pubrica. https://pubrica.com/academy/research/an-overview-of-regulatory-affairs-causal-inference-safe-and-effective-health-care-in-machine-learningfor-bio-statistical-services/

This article is written to be learned from. Cite it if you are referring to what it says; if you are using the evidence behind it, cite the sources listed above instead.

Claim-and-design audit for a clinical machine learning paper

Complete this before writing the abstract. The purpose is to make the claim in the abstract no larger than the design can carry — which is the single objection methods reviewers in this field raise most often, and the one that cannot be answered in revision.

CLAIM-AND-DESIGN AUDIT
Model / manuscript: ______________________________   Date: __________

PART A -- WHICH QUESTION
   The model answers (circle ONE):
     (1) prediction under current care
     (2) prediction under a hypothetical intervention
     (3) a causal effect of an intervention
   Stated in the methods in those terms?                   yes / no
   If (2) or (3): causal structure stated?                  yes / no
   If (2) or (3): identification assumptions listed?        yes / no

PART B -- FEATURE AUDIT
   For each predictor, mark T if it could record a clinical RESPONSE
   to the suspected condition rather than a sign of it.
     ____________________________  T / -
     ____________________________  T / -
     ____________________________  T / -
     ____________________________  T / -
   Any T marked?  -> then EITHER remove it,
                    OR add this sentence to the discussion:
     "Because [predictor] records a clinical response to suspected
      [condition], this model is prognostic under current care and must
      not be used as a trigger for [action]."
   Would each remaining predictor keep its relationship with the
   outcome at another site, in another year, under different practice?
     Predictors expected to be stable:   ______________________________
     Predictors expected to be unstable: ______________________________

PART C -- ADJUSTMENT AND MISSINGNESS       (skip if purely predictive)
   [ ] Causal diagram drawn and included or described
   [ ] Adjustment set justified from that diagram, not from the data
   [ ] No collider in the adjustment set
   [ ] Mediators handled deliberately, with the estimand stated
   [ ] Missing-data mechanism stated, not just the imputation method
   [ ] Informative missingness (test not ordered) considered explicitly

PART D -- TARGET TRIAL              (complete only for question 2 or 3)
   Eligibility criteria: _____________________________________________
   Treatment strategies compared: ____________________________________
   Assignment procedure emulated how: ________________________________
   Time zero: ________________________________________________________
   Follow-up start / end: ____________________________________________
   Outcome: __________________________________________________________
   Causal contrast: __________________________________________________
   [ ] No eligibility criterion uses post-baseline information
   [ ] Comparison group members were all treatment-eligible at time zero

PART E -- VALIDATION AND TRANSPORT
   Internal validation design: _______________________________________
   External validation: same system / different system / different
                        country / different case mix / none
   Target population the claim is made for: __________________________
   What makes the transport valid: ___________________________________
   [ ] Discrimination with confidence intervals
   [ ] Calibration slope and intercept, with plot
   [ ] Performance by relevant subgroup

PART F -- CLAIM CHECK
   Abstract conclusion, as currently written:
     _________________________________________________________________
   Does it contain a recommendation to ACT?                 yes / no
   If yes: was any outcome under that action measured?      yes / no
   If no:  rewrite as -> "Whether acting on this model changes
           [outcome] is untested and would require [design]."
   [ ] Intended use stated, and restricted where necessary
   [ ] No comparator performance imported from another study
   [ ] Explanation / interpretability reported as debugging only

PART G -- REGULATORY               (complete if a submission is in view)
   Intended use statement drafted?                          yes / no
   Model locked, or updating?                        locked / updating
   If updating: permitted changes pre-specified?            yes / no
   Evidence type: retrospective / prospective / randomised
   Subgroup performance available for the deployed population? yes / no

Pubrica AcademyWritten and maintained by the subject matter experts of Pubrica who do this work. Articles are revised when the standard underneath them moves, and each one carries the date it was last checked.

How Pubrica helps

Pubrica’s biostatistics group works on the part of this that cannot be fixed in editing: whether the estimand matches the question, whether the adjustment set follows from a stated structure, whether a target trial protocol can be specified at all, and what the transportability claim actually is. Our regulatory writers then build the clinical evaluation argument from that, rather than from a performance table.

Find something else

Search the rest of the Academy.

Open with no account. Type what you are working on, or open the full library to filter by category, sub-topic or resource type.

Send us the protocol or the manuscript, and we will tell you whether the claim and the design agree.

A scoping read costs nothing and takes two business days. You get a written view of what the work needs and who would hold it.

Get a free quote

Regulations

Check your project against this rule

  1. 1Your work
  2. 2Your details
  3. 3Send
What do you need?

Material you send is held under signed confidentiality, with access limited to the assigned team and encrypted transfer. It is not used to train anything and is not passed to third-party tools. Ask for the agreement before you send anything you would rather cover first.