Contents
On this page
- Regulatory affairs for machine learning in health care: the FDA’s approach
- Three questions, routinely confused
- How a well-predicting model becomes a bad decision rule
- Shortcut features are a causal problem, not a data-quality one
- What “we adjusted for confounders” has to mean
- Asking a causal question of observational data: the target trial
- Generalisability and transportability are different problems
- How regulators actually approach machine-learning tools
- Explainability does not substitute for evidence
- Writing it up so the claim matches the design
- Take-home message
- How to apply this to your study
- Common questions
Academy · Research Data Analytics & AI · Causal inference and regulation
Why Prediction Is Not Causation in Machine Learning for Health Care
A model can predict an outcome extremely well and still be useless, or actively harmful, as the basis for a decision. The two tasks answer different questions — who will experience this outcome under current care, versus what will happen if we act — and they require different data, different designs and different evidence. Regulators are increasingly explicit about the distinction; most published models are not.
The sentence that should stop a reviewer is “patients identified as high risk by the model should receive the intervention.” Nothing in a predictive model licenses that step. The model was fitted to data generated under existing practice, which means the clinicians’ own decisions are part of what it learned, and a feature can predict an outcome precisely because it already triggers treatment. This page sets out why the two questions come apart, what it takes to answer the causal one, and how regulatory frameworks for machine-learning tools treat the difference.
What you will learn
- The three distinct questions a clinical model can be asked, and which design each one needs
- Why a feature can predict an outcome because it triggers treatment — and what that does to a deployed model
- What a shortcut feature is in structural terms, and why it is a causal problem rather than a data-cleaning one
- How target trial emulation turns an observational dataset into an answerable causal question
- The difference between generalisability and transportability, and why a validated model can still fail in a new population
- How regulators approach machine-learning-based devices, including the evidence base behind clearances and the problem of models that update
- How to write a prediction-model paper so the causal claim matches the design
What to unlearn
- “If the model predicts well, it will support decisions well.” Predictive accuracy under current care says nothing about what happens under a different care pattern. A model deployed to change practice invalidates the data it was fitted to.
- “We adjusted for confounders, so the estimate is causal.” Adjustment is only valid for a set identified from an assumed causal structure. Putting every available variable into a model can introduce bias rather than remove it, including through colliders and mediators.
- “Machine learning avoids the assumptions that regression needs.” It avoids some functional-form assumptions and none of the identification assumptions. No algorithm substitutes for knowing what adjusts away confounding.
- “An explanation tells us the model is safe.” Post-hoc explanation describes the model’s behaviour, not the world. It cannot establish that an association is causal, and it is not a substitute for prospective evidence.
- “Regulatory clearance means the tool was shown to improve outcomes.” Clearance pathways turn substantially on equivalence and on the adequacy of specified evidence, which is frequently retrospective. Clearance is not an outcome claim.
Machine learning, regulation and causation at a glance
- Subject of the original article
- Regulatory affairs for machine-learning tools in health care, and the causal inference needed to show they are safe and effective
- Main US route to market
- 85.9% of 64 AI/ML devices in one analysis, and 95.0% of 343 in another, were cleared through the 510(k) pathway4025
- Core distinction
- Prediction under current care, prediction under intervention and causal effect need different designs2657
- Commonest error
- Recommending an action on the strength of predictive accuracy alone
- Reading time
- About 20 minutes, with 4 new figures and 3 new tables
Regulatory affairs for machine learning in health care: the FDA’s approach
The title of this page refers to regulatory affairs for machine learning in health care: how tools that use machine learning to support diagnosis or treatment are authorised, and what evidence regulators ask for. In the United States most such tools reach the market as medical devices. Benjamens, Dhunnoo and Meskó cross-checked FDA approvals and identified 64 AI or machine-learning-based devices, of which 85.9% were cleared through the 510(k) pathway, 12.5% through the de novo pathway and one through premarket approval; only 45% of them mentioned AI or machine learning in the official FDA announcement40. Zhu and colleagues reviewed the approval letters of 343 AI/ML-enabled devices authorised by June 2021 and found that 95.0% were cleared under 510(k), that 69.4% were software as a medical device, and that 70.3% were reviewed by the radiology panel25.
Two further themes run through the regulatory literature. The first is the gap between what is cleared and what is claimed: Clark, Kim and Aphinyanaphongs compared 119 510(k) summaries with the devices’ marketing materials and classed 15 devices as discrepant and 8 as contentious, concluding that discrepancies were present in about one-fifth of the devices surveyed62. The second is the model that keeps learning. Lee and colleagues describe the United States as advancing mechanisms for predictable adaptation, including predetermined change control plans, while the European Union and South Korea have taken different routes63; the FDA, Health Canada and the MHRA have published guiding principles for such plans1.
Three regulatory orientations to learning medical AI, as described by Lee et al. (2026)
| Jurisdiction | Orientation | Mechanisms named |
|---|---|---|
| United States | Predictable adaptation | Predetermined change control plans, real-world evidence frameworks and lifecycle-oriented quality management reforms |
| European Union | Precautionary, rights-based governance | The Artificial Intelligence Act and modernised liability rules |
| South Korea | Statutory framework for learning medical AI | The Digital Medical Products Act, described as one of the first comprehensive statutory frameworks of its kind |
Summarised from the abstract of Lee et al.63, who propose four pillars for oversight: continuous validation, agile change management, proactive performance surveillance and distributed accountability.
Why the rest of this article focuses on prediction versus causation
Each regulatory question above (whether a device performs as specified in its intended population, whether an update is safe, whether marketing claims match what was cleared) turns on what a model’s output licenses a clinician to do. That is a causal question, and the commonest weakness in published clinical models is to answer it with predictive evidence. The rest of this article therefore concentrates on the difference between prediction and causation, and returns to regulators in a later section.
Three questions, routinely confused
Almost every argument about machine learning in health care becomes clearer once you separate three questions that a model might be asked.
The first is prediction under current care: given what I know now, who is likely to experience this outcome if nothing changes? This is what a conventional predictive model estimates, and it is a legitimate and useful quantity — for resource planning, for triage, for prognostic information.
The second is prediction under a hypothetical intervention: what is this patient’s risk if we treat, and what is it if we do not? Lin and colleagues, in Diagnostic and Prognostic Research, carried out a scoping review of the causal methods that make this kind of prediction possible, and the point of their review is that it requires methods a standard predictive pipeline does not contain 26.
The third is a causal effect: does this intervention change the outcome, and by how much, in this population? Pearl’s overview in Statistics Surveys gives the formal machinery for why these are different objects and why no amount of data on the first identifies the third 57.
The confusion is not academic. A model answering the first question is routinely deployed as if it answered the second, which is the single most consequential methodological error in clinical machine learning. Piccininni and colleagues, in BMC Medical Research Methodology, show how directed acyclic graphs make the distinction visible in clinical risk prediction modelling specifically — drawing the graph is often enough to reveal that the intended use requires a quantity the model does not estimate 35.
Three questions a clinical model can be asked, and what each one requires
| Question | What is estimated | What the design must provide | The error that follows from confusing it |
|---|---|---|---|
| Prediction under current care who will have this outcome? | Risk given observed features, in a population managed as the training population was | Representative sampling, a stated prediction time point, calibration in the target population | Deployed as a treatment trigger. The model flags patients already being treated and misses those not yet acted on. |
| Prediction under a hypothetical intervention what is the risk if we treat, and if we do not? | Risk under each of two or more specified treatment strategies | An identified causal structure, treatment strategies defined well enough to be assigned, methods for predictions under intervention | Reported as if it were ordinary prediction, hiding the assumptions the estimate depends on. |
| Causal effect does acting change the outcome? | Contrast between outcomes under treatment and under the comparator in a defined population | Randomisation, or an emulated target trial with a stated protocol and no post-baseline eligibility | Inferred from predictive performance, which identifies no causal quantity at all. |
Framing follows Pearl in Statistics Surveys on the formal distinction, Lin and colleagues in Diagnostic and Prognostic Research on prediction under hypothetical interventions, Piccininni and colleagues in BMC Medical Research Methodology on causal thinking in clinical risk prediction, and Hernan and Robins in the American Journal of Epidemiology on target trial emulation. No performance figures are implied.
How a well-predicting model becomes a bad decision rule
Consider a model that predicts in-hospital deterioration and performs excellently. Among its strongest features is whether the patient is receiving a particular escalation of care. The feature predicts deterioration well because clinicians give it to patients they judge to be deteriorating. The model has learned the clinicians’ judgement, encoded as a treatment variable, and it is an excellent predictor precisely because of that.
Now deploy it as a decision rule. The model flags patients already receiving the escalation — the ones who least need flagging — and misses the patients whose clinicians have not yet acted, who are the entire reason for building it. If the deployment then changes who receives the escalation, the relationship the model learned no longer holds, and performance degrades in a way that monitoring the output alone will not explain.
This is not a contrived example; it is the structure of a whole class of failures, and it generalises. Any variable that records a clinical response to the outcome will predict the outcome, and will invert or collapse the moment the model changes practice. The implication for authors is specific: when you list candidate features, flag every one that could be a response to the condition rather than a sign of it, and either exclude it or state that the model is prognostic under current care only and must not be read as a treatment trigger. The implication for readers is to look for that sentence and distrust the paper when it is absent.
Shortcut features are a causal problem, not a data-quality one
Bellamy, Hernán and Beam, in the European Journal of Epidemiology, give a structural characterisation of shortcut features: a shortcut is a feature associated with the outcome through a path that is specific to the dataset rather than stable across settings 22. Characterising it this way matters because it tells you what will and will not fix the problem. More data from the same source will not; a larger model will not; regularisation will not. What helps is identifying, from the assumed causal structure, which associations should be stable and which are artefacts of how this dataset came to exist.
Subbaswamy and Saria, in Biostatistics, take this from diagnosis to construction, showing how causal reasoning about dataset shift supports building models that are deliberately stable across the shifts you anticipate — accepting lower apparent performance in the development data in exchange for performance that survives deployment 37. Kyono and van der Schaar, in IEEE Transactions on Artificial Intelligence, address the related practical problem of selecting between models when you have no labels in the target domain, exploiting causal structure to do it.
The appraisal question that follows is short and rarely asked: which of this model’s inputs would you expect to retain their relationship with the outcome at a different hospital, in a different year, under a different care pattern — and which would not? A paper that cannot answer it has not established that it has a model rather than a description of one dataset.
What “we adjusted for confounders” has to mean
Confounding has a precise definition and the loose usage causes real errors. VanderWeele and Shpitser, in the Annals of Statistics, work through what it takes for a variable to be a confounder, and Greenland, Robins and Pearl, in Statistical Science, set out the relationship between confounding and collapsibility — the reason a measure of association can change on adjustment without any confounding being removed 56,60. Greenland and Morgenstern’s treatment in the Annual Review of Public Health remains the clearest practical statement for health researchers.
The operational consequence is that a valid adjustment set cannot be chosen from the data. It follows from an assumed causal structure, which has to be stated. “We included all available covariates” is not a conservative choice: conditioning on a collider induces association where none existed, and conditioning on a mediator removes part of the effect you were trying to estimate. A high-capacity machine learning model with every variable in the feature set does this enthusiastically and silently.
Missing data sits in the same category. Rubin’s paper in Biometrika established that the handling of missing data depends on assumptions about why it is missing, and in clinical data missingness is very often informative — a test was not ordered because the clinician did not suspect the condition 61. A model that treats missingness as a feature is then learning the clinician’s suspicion, with all the consequences described in the section above. Glymour, Zhang and Spirtes, in Frontiers in Genetics, review causal discovery methods that attempt to learn structure from data rather than assume it; these are useful tools and they do not remove the need to state what you assumed.
Weak adjustment statement
“We adjusted for all available covariates to control for confounding.”
No causal structure is stated, so a reader cannot tell whether colliders or mediators were conditioned on, and the claim that confounding was controlled cannot be checked.
Stronger adjustment statement
“The adjustment set was chosen from the causal diagram in the supplement. Variables measured after treatment initiation were excluded as potential mediators, and laboratory-test missingness was treated as informative rather than as a predictor.”
States where the set came from, why some variables were left out, and how missingness was handled, so each assumption can be challenged.
Illustrative wording written for this article; not quoted from a published paper.
Asking a causal question of observational data: the target trial
Where a randomised trial is not available, the most disciplined available approach is to specify the trial you would have run and then emulate it. Hernán and Robins, in the American Journal of Epidemiology, set out the protocol: write down the eligibility criteria, the treatment strategies being compared, the assignment procedure, the start and end of follow-up, the outcome, the causal contrast of interest and the analysis plan — and only then go to the data 54.
The value of the exercise is that it forces the ambiguities into the open before any estimate exists. Most of the irreparable problems in observational treatment-effect studies are visible at the protocol stage: there is no clear time zero, the treatment strategy is not well defined, eligibility is determined using information from after baseline, or the comparison group is composed of people who were never candidates for the treatment. Each of these is fatal, each is cheap to find on paper, and each is expensive to discover after the analysis.
For machine learning specifically, the target trial framing also clarifies where flexible models genuinely help. They are well suited to estimating the nuisance components — the probability of treatment given covariates, the outcome regression — within an estimator whose identification rests on a stated causal structure. They are not a route to skipping that structure. If you are planning this kind of analysis, the design work belongs with the statistical team from the start: see the biostatistics role hub and our statistical analysis plan service.
Target trial protocol components and the failure each one exposes
| Component | What to specify | Failure it exposes if left vague |
|---|---|---|
| Eligibility criteria | Who could have been enrolled, using information available at baseline only | Eligibility determined from information after baseline |
| Treatment strategies | Each strategy defined well enough to be assigned | A treatment strategy that is not well defined |
| Assignment procedure | How assignment is emulated, and the adjustment set that justifies it | A comparison group of people who were never candidates for treatment |
| Start and end of follow-up | Time zero aligned with eligibility and assignment | No clear time zero |
| Outcome and causal contrast | The outcome and the contrast of interest, stated before analysis | An estimand chosen after seeing the data |
| Analysis plan | Estimator, nuisance models (where flexible machine learning can help) and sensitivity analyses | Flexible models used as a substitute for an identified structure |
Components follow Hernán and Robins54; the failures are those listed in the section above.
Generalisability and transportability are different problems
A model is fitted to a population, not to a disease, and the question of whether it carries to another population has two distinct forms. Degtiar and Rose, in the Annual Review of Statistics and Its Application, separate them: generalisability concerns extending inference from a study sample to the population it was drawn from, transportability concerns extending it to a different population altogether 17. Justice and colleagues, in the Annals of Internal Medicine, framed the same issue for prognostic information decades ago, and the framing has not improved on.
The methodological literature here is mature and under-used by model developers. Steingrimsson and colleagues, in the American Journal of Epidemiology, address transporting a prediction model to a new target population directly 21. Dahabreh and colleagues, in Statistics in Medicine, extend inferences from a randomised trial to a new target population; Westreich and colleagues, in the American Journal of Epidemiology, give the inverse odds of sampling weights approach; Rudolph and van der Laan, in the Journal of the Royal Statistical Society Series B, treat the transport of encouragement-design effects across sites; and Colnet and colleagues, in the Journal of Causal Inference, provide a sensitivity analysis for the common situation where the target population is missing covariates you would need 23,36,51,52. Bareinboim and Pearl, in the Proceedings of the National Academy of Sciences, give the general data-fusion framework in which all of these sit 53.
The practical point for a prediction-model paper is that “externally validated” is not one claim. Validated in a second hospital in the same health system, in a different country, or in a population with materially different prevalence and case mix are three different findings. Say which one you have, describe the target population you are claiming transportability to, and state what you assumed makes the transport valid.
How regulators actually approach machine-learning tools
Two features of the regulatory landscape surprise researchers coming from the methods literature. The first is how much of it rests on equivalence rather than on fresh clinical evidence. Muehlematter, Bluethgen and Vokinger, in The Lancet Digital Health, analysed the predicate networks behind cleared AI and machine-learning-based medical devices and found that more than a third of them originated from predicate devices that were not themselves AI or machine-learning-based. Zhu and colleagues, in the International Journal of Medical Informatics, catalogued the characteristics and intended uses of the devices approved to that point, and Wu and colleagues, in Nature Medicine, examined how medical AI devices are evaluated and documented the limitations: evaluations frequently retrospective, test data often from few sites, and subgroup performance rarely reported 25,31.
The second is the problem of a model that changes. Babic and colleagues, in Science, described the dilemma of algorithms held on regulatory lockdown: freezing a model guarantees that what was evaluated is what is deployed, at the cost of forbidding the updating that is the method’s chief advantage 42. Gilbert and colleagues, in the Journal of Medical Internet Research, discuss algorithm change protocols as the mechanism for pre-specifying permitted changes, and Gerke and colleagues, in npj Digital Medicine, argue for a system-level view in which the regulated object is not the algorithm alone but the algorithm in its clinical workflow 27,39. Larson and colleagues, in the Journal of the American College of Radiology, set out concrete recommendations for imaging algorithms along these lines.
The implication for anyone writing a submission or a clinical evaluation is that the causal and transportability questions above are not academic extras. They are the content of the argument that the device will perform as specified in the population it will be used on. Our regulatory writing and clinical evaluation report pages cover the documents; what belongs in them is decided here.
Regulatory-readiness checklist for a machine-learning tool
- State the intended use and which of the three questions the model answers, in the same words in the paper, the submission and the marketing material.62
- Identify the likely pathway and predicate device, and say where the predicate differs from the proposed device.18
- Describe the evaluation data: retrospective or prospective, number of sites, and performance by subgroup.
- Declare whether the model is locked; if not, pre-specify permitted changes and how each will be evaluated.1
- Name the target population and the argument that performance will transport to it.
- Plan post-market performance monitoring rather than treating clearance as the end of evaluation.63
From an analysis of the 510(k) predicate networks behind cleared AI and machine-learning-based medical devices. Source: Muehlematter UJ, Bluethgen C, Vokinger KN. FDA-cleared artificial intelligence and machine learning-based medical devices and their 510(k) predicate networks. The Lancet Digital Health.
Explainability does not substitute for evidence
Babic, Gerke, Evgeniou and Cohen, in Science, argue against treating explainability as a safety requirement for medical AI, and the argument is worth understanding rather than dismissing 28. Post-hoc explanations describe how a model behaves. They do not establish that the relationships the model exploits are causal, they can be unstable across methods and runs, and they offer a false sense of assurance that displaces the evidence that would actually warrant trust — prospective evaluation, external validation and monitoring.
Moraffah and colleagues, in the ACM SIGKDD Explorations Newsletter, survey causal interpretability as a more demanding alternative: explanations framed in terms of interventions rather than input sensitivity 34. That is the right direction, and it is also harder, because it inherits all the identification problems in the sections above. The honest position for a paper is to report what explanation methods revealed as a debugging finding, state plainly what they cannot establish, and not offer interpretability as a reason the model is fit for use.
Writing it up so the claim matches the design
Reporting guidance for this field has converged quickly. Collins and colleagues’ TRIPOD+AI statement in BMJ is now the reference point for reporting clinical prediction models developed with regression or machine learning, and it is explicit about the items this page has been circling: the source of data and the eligibility criteria, the predictors and when they were measured relative to the outcome, how the model was developed and validated, calibration as well as discrimination, performance in relevant subgroups, and a statement of intended use 14. The earlier protocol by Collins and colleagues in BMJ Open also describes PROBAST-AI, the companion risk-of-bias tool, which is what a methods reviewer will reach for 32.
For diagnostic accuracy studies specifically, Sounderajah and colleagues, in BMJ Open, set out the development of STARD-AI. Trial-stage guidance for interventions involving AI — the SPIRIT-AI, CONSORT-AI and DECIDE-AI family — is covered on our page on AI and machine learning in clinical development, and the general checklist landscape on reporting guidelines 29. Where a guideline itself was produced by a consensus process, Gattrell and colleagues’ ACCORD statement in PLOS Medicine now sets the standard for reporting how that consensus was reached.
What none of these guidelines can do is fix a mismatch between claim and design after the fact. The commonest version of that mismatch is a prediction model whose abstract recommends an action, and it is visible in a single sentence.
A prediction model’s conclusion, written two ways
Claims more than the design supports
Our machine learning model identified patients at high risk of deterioration with excellent discrimination, outperforming the existing clinical score. Patients flagged by the model should be prioritised for early escalation of care, and the model could be integrated into the electronic record to reduce adverse events.
“Should be prioritised” is a causal claim derived from a predictive estimate, which identifies no causal quantity.
“Reduce adverse events” was not measured. No outcome under the proposed intervention exists anywhere in the study.
Treatment-response predictors are unexamined. If the model learned the clinicians’ judgement, deployment inverts the relationship it relies on.
“Outperforming the existing score” compares discrimination, not decisions — and discrimination is prevalence-dependent and threshold-free.
No target population named, so the transportability claim implicit in “integrated into the electronic record” cannot be assessed.
Matches claim to design
The model discriminates between patients who did and did not deteriorate under the care patterns prevailing in the development cohort, with discrimination and calibration reported for the external cohort and by subgroup. Because several predictors record clinical responses to suspected deterioration rather than signs of it, the model is prognostic under current care and must not be used as a trigger for escalation: it cannot estimate risk under an intervention, and we have not evaluated what happens to its predictors once escalation decisions change. Whether acting on the model reduces adverse events is an untested causal question that would require a prospective comparison against current practice.
The estimand is stated: discrimination under the care patterns of the development cohort, not risk under intervention.
The treatment-response problem is named rather than left for a reviewer to find, with the consequence spelled out.
Calibration and subgroup performance appear, which is what TRIPOD+AI asks for and what decisions at a threshold depend on.
The intended use is restricted explicitly, so the paper cannot be cited for a use it does not support.
The causal question is identified as open, with the design that would answer it named.
The second version makes a smaller claim and is far harder to refute, which is why it ages better. More importantly, it is the version that can be used: a reader can tell what the model is for, a regulator can tell what has been evidenced, and a subsequent trial has a stated hypothesis to test. The first version will be cited for a conclusion the study never reached.
Take-home message
Take-home message
Prediction and causation answer different questions and require different designs. A model fitted to data generated under current practice has learned that practice, including the clinicians’ own treatment decisions, so a feature can predict an outcome precisely because it triggers treatment — which makes the model a poor and sometimes harmful decision rule. Valid causal claims need an identified structure, stated assumptions and, where no trial exists, an explicitly emulated target trial. Transportability to a new population is a further claim requiring its own argument. Regulatory frameworks rest substantially on equivalence rather than fresh outcome evidence, so clearance is not a demonstration of benefit. Report under TRIPOD+AI, state the intended use, and make the claim no larger than the design.
How to apply this to your study
How to apply this to your own study
- Write down which of the three questions your model answers — prediction under current care, prediction under intervention, or causal effect — and put the answer in the methods. Most papers never state it, and most confusion follows from that.
- Go through your feature list and flag every treatment-response variable. Anything that records a clinical reaction to the suspected condition will predict it and will invert on deployment. Exclude it, or restrict the intended use in writing.
- Draw the causal diagram before choosing the adjustment set. If you cannot draw it, you cannot justify the adjustment, and “we included all covariates” is a choice with its own bias, not a neutral default.
- If the question is causal and there is no trial, write the target trial protocol first — eligibility, strategies, assignment, time zero, follow-up, outcome, contrast, analysis — and only then open the dataset.
- Name the target population you claim transportability to, and state what makes the transport valid. “Externally validated” in a second hospital of the same system is a much weaker claim than it sounds.
- Report calibration and subgroup performance, not discrimination alone. Decisions happen at thresholds, and TRIPOD+AI treats both as required items.
- Remove every action recommendation from the abstract that the design cannot support. Replace it with the study that would support it.
- Treat explanation output as a debugging finding. Report what it revealed; do not offer it as evidence of fitness for use.
- If a regulatory submission is in view, specify the change-control position early — whether the model is locked, and if not, what changes are pre-specified and how they will be evaluated.
Common questions
Prediction estimates who will experience an outcome given observed features, in a population managed as the training population was. Causal inference estimates what would happen if we acted. They are different mathematical objects, as Pearl sets out in Statistics Surveys, and no quantity of predictive data identifies a causal effect. The practical consequence is that a model can discriminate excellently and still be a poor decision rule, because the features driving the prediction may be consequences of clinical decisions rather than causes of the outcome. Lin and colleagues, in Diagnostic and Prognostic Research, review the methods needed for the intermediate case: prediction under a hypothetical intervention, which requires causal machinery a standard pipeline lacks.
Through a specific and common mechanism. Suppose a strong predictor of deterioration is receipt of an escalation of care — strong precisely because clinicians give it to patients they judge to be deteriorating. The model has learned their judgement. Deployed as a trigger, it flags patients already being treated and misses those whose clinicians have not yet acted, which is the entire use case. Worse, if deployment changes who receives the escalation, the learned relationship no longer holds and performance degrades for reasons that output monitoring will not reveal. Any variable recording a clinical response to the condition behaves this way, which is why feature lists need auditing for it explicitly.
No. Adjustment is only valid for a set of variables identified from an assumed causal structure, and that structure has to be stated rather than inferred from the data. VanderWeele and Shpitser, in the Annals of Statistics, work through what it takes for a variable to be a confounder; Greenland, Robins and Pearl, in Statistical Science, show why an association measure can change on adjustment without confounding being removed. Including every available covariate is not conservative: conditioning on a collider creates bias that was not there, and conditioning on a mediator removes part of the effect you wanted. A flexible model with a large feature set does both silently.
It is the practice of specifying the randomised trial you would have run, then emulating it in observational data. Hernán and Robins, in the American Journal of Epidemiology, set out the components: eligibility criteria, the treatment strategies compared, the assignment procedure, time zero, start and end of follow-up, the outcome, the causal contrast and the analysis plan. Use it whenever the question is causal and a trial is unavailable or not yet done. Its main benefit arrives before any analysis: it exposes the fatal problems — no clear time zero, an ill-defined treatment strategy, eligibility determined using post-baseline information, a comparison group that was never treatment-eligible — while they are still cheap to find.
Because validation and transportability are different claims. Degtiar and Rose, in the Annual Review of Statistics and Its Application, separate generalisability — extending from a sample to the population it came from — from transportability, extending to a different population entirely. A new population can differ in prevalence, case mix, measurement practice, treatment patterns and the stability of the associations the model relies on. Methods exist for this: Steingrimsson and colleagues on transporting a prediction model, Dahabreh and colleagues and Westreich and colleagues on extending trial inferences, Colnet and colleagues on sensitivity analysis when target covariates are missing. What is usually absent from papers is any statement of which target population is being claimed.
Substantially through equivalence to existing devices, plus evidence specified for the particular claim. Muehlematter, Bluethgen and Vokinger, in The Lancet Digital Health, found that more than a third of cleared AI and machine-learning-based devices traced to predicate devices that were not themselves AI or machine-learning-based. Wu and colleagues, in Nature Medicine, documented the resulting limitations: evaluations commonly retrospective, test data often from few sites, subgroup performance rarely reported. Zhu and colleagues catalogued the devices and their intended uses. The upshot is that clearance establishes a regulatory status, not a demonstrated improvement in patient outcomes, and a clinical evaluation argument still has to be made on its own merits.
This is an unresolved tension rather than a settled rule. Babic and colleagues, in Science, described the lockdown dilemma: freezing the model guarantees that what was evaluated is what is deployed, but forbids the continuous learning that is the method’s main advantage. Gilbert and colleagues, in the Journal of Medical Internet Research, discuss algorithm change protocols, which pre-specify the permitted changes and how they will be evaluated so that updating does not require a fresh submission each time. Gerke and colleagues, in npj Digital Medicine, argue the regulated object should be the whole system — algorithm plus workflow plus the people using it — rather than the model alone. Expect requirements in this area to keep moving.
Babic, Gerke, Evgeniou and Cohen argue in Science that it should not be treated as a safety requirement, and the reasoning is strong. Post-hoc explanations describe the model’s behaviour, not the world; they can be unstable across methods; and they cannot establish that an exploited association is causal. Treated as assurance, they displace the evidence that would actually warrant trust: external validation, prospective evaluation, subgroup performance and monitoring. Moraffah and colleagues survey causal interpretability as a more demanding alternative framed in terms of interventions, which inherits all the identification problems involved. Report explanation findings as debugging, and make the trust case from validation design.
Fewer of one kind, none fewer of the kind that matters here. Flexible models relax assumptions about functional form — linearity, additivity, specific interaction terms — and that is a genuine advantage. They relax no identification assumptions: exchangeability given the adjustment set, positivity, consistent treatment definition, and correct assumptions about why data are missing, which Rubin addressed in Biometrika. Those come from substantive knowledge of the causal structure, not from the algorithm. The productive role for machine learning in causal analysis is estimating nuisance components within an estimator whose identification is argued separately, which is where most of the current methodological work sits.
References and sources
- U.S. Food and Drug Administration, Health Canada, Medicines and Healthcare products Regulatory Agency. Predetermined Change Control Plans for Machine Learning-Enabled Medical Devices: Guiding Principles. U.S. Food and Drug Administration. 2025. read 9 October 2026. fda.gov
- Artificial Intelligence and Machine Learning in Software as a Medical Device. U.S. Food and Drug Administration (Digital Health Center of Excellence). 2024. read 9 October 2026. fda.gov
- European Medicines Agency, Committee for Medicinal Products for Human Use and Committee for Medicinal Products for Veterinary Use. Reflection paper on the use of Artificial Intelligence (AI) in the medicinal product lifecycle (EMA/CHMP/CVMP/83833/2023). European Medicines Agency. 2024. read 9 October 2026. ema.europa.eu
- Lee H, Qiu S, Haupert S, Innes GK, Naumann T, Alemayehu D, van der Laan M. Methodological and regulatory considerations for causal AI in drug development. npj Digital Medicine. 2026;9(1):295. doi:10.1038/s41746-026-02477-w read 9 October 2026. doi.org/10.1038/s41746-026-02477-w
- Teodoro D, Naderi N, Yazdani A, Zhang B, et al. , Bornet A, et al. A scoping review of artificial intelligence applications in clinical trial risk assessment. npj Digital Medicine. 2025;8(1):486. doi:10.1038/s41746-025-01886-7 read 9 October 2026. doi.org/10.1038/s41746-025-01886-7
- Rademakers FE, Biasin E, Bruining N, Caiani EG, Davies RH, Gilbert SH, Kamenjasevic E, McGauran G, O'Connor G, Rouffet JB, Vasey B, et al. , Fraser AG, et al. CORE-MD clinical risk score for regulatory evaluation of artificial intelligence-based medical device software. npj Digital Medicine. 2025;8(1):90. doi:10.1038/s41746-025-01459-8 read 9 October 2026. doi.org/10.1038/s41746-025-01459-8
- Zhang J, Li H, Zhang Y, Huang J, Ren L, Zhang C, Zou Q, Zhang Y. Computational toxicology in drug discovery: applications of artificial intelligence in ADMET and toxicity prediction. Briefings in Bioinformatics. 2025;26(5):bbaf533. doi:10.1093/bib/bbaf533 read 9 October 2026. doi.org/10.1093/bib/bbaf533
- Gattrell WT, Logullo P, van Zuuren EJ, Price A, Hughes EL, Blazey P, Winchester CC, Tovey D, Goldman K, Hungin AP, Harrison N. ACCORD (ACcurate COnsensus Reporting Document): a reporting guideline for consensus methods in biomedicine developed via a modified Delphi. PLOS Medicine. 2024;21:e1004326. doi:10.1371/journal.pmed.1004326 read 9 October 2026. doi.org/10.1371/journal.pmed.1004326
- Johnson D, Del Fiol G, Kawamoto K, Romagnoli KM, Sanders N, Isaacson G, Jenkins E, Williams MS. Genetically guided precision medicine clinical decision support tools: a systematic review. Journal of the American Medical Informatics Association. 2024;31(5):1183-1194. doi:10.1093/jamia/ocae033 read 9 October 2026. doi.org/10.1093/jamia/ocae033
- Li YH, Li YL, Wei MY, Li GY, et al, et al. Innovation and challenges of artificial intelligence technology in personalized healthcare. Scientific Reports. 2024;14(1):18994. doi:10.1038/s41598-024-70073-7 read 9 October 2026. doi.org/10.1038/s41598-024-70073-7
- Becker R, Chokoshvili D, Dove ES. Legal bases for effective secondary use of health and genetic data in the EU: time for new legislative solutions to better harmonize data for cross-border sharing? International Data Privacy Law. 2024;14(3):223-246. doi:10.1093/idpl/ipae014 read 9 October 2026. doi.org/10.1093/idpl/ipae014
- European Parliament and Council of the European Union. Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act). Official Journal of the European Union (EUR-Lex, CELEX 32024R1689). 2024. read 9 October 2026. eur-lex.europa.eu
- Liu Y, Yu W, Dillon T. Regulatory responses and approval status of artificial intelligence medical devices with a focus on China. npj Digital Medicine. 2024;7(1):255. doi:10.1038/s41746-024-01254-x read 9 October 2026. doi.org/10.1038/s41746-024-01254-x
- Collins GS, Moons KGM, Dhiman P, Riley RD, Beam AL, Van Calster B, Ghassemi M, Liu X, Reitsma JB, van Smeden M, Boulesteix AL, Camaradou JC, Celi LA, Denaxas S, Denniston AK, Glocker B, Golub RM, Harvey H, Heinze G, Hoffman MM, Kengne AP, Lam E, Lee N, Loder EW, Maier-Hein L, Mateen BA, McCradden MD, Oakden-Rayner L, Ordish J, Parnell R, Rose S, Singh K, Wynants L, Logullo P. TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods. BMJ. 2024;385:e078378. doi:10.1136/bmj-2023-078378 read 9 October 2026. doi.org/10.1136/bmj-2023-078378
- Van Spall HGC, Bastien A, Gersh B, Greenberg B, Mohebi R, Min J, Strauss K, Thirstrup S, Zannad F. The role of early-phase trials and real-world evidence in drug development. Nature Cardiovascular Research. 2024;3(2):110-117. doi:10.1038/s44161-024-00420-4 read 9 October 2026. doi.org/10.1038/s44161-024-00420-4
- Gerdes A. The role of explainability in AI-supported medical decision-making. Discover Artificial Intelligence. 2024;4(1):29. doi:10.1007/s44163-024-00119-2 read 9 October 2026. doi.org/10.1007/s44163-024-00119-2
- Degtiar I, Rose S. A review of generalizability and transportability. Annual Review of Statistics and Its Application. 2023;10:501-24. read 9 October 2026.
- Muehlematter UJ, Bluethgen C, Vokinger KN. FDA-cleared artificial intelligence and machine learning-based medical devices and their 510(k) predicate networks. The Lancet Digital Health. 2023;5(9):e618-e626. doi:10.1016/S2589-7500(23)00126-7 read 9 October 2026. doi.org/10.1016/S2589-7500(23)00126-7
- Chou WC, Lin Z. Machine learning and artificial intelligence in physiologically based pharmacokinetic modeling. Toxicological Sciences. 2023;191(1):1-14. doi:10.1093/toxsci/kfac101 read 9 October 2026. doi.org/10.1093/toxsci/kfac101
- Arora A, Alderman JE, Palmer J, Ganapathi S, Laws E, McCradden MD, Oakden-Rayner L, Pfohl SR, Ghassemi M, McKay F, et al. , Liu X, et al. The value of standards for health datasets in artificial intelligence-based applications. Nature Medicine. 2023;29(11):2929-2938. doi:10.1038/s41591-023-02608-w read 9 October 2026. doi.org/10.1038/s41591-023-02608-w
- Steingrimsson JA, Gatsonis C, Li B, Dahabreh IJ. Transporting a prediction model for use in a new target population. American Journal of Epidemiology. 2023;192:296-304. read 9 October 2026. View via DOI
- Bellamy D, Hernan MA, Beam A. A structural characterization of shortcut features for prediction. European Journal of Epidemiology. 2022;37:563-8. read 9 October 2026. View via DOI
- Colnet B, Josse J, Varoquaux G, Scornet E. Causal effect on a target population: a sensitivity analysis to handle missing covariates. Journal of Causal Inference. 2022;10:372-414. read 9 October 2026.
- Lin Z, Chou WC. Machine learning and artificial intelligence in toxicological sciences. Toxicological Sciences. 2022;189(1):7-19. doi:10.1093/toxsci/kfac075 read 9 October 2026. doi.org/10.1093/toxsci/kfac075
- Zhu S, Gilbert M, Chetty I, Siddiqui F. The 2021 landscape of FDA-approved artificial intelligence/machine learning-enabled medical devices: an analysis of the characteristics and intended use. International Journal of Medical Informatics. 2022;165:104828. read 13 November 2026. doi.org/10.1016/j.ijmedinf.2022.104828
- Lin L, Sperrin M, Jenkins DA, Martin GP, Peek N. A scoping review of causal methods enabling predictions under hypothetical interventions. Diagnostic and Prognostic Research. 2021;5:3. read 9 October 2026. View via DOI
- Gilbert S, Fenech M, Hirsch M, Upadhyay S, Biasiucci A, Starlinger J. Algorithm change protocols in the regulation of adaptive machine learning-based medical devices. Journal of Medical Internet Research. 2021;23(10):e30545. read 9 October 2026. View via DOI
- Babic B, Gerke S, Evgeniou T, Cohen IG. Beware explanations from AI in health care. Science. 2021;373:284-6. doi:10.1126/science.abg1834 read 9 October 2026. doi.org/10.1126/science.abg1834
- Sounderajah V, Ashrafian H, Golub RM, Shetty S, De Fauw J, Hooft L, Moons KGM, Collins GS, Moher D, Bossuyt PM, Darzi A. Developing a reporting guideline for artificial intelligence-centred diagnostic test accuracy studies: the STARD-AI protocol. BMJ Open. 2021;11:e047709. doi:10.1136/bmjopen-2020-047709 read 9 October 2026. doi.org/10.1136/bmjopen-2020-047709
- Kyono T, van der Schaar M. Exploiting causal structure for robust model selection in unsupervised domain adaptation. IEEE Transactions on Artificial Intelligence. 2021;2:494-507. read 9 October 2026.
- Wu E, Wu K, Daneshjou R, Ouyang D, Ho DE, Zou J. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine. 2021;27(4):582-4. read 9 October 2026. View via DOI
- Collins GS, Dhiman P, Andaur Navarro CL, Ma J, Hooft L, Reitsma JB, Logullo P, Beam AL, Peng L, Van Calster B, van Smeden M, Riley RD, Moons KGM. Protocol for development of a reporting guideline (TRIPOD-AI) and risk of bias tool (PROBAST-AI) for diagnostic and prognostic prediction model studies based on artificial intelligence. BMJ Open. 2021;11:e048008. doi:10.1136/bmjopen-2020-048008 read 9 October 2026. doi.org/10.1136/bmjopen-2020-048008
- Larson DB, Harvey H, Rubin DL, Irani N, Tse JR, Langlotz CP. Regulatory frameworks for development and evaluation of artificial intelligence-based diagnostic imaging algorithms: summary and recommendations. Journal of the American College of Radiology. 2021;18(3 Pt A):413-24. read 9 October 2026. View via DOI
- Moraffah R, Karami M, Guo R, Raglin A, Liu H. Causal interpretability for machine learning: problems, methods and evaluation. ACM SIGKDD Explorations Newsletter. 2020;22(1):18-33. read 9 October 2026.
- Piccininni M, Konigorski S, Rohmann JL, Kurth T. Directed acyclic graphs and causal thinking in clinical risk prediction modeling. BMC Medical Research Methodology. 2020;20:179. read 9 October 2026. View via DOI
- Dahabreh IJ, Robertson SE, Steingrimsson JA, Stuart EA, Hernan MA. Extending inferences from a randomized trial to a new target population. Statistics in Medicine. 2020;39:1999-2014. read 9 October 2026. View via DOI
- Subbaswamy A, Saria S. From development to deployment: dataset shift, causality, and shift-stable models in health AI. Biostatistics. 2020;21:345-52. read 9 October 2026. View via DOI
- Raphael MJ, Gyawali B, Booth CM. Real-world evidence and regulatory drug approval. Nature Reviews Clinical Oncology. 2020;17(5):271-272. doi:10.1038/s41571-020-0345-7 read 9 October 2026. doi.org/10.1038/s41571-020-0345-7
- Gerke S, Babic B, Evgeniou T, Cohen IG. The need for a system view to regulate artificial intelligence/machine learning-based software as medical device. npj Digital Medicine. 2020;3:53. read 9 October 2026. View via DOI
- Benjamens S, Dhunnoo P, Mesko B. The state of artificial intelligence-based FDA-approved medical devices and algorithms: an online database. npj Digital Medicine. 2020;3(1):118. doi:10.1038/s41746-020-00324-0 read 9 October 2026. doi.org/10.1038/s41746-020-00324-0
- Liu X, Faes L, Kale AU, Wagner SK, Fu DJ, Bruynseels A, Mahendiran T, Moraes G, Shamdas M, Kern C, Ledsam JR, Schmid MK, Balaskas K, Topol EJ, Bachmann LM, Keane PA, Denniston AK. A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis. The Lancet Digital Health. 2019;1(6):e271-e297. doi:10.1016/S2589-7500(19)30123-2 read 9 October 2026. doi.org/10.1016/S2589-7500(19)30123-2
- Babic B, Gerke S, Evgeniou T, Cohen IG. Algorithms on regulatory lockdown in medicine. Science. 2019;366(6470):1202-4. read 9 October 2026. View via DOI
- Shah P, Kendall F, Khozin S, Goosen R, Hu J, Laramie J, Ringel M, Schork N. Artificial intelligence and machine learning in clinical development: a translational perspective. npj Digital Medicine. 2019;2(1):69. doi:10.1038/s41746-019-0148-3 read 9 October 2026. doi.org/10.1038/s41746-019-0148-3
- Topol EJ. High-performance medicine: the convergence of human and artificial intelligence. Nature Medicine. 2019;25(1):44-56. doi:10.1038/s41591-018-0300-7 read 9 October 2026. doi.org/10.1038/s41591-018-0300-7
- Bourassa Forcier M, Gallois H, Mullan S, Joly Y. Integrating artificial intelligence into health care through data access: can the GDPR act as a beacon for policymakers? Journal of Law and the Biosciences. 2019;6(1):317-335. doi:10.1093/jlb/lsz013 read 9 October 2026. doi.org/10.1093/jlb/lsz013
- Kelly CJ, Karthikesalingam A, Suleyman M, Corrado G, King D. Key challenges for delivering clinical impact with artificial intelligence. BMC Medicine. 2019. doi:10.1186/s12916-019-1426-2 read 9 October 2026. doi.org/10.1186/s12916-019-1426-2
- Glymour C, Zhang K, Spirtes P. Review of causal discovery methods based on graphical models. Frontiers in Genetics. 2019;10:524. read 9 October 2026. View via DOI
- Ioannidis JPA, Khoury MJ. Evidence-based medicine and big genomic data. Human Molecular Genetics. 2018;27(R1):R2-R7. doi:10.1093/hmg/ddy065 read 9 October 2026. doi.org/10.1093/hmg/ddy065
- Ballantyne A. Where is the human in the data? A guide to ethical data use. GigaScience. 2018;7(7):giy076. doi:10.1093/gigascience/giy076 read 9 October 2026. doi.org/10.1093/gigascience/giy076
- Burton PR, Banner N, Elliot MJ, Knoppers BM, Banks J. Policies and strategies to facilitate secondary use of research data in the health sciences. International Journal of Epidemiology. 2017;46(6):1729-1733. doi:10.1093/ije/dyx195 read 9 October 2026. doi.org/10.1093/ije/dyx195
- Rudolph KE, van der Laan MJ. Robust estimation of encouragement design intervention effects transported across sites. Journal of the Royal Statistical Society Series B: Statistical Methodology. 2017;79:1509-25. read 9 October 2026. View via DOI
- Westreich D, Edwards JK, Lesko CR, Stuart E, Cole SR. Transportability of trial results using inverse odds of sampling weights. American Journal of Epidemiology. 2017;186:1010-4. read 9 October 2026. View via DOI
- Bareinboim E, Pearl J. Causal inference and the data-fusion problem. Proceedings of the National Academy of Sciences. 2016;113(27):7345-52. read 9 October 2026. View via DOI
- Hernan MA, Robins JM. Using big data to emulate a target trial when a randomized trial is not available. American Journal of Epidemiology. 2016;183:758-64. read 9 October 2026. View via DOI
- Dahabreh IJ, Hayward R, Kent DM. Using group data to treat individuals: understanding heterogeneous treatment effects in the age of precision medicine and patient-centred evidence. International Journal of Epidemiology. 2016;45(6):2184-2193. doi:10.1093/ije/dyw125 read 9 October 2026. doi.org/10.1093/ije/dyw125
- VanderWeele TJ, Shpitser I. On the definition of a confounder. Annals of Statistics. 2013;41(1):196-220. read 9 October 2026. View via DOI
- Pearl J. Causal inference in statistics: an overview. Statistics Surveys. 2009;3:96-146. read 9 October 2026.
- Greenland S, Morgenstern H. Confounding in health research. Annual Review of Public Health. 2001;22(1):189-212. read 9 October 2026. View via DOI
- Justice AC, Covinsky KE, Berlin JA. Assessing the generalizability of prognostic information. Annals of Internal Medicine. 1999;130:515-24. read 9 October 2026. View via DOI
- Greenland S, Robins JM, Pearl J. Confounding and collapsibility in causal inference. Statistical Science. 1999:29-46. read 9 October 2026.
- Rubin DB. Inference and missing data. Biometrika. 1976;63(3):581-92. read 9 October 2026.
- Clark P, Kim J, Aphinyanaphongs Y. Marketing and US Food and Drug Administration clearance of artificial intelligence and machine learning enabled software in and as medical devices: a systematic review. JAMA Network Open. 2023;6(7):e2321792. read 13 November 2026. doi.org/10.1001/jamanetworkopen.2023.21792
- Lee JH, Choi B, Jeong K, Suh SW, Kim JH, Son DS. Co-lifecycle governance for learning medical AI: a hybrid convergence framework for adaptive regulatory oversight. Journal of Medical Internet Research. 2026;28:e90654. read 13 November 2026. doi.org/10.2196/90654
Every source above is a peer-reviewed article, a publisher’s record of one, or a standard issued by the body that owns it. Each carries the date it was last read. Where a standard moves, this article is revised rather than left up with a note.
How to cite this article
Pick your referencing style and copy it. Your university or target journal may vary the punctuation — check its guide if it has one.
Pubrica Academy. (2026, October 9). Why Prediction Is Not Causation in Machine Learning for Health Care. Pubrica. https://pubrica.com/academy/research/an-overview-of-regulatory-affairs-causal-inference-safe-and-effective-health-care-in-machine-learningfor-bio-statistical-services/
Pubrica Academy. “Why Prediction Is Not Causation in Machine Learning for Health Care.” Pubrica, 9 Oct. 2026, pubrica.com/academy/research/an-overview-of-regulatory-affairs-causal-inference-safe-and-effective-health-care-in-machine-learningfor-bio-statistical-services/.
Pubrica Academy (2026) Why Prediction Is Not Causation in Machine Learning for Health Care. Pubrica. Available at: https://pubrica.com/academy/research/an-overview-of-regulatory-affairs-causal-inference-safe-and-effective-health-care-in-machine-learningfor-bio-statistical-services/ (Accessed: 15 October 2026).
Pubrica Academy. “Why Prediction Is Not Causation in Machine Learning for Health Care.” Pubrica. October 9, 2026. https://pubrica.com/academy/research/an-overview-of-regulatory-affairs-causal-inference-safe-and-effective-health-care-in-machine-learningfor-bio-statistical-services/
Pubrica Academy, ‘Why Prediction Is Not Causation in Machine Learning for Health Care’ (Pubrica, 9 October 2026) <https://pubrica.com/academy/research/an-overview-of-regulatory-affairs-causal-inference-safe-and-effective-health-care-in-machine-learningfor-bio-statistical-services/> accessed 15 October 2026.
Pubrica Academy. Why Prediction Is Not Causation in Machine Learning for Health Care. [Internet]. Pubrica; 2026 Oct 9 [cited 2026 Oct 15]. Available from: https://pubrica.com/academy/research/an-overview-of-regulatory-affairs-causal-inference-safe-and-effective-health-care-in-machine-learningfor-bio-statistical-services/
Pubrica Academy, “Why Prediction Is Not Causation in Machine Learning for Health Care,” Pubrica, Oct. 9, 2026. [Online]. Available: https://pubrica.com/academy/research/an-overview-of-regulatory-affairs-causal-inference-safe-and-effective-health-care-in-machine-learningfor-bio-statistical-services/
@misc{an_overview_of_regulatory_affairs_caus_2026,
author = {{Pubrica Academy}},
title = {Why Prediction Is Not Causation in Machine Learning for Health Care},
howpublished = {Pubrica},
year = {2026},
month = {oct},
url = {https://pubrica.com/academy/research/an-overview-of-regulatory-affairs-causal-inference-safe-and-effective-health-care-in-machine-learningfor-bio-statistical-services/},
note = {Accessed: <span data-cite-acc="iso">2026-10-15</span>}
}This article is written to be learned from. Cite it if you are referring to what it says; if you are using the evidence behind it, cite the sources listed above instead.
Claim-and-design audit for a clinical machine learning paper
Complete this before writing the abstract. The purpose is to make the claim in the abstract no larger than the design can carry — which is the single objection methods reviewers in this field raise most often, and the one that cannot be answered in revision.
CLAIM-AND-DESIGN AUDIT
Model / manuscript: ______________________________ Date: __________
PART A -- WHICH QUESTION
The model answers (circle ONE):
(1) prediction under current care
(2) prediction under a hypothetical intervention
(3) a causal effect of an intervention
Stated in the methods in those terms? yes / no
If (2) or (3): causal structure stated? yes / no
If (2) or (3): identification assumptions listed? yes / no
PART B -- FEATURE AUDIT
For each predictor, mark T if it could record a clinical RESPONSE
to the suspected condition rather than a sign of it.
____________________________ T / -
____________________________ T / -
____________________________ T / -
____________________________ T / -
Any T marked? -> then EITHER remove it,
OR add this sentence to the discussion:
"Because [predictor] records a clinical response to suspected
[condition], this model is prognostic under current care and must
not be used as a trigger for [action]."
Would each remaining predictor keep its relationship with the
outcome at another site, in another year, under different practice?
Predictors expected to be stable: ______________________________
Predictors expected to be unstable: ______________________________
PART C -- ADJUSTMENT AND MISSINGNESS (skip if purely predictive)
[ ] Causal diagram drawn and included or described
[ ] Adjustment set justified from that diagram, not from the data
[ ] No collider in the adjustment set
[ ] Mediators handled deliberately, with the estimand stated
[ ] Missing-data mechanism stated, not just the imputation method
[ ] Informative missingness (test not ordered) considered explicitly
PART D -- TARGET TRIAL (complete only for question 2 or 3)
Eligibility criteria: _____________________________________________
Treatment strategies compared: ____________________________________
Assignment procedure emulated how: ________________________________
Time zero: ________________________________________________________
Follow-up start / end: ____________________________________________
Outcome: __________________________________________________________
Causal contrast: __________________________________________________
[ ] No eligibility criterion uses post-baseline information
[ ] Comparison group members were all treatment-eligible at time zero
PART E -- VALIDATION AND TRANSPORT
Internal validation design: _______________________________________
External validation: same system / different system / different
country / different case mix / none
Target population the claim is made for: __________________________
What makes the transport valid: ___________________________________
[ ] Discrimination with confidence intervals
[ ] Calibration slope and intercept, with plot
[ ] Performance by relevant subgroup
PART F -- CLAIM CHECK
Abstract conclusion, as currently written:
_________________________________________________________________
Does it contain a recommendation to ACT? yes / no
If yes: was any outcome under that action measured? yes / no
If no: rewrite as -> "Whether acting on this model changes
[outcome] is untested and would require [design]."
[ ] Intended use stated, and restricted where necessary
[ ] No comparator performance imported from another study
[ ] Explanation / interpretability reported as debugging only
PART G -- REGULATORY (complete if a submission is in view)
Intended use statement drafted? yes / no
Model locked, or updating? locked / updating
If updating: permitted changes pre-specified? yes / no
Evidence type: retrospective / prospective / randomised
Subgroup performance available for the deployed population? yes / no Pubrica AcademyWritten and maintained by the subject matter experts of Pubrica who do this work. Articles are revised when the standard underneath them moves, and each one carries the date it was last checked.
How Pubrica helps
Pubrica’s biostatistics group works on the part of this that cannot be fixed in editing: whether the estimand matches the question, whether the adjustment set follows from a stated structure, whether a target trial protocol can be specified at all, and what the transportability claim actually is. Our regulatory writers then build the clinical evaluation argument from that, rather than from a performance table.