Free scoping read. A written assessment of your manuscript in two business days, at no cost.

Start one
Contents

Academy · Research Data Analytics & AI · AI clinical impact challenges

Challenges of Machine Learning and Artificial Intelligence in Delivering Clinical Impact

Diagnostic accuracy is the least difficult aspect of clinical artificial intelligence (AI) and the aspect most frequently reported in the literature. The principal challenges are external validation in the setting of intended use, robustness to dataset shift, lawful access to data, integration into a workflow that changes once the model is introduced, and a regulatory pathway that currently limits the visibility of these systems. Each of these challenges also concerns reporting, because a reviewer judges a claim by the precision with which it is stated.

Pubrica AcademyWritten by the medical writing and biostatistics teams Reviewed byPubrica biostatistics team Published9 October 2026 Last checked9 October 2026 Reading time14 minutes Sources53 peer-reviewed

Kelly, Karthikesalingam, Suleyman, Corrado and King set out the problem in BMC Medicine: the gap between demonstrated algorithmic performance and delivered clinical benefit is not principally a modelling gap. The most frequently cited evidence on performance supports this conclusion from a different direction. Liu and colleagues, in The Lancet Digital Health, pooled deep learning at a sensitivity of 87.0% and a specificity of 92.5%, statistically comparable to health-care professionals, but only 14 of the studies they reviewed provided a comparison on the same sample. Claims of parity therefore rest on a limited number of fair comparisons, and this pattern of strong performance figures with weak comparability recurs at every stage between a model and a patient.

Learning objectives

  • The extent to which the pooled accuracy evidence establishes parity with clinicians
  • The reasons “a validated model” is not a meaningful claim, and the appropriate alternative formulation
  • The forms of dataset shift and those detectable by a given validation design
  • The data and governance constraints that apply before any modelling decision
  • The reasons a deployed model alters the setting that generated its training data
  • The reporting standard applicable to each study type, and the information each requires

Common misconceptions

  • “Deep learning matches or exceeds clinicians in diagnosis.” On pooled estimates its performance is comparable, but direct comparisons are scarce. Liu and colleagues, in The Lancet Digital Health, found deep learning’s pooled sensitivity of 87.0% and specificity of 92.5% statistically comparable to that of health-care professionals, with only 14 of the reviewed studies providing a comparison on the same sample. The remaining comparisons used a different sample, a different set of readers or no comparator.
  • “High accuracy in one task generalises to similar tasks.” It does not, and counterexamples exist in the same literature. Lee and colleagues, in Cancer Imaging, found that deep learning for axillary nodal staging pooled to a sensitivity of only 0.76 and specificity of 0.81, with a weighted AUC of 0.788, a moderate result considerably below the most frequently reported figures, on a clinically important task.
  • “The model has been validated.” Van Calster, Steyerberg, Wynants and van Smeden addressed this point directly in the title of their BMC Medicine paper: there is no such thing as a validated prediction model. Validation is a statement about a population, a setting and a time, not a property a model acquires. Collins, Riley and colleagues set out in the BMJ the specifications a validation study requires in order to be meaningful.
  • “A model that performs well at present will continue to do so.” Finlayson and colleagues, writing in the New England Journal of Medicine, describe dataset shift as a clinical problem rather than a technical one: practice patterns, coding, equipment, case mix and referral behaviour all change, and a model fitted to data from a previous period may degrade without warning. Lasko, Strobl and Stead, in npj Digital Medicine, examine why probabilistic clinical models fail to transfer between sites.
  • “Regulatory clearance identifies which systems use machine learning.” At present it does not. Benjamens, Dhunnoo and Meskó, in npj Digital Medicine, identified 64 FDA-approved AI and machine learning devices and algorithms, of which only 29 were described as AI or ML in the official FDA announcement. The public record therefore understates the extent to which this technology has already been cleared.

Definition of clinical impact and its distinction from accuracy

Clinical impact is a change in what happens to patients: a decision made differently, earlier, more consistently or at lower cost, with a consequent change in outcome. Diagnostic accuracy precedes clinical impact, and the chain between them comprises several links about which accuracy alone provides no information.

These links should be identified explicitly, because manuscripts frequently claim the last on the basis of evidence for the first. A model must be accurate in the population in which it will be used, not only in the population in which it was developed. Its errors must be distributed in a manner acceptable to clinicians, which concerns the type of errors rather than their number. Its output must reach a point in the workflow at which a decision remains open. A clinician must act on it, which depends on the degree of trust in the output and the cost of following it. Finally, the action must change an outcome, which is a question of effectiveness that accuracy cannot address.

Kelly and colleagues’ framing in BMC Medicine is useful precisely because it treats these as separate obstacles rather than a single translation gap 41. Topol, in Nature Medicine, surveys the same territory with greater optimism about the outcome; Wiens and colleagues, in the same journal, set out a roadmap for responsible machine learning in health care organised around the obligations at each link. For the purposes of writing, the principle is straightforward: authors should state the link for which the study provides evidence and should not allow the discussion section to advance implicitly to the next.

Scope of the evidence on diagnostic accuracy

The pooled evidence is stronger than sceptics assume and considerably narrower than prominent claims suggest, and both observations derive from the same source. Liu and colleagues’ systematic review and meta-analysis in The Lancet Digital Health found deep learning’s pooled sensitivity at 87.0% and specificity at 92.5%, statistically comparable to those of health-care professionals. This is a genuine finding about a substantial body of work.

The qualification appears in the same paper: only 14 of the reviewed studies provided a same-sample comparison 35. A comparison between an algorithm tested on one dataset and clinicians tested on another is not a comparison of the algorithm with the clinicians; it consists of two separate measurements presented together 35. Nagendran and colleagues, in the BMJ, examined the design, reporting standards and claims of deep learning studies comparing artificial intelligence with clinicians and found that the claims systematically exceeded what the designs could support, the same conclusion reached by auditing the studies rather than pooling them 30.

The second qualification is task dependence. Lee and colleagues, in Cancer Imaging, found that deep learning for axillary nodal staging pooled to a sensitivity of 0.76 and a specificity of 0.81 with a weighted AUC of 0.788 5. This is a useful result for a difficult task, but it is considerably below the aggregate figures. Any statement that deep learning achieves a given level of performance in medical imaging therefore describes a mixture of tasks, and the composition of that mixture determines the result. The Pubrica articles on deep learning versus machine learning in medical imaging, meta-analysis of convolutional neural networks for radiological images and systematic review of AI in imaging discuss the imaging evidence in greater detail.

Two prominent single-study results should be regarded as reference points rather than as evidence of parity: Esteva and colleagues’ dermatologist-level classification of skin cancer in Nature, and McKinney and colleagues’ international evaluation of an AI system for breast cancer screening in the same journal. Both are rigorous studies. Neither establishes that a system performs at that level in a different health system, which is the subject of the next section 32,47.

87.0%pooled sensitivity for deep learning
92.5%pooled specificity for deep learning
14reviewed studies providing a same-sample comparison with health-care professionals

The principal finding and its qualification derive from the same paper. Figures from Liu X, Faes L, Kale AU, et al. A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis. The Lancet Digital Health.

The performance claim in an abstract

Claim outruns the design

Our deep learning model achieved an AUC of 0.97 for the detection of the target condition, outperforming radiologists and demonstrating that artificial intelligence can be used to improve diagnostic accuracy in clinical practice.

One AUC, no setting. Whether this is internal or external performance is not stated, and the difference between them is usually decisive.

“Outperforming radiologists” with no comparison design. The report does not state whether the same sample was used, the number of readers or the clinical information available to them. Liu and colleagues found that only 14 studies in their review provided a same-sample comparison.

No calibration. An AUC indicates that the model ranks patients well; it provides no information on whether the probabilities it outputs can inform action.

No operating point. “Outperformed” on a sensitivity-specificity trade-off is undefined unless the threshold was chosen in advance.

“Can be used to improve diagnostic accuracy in clinical practice” is three links further down the chain than anything measured here; it asserts clinical impact on the basis of offline accuracy.

No negative statement. The reader is not told what was not tested, and must therefore either assume the worst or accept the claim without evidence.

Claim matched to the design

On a held-out test set of 1,200 studies from the development institution, the model achieved an AUC of 0.97 (95% CI reported in Table 2) with calibration slope and intercept as shown; on an external test set from two institutions not contributing to development, the AUC was 0.89. Three radiologists interpreted the same external test set independently, blinded to the model output and with the clinical information available in routine reporting; the model’s sensitivity was higher and its specificity lower than the pooled reader performance, with the operating point chosen prospectively. The model has not been evaluated prospectively in a clinical pathway, and its effect on diagnostic decisions, downstream investigation or patient outcome is unknown.

Internal and external performance are reported separately, and the decrease between them is visible rather than obscured by averaging.

The comparison is fully specified (same sample, number of readers, blinding and information available), which makes it a comparison rather than two separate measurements.

Calibration is reported alongside discrimination, so the reader can determine whether the output probability is accurate.

The operating point was chosen prospectively, which removes the possibility that the threshold was selected to favour the model in the comparison.

The sensitivity-specificity trade-off is stated in both directions rather than summarised as “outperformed”, which allows a clinician to judge whether the error profile suits the clinical setting.

The final sentence states exactly what has not been established. It anticipates the reviewer’s principal objection and increases the credibility of every other claim in the abstract.

The second abstract makes a more limited claim that a reader can act on and a reviewer can verify. The first makes a broader claim that a careful reviewer is likely to discount entirely, because it is not possible to determine which part of it the study supports.

Validation as a setting-specific claim

The most common overstatement in clinical AI manuscripts is the phrase “the validated model”. Van Calster, Steyerberg, Wynants and van Smeden argued in BMC Medicine that no such entity exists, and the argument is substantive rather than semantic: a model’s performance is a joint property of the model, the population, the measurement process and the time of measurement 21.

Collins, Dhiman, Ma and colleagues, in the BMJ, set out the pathway from development to external validation, and Riley, Archer, Snell and colleagues subsequently describe how to undertake an external validation study 10,11. This work yields three requirements against which a methods section can be assessed. First, the validation population must be described in sufficient detail for readers to judge whether it resembles their own, including case mix, prevalence, setting and the measurement of predictors. Second, calibration must be reported in addition to discrimination, because a model with an excellent AUC and poor calibration produces confidently incorrect probabilities, and probabilities are the basis of clinical action. Third, validation must take place in the setting of intended use, which is the point Sperrin, Riley, Collins and Martin make under the term targeted validation in Diagnostic and Prognostic Research 25.

Two further findings clarify what constitutes a fair test. Pajouheshnia, van Smeden, Peelen and Groenwold showed in the Journal of Clinical Epidemiology that variation in the measurement of a predictor affects both discrimination and transportability, so a model validated in a setting where a predictor is measured differently is, in effect, a different model 39. Schinkel and colleagues, in Scientific Reports, argue for incorporating cohort heterogeneity during development rather than minimising it, because a model fitted to a homogeneous development cohort has been optimised for a setting that does not exist 18. See statistical review and biostatistics support for the stages at which these checks are performed in practice.

The specifications required for a validation claim to be meaningful

ElementThe reader’s questionExample of an uninformative claim
PopulationDoes the validation population match the reader’s case mix, prevalence and setting?“Validated on an independent dataset” with no description of the dataset
Setting of intended useWas the model tested in the setting of intended use, at the decision point at which it will act?A screening-population model validated on a referred secondary-care sample
Predictor measurementWere the predictors measured as they are in the reader’s setting?No statement of how predictors were ascertained, so the model tested may differ from the model reported
DiscriminationDoes it rank patients correctly?An AUC reported alone, as though it were sufficient
CalibrationCan the output probabilities be accepted at face value?Omitted entirely; the most common gap, and the most consequential for a clinician acting on a predicted value
ComparisonAgainst which comparator, on which sample and with what information available?Algorithm on dataset A, clinicians on dataset B, reported as a comparison
TimeWhen were the data collected, and what has changed since?No data period, so dataset shift cannot be assessed

Elements compiled from Van Calster B, Steyerberg EW, Wynants L, van Smeden M, BMC Medicine; Collins GS and colleagues and Riley RD and colleagues, BMJ; Sperrin M, Riley RD, Collins GS, Martin GP, Diagnostic and Prognostic Research; and Pajouheshnia R and colleagues, Journal of Clinical Epidemiology.

Forms of dataset shift and their detectability by study design

Dataset shift explains why a model that performed well on recent data performs worse on data from the following year, and why a model developed at one hospital underperforms at another. Finlayson and colleagues, in the New England Journal of Medicine, present it for clinicians: the causes are ordinary operational events, such as the replacement of a scanner, a change in coding practice, the reorganisation of a referral pathway, a guideline revision that alters a treatment threshold, or a shift in case mix following the introduction of a new service 29.

The forms of shift should be distinguished, because they require different methods of detection. The input distribution may change while the relationship between inputs and outcome remains stable. The relationship itself may change, which is the more difficult case and the one that external validation at a single later time point will not detect. Prevalence may change, which leaves discrimination intact but severely compromises calibration and predictive values. The measurement process may change, which is the form Pajouheshnia and colleagues quantified. Dockès, Varoquaux and Poline, in GigaScience, treat the prevention of shift-induced failure as a design problem rather than a monitoring problem, and Lemmon and colleagues, in Methods of Information in Medicine, evaluate feature selection methods specifically for preserving performance under temporal dataset shift 19,28.

Transportability between sites has its own literature, which reaches a cautionary conclusion. Lasko, Strobl and Stead, in npj Digital Medicine, examine why probabilistic clinical models fail to transport between sites; Wan, Caffo and Vedula offer a unified framework for generalisability in Frontiers in Artificial Intelligence; Reps and colleagues, in BMC Medical Informatics and Decision Making, evaluate ensembles across multiple healthcare databases as a route to transportability; and de Jong and colleagues, in Statistics in Medicine, develop models from pooled studies and large clustered datasets for the same reason 17,22,23,26. The practical implication of this body of work is that transportability must be designed for and measured rather than assumed, and that single-site development with single-site validation provides very little information about it.

Data and governance constraints preceding model development

Most clinical AI projects are shaped more by the data that can lawfully and practically be assembled than by any methodological choice. This should be stated explicitly in a manuscript, because reviewers regard an unexplained convenience dataset as a weakness and an explained one as a constraint.

Three constraints recur. Permission. Becker, Chokoshvili and Dove, in International Data Privacy Law, examine the legal bases for secondary use of health and genetic data in the EU and the difficulty of harmonising them for cross-border sharing; Bourassa Forcier, Gallois, Mullan and Joly, in the Journal of Law and the Biosciences, ask whether the GDPR can serve as a model for policymakers integrating AI into care 13,40. Burton and colleagues, in the International Journal of Epidemiology, set out policies and strategies for facilitating secondary use, and Ballantyne, in GigaScience, addresses the prior question of where the human is located in the data. These considerations are not merely compliance matters; they determine the population on which a model can be developed and therefore the population for which it will perform.

Quality and documentation. Arora and colleagues, in Nature Medicine, make the case for standards for health datasets used in AI applications, on the grounds that a dataset without documented provenance, composition and known limitations cannot be assessed for fitness 20,43. Miller, in npj Digital Medicine, presents the same argument for practising physicians. The Pubrica articles on challenges in medical data collection and the limitations of digital health records describe these issues from the perspective of the data.

Technical routes around the constraint. Kaissis and colleagues, in Nature Machine Intelligence, demonstrate end-to-end privacy-preserving deep learning on multi-institutional medical imaging, a viable option for multi-site work where pooling raw data is not permissible 27. It should be regarded as a method of training across sites whose data could not otherwise be combined, not as a solution to the question of whether those sites’ populations resemble each other.

Effect of model deployment on the clinical setting

This is the challenge least often anticipated at the design stage and the one most likely to be unexpected after deployment. A model that influences care alters the data-generating process that produced its training data.

Lenert, Matheny and Walsh state this directly in the Journal of the American Medical Informatics Association: prognostic models will be victims of their own success 42. A model that identifies high-risk patients prompts intervention, the intervention reduces the predicted events, and the model’s apparent accuracy declines while its clinical value may have increased. Performance monitoring that does not account for this effect may interpret a successful deployment as a failing model.

The available responses are established, and each has costs. Lee and Lee, in The Lancet Digital Health, describe clinical applications of continual learning; Su and colleagues review statistical updating methods for clinical prediction models in Statistical Methods in Medical Research; Jenkins, Sperrin, Martin and Peek set out the methodological challenges of dynamic models in Diagnostic and Prognostic Research; and Janssen and colleagues showed in the Journal of Clinical Epidemiology that updating methods improved performance in new patients 31,44,45,53. A model that updates is a model whose behaviour changes after approval, which is precisely the problem regulators are addressing.

The human factors are equally consequential. Gerdes, in Discover Artificial Intelligence, examines the role of explainability in AI-supported medical decision-making; the underlying practical question is how a clinician responds to an output that cannot be interrogated when it conflicts with clinical judgement. This is an empirical question concerning a specific tool in a specific setting, and it should be addressed in an early-stage clinical evaluation rather than in the closing paragraph of a discussion section.

Regulatory approval and the visibility of AI devices

The regulatory position is changing rapidly, and one empirical finding concerning it is particularly relevant. Benjamens, Dhunnoo and Meskó, in npj Digital Medicine, built a database of FDA-approved AI and machine learning based medical devices and algorithms and identified 64, of which only 29 were described as AI or ML in the official FDA announcement 34. Fewer than half of the cleared systems identified were described as AI or ML in the public record 34.

Two implications follow. For researchers, the regulatory literature is an unreliable sampling frame for technologies in clinical use, so any claim about the extent of AI approval requires careful sourcing. For developers, the absence of AI terminology in a clearance announcement is not evidence that a comparable product has not already been cleared.

Beyond the difficulty of enumeration, the substantive issues converge on the themes discussed above. The FDA’s framework for artificial intelligence and machine learning in software as a medical device addresses the problem of post-approval change directly, since a learning system is a device whose behaviour changes after clearance. Liu, Yu and Dillon, in npj Digital Medicine, survey regulatory responses and approval status with a focus on China, which usefully complements a discussion centred largely on the US and the EU. Rademakers and colleagues, writing for the CORE-MD group in the same journal, propose a clinical risk score for the regulatory evaluation of AI-based medical device software, intended to make the level of evidence required proportionate to the risk the software carries 4. Regulatory affairs and causal inference in machine learning and regulatory and ethical challenges for medical devices take this further.

64FDA-approved AI and machine learning devices and algorithms identified
29described as AI or ML in the official FDA announcement

The public regulatory record understates the extent to which machine learning has already been cleared. Figures from Benjamens S, Dhunnoo P, Meskó B. The state of artificial intelligence-based FDA-approved medical devices and algorithms: an online database. npj Digital Medicine.

Reporting clinical AI studies for peer review

The principal question for a reader of a clinical AI paper is not whether the model performs well but whether the claim made corresponds to the claim the design supports. The AI-specific reporting standards exist to enforce that correspondence, and selecting the appropriate standard is the most efficient means of establishing credibility.

The standard should be matched to the study. A study developing or validating a prediction model reports to TRIPOD+AI. A randomised trial of an AI intervention reports to CONSORT-AI, and its protocol to SPIRIT-AI. A diagnostic accuracy study reports to STARD-AI. An early-stage clinical evaluation of a decision support system driven by AI reports to DECIDE-AI, published by Vasey and colleagues in Nature Medicine, the standard covering the stage between offline accuracy and a definitive trial, which is where most of the challenges described in this article become apparent 24. For the appraisal of prediction models, as distinct from their reporting, PROBAST+AI is the risk-of-bias instrument.

Three statements are essential in a methods section, whichever standard applies. Authors should name the setting of intended use and show that the validation population resembles it; report calibration alongside discrimination and, where applicable, describe how probabilities were calibrated; and state the comparison precisely, including whether the same sample was used, the number of readers and the information available to them, because this is the detail that the review by Liu and colleagues shows to be most frequently omitted. Reporting checklist review and bias and fairness review are the Pubrica services in which these checks are performed.

The AI reporting standard applicable to each study type

If your study isReport toRequired statements
Development or validation of a clinical prediction model, including machine learning modelsTRIPOD+AIThe intended use, how the model was built, the validation design, and calibration as well as discrimination
A randomised trial of an AI-based interventionCONSORT-AIThe intervention version and inputs, how the AI output entered the care pathway, and handling of poor-quality or unavailable input data
The protocol for such a trialSPIRIT-AIThe same specification, pre-specified, before results exist
A diagnostic accuracy study of an AI systemSTARD-AIThe reference standard, the index test as a specified version, and the sample on which both were applied
An early-stage clinical evaluation of an AI-driven decision support systemDECIDE-AI — Vasey and colleagues, Nature MedicineHuman factors, implementation detail, safety and the clinician’s interaction with the output, at the stage between offline accuracy and a definitive trial
An appraisal of existing prediction models, for example in a systematic reviewPROBAST+AIRisk of bias and applicability, judged by domain rather than as a single grade

Standards as published by their respective development groups. DECIDE-AI is cited to Vasey B, Nagendran M, Campbell B, et al. Reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI. Nature Medicine.

Key points

Key points

The principal obstacle between clinical AI and clinical impact is not model performance. Liu and colleagues pooled deep learning at 87.0% sensitivity and 92.5% specificity, statistically comparable to health-care professionals, but only 14 of the reviewed studies compared the two on the same sample, and performance is highly task-dependent, with deep learning for axillary nodal staging pooling to 0.76 sensitivity and 0.81 specificity. The factors that lie between a strong model and an improved outcome are validation in the setting of intended use rather than “a validated model”, which Van Calster and colleagues argue does not exist; robustness to dataset shift, which Finlayson and colleagues present as an operational certainty rather than a risk; lawful access to data that resemble the deployment population; a workflow in which, as Lenert and colleagues observe, an effective prognostic model reduces its own apparent accuracy; and a regulatory record that labelled only 29 of 64 identified FDA-cleared AI devices as AI. In reporting, authors should identify the link in the chain for which the study provides evidence, report calibration together with discrimination, specify the comparison exactly, and use the reporting standard appropriate to the study: TRIPOD+AI, CONSORT-AI, SPIRIT-AI, STARD-AI, DECIDE-AI or PROBAST+AI.

Application to your study

Application to your study

  1. Identify the link in the evidential chain that the study addresses. The possible links are offline accuracy, external validity, change in clinical decisions and effect on outcomes. No sentence in the abstract or discussion should claim the subsequent link.
  2. Replace every use of “validated” with a specific claim. The claim should state the population, the setting, the period and the method of measuring predictors. If these cannot be specified, the term conveys no information and is likely to be challenged by reviewers.
  3. Report calibration alongside discrimination in every case. An AUC alone provides a clinician with no information about the reliability of the value the model outputs, which is the value on which clinical action is based.
  4. Specify any comparison in full or omit it. The specification should include the sample, the number of readers, blinding and the clinical information available to the readers. A cross-dataset comparison presented as a direct comparison is the error most prevalent in the evidence base.
  5. Write the paragraph on dataset shift before discussing deployment. The paragraph should state the data period, subsequent changes, differences in prevalence in the target setting and the monitoring that would detect degradation, including the possibility that an effective model may appear to be failing.
  6. Select the reporting standard first and draft in accordance with it. TRIPOD+AI applies to a prediction model, CONSORT-AI and SPIRIT-AI to a trial and its protocol, STARD-AI to diagnostic accuracy, DECIDE-AI to early clinical evaluation, and PROBAST+AI to the appraisal of other researchers’ models.

Frequently asked questions

On pooled estimates, its performance is comparable rather than superior, and the comparison rests on a limited evidence base. Liu and colleagues’ systematic review and meta-analysis in The Lancet Digital Health found deep learning’s pooled sensitivity of 87.0% and specificity of 92.5% statistically comparable to those of health-care professionals, but only 14 of the studies they reviewed provided a comparison on the same sample. Performance also varies considerably by task: Lee and colleagues found that deep learning for axillary nodal staging pooled to a sensitivity of 0.76 and specificity of 0.81 with a weighted AUC of 0.788. Nagendran and colleagues, examining design and claims in the BMJ, found that the claims in this literature frequently exceeded what the study designs could support.

It means that its performance was measured on data from a population, institution or time period that did not contribute to its development, and in the setting in which it is intended to be used. Van Calster, Steyerberg, Wynants and van Smeden argued in BMC Medicine that there is no such thing as a validated model, because performance is a joint property of the model, the population, the measurement process and the time of measurement. A usable validation report specifies the case mix, prevalence and setting of the validation population; the method of measuring predictors in that population; discrimination and calibration; the operating point; and the data period. Riley and colleagues set out how to undertake such a study in the BMJ, and Sperrin and colleagues argue for targeted validation specifically in the intended population.

Four reasons predominate, and most do not concern the model itself. The first is dataset shift: Finlayson and colleagues, in the New England Journal of Medicine, describe how scanner replacement, coding changes, pathway reorganisation and changes in case mix degrade a fitted model, and Lasko, Strobl and Stead address in npj Digital Medicine why models fail to transport between sites. The second is calibration: a model may rank patients well yet output unreliable probabilities. The third is workflow: the output must reach the point at which a decision remains open, and a clinician must act on it. The fourth is the self-defeating effect described by Lenert, Matheny and Walsh, whereby a prognostic model that prompts effective intervention reduces the events it predicts, so that its apparent accuracy falls while its value increases.

More than the public record indicates, and enumeration is genuinely difficult. Benjamens, Dhunnoo and Meskó, in npj Digital Medicine, built an online database of FDA-approved AI and machine learning based medical devices and algorithms and identified 64, of which only 29 were described as AI or ML in the official FDA announcement. Fewer than half of the systems identified were therefore labelled as such in the source a reader would consult. Two consequences follow: regulatory announcements are an unreliable sampling frame for technologies in clinical use, and the absence of AI terminology in a clearance notice is not evidence that a comparable product has not been cleared. Liu, Yu and Dillon survey the position outside the US, with a focus on China, in the same journal.

The standard should be matched to the study type: TRIPOD+AI for the development or validation of a clinical prediction model; CONSORT-AI for a randomised trial of an AI intervention, with SPIRIT-AI for its protocol; STARD-AI for a diagnostic accuracy study; and DECIDE-AI, published by Vasey and colleagues in Nature Medicine, for an early-stage clinical evaluation of a decision support system driven by AI, which is the stage between offline accuracy and a definitive trial and the stage at which human factors, implementation and safety first become apparent. PROBAST+AI applies to the appraisal of existing prediction models rather than the reporting of a new one. Selecting the standard before drafting is more useful than mapping the manuscript onto it afterwards, because several items require information that must be pre-specified.

Dataset shift is any change between the data on which a model was fitted and the data to which it is applied, and it takes distinguishable forms: the input distribution changes; the relationship between inputs and outcome changes; prevalence changes, which leaves discrimination intact but compromises calibration; or the measurement process changes, which Pajouheshnia and colleagues showed affects both discrimination and transportability. Authors should report the data period for development and for each validation, describe subsequent changes in the setting, state the prevalence in the target population relative to the development population, and specify the monitoring that would detect degradation. Dockès, Varoquaux and Poline treat prevention as a design problem in GigaScience, and Lemmon and colleagues evaluate feature selection for stability under temporal shift.

Technically, yes, although it is important to be precise about what this achieves. Kaissis and colleagues demonstrated end-to-end privacy-preserving deep learning on multi-institutional medical imaging in Nature Machine Intelligence, so training across sites that cannot pool raw data is a practical option rather than a proposal. It addresses the legal and governance barriers to combining data, which Becker, Chokoshvili and Dove describe for the EU in International Data Privacy Law and Burton and colleagues address in the International Journal of Epidemiology. It does not address whether the participating sites’ populations resemble each other or the deployment setting. Privacy-preserving training resolves a problem of permission but leaves the problem of transportability unchanged.

The specific matters that the design could not establish, stated explicitly rather than alluded to. These typically include the following: that performance was measured on retrospective data and the prospective effect on decisions is unknown; which populations are absent or under-represented in development and validation, and for whom the model is therefore likely to perform less well; that calibration in the deployment setting will differ if prevalence differs; that the data period predates known changes in practice, equipment or coding; whether or not the reader comparison was performed on the same sample; and that no effect on outcomes has been demonstrated. A limitations paragraph that names these matters demonstrates competence; a generic paragraph appears to be an attempt to satisfy a journal requirement.

References

  1. Artificial Intelligence and Machine Learning in Software as a Medical Device. U.S. Food and Drug Administration (Digital Health Center of Excellence). 2024. read 9 October 2026. fda.gov
  2. Farahani S, Hejazi M, Tabassum M, Di Ieva A, Mahdavifar N, et al. , Liu S, et al. Diagnostic performance of deep learning for predicting glioma isocitrate dehydrogenase and 1p/19q co-deletion in MRI: a systematic review and meta-analysis. European Radiology. 2026;36(2):1562-1591. doi:10.1007/s00330-025-11898-2 read 9 October 2026. doi.org/10.1007/s00330-025-11898-2
  3. Teodoro D, Naderi N, Yazdani A, Zhang B, et al. , Bornet A, et al. A scoping review of artificial intelligence applications in clinical trial risk assessment. npj Digital Medicine. 2025;8(1):486. doi:10.1038/s41746-025-01886-7 read 9 October 2026. doi.org/10.1038/s41746-025-01886-7
  4. Rademakers FE, Biasin E, Bruining N, Caiani EG, Davies RH, Gilbert SH, Kamenjasevic E, McGauran G, O'Connor G, Rouffet JB, Vasey B, et al. , Fraser AG, et al. CORE-MD clinical risk score for regulatory evaluation of artificial intelligence-based medical device software. npj Digital Medicine. 2025;8(1):90. doi:10.1038/s41746-025-01459-8 read 9 October 2026. doi.org/10.1038/s41746-025-01459-8
  5. Lee CF, Lin J, Huang YL, Chen ST, Chou CT, Chen DR, et al. , Wu WP, et al. Deep learning-based breast MRI for predicting axillary lymph node metastasis: a systematic review and meta-analysis. Cancer Imaging. 2025;25(1):44. doi:10.1186/s40644-025-00863-3 read 9 October 2026. doi.org/10.1186/s40644-025-00863-3
  6. Abdullah KA, Marziali S, Nanaa M, Escudero Sanchez L, Payne NR, et al. , Gilbert FJ, et al. Deep learning-based breast cancer diagnosis in breast MRI: systematic review and meta-analysis. European Radiology. 2025;35(8):4474-4489. doi:10.1007/s00330-025-11406-6 read 9 October 2026. doi.org/10.1007/s00330-025-11406-6
  7. Du S, Guo J, Huang D, Liu Y, Zhang X, Lu S. Diagnostic accuracy of deep learning-based algorithms in laryngoscopy: a systematic review and meta-analysis. European Archives of Oto-Rhino-Laryngology. 2025;282(1):351-360. doi:10.1007/s00405-024-09049-2 read 9 October 2026. doi.org/10.1007/s00405-024-09049-2
  8. Valizadeh P, Jannatdoust P, Pahlevan-Fallahy MT, Hassankhani A, Amoukhteh M, Bagherieh S, Ghadimi DJ, et al. , Gholamrezanezhad A, et al. Diagnostic accuracy of radiomics and artificial intelligence models in diagnosing lymph node metastasis in head and neck cancers: a systematic review and meta-analysis. Neuroradiology. 2025;67(2):449-467. doi:10.1007/s00234-024-03485-x read 9 October 2026. doi.org/10.1007/s00234-024-03485-x
  9. Coveney P, Highfield R, Stahlberg E, et al. , Vazquez M, et al. Digital twins and Big AI: the future of truly individualised healthcare. npj Digital Medicine. 2025;8(1):494. doi:10.1038/s41746-025-01874-x read 9 October 2026. doi.org/10.1038/s41746-025-01874-x
  10. Collins GS, Dhiman P, Ma J, Schlussel MM, Archer L, Van Calster B, Harrell FE Jr, Martin GP, Moons KGM, van Smeden M, Sperrin M, Bullock GS, Riley RD. Evaluation of clinical prediction models (part 1): from development to external validation. BMJ. 2024;384:e074819. read 9 October 2026. arxiv.org
  11. Riley RD, Archer L, Snell KIE, Ensor J, Dhiman P, Martin GP, Bonnett LJ, Collins GS. Evaluation of clinical prediction models (part 2): how to undertake an external validation study. BMJ. 2024;384:e074820. read 9 October 2026. arxiv.org
  12. Li YH, Li YL, Wei MY, Li GY, et al, et al. Innovation and challenges of artificial intelligence technology in personalized healthcare. Scientific Reports. 2024;14(1):18994. doi:10.1038/s41598-024-70073-7 read 9 October 2026. doi.org/10.1038/s41598-024-70073-7
  13. Becker R, Chokoshvili D, Dove ES. Legal bases for effective secondary use of health and genetic data in the EU: time for new legislative solutions to better harmonize data for cross-border sharing? International Data Privacy Law. 2024;14(3):223-246. doi:10.1093/idpl/ipae014 read 9 October 2026. doi.org/10.1093/idpl/ipae014
  14. Liu Y, Yu W, Dillon T. Regulatory responses and approval status of artificial intelligence medical devices with a focus on China. npj Digital Medicine. 2024;7(1):255. doi:10.1038/s41746-024-01254-x read 9 October 2026. doi.org/10.1038/s41746-024-01254-x
  15. Wang TW, Hong JS, Chiu HY, Chao HS, Chen YM, Wu YT. Standalone deep learning versus experts for diagnosis lung cancer on chest computed tomography: a systematic review. European Radiology. 2024;34(11):7397-7407. doi:10.1007/s00330-024-10804-6 read 9 October 2026. doi.org/10.1007/s00330-024-10804-6
  16. Gerdes A. The role of explainability in AI-supported medical decision-making. Discover Artificial Intelligence. 2024;4(1):29. doi:10.1007/s44163-024-00119-2 read 9 October 2026. doi.org/10.1007/s44163-024-00119-2
  17. Lasko TA, Strobl EV, Stead WW. Why do probabilistic clinical models fail to transport between sites. npj Digital Medicine. 2024;7. read 9 October 2026. arxiv.org
  18. Schinkel M, Bennis FC, Boerman AW, Wiersinga WJ, Nanayakkara PWB. Embracing cohort heterogeneity in clinical machine learning development: a step toward generalizable models. Scientific Reports. 2023;13:8363. read 9 October 2026. arxiv.org
  19. Lemmon J, Guo LL, Posada J, Pfohl SR, Fries J, Fleming SL, Aftandilian C, Shah N, Sung L. Evaluation of feature selection methods for preserving machine learning performance in the presence of temporal dataset shift in clinical medicine. Methods of Information in Medicine. 2023;62:60-70. read 9 October 2026. arxiv.org
  20. Arora A, Alderman JE, Palmer J, Ganapathi S, Laws E, McCradden MD, Oakden-Rayner L, Pfohl SR, Ghassemi M, McKay F, et al. , Liu X, et al. The value of standards for health datasets in artificial intelligence-based applications. Nature Medicine. 2023;29(11):2929-2938. doi:10.1038/s41591-023-02608-w read 9 October 2026. doi.org/10.1038/s41591-023-02608-w
  21. Van Calster B, Steyerberg EW, Wynants L, van Smeden M. There is no such thing as a validated prediction model. BMC Medicine. 2023;21:70. read 9 October 2026. arxiv.org
  22. Wan B, Caffo B, Vedula SS. A unified framework on generalizability of clinical prediction models. Frontiers in Artificial Intelligence. 2022;5:872720. read 9 October 2026. arxiv.org
  23. Reps JM, Williams RD, Schuemie MJ, Ryan PB, Rijnbeek PR. Learning patient-level prediction models across multiple healthcare databases: evaluation of ensembles for increasing model transportability. BMC Medical Informatics and Decision Making. 2022;22:142. read 9 October 2026. arxiv.org
  24. Vasey B, Nagendran M, Campbell B, Clifton DA, Collins GS, Denaxas S, Denniston AK, Faes L, Geerts B, Ibrahim M, Liu X, Mateen BA, Mathur P, McCradden MD, Morgan L, Ordish J, Rogers C, Saria S, Ting DSW, Watkinson P, Weber W, Wheatstone P, McCulloch P. Reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI. Nature Medicine. 2022;28:924-33. doi:10.1038/s41591-022-01772-9 read 9 October 2026. doi.org/10.1038/s41591-022-01772-9
  25. Sperrin M, Riley RD, Collins GS, Martin GP. Targeted validation: validating clinical prediction models in their intended population and setting. Diagnostic and Prognostic Research. 2022;6:24. read 9 October 2026. arxiv.org
  26. de Jong VMT, Moons KGM, Eijkemans MJC, Riley RD, Debray TPA. Developing more generalizable prediction models from pooled studies and large clustered data sets. Statistics in Medicine. 2021;40:3533-59. read 9 October 2026. arxiv.org
  27. Kaissis G, Ziller A, Passerat-Palmbach J, Ryffel T, Usynin D, Trask A, Lima I, Mancuso J, Jungmann F, Steinborn MM, Saleh A, Makowski M, Rueckert D, Braren R. End-to-end privacy preserving deep learning on multi-institutional medical imaging. Nature Machine Intelligence. 2021;3(6):473-484. doi:10.1038/s42256-021-00337-8 read 9 October 2026. doi.org/10.1038/s42256-021-00337-8
  28. Dockes J, Varoquaux G, Poline JB. Preventing dataset shift from breaking machine-learning biomarkers. GigaScience. 2021;10:giab055. read 9 October 2026. arxiv.org
  29. Finlayson SG, Subbaswamy A, Singh K, Bowers J, Kupke A, Zittrain J, Kohane IS, Saria S. The clinician and dataset shift in artificial intelligence. New England Journal of Medicine. 2021;385:283-6. read 9 October 2026. arxiv.org
  30. Nagendran M, Chen Y, Lovejoy CA, Gordon AC, Komorowski M, Harvey H, Topol EJ, Ioannidis JPA, Collins GS, Maruthappu M. Artificial intelligence versus clinicians: systematic review of design, reporting standards, and claims of deep learning studies. BMJ. 2020;368:m689. doi:10.1136/bmj.m689 read 9 October 2026. doi.org/10.1136/bmj.m689
  31. Lee CS, Lee AY. Clinical applications of continual learning machine learning. The Lancet Digital Health. 2020;2:e279-81. doi:10.1016/S2589-7500(20)30102-3 read 9 October 2026. doi.org/10.1016/S2589-7500(20)30102-3
  32. McKinney SM, Sieniek M, Godbole V, Godwin J, Antropova N, Ashrafian H, Back T, Chesus M, Corrado GS, Darzi A, Etemadi M, Garcia-Vicente F, Gilbert FJ, Halling-Brown M, Hassabis D, Jansen S, Karthikesalingam A, Kelly CJ, King D, Ledsam JR, Melnick D, Mostofi H, Peng L, Reicher JJ, Romera-Paredes B, Sidebottom R, Suleyman M, Tse D, Young KC, De Fauw J, Shetty S. International evaluation of an AI system for breast cancer screening. Nature. 2020;577:89-94. doi:10.1038/s41586-019-1799-6 read 9 October 2026. doi.org/10.1038/s41586-019-1799-6
  33. Raphael MJ, Gyawali B, Booth CM. Real-world evidence and regulatory drug approval. Nature Reviews Clinical Oncology. 2020;17(5):271-272. doi:10.1038/s41571-020-0345-7 read 9 October 2026. doi.org/10.1038/s41571-020-0345-7
  34. Benjamens S, Dhunnoo P, Mesko B. The state of artificial intelligence-based FDA-approved medical devices and algorithms: an online database. npj Digital Medicine. 2020;3(1):118. doi:10.1038/s41746-020-00324-0 read 9 October 2026. doi.org/10.1038/s41746-020-00324-0
  35. Liu X, Faes L, Kale AU, Wagner SK, Fu DJ, Bruynseels A, Mahendiran T, Moraes G, Shamdas M, Kern C, Ledsam JR, Schmid MK, Balaskas K, Topol EJ, Bachmann LM, Keane PA, Denniston AK. A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis. The Lancet Digital Health. 2019;1(6):e271-e297. doi:10.1016/S2589-7500(19)30123-2 read 9 October 2026. doi.org/10.1016/S2589-7500(19)30123-2
  36. Shah P, Kendall F, Khozin S, Goosen R, Hu J, Laramie J, Ringel M, Schork N. Artificial intelligence and machine learning in clinical development: a translational perspective. npj Digital Medicine. 2019;2(1):69. doi:10.1038/s41746-019-0148-3 read 9 October 2026. doi.org/10.1038/s41746-019-0148-3
  37. Wiens J, Saria S, Sendak M, Ghassemi M, Liu VX, Doshi-Velez F, Jung K, Heller K, Kale D, Saeed M, Ossorio PN, Thadaney-Israni S, Goldenberg A. Do no harm: a roadmap for responsible machine learning for health care. Nature Medicine. 2019;25:1337-40. doi:10.1038/s41591-019-0548-6 read 9 October 2026. doi.org/10.1038/s41591-019-0548-6
  38. Topol EJ. High-performance medicine: the convergence of human and artificial intelligence. Nature Medicine. 2019;25(1):44-56. doi:10.1038/s41591-018-0300-7 read 9 October 2026. doi.org/10.1038/s41591-018-0300-7
  39. Pajouheshnia R, van Smeden M, Peelen LM, Groenwold RHH. How variation in predictor measurement affects the discriminative ability and transportability of a prediction model. Journal of Clinical Epidemiology. 2019;105:136-41. read 9 October 2026. arxiv.org
  40. Bourassa Forcier M, Gallois H, Mullan S, Joly Y. Integrating artificial intelligence into health care through data access: can the GDPR act as a beacon for policymakers? Journal of Law and the Biosciences. 2019;6(1):317-335. doi:10.1093/jlb/lsz013 read 9 October 2026. doi.org/10.1093/jlb/lsz013
  41. Kelly CJ, Karthikesalingam A, Suleyman M, Corrado G, King D. Key challenges for delivering clinical impact with artificial intelligence. BMC Medicine. 2019. doi:10.1186/s12916-019-1426-2 read 9 October 2026. doi.org/10.1186/s12916-019-1426-2
  42. Lenert MC, Matheny ME, Walsh CG. Prognostic models will be victims of their own success, unless. Journal of the American Medical Informatics Association. 2019;26:1645-50. read 9 October 2026. arxiv.org
  43. Miller DD. The medical AI insurgency: what physicians must know about data to practice with intelligent machines. npj Digital Medicine. 2019;2(1):62. doi:10.1038/s41746-019-0138-5 read 9 October 2026. doi.org/10.1038/s41746-019-0138-5
  44. Su TL, Jaki T, Hickey GL, Buchan I, Sperrin M. A review of statistical updating methods for clinical prediction models. Statistical Methods in Medical Research. 2018;27:185-97. read 9 October 2026. arxiv.org
  45. Jenkins DA, Sperrin M, Martin GP, Peek N. Dynamic models to predict health outcomes: current status and methodological challenges. Diagnostic and Prognostic Research. 2018;2:23. read 9 October 2026. arxiv.org
  46. Ballantyne A. Where is the human in the data? A guide to ethical data use. GigaScience. 2018;7(7):giy076. doi:10.1093/gigascience/giy076 read 9 October 2026. doi.org/10.1093/gigascience/giy076
  47. Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM, Thrun S. Dermatologist-level classification of skin cancer with deep neural networks. Nature. 2017;542(7639):115-118. doi:10.1038/nature21056 read 9 October 2026. doi.org/10.1038/nature21056
  48. Burton PR, Banner N, Elliot MJ, Knoppers BM, Banks J. Policies and strategies to facilitate secondary use of research data in the health sciences. International Journal of Epidemiology. 2017;46(6):1729-1733. doi:10.1093/ije/dyx195 read 9 October 2026. doi.org/10.1093/ije/dyx195
  49. Dart A. The algorithm will see you now. Nature Reviews Cancer. 2017;17(3):142. doi:10.1038/nrc.2017.10 read 9 October 2026. doi.org/10.1038/nrc.2017.10
  50. Riley RD, Ensor J, Snell KIE, Debray TPA, Altman DG, Moons KGM, Collins GS. External validation of clinical prediction models using big datasets from e-health records or IPD meta-analysis: opportunities and challenges. BMJ. 2016;353:i3140. read 9 October 2026. arxiv.org
  51. Debray TPA, Moons KGM, Ahmed I, Koffijberg H, Riley RD. A framework for developing, implementing, and evaluating clinical prediction models in an individual participant data meta-analysis. Statistics in Medicine. 2013;32:3158-80. read 9 October 2026. arxiv.org
  52. Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, Pencina MJ, Kattan MW. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010;21:128. read 9 October 2026. arxiv.org
  53. Janssen KJM, Moons KGM, Kalkman CJ, Grobbee DE, Vergouwe Y. Updating methods improved the performance of a clinical prediction model in new patients. Journal of Clinical Epidemiology. 2008;61:76-86. read 9 October 2026. arxiv.org

Every source above is a peer-reviewed article, a publisher’s record of one, or a standard issued by the body that owns it. Each carries the date it was last read. Where a standard moves, this article is revised rather than left up with a note.

How to cite this article

Pick your referencing style and copy it. Your university or target journal may vary the punctuation — check its guide if it has one.

Pubrica Academy. (2026, October 9). Challenges of Machine Learning and Artificial Intelligence in Delivering Clinical Impact. Pubrica. https://pubrica.com/academy/medical-writing/list-out-the-challenges-of-machine-learning-artificial-intelligence-for-delivering-clinical-impact/

This article is written to be learned from. Cite it if you are referring to what it says; if you are using the evidence behind it, cite the sources listed above instead.

Clinical AI claim-and-evidence audit

This checklist should be applied to the manuscript before submission. Every claim in the abstract should correspond to a completed item, and any item that cannot be completed should be addressed in the limitations section.

CLINICAL AI MANUSCRIPT: CLAIM-AND-EVIDENCE AUDIT

0. WHICH LINK IN THE CHAIN DOES THIS STUDY ADDRESS?
   [ ] offline accuracy on retrospective data
   [ ] external validity in a new population/site/time
   [ ] change in clinician decisions
   [ ] change in patient outcome or resource use
   Strongest claim my design supports, in one sentence:
   ______________________________________________________
   [ ] No sentence in the abstract or discussion claims a later link

1. REPORTING STANDARD
   Standard used: [ ] TRIPOD+AI  [ ] CONSORT-AI  [ ] SPIRIT-AI
                  [ ] STARD-AI   [ ] DECIDE-AI   [ ] PROBAST+AI
   [ ] Completed checklist with section locations attached

2. INTENDED USE
   Population: ______________________
   Setting and decision point the output enters: ______________
   Prevalence in intended-use setting: ______
   Prevalence in development data: ______
   [ ] Validation population described well enough to compare

3. PERFORMANCE  (fill both columns or explain why not)
                              Internal        External
   Discrimination (AUC)       ________        ________
   Sensitivity                ________        ________
   Specificity                ________        ________
   Calibration slope          ________        ________
   Calibration intercept      ________        ________
   Operating point            ________        ________
   [ ] Operating point chosen PROSPECTIVELY
   [ ] Calibration reported, not just discrimination
   [ ] External data contributed nothing to development

4. COMPARISON WITH CLINICIANS  (skip if none - do not imply one)
   Same sample as the model was tested on?   [ ] yes  [ ] no
   Number of readers: ____
   Readers blinded to model output?          [ ] yes  [ ] no
   Clinical information available to readers: ______________
   [ ] The comparison is described in these terms in the abstract

5. DATA AND GOVERNANCE
   Data period: ______ to ______
   Legal basis / approval for use: ______________________
   Sites contributing: ____  Multi-site training method: ______
   Dataset documentation available (provenance, composition,
     known limitations)?  [ ]
   Groups under-represented or absent: ______________________

6. SHIFT AND MONITORING
   Known changes in practice/equipment/coding since data period:
   ______________________________________________________
   Planned monitoring metric: ______________________
   [ ] Monitoring plan accounts for a model that reduces the events
       it predicts (apparent accuracy falling while value rises)
   Update/retraining policy: ______________________

7. LIMITATIONS  (one specific sentence each - no generics)
   Retrospective design: ______________________
   Population coverage: ______________________
   Calibration in deployment setting: ______________________
   Reader comparison: ______________________
   Outcome effect not established: ______________________

Pubrica AcademyWritten and maintained by Pubrica’s subject matter experts. Articles are revised when the underlying standards change, and each article states the date on which it was last reviewed.

How Pubrica can help

Pubrica’s data and biostatistics teams assess the aspect that reviewers of clinical AI papers examine first: whether the claim corresponds to the design. Pubrica maps the manuscript to TRIPOD+AI, CONSORT-AI, SPIRIT-AI, STARD-AI or DECIDE-AI as the study type requires, identifies instances in which discrimination is reported without calibration, revises comparison statements so that the sample and reader conditions are explicit, and identifies where the discussion advances from offline accuracy to clinical impact without supporting evidence. Where the justified conclusion is narrower than that in the draft, Pubrica states this and assists in presenting the narrower conclusion.

Find something else

Search the Pubrica Academy

Access is free and requires no account. Enter a topic, or open the full library to filter by category, sub-topic or resource type.

Pubrica reviews AI manuscripts and identifies which claims the study design supports and which a reviewer is likely to discount.

A scoping assessment is free and takes two business days. It provides a written assessment of what the work requires and which specialist would undertake it.

Get a free quote

Regulations

Check your project against this rule

  1. 1Your work
  2. 2Your details
  3. 3Send
What do you need?

Material you send is held under signed confidentiality, with access limited to the assigned team and encrypted transfer. It is not used to train anything and is not passed to third-party tools. Ask for the agreement before you send anything you would rather cover first.