Free scoping read. A written assessment of your manuscript in two business days, at no cost.

Start one
Contents

Academy · Research Data Analytics & AI · AI in medical imaging

Artificial Intelligence in Medical Imaging Research: Demonstrated Capabilities and Evidence Gaps

Imaging artificial intelligence (AI) models perform well at narrow, well-defined tasks such as segmenting a structure, flagging a probable abnormality or performing a measurement that would otherwise be made visually. The principal evidential problem is not whether the technology works in principle but that most published studies have not evaluated it to the standard a clinical claim requires.

Pubrica AcademyWritten by the medical writing and biostatistics teams Reviewed byPubrica biostatistics team Published8 October 2026 Last checked8 October 2026 Reading time7 minutes

This article is intended for research that uses, validates or reports an imaging AI tool. It describes the tasks these models have been shown to perform well, the two reporting standards that now govern how such work is reported, and the areas in which the published evidence is commonly deficient.

Functions of imaging AI models

Clinical imaging AI predominantly performs one of three functions: segmentation of a structure on a scan (delineating an organ or lesion boundary), classification of an image (flagging a probable finding for human review), or continuous quantification, such as frame-by-frame estimation of left ventricular ejection fraction from an echocardiogram video. In routine practice, none of these tasks replaces clinical diagnosis independently; each functions as a measurement or triage aid within the clinician’s workflow.

Applicable reporting standards

TRIPOD+AI (Collins GS, et al. BMJ. 2024;385:e078378) governs the reporting of clinical prediction models developed with regression or machine-learning methods, including models that use imaging as an input to predict a future outcome or a current diagnosis. STARD-AI (Sounderajah V, et al. Nat Med. 2025) governs diagnostic-accuracy studies that use AI specifically, in which the question is how well the model distinguishes disease from its absence against a reference standard. A study that combines both (for example, an AI model predicting diagnosis from imaging, evaluated against pathology) may need to satisfy items from each.

Characteristics of a well-evidenced example

EchoNet-Dynamic (Ouyang D, et al. Nature. 2020;580:252–256) provides a useful reference point for rigorous external validation: a deep learning model that estimates ejection fraction beat by beat from echocardiogram video was tested against expert human measurements on a held-out Stanford set (mean absolute error 4.1%) and on an entirely separate Cedars-Sinai dataset (mean absolute error 6.0%), and a further prospective assessment showed that its repeat-measurement variance was comparable to or better than that of sonographers. The external, out-of-institution dataset is the critical feature, and it is the element that most published imaging-AI studies omit.

EchoNet-Dynamic’s own validation, two datasets

Test setWhat it wasMean absolute errorAUC
Stanford (held-out)Same institution as training data4.1%0.97
Cedars-Sinai (external)A wholly separate institution6.0%0.96

Source: Ouyang D, et al. Nature. 2020;580:252–256.

Common limitations of the published evidence

A systematic review of studies comparing deep learning against clinicians on medical imaging tasks (Nagendran M, et al. BMJ. 2020;368:m689) identified only 10 registered trials compared with 81 published non-randomised studies. Most of the non-randomised studies (61 of 81) reported AI performance comparable to or better than that of clinicians; few studies were prospective or conducted in a real clinical setting; and adherence to reporting standards was poor overall, with a generally high risk of bias. These findings do not indicate that the underlying technology is ineffective; rather, a claim in a single published paper provides, on its own, a weak basis for inferring that a tool will perform well in a new setting.

81published non-randomised studies found
10registered trials found, by contrast
61/81reported AI comparable to or better than clinicians

Source: Nagendran M, et al. BMJ. 2020;368:m689.

Implications for studies that use imaging AI tools

Considerations before relying on the output of an imaging AI tool

Establish what the tool was validated against.A held-out split of data from the same institution is a weaker test than an external dataset from a different site
Establish whether validation was prospective.Retrospective performance on previously acquired images does not provide the same evidence as performance on newly acquired scans
Select the appropriate reporting standard.TRIPOD+AI for a prediction model and STARD-AI for a diagnostic-accuracy claim; the choice depends on the function of the model, not on whether it uses AI
Report the population on which the tool was developed.Performance reported on one patient population does not transfer automatically to a different age range, disease prevalence or scanner type

Frequently asked questions

No. TRIPOD+AI covers prediction models generally (estimating a future outcome, or a patient’s current condition, from inputs that may include imaging). STARD-AI covers diagnostic-accuracy studies specifically (how well a test, including an AI-read image, distinguishes disease from its absence against a confirmed reference standard). The applicable standard depends on the claim the study makes, not on the involvement of AI.

No. This is a substantive methodological distinction rather than a technicality: a second-reader or triage tool that a clinician can override requires different validation from a tool that makes an unreviewed decision. Neither STARD-AI nor TRIPOD+AI treats these roles as interchangeable, and a study should state explicitly which role the tool performed.

References

  1. Collins GS, Moons KGM, Dhiman P, et al. "TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods." read 8 October 2026. BMJ. 2024;385:e078378
  2. Sounderajah V, Guni A, Liu X, et al. "The STARD-AI reporting guideline for diagnostic accuracy studies using artificial intelligence." read 8 October 2026. Nat Med. 2025
  3. Nagendran M, Chen Y, Lovejoy CA, et al. "Artificial intelligence versus clinicians: systematic review of design, reporting standards, and claims of deep learning studies." read 8 October 2026. BMJ. 2020;368:m689
  4. Ouyang D, He B, Ghorbani A, et al. "Video-based AI for beat-to-beat assessment of cardiac function." read 8 October 2026. Nature. 2020;580:252–256

Each source carries the date it was last read. Where a standard moves, this article is revised rather than left up with a note.

How to cite this article

Pick your referencing style and copy it. Your university or target journal may vary the punctuation — check its guide if it has one.

Pubrica Academy. (2026, October 8). Artificial Intelligence in Medical Imaging Research: Demonstrated Capabilities and Evidence Gaps. Pubrica. https://pubrica.com/academy/medical-writing/clinical-applications-of-machine-learning-in-radiology/

This article is written to be learned from. Cite it if you are referring to what it says; if you are using the evidence behind it, cite the sources listed above instead.

Pubrica AcademyWritten and maintained by Pubrica’s subject matter experts. Articles are revised when the underlying standards change, and each article states the date on which it was last reviewed.

How Pubrica can help

Pubrica’s biostatistics and medical writing teams determine which reporting standard applies to an AI-related claim before drafting, and write the methods and results sections to satisfy it item by item.

Find something else

Search the Pubrica Academy

Access is free and requires no account. Enter a topic, or open the full library to filter by category, sub-topic or resource type.

Confirming the reporting standard for studies using imaging AI tools

A scoping assessment is free and takes two business days. It provides a written assessment of what the work requires and which specialist would undertake it.

Get a free quote

Regulations

Check your project against this rule

  1. 1Your work
  2. 2Your details
  3. 3Send
What do you need?

Material you send is held under signed confidentiality, with access limited to the assigned team and encrypted transfer. It is not used to train anything and is not passed to third-party tools. Ask for the agreement before you send anything you would rather cover first.