Contents
On this page
- What actually changed, and what did not
- The accuracy evidence, task by task
- Where it loses, and why that case matters
- The labels are usually the constraint
- Shortcut learning, and why internal validation cannot find it
- Interpretability: useful, and not what it is usually claimed to be
- Generalisation across scanners, sites and time
- Reporting an imaging AI study
- The research issues that are actually open
- Take-home message
- How to apply this to your study
- Common questions
Academy · Research Data Analytics & AI · deep learning imaging challenges
Deep Learning Over Machine Learning in Medical Imaging: The Real Difficulties
Deep learning removed the feature-engineering bottleneck in medical imaging and moved the difficulty somewhere harder to see: into the labels, the confounds the network is free to exploit, and the gap between internal and external performance. The pooled accuracy figures are good. The variation between tasks is enormous, and on at least one clinically important task the human expert is still far ahead on specificity.
The pooled numbers everybody quotes come from Liu and colleagues in The Lancet Digital Health: deep learning at a pooled sensitivity of 87.0% and specificity of 92.5%, against health-care professionals on the same samples at 86.4% and 90.5%. Read as a single headline, that supports a story about parity. Read task by task, the same literature supports a different story. On low-dose chest CT, Wang and colleagues found standalone deep learning at 39% specificity against 82% for experts. Both findings are in the peer-reviewed record, and a manuscript that cites only the first is not reporting the evidence base.
What you will learn
- What actually differs between classical machine learning and deep learning on images
- What the pooled diagnostic accuracy evidence says, task by task, with its confidence intervals
- Where deep learning currently loses badly to human readers, and what that implies about task selection
- Why label quality and class imbalance limit performance more often than architecture does
- What shortcut learning is, how it is detected, and why it survives internal validation
- Which reporting standards apply to an imaging AI study, and what reviewers of these papers criticise
What to unlearn
- “Deep learning has overtaken radiologists.” On pooled estimates it is comparable rather than superior. Liu and colleagues reported deep learning at a pooled sensitivity of 87.0% (95% CI 83.0–90.2) and specificity of 92.5% (95% CI 85.1–96.4), against health-care professionals on the same samples at 86.4% (95% CI 79.9–91.0) and 90.5% (95% CI 80.6–95.7). Those confidence intervals overlap substantially.
- “Performance generalises across imaging tasks.” It does not, and the spread is wide. Abdullah and colleagues pooled deep learning on breast MRI to an AUC of 0.90 with 88% sensitivity and 90% specificity, while Lee and colleagues pooled axillary nodal staging on breast MRI to only 0.76 sensitivity and 0.81 specificity with a weighted AUC of 0.788 — the same modality, the same organ, a different question.
- “Deep learning is better than humans at ruling things out.” Sometimes the reverse, dramatically. Wang and colleagues, in European Radiology, found that pooled across all chest-CT tasks standalone deep learning and experts had statistically indistinguishable sensitivity (82% versus 81%, p=0.06) with deep learning higher on specificity (75% versus 68%, p<0.01) — but on low-dose CT specifically, deep learning reached 39% specificity against 82% for experts.
- “A model that validates internally has learned the clinical signal.” It may have learned something else that correlates with the label in your data. DeGrave, Janizek and Lee showed in Nature Machine Intelligence that AI for radiographic COVID-19 detection selects shortcuts over signal — and shortcuts survive internal validation by construction, because the correlation they exploit is present in the held-out split too.
- “More architecture is the answer to poor performance.” Usually the labels are. Greenwald and colleagues reached human-level whole-cell segmentation in Nature Biotechnology through large-scale data annotation as much as through modelling, and Escobar Diaz Guerrero and colleagues address class imbalance in histopathology images through data augmentation methodology rather than through network design.
What actually changed, and what did not
The difference between classical machine learning and deep learning in imaging is where the features come from. In a classical pipeline a human decides what to measure — shape, texture, intensity statistics, wavelet descriptors — and a classifier is fitted to those measurements. Aerts and colleagues’ radiomics work in Nature Communications is the canonical demonstration of that approach decoding tumour phenotype from quantitative image features, and Oikonomou, Siddique and Antoniades give the radiomic guide for cardiovascular phenotyping in Cardiovascular Research 32,50. In a deep learning pipeline the representation is learned from the pixels, so no one specifies what to measure. LeCun, Bengio and Hinton’s review in Nature remains the reference statement of why that works.
What this buys is the removal of a real bottleneck: performance is no longer capped by whether anyone thought of the right feature. What it costs is specific and under-discussed. The learned representation is not inspectable in the way a hand-crafted feature set is, so you cannot read off what the model is using. The data requirement rises substantially, because the representation has to be learned rather than supplied. And the model is free to use any pixel-level regularity that predicts the label, including ones that have nothing to do with the disease — which is the subject of the section on shortcuts below.
What did not change is everything downstream of the model output. Calibration, external validity, prevalence dependence, workflow integration and the comparison design are the same problems they were with a logistic regression on radiomic features. Kamal and colleagues survey deep learning models for medical image analysis, and Huang and colleagues review fully convolutional networks for semantic segmentation in Diagnostics; both are useful on what architectures do and silent, as architecture reviews generally are, on whether the resulting system helps anyone. Our pages on artificial neural networks in medical diagnosis and the steps in ANN-based medical diagnosis cover the mechanics.
Classical machine learning and deep learning on medical images
| Classical machine learning on engineered features | Deep learning on pixels | |
|---|---|---|
| Where features come from | Specified by a human — shape, texture, intensity, wavelet descriptors | Learned from the data as part of training |
| Performance ceiling | Capped by whether anyone thought of the right feature | Not capped that way; capped by labels, data and confounds instead |
| Data requirement | Moderate — the representation is supplied, not learned | Substantially higher, or a transfer or self-supervised strategy |
| Inspectability | The feature set is readable and each feature has a name | The representation is not readable; attribution methods give a partial view |
| Exposure to confounds | Limited to confounds present in the chosen features | Any pixel-level regularity that predicts the label is available to the model |
| What a random train-test split establishes | That the classifier generalises within the distribution | The same — and that is the problem, because shortcuts are inside the distribution |
| What is unchanged | Calibration, prevalence dependence, external validity, workflow integration, comparison design | Calibration, prevalence dependence, external validity, workflow integration, comparison design |
Our summary. The classical radiomic approach is exemplified by Aerts HJWL and colleagues, Nature Communications, and Oikonomou, Siddique and Antoniades, Cardiovascular Research; the learned-representation approach by LeCun, Bengio and Hinton, Nature.
The accuracy evidence, task by task
The single most useful thing a researcher in this area can do is stop quoting the aggregate and start quoting the task. Liu and colleagues’ systematic review and meta-analysis in The Lancet Digital Health gives the aggregate: pooled sensitivity 87.0% (95% CI 83.0–90.2) and specificity 92.5% (95% CI 85.1–96.4) for deep learning, with health-care professionals on the same samples at 86.4% (95% CI 79.9–91.0) and 90.5% (95% CI 80.6–95.7) 39. The honest reading of those four numbers with their intervals is comparability, not superiority 39.
Task-level syntheses fill in the variation. On chest CT, Wang and colleagues found pooled sensitivity of 82% for standalone deep learning against 81% for experts, statistically indistinguishable at p=0.06, with specificity of 75% against 68% at p<0.01 13. On breast MRI for cancer diagnosis, Abdullah and colleagues reviewed 40 studies, of which 21 entered the meta-analysis, pooling to an AUC of 0.90 with 88% sensitivity and 90% specificity 7. On glioma molecular markers from MRI, Farahani and colleagues reviewed 104 studies and pooled 72, reporting sensitivity and specificity of 0.80 and 0.85 for IDH mutation and 0.75 and 0.82 for 1p/19q co-deletion 3. And on axillary nodal staging from breast MRI, Lee and colleagues pooled 10 studies to a sensitivity of 0.76, specificity of 0.81 and weighted AUC of 0.788 6.
Two patterns are worth extracting. The first is that task difficulty dominates: breast cancer diagnosis on MRI and axillary nodal staging on MRI differ by more than a tenth in sensitivity, with the same modality and overlapping study populations 6,7. The second is a structural observation about the evidence base rather than the models — in both of the larger reviews, well under half of the identified studies could be pooled (21 of 40 and 72 of 104), which means the meta-analytic estimates rest on the subset of studies reporting usably. That selection is not random, and it almost certainly favours the better-performing and better-reported work. See meta-analysis of convolutional neural networks for radiological images and meta-analysis in cardiovascular imaging AI for how these syntheses are constructed.
Pooled diagnostic accuracy of deep learning, by task
| Task and source | Sensitivity | Specificity | Studies | Comparator, where reported |
|---|---|---|---|---|
| Detecting disease from medical imaging, all tasks pooled — Liu and colleagues, The Lancet Digital Health | 87.0% (95% CI 83.0–90.2) | 92.5% (95% CI 85.1–96.4) | — | Health-care professionals on the same samples: 86.4% (79.9–91.0) and 90.5% (80.6–95.7) |
| Lung cancer on chest CT, all tasks pooled — Wang and colleagues, European Radiology | 82% | 75% | — | Experts: 81% sensitivity (p=0.06) and 68% specificity (p<0.01) |
| Lung cancer on low-dose chest CT — Wang and colleagues, European Radiology | — | 39% | — | Experts: 82% specificity |
| Breast cancer diagnosis on breast MRI — Abdullah and colleagues, European Radiology | 88% | 90% | 40 reviewed, 21 pooled | Pooled AUC 0.90 |
| IDH mutation on MRI in glioma — Farahani and colleagues, European Radiology | 0.80 | 0.85 | 104 reviewed, 72 pooled | — |
| 1p/19q co-deletion on MRI in glioma — Farahani and colleagues, European Radiology | 0.75 | 0.82 | 104 reviewed, 72 pooled | — |
| Axillary lymph node metastasis on breast MRI — Lee and colleagues, Cancer Imaging | 0.76 | 0.81 | 10 pooled | Weighted AUC 0.788 |
Figures exactly as reported in each systematic review and meta-analysis. They are not directly comparable: tasks, reference standards, populations and the proportion of identified studies eligible for pooling all differ. Note that in two of the reviews, well under half of the identified studies could be pooled — 21 of 40 and 72 of 104 — so the pooled estimates rest on the subset reporting usably.
Comparable, with overlapping intervals — not superior. Figures from Liu X, Faes L, Kale AU, et al. A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis. The Lancet Digital Health.
Where it loses, and why that case matters
The most instructive single result in this literature is a failure. Wang and colleagues found that on low-dose CT, standalone deep learning reached a specificity of 39% against 82% for human experts 13. Pooled across all chest-CT tasks in the same review, deep learning had the higher specificity — 75% against 68%. The direction of the comparison reverses within one review, on one modality, between protocols 13.
Why this happens is worth thinking through, because the mechanism generalises. Low-dose acquisition is noisier, and noise produces exactly the kind of local texture a network can mistake for a finding. A human reader has a model of what the noise looks like and discounts it; a network trained predominantly on standard-dose images does not. The result is a flood of false positives, which is what a 39% specificity means in practice: on a screening population, the majority of flagged studies would be normal.
Three lessons follow for anyone designing or writing up an imaging AI study. Acquisition protocol is part of the task definition, not a technical detail — a model validated on standard-dose images has not been validated for low-dose use. Specificity failures are more dangerous than sensitivity failures in screening, because they generate downstream investigation in healthy people, and a paper reporting only sensitivity or only AUC conceals this. And a reversal of this size inside a single review is a strong argument for reporting your own results by protocol and subgroup rather than pooling them into one figure.
The comparison reverses by acquisition protocol within a single review. Figures from Wang TW, Hong JS, Chiu HY, Chao HS, Chen YM, Wu YT. Standalone deep learning versus experts for diagnosis lung cancer on chest computed tomography: a systematic review. European Radiology. Pooled across all chest-CT tasks in the same review, deep learning had higher specificity than experts, 75% against 68%.
The labels are usually the constraint
Teams reach for a better architecture when performance plateaus. In medical imaging the ceiling is more often the label set, and the reasons are structural rather than careless.
Annotation at the scale deep learning wants is expensive and requires the people whose time is scarcest. Greenwald and colleagues’ whole-cell segmentation work in Nature Biotechnology is explicit that human-level performance came from large-scale data annotation alongside the modelling — the annotation effort is the contribution as much as the network is 24. Where exhaustive annotation is impossible, the alternatives have their own characteristics: Krishnan, Rajpurkar and Topol review self-supervised learning in medicine and health care in Nature Biomedical Engineering, and Pinckaers, van Ginneken and Litjens address the specific problem of end-to-end learning on multi-megapixel images in IEEE Transactions on Pattern Analysis and Machine Intelligence, where a single image cannot fit a conventional training step at all 23.
Three label problems recur and each needs stating in a methods section. Label noise. If the reference standard is a clinical report rather than an adjudicated panel, the model is being trained towards the reporting radiologist’s judgement, including its errors, and its apparent ceiling is that reader’s accuracy. Class imbalance. Disease prevalence in a clinical archive is low, and a model trained on it will learn the base rate unless something is done; Escobar Diaz Guerrero and colleagues set out an augmentation methodology to reduce class imbalance in histopathology images in the Journal of Imaging Informatics in Medicine, and whatever approach is taken has to be reported because it changes what the output probabilities mean 11. Label granularity. A study-level label on a volumetric scan tells the model that something is somewhere, which is a much weaker supervision signal than a localised annotation and produces models that are correspondingly harder to interrogate. AI data preparation is where most of this work actually happens.
Shortcut learning, and why internal validation cannot find it
A network minimises error on the training distribution. If a non-clinical feature predicts the label in your data, using it is not a malfunction — it is the correct solution to the problem as posed.
DeGrave, Janizek and Lee demonstrated this concretely in Nature Machine Intelligence: AI for radiographic COVID-19 detection selects shortcuts over signal 25. The general shape of the problem is that cases and controls in a medical imaging dataset often come from different sources, and anything that differs systematically between those sources — scanner model, acquisition protocol, patient positioning, text markers and annotations burned into the image, image processing pipeline, even the proportion of the field of view occupied by the body — is available as a predictor.
The critical point for study design is that a random train-test split cannot detect this 25,41. The shortcut is present on both sides of the split, so held-out performance is excellent and the model is worthless. The things that do detect it are specific: splitting by site, scanner or time rather than at random; testing on data from a source that contributed nothing to training; examining saliency or attribution maps for attention outside the anatomy of interest; and deliberately perturbing or removing the suspected shortcut to see whether performance collapses. Dinsdale, Jenkinson and Namburete, in NeuroImage, take the active approach, using deep learning to unlearn dataset bias for MRI harmonisation and confound removal, which is the constructive version of the same insight 27. Bias and fairness review is where we look for these.
Interpretability: useful, and not what it is usually claimed to be
Interpretability methods are now standard in imaging AI papers, and it is worth being precise about what they provide. An attribution map tells you which pixels most influenced this output for this input. It does not tell you why, it does not constitute a clinical explanation, and it is not a validation.
The methods are well developed. Bach and colleagues introduced layer-wise relevance propagation in PLOS ONE; Böhle and colleagues applied it to MRI-based Alzheimer’s disease classification in Frontiers in Aging Neuroscience and Eitel and colleagues to multiple sclerosis diagnosis in NeuroImage: Clinical 42,44,49. Chen, Bei and Rudin’s concept whitening in Nature Machine Intelligence and Erion and colleagues’ axiomatic attribution priors in the same journal both build interpretability into training rather than applying it afterwards, which is the more defensible construction 30,34. Biffi and colleagues do explainable anatomical shape analysis through hierarchical generative models in IEEE Transactions on Medical Imaging; Faust and colleagues visualise histopathological classification and anomaly detection through non-linear dimensionality reduction in BMC Bioinformatics; Ghorbani and colleagues interpret echocardiograms in npj Digital Medicine; Camalan and colleagues use class activation map analysis for oral dysplasia prediction in Cancers. Barredo Arrieta and colleagues survey the field in Information Fusion, and Carvalho, Pereira and Cardoso review methods and metrics in Electronics.
Two cautions. First, the epistemic one: Durán and Jongsma argue in the Journal of Medical Ethics that the basis for trusting medical AI is not primarily transparency, and Gerdes, in Discover Artificial Intelligence, examines what role explainability actually plays in AI-supported medical decision-making 14,31. An interpretable model that is wrong is still wrong. Second, the practical one: attribution maps are most valuable as a debugging tool for exactly the shortcut problem above — attention outside the relevant anatomy is strong evidence of a confound — and least valuable as a figure asserting that the model is trustworthy. Use them for the first purpose and say so.
Generalisation across scanners, sites and time
Medical images are not standardised objects. The same patient on two scanners produces two different pixel arrays, and a model fitted to one manufacturer’s characteristics has partly fitted the manufacturer.
This is why harmonisation is a research area in its own right rather than a preprocessing step. Dinsdale, Jenkinson and Namburete’s unlearning approach in NeuroImage removes site-related information from a learned representation deliberately. Zhang and colleagues, in Diagnostics, address informative frame selection through manifold learning cascade-clustering, which is the related problem of which images in a sequence are usable at all 15. Abu and colleagues analyse transfer learning optimisation for visual field defect classification in the same journal, and Santosh and colleagues review five years of deep learning for tuberculosis screening on chest X-rays in the Journal of Medical Systems — a task where deployment settings differ from development settings about as much as they can 18,19.
Multi-site training is the other route and now has a demonstrated privacy-preserving form: Kaissis and colleagues, in Nature Machine Intelligence, show end-to-end privacy preserving deep learning on multi-institutional medical imaging 29. It is worth being clear about what that solves. It makes training across institutions possible where pooling raw images is not permitted. It does not make the participating institutions’ populations, scanners or protocols representative of a deployment site that was not in the federation.
For a methods section the requirements are narrow and rarely met. Report scanner manufacturers, models and acquisition parameters for development and for each validation set. Split by site and by time, not at random, and say that you did. Report performance per site rather than pooled, because a pooled figure hides the site where the model does not work. And state the data period, because protocols and equipment change. Kelly, Karthikesalingam, Suleyman, Corrado and King’s framing in BMC Medicine is the general statement of why this matters; our page on the challenges of delivering clinical impact with AI follows it through.
Reporting an imaging AI study
Reviewers of these manuscripts raise the same issues repeatedly, which is itself a finding: Gregory, Welliver and Chong catalogued the top reviewer critiques of radiology artificial intelligence articles through a qualitative thematic analysis of reviewer critiques of machine learning and deep learning manuscripts submitted to the Journal of Magnetic Resonance Imaging 38. The existence of a stable list of recurring critiques means most of these objections are foreseeable and therefore avoidable.
Use the standard that matches the study. CLAIM — the Checklist for Artificial Intelligence in Medical Imaging, published by Mongan, Moy and Kahn in Radiology: Artificial Intelligence — is the imaging-specific guide for authors and reviewers and is the one to work from for a model development or evaluation paper in imaging 33. Alongside it: TRIPOD+AI for prediction model development and validation, STARD-AI for a diagnostic accuracy study, CONSORT-AI and SPIRIT-AI for a randomised trial of an imaging AI intervention and its protocol, DECIDE-AI for an early-stage clinical evaluation, and PROBAST+AI where you are appraising other people’s models in a review.
Four items carry most of the credibility. The data description: sources, scanners, protocols, dates, and what the reference standard was and who established it. The split: by site or time rather than at random, with the reason stated. The metrics: sensitivity and specificity at a prospectively chosen operating point, with calibration, rather than AUC alone — and reported by subgroup and acquisition protocol, because of the low-dose CT case above. And the comparison: if human readers are involved, how many, on which sample, blinded to what, with what clinical information. The Liu and colleagues review is the standing reminder of why that last item needs stating: comparisons on the same sample are the minority 39. Reporting checklist review and statistical review are where this is checked.
Which standard to report an imaging AI study against
| Study type | Standard | The item it is most often used to catch |
|---|---|---|
| Development or evaluation of an imaging AI model | CLAIM — Mongan, Moy and Kahn, Radiology: Artificial Intelligence | Incomplete data and reference-standard description; undefined train, tune and test partitions |
| Development or validation of a clinical prediction model | TRIPOD+AI | Discrimination reported without calibration; intended use left unstated |
| Diagnostic accuracy evaluation | STARD-AI | Reference standard and index-test version; the sample on which both were applied |
| Randomised trial of an imaging AI intervention | CONSORT-AI, with SPIRIT-AI for the protocol | How the AI output entered the reporting pathway, and handling of poor-quality or missing input |
| Early-stage clinical evaluation in a reading workflow | DECIDE-AI | Human factors, reader interaction with the output, safety and implementation detail |
| Systematic review appraising others’ models | PROBAST+AI | Risk of bias and applicability, judged domain by domain |
Standards as published by their development groups. CLAIM is cited to Mongan J, Moy L, Kahn CE Jr. Checklist for Artificial Intelligence in Medical Imaging (CLAIM): a guide for authors and reviewers. Radiology: Artificial Intelligence. The recurring reviewer critiques are catalogued by Gregory J, Welliver S, Chong J in the Journal of Magnetic Resonance Imaging.
The data and validation paragraph of an imaging AI methods section
Unauditable
A total of 8,400 images were collected retrospectively from our institutional archive and randomly divided into training (70%), validation (15%) and test (15%) sets. The model was trained until convergence and evaluated on the test set, achieving an AUC of 0.96. Ground truth was established from the clinical radiology reports.
Random split. This cannot detect a shortcut, because any site, scanner or protocol artefact that predicts the label is present on both sides of the split — which is exactly how DeGrave and colleagues’ shortcut models pass internal validation.
No scanner, protocol or date information. The reader cannot tell whether the model has learned the disease or the equipment, and cannot judge transfer to their own setting.
Reference standard is the clinical report. The model is being trained towards one reader’s judgement including its errors, so its ceiling is that reader — and this is stated as though it were ground truth.
Patient-level leakage not excluded. With images rather than studies as the unit and a random split, the same patient can appear in training and test, which inflates performance.
AUC alone. No operating point, no sensitivity or specificity, no calibration — so nothing about how the model would behave in use, and no visibility of the low-specificity failure mode.
No prevalence statement. Predictive values depend on prevalence, and a model trained at archive prevalence will behave differently in screening.
Auditable against CLAIM
We retrospectively collected 8,400 studies acquired between January 2017 and June 2022 at four sites, on scanners from three manufacturers; manufacturer, model and the acquisition parameters relevant to this task are given per site in Supplementary Table 1. Partitioning was by site and by time rather than at random: sites 1 to 3 (acquired to December 2020) formed the development set, which was further split by patient into training and tuning partitions with no patient appearing in both; site 4, and all studies at sites 1 to 3 acquired from January 2021, formed two held-out test sets evaluated separately. The reference standard was histopathology where available (n reported per set) and otherwise an adjudicated read by two radiologists blinded to the model output, with a third resolving disagreement; the proportion established by each route is reported per set, because they are not equivalent. The operating point was fixed on the tuning partition before any test-set evaluation. We report sensitivity, specificity and calibration at that operating point for each test set separately, and stratified by acquisition protocol, alongside AUC; results are not pooled across test sets. Class prevalence in each set is reported, and because it differs from the prevalence in the intended screening population we additionally report predicted values recalibrated to that prevalence. Attribution maps were inspected on a random sample of true positives to check for attention outside the relevant anatomy; cases where this occurred are described in the Supplement.
The split is by site and by time, with a held-out site, which is the design that can actually detect shortcut reliance and site overfitting.
Scanners, protocols and dates are reported per site, so a reader can judge whether their own equipment is represented.
The reference standard is disaggregated by how it was established, with the proportions given — histopathology and an adjudicated read are not interchangeable and the paper says so.
Patient-level separation is stated explicitly, closing the commonest source of inflated performance in imaging work.
Sensitivity, specificity and calibration are reported at a prospectively fixed operating point, per test set and by acquisition protocol, which is what the low-dose CT reversal shows is necessary.
Prevalence is reported and predictive values are recalibrated to the intended-use prevalence, so the numbers mean something in the setting the model is for.
The attribution check is reported as a debugging step with its negative findings included, which is the honest use of interpretability rather than the decorative one.
The second version is longer by one paragraph and converts an unfalsifiable AUC into a set of claims a reader can check and a purchaser could act on. Every element of it is something the team already knows; the only cost is writing it down.
The research issues that are actually open
Worth separating from the solved and the fashionable, because a discussion section that identifies a genuine open problem reads very differently from one that lists challenges.
Label-efficient learning that works clinically. Self-supervised and weakly supervised methods are the most promising direction for the annotation bottleneck, and the open question is not whether they learn useful representations but whether the representations transfer to tasks and populations the pretraining data did not cover.
Detecting shortcut reliance before deployment, routinely. The methods exist in research settings. There is no standard, auditable procedure a developer must pass, and a cleared product gives a purchaser no information about whether it was done.
Prospective evaluation at the decision point. Almost all of the accuracy evidence above is retrospective and offline. DECIDE-AI exists because the stage between offline accuracy and a definitive trial has its own questions — how the output changes a reader’s judgement, what happens when the model is unavailable or the input is poor quality, and how performance drifts in service.
Calibration and prevalence transfer. A model’s predicted probabilities depend on the prevalence it was trained at, and screening prevalence differs from clinical prevalence by orders of magnitude. This is well understood statistically and routinely unreported in imaging papers.
Harmonisation that does not discard signal. Removing site information improves transfer and may remove clinically relevant variation along with it. The trade-off is currently handled case by case.
Topol’s survey in Nature Medicine and Lin and colleagues’ account of artificial intelligence in cardiovascular imaging in BJR|Open are useful on where the field believes it is going 16,40. The gap between that and the five items above is roughly the gap this article has been about. See a systematic review of artificial intelligence in imaging and machine learning in computational pathology for adjacent evidence.
Take-home message
Take-home message
Deep learning replaced feature engineering with representation learning and moved the hard problems into labels, confounds and generalisation. The pooled evidence supports comparability with human readers rather than superiority: Liu and colleagues reported deep learning at 87.0% sensitivity (95% CI 83.0–90.2) and 92.5% specificity (95% CI 85.1–96.4) against health-care professionals on the same samples at 86.4% (79.9–91.0) and 90.5% (80.6–95.7). Task variation dwarfs that difference — breast MRI diagnosis pools to 88% and 90% across 21 of 40 studies, glioma IDH status to 0.80 and 0.85 from 72 of 104, axillary nodal staging to only 0.76 and 0.81 — and on low-dose chest CT standalone deep learning reached 39% specificity against 82% for experts, a reversal inside a single review. The methodological consequences: a random train-test split cannot detect shortcut learning, because the shortcut is inside the distribution; labels, not architecture, usually set the ceiling; and reporting has to be by site, protocol and operating point, with calibration, against CLAIM and the AI-specific reporting standards.
How to apply this to your study
How to apply this to your own study
- Split by site and by time, never at random, and say so. Hold out at least one source that contributed nothing to training. A random split cannot distinguish a model that learned the disease from one that learned the scanner.
- Describe the reference standard by how it was established. Histopathology, adjudicated read, single clinical report — with the proportion from each route per dataset. If it is the clinical report, say that the ceiling is that reader’s accuracy.
- Report sensitivity, specificity and calibration at a prospectively fixed operating point. Per test set and per acquisition protocol, not pooled. AUC alone conceals exactly the low-specificity failure the low-dose CT result exposes.
- Check for shortcuts deliberately, and report the check. Attribution maps inspected for attention outside the anatomy, and a perturbation test that removes the suspected confound to see whether performance collapses.
- State prevalence in every set, and recalibrate to intended-use prevalence. Predictive values are prevalence-dependent, and archive prevalence is not screening prevalence.
- Draft against CLAIM, plus the standard for your study type. TRIPOD+AI, STARD-AI, CONSORT-AI with SPIRIT-AI, DECIDE-AI, or PROBAST+AI for a review. The recurring reviewer critiques in this field are a known list, which means they are avoidable.
Common questions
Where the features come from. In classical machine learning a human specifies what to measure in the image — shape, texture, intensity statistics, wavelet descriptors — and a classifier is fitted to those measurements; Aerts and colleagues’ radiomics work in Nature Communications is the reference example. In deep learning the representation is learned directly from the pixels, so performance is no longer capped by whether anyone thought of the right feature. The costs are a substantially higher data requirement, a representation that is not directly inspectable, and exposure to any pixel-level regularity that happens to predict the label in your data. Everything downstream — calibration, external validity, prevalence dependence, workflow integration, comparison design — is unchanged.
On pooled estimates, comparable rather than better, with overlapping confidence intervals. Liu and colleagues, in The Lancet Digital Health, reported deep learning at a pooled sensitivity of 87.0% (95% CI 83.0–90.2) and specificity of 92.5% (95% CI 85.1–96.4), with health-care professionals on the same samples at 86.4% (95% CI 79.9–91.0) and 90.5% (95% CI 80.6–95.7). Task-level results vary much more than that difference: Wang and colleagues found pooled chest-CT sensitivity of 82% for deep learning against 81% for experts with specificity of 75% against 68%, while on low-dose CT specifically deep learning reached 39% specificity against 82%. The aggregate answer is therefore less useful than the task-specific one, and “as accurate as radiologists” is not a property any system has in general.
It is a model solving your task using a feature that predicts the label in your data but has nothing to do with the disease — scanner model, acquisition protocol, patient positioning, burned-in text markers, image processing pipeline, how much of the field of view the body occupies. DeGrave, Janizek and Lee demonstrated it for radiographic COVID-19 detection in Nature Machine Intelligence. The crucial point is that a random train-test split cannot detect it, because the shortcut is present on both sides. What detects it: splitting by site, scanner or time; testing on a source that contributed nothing to training; inspecting attribution maps for attention outside the relevant anatomy; and perturbing or removing the suspected confound to see whether performance collapses. Dinsdale and colleagues’ unlearning approach in NeuroImage addresses it during training.
Usually because it partly learned your hospital. Medical images are not standardised objects: the same patient on two scanners yields different pixel arrays, and acquisition protocols, reconstruction pipelines, patient populations and disease prevalence all differ between sites. A model fitted to one site’s characteristics has fitted some of those characteristics along with the disease. The responses are harmonisation — Dinsdale, Jenkinson and Namburete deliberately unlearn site information in NeuroImage — and multi-site training, which Kaissis and colleagues showed can be done in privacy-preserving form in Nature Machine Intelligence. Neither makes a federation representative of a site outside it. The reporting requirement is to give scanner, protocol and date information per set and to report performance per site rather than pooled.
There is no transferable number, and the more useful framing is that labels rather than architecture usually set the ceiling. Greenwald and colleagues reached human-level whole-cell segmentation in Nature Biotechnology through large-scale annotation as much as through modelling. Where exhaustive annotation is impossible, Krishnan, Rajpurkar and Topol review self-supervised learning in Nature Biomedical Engineering, and weakly supervised approaches work from study-level labels at the cost of a weaker supervision signal. Three label properties matter as much as quantity: noise, which caps performance at the accuracy of whoever produced the reference standard; class imbalance, since archive prevalence is low and must be handled and reported; and granularity, because a study-level label on a volume teaches the model much less than a localised one.
No, and it is worth being precise about what they do. An attribution or saliency map tells you which pixels most influenced this output for this input. It does not explain why, it is not a clinical justification, and it is not a validation — an interpretable model that is wrong is still wrong. Durán and Jongsma argue in the Journal of Medical Ethics that the basis for trusting medical AI is not primarily transparency, and Gerdes examines what role explainability actually plays in decision-making. Where these methods genuinely earn their place is as a debugging tool: attention falling outside the relevant anatomy is strong evidence of a confound. Methods that build interpretability into training, such as Chen, Bei and Rudin’s concept whitening or Erion and colleagues’ attribution priors, are more defensible than post hoc maps.
Start with CLAIM, the Checklist for Artificial Intelligence in Medical Imaging, published by Mongan, Moy and Kahn in Radiology: Artificial Intelligence as a guide for authors and reviewers — it is the imaging-specific standard for a model development or evaluation study. Then add the standard matching your design: TRIPOD+AI for prediction model development and validation, STARD-AI for a diagnostic accuracy evaluation, CONSORT-AI with SPIRIT-AI for a randomised trial of an imaging AI intervention and its protocol, DECIDE-AI for an early-stage clinical evaluation in a reading workflow, and PROBAST+AI if you are appraising other people’s models in a review. Gregory, Welliver and Chong’s thematic analysis of reviewer critiques of these manuscripts is worth reading alongside, because the objections recur and are therefore predictable.
Because a specificity failure generates investigation, anxiety and cost in people who do not have the disease, and at screening prevalence most of the people a model flags are healthy. The low-dose CT result makes this concrete: Wang and colleagues found standalone deep learning at 39% specificity against 82% for experts on that protocol, which in a screening population means the majority of flagged studies would be normal. A paper reporting AUC alone, or sensitivity alone, conceals this entirely. The practical requirements are to report sensitivity and specificity together at a prospectively chosen operating point, to report them by acquisition protocol and subgroup rather than pooled, and to state the prevalence in each dataset alongside predictive values recalibrated to the intended-use prevalence.
References and sources
- Basile C, Villaschi A, Ambrosino P, Canonico ME, Rawshani A, Bobbio E. Artificial intelligence in cardiovascular medicine: a practical guide for clinicians and researchers. European Heart Journal - Digital Health. 2026;7(8):ztag153. doi:10.1093/ehjdh/ztag153 read 9 October 2026. doi.org/10.1093/ehjdh/ztag153
- Aladwani F, Perelli A, Mordi I, Khan F. Deep learning for cardiac MRI: performance evidence and barriers to clinical integration. A Systematic Review and Meta-Analysis. European Heart Journal - Imaging Methods and Practice. 2026;4(1):qyag045. doi:10.1093/ehjimp/qyag045 read 9 October 2026. doi.org/10.1093/ehjimp/qyag045
- Farahani S, Hejazi M, Tabassum M, Di Ieva A, Mahdavifar N, et al. , Liu S, et al. Diagnostic performance of deep learning for predicting glioma isocitrate dehydrogenase and 1p/19q co-deletion in MRI: a systematic review and meta-analysis. European Radiology. 2026;36(2):1562-1591. doi:10.1007/s00330-025-11898-2 read 9 October 2026. doi.org/10.1007/s00330-025-11898-2
- Yang Z, Wei T, Liang Y, Yuan X, Gao R, Xia Y, Zhou J, Zhang Y, et al. , Yu Z, et al. A foundation model for generalizable cancer diagnosis and survival prediction from histopathological images. Nature Communications. 2025;16(1):2366. doi:10.1038/s41467-025-57587-y read 9 October 2026. doi.org/10.1038/s41467-025-57587-y
- Ding T, Wagner SJ, Song AH, Chen RJ, Lu MY, Zhang A, Vaidya AJ, Jaume G, Shaban M, Kim A, et al. , Mahmood F, et al. A multimodal whole-slide foundation model for pathology. Nature Medicine. 2025;31(11):3749-3761. doi:10.1038/s41591-025-03982-3 read 9 October 2026. doi.org/10.1038/s41591-025-03982-3
- Lee CF, Lin J, Huang YL, Chen ST, Chou CT, Chen DR, et al. , Wu WP, et al. Deep learning-based breast MRI for predicting axillary lymph node metastasis: a systematic review and meta-analysis. Cancer Imaging. 2025;25(1):44. doi:10.1186/s40644-025-00863-3 read 9 October 2026. doi.org/10.1186/s40644-025-00863-3
- Abdullah KA, Marziali S, Nanaa M, Escudero Sanchez L, Payne NR, et al. , Gilbert FJ, et al. Deep learning-based breast cancer diagnosis in breast MRI: systematic review and meta-analysis. European Radiology. 2025;35(8):4474-4489. doi:10.1007/s00330-025-11406-6 read 9 October 2026. doi.org/10.1007/s00330-025-11406-6
- Du S, Guo J, Huang D, Liu Y, Zhang X, Lu S. Diagnostic accuracy of deep learning-based algorithms in laryngoscopy: a systematic review and meta-analysis. European Archives of Oto-Rhino-Laryngology. 2025;282(1):351-360. doi:10.1007/s00405-024-09049-2 read 9 October 2026. doi.org/10.1007/s00405-024-09049-2
- Valizadeh P, Jannatdoust P, Pahlevan-Fallahy MT, Hassankhani A, Amoukhteh M, Bagherieh S, Ghadimi DJ, et al. , Gholamrezanezhad A, et al. Diagnostic accuracy of radiomics and artificial intelligence models in diagnosing lymph node metastasis in head and neck cancers: a systematic review and meta-analysis. Neuroradiology. 2025;67(2):449-467. doi:10.1007/s00234-024-03485-x read 9 October 2026. doi.org/10.1007/s00234-024-03485-x
- Yan F, Wu J, Li J, Wang W, Chen Y, Wei L, Lu J, Chen W, Gao Z, Li J, et al. , Wang Z, et al. PathOrchestra: a comprehensive foundation model for computational pathology with over 100 diverse clinical-grade tasks. npj Digital Medicine. 2025;8(1):695. doi:10.1038/s41746-025-02027-w read 9 October 2026. doi.org/10.1038/s41746-025-02027-w
- Escobar Diaz Guerrero R, Carvalho L, Bocklitz T, et al, et al. A data augmentation methodology to reduce the class imbalance in histopathology images. Journal of Imaging Informatics in Medicine. 2024. doi:10.1007/s10278-024-01018-9 read 9 October 2026. doi.org/10.1007/s10278-024-01018-9
- Xu H, Usuyama N, Bagga J, Zhang S, Rao R, Naumann T, Wong C, Gero Z, Gonzalez J, Gu Y, et al. , Poon H, et al. A whole-slide foundation model for digital pathology from real-world data. Nature. 2024;630(8015):181-188. doi:10.1038/s41586-024-07441-w read 9 October 2026. doi.org/10.1038/s41586-024-07441-w
- Wang TW, Hong JS, Chiu HY, Chao HS, Chen YM, Wu YT. Standalone deep learning versus experts for diagnosis lung cancer on chest computed tomography: a systematic review. European Radiology. 2024;34(11):7397-7407. doi:10.1007/s00330-024-10804-6 read 9 October 2026. doi.org/10.1007/s00330-024-10804-6
- Gerdes A. The role of explainability in AI-supported medical decision-making. Discover Artificial Intelligence. 2024;4(1):29. doi:10.1007/s44163-024-00119-2 read 9 October 2026. doi.org/10.1007/s44163-024-00119-2
- Zhang L, Wu L, Wei L, Wu H, Lin Y. A Novel Framework of Manifold Learning Cascade-Clustering for the Informative Frame Selection. Diagnostics. 2023;13(6):1151. doi:10.3390/diagnostics13061151 read 9 October 2026. doi.org/10.3390/diagnostics13061151
- Lin A, Pieszko K, Park C, Ignor K, Williams MC, Slomka P, Dey D. Artificial intelligence in cardiovascular imaging: enhancing image analysis and risk stratification. BJR|Open. 2023;5(1):20220021. doi:10.1259/bjro.20220021 read 9 October 2026. doi.org/10.1259/bjro.20220021
- Kamal S, Lazaar M, Bennani Othmani M, Bourzgui F, Mahboub O. Deep Learning Models for Medical Image Analysis in Smart Healthcare System: A Review. Lecture Notes in Networks and Systems (Proceedings of the 6th International Conference on Big Data and Internet of Things, BDIoT 2022), Springer. 2023;625:376-387. doi:10.1007/978-3-031-28387-1_32 read 9 October 2026. doi.org/10.1007/978-3-031-28387-1_32
- Abu M, Zahri NAH, Amir A, Ismail MI, Yaakub A, Anwar SA, Ahmad MI. A Comprehensive Performance Analysis of Transfer Learning Optimization in Visual Field Defect Classification. Diagnostics. 2022;12(5):1258. doi:10.3390/diagnostics12051258 read 9 October 2026. doi.org/10.3390/diagnostics12051258
- Santosh K, Allu S, Rajaraman S, Antani S. Advances in deep learning for tuberculosis screening using chest X-rays: the last 5 years review. Journal of Medical Systems. 2022;46(11):82. doi:10.1007/s10916-022-01870-8 read 9 October 2026. doi.org/10.1007/s10916-022-01870-8
- Mao LK, Huang MH, Lai CH, Sun YN, Chen CY. Detecting Endotracheal Tube and Carina on Portable Supine Chest Radiographs Using One-Stage Detector with a Coarse-to-Fine Attention. Diagnostics. 2022;12(8):1913. doi:10.3390/diagnostics12081913 read 9 October 2026. doi.org/10.3390/diagnostics12081913
- Nuvoli S, Bianconi F, Rondini M, Lazzarato A, Marongiu A, Fravolini ML, Cascianelli S, Amici S, Filippi L, Spanu A, Palumbo B. Differential Diagnosis of Alzheimer Disease vs. Mild Cognitive Impairment Based on Left Temporal Lateral Lobe Hypomethabolism on 18F-FDG PET/CT and Automated Classifiers. Diagnostics. 2022;12(10):2425. doi:10.3390/diagnostics12102425 read 9 October 2026. doi.org/10.3390/diagnostics12102425
- Huang SY, Hsu WL, Hsu RJ, Liu DW. Fully Convolutional Network for the Semantic Segmentation of Medical Images: A Survey. Diagnostics. 2022;12(11):2765. doi:10.3390/diagnostics12112765 read 9 October 2026. doi.org/10.3390/diagnostics12112765
- Krishnan R, Rajpurkar P, Topol EJ. Self-supervised learning in medicine and healthcare. Nature Biomedical Engineering. 2022. read 9 October 2026. arxiv.org
- Greenwald NF, Miller G, Moen E, Kong A, Kagel A, Dougherty T, Fullaway CC, McIntosh BJ, Leow KX, Schwartz MS, Pavelchek C, Cui S, Camplisson I, Bar-Tal O, Singh J, Fong M, Chaudhry G, Abraham Z, Moseley J, Warshawsky S, Soon E, Greenbaum S, Risom T, Hollmann T, Bendall SC, Keren L, Graf W, Angelo M, Van Valen D. Whole-cell segmentation of tissue images with human-level performance using large-scale data annotation and deep learning. Nature Biotechnology. 2022;40:555-65. read 9 October 2026. arxiv.org
- DeGrave AJ, Janizek JD, Lee SI. AI for radiographic COVID-19 detection selects shortcuts over signal. Nature Machine Intelligence. 2021;3:610-19. doi:10.1038/s42256-021-00338-7 read 9 October 2026. doi.org/10.1038/s42256-021-00338-7
- Camalan S, Mahmood H, Binol H, Araujo ALD, Santos-Silva AR, Vargas PA, Lopes MA, Khurram SA, Gurcan MN. Convolutional neural network-based clinical predictors of oral dysplasia: class activation map analysis of deep learning results. Cancers. 2021;13. doi:10.3390/cancers13061291 read 9 October 2026. doi.org/10.3390/cancers13061291
- Dinsdale NK, Jenkinson M, Namburete AIL. Deep learning-based unlearning of dataset bias for MRI harmonisation and confound removal. NeuroImage. 2021;228:117689. doi:10.1016/j.neuroimage.2020.117689 read 9 October 2026. doi.org/10.1016/j.neuroimage.2020.117689
- Gamble P, Jaroensri R, Wang H, Tan F, Moran M, Brown T, Flament-Auvigne I, Rakha EA, Toss M, Dabbs DJ, Regitnig P, Olson N, Wren JH, Robinson C, Corrado GS, Peng LH, Liu Y, Mermel CH, Steiner DF, Chen PHC. Determining breast cancer biomarker status and associated morphological features using deep learning. Communications Medicine. 2021;1:14. doi:10.1038/s43856-021-00013-3 read 9 October 2026. doi.org/10.1038/s43856-021-00013-3
- Kaissis G, Ziller A, Passerat-Palmbach J, Ryffel T, Usynin D, Trask A, Lima I, Mancuso J, Jungmann F, Steinborn MM, Saleh A, Makowski M, Rueckert D, Braren R. End-to-end privacy preserving deep learning on multi-institutional medical imaging. Nature Machine Intelligence. 2021;3(6):473-484. doi:10.1038/s42256-021-00337-8 read 9 October 2026. doi.org/10.1038/s42256-021-00337-8
- Erion G, Janizek JD, Sturmfels P, Lundberg SM, Lee SI. Improving performance of deep learning models with axiomatic attribution priors and expected gradients. Nature Machine Intelligence. 2021;3:620-31. doi:10.1038/s42256-021-00343-w read 9 October 2026. doi.org/10.1038/s42256-021-00343-w
- Duran JM, Jongsma KR. Who is afraid of black box algorithms? On the epistemological and ethical basis of trust in medical AI. Journal of Medical Ethics. 2021;47:329-35. read 9 October 2026. arxiv.org
- Oikonomou EK, Siddique M, Antoniades C. Artificial intelligence in medical imaging: a radiomic guide to precision phenotyping of cardiovascular disease. Cardiovascular Research. 2020;116(13):2040-2054. doi:10.1093/cvr/cvaa021 read 9 October 2026. doi.org/10.1093/cvr/cvaa021
- Mongan J, Moy L, Kahn CE Jr. Checklist for Artificial Intelligence in Medical Imaging (CLAIM): a guide for authors and reviewers. Radiology: Artificial Intelligence. 2020;2:e200029. doi:10.1148/ryai.2020200029 read 9 October 2026. doi.org/10.1148/ryai.2020200029
- Chen Z, Bei Y, Rudin C. Concept whitening for interpretable image recognition. Nature Machine Intelligence. 2020;2:772-82. doi:10.1038/s42256-020-00265-z read 9 October 2026. doi.org/10.1038/s42256-020-00265-z
- Ghorbani A, Ouyang D, Abid A, He B, Chen JH, Harrington RA, Liang DH, Ashley EA, Zou JY. Deep learning interpretation of echocardiograms. npj Digital Medicine. 2020;3:10. doi:10.1038/s41746-019-0216-8 read 9 October 2026. doi.org/10.1038/s41746-019-0216-8
- Biffi C, Cerrolaza JJ, Tarroni G, Bai W, de Marvao A, Oktay O, Ledig C, Le Folgoc L, Kamnitsas K, Doumou G, Duan J, Prasad SK, Cook SA, O'Regan DP, Rueckert D. Explainable anatomical shape analysis through deep hierarchical generative models. IEEE Transactions on Medical Imaging. 2020;39:2088-99. read 9 October 2026. arxiv.org
- Barredo Arrieta A, Diaz-Rodriguez N, Del Ser J, Bennetot A, Tabik S, Barbado A, Garcia S, Gil-Lopez S, Molina D, Benjamins R, Chatila R, Herrera F. Explainable artificial intelligence (XAI): concepts, taxonomies, opportunities and challenges toward responsible AI. Information Fusion. 2020;58:82-115. doi:10.1016/j.inffus.2019.12.012 read 9 October 2026. doi.org/10.1016/j.inffus.2019.12.012
- Gregory J, Welliver S, Chong J. Top 10 reviewer critiques of radiology artificial intelligence (AI) articles: qualitative thematic analysis of reviewer critiques of machine learning/deep learning manuscripts submitted to JMRI. Journal of Magnetic Resonance Imaging. 2020;52:248-54. doi:10.1002/jmri.27035 read 9 October 2026. doi.org/10.1002/jmri.27035
- Liu X, Faes L, Kale AU, Wagner SK, Fu DJ, Bruynseels A, Mahendiran T, Moraes G, Shamdas M, Kern C, Ledsam JR, Schmid MK, Balaskas K, Topol EJ, Bachmann LM, Keane PA, Denniston AK. A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis. The Lancet Digital Health. 2019;1(6):e271-e297. doi:10.1016/S2589-7500(19)30123-2 read 9 October 2026. doi.org/10.1016/S2589-7500(19)30123-2
- Topol EJ. High-performance medicine: the convergence of human and artificial intelligence. Nature Medicine. 2019;25(1):44-56. doi:10.1038/s41591-018-0300-7 read 9 October 2026. doi.org/10.1038/s41591-018-0300-7
- Kelly CJ, Karthikesalingam A, Suleyman M, Corrado G, King D. Key challenges for delivering clinical impact with artificial intelligence. BMC Medicine. 2019. doi:10.1186/s12916-019-1426-2 read 9 October 2026. doi.org/10.1186/s12916-019-1426-2
- Boehle M, Eitel F, Weygandt M, Ritter K. Layer-wise relevance propagation for explaining deep neural network decisions in MRI-based Alzheimer's disease classification. Frontiers in Aging Neuroscience. 2019;11. read 9 October 2026. arxiv.org
- Carvalho DV, Pereira EM, Cardoso JS. Machine learning interpretability: a survey on methods and metrics. Electronics. 2019;8:832. read 9 October 2026. arxiv.org
- Eitel F, Soehler E, Bellmann-Strobl J, Brandt AU, Ruprecht K, Giess RM, Kuchling J, Asseyer S, Weygandt M, Haynes JD, Scheel M, Paul F, Ritter K. Uncovering convolutional neural network decisions for diagnosing multiple sclerosis on conventional MRI using layer-wise relevance propagation. NeuroImage: Clinical. 2019;24:102003. doi:10.1016/j.nicl.2019.102003 read 9 October 2026. doi.org/10.1016/j.nicl.2019.102003
- Ding Y, Sohn JH, Kawczynski MG, Trivedi H, Harnish R, Jenkins NW, Lituiev D, Copeland TP, Aboian MS, Mari Aparici C, Behr SC, Flavell RR, Huang SY, Zalocusky KA, Nardo L, Seo Y, Hawkins RA, Hernandez Pampaloni M, Hadley D, Franc BL. A deep learning model to predict a diagnosis of Alzheimer disease by using 18F-FDG PET of the brain. Radiology. 2018;290:180958. doi:10.1148/radiol.2018180958 read 9 October 2026. doi.org/10.1148/radiol.2018180958
- Rajpurkar P, Irvin J, Ball RL, Zhu K, Yang B, Mehta H, Duan T, Ding D, Bagul A, Langlotz CP, Patel BN, Yeom KW, Shpanskaya K, Blankenberg FG, Seekins J, Amrhein TJ, Mong DA, Halabi SS, Zucker EJ, Ng AY, Lungren MP. Deep learning for chest radiograph diagnosis: a retrospective comparison of the CheXNeXt algorithm to practicing radiologists. PLOS Medicine. 2018;15:e1002686. doi:10.1371/journal.pmed.1002686 read 9 October 2026. doi.org/10.1371/journal.pmed.1002686
- Faust K, Xie Q, Han D, Goyle K, Volynskaya Z, Djuric U, Diamandis P. Visualizing histopathologic deep learning classification and anomaly detection using nonlinear feature space dimensionality reduction. BMC Bioinformatics. 2018;19. read 9 October 2026. arxiv.org
- Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM, Thrun S. Dermatologist-level classification of skin cancer with deep neural networks. Nature. 2017;542(7639):115-118. doi:10.1038/nature21056 read 9 October 2026. doi.org/10.1038/nature21056
- Bach S, Binder A, Montavon G, Klauschen F, Muller KR, Samek W. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLOS ONE. 2015;10. read 9 October 2026. arxiv.org
- Aerts HJWL, Velazquez ER, Leijenaar RTH, Parmar C, Grossmann P, Carvalho S, Bussink J, Monshouwer R, Haibe-Kains B, Rietveld D, Hoebers F, Rietbergen MM, Leemans CR, Dekker A, Quackenbush J, Gillies RJ, Lambin P. Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach. Nature Communications. 2014;5:4006. doi:10.1038/ncomms5006 read 9 October 2026. doi.org/10.1038/ncomms5006
Every source above is a peer-reviewed article, a publisher’s record of one, or a standard issued by the body that owns it. Each carries the date it was last read. Where a standard moves, this article is revised rather than left up with a note.
How to cite this article
Pick your referencing style and copy it. Your university or target journal may vary the punctuation — check its guide if it has one.
Pubrica Academy. (2026, October 9). Deep Learning Over Machine Learning in Medical Imaging: The Real Difficulties. Pubrica. https://pubrica.com/academy/latest-topics/deep-learning-over-machine-learning-mention-the-challenges-and-difficulties-in-the-medical-imaging-process-and-research-issues/
Pubrica Academy. “Deep Learning over Machine Learning in Medical Imaging: The Real Difficulties.” Pubrica, 9 Oct. 2026, pubrica.com/academy/latest-topics/deep-learning-over-machine-learning-mention-the-challenges-and-difficulties-in-the-medical-imaging-process-and-research-issues/.
Pubrica Academy (2026) Deep Learning Over Machine Learning in Medical Imaging: The Real Difficulties. Pubrica. Available at: https://pubrica.com/academy/latest-topics/deep-learning-over-machine-learning-mention-the-challenges-and-difficulties-in-the-medical-imaging-process-and-research-issues/ (Accessed: 15 October 2026).
Pubrica Academy. “Deep Learning over Machine Learning in Medical Imaging: The Real Difficulties.” Pubrica. October 9, 2026. https://pubrica.com/academy/latest-topics/deep-learning-over-machine-learning-mention-the-challenges-and-difficulties-in-the-medical-imaging-process-and-research-issues/
Pubrica Academy, ‘Deep Learning Over Machine Learning in Medical Imaging: The Real Difficulties’ (Pubrica, 9 October 2026) <https://pubrica.com/academy/latest-topics/deep-learning-over-machine-learning-mention-the-challenges-and-difficulties-in-the-medical-imaging-process-and-research-issues/> accessed 15 October 2026.
Pubrica Academy. Deep Learning Over Machine Learning in Medical Imaging: The Real Difficulties. [Internet]. Pubrica; 2026 Oct 9 [cited 2026 Oct 15]. Available from: https://pubrica.com/academy/latest-topics/deep-learning-over-machine-learning-mention-the-challenges-and-difficulties-in-the-medical-imaging-process-and-research-issues/
Pubrica Academy, “Deep Learning Over Machine Learning in Medical Imaging: The Real Difficulties,” Pubrica, Oct. 9, 2026. [Online]. Available: https://pubrica.com/academy/latest-topics/deep-learning-over-machine-learning-mention-the-challenges-and-difficulties-in-the-medical-imaging-process-and-research-issues/
@misc{deep_learning_over_machine_learning_me_2026,
author = {{Pubrica Academy}},
title = {Deep Learning Over Machine Learning in Medical Imaging: The Real Difficulties},
howpublished = {Pubrica},
year = {2026},
month = {oct},
url = {https://pubrica.com/academy/latest-topics/deep-learning-over-machine-learning-mention-the-challenges-and-difficulties-in-the-medical-imaging-process-and-research-issues/},
note = {Accessed: <span data-cite-acc="iso">2026-10-15</span>}
}This article is written to be learned from. Cite it if you are referring to what it says; if you are using the evidence behind it, cite the sources listed above instead.
Imaging AI methods and validation stub
Complete this before writing the methods section. It is built around the items that decide whether an imaging AI result is believable: how the data were partitioned, what the reference standard was, what was reported at which operating point, and whether anyone checked for a shortcut.
IMAGING AI STUDY: METHODS AND VALIDATION RECORD
1. TASK DEFINITION
Clinical question the model answers: ______________________
Modality: ____________ Acquisition protocol(s): ____________
[ ] Acquisition protocol is treated as PART of the task, not a detail
Intended use setting: ______________________
Prevalence in intended-use setting: ______
2. DATA
Studies (not images): ______ Patients: ______
Sites: ____ Date range: ______ to ______
Scanner manufacturers and models, per site:
Site 1: ______________________
Site 2: ______________________
Site 3: ______________________
Acquisition parameters relevant to this task: ______________
3. REFERENCE STANDARD (disaggregate - these are not equivalent)
Established by histopathology ................ ____ (__%)
Established by adjudicated multi-reader read .. ____ (__%)
Established by single clinical report ......... ____ (__%)
Readers blinded to model output? [ ] yes [ ] no
Disagreement resolution: ______________________
[ ] If the standard is the clinical report, the text states that
performance is capped by that reader's accuracy
4. PARTITIONING (random splits do not detect shortcuts)
Split by: [ ] site [ ] time [ ] both [ ] random (justify: ____)
Development set: ______________________
Tuning partition: ______________________
Held-out test set 1 (external site): ______________
Held-out test set 2 (later time period): ______________
[ ] No patient appears in more than one partition
[ ] Test sets evaluated and reported SEPARATELY, not pooled
5. CLASS BALANCE AND PREVALENCE
Prevalence per set: dev ____ test 1 ____ test 2 ____
Imbalance handling (resampling, augmentation, loss weighting):
______________________________________________________
[ ] Effect of this on output probabilities stated
[ ] Predictive values recalibrated to intended-use prevalence
6. RESULTS TO REPORT (per test set, per protocol)
Operating point: ______ Fixed before test evaluation? [ ]
Sensitivity ______ Specificity ______
Calibration slope ______ Intercept ______
AUC ______
[ ] Reported stratified by acquisition protocol
[ ] Reported stratified by site
[ ] NOT reported as AUC alone
7. SHORTCUT CHECK (report the result, including a negative one)
[ ] Attribution maps inspected on a sample of true positives
Attention outside relevant anatomy found in: ____ cases
[ ] Perturbation / removal test on suspected confound
Confound tested: ____________ Performance change: ______
[ ] Performance on the external site compared with internal
Internal ______ External ______ Drop ______
8. READER COMPARISON (omit entirely if none - do not imply one)
Same sample as the model test set? [ ] yes [ ] no
Readers: ____ Blinded to model output? [ ]
Clinical information available to readers: ______________
9. REPORTING STANDARD
[ ] CLAIM [ ] TRIPOD+AI [ ] STARD-AI
[ ] CONSORT-AI / SPIRIT-AI [ ] DECIDE-AI [ ] PROBAST+AI
Completed checklist with section locations attached? [ ] Pubrica AcademyWritten and maintained by the subject matter experts of Pubrica who do this work. Articles are revised when the standard underneath them moves, and each one carries the date it was last checked.
How Pubrica helps
Pubrica’s imaging and data teams review AI imaging manuscripts on the points that decide whether the result holds: whether the split can detect a shortcut, whether the reference standard is disaggregated by how it was established, whether sensitivity, specificity and calibration are reported at a fixed operating point by site and protocol, and whether the reader comparison is on the same sample. We map drafts against CLAIM and the AI reporting standard that fits the design, and we flag where a pooled figure is concealing a protocol or site on which the model does not work.