Contents
On this page
- Study selection: from screened records to poolable studies
- Effect of independent validation on pooled accuracy
- Unit of analysis: lesion, image, scan or patient
- Joint bivariate modelling of sensitivity and specificity
- Confidence interval performance in small heterogeneous sets
- Description and interpretation of heterogeneity
- Small-study effects and the limits of asymmetry testing
- Appraisal and reporting instruments
- Formulating defensible diagnostic accuracy claims
- Planning considerations for a CNN accuracy review
- Key points
- Application to your study
- Frequently asked questions
Academy · Research Data Analytics & AI · CNN radiology meta-analysis
Meta-Analysis of Convolutional Neural Network (CNN) Performance on Radiological Images
Pooled sensitivity for deep-learning lung-nodule detection was 85% across all validation datasets and 79% on independent datasets, for algorithms tested under different validation conditions. This six-point gap is arguably the most important figure in this literature, because almost every headline accuracy claim derives from the first type of dataset yet is interpreted as though it derived from the second.
Convolutional neural networks are now evaluated in hundreds of radiology papers, yet only a very small proportion of those evaluations can be pooled. In the largest systematic review of deep learning against clinicians in medical imaging, 31,587 records were identified, 82 studies entered the review, and 25 had out-of-sample external validation suitable for meta-analysis. The principal barrier is not the volume of research but the fact that most studies report a discrimination statistic without the validation design, threshold or counts that a diagnostic accuracy model requires.
Learning objectives
- The proportion of the CNN imaging literature that is poolable, and the reasons the remainder is not
- The quantified effect of independent validation on pooled sensitivity
- The reasons lesion-level and patient-level results address different questions
- The rationale for pooling sensitivity and specificity jointly with a bivariate or hierarchical model
- The poor performance of confidence intervals in small, heterogeneous sets, and the preferred alternative methods
- Appraisal and reporting of this review type with QUADAS-2, CLAIM and PRISMA-DTA
Common misconceptions
- “Reported accuracy is reported accuracy.” Wang and colleagues, in Cancers, reported a pooled lesion-wise sensitivity of 85% across all validation datasets and of 79% on independent datasets, from 48 studies covering 64 and 16 algorithms respectively.
- “The literature is large, so the meta-analysis will be well powered.” Liu and colleagues screened 20,530 records, included 82 studies and found 25 with out-of-sample external validation suitable for pooling.
- “Pool sensitivity, then pool specificity.” Reitsma and colleagues showed that the two are linked within studies by the threshold; treating them as independent misrepresents the joint uncertainty.
- “A 95% confidence interval covers 95% of the time.” Veroniki and colleagues found that Wald-type coverage fell as low as 0.65 against a nominal 0.95 when heterogeneity was high and only two or three studies were pooled.
- “A non-significant funnel plot test means no publication bias.” Sterne and colleagues advise against these tests when fewer than 10 studies are available, which is the range in which most CNN accuracy sets lie.
Study selection: from screened records to poolable studies
Liu and colleagues’ systematic review and meta-analysis in The Lancet Digital Health, comparing deep-learning performance against health-care professionals in medical imaging, documents the attrition precisely: 31,587 records identified, 20,530 screened, 122 full texts assessed, 82 studies included in the systematic review, and 25 with out-of-sample external validation suitable for meta-analysis 32. Fewer than one third of the included studies could be pooled, from a screened set of more than twenty thousand records.
The reasons are consistent across this literature. Performance is reported on a random split of the dataset on which the model was developed, which measures fit rather than transportability. The operating threshold is not stated, so no 2x2 table can be extracted 36. The unit of analysis (per lesion, per image, per scan or per patient) is not stated or differs from that of the rest of the set. The ground truth differs between studies: expert consensus in one, histopathology in another and follow-up imaging in a third. Comparator data, where clinicians were assessed at all, are often drawn from a different sample from that used for the algorithm.
For a protocol, this means that the eligibility criteria must be framed in terms of reportability, and the expected yield stated in advance 41. A review that promises a pooled estimate but includes only four studies cannot fulfil its stated aim. The Academy page on abstract screening describes how such a pronounced reduction in eligible studies can be managed.
Proportion of this literature that reaches a pooled estimate
| Stage | Count | What drops out here |
|---|---|---|
| Records identified | 31,587 | Duplicates and clearly irrelevant records |
| Records screened | 20,530 | Non-imaging, non-deep-learning, no accuracy outcome |
| Full texts assessed | 122 | No extractable accuracy data; wrong study design |
| Included in the systematic review | 82 | — |
| Suitable for meta-analysis | 25 | No out-of-sample external validation |
Liu X, Faes L, Kale AU, Wagner SK, Fu DJ, Bruynseels A, et al. A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis. The Lancet Digital Health 2019. The final row reflects the review’s own criterion of out-of-sample external validation.
Effect of independent validation on pooled accuracy
Wang and colleagues’ meta-analytical review of deep learning for lung nodule early detection, in Cancers, analysed the two separately and quantified the difference. Pooled lesion-wise sensitivity was 79% (95% CI 72 to 86) on independent datasets and 85% (83 to 88) across all validation datasets, from 48 included studies covering 16 algorithms in the independent-dataset analysis and 64 across all validation datasets 6,7,39.
Two features of this result merit attention. The point estimate is six percentage points lower on independent data, which is the expected direction, but the magnitude is informative. The interval is also much wider (72 to 86 against 83 to 88) because far fewer algorithms were evaluated in this way. The more trustworthy estimate is therefore also the less precise one, and this literature cannot yet provide an estimate that combines precision with external validity 7.
Validation design should be treated as a stratification variable rather than as a covariate confined to a sensitivity analysis, and both strata should be reported. A single pooled figure that combines held-out splits with independent samples is the most common means by which this literature is overstated, and the problem cannot be detected from the abstract. The same problem in cardiovascular imaging, where patient-level and vessel-level estimates diverge, is addressed on the Academy cardiac imaging AI page.
The same deep-learning lung-nodule detection task, pooled under two definitions of validation data, from 48 included studies. Source: Wang TW, Wang CK, Hong JS, Chao HS, Chen YM, Wu YT. Deep Learning in Thoracic Oncology: Meta-Analytical Insights into Lung Nodule Early-Detection Technologies. Cancers 2025;17(4):621.
Unit of analysis: lesion, image, scan or patient
Nodule detection makes the unit-of-analysis problem particularly acute, because a single scan contains many candidate lesions and a single patient may have several scans. Lesion-wise sensitivity, the quantity pooled by Wang and colleagues, is the proportion of true nodules that the algorithm detected 7. Patient-wise sensitivity is the proportion of patients with a nodule who were flagged, and it will generally be higher, because detection of any one nodule in a patient is sufficient.
Specificity is even more problematic. At lesion level there is no natural denominator of true negatives, which is why detection studies conventionally report false positives per scan instead. That quantity cannot be entered into a bivariate model alongside a specificity, so a set combining the two conventions cannot be pooled without a decision about which to exclude. That decision should be made and stated in the protocol, and the number of studies it excluded should be reported.
The corresponding requirement is to treat multiple results from one study (several thresholds, units or datasets) as dependent rather than as additional studies 41. Entering them as independent observations inflates the apparent information and spuriously narrows the interval, and in a set of twenty-five studies this distortion is not marginal.
Joint bivariate modelling of sensitivity and specificity
Sensitivity and specificity move together with the operating threshold, and within studies they are typically negatively correlated. Reitsma and colleagues, in the Journal of Clinical Epidemiology, set out the bivariate random-effects model that estimates both jointly with a correlation parameter, giving summary estimates whose uncertainty reflects the structure of the data 45. Harbord and colleagues, in Biostatistics, showed how that formulation relates to the hierarchical summary ROC approach, so that the choice between them depends on what is to be reported rather than on validity: a summary operating point where a common threshold exists, and a summary curve where thresholds differ 43.
Either model requires a 2x2 table for each study: true positives, false positives, false negatives and true negatives at a stated threshold, or sufficient information to reconstruct them. An area under the curve (AUC) cannot be converted into those counts. This is the most common reason for the exclusion of a CNN study from a meta-analysis, and the number of such exclusions should be recorded explicitly, because that number is itself a finding about the field. Deeks, Macaskill and Irwig, also in the Journal of Clinical Epidemiology, describe how small-study effects behave differently in diagnostic reviews from intervention reviews 44.
Where the review question concerns risk prediction rather than detection (for example, the work of Mikhael and colleagues on predicting future lung cancer risk from a single low-dose CT), the synthesis problem is different again, and a diagnostic accuracy model is not the appropriate target 20.
Confidence interval performance in small heterogeneous sets
This technical point is the one most often overlooked, and it has the greatest consequences in precisely the situation in which CNN reviews find themselves. Veroniki and colleagues, reviewing methods for calculating uncertainty in the random-effects summary effect, found that coverage of the Wald-type normal confidence interval fell as low as 0.65 against a nominal 0.95 when heterogeneity was high and only two or three studies were pooled 10,51. An interval labelled 95% that contains the true value only about two-thirds of the time is not a conservative approximation but a misstatement 39.
CNN accuracy sets are small and heterogeneous almost by construction, with different scanners, acquisition protocols, prevalence and ground truth. The interval method is therefore a consequential pre-specified choice rather than a software default, and it should be stated in the protocol alongside the model 51. Our methodology walkthrough sets out the current recommendations; biostatistics support is where the fitting happens.
Description and interpretation of heterogeneity
Higgins and Thompson, in Statistics in Medicine, proposed three statistics — H, I-squared and R — to describe heterogeneity independently of the effect metric and the number of studies, illustrated on five datasets; Higgins, Thompson, Deeks and Altman’s later BMJ paper on measuring inconsistency is the version most reviewers cite. In diagnostic accuracy reviews these statistics describe variation in the summary estimates but do not identify its source, and in CNN imaging the source is usually apparent from the study table rather than from a statistic 46.
The candidate explanations should therefore be tabulated rather than tested: scanner and vendor, slice thickness and reconstruction, nodule size distribution, disease prevalence in the validation sample, ground truth definition, threshold provenance, and whether the validation data came from the same institutions as the development data. With twenty-five studies and this many plausible moderators, formal meta-regression is underpowered, and a table that allows readers to see the clustering is more transparent than a non-significant coefficient.
Small-study effects and the limits of asymmetry testing
Sterne and colleagues’ recommendations in the BMJ are explicit: tests for funnel plot asymmetry should generally not be used when a meta-analysis contains fewer than 10 studies, and the minimum may be substantially higher when heterogeneity is substantial 39. The underlying logic of the plot is that, in the absence of bias and heterogeneity, about 95% of studies should lie inside the triangle defined by the fixed-effect estimate plus or minus 1.96 standard errors; this expectation about scatter carries little weight with eight or nine data points 39. Egger and colleagues’ original test, also in the BMJ, remains widely used, and is frequently misapplied, in sets of this size.
The concern itself is genuine and arguably greater here than in drug trials. An algorithm that underperforms is often retrained rather than published; versions are iterated until a result is reportable; and benchmark datasets are shared, so the same images appear in multiple papers evaluating related models. None of these mechanisms can be detected by a funnel plot. They should be reported as a substantive limitation, with the specific mechanisms named, rather than by running a test below its stated minimum and presenting the non-significant result as reassurance 39.
Appraisal and reporting instruments
QUADAS-2, from Whiting and colleagues in the Annals of Internal Medicine, is the risk-of-bias tool, covering patient selection, index test, reference standard, and flow and timing, each with an applicability judgement. For CNN studies the index test domain is the most important, because threshold selection, model version and whether the algorithm was run blind to the reference standard are all assessed there.
CLAIM, the Checklist for Artificial Intelligence in Medical Imaging from Mongan, Moy and Kahn in Radiology: Artificial Intelligence, specifies what the primary study should report: data sources, ground truth definition, data partitioning, model description and validation design 31. Extraction against CLAIM converts “insufficiently reported to pool” from a judgement into a documented, itemised exclusion reason, and the resulting tally is publishable in its own right.
PRISMA-DTA, from McInnes and colleagues in JAMA, governs the reporting of the review. It is a reporting guideline rather than a method, a distinction explained on the Academy PRISMA definition page, and it explicitly requires the target condition, the reference standard and the unit of analysis, the three items that this literature most often leaves implicit 35.
Formulating defensible diagnostic accuracy claims
The typical failure is rarely an incorrect number. It is a correct number reported without the validation design that produced it, which converts a modest finding into an overstatement although nothing false has been written.
The same pooled sensitivity, reported two ways
Overstates without saying anything false
Deep learning algorithms detect pulmonary nodules with a pooled sensitivity of 85%, supporting their readiness for clinical deployment in lung cancer screening.
Which validation datasets? The 85% figure includes held-out splits of development data; on independent datasets the same review reports 79%.
At what unit of analysis? Lesion-wise sensitivity is not patient-wise sensitivity, and the clinical implication differs.
Where is specificity, or the false-positive rate? A detection sensitivity reported without the false-positive burden provides no information about workload or harm.
How many algorithms, evaluated how often? 16 algorithms on independent data against 64 overall; the headline figure rests on the weaker design.
“Readiness for clinical deployment” is not an output of a meta-analysis. No accuracy estimate, however it is pooled, establishes such readiness.
Reports the design with the number
Across 48 included studies, pooled lesion-wise sensitivity for deep-learning nodule detection was 85% (95% CI 83 to 88) when all validation datasets were included, covering 64 algorithms. Restricted to independent datasets, pooled sensitivity fell to 79% (72 to 86) from 16 algorithms. The estimate on independent data is both lower and markedly less precise, because few algorithms have been evaluated that way; it is the more relevant figure for deployment, and the width of its interval is the reason we do not draw a readiness conclusion from it.
Both strata are reported, so the reader can see that the figure depends on which validation data are included.
The unit of analysis is named, making the estimate comparable with others of the same kind.
Algorithm counts are given for each stratum, showing that the more trustworthy estimate rests on fewer evaluations.
The wider interval is interpreted rather than concealed, which makes the uncertainty usable.
The deployment claim is explicitly declined, with the reason stated.
Every number in the first version appears in the second. The difference is that the second states the validation design, the unit of analysis and the number of algorithms underlying each figure, which are the three facts that determine the meaning of an accuracy estimate, and it declines the inference that the data cannot support.
Planning considerations for a CNN accuracy review
The first decision is whether the question concerns detection accuracy, clinical comparison against readers or the mapping of what has been developed. These require different designs, and only the first two can produce a pooled estimate 41. If the question concerns accuracy, eligibility should be defined in terms of reportability: a stated threshold, reconstructable 2x2 counts, a single declared unit of analysis, a ground truth from a pre-specified list, and independent validation. The search should then be run and the eligible studies counted before a synthesis is planned.
If fewer than ten studies remain, asymmetry testing is already known to be unavailable and interval coverage to be fragile, and the review can be designed around those constraints. The alternative output, a structured map of what has been developed, how it was validated and on which populations, with the reporting gaps itemised against CLAIM, is more useful to this field than a pooled sensitivity from five papers. Pubrica’s AI imaging hub and systematic review support pages cover both routes.
Key points
Key points
Only a small proportion of the CNN radiology literature is poolable: one major review screened 20,530 records, included 82 studies and found 25 with out-of-sample external validation suitable for meta-analysis. Where validation design has been analysed separately, it has a material effect: pooled lesion-wise sensitivity for nodule detection was 85% across all validation datasets and 79% on independent datasets, from 48 studies covering 64 and 16 algorithms. Sensitivity and specificity should be pooled jointly with a bivariate or hierarchical model, never separately; analyses should be stratified by validation design and unit of analysis; the confidence-interval method should be pre-specified, because Wald-type coverage can fall to 0.65 in small heterogeneous sets; and asymmetry tests should not be run below 10 studies.
Application to your study
Application to your study
- Define the review type first. The options are detection accuracy, clinician comparison or a map of existing evidence; only the first two can yield a pooled estimate.
- Treat independent validation as an eligibility criterion or a stratum, not as a footnote. Report both strata where both exist, with their algorithm counts.
- Declare one unit of analysis per stratum. Lesion-wise and patient-wise sensitivity are different quantities; false positives per scan cannot sit alongside a specificity in the same model.
- Require reconstructable 2x2 counts at a stated threshold. Count the studies excluded for reporting only the area under the curve (AUC), and publish that count as a finding.
- Record threshold provenance per study. Classify each threshold as pre-specified, inherited or optimised in the same data; the last introduces an optimistic bias that no model removes.
- Pre-specify the confidence-interval method in the protocol. Wald-type coverage has been shown to fall to 0.65 against a nominal 0.95 with high heterogeneity and two or three studies.
- Tabulate heterogeneity sources instead of regressing on them. Scanner, protocol, prevalence, ground truth, threshold and site overlap constitute too many moderators for twenty-five studies.
- Omit the funnel plot asymmetry test below 10 studies and identify the actual bias mechanisms instead: unpublished underperforming models, iterated versions and shared benchmark datasets.
Frequently asked questions
A small minority. Liu and colleagues, in The Lancet Digital Health, identified 31,587 records, screened 20,530, assessed 122 full texts, included 82 studies in the systematic review and found 25 with out-of-sample external validation suitable for meta-analysis. The exclusions are mostly structural rather than related to quality: performance reported on a held-out split of the development data, an unstated operating threshold so that no 2x2 table can be reconstructed, an unstated or incomparable unit of analysis, a ground truth that differs from the rest of the set, or comparator reader data drawn from a different sample from that used for the algorithm.
In the clearest quantification available, by approximately six percentage points, with a much wider interval. Wang and colleagues, in Cancers, pooled lesion-wise sensitivity for deep-learning lung-nodule detection at 79% (95% CI 72 to 86) on independent datasets against 85% (83 to 88) across all validation datasets, from 48 included studies covering 16 and 64 algorithms respectively. The more externally valid estimate is also the less precise one, because far fewer algorithms have been evaluated in this way. Both strata should be reported; a single figure combining them is the usual route to overstatement.
Because they are not independent. Within any study the two are linked by the operating threshold and are typically negatively correlated: a cut-off that detects more disease necessarily produces more false positives. Reitsma and colleagues, in the Journal of Clinical Epidemiology, set out the bivariate random-effects model that estimates both jointly with a correlation parameter. Harbord and colleagues, in Biostatistics, showed how that model relates to the hierarchical summary ROC formulation, which is the more appropriate target where studies report at different thresholds. Separate univariate pooling misrepresents the joint uncertainty and can yield a summary point that lies off any plausible ROC curve.
One unit should be selected for each stratum and declared. Lesion-wise sensitivity is the proportion of true nodules that the algorithm detected; patient-wise sensitivity is the proportion of affected patients who were flagged, and it is generally higher because one detection suffices. Specificity is the more difficult problem: at lesion level there is no natural denominator of true negatives, which is why detection studies conventionally report false positives per scan instead, a quantity that cannot enter a bivariate model alongside a specificity. The protocol should specify which convention is required, and the number of studies excluded by that decision should be reported.
Not automatically, and least of all in small heterogeneous sets. Veroniki and colleagues found that coverage of the Wald-type normal confidence interval fell as low as 0.65 against a nominal 0.95 when heterogeneity was high and only two or three studies were pooled. CNN accuracy sets are small and heterogeneous by construction, with different scanners, protocols, prevalence and ground truth definitions, so the interval method is a consequential choice that should be specified in the protocol rather than left to a software default, and it should be reported alongside the model.
Usually by explaining why it cannot be assessed formally. Sterne and colleagues advise that tests for funnel plot asymmetry should generally not be used with fewer than 10 studies, and that the minimum is substantially higher under substantial heterogeneity; the plot assumes that, in the absence of bias and heterogeneity, about 95% of studies fall inside the triangle defined by the fixed-effect estimate plus or minus 1.96 standard errors, an expectation that carries little weight with eight or nine data points. The specific mechanisms in this field should instead be named in the limitations: underperforming models are retrained rather than published, versions are iterated until reportable, and shared benchmark datasets mean that the same images recur across papers.
Three tools, each with a different function. QUADAS-2, from Whiting and colleagues in the Annals of Internal Medicine, appraises risk of bias and applicability in each included accuracy study; for CNN work the index test domain carries most of the weight, covering threshold selection, model version and blinding to the reference standard. CLAIM, from Mongan, Moy and Kahn in Radiology: Artificial Intelligence, covers what the primary AI study should report and is the practical screen for whether a study is poolable. PRISMA-DTA, from McInnes and colleagues in JAMA, governs reporting of the review itself, including target condition, reference standard and unit of analysis.
Yes, provided that the output rather than the standard is changed. A structured map of what has been developed, against which ground truth it was validated, and on which populations and scanners, with the reporting gaps itemised against CLAIM, answers a question that this field genuinely faces, and the tally of studies excluded for AUC-only reporting or internal-validation-only designs is a publishable finding in itself. A pooled sensitivity from four or five studies, presented with a confidence interval of unknown coverage and an asymmetry test that cannot be run, is not worthwhile.
References
- Basile C, Villaschi A, Ambrosino P, Canonico ME, Rawshani A, Bobbio E. Artificial intelligence in cardiovascular medicine: a practical guide for clinicians and researchers. European Heart Journal - Digital Health. 2026;7(8):ztag153. doi:10.1093/ehjdh/ztag153 read 9 October 2026. doi.org/10.1093/ehjdh/ztag153
- Aladwani F, Perelli A, Mordi I, Khan F. Deep learning for cardiac MRI: performance evidence and barriers to clinical integration. A Systematic Review and Meta-Analysis. European Heart Journal - Imaging Methods and Practice. 2026;4(1):qyag045. doi:10.1093/ehjimp/qyag045 read 9 October 2026. doi.org/10.1093/ehjimp/qyag045
- Farahani S, Hejazi M, Tabassum M, Di Ieva A, Mahdavifar N, et al. , Liu S, et al. Diagnostic performance of deep learning for predicting glioma isocitrate dehydrogenase and 1p/19q co-deletion in MRI: a systematic review and meta-analysis. European Radiology. 2026;36(2):1562-1591. doi:10.1007/s00330-025-11898-2 read 9 October 2026. doi.org/10.1007/s00330-025-11898-2
- Yang Z, Wei T, Liang Y, Yuan X, Gao R, Xia Y, Zhou J, Zhang Y, et al. , Yu Z, et al. A foundation model for generalizable cancer diagnosis and survival prediction from histopathological images. Nature Communications. 2025;16(1):2366. doi:10.1038/s41467-025-57587-y read 9 October 2026. doi.org/10.1038/s41467-025-57587-y
- Ding T, Wagner SJ, Song AH, Chen RJ, Lu MY, Zhang A, Vaidya AJ, Jaume G, Shaban M, Kim A, et al. , Mahmood F, et al. A multimodal whole-slide foundation model for pathology. Nature Medicine. 2025;31(11):3749-3761. doi:10.1038/s41591-025-03982-3 read 9 October 2026. doi.org/10.1038/s41591-025-03982-3
- Andreadis T, Gasteratos A, Seimenis I, Koulouriotis D. Application of CAD systems in breast cancer diagnosis using machine learning techniques: an overview of systematic reviews. Bioengineering. 2025;12(11):1160. doi:10.3390/bioengineering12111160 read 9 October 2026. doi.org/10.3390/bioengineering12111160
- Wang TW, Wang CK, Hong JS, Chao HS, Chen YM, Wu YT. Deep Learning in Thoracic Oncology: Meta-Analytical Insights into Lung Nodule Early-Detection Technologies. Cancers. 2025;17(4):621. doi:10.3390/cancers17040621 read 9 October 2026. doi.org/10.3390/cancers17040621
- Lee CF, Lin J, Huang YL, Chen ST, Chou CT, Chen DR, et al. , Wu WP, et al. Deep learning-based breast MRI for predicting axillary lymph node metastasis: a systematic review and meta-analysis. Cancer Imaging. 2025;25(1):44. doi:10.1186/s40644-025-00863-3 read 9 October 2026. doi.org/10.1186/s40644-025-00863-3
- Abdullah KA, Marziali S, Nanaa M, Escudero Sanchez L, Payne NR, et al. , Gilbert FJ, et al. Deep learning-based breast cancer diagnosis in breast MRI: systematic review and meta-analysis. European Radiology. 2025;35(8):4474-4489. doi:10.1007/s00330-025-11406-6 read 9 October 2026. doi.org/10.1007/s00330-025-11406-6
- Du S, Guo J, Huang D, Liu Y, Zhang X, Lu S. Diagnostic accuracy of deep learning-based algorithms in laryngoscopy: a systematic review and meta-analysis. European Archives of Oto-Rhino-Laryngology. 2025;282(1):351-360. doi:10.1007/s00405-024-09049-2 read 9 October 2026. doi.org/10.1007/s00405-024-09049-2
- Valizadeh P, Jannatdoust P, Pahlevan-Fallahy MT, Hassankhani A, Amoukhteh M, Bagherieh S, Ghadimi DJ, et al. , Gholamrezanezhad A, et al. Diagnostic accuracy of radiomics and artificial intelligence models in diagnosing lymph node metastasis in head and neck cancers: a systematic review and meta-analysis. Neuroradiology. 2025;67(2):449-467. doi:10.1007/s00234-024-03485-x read 9 October 2026. doi.org/10.1007/s00234-024-03485-x
- Xu H, Usuyama N, Bagga J, Zhang S, Rao R, Naumann T, Wong C, Gero Z, Gonzalez J, Gu Y, et al. , Poon H, et al. A whole-slide foundation model for digital pathology from real-world data. Nature. 2024;630(8015):181-188. doi:10.1038/s41586-024-07441-w read 9 October 2026. doi.org/10.1038/s41586-024-07441-w
- Wang TW, Hong JS, Chiu HY, Chao HS, Chen YM, Wu YT. Standalone deep learning versus experts for diagnosis lung cancer on chest computed tomography: a systematic review. European Radiology. 2024;34(11):7397-7407. doi:10.1007/s00330-024-10804-6 read 9 October 2026. doi.org/10.1007/s00330-024-10804-6
- Gerdes A. The role of explainability in AI-supported medical decision-making. Discover Artificial Intelligence. 2024;4(1):29. doi:10.1007/s44163-024-00119-2 read 9 October 2026. doi.org/10.1007/s44163-024-00119-2
- Lin A, Pieszko K, Park C, Ignor K, Williams MC, Slomka P, Dey D. Artificial intelligence in cardiovascular imaging: enhancing image analysis and risk stratification. BJR|Open. 2023;5(1):20220021. doi:10.1259/bjro.20220021 read 9 October 2026. doi.org/10.1259/bjro.20220021
- Nasser M, Yusof UK. Deep learning based methods for breast cancer diagnosis: a systematic review and future direction. Diagnostics. 2023;13(1):161. doi:10.3390/diagnostics13010161 read 9 October 2026. doi.org/10.3390/diagnostics13010161
- Hendrix W, Hendrix N, Scholten ET, Mourits M, Trap-de Jong J, Schalekamp S, et al. Deep learning for the detection of benign and malignant pulmonary nodules in non-screening chest CT scans. Communications Medicine. 2023;3:156. doi:10.1038/s43856-023-00388-5 read 9 October 2026. doi.org/10.1038/s43856-023-00388-5
- Thong LT, Chou HS, Chew HSJ, Lau Y. Diagnostic test accuracy of artificial intelligence-based imaging for lung cancer screening: a systematic review and meta-analysis. Lung Cancer. 2023;176:4-13. doi:10.1016/j.lungcan.2022.12.002 read 9 October 2026. doi.org/10.1016/j.lungcan.2022.12.002
- Frank RA, Salameh JP, Islam N, Yang B, Murad MH, Mustafa R, Leeflang M, Bossuyt PM, Takwoingi Y, Whiting P, Dawit H, Kang SK, Ebrahimzadeh S, Levis B, Hutton B, McInnes MDF. How to critically appraise and interpret systematic reviews and meta-analyses of diagnostic accuracy: a user guide. Radiology. 2023;307(3):e221437. doi:10.1148/RADIOL.221437 read 9 October 2026. doi.org/10.1148/RADIOL.221437
- Mikhael PG, Wohlwend J, Yala A, Karstens L, Xiang J, Takigami AK, et al. Sybil: a validated deep learning model to predict future lung cancer risk from a single low-dose chest computed tomography. Journal of Clinical Oncology. 2023;41:2191-2200. doi:10.1200/JCO.22.01345 read 9 October 2026. doi.org/10.1200/JCO.22.01345
- Zhang X, Liu B, Liu K, Wang L. The diagnosis performance of convolutional neural network in the detection of pulmonary nodules: a systematic review and meta-analysis. Acta Radiologica. 2023;64:2987-2998. doi:10.1177/02841851231201514 read 9 October 2026. doi.org/10.1177/02841851231201514
- Liu M, Wu J, Wang N, Zhang X, Bai Y, Guo J, et al. The value of artificial intelligence in the diagnosis of lung cancer: a systematic review and meta-analysis. PLoS ONE. 2023;18:e0273445. doi:10.1371/journal.pone.0273445 read 9 October 2026. doi.org/10.1371/journal.pone.0273445
- Santosh K, Allu S, Rajaraman S, Antani S. Advances in deep learning for tuberculosis screening using chest X-rays: the last 5 years review. Journal of Medical Systems. 2022;46(11):82. doi:10.1007/s10916-022-01870-8 read 9 October 2026. doi.org/10.1007/s10916-022-01870-8
- Forte GC, Altmayer S, Silva RF, Stefani MT, Libermann LL, Cavion CC, et al. Deep learning algorithms for diagnosis of lung cancer: a systematic review and meta-analysis. Cancers. 2022;14:3856. doi:10.3390/cancers14163856 read 9 October 2026. doi.org/10.3390/cancers14163856
- Katase S, Ichinose A, Hayashi M, Watanabe M, Chin K, Takeshita Y, et al. Development and performance evaluation of a deep learning lung nodule detection system. BMC Medical Imaging. 2022;22:203. doi:10.1186/s12880-022-00938-8 read 9 October 2026. doi.org/10.1186/s12880-022-00938-8
- Suzuki K, Otsuka Y, Nomura Y, Kumamaru KK, Kuwatsuru R, Aoki S. Development and validation of a modified three-dimensional U-Net deep-learning model for automated detection of lung nodules on chest CT images from the Lung Image Database Consortium and Japanese datasets. Academic Radiology. 2022;29:S11-S17. doi:10.1016/j.acra.2020.07.030 read 9 October 2026. doi.org/10.1016/j.acra.2020.07.030
- Mei J, Cheng MM, Xu G, Wan LR, Zhang H. SANet: a slice-aware network for pulmonary nodule detection. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2022;44:4374-4387. doi:10.1109/TPAMI.2021.3065086 read 9 October 2026. doi.org/10.1109/TPAMI.2021.3065086
- Luo X, Song T, Wang G, Chen J, Chen Y, Li K, et al. SCPM-Net: an anchor-free 3D lung nodule detection network using sphere representation and center points matching. Medical Image Analysis. 2022;75:102287. doi:10.1016/j.media.2021.102287 read 9 October 2026. doi.org/10.1016/j.media.2021.102287
- Kaissis G, Ziller A, Passerat-Palmbach J, Ryffel T, Usynin D, Trask A, Lima I, Mancuso J, Jungmann F, Steinborn MM, Saleh A, Makowski M, Rueckert D, Braren R. End-to-end privacy preserving deep learning on multi-institutional medical imaging. Nature Machine Intelligence. 2021;3(6):473-484. doi:10.1038/s42256-021-00337-8 read 9 October 2026. doi.org/10.1038/s42256-021-00337-8
- Oikonomou EK, Siddique M, Antoniades C. Artificial intelligence in medical imaging: a radiomic guide to precision phenotyping of cardiovascular disease. Cardiovascular Research. 2020;116(13):2040-2054. doi:10.1093/cvr/cvaa021 read 9 October 2026. doi.org/10.1093/cvr/cvaa021
- Mongan J, Moy L, Kahn CE. Checklist for Artificial Intelligence in Medical Imaging (CLAIM): a guide for authors and reviewers. Radiology: Artificial Intelligence. 2020;2:e200029. doi:10.1148/ryai.2020200029 read 9 October 2026. doi.org/10.1148/ryai.2020200029
- Liu X, Faes L, Kale AU, Wagner SK, Fu DJ, Bruynseels A, Mahendiran T, Moraes G, Shamdas M, Kern C, Ledsam JR, Schmid MK, Balaskas K, Topol EJ, Bachmann LM, Keane PA, Denniston AK. A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis. The Lancet Digital Health. 2019. doi:10.1016/S2589-7500(19)30123-2 read 9 October 2026. doi.org/10.1016/S2589-7500(19)30123-2
- Ardila D, Kiraly AP, Bharadwaj S, Choi B, Reicher JJ, Peng L, et al. End-to-end lung cancer screening with three-dimensional deep learning on low-dose chest computed tomography. Nature Medicine. 2019;25:954-961. doi:10.1038/s41591-019-0447-x read 9 October 2026. doi.org/10.1038/s41591-019-0447-x
- Luo D, Wan X, Liu J, Tong T. Optimally estimating the sample mean from the sample size, median, mid-range, and/or mid-quartile range. Statistical Methods in Medical Research. 2018;27:1785-1805. doi:10.1177/0962280216669183 read 9 October 2026. doi.org/10.1177/0962280216669183
- McInnes MDF, Moher D, Thombs BD, McGrath TA, Bossuyt PM, et al. Preferred Reporting Items for a Systematic Review and Meta-analysis of Diagnostic Test Accuracy Studies: The PRISMA-DTA Statement. JAMA. 2018;319(4):388-396. doi:10.1001/jama.2017.19163 read 9 October 2026. doi.org/10.1001/jama.2017.19163
- Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig L, Lijmer JG, Moher D, Rennie D, de Vet HCW, Kressel HY, Rifai N, Golub RM, Altman DG, Hooft L, Korevaar DA, Cohen JF; STARD Group. STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies. BMJ. 2015;351:h5527. doi:10.1136/bmj.h5527 read 9 October 2026. doi.org/10.1136/bmj.h5527
- Wan X, Wang W, Liu J, Tong T. Estimating the sample mean and standard deviation from the sample size, median, range and/or interquartile range. BMC Medical Research Methodology. 2014;14:135. doi:10.1186/1471-2288-14-135 read 9 October 2026. doi.org/10.1186/1471-2288-14-135
- Riley RD, Higgins JPT, Deeks JJ. Interpretation of random effects meta-analyses. BMJ. 2011;342:d549. doi:10.1136/bmj.d549 read 9 October 2026. doi.org/10.1136/bmj.d549
- Sterne JAC, Sutton AJ, Ioannidis JPA, Terrin N, Jones DR, Lau J, Carpenter J, Rucker G, Harbord RM, Schmid CH, Tetzlaff J, Deeks JJ, Peters J, Macaskill P, Schwarzer G, Duval S, Altman DG, Moher D, Higgins JPT. Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials. BMJ. 2011;343:d4002. doi:10.1136/bmj.d4002 read 9 October 2026. doi.org/10.1136/bmj.d4002
- Viechtbauer W. Conducting meta-analyses in R with the metafor package. Journal of Statistical Software. 2010;36(3):1-48. doi:10.18637/jss.v036.i03 read 9 October 2026. doi.org/10.18637/jss.v036.i03
- Leeflang MMG, Deeks JJ, Gatsonis C, Bossuyt PMM; Cochrane Diagnostic Test Accuracy Working Group. Systematic reviews of diagnostic test accuracy. Annals of Internal Medicine. 2008;149(12):889-897. doi:10.7326/0003-4819-149-12-200812160-00008 read 9 October 2026. doi.org/10.7326/0003-4819-149-12-200812160-00008
- Bax L, Yu LM, Ikeda N, Tsuruta N, Moons KGM. A systematic comparison of software dedicated to meta-analysis of causal studies. BMC Medical Research Methodology. 2007;7:40. doi:10.1186/1471-2288-7-40 read 9 October 2026. doi.org/10.1186/1471-2288-7-40
- Harbord RM, Deeks JJ, Egger M, Whiting P, Sterne JAC. A unification of models for meta-analysis of diagnostic accuracy studies. Biostatistics. 2007;8:239-251. read 9 October 2026. arxiv.org
- Deeks JJ, Macaskill P, Irwig L. Journal of Clinical Epidemiology. 2005;58:882-893. read 9 October 2026. lshtm-eprints-repo-prod.lshtm-eprints.cdl.cosector.com
- Reitsma JB, Glas AS, Rutjes AWS, Scholten RJPM, Bossuyt PM, Zwinderman AH. Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. Journal of Clinical Epidemiology. 2005;58:982-990. read 9 October 2026. arxiv.org
- Higgins JPT, Thompson SG. Quantifying heterogeneity in a meta-analysis. Statistics in Medicine. 2002;21(11):1539-1558. doi:10.1002/sim.1186 read 9 October 2026. doi.org/10.1002/sim.1186
- Sutton AJ, Abrams KR, Jones DR, Sheldon TA, Song F. Methods for meta-analysis in medical research. Wiley (book). 2000. read 9 October 2026. link.springer.com
- DerSimonian R, Laird N. Meta-analysis in clinical trials. Controlled Clinical Trials. 1986;7(3):177-188. doi:10.1016/0197-2456(86)90046-2 read 9 October 2026. doi.org/10.1016/0197-2456(86)90046-2
- Fleiss JL. Measuring nominal scale agreement among many raters. Psychological Bulletin. 1971;76:378-382. read 9 October 2026. trialsjournal.biomedcentral.com
- Cochran WG. The combination of estimates from different experiments. Biometrics. 1954;10(1):101-129. doi:10.2307/3001666 read 9 October 2026. doi.org/10.2307/3001666
- Veroniki AA, Jackson D, Bender R, Kuss O, Langan D, Higgins JPT, Knapp G, Salanti G. Methods to calculate uncertainty in the estimated overall effect size from a random-effects meta-analysis. doi:10.1002/jrsm.1319 read 9 October 2026. doi.org/10.1002/jrsm.1319
Every source above is a peer-reviewed article, a publisher’s record of one, or a standard issued by the body that owns it. Each carries the date it was last read. Where a standard moves, this article is revised rather than left up with a note.
How to cite this article
Pick your referencing style and copy it. Your university or target journal may vary the punctuation — check its guide if it has one.
Pubrica Academy. (2026, October 9). Meta-Analysis of Convolutional Neural Network (CNN) Performance on Radiological Images. Pubrica. https://pubrica.com/academy/medical-writing/meta-analysis-of-convolutional-neural-networks-for-radiological-images/
Pubrica Academy. “Meta-Analysis of Convolutional Neural Network (CNN) Performance on Radiological Images.” Pubrica, 9 Oct. 2026, pubrica.com/academy/medical-writing/meta-analysis-of-convolutional-neural-networks-for-radiological-images/.
Pubrica Academy (2026) Meta-Analysis of Convolutional Neural Network (CNN) Performance on Radiological Images. Pubrica. Available at: https://pubrica.com/academy/medical-writing/meta-analysis-of-convolutional-neural-networks-for-radiological-images/ (Accessed: 15 October 2026).
Pubrica Academy. “Meta-Analysis of Convolutional Neural Network (CNN) Performance on Radiological Images.” Pubrica. October 9, 2026. https://pubrica.com/academy/medical-writing/meta-analysis-of-convolutional-neural-networks-for-radiological-images/
Pubrica Academy, ‘Meta-Analysis of Convolutional Neural Network (CNN) Performance on Radiological Images’ (Pubrica, 9 October 2026) <https://pubrica.com/academy/medical-writing/meta-analysis-of-convolutional-neural-networks-for-radiological-images/> accessed 15 October 2026.
Pubrica Academy. Meta-Analysis of Convolutional Neural Network (CNN) Performance on Radiological Images. [Internet]. Pubrica; 2026 Oct 9 [cited 2026 Oct 15]. Available from: https://pubrica.com/academy/medical-writing/meta-analysis-of-convolutional-neural-networks-for-radiological-images/
Pubrica Academy, “Meta-Analysis of Convolutional Neural Network (CNN) Performance on Radiological Images,” Pubrica, Oct. 9, 2026. [Online]. Available: https://pubrica.com/academy/medical-writing/meta-analysis-of-convolutional-neural-networks-for-radiological-images/
@misc{meta_analysis_of_convolutional_neural__2026,
author = {{Pubrica Academy}},
title = {Meta-Analysis of Convolutional Neural Network (CNN) Performance on Radiological Images},
howpublished = {Pubrica},
year = {2026},
month = {oct},
url = {https://pubrica.com/academy/medical-writing/meta-analysis-of-convolutional-neural-networks-for-radiological-images/},
note = {Accessed: <span data-cite-acc="iso">2026-10-15</span>}
}This article is written to be learned from. Cite it if you are referring to what it says; if you are using the evidence behind it, cite the sources listed above instead.
Protocol stub for a CNN imaging accuracy review
This stub should be completed before the search. Its purpose is to identify exclusions that would otherwise emerge at the synthesis stage, and to fix the stratification before any estimate exists.
CNN IMAGING ACCURACY REVIEW — PROTOCOL STUB
1. REVIEW TYPE (choose one; they are not interchangeable)
[ ] Diagnostic accuracy synthesis (needs 2x2 data)
[ ] Algorithm vs reader comparison (needs SAME sample for both)
[ ] Mapping / scoping review (no pooled estimate)
2. TARGET CONDITION AND GROUND TRUTH
Target condition, defined precisely: ______________________
Acceptable ground truth, pre-specified list:
[ ] Histopathology [ ] Follow-up imaging at ____ months
[ ] Expert consensus, ____ readers [ ] Other: ____________
Studies using anything else: excluded, and counted.
3. UNIT OF ANALYSIS (one per stratum; strata analysed separately)
[ ] Lesion-wise [ ] Image-wise [ ] Scan-wise [ ] Patient-wise
Negative-class convention required:
[ ] Specificity (true negatives defined)
[ ] False positives per scan <-- cannot be pooled with the above
4. VALIDATION DESIGN (stratify; do not merge)
[ ] Independent / external dataset — primary stratum
[ ] Held-out split of development data — secondary stratum
Record per study: development cohort, validation cohort, whether
institutions or scanners overlap, whether patients overlap.
Algorithm count to be reported per stratum: [ ]
5. EXTRACTION REQUIREMENTS
[ ] TP / FP / FN / TN at a stated threshold, or reconstructable
[ ] Threshold provenance: pre-specified / inherited / optimised here
[ ] Scanner, vendor, slice thickness, reconstruction
[ ] Prevalence in the validation sample
[ ] Model family and version; training set size
Studies with AUC only: [ ] excluded — COUNT: ____
[ ] authors contacted — log kept
6. SYNTHESIS
[ ] Bivariate random-effects (common threshold)
[ ] Hierarchical summary ROC (thresholds differ)
Confidence interval method, pre-specified: ______________________
Dependency handling for multiple results per study: ______________
7. HETEROGENEITY AND BIAS
Expected poolable studies: ____
[ ] Heterogeneity sources TABULATED (not regressed) : scanner /
protocol / nodule size / prevalence / ground truth / threshold /
site overlap
[ ] If fewer than 10 studies: no funnel plot asymmetry test
[ ] Limitations to name explicitly: unpublished underperforming
models; iterated versions; shared benchmark datasets
8. APPRAISAL AND REPORTING
[ ] QUADAS-2 per included study, with supporting quotes
[ ] CLAIM completeness recorded per study; tally reported
[ ] Review reported following PRISMA-DTA (reporting, not conduct) Pubrica AcademyWritten and maintained by Pubrica’s subject matter experts. Articles are revised when the underlying standards change, and each article states the date on which it was last reviewed.
How Pubrica can help
Pubrica scopes AI imaging accuracy reviews against what the literature reports rather than what it claims, so that the poolable yield is known before the search closes, and fits the bivariate or hierarchical model with a pre-specified interval method once a study set exists. Data are extracted against CLAIM and appraised with QUADAS-2; where pooling is not defensible, Pubrica develops a mapping review and reports the reporting gaps as findings.