Contents
Academy · Research Data, Analytics & AI · How Machine learning help in fighting against COVID-19
Machine Learning Prediction Models for COVID-19: Contributions and Methodological Limitations
Hundreds of prediction models were published for COVID-19 diagnosis and prognosis. A systematic review of nearly all of them concluded that almost none were ready for clinical use. This article summarises the review’s findings and their implications for evaluating rapidly developed prediction models.
Machine learning contributed to the COVID-19 response primarily through prediction models for diagnosis (distinguishing COVID-19 from other conditions) and prognosis (predicting disease progression or mortality risk). However, a large living systematic review of these models found that the great majority carried a high risk of bias, which limited the extent to which this work produced models ready for clinical deployment.
Learning objectives
- What the systematic review of COVID-19 prediction models by Wynants and colleagues found.
- The specific methodological problems the review identified as recurring across most models.
- Why pressure for rapid development and methodological rigour were in tension during this period.
- What this episode indicates about the evaluation of any rapidly produced prediction model.
Common misconceptions
- “Hundreds of published COVID-19 prediction models represent hundreds of validated tools available to clinicians.” The systematic review and critical appraisal by Wynants and colleagues found that the large majority of published models carried a high risk of bias, largely owing to non-representative patient selection, unclear or inappropriate handling of predictors, and lack of external validation; publication volume therefore did not equate to clinical readiness.
- “High reported accuracy during a health emergency is sufficient to justify a model’s use.” The review’s critical appraisal specifically noted that many models were developed and validated on small, non-representative samples under time pressure, which can produce high apparent accuracy in the original sample but poor performance when applied elsewhere.
Findings of the Wynants systematic review
Wynants and colleagues conducted a systematic review and critical appraisal of prediction models for COVID-19 diagnosis and prognosis, published in the BMJ and subsequently maintained as a living, continuously updated review as more models were published. Their critical appraisal, using a structured risk-of-bias assessment tool for prediction models, found that the substantial majority of the models reviewed were rated at high risk of bias, meaning their reported performance could not be relied upon to generalise beyond the specific sample and conditions under which they were developed.
Recurring methodological problems
The review identified recurring, specific problems across many of the models assessed: patient samples that were small or not representative of the populations in which the model was intended to be used, predictor variables that were poorly defined or handled in ways likely to overstate performance, and, most consistently, an absence of external validation on data independent of the development sample. This last deficiency reflects the general caution discussed in the related article on machine learning models in data extraction: a model's performance on its own training or development data is not evidence that it generalises.
Wider implications for prediction-model research
The findings of the Wynants review reflect a specific tension of this period: the speed at which models needed to be developed and published to be useful during a fast-moving public health emergency worked directly against the time and data typically needed to validate a model properly. The implication extends beyond COVID-19: any rapidly produced prediction model, in any health emergency, should be assessed against the same risk-of-bias criteria that this review applied before being treated as ready for clinical decision-making, irrespective of the perceived urgency of the need at the time.
Key points
Key points
Machine learning produced a very large volume of COVID-19 diagnosis and prognosis prediction models during the pandemic, but a systematic, critical appraisal found the large majority at high risk of bias owing to non-representative samples, poor predictor handling, and lack of external validation; publication volume under time pressure is therefore not equivalent to validated, deployment-ready performance.
Practical application
Practical application
- Any rapidly developed prediction model should be assessed with a structured risk-of-bias tool. A recognised prediction-model risk-of-bias assessment should be applied before a model’s reported performance is relied upon, particularly where the model was developed under emergency time pressure.
- External validation should be required before deployment. A model’s performance on its own development sample should be regarded as preliminary, and validation on an independent dataset required before the model is used to inform decisions.
- Sample representativeness should be documented explicitly. The comparison between the development sample and the intended-use population should be stated, since a mismatch was one of the most common sources of overstated performance identified in the review.
Frequently asked questions
The review identified a small number of models with more favourable methodological assessments relative to the majority, though it emphasised that even these required further independent validation before clinical use.
The review was maintained as a living review with periodic updates during the period it tracked new publications; current status should be checked against the review's most recent published version.
Not necessarily; it means the available evidence does not establish that its performance will generalise beyond the development sample, which is a different claim from the model being definitely inaccurate.
References
- Wynants L, Van Calster B, Collins GS, et al. Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal. BMJ. 2020;369:m1328. read 10 October 2026. View on The BMJ
Every source above is a peer-reviewed article, a publisher’s record of one, or a standard issued by the body that owns it. Each carries the date it was last read. Where a standard moves, this article is revised rather than left up with a note.
How to cite this article
Pick your referencing style and copy it. Your university or target journal may vary the punctuation — check its guide if it has one.
Pubrica Academy. (2026, October 10). Machine Learning Prediction Models for COVID-19: Contributions and Methodological Limitations. Pubrica. https://pubrica.com/academy/concepts-definitions/how-machine-learning-help-in-fighting-against-covid-19/
Pubrica Academy. “Machine Learning Prediction Models for COVID-19: Contributions and Methodological Limitations.” Pubrica, 10 Oct. 2026, pubrica.com/academy/concepts-definitions/how-machine-learning-help-in-fighting-against-covid-19/.
Pubrica Academy (2026) Machine Learning Prediction Models for COVID-19: Contributions and Methodological Limitations. Pubrica. Available at: https://pubrica.com/academy/concepts-definitions/how-machine-learning-help-in-fighting-against-covid-19/ (Accessed: 15 October 2026).
Pubrica Academy. “Machine Learning Prediction Models for COVID-19: Contributions and Methodological Limitations.” Pubrica. October 10, 2026. https://pubrica.com/academy/concepts-definitions/how-machine-learning-help-in-fighting-against-covid-19/
Pubrica Academy, ‘Machine Learning Prediction Models for COVID-19: Contributions and Methodological Limitations’ (Pubrica, 10 October 2026) <https://pubrica.com/academy/concepts-definitions/how-machine-learning-help-in-fighting-against-covid-19/> accessed 15 October 2026.
Pubrica Academy. Machine Learning Prediction Models for COVID-19: Contributions and Methodological Limitations. [Internet]. Pubrica; 2026 Oct 10 [cited 2026 Oct 15]. Available from: https://pubrica.com/academy/concepts-definitions/how-machine-learning-help-in-fighting-against-covid-19/
Pubrica Academy, “Machine Learning Prediction Models for COVID-19: Contributions and Methodological Limitations,” Pubrica, Oct. 10, 2026. [Online]. Available: https://pubrica.com/academy/concepts-definitions/how-machine-learning-help-in-fighting-against-covid-19/
@misc{ml_help_fighting_covid_19_2026,
author = {{Pubrica Academy}},
title = {Machine Learning Prediction Models for COVID-19: Contributions and Methodological Limitations},
howpublished = {Pubrica},
year = {2026},
month = {oct},
url = {https://pubrica.com/academy/concepts-definitions/how-machine-learning-help-in-fighting-against-covid-19/},
note = {Accessed: <span data-cite-acc="iso">2026-10-15</span>}
}This article is written to be learned from. Cite it if you are referring to what it says; if you are using the evidence behind it, cite the sources listed above instead.
Rapid prediction-model risk-of-bias stub
This checklist applies to the evaluation of a rapidly developed prediction model.
Development sample size and representativeness: ____ Predictor definition and handling reviewed? Y/N External validation performed? Y/N, on what data Risk-of-bias tool applied: ____ Overall risk-of-bias rating: ____ Decision: deploy / validate further / reject
Pubrica AcademyWritten and maintained by Pubrica’s subject matter experts. Articles are revised when the underlying standards change, and each article states the date on which it was last reviewed.
How Pubrica can help
Pubrica’s data-analytics team applies structured risk-of-bias assessment to rapidly developed prediction models and designs the external validation studies required before such models are used for clinical or research decisions.