Free scoping read. A written assessment of your manuscript in two business days, at no cost.

Start one
Contents

Academy · Research Data, Analytics & AI · How Machine learning help in fighting against COVID-19

Machine Learning Prediction Models for COVID-19: Contributions and Methodological Limitations

Hundreds of prediction models were published for COVID-19 diagnosis and prognosis. A systematic review of nearly all of them concluded that almost none were ready for clinical use. This article summarises the review’s findings and their implications for evaluating rapidly developed prediction models.

Pubrica AcademyWritten by a data-analytics reviewer Reviewed bya systematic-review methodologist Published10 October 2026 Last checked10 October 2026 Reading time8 minutes Sources1 peer-reviewed

Machine learning contributed to the COVID-19 response primarily through prediction models for diagnosis (distinguishing COVID-19 from other conditions) and prognosis (predicting disease progression or mortality risk). However, a large living systematic review of these models found that the great majority carried a high risk of bias, which limited the extent to which this work produced models ready for clinical deployment.

Learning objectives

  • What the systematic review of COVID-19 prediction models by Wynants and colleagues found.
  • The specific methodological problems the review identified as recurring across most models.
  • Why pressure for rapid development and methodological rigour were in tension during this period.
  • What this episode indicates about the evaluation of any rapidly produced prediction model.

Common misconceptions

  • “Hundreds of published COVID-19 prediction models represent hundreds of validated tools available to clinicians.” The systematic review and critical appraisal by Wynants and colleagues found that the large majority of published models carried a high risk of bias, largely owing to non-representative patient selection, unclear or inappropriate handling of predictors, and lack of external validation; publication volume therefore did not equate to clinical readiness.
  • “High reported accuracy during a health emergency is sufficient to justify a model’s use.” The review’s critical appraisal specifically noted that many models were developed and validated on small, non-representative samples under time pressure, which can produce high apparent accuracy in the original sample but poor performance when applied elsewhere.

Findings of the Wynants systematic review

Wynants and colleagues conducted a systematic review and critical appraisal of prediction models for COVID-19 diagnosis and prognosis, published in the BMJ and subsequently maintained as a living, continuously updated review as more models were published. Their critical appraisal, using a structured risk-of-bias assessment tool for prediction models, found that the substantial majority of the models reviewed were rated at high risk of bias, meaning their reported performance could not be relied upon to generalise beyond the specific sample and conditions under which they were developed.

Recurring methodological problems

The review identified recurring, specific problems across many of the models assessed: patient samples that were small or not representative of the populations in which the model was intended to be used, predictor variables that were poorly defined or handled in ways likely to overstate performance, and, most consistently, an absence of external validation on data independent of the development sample. This last deficiency reflects the general caution discussed in the related article on machine learning models in data extraction: a model's performance on its own training or development data is not evidence that it generalises.

Wider implications for prediction-model research

The findings of the Wynants review reflect a specific tension of this period: the speed at which models needed to be developed and published to be useful during a fast-moving public health emergency worked directly against the time and data typically needed to validate a model properly. The implication extends beyond COVID-19: any rapidly produced prediction model, in any health emergency, should be assessed against the same risk-of-bias criteria that this review applied before being treated as ready for clinical decision-making, irrespective of the perceived urgency of the need at the time.

Key points

Key points

Machine learning produced a very large volume of COVID-19 diagnosis and prognosis prediction models during the pandemic, but a systematic, critical appraisal found the large majority at high risk of bias owing to non-representative samples, poor predictor handling, and lack of external validation; publication volume under time pressure is therefore not equivalent to validated, deployment-ready performance.

Practical application

Practical application

  1. Any rapidly developed prediction model should be assessed with a structured risk-of-bias tool. A recognised prediction-model risk-of-bias assessment should be applied before a model’s reported performance is relied upon, particularly where the model was developed under emergency time pressure.
  2. External validation should be required before deployment. A model’s performance on its own development sample should be regarded as preliminary, and validation on an independent dataset required before the model is used to inform decisions.
  3. Sample representativeness should be documented explicitly. The comparison between the development sample and the intended-use population should be stated, since a mismatch was one of the most common sources of overstated performance identified in the review.

Frequently asked questions

The review identified a small number of models with more favourable methodological assessments relative to the majority, though it emphasised that even these required further independent validation before clinical use.

The review was maintained as a living review with periodic updates during the period it tracked new publications; current status should be checked against the review's most recent published version.

Not necessarily; it means the available evidence does not establish that its performance will generalise beyond the development sample, which is a different claim from the model being definitely inaccurate.

References

  1. Wynants L, Van Calster B, Collins GS, et al. Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal. BMJ. 2020;369:m1328. read 10 October 2026. View on The BMJ

Every source above is a peer-reviewed article, a publisher’s record of one, or a standard issued by the body that owns it. Each carries the date it was last read. Where a standard moves, this article is revised rather than left up with a note.

How to cite this article

Pick your referencing style and copy it. Your university or target journal may vary the punctuation — check its guide if it has one.

Pubrica Academy. (2026, October 10). Machine Learning Prediction Models for COVID-19: Contributions and Methodological Limitations. Pubrica. https://pubrica.com/academy/concepts-definitions/how-machine-learning-help-in-fighting-against-covid-19/

This article is written to be learned from. Cite it if you are referring to what it says; if you are using the evidence behind it, cite the sources listed above instead.

Rapid prediction-model risk-of-bias stub

This checklist applies to the evaluation of a rapidly developed prediction model.

Development sample size and representativeness: ____
Predictor definition and handling reviewed? Y/N
External validation performed? Y/N, on what data
Risk-of-bias tool applied: ____
Overall risk-of-bias rating: ____
Decision: deploy / validate further / reject

Pubrica AcademyWritten and maintained by Pubrica’s subject matter experts. Articles are revised when the underlying standards change, and each article states the date on which it was last reviewed.

How Pubrica can help

Pubrica’s data-analytics team applies structured risk-of-bias assessment to rapidly developed prediction models and designs the external validation studies required before such models are used for clinical or research decisions.

Find something else

Search the Pubrica Academy

Access is free and requires no account. Enter a topic, or open the full library to filter by category, sub-topic or resource type.

Risk-of-bias assessment of rapidly developed prediction models

A scoping assessment is free and takes two business days. It provides a written assessment of what the work requires and which specialist would undertake it.

Get a free quote

Regulations

Check your project against this rule

  1. 1Your work
  2. 2Your details
  3. 3Send
What do you need?

Material you send is held under signed confidentiality, with access limited to the assigned team and encrypted transfer. It is not used to train anything and is not passed to third-party tools. Ask for the agreement before you send anything you would rather cover first.