Free scoping read. A written assessment of your manuscript in two business days, at no cost.

Start one
Contents

Academy · Research Data Analytics & AI · ML in pharma reviews

Systematic Reviews of Machine Learning in the Pharmaceutical Industry: Scope, Appraisal and Synthesis

“Machine learning in the pharmaceutical industry” is not a single subject and cannot be addressed by a single review. It spans target identification, molecular property prediction, trial operations, manufacturing control and pharmacovigilance, domains with different data, different success criteria and different evidence standards. A review that does not select one domain, and does not state the comparator against which the models are evaluated, produces a catalogue rather than a synthesis.

Pubrica AcademyWritten by the medical writing and biostatistics teams Reviewed byPubrica biostatistics team Published9 October 2026 Last checked9 October 2026 Reading time13 minutes Sources32 peer-reviewed

The literature on machine learning in drug research grows faster than any review can cover, and reports its results in a form that makes synthesis unusually difficult: retrospective performance on curated datasets, with success criteria chosen by the authors. The useful question for a reviewer is rarely “how well do the models perform”, because the reported figures are not commensurable. It is rather what has been attempted, against which baseline, with what validation and with what prospective confirmation; the last of these is where the literature is weakest.

Learning objectives

  • Scope the review to one domain of pharmaceutical machine learning rather than the whole field
  • Frame a question where the object under study is a method rather than a treatment
  • Require a baseline: the comparator against which the model was evaluated, and over what time horizon
  • Search a literature that appears partly in preprints, patents and conference proceedings
  • Extract the fields that distinguish a validated model from a well-fitted one
  • Determine whether any part of the included set can be pooled, and what to present when it cannot

Common misconceptions

  • “Reported performance metrics across studies can be compared.” Different datasets, splits, endpoints and author-selected metrics render the numbers incommensurable. A table of such figures has the appearance of evidence without its substance.
  • “A model that outperforms a baseline in a retrospective benchmark is validated.” Retrospective benchmark performance is a hypothesis about prospective performance. The review should report which studies tested that hypothesis; most have not.
  • “Drug discovery success can be attributed to a method.” Attribution over a development timeline this long is extremely difficult. Analyses of where new drugs come from span decades for a reason — one well-known classification of natural products as sources of new drugs covers approvals from 1981 to 2014, a 34-year window.
  • “Published code and data make a study reproducible.” They make it potentially reproducible. Whether the training data are available, whether the split can be regenerated and whether the evaluation set was ever used for selection are separate questions, and the extraction form should address all three.
  • “Searching MEDLINE and Embase covers this field.” Substantial parts of the field are reported first in preprints and conference proceedings, and commercially significant work appears in patents or not at all. A search restricted to bibliographic databases has a structural gap that should be acknowledged.

Five domains without a shared evidence standard

The phrase covers at least five distinct activities, and the first task of the protocol is to select one.

Discovery and target work: target identification and prioritisation, virtual screening, molecular property and activity prediction, generative design 7,8. Evaluated against retrospective benchmarks, occasionally against prospective synthesis and assay.

Preclinical and translational: toxicity prediction, pharmacokinetic modelling, biomarker discovery, image-based phenotyping 7,29. Evaluated against experimental ground truth that is itself variable.

Clinical development: site and patient selection, eligibility screening, dropout prediction, endpoint adjudication, synthetic control arms. Evaluated against trial operational outcomes, and occasionally regulated.

Manufacturing and quality: process control, deviation prediction, batch release support. Evaluated against process outcomes in a regulated environment with its own validation expectations.

Post-market: signal detection in pharmacovigilance, literature surveillance, real-world evidence generation. Evaluated against reference signal sets, with a well-known difficulty in defining truth.

These domains cannot share one eligibility criterion, one appraisal instrument or one outcome definition. A review spanning them is a map, and should be labelled as a scoping review 13,15. A review intended to synthesise evidence must select one domain and one task within it. The pages on AI use cases in clinical development, machine learning in computational pathology and literature surveillance in pharmacovigilance each cover one of these domains in its own terms.

Five domains, and why they cannot share one review

DomainTypical taskWhat counts as ground truthWhat the strongest available evidence looks like
Discovery and target workVirtual screening, property and activity prediction, generative design, target prioritisationAssay results, where they exist; otherwise a curated benchmark labelProspective synthesis and assay of predicted compounds, reported with the chronology
Preclinical and translationalToxicity and pharmacokinetic prediction, biomarker discovery, image-based phenotypingExperimental measurement, itself variable between labs and protocolsExternal validation on data from a different laboratory or platform
Clinical developmentSite and patient selection, dropout prediction, eligibility screening, endpoint adjudicationTrial operational records and clinical outcomesProspective use in a trial with the operational outcome reported
Manufacturing and qualityProcess control, deviation prediction, batch release supportProcess and quality outcomes in a regulated environmentValidated deployment with documented performance over time
Post-marketPharmacovigilance signal detection, literature surveillance, real-world evidenceReference signal sets, with a known difficulty in defining truthEvaluation against an independently constructed reference set

Compiled as a scoping structure; the domains differ in data, ground truth and evidence standard, which is why one eligibility criterion cannot serve all five.

Framing a review question when the object of study is a method

When the object under study is a computational method rather than a treatment, the intervention-review frame does not apply, and the elements that do are the task, the input data, the baseline, the validation design and the criterion of success 32.

The task must be stated as an input and an output: predict a measured property for a compound given a structure, rank candidates for a target given assay data, classify a trial site by enrolment risk given historical records. “Applying machine learning to drug discovery” is not a task.

The baseline is the element most reviews omit and the one that determines whether any included study is interpretable. A model with no stated comparator makes no claim. Legitimate baselines include an established physics-based or statistical method, a simpler model of the same type, a human expert workflow, or a documented historical rate. If a study has none, this should be recorded — the proportion of studies without a baseline is a review-level finding.

The validation design ranges from a random split of a single dataset, through scaffold or temporal splits, to evaluation on data generated after the model was built, to prospective experimental confirmation. These are not degrees of the same property: a random split and a prospective assay answer different questions, and a review that combines them yields no interpretable result.

The criterion of success must be declared because authors select it. A model that improves a ranking metric and a model that yields a confirmed active compound are both described as “successful”. The review should specify which of these it addresses. See framing review questions for the reasons why specifying a different element set is preferable to forcing the PICO (population, intervention, comparator, outcome) framework.

Baselines and time horizons

Claims that machine learning is changing pharmaceutical research are claims about a counterfactual over a long horizon, and the horizon is the central difficulty. A compound entering discovery today reaches approval, if at all, after a timeline measured in years, which means that attributing an approval to a method used at the outset is a difficult inferential problem rather than a reporting one.

The scale of that difficulty is visible in how the field studies drug origins at all: a well-known classification of natural products as sources of new drugs covers approvals from 1981 to 2014, a 34-year window 19. Analyses of where new medicines come from are built on multi-decade windows because that is the timescale on which anything can be said about sources of new drugs 19,24. A review of machine learning in discovery is therefore examining a literature whose outcome of interest largely has not had time to be observed.

Three consequences follow for the review. The outcome under review should be stated precisely — retrospective predictive performance, prospective experimental confirmation, candidates entering development, or approvals — and the discussion should not shift between them. The strongest evidence is likely to be at the retrospective end and the claims at the approval end, and the review should state this. Where a study claims a discovery outcome, the chronology should be extracted: when the model was built, when the prediction was made, when the experimental confirmation occurred, and whether the confirmation was independent of those who made the prediction.

1981–2014approval window covered by one widely cited classification of natural products as sources of new drugs
34years of approvals in that single analysis

The time horizon on which questions about the sources of new drugs are examined, from the classification of natural products as sources of new drugs covering approvals from 1981 to 2014. It is given here to indicate the timescale over which discovery outcomes become observable, not as a performance figure for any method.

Searching a literature that extends beyond journals

Four source types, and a bibliographic-only search misses two of them 1.

Bibliographic databases cover the peer-reviewed journal literature and should be searched database-by-database in the usual way — see sources of review data. The terminology problem is acute here: the same work is described as machine learning, deep learning, artificial intelligence, data-driven modelling, cheminformatics, quantitative structure-activity modelling or simply as a named architecture, and indexing lags behind all of it 12.

Preprint servers and conference proceedings carry a substantial share of methods work, often months or years before journal publication and sometimes instead of it. The protocol should specify whether preprints are eligible and, if so, how the later journal version and any changes between the two will be handled 4. The decision should be reported in either case.

Patents and regulatory filings are where commercially significant work surfaces, if it surfaces 23. Methodological detail can rarely be extracted from them, and a review that omits them should state that commercially developed methods are structurally underrepresented in its evidence base. Such a statement is more informative than an unqualified claim about the state of the field.

Code and model repositories are primary sources about the artefacts themselves: whether code exists, whether weights are released, and what the training data were. Searching them is unusual in a review but is worthwhile when reproducibility is part of the question.

Where a field’s evidence is partly unpublished, a transparent review states the gap specifically rather than describing its search as comprehensive.

A search strategy for machine learning in pharmaceutical research

Retrieves the wrong set and misses the field

("machine learning" OR "artificial intelligence") AND ("drug discovery" OR "pharmaceutical") — searched in PubMed and Google Scholar, 2015 to present.

Two method terms only. The same work is published as deep learning, QSAR, cheminformatics or a named architecture, and all of that is outside this string.

No controlled vocabulary. Free-text-only searching in a database with mature indexing forgoes the most reliable component of retrieval.

“Drug discovery” as the task concept. It retrieves the entire pharmaceutical literature and still misses papers that describe their task specifically.

A date limit of 2015. It excludes the modelling traditions the field was built on, and it makes any claim about recent growth circular.

Google Scholar as a database. It cannot be searched with a documented, stable strategy, so it cannot be reported in the form a systematic review requires.

No preprint or proceedings coverage. A substantial share of methods work appears there first, and sometimes exclusively, so the search has a structural gap that is never declared.

No statement of what is unreachable. Commercially developed methods are invisible to this search and the review will read as a description of the whole field.

Covers the terminology and names its gaps

Searched in MEDLINE via Ovid, Embase via Ovid and Web of Science Core Collection, each on a stated date with the full line-by-line strategy in the supplement. The method concept combined controlled vocabulary terms for machine learning, artificial intelligence and neural networks with free-text terms for the architecture families and modelling traditions the same work is published under, including deep learning, neural network, random forest, gradient boosting, support vector, graph neural network, transformer, variational autoencoder, quantitative structure-activity relationship, cheminformatics and data-driven modelling, each truncated appropriately for the platform. The task concept was restricted to the single domain under review and expressed as its own set of terms — for a molecular property prediction review, terms for virtual screening, activity prediction, property prediction, hit identification and lead optimisation — rather than the phrase drug discovery, which retrieves the whole of pharmaceutical research. No date limit was applied, because restricting to recent years excludes the modelling traditions the current literature builds on and makes any temporal claim circular. Separately, arXiv, bioRxiv and ChemRxiv were searched on a stated date with a simplified strategy recorded in the supplement, and the proceedings of the named machine learning conferences were searched for the same period; the protocol states that where a preprint and a later journal version both exist the journal version is used and any change in the reported result between the two is recorded. We did not search patent databases, and we state in the limitations that methods developed commercially are therefore structurally underrepresented in this evidence base. Code repositories were searched only to establish artefact availability for already-included studies, not to identify studies, and this is reported as a supplementary method.

The method concept spans the vocabularies under which the same work is published, including the modelling traditions that predate the current terminology.

Controlled vocabulary and free text are combined per platform, which is what makes retrieval both sensitive and reportable.

The task concept is restricted to one domain and expressed in task terms, so the retrieved set is the review’s subject rather than the whole field.

No date limit is applied, and the reason is stated, which removes a circularity from any claim the review makes about change over time.

Preprints and proceedings are searched as named sources with their own recorded strategy, and the preprint-versus-journal handling rule is pre-specified.

The patent gap is declared explicitly, so the review’s claims are about the published evidence base rather than about the field.

Repository searching is labelled as a supplementary method for a specific purpose, so it is not mistaken for part of the identification strategy.

The first string will retrieve tens of thousands of records, most of which are irrelevant, while the most relevant are missing — the least favourable combination. The second is longer, reportable line by line, and states the two gaps — patents and unpublished work — that determine what the review may claim about the field as a whole.

Data extraction from machine learning studies

A model paper does not fit a clinical extraction form 2. Six blocks cover the information a reader requires.

Task and data. Input representation, output definition, data source and whether it is public, sample size in the relevant unit — compounds, assays, patients, sites — label provenance and label noise, and class balance.

Split and leakage controls. How the data were split, whether the split was random, scaffold-based, temporal or by source; whether any preprocessing, feature selection or hyperparameter choice used the evaluation data; and whether near-duplicate records appear on both sides of the split, which in chemical datasets is a standard and under-reported failure.

Model and training. Model family, feature representation, how hyperparameters were chosen, and whether the comparison baselines were tuned with the same effort as the proposed model — an untuned baseline is not a valid baseline.

Evaluation. Metrics, with the primary one identified; uncertainty; performance against the baseline rather than in isolation; and performance in the relevant subsets, such as chemical series or patient subgroups.

Prospective component. Whether any prediction was subsequently tested, how many predictions were tested, who tested them, and what the outcome was. Absence should be recorded explicitly; this field is empty for most of the literature, and the proportion is itself a finding.

Availability. Code, trained model, data, and the terms of access. The general extraction approach is described on the page on what data to extract.

Appraisal: data leakage, baselines and optimism

There is no single accepted risk-of-bias instrument for machine learning methods papers across these domains. Where the model is a clinical prediction model, a prediction-model risk-of-bias instrument applies and should be named with its version. Where it is not — for example, a virtual screening pipeline or a process control model — a transparent item list should be used and results reported item by item rather than as an invented score.

Five items merit inclusion in any such list. Leakage, in all its forms: evaluation data used in selection, duplicate or near-duplicate records across the split, temporal leakage where future information is available to a model predicting the past. Baseline adequacy: whether a comparator existed and whether it was given the same tuning effort. Metric selection: whether the primary metric was declared in advance or appears to have been chosen after the results. Dataset provenance: whether the benchmark is one on which many models have been developed and tuned, in which case reported performance on it carries community-wide optimism. And prospective confirmation: present or absent.

These should be reported as a matrix of studies against items. This matrix is usually the most informative output of a review in this area, because it indicates what the literature permits readers to conclude — which is often less than the literature claims. The page on regulatory affairs and causal inference in machine learning covers the appraisal questions that arise when a model enters a regulated process.

Appraisal items for a machine learning study, reported individually and never as a score

ItemWhat to look forWhy it matters
LeakageEvaluation data used in preprocessing, feature selection, hyperparameter choice or threshold setting; near-duplicate records across the split; future information available to a model predicting the pastEvery form inflates reported performance, and none is visible in the headline metric
Baseline adequacyWhether a comparator existed at all, what it was, and whether it received the same tuning effort as the proposed modelAn untuned baseline produces an artificial improvement; the absence of a baseline means that no claim is made
Split strategyRandom, scaffold, temporal, by source, or evaluation on data generated after the model was builtA random split answers a different question from a temporal or source-held-out split
Metric selectionWhether a primary metric was declared in advance, and whether reported metrics favour the proposed modelPost hoc metric choice is the least costly and least detectable form of selective reporting
Benchmark provenanceWhether the dataset is one on which many models have been developed and tunedShared benchmarks accumulate community-wide optimism that no single study can detect
Prospective confirmationWhether any prediction was tested afterwards, how many, by whom, and with what resultThis is the scarcest evidence in the field and the only direct test of the retrospective hypothesis
AvailabilityCode, trained model, training data, and the terms of accessDetermines whether any claim in the paper can be independently checked

Compiled as a transparent item list. Where the model is a clinical prediction model, a published prediction-model risk-of-bias instrument applies instead and should be named with its version.

Pooling and preferable alternatives

Meta-analysis of model performance across studies is almost never defensible in this field 2. The estimates derive from different datasets of differing difficulty, different endpoints, different split strategies and different metrics; an average of them estimates no defined quantity. Where a set of studies evaluated different models on the same public benchmark with the same split, a structured comparison is possible — but the unit of evidence is then the benchmark rather than the study, and the shared-benchmark optimism applies to all of them equally.

The following alternatives are listed in descending order of usefulness. A task-by-task structured synthesis, with one table per task giving each study’s data, split, baseline, primary metric and prospective status. A baseline-relative summary: how many studies beat a stated baseline, how many had none, and how many were compared only against an untuned alternative. A prospective-evidence count: how many included studies subsequently tested a prediction, and with what result. Finally, an evidence gap map of tasks against validation levels, which shows where the field has conducted only retrospective work. See evidence gap mapping and, where pooling genuinely applies, meta-analysis methodology.

The approach to avoid is a forest plot of accuracy figures drawn from unrelated datasets, with a heterogeneity statistic reported as though it were a finding. Such a plot is interpreted as evidence but contains none.

Reporting the review and the evidence gaps identified

The review process should be reported in accordance with the PRISMA 2020 statement, and if the design is a map rather than a synthesis, it should be reported as a scoping review and labelled as such in the title 10,13. The structural limits of the search should be stated specifically: which preprint servers were and were not searched, whether patents were searched, and, where applicable, that commercially developed methods are underrepresented.

Beyond the standard items, three review-level outputs should be specified in the protocol because they make this kind of review reusable: the study-by-item appraisal matrix; the counts of studies with no baseline, with an untuned baseline, and with prospective confirmation; and a defined research agenda stating which task, evaluated on which data, against which baseline and with what prospective test, would advance the question. A review that concludes with such an agenda is useful to the field; one that concludes with “further research is needed” is not.

Where performance figures are reported at all, the dataset should remain attached to the number in the same cell. A metric without its dataset is not a result, and separating them in a table invites precisely the cross-study comparison that the review should caution against.

Limits of what this literature can currently support

Four statements should be made explicitly in a discussion section, because their absence causes reviews in this area to read as promotional.

Retrospective benchmark performance does not establish prospective usefulness. It constitutes a hypothesis about it, and that hypothesis has been tested for only a small fraction of published models.

Nothing in the published methods literature establishes an effect on development timelines or success rates. Those outcomes occur on a timescale over which attribution to a single method is extremely difficult — the multi-decade windows used in analyses of where new drugs come from, such as the 1981 to 2014 span of one widely cited natural products classification, exist because that is the timescale on which such questions can be examined at all 19.

Comparisons between published models are mostly comparisons between datasets. Where two models were never evaluated on the same data with the same split, the literature does not support a statement that one is better.

Absence of a reported failure is not absence of failure. A field in which negative results are rarely published will show a distribution of reported performance that is shifted upwards, and a review cannot correct for this — it can only state it. Specifying these four limits, with reference to what the included studies did and did not do, is the most defensible contribution a review in this field can make.

Key points

Key points

A review should address one domain and one task: discovery, preclinical, clinical development, manufacturing and post-market work do not share an evidence standard, and a review spanning them is a map rather than a synthesis. The question should be framed as task, input data, baseline, validation design and success criterion, and the baseline treated as mandatory; a model with no comparator, or with an untuned one, makes no claim, and the proportion of such studies is itself a review-level finding. The search should extend beyond bibliographic databases, because much of this literature appears first in preprints and proceedings and some of it only in patents or not at all, and this gap should be acknowledged rather than the search described as comprehensive. Split strategy, leakage controls, baseline tuning effort, primary metric and prospective confirmation should be extracted, and a study-by-item appraisal matrix reported instead of a pooled accuracy estimate, since averaging metrics measured on different datasets estimates nothing. The review should state explicitly that retrospective benchmark performance is a hypothesis about prospective usefulness, and that development-timeline and approval outcomes occur over horizons on which attribution to a single method is extremely difficult; the 1981 to 2014 window of one widely cited natural products analysis spans 34 years.

Application to your study

Application to your study

  1. One domain and one task should be named in the protocol. The relationship between input and output should be stated in one sentence before any search terms are written.
  2. The baseline should be both an extraction field and a reporting item. The record should state what the baseline was and whether it was tuned to the same extent, or that there was none.
  3. Every study should be classified by validation design. Random split, scaffold or temporal split, held-out source, data generated after the model, prospective experimental test.
  4. The search should be built from controlled vocabularies rather than general catch-all phrases. It should combine controlled terms, architecture families and older modelling traditions, with the task expressed in task-specific terms.
  5. The rule for preprints should be decided in advance. The protocol should state whether preprints are eligible and how a later journal version with a changed result will be handled.
  6. Leakage controls should be extracted explicitly. These include the split method, duplicate handling, whether evaluation data informed any selection decision, and whether the paper reports this.
  7. The appraisal matrix, rather than a summary score, should be published. Presenting studies against items allows readers to see which items the field omits.
  8. The review should conclude with a specific research agenda. The agenda should specify the task, the data, the baseline and the prospective test required.

Frequently asked questions

Almost never in this field, and the reason is not heterogeneity in the statistical sense but incommensurability. The reported estimates derive from different datasets of differing difficulty, with different endpoints, different split strategies and metrics chosen by the authors; averaging them produces a number that estimates no defined quantity. The narrow exception is a set of studies that evaluated different models on the same public benchmark with the same split, where a structured comparison is possible — but the unit of evidence is then the benchmark rather than the study, and any community-wide optimism from repeated tuning on that benchmark applies to all of them equally. More appropriate outputs are a task-by-task structured synthesis, a baseline-relative summary, a count of studies with prospective confirmation, and an evidence gap map of tasks against validation levels.

Three properties. The baseline exists: a model reported without any comparator makes no claim, since performance figures cannot be interpreted in isolation. It is the appropriate comparator: an established physics-based or statistical method, a simpler model of the same family, the human expert workflow the model would replace, or a documented historical rate — whichever the study implicitly claims to improve upon. It received comparable tuning effort, because a lightly configured baseline set against a heavily optimised proposed model produces an improvement that reflects effort rather than method. The review should extract all three as separate fields and report the counts: how many included studies had no baseline, how many had an untuned one, and how many compared against an established alternative.

Leakage is any route by which information from the evaluation data influenced the model or its configuration. The common forms are preprocessing, normalisation or feature selection performed before the split; hyperparameter or threshold selection using the evaluation set; near-duplicate records appearing on both sides of a split, which in chemical datasets is standard and under-reported; and temporal leakage, where a model predicting an earlier period had access to later information. Detecting it from a paper is partly impossible, which is why the extraction field has three states: controlled, not controlled, or cannot tell. The third state should be recorded accurately and its frequency reported — a literature in which most papers do not permit a leakage assessment constitutes a finding in itself.

There are legitimate arguments for both positions, and the essential requirement is to decide in the protocol and report the decision. Inclusion acknowledges that a substantial share of methods work appears first on preprint servers, and sometimes only there, so exclusion leaves a systematic gap in a rapidly developing field. Exclusion restricts the evidence base to peer-reviewed work, which is defensible provided the review states that recent methods work is therefore underrepresented. If preprints are included, the handling rule should be pre-specified: where a preprint and a later journal version both exist, the journal version should be used and any change in the reported result between the two recorded, since that change is itself informative about how these results are modified under peer review.

Because the outcome occurs on a timescale over which attribution to a single method is extremely difficult, and because the published methods literature does not measure that outcome at all. Analyses of where new medicines come from are built on multi-decade windows for this reason — one widely cited classification of natural products as sources of new drugs covers approvals from 1981 to 2014, a 34-year span. A compound influenced by a model at the discovery stage passes through many subsequent decisions before any approval, so even a well-documented case is a narrative rather than an estimate of effect. A review can legitimately report which studies tested predictions prospectively, how many predictions were tested, who tested them, and with what result — and that the longer-horizon outcome has largely not yet had time to be observed.

This depends on the type of model. Where the model is a clinical prediction model — predicting an outcome in patients — a published prediction-model risk-of-bias instrument applies, and it should be named with its version and date, with any modified domain reported. Where the model is not a clinical prediction model, such as a virtual screening pipeline or a manufacturing process control model, no single accepted instrument exists, and devising a scored instrument is less appropriate than using a transparent item list. Leakage, baseline adequacy, split strategy, metric selection, benchmark provenance, prospective confirmation and artefact availability should be assessed, and the results presented as a matrix of studies against items so that a reader can see which specific item failed rather than a composite number.

By stating it as a specific rather than a generic limitation. Commercially developed methods appear in patents, in regulatory filings, or not at all, and detailed methodology can rarely be extracted from any of these. A review of the published literature therefore describes the published evidence base, not the field, and should state this explicitly. Two further asymmetries should be noted: negative results are rarely published in methods work, so the distribution of reported performance is shifted upwards in a way that no review can correct for; and the published record favours approaches that are publishable, which are not necessarily approaches that work. A review that acknowledges these three gaps makes a more defensible claim than one that describes its search as comprehensive.

A transparent map of what has been validated and by what means, together with a specific research agenda. Specifically: a task-by-task table giving each study’s data, split strategy, baseline, primary metric and prospective status; a matrix of studies against appraisal items showing which methodological safeguards the literature does and does not report; counts of studies with no baseline, an untuned baseline and prospective confirmation; and a gap map of tasks against validation levels that shows where the field has conducted only retrospective work. This should be followed by a defined agenda: which task, evaluated on which data, against which baseline and with what prospective test, would advance the question. Such a list is reusable by subsequent researchers, which a pooled accuracy figure is not.

References

  1. Chapter 4: Searching for and selecting studies. Cochrane Handbook for Systematic Reviews of Interventions, version 6.5. Cochrane. 2024;6.5. read 9 October 2026. cochrane.org
  2. Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions, version 6.5. Cochrane. 2024;6.5. read 9 October 2026. cochrane.org
  3. Jonnalagadda SR, Goyal P, Huffman MD. Automating data extraction in systematic reviews: a systematic review. Systematic Reviews. 2015;4:78. read 9 October 2026. cochrane.org
  4. Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Systematic Reviews. 2015;4(1):1. doi:10.1186/2046-4053-4-1 read 9 October 2026. doi.org/10.1186/2046-4053-4-1
  5. Tsafnat G, Glasziou P, Choong MK, Dunn A, Galgani F, Coiera E. Systematic review automation technologies. Systematic Reviews. 2014;3(1):74. doi:10.1186/2046-4053-3-74 read 9 October 2026. doi.org/10.1186/2046-4053-3-74
  6. Bolanos F, Salatino A, Osborne F, Motta E. Artificial intelligence for literature reviews: opportunities and challenges. Artificial Intelligence Review. 2024;57(10):259. doi:10.1007/s10462-024-10902-3 read 9 October 2026. doi.org/10.1007/s10462-024-10902-3
  7. Qureshi R, Irfan M, Gondal TM, Khan S, Wu J, Hadi MU, et al. AI in drug discovery and its clinical relevance. Heliyon. 2023;9(7):e17575. doi:10.1016/j.heliyon.2023.e17575 read 9 October 2026. doi.org/10.1016/j.heliyon.2023.e17575
  8. Zhang Y, Luo M, Wu P, Wu S, Lee TY, Bai C. Application of Computational Biology and Artificial Intelligence in Drug Design. International Journal of Molecular Sciences. 2022;23(21):13568. doi:10.3390/ijms232113568 read 9 October 2026. doi.org/10.3390/ijms232113568
  9. Karim MR, Beyan O, Zappa A, Costa IG, Rebholz-Schuhmann D, Cochez M, Decker S. Deep learning-based clustering approaches for bioinformatics. Briefings in Bioinformatics. 2021;22(1):393-415. doi:10.1093/bib/bbz170 read 9 October 2026. doi.org/10.1093/bib/bbz170
  10. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71. doi:10.1136/bmj.n71 read 9 October 2026. doi.org/10.1136/bmj.n71
  11. Gauthier J, Vincent AT, Charette SJ, Derome N. A brief history of bioinformatics. Briefings in Bioinformatics. 2019;20(6):1981-1996. doi:10.1093/bib/bby063 read 9 October 2026. doi.org/10.1093/bib/bby063
  12. Bramer WM, de Jonge GB, Rethlefsen ML, Mast F, Kleijnen J. A systematic approach to searching: an efficient and complete method to develop literature searches. Journal of the Medical Library Association. 2018;106(4):531-541. doi:10.5195/jmla.2018.283 read 9 October 2026. doi.org/10.5195/jmla.2018.283
  13. Tricco AC, Lillie E, Zarin W, O'Brien KK, Colquhoun H, Levac D, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation. Annals of Internal Medicine. 2018;169(7):467-473. doi:10.7326/M18-0850 read 9 October 2026. doi.org/10.7326/M18-0850
  14. Yang Y, Gao J, Wang J, Heffernan R, Hanson J, Paliwal K, Zhou Y. Sixty-five years of the long march in protein secondary structure prediction: the final stretch? Briefings in Bioinformatics. 2018;19(3):482-494. doi:10.1093/bib/bbw129 read 9 October 2026. doi.org/10.1093/bib/bbw129
  15. Munn Z, Peters MD, Stern C, Tufanaru C, McArthur A, Aromataris E. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Medical Research Methodology. 2018;18(1):143. doi:10.1186/s12874-018-0611-x read 9 October 2026. doi.org/10.1186/s12874-018-0611-x
  16. Baudard M, Yavchitz A, Ravaud P, Perrodeau E, Boutron I. Impact of searching clinical trial registries in systematic reviews of pharmaceutical treatments: methodological systematic review and reanalysis of meta-analyses. BMJ. 2017;356:j448. read 9 October 2026. cochrane.org
  17. Angermueller C, Parnamaa T, Parts L, Stegle O. Deep learning for computational biology. Molecular Systems Biology. 2016;12(7):878. doi:10.15252/msb.20156651 read 9 October 2026. doi.org/10.15252/msb.20156651
  18. DiMasi JA, Grabowski HG, Hansen RW. Innovation in the pharmaceutical industry: New estimates of R&D costs. Journal of Health Economics. 2016;47:20-33. doi:10.1016/j.jhealeco.2016.01.012 read 9 October 2026. doi.org/10.1016/j.jhealeco.2016.01.012
  19. Newman DJ, Cragg GM. Natural Products as Sources of New Drugs from 1981 to 2014. Journal of Natural Products. 2016;79(3):629-661. doi:10.1021/acs.jnatprod.5b01055 read 9 October 2026. doi.org/10.1021/acs.jnatprod.5b01055
  20. Libbrecht MW, Noble WS. Machine learning applications in genetics and genomics. Nature Reviews Genetics. 2015;16(6):321-332. doi:10.1038/nrg3920 read 9 October 2026. doi.org/10.1038/nrg3920
  21. Welch L, Lewitter F, Schwartz R, Brooksbank C, Radivojac P, Gaeta B, Schneider MV. Bioinformatics Curriculum Guidelines: Toward a Definition of Core Competencies. PLOS Computational Biology. 2014;10(3):e1003496. doi:10.1371/journal.pcbi.1003496 read 9 October 2026. doi.org/10.1371/journal.pcbi.1003496
  22. Gross A, Schirm S, Scholz M. Ycasd - a tool for capturing and scaling data from graphical representations. BMC Bioinformatics. 2014;15(1):219. doi:10.1186/1471-2105-15-219 read 9 October 2026. doi.org/10.1186/1471-2105-15-219
  23. Turner EH. How to access and process FDA drug approval packages for use in research. BMJ. 2013;347. read 9 October 2026. cochrane.org
  24. Swinney DC, Anthony J. How were new medicines discovered? Nature Reviews Drug Discovery. 2011;10(7):507-519. doi:10.1038/nrd3480 read 9 October 2026. doi.org/10.1038/nrd3480
  25. Zarin DA, Tse T, Williams RJ, Califf RM, Ide NC. The ClinicalTrials.gov results database: update and key issues. New England Journal of Medicine. 2011;364:852-860. read 9 October 2026. cochrane.org
  26. Paul SM, Mytelka DS, Dunwiddie CT, Persinger CC, Munos BH, Lindborg SR, Schacht AL. How to improve R&D productivity: the pharmaceutical industry's grand challenge. Nature Reviews Drug Discovery. 2010;9(3):203-214. doi:10.1038/nrd3078 read 9 October 2026. doi.org/10.1038/nrd3078
  27. Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information and Libraries Journal. 2009;26(2):91-108. doi:10.1111/j.1471-1842.2009.00848.x read 9 October 2026. doi.org/10.1111/j.1471-1842.2009.00848.x
  28. Benson DA, Karsch-Mizrachi I, Lipman DJ, Ostell J, Wheeler DL. GenBank. Nucleic Acids Research. 2008;36(Database issue):D25-D30. doi:10.1093/nar/gkm929 read 9 October 2026. doi.org/10.1093/nar/gkm929
  29. Materi W, Wishart DS. Computational systems biology in drug discovery and development: methods and applications. Drug Discovery Today. 2007;12(7-8):295-303. doi:10.1016/j.drudis.2007.02.013 read 9 October 2026. doi.org/10.1016/j.drudis.2007.02.013
  30. Larranaga P, Calvo B, Santana R, Bielza C, Galdiano J, Inza I, et al. Machine learning in bioinformatics. Briefings in Bioinformatics. 2006;7(1):86-112. doi:10.1093/bib/bbk007 read 9 October 2026. doi.org/10.1093/bib/bbk007
  31. Ouzounis CA, Valencia A. Early bioinformatics: the birth of a discipline - a personal view. Bioinformatics. 2003;19(17):2176-2190. doi:10.1093/bioinformatics/btg309 read 9 October 2026. doi.org/10.1093/bioinformatics/btg309
  32. Richardson WS, Wilson MC, Nishikawa J, Hayward RS. The well-built clinical question: a key to evidence-based decisions. ACP Journal Club. 1995;123(3):A12-A13. doi:10.7326/ACPJC-1995-123-3-A12 read 9 October 2026. doi.org/10.7326/ACPJC-1995-123-3-A12

Every source above is a peer-reviewed article, a publisher’s record of one, or a standard issued by the body that owns it. Each carries the date it was last read. Where a standard moves, this article is revised rather than left up with a note.

How to cite this article

Pick your referencing style and copy it. Your university or target journal may vary the punctuation — check its guide if it has one.

Pubrica Academy. (2026, October 9). Systematic Reviews of Machine Learning in the Pharmaceutical Industry: Scope, Appraisal and Synthesis. Pubrica. https://pubrica.com/academy/latest-topics/systematic-review-of-machine-learning-in-the-pharmaceutical-industry/

This article is written to be learned from. Cite it if you are referring to what it says; if you are using the evidence behind it, cite the sources listed above instead.

Protocol and extraction stub for a machine learning review

The first block prevents the most common failure, namely a review spanning five domains. The leakage and baseline fields are those that distinguish a validated model from a well-fitted one.

MACHINE LEARNING IN PHARMA — REVIEW STUB

=== BLOCK 1: SCOPE (one domain, one task) ===
  Domain (tick ONE):
    [ ] discovery / target work
    [ ] preclinical / translational
    [ ] clinical development
    [ ] manufacturing / quality
    [ ] post-market / pharmacovigilance
  TASK as input -> output: _____________________________
  Success criterion being reviewed (tick ONE):
    [ ] retrospective predictive performance
    [ ] external or temporal validation
    [ ] prospective experimental confirmation
    [ ] candidates entering development
    [ ] approvals
  Design label: [ ] systematic review [ ] scoping review
                [ ] evidence gap map

=== BLOCK 2: SEARCH ===
  Databases, platforms, dates: _________________________
  Method concept — controlled vocabulary terms: __________
  Method concept — architecture and tradition terms:
    deep learning / neural network / random forest / gradient
    boosting / support vector / graph neural network / transformer /
    autoencoder / QSAR / cheminformatics / data-driven modelling
    plus: ______________________________________________
  TASK concept terms (NOT "drug discovery"): ____________
  Date limit:  [ ] none (recommended)  [ ] from ____ because ______
  Preprint servers searched: ___________ date ______
  Conference proceedings searched: __________ date ______
  PREPRINT RULE: where preprint and journal version both exist ->
    [ ] use journal version and record any change in the result
  Patents searched?  [ ] yes  [ ] NO — declared as a limitation [ ]
  Code repositories: [ ] availability checks only, not identification

=== BLOCK 3: EXTRACTION, PER STUDY ===
TASK AND DATA
  Input representation: ________________________________
  Output definition: __________________________________
  Dataset(s) and whether public: ______________________
  Sample size in the unit that matters: ____ (compounds / assays /
    patients / sites — state which)
  Label provenance and known label noise: _____________
  Class balance: ______________________________________
SPLIT AND LEAKAGE
  Split type: [ ] random [ ] scaffold [ ] temporal [ ] by source
              [ ] data generated after the model was built
  Near-duplicate handling across the split: [ ] controlled
              [ ] not controlled  [ ] cannot tell
  Did evaluation data influence preprocessing, feature selection,
    hyperparameters or thresholds?  [ ] no [ ] yes [ ] cannot tell
MODEL AND TRAINING
  Model family: _______________________________________
  Hyperparameter selection method: ____________________
  BASELINE: [ ] none  [ ] named: ______________________
    Baseline tuned with comparable effort?  [ ] yes [ ] no [ ] cannot tell
EVALUATION
  Metrics reported: ___________________________________
  PRIMARY metric, and was it declared in advance? _______
  Uncertainty reported?  [ ] yes, how: ______  [ ] no
  Performance relative to baseline: ___________________
  Subgroup / series / site results?  [ ] yes  [ ] no
  Benchmark widely used for model development?  [ ] yes [ ] no
PROSPECTIVE COMPONENT
  Any prediction tested afterwards?  [ ] no  [ ] yes
    How many predictions tested: ____  By whom: ______
    Independent of the prediction team?  [ ] yes  [ ] no
    Outcome: __________________________________________
  Chronology recorded (model built / predicted / confirmed): ______
AVAILABILITY
  Code [ ]  Trained model [ ]  Training data [ ]  Terms: ______

=== BLOCK 4: REVIEW-LEVEL OUTPUTS (commit to these) ===
  Study x item appraisal MATRIX (not a score): [ ] planned
  Counts: studies with no baseline ____
          studies with an untuned baseline ____
          studies with prospective confirmation ____
          studies where leakage could not be assessed ____
  Task x validation-level GAP MAP: [ ] planned
  NAMED RESEARCH AGENDA: which task, on which data, against which
    baseline, with what prospective test:
    ___________________________________________________

=== BLOCK 5: LIMITS TO STATE EXPLICITLY ===
  [ ] Retrospective benchmark performance is a hypothesis about
      prospective usefulness, tested for few of the included models
  [ ] No claim about development timelines or success rates
  [ ] Cross-study performance comparisons are dataset comparisons
  [ ] Unpublished and commercially developed methods are
      structurally underrepresented
  [ ] Negative results are rarely published, so reported performance
      is shifted upwards in a way the review cannot correct

Pubrica AcademyWritten and maintained by Pubrica’s subject matter experts. Articles are revised when the underlying standards change, and each article states the date on which it was last reviewed.

How Pubrica can help

Pubrica scopes these reviews to one domain and one task before the search, develops a strategy covering the vocabularies under which the same work is published, and extracts the split, leakage and baseline fields that distinguish a validated model from a well-fitted one. The deliverable is a study-by-item appraisal matrix and an evidence gap map rather than a pooled accuracy figure, together with a clearly worded statement of what the published evidence base cannot support.

Find something else

Search the Pubrica Academy

Access is free and requires no account. Enter a topic, or open the full library to filter by category, sub-topic or resource type.

Pubrica narrows a review question to a task the literature can answer and identifies what it cannot address.

A scoping assessment is free and takes two business days. It provides a written assessment of what the work requires and which specialist would undertake it.

Get a free quote

Regulations

Check your project against this rule

  1. 1Your work
  2. 2Your details
  3. 3Send
What do you need?

Material you send is held under signed confidentiality, with access limited to the assigned team and encrypted transfer. It is not used to train anything and is not passed to third-party tools. Ask for the agreement before you send anything you would rather cover first.