Contents
On this page
- Resistance of computational fields to systematic review
- Selection of the review instrument
- Long research horizons and the history of the field
- Location of the primary record
- Definition of the unit of analysis
- Data extraction from methods papers
- Appraisal of performance claims
- Synthesis without statistical pooling
- Reporting, date-stamping and the limits of the review’s claims
- Key points
- Application to your study
- Frequently asked questions
Academy · Research Data Analytics & AI · reviewing computational biology
Systematically Reviewing Developments in Bioinformatics and Computational Biology: Scope, Instruments and Reporting
“Current developments in bioinformatics” is a subject for a textbook chapter rather than a systematic review. A review can answer a bounded question about a defined task — which methods exist for it, how they were evaluated, on what data and against what baseline — and it must be date-stamped, because its subject continues to change. The choice of review instrument largely determines whether the result is useful or merely extensive.
Computational biology combines a long research history with a short publication cycle. Protein secondary structure prediction has been pursued for sixty-five years and has been described as being in its final stretch, while the same field publishes methods faster than any review can track them. These two characteristics place opposing demands on a reviewer, and many reviews resolve the tension by widening the question until little in the result can be verified. The alternative is to narrow the task, declare the date and state what the evidence does and does not establish.
Learning objectives
- Distinguishing between a systematic review, a scoping review, an evidence map, a state-of-the-art review and a living review for a computational question
- Bounding a question so that the included set is finite and the synthesis is meaningful
- Searching a literature distributed across journals, preprints, repositories and benchmark competitions
- Defining the unit of analysis when the included items are methods rather than studies
- Appraising self-reported performance, benchmark reuse and the absence of independent evaluation
- Reporting a review whose subject will have changed by the time it is read
Common misconceptions
- “A systematic review is the most rigorous option, so it is always the right one.” For a mapping question it is the wrong instrument, and a systematic review that cannot specify a finite included set is less rigorous than a well-conducted scoping review.
- “The field moves too fast to review, so completeness is impossible.” Completeness at a stated date for a bounded task is achievable. Completeness for an unbounded subject is not, and the remedy is a defined boundary rather than an apology.
- “Everything important is in the last five years.” Some problems in this field have been studied for decades — secondary structure prediction for sixty-five years — and a date-limited search renders any claim about progress circular.
- “Reported benchmark performance is comparable across papers.” Different versions of a benchmark, different splits, different preprocessing and different evaluation scripts make the numbers non-comparable, and the paper usually does not report sufficient detail to establish this.
- “Peer-reviewed journal articles are the evidence base.” In parts of this field the primary record is a preprint, a repository release or a community assessment result, and a journal-only search captures only a delayed and partial representation of the literature.
Resistance of computational fields to systematic review
Four structural features each violate an assumption on which systematic review methods rely.
The unit is unstable. In a clinical review the unit is a study: a discrete event with a population and a result. Here the unit might be a method, an implementation, a benchmark result, a release version or a paper, and the same method appears across several of these. If the protocol does not fix the unit, the included set cannot be counted and the flow diagram becomes meaningless.
The record is distributed. Journals, preprint servers, software repositories and community assessment exercises each hold part of the primary evidence, and the authoritative version of a method may be a repository release rather than a publication.
Evaluation is largely self-reported. A new method is usually introduced with performance figures produced by its own authors on datasets they selected. This is standard practice, and it means that the evidence base is overwhelmingly non-independent, which a review must state rather than aggregate.
The subject changes during the review. A search run in March is out of date by the time of publication; this is true of all reviews but particularly consequential here. The remedy is a date stamp and, where appropriate, a living review format with a declared update cycle, rather than a sentence about the pace of the field.
Selection of the review instrument
Five instruments are available, and selection of the wrong one accounts for most unsatisfactory reviews in this area 24.
A systematic review suits a bounded question with a specifiable included set and a comparable outcome: which methods have been evaluated for one named task on one named benchmark, and how. It requires eligibility criteria capable of returning a finite set 2.
A scoping review suits a mapping question: what has been attempted in an area, by whom, on what data, with what gaps 12,24. It makes no claim about effects, and this should not be presented as a weakness. See scoping review.
An evidence map suits the case where the useful output is a matrix — tasks against validation levels, or methods against data types — rather than a narrative. See evidence gap mapping.
A state-of-the-art review suits a question about where a field currently stands, written by people with the expertise to judge, and does not claim exhaustive retrieval. See state-of-the-art review.
A living review suits a question worth keeping current, with a stated search cadence and a version history.
The instrument should be chosen before the search and named in the title, and a scoping review should not be described as a systematic review because the latter appears more rigorous. A reader who knows which instrument was used can interpret the result; a reader who has been misinformed cannot.
Which review instrument fits which question
| Instrument | Question it answers | Requires | Must not claim |
|---|---|---|---|
| Systematic review | Which methods have been evaluated for one named task, on what data, how? | Eligibility criteria that return a finite, countable included set | Coverage of an unbounded subject |
| Scoping review | What has been attempted in this area, by whom, with what gaps? | A systematic search and a transparent charting framework | An effect estimate or a performance comparison |
| Evidence map | Where in the task-by-evaluation space has work been done? | A pre-specified matrix structure and consistent coding | A narrative judgement about which method is best |
| State-of-the-art review | Where does this field currently stand, in the judgement of people who work in it? | Subject expertise and an explicit account of how material was selected | Exhaustive retrieval |
| Living review | What is currently true, kept current? | A declared search cadence, a version history and a maintenance commitment | Permanence — each version is dated and superseded |
Compiled from the published typology of review types and their associated methodologies; the choice should be stated in the title and in the protocol.
Long research horizons and the history of the field
A review of “current developments” carries an implicit claim about change over time, and that claim requires a baseline from before the search window.
Two points are relevant. First, some problems in this field are very old: protein secondary structure prediction has been pursued for sixty-five years and is described as being in its final stretch 13. A review that searches the last five years and reports rapid recent progress on such a problem has measured the last five years of a sixty-five-year effort, and cannot say what share of the progress is recent without a longer view.
Second, the field has a historical literature of its own, which should be consulted before a developments review is written. A history of bioinformatics has been published in Briefings in Bioinformatics, in volume 20, issue 6, at pages 1981 to 1996 10. Consulting the field’s own account of its development typically benefits a review in two ways: it supplies the baseline against which “current” acquires meaning, and it reveals which apparently new ideas are re-entries 10,32.
The practical consequence for the protocol is that a date limit should be avoided unless it can be justified, and that any claim about acceleration or novelty should be treated as requiring evidence rather than as framing. Where a date limit is unavoidable for feasibility, the protocol should state that it is a feasibility restriction and that claims about change over time are therefore outside the review’s scope.
The time horizon of a long-standing problem in the field, and the location of the field’s own published history. Both are given to indicate the baseline a developments review needs, not as performance figures.
Location of the primary record
Five source types are relevant, and a review that searches only the first describes a delayed subset of the literature 1.
Bibliographic databases for the journal literature, searched database by database with platform and date recorded — see sources of review data. Terminology is the main difficulty: methods are named, renamed and referred to by architecture, so the strategy needs method names, task terms and the older vocabulary the field used before the current one 11.
Preprint servers, where a substantial share of methods work appears first and occasionally only. Eligibility should be decided in the protocol, together with the handling rule for a later journal version.
Conference proceedings, which in computational fields carry primary rather than preliminary results, unlike in clinical research. Treating a proceedings paper as grey literature is a category error here.
Software and model repositories, which hold release histories, versions and in many cases the only authoritative description of what a method currently does. Access dates should be recorded.
Community assessment exercises and shared benchmarks, which are the field’s nearest approach to independent evaluation. Where one exists for the task under review, its results are more informative than any number of self-reported comparisons, and the review should foreground them.
Citation tracking on the key method papers is usually the highest-yield single identification method. It should be reported with the index and date, not as an afterthought.
Definition of the unit of analysis
This is the decision that makes a computational review countable, and it must be made explicitly.
If the unit is the method, a method described across a preprint, a journal paper, two update papers and a repository release is one included item with several records attached. The flow diagram then counts methods, and a linking rule is needed for deciding when two papers describe the same method and when a modification constitutes a new one.
If the unit is the evaluation, each reported performance assessment is an included item, and the same method appears several times — which is the right choice when the question is about how methods have been evaluated rather than about what exists.
If the unit is the paper, the review is about the literature rather than about the methods, which is a legitimate question — reporting practice, data availability, benchmark usage — and should be stated as such.
The eligibility criteria should then be written around that unit: the task, stated as input and output; the data type; whether an implementation must be available; whether evaluation against a named benchmark is required; and whether methods available only as a repository release, with no publication, are eligible. That last criterion matters more in this field than in any other, and if it is left unstated it will be decided inconsistently during screening. The Pubrica page on screening describes how to write criteria that remain workable when applied to actual records.
The objective and eligibility statement in the protocol
Cannot produce a countable included set
This systematic review aims to summarise current developments in bioinformatics and computational biology, covering recent advances in machine learning approaches, tools and databases published in the last five years.
“Current developments in bioinformatics” has no boundary. No eligibility criterion can return a finite set from it, so the included set will be whatever the search happened to retrieve.
No unit of analysis. Methods, papers, tools and databases are all mentioned, so the counts in the flow diagram will be uninterpretable.
“Tools and databases” alongside methods. Three different kinds of object with three different evidence bases, pooled into one review.
A five-year window with no justification. It excludes the lineage the current work builds on, and it makes the implicit claim about recent advances unfalsifiable.
Journal publication assumed. Preprints, proceedings and repository-only releases carry primary evidence in this field and are silently excluded.
No date stamp and no version record. Nothing in the result will be attributable to a defined state of the field.
Labelled a systematic review. The question is a mapping question, so the instrument is inappropriate before any work begins.
Bounded, countable and date-stamped
This review asks which computational methods have been proposed for one named prediction task, defined by a stated input and a stated output, and how each was evaluated. The unit of analysis is the method: where a preprint, a journal article, subsequent update papers and a repository release describe the same method, these are treated as one included item with all records attached, and a modification is treated as a new method only where the authors assign it a new name or version and report separate performance. Eligible items propose or substantially modify a method for that task, report an evaluation on data not used for development, and have an available implementation or sufficient methodological detail for reimplementation; methods released only in a public repository, with no accompanying publication, are eligible and are identified by searching the named repositories on a stated date. No publication-date limit is applied, because the task has been worked on for decades and a recent-only window would make any claim about progress circular. Searches cover the named bibliographic databases with platform and date recorded, the named preprint servers, the proceedings of the named conferences, the named repositories, and the results of the community assessment exercise for this task, which are reported separately from developer-run evaluations throughout. Findings describe the literature as it stood on the stated search date; method versions and benchmark versions examined are recorded for every included item, and both are expected to have changed by publication.
The task is defined as an input and an output, which is what makes an eligibility criterion able to return a finite set.
The unit of analysis is stated, with an explicit linking rule, so the counts are interpretable and the flow diagram is accurate.
Repository-only releases are explicitly eligible and explicitly searched for, which is the criterion most reviews in this field leave to chance.
No date limit is applied and the reason is given, removing the circularity from any statement about progress.
Community assessment results are kept separate from developer-run evaluations throughout, so independent evidence is not averaged into self-reports.
Method and benchmark versions are recorded per item and the search date is stated, so the review describes a defined state of the field.
The expiry is stated at the outset rather than conceded in the limitations, which is the appropriate treatment of a changing subject.
The first statement will always produce a review, but never one that can be verified. The second produces a smaller, countable result that a reader can use and a later team can update, and it makes the one genuinely independent evidence source in the field visible instead of merging it with the rest.
Data extraction from methods papers
The extraction form should be built around what would allow a reader to judge the claim rather than around what the paper advertises.
Identity and provenance. Method name and version, all records describing it, implementation availability and licence, whether it supersedes an earlier method by the same group.
Task and data. Input and output definitions, the datasets used for development and evaluation, their versions and release dates, and the size in the unit that matters.
Evaluation design. Which benchmark, which version of it, which split, which evaluation script or metric implementation, and whether the evaluation data postdates the method’s development. The last is the single most informative field: a method evaluated on data that existed before it was built carries a risk of contamination that one evaluated on later data does not.
Baselines. Which comparators, which versions of them, who ran them, and whether they were tuned comparably. Comparators run by the proposing authors with default settings are weak evidence and should be recorded as such.
Independent evaluation. Whether the method has been assessed by anyone other than its developers, in a community assessment or an independent benchmark, with the result. Most entries will be empty; the proportion is a finding.
Reporting and reproducibility. Code, trained parameters, data, environment specification, and whether the reported result has been reproduced. The general principles of extraction are described on the Pubrica page on what data to extract.
Extraction fields for a computational methods paper
| Block | Fields | Why it matters |
|---|---|---|
| Identity | Method name and version; every record describing it; implementation availability and licence; whether it supersedes an earlier method | A method described across a preprint, a paper, two updates and a release is one item, not five |
| Task and data | Input and output definitions; development and evaluation datasets with versions and release dates; size in the relevant unit | Dataset versions change; without them a reported number has no referent |
| Evaluation design | Benchmark and its version; split; evaluation script or metric implementation; whether the evaluation data postdates development | Evaluation on data that existed before the method was built carries a contamination risk that later data does not |
| Baselines | Comparators and their versions; who ran them; whether they were tuned comparably | Comparators run by the proposing authors at default settings are weak evidence |
| Independent evaluation | Any assessment by third parties or in a community assessment, with the result | Most entries will be empty, and the proportion that are is a review-level finding |
| Reproducibility | Code, trained parameters, data, environment specification, and any reproduction attempt | Determines whether any reported number can be checked by anyone |
Compiled as a working extraction structure for reviews whose included items are methods rather than studies.
Appraisal of performance claims
There is no accepted risk-of-bias instrument for computational methods papers, and a scored instrument invented for one review is worse than a transparent item list reported item by item.
Five items carry the weight. Independence of the evaluation: developer-run or third-party. Benchmark contamination: whether the evaluation data could have influenced development, directly or through the community’s accumulated tuning on a shared dataset 15,16. Baseline fairness: whether comparators were run competently and tuned comparably. Metric and split transparency: whether enough is reported to reproduce the number, including the evaluation script and the benchmark version. And reproducibility artefacts: code, parameters, data and environment.
Shared benchmarks warrant particular care. A dataset that an entire subfield has developed against for years accumulates optimism that no individual paper can detect or report: methods are selected, implicitly, for performance on it. A review that reports performance on such a benchmark without noting this transmits a systematically inflated set of numbers. Where a community assessment with held-out data exists, it provides the better evidence, and the review should state this explicitly rather than treating all performance figures as equivalent.
The response “cannot tell” should be recorded freely. In this literature it is frequently the accurate answer, and its frequency is informative about reporting practice.
Synthesis without statistical pooling
Numerical pooling is almost never appropriate: performance figures come from different benchmark versions, splits, preprocessing pipelines and metric implementations, and averaging them estimates no meaningful quantity 2. The review should state this rather than pool the figures with a heterogeneity caveat.
Four outputs work better. A task-structured table, one row per included item, giving data, evaluation design, baseline, independence and reproducibility artefacts — which lets a reader compare like with like where that is possible and see immediately where it is not. An evidence map of tasks against evaluation levels, which makes the field’s blind spots visible: typically a dense cluster of developer-evaluated methods and a sparse column of independently assessed ones. A methods-lineage account, describing how approaches derive from one another, which is what readers of a developments review generally require and which a flat catalogue cannot provide 13,30. And an explicit statement of what would settle the open questions: which held-out data, which community assessment, which independent reimplementation.
Where a community assessment exists for the task, the synthesis should be organised around it rather than around the papers. It is the one place in this literature where comparable numbers exist, and giving it prominence is both more informative and more accurate than a table assembled from self-reports.
Reporting, date-stamping and the limits of the review’s claims
The process should be reported in accordance with the PRISMA 2020 statement, with the extension matching the chosen instrument where one applies, and the instrument should be labelled in the title 9,12. The items that this type of review specifically requires should then be added.
All elements should be date-stamped: the search date per source, the repository access dates, the benchmark versions, and the version of each method examined. The unit of analysis should be stated in the methods, because a reader cannot interpret the counts without it. The included-item table and the evidence map should be published as supplementary files, since those are the parts others will reuse. The review’s expiry should also be stated plainly: findings describe the literature as it stood on a stated date, and both the methods and their benchmarks will have changed.
Three claims should be avoided. The first is that the review is comprehensive, if repositories, proceedings or preprints were not searched; the sources not searched should be named instead. The second is that one method outperforms another, where the comparison rests on separate developer-run evaluations on different data. The third is that the field is advancing rapidly, unless the review has a baseline long enough to support that comparison; given that some problems here have been worked on for sixty-five years, a five-year window is not such a baseline 13. Reporting checklist review identifies these before submission, and the Pubrica bioinformatics team works on the underlying analyses.
Key points
Key points
The question should be narrowed to a task defined by an input and an output, the unit of analysis — method, evaluation or paper — should be fixed, and a linking rule should be written for multiple records that describe a single method. The instrument should be chosen deliberately: a systematic review requires eligibility criteria that return a finite set, whereas a mapping question is better served by a scoping review or an evidence map, labelled as such. The search should extend beyond journals, because preprints, conference proceedings and repository releases carry primary evidence in this field, and repository-only methods should be made explicitly eligible or explicitly excluded rather than left to screening. Date limits should be avoided: protein secondary structure prediction has been pursued for sixty-five years, so a five-year window cannot support a claim about progress, and the field’s own published history — in Briefings in Bioinformatics, volume 20, issue 6, pages 1981 to 1996 — supplies the baseline that gives “current” a defined meaning. Developer-run and independent evaluations should be kept in separate columns, performance figures should not be pooled across benchmark versions, and the review and every method and benchmark version in it should be date-stamped.
Application to your study
Application to your study
- The task should be specified in terms of input and output. If this is not possible, the question is a mapping question and requires a different instrument.
- The unit of analysis and the linking rule should be declared. The rule should state when two papers describe one method and when a modification constitutes a new method.
- The instrument should be selected and named in the title. The options are a systematic review, scoping review, evidence map, state-of-the-art review or living review; a map should not be relabelled as a systematic review.
- The eligibility of repository-only methods should be decided explicitly. The named repositories should then be searched on a recorded date, because a method without an accompanying paper remains a method.
- The date limit should be removed or justified on grounds of feasibility. If it is retained, claims about change over time should be stated to be outside the scope of the review.
- Developer-run and independent evaluations should be reported separately. They should occupy two separate columns, and any community assessment result for the task should be given prominence.
- Benchmark and method versions should be recorded for each item. A performance figure without its benchmark version has no defined referent.
- The review should be date-stamped and its expiry stated. This includes search dates for each source, repository access dates and an explicit statement that the subject has since changed.
Frequently asked questions
A systematic review can address a bounded question within the field, at a stated date. A systematic review of an unbounded subject such as “current developments in bioinformatics” is not feasible, because no eligibility criterion can return a finite included set from it and the result is a catalogue of whatever the search retrieved. The question should be bounded to one task, defined by an input and an output, the unit of analysis fixed, and the search date-stamped. If the question is genuinely a mapping question — what has been attempted, where the gaps are — a scoping review or an evidence map should be used and labelled as such, which is more rigorous than a systematic review that cannot specify its own boundary.
The reviewer should choose among three options and record the choice in the protocol. If the unit is the method, a preprint, a journal paper, two update papers and a repository release describing the same method are one included item with records attached, and a rule is needed for when a modification counts as a new method, usually tied to the authors assigning a new name or version and reporting separate performance. If the unit is the evaluation, each reported performance assessment is an item and one method may appear several times, which suits a question about how methods have been evaluated. If the unit is the paper, the review is about the literature — reporting practice, data availability, benchmark usage — which is a legitimate question that should be stated as such. Without this decision, the counts in the flow diagram have no defined meaning.
In this field the default should be yes, with the decision stated explicitly either way. Substantial methods work appears first on preprint servers and occasionally only there, conference proceedings carry primary rather than preliminary results unlike in clinical research, and a method may exist authoritatively as a repository release with no accompanying publication at all. Excluding these is defensible only if the review states that it is describing the journal-published subset rather than the field. If they are included, the handling rules should be pre-specified: how a later journal version of a preprint is treated and whether a changed result between the two is recorded, and how repository-only methods are identified, which requires searching named repositories on a recorded date rather than relying on screening to identify them.
Because the review aggregates the problem and presents it as consistency. A shared benchmark that an entire subfield has developed against for years accumulates optimism no individual paper can detect: methods are implicitly selected for performance on that dataset, so the published distribution of results on it is shifted upwards. When a review assembles those figures into a table, the apparent agreement between methods partly reflects their common target rather than genuine convergence. The appropriate handling is to record the benchmark and its version for every item, note which benchmarks are long-established community targets, keep developer-run evaluations separate from independent ones, and foreground any community assessment with genuinely held-out data, which is the field’s closest approach to independent evidence.
Pooling is essentially never appropriate, and comparison is appropriate only under conditions that should be checked and reported. The figures come from different benchmark versions, different splits, different preprocessing and sometimes different implementations of the same metric, and papers frequently do not report enough to determine which. Averaging them estimates no quantity. Direct comparison is defensible where two methods were evaluated on the same version of the same benchmark with the same split and the same evaluation script, and where the baselines were run competently by someone with an interest in their performing well — which usually means a community assessment rather than a methods paper. Otherwise, the review should present a task-structured table that keeps each number attached to its benchmark version, and state that cross-study comparison is not supported.
Only as a declared feasibility restriction, and not if the review makes any claim about progress or novelty. Some problems in this field have been worked on for decades — protein secondary structure prediction has been pursued for sixty-five years and is described as being in its final stretch — so a five-year window examines a thin slice of a long effort and cannot establish what share of the progress is recent. A date limit also excludes the lineage that current work builds on, which matters because apparently new ideas in computational biology are often re-entries. Consulting the field’s own published history is a simple corrective: one such history appears in Briefings in Bioinformatics, volume 20, issue 6, pages 1981 to 1996.
A transparent item list, reported item by item, is preferable to a scored instrument devised for the occasion. Five items carry the weight: independence of the evaluation, meaning developer-run or third-party; benchmark contamination, including community-wide tuning on a shared dataset; baseline fairness, meaning whether comparators were run competently and tuned comparably; metric and split transparency, meaning whether enough is reported to reproduce the number including the benchmark version and evaluation script; and reproducibility artefacts, meaning code, trained parameters, data and environment. Reviewers should record “cannot tell” freely, because in this literature it is frequently the accurate answer, and report how often it was used, since that frequency is a substantive finding about reporting practice.
By date-stamping rather than apologising, and by considering a format designed for the purpose. The review should state the search date for each source, the repository access dates, the benchmark versions and the method version examined for every included item, and then state plainly that findings describe the literature as it stood on that date and that methods and benchmarks are expected to have changed. The included-item table and the evidence map should be published as supplementary files, since those are the parts a later team will update rather than rewrite. Where the question is worth keeping current, a living review with a declared search cadence and a version history is the appropriate instrument, and a state-of-the-art review that does not claim exhaustive retrieval is a legitimate alternative to a one-off systematic review of a changing subject.
References
- Chapter 4: Searching for and selecting studies. Cochrane Handbook for Systematic Reviews of Interventions, version 6.5. Cochrane. 2024;6.5. read 9 October 2026. cochrane.org
- Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions, version 6.5. Cochrane. 2024;6.5. read 9 October 2026. cochrane.org
- Jonnalagadda SR, Goyal P, Huffman MD. Automating data extraction in systematic reviews: a systematic review. Systematic Reviews. 2015;4:78. read 9 October 2026. cochrane.org
- Tsafnat G, Glasziou P, Choong MK, Dunn A, Galgani F, Coiera E. Systematic review automation technologies. Systematic Reviews. 2014;3(1):74. doi:10.1186/2046-4053-3-74 read 9 October 2026. doi.org/10.1186/2046-4053-3-74
- Bolanos F, Salatino A, Osborne F, Motta E. Artificial intelligence for literature reviews: opportunities and challenges. Artificial Intelligence Review. 2024;57(10):259. doi:10.1007/s10462-024-10902-3 read 9 October 2026. doi.org/10.1007/s10462-024-10902-3
- Qureshi R, Irfan M, Gondal TM, Khan S, Wu J, Hadi MU, et al. AI in drug discovery and its clinical relevance. Heliyon. 2023;9(7):e17575. doi:10.1016/j.heliyon.2023.e17575 read 9 October 2026. doi.org/10.1016/j.heliyon.2023.e17575
- Zhang Y, Luo M, Wu P, Wu S, Lee TY, Bai C. Application of Computational Biology and Artificial Intelligence in Drug Design. International Journal of Molecular Sciences. 2022;23(21):13568. doi:10.3390/ijms232113568 read 9 October 2026. doi.org/10.3390/ijms232113568
- Karim MR, Beyan O, Zappa A, Costa IG, Rebholz-Schuhmann D, Cochez M, Decker S. Deep learning-based clustering approaches for bioinformatics. Briefings in Bioinformatics. 2021;22(1):393-415. doi:10.1093/bib/bbz170 read 9 October 2026. doi.org/10.1093/bib/bbz170
- Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71. doi:10.1136/bmj.n71 read 9 October 2026. doi.org/10.1136/bmj.n71
- Gauthier J, Vincent AT, Charette SJ, Derome N. A brief history of bioinformatics. Briefings in Bioinformatics. 2019;20(6):1981-1996. doi:10.1093/bib/bby063 read 9 October 2026. doi.org/10.1093/bib/bby063
- Bramer WM, de Jonge GB, Rethlefsen ML, Mast F, Kleijnen J. A systematic approach to searching: an efficient and complete method to develop literature searches. Journal of the Medical Library Association. 2018;106(4):531-541. doi:10.5195/jmla.2018.283 read 9 October 2026. doi.org/10.5195/jmla.2018.283
- Tricco AC, Lillie E, Zarin W, O'Brien KK, Colquhoun H, Levac D, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation. Annals of Internal Medicine. 2018;169(7):467-473. doi:10.7326/M18-0850 read 9 October 2026. doi.org/10.7326/M18-0850
- Yang Y, Gao J, Wang J, Heffernan R, Hanson J, Paliwal K, Zhou Y. Sixty-five years of the long march in protein secondary structure prediction: the final stretch? Briefings in Bioinformatics. 2018;19(3):482-494. doi:10.1093/bib/bbw129 read 9 October 2026. doi.org/10.1093/bib/bbw129
- Bramer WM, Giustini D, de Jonge GB, Holland L, Bekhuis T. De-duplication of database search results for systematic reviews in EndNote. Journal of the Medical Library Association. 2016;104(3):240-243. doi:10.3163/1536-5050.104.3.014 read 9 October 2026. doi.org/10.3163/1536-5050.104.3.014
- Angermueller C, Parnamaa T, Parts L, Stegle O. Deep learning for computational biology. Molecular Systems Biology. 2016;12(7):878. doi:10.15252/msb.20156651 read 9 October 2026. doi.org/10.15252/msb.20156651
- Libbrecht MW, Noble WS. Machine learning applications in genetics and genomics. Nature Reviews Genetics. 2015;16(6):321-332. doi:10.1038/nrg3920 read 9 October 2026. doi.org/10.1038/nrg3920
- Welch L, Lewitter F, Schwartz R, Brooksbank C, Radivojac P, Gaeta B, Schneider MV. Bioinformatics Curriculum Guidelines: Toward a Definition of Core Competencies. PLOS Computational Biology. 2014;10(3):e1003496. doi:10.1371/journal.pcbi.1003496 read 9 October 2026. doi.org/10.1371/journal.pcbi.1003496
- Gross A, Schirm S, Scholz M. Ycasd - a tool for capturing and scaling data from graphical representations. BMC Bioinformatics. 2014;15(1):219. doi:10.1186/1471-2105-15-219 read 9 October 2026. doi.org/10.1186/1471-2105-15-219
- Favrin B. esyN: Network Building, Sharing and Publishing. PLOS ONE. 2014;9(9):e106035. doi:10.1371/journal.pone.0106035 read 9 October 2026. doi.org/10.1371/journal.pone.0106035
- Langfelder P, Mischel PS, Horvath S, Ravasi T. When Is Hub Gene Selection Better than Standard Meta-Analysis? PLOS ONE. 2013;8(4):e61505. doi:10.1371/journal.pone.0061505 read 9 October 2026. doi.org/10.1371/journal.pone.0061505
- Shabalin AA. Matrix eQTL: ultra fast eQTL analysis via large matrix operations. Bioinformatics. 2012;28(10):1353-1358. doi:10.1093/bioinformatics/bts163 read 9 October 2026. doi.org/10.1093/bioinformatics/bts163
- Langfelder P, Luo R, Oldham MC, Horvath S. Is my network module preserved and reproducible? PLOS Computational Biology. 2011;7(1):e1001057. doi:10.1371/journal.pcbi.1001057 read 9 October 2026. doi.org/10.1371/journal.pcbi.1001057
- Paul SM, Mytelka DS, Dunwiddie CT, Persinger CC, Munos BH, Lindborg SR, Schacht AL. How to improve R&D productivity: the pharmaceutical industry's grand challenge. Nature Reviews Drug Discovery. 2010;9(3):203-214. doi:10.1038/nrd3078 read 9 October 2026. doi.org/10.1038/nrd3078
- Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information and Libraries Journal. 2009;26(2):91-108. doi:10.1111/j.1471-1842.2009.00848.x read 9 October 2026. doi.org/10.1111/j.1471-1842.2009.00848.x
- Koschutzki D, Schreiber F. Centrality Analysis Methods for Biological Networks and Their Application to Gene Regulatory Networks. Gene Regulation and Systems Biology. 2008;2:193-201. doi:10.4137/grsb.s702 read 9 October 2026. doi.org/10.4137/grsb.s702
- Benson DA, Karsch-Mizrachi I, Lipman DJ, Ostell J, Wheeler DL. GenBank. Nucleic Acids Research. 2008;36(Database issue):D25-D30. doi:10.1093/nar/gkm929 read 9 October 2026. doi.org/10.1093/nar/gkm929
- Langfelder P, Horvath S. WGCNA: an R package for weighted correlation network analysis. BMC Bioinformatics. 2008;9:559. doi:10.1186/1471-2105-9-559 read 9 October 2026. doi.org/10.1186/1471-2105-9-559
- Materi W, Wishart DS. Computational systems biology in drug discovery and development: methods and applications. Drug Discovery Today. 2007;12(7-8):295-303. doi:10.1016/j.drudis.2007.02.013 read 9 October 2026. doi.org/10.1016/j.drudis.2007.02.013
- Yip AM, Horvath S. Gene network interconnectedness and the generalized topological overlap measure. BMC Bioinformatics. 2007;8:22. doi:10.1186/1471-2105-8-22 read 9 October 2026. doi.org/10.1186/1471-2105-8-22
- Larranaga P, Calvo B, Santana R, Bielza C, Galdiano J, Inza I, et al. Machine learning in bioinformatics. Briefings in Bioinformatics. 2006;7(1):86-112. doi:10.1093/bib/bbk007 read 9 October 2026. doi.org/10.1093/bib/bbk007
- Zhang B, Horvath S. A general framework for weighted gene co-expression network analysis. Statistical Applications in Genetics and Molecular Biology. 2005;4:17. doi:10.2202/1544-6115.1128 read 9 October 2026. doi.org/10.2202/1544-6115.1128
- Ouzounis CA, Valencia A. Early bioinformatics: the birth of a discipline - a personal view. Bioinformatics. 2003;19(17):2176-2190. doi:10.1093/bioinformatics/btg309 read 9 October 2026. doi.org/10.1093/bioinformatics/btg309
- Ravasz E, Somera AL, Mongru DA, Oltvai ZN, Barabasi AL. Hierarchical organization of modularity in metabolic networks. Science. 2002;297(5586):1551-1555. doi:10.1126/science.1073374 read 9 October 2026. doi.org/10.1126/science.1073374
Every source above is a peer-reviewed article, a publisher’s record of one, or a standard issued by the body that owns it. Each carries the date it was last read. Where a standard moves, this article is revised rather than left up with a note.
How to cite this article
Pick your referencing style and copy it. Your university or target journal may vary the punctuation — check its guide if it has one.
Pubrica Academy. (2026, October 9). Systematically Reviewing Developments in Bioinformatics and Computational Biology: Scope, Instruments and Reporting. Pubrica. https://pubrica.com/academy/research/current-developments-in-bioinformatics-and-computational-biology-a-systematic-review/
Pubrica Academy. “Systematically Reviewing Developments in Bioinformatics and Computational Biology: Scope, Instruments and Reporting.” Pubrica, 9 Oct. 2026, pubrica.com/academy/research/current-developments-in-bioinformatics-and-computational-biology-a-systematic-review/.
Pubrica Academy (2026) Systematically Reviewing Developments in Bioinformatics and Computational Biology: Scope, Instruments and Reporting. Pubrica. Available at: https://pubrica.com/academy/research/current-developments-in-bioinformatics-and-computational-biology-a-systematic-review/ (Accessed: 15 October 2026).
Pubrica Academy. “Systematically Reviewing Developments in Bioinformatics and Computational Biology: Scope, Instruments and Reporting.” Pubrica. October 9, 2026. https://pubrica.com/academy/research/current-developments-in-bioinformatics-and-computational-biology-a-systematic-review/
Pubrica Academy, ‘Systematically Reviewing Developments in Bioinformatics and Computational Biology: Scope, Instruments and Reporting’ (Pubrica, 9 October 2026) <https://pubrica.com/academy/research/current-developments-in-bioinformatics-and-computational-biology-a-systematic-review/> accessed 15 October 2026.
Pubrica Academy. Systematically Reviewing Developments in Bioinformatics and Computational Biology: Scope, Instruments and Reporting. [Internet]. Pubrica; 2026 Oct 9 [cited 2026 Oct 15]. Available from: https://pubrica.com/academy/research/current-developments-in-bioinformatics-and-computational-biology-a-systematic-review/
Pubrica Academy, “Systematically Reviewing Developments in Bioinformatics and Computational Biology: Scope, Instruments and Reporting,” Pubrica, Oct. 9, 2026. [Online]. Available: https://pubrica.com/academy/research/current-developments-in-bioinformatics-and-computational-biology-a-systematic-review/
@misc{current_developments_in_bioinformatics_2026,
author = {{Pubrica Academy}},
title = {Systematically Reviewing Developments in Bioinformatics and Computational Biology: Scope, Instruments and Reporting},
howpublished = {Pubrica},
year = {2026},
month = {oct},
url = {https://pubrica.com/academy/research/current-developments-in-bioinformatics-and-computational-biology-a-systematic-review/},
note = {Accessed: <span data-cite-acc="iso">2026-10-15</span>}
}This article is written to be learned from. Cite it if you are referring to what it says; if you are using the evidence behind it, cite the sources listed above instead.
Protocol stub for a computational methods review
The first two blocks determine whether these reviews succeed or fail. They should be completed before any search term is written, and the version and date fields should be kept current throughout the work.
COMPUTATIONAL METHODS REVIEW — PROTOCOL STUB
=== BLOCK 1: INSTRUMENT AND BOUNDARY ===
INSTRUMENT (tick one; put it in the title):
[ ] systematic review — requires a finite, countable included set
[ ] scoping review — mapping, no performance claim
[ ] evidence map — matrix output
[ ] state-of-the-art review — no exhaustive-retrieval claim
[ ] living review — update cadence: ____________
TASK, as input -> output: ____________________________
If you cannot write that in one line, the instrument is wrong.
Data type(s) in scope: _______________________________
Explicitly OUT of scope: _____________________________
=== BLOCK 2: UNIT OF ANALYSIS ===
The unit is: [ ] the METHOD [ ] the EVALUATION [ ] the PAPER
LINKING RULE: when do several records describe one item?
___________________________________________________
When does a modification become a NEW item?
[ ] new name assigned by the authors
[ ] new version with separately reported performance
[ ] other: _________________________________________
Flow diagram counts: ________________________________
=== BLOCK 3: ELIGIBILITY ===
Must propose or substantially modify a method for the task [ ]
Must report evaluation on data not used for development [ ]
Implementation required? [ ] yes [ ] no, detail sufficient
Benchmark evaluation required? [ ] yes, named: ______ [ ] no
REPOSITORY-ONLY methods (no publication): [ ] eligible
[ ] excluded, stated why
Preprints: [ ] eligible [ ] excluded
Rule where a journal version also exists: ____________
Conference proceedings: [ ] eligible (primary in this field)
DATE LIMIT: [ ] none (recommended)
[ ] from ____ as a FEASIBILITY restriction, and claims
about change over time are out of scope [ ]
=== BLOCK 4: SOURCES (record a date for every one) ===
Databases with platform and date: ____________________
Method-name, task and LEGACY vocabulary terms: ________
Preprint servers + dates: ____________________________
Conference proceedings + years: ______________________
Repositories + access dates: _________________________
Community assessment / shared benchmark results: ______
accessed: ______ reported SEPARATELY throughout [ ]
Citation tracking on key method papers: index ____ date ____
NOT searched, and declared as a limitation: ____________
=== BLOCK 5: EXTRACTION, PER ITEM ===
IDENTITY
Method name and VERSION: _____________________________
All records describing it: ___________________________
Implementation available? [ ] yes, licence ______ [ ] no
Supersedes an earlier method by the same group? [ ] yes: ______
TASK AND DATA
Input / output definitions: __________________________
Development dataset + VERSION + release date: ________
Evaluation dataset + VERSION + release date: _________
Size in the relevant unit: ___________________________
EVALUATION DESIGN
Benchmark + VERSION: _________________________________
Split: _______________________________________________
Evaluation script / metric implementation: ___________
Does evaluation data POSTDATE development? [ ] yes [ ] no [ ] cannot tell
BASELINES
Comparators + versions: ______________________________
Run by: [ ] proposing authors [ ] third party
Tuned comparably? [ ] yes [ ] no [ ] cannot tell
INDEPENDENT EVALUATION
Assessed by anyone other than the developers? [ ] no [ ] yes:
Source: ______ Result: ____________________________
REPRODUCIBILITY
Code [ ] Trained parameters [ ] Data [ ] Environment spec [ ]
Reported result reproduced by anyone? [ ] yes [ ] no [ ] unknown
=== BLOCK 6: SYNTHESIS PLAN (no pooling) ===
[ ] task-structured table, each number kept with its benchmark version
[ ] evidence map: tasks x evaluation levels
[ ] methods-lineage account: how approaches derive from each other
[ ] developer-run and independent evaluations in SEPARATE columns
[ ] statement of what would settle the open questions
[ ] NO pooled performance estimate
=== BLOCK 7: DATE STAMPS AND EXPIRY ===
Search dates per source: _____________________________
Repository access dates: _____________________________
Benchmark versions examined: _________________________
Expiry sentence included in the discussion? [ ] yes
Historical baseline consulted (the field's own history)? [ ] yes Pubrica AcademyWritten and maintained by Pubrica’s subject matter experts. Articles are revised when the underlying standards change, and each article states the date on which it was last reviewed.
How Pubrica can help
Pubrica scopes these reviews to a task that the available evidence can answer, fixes the unit of analysis and the linking rule before screening, and searches repositories, preprint servers and community assessment results alongside the journal literature. The deliverable keeps developer-run and independent evaluations in separate columns, records every method and benchmark version, and states what the review cannot claim, which in this field is frequently the most consequential paragraph in the manuscript.