Free scoping read. A written assessment of your manuscript in two business days, at no cost.

Start one
Contents

Academy · Research Data, Analytics & AI · Text mining in Data Extraction

Text Mining in Data Extraction for Systematic Reviews: Capabilities, Evidence and Limitations

Text mining does not replace the reviewer who reads a trial report and determines, for example, the sample size. It replaces the initial reading that locates where such a value is reported.

Pubrica AcademyWritten by a systematic-review methodologist Reviewed bya data-extraction lead Published10 October 2026 Last checked10 October 2026 Reading time9 minutes Sources3 peer-reviewed

In the context of data extraction, text mining refers to the use of natural language processing software to locate and extract candidate values, such as sample size, outcome definitions and risk-of-bias judgements, from the free text of included studies, so that a human extractor checks and corrects a pre-filled field rather than starting from a blank one.

Learning objectives

  • The functions of text-mining tools for data extraction, and the task against which they are evaluated.
  • The measured efficiency gains reported for these tools in prospective evaluations, as distinct from vendor claims.
  • The fields that text mining tends to extract reliably, and those that still require a human reviewer to read the source text.
  • Why “semi-automated” rather than “automated” is the accurate description.

Common misconceptions

  • “Text mining extracts data automatically.” Every published evaluation of these tools describes a semi-automated workflow in which the software proposes values and a trained reviewer verifies them against the source document before any value is accepted.
  • “If the tool pre-fills a field, the value is correct.” Prospective evaluations found meaningful miss rates for some field types; pre-filled values still require verification against the source text, particularly for numeric and conditional fields.
  • “A single tool performs equally well for every field type.” Published evaluations indicate that accuracy varies substantially by field: simple, consistently reported fields are extracted more reliably than fields that depend on interpretation or appear in variable formats across papers.

Functions of text-mining tools

Text-mining tools for data extraction operate against a defined target: a list of fields that a systematic review requires from each included study (population size, intervention, comparator, outcome, numeric results) and a corpus of PDFs or full texts to be searched. The tool parses each document, locates text spans likely to contain each field, and proposes a value with a link to its location in the source; the reviewer then opens that link, confirms the value against the original sentence and corrects it where necessary. Gates and colleagues, evaluating one such tool prospectively against trials being extracted for an ongoing systematic review, describe it explicitly as a tool intended to create efficiencies in a human-led process rather than to remove the human step1.

Outcomes measured in prospective evaluations

Gates et al. (2021) conducted their evaluation prospectively, extracting data with the tool from trials included in a live Cochrane-style review and comparing the tool-assisted extraction with full dual-reviewer manual extraction of the same trials, and reported both the accuracy of the machine-suggested values and the time saved by starting from those values rather than from a blank form1. Panayi and colleagues (2023) evaluated a different prototype tool against systematic-review extraction tasks and likewise reported that it reduced extraction time for the fields it handled well, while identifying specific field types for which reviewer correction was still routinely required2. A comprehensive 2024 review by Ofori-Boateng and colleagues, surveying natural language processing (NLP), machine learning and deep-learning methods applied across the systematic-review pipeline (not only extraction), identifies data extraction as one of the steps in which automation has advanced furthest but which still operates as a decision-support layer under reviewer control3.

Reliability of text mining by field type

Across these evaluations the pattern is consistent: fields that are consistently reported, consistently located (the stated sample size in an abstract, a clearly labelled outcome table), and numeric or short in form are extracted more reliably than fields that require the synthesis or interpretation of information distributed across the methods and results sections, or that depend on judgement (such as the assessment of risk of bias, or a decision on whether an outcome definition matches the pre-specified definition of the review)13. None of the three sources above reports a tool suitable for auto-populating the evidence table of a review without a reviewer checking every value against the source PDF before it is used in analysis.

Key points

Key points

Text mining for data extraction yields measurable time savings for well-defined, consistently reported fields. However, every published evaluation describes a semi-automated process with human verification rather than a replacement for the reviewer who reads the source paper.

Practical application

Practical application

  1. The use of the tool should be limited to the fields for which it performs well. Text-mining assistance should be applied first to sample size, simple numeric outcomes and clearly labelled table values; fields that depend on judgement should be expected to require manual extraction.
  2. The link to the source text should be retained in the workflow. Every machine-suggested value should be checked against the linked source sentence before it enters the evidence table, rather than being approved without verification.
  3. The extraction method should be reported accurately. If a review used text-mining-assisted extraction, authors should name the tool, describe the fields for which it was used and describe the human verification step, rather than describing the extraction as automated.

Frequently asked questions

No published evaluation of these tools supports the removal of human verification; the reported efficiency gain derives from starting with a pre-filled, source-linked value rather than a blank field, not from removing the reviewer.

Consistently reported numeric or short fields, such as sample size and clearly tabulated outcome values, benefit most; fields that require interpretation or synthesis across sections benefit least.

No. The tools evaluated in the cited studies locate and extract specific, pre-defined fields with a traceable link to source text, which is a narrower and more checkable task than open-ended summarisation.

References

  1. Gates A, Gates M, Sim S, Elliott SA, Pillay J, Hartling L. Creating efficiencies in the extraction of data from randomized trials: a prospective evaluation of a machine learning and text mining tool. BMC Med Res Methodol. 2021;21:169. read 10 October 2026. View on BMC Medical Research Methodology
  2. Panayi A, Ward K, Benhadji-Schaff A, Ibanez-Lopez AS, Xia A, Barzilay R. Evaluation of a prototype machine learning tool to semi-automate data extraction for systematic literature reviews. Syst Rev. 2023;12:187. read 10 October 2026. View on Systematic Reviews
  3. Ofori-Boateng R, Aceves-Martins M, Wiratunga N, Moreno-Garcia CF. Towards the automation of systematic reviews using natural language processing, machine learning, and deep learning: a comprehensive review. Artif Intell Rev. 2024;57:200. read 10 October 2026. View on Artificial Intelligence Review

Every source above is a peer-reviewed article, a publisher’s record of one, or a standard issued by the body that owns it. Each carries the date it was last read. Where a standard moves, this article is revised rather than left up with a note.

How to cite this article

Pick your referencing style and copy it. Your university or target journal may vary the punctuation — check its guide if it has one.

Pubrica Academy. (2026, October 10). Text Mining in Data Extraction for Systematic Reviews: Capabilities, Evidence and Limitations. Pubrica. https://pubrica.com/academy/concepts-definitions/text-mining-in-data-extraction/

This article is written to be learned from. Cite it if you are referring to what it says; if you are using the evidence behind it, cite the sources listed above instead.

Text-mining-assisted extraction verification log

The template contains one row per field per study, for use by the reviewer verifying machine-suggested values.

Study ID | Field | Machine-suggested value | Source sentence (quoted) | Verified? (Y/N) | Correction (if any) | Reviewer initials

Pubrica AcademyWritten and maintained by Pubrica’s subject matter experts. Articles are revised when the underlying standards change, and each article states the date on which it was last reviewed.

How Pubrica can help

Pubrica’s data extraction teams use text-mining-assisted tools to accelerate the identification of candidate values, but every value entered into the evidence table is checked against the original source text by a trained reviewer before it is used in analysis.

Find something else

Search the Pubrica Academy

Access is free and requires no account. Enter a topic, or open the full library to filter by category, sub-topic or resource type.

Pubrica can conduct data extraction for systematic reviews to a reportable standard.

A scoping assessment is free and takes two business days. It provides a written assessment of what the work requires and which specialist would undertake it.

Get a free quote

Regulations

Check your project against this rule

  1. 1Your work
  2. 2Your details
  3. 3Send
What do you need?

Material you send is held under signed confidentiality, with access limited to the assigned team and encrypted transfer. It is not used to train anything and is not passed to third-party tools. Ask for the agreement before you send anything you would rather cover first.