Free scoping read. A written assessment of your manuscript in two business days, at no cost.

Start one
Contents

Academy · Research Data, Analytics & AI · Clinical Data Collection Approaches

Common Medical Data Collection Approaches in Clinical Research and the Standardization of Data Elements

Electronic health record (EHR) extraction, chart abstraction, patient-reported surveys and registries are not interchangeable, and none produces data that are automatically comparable across studies without a shared data-element standard.

Pubrica AcademyWritten by a clinical research informaticist Reviewed bya research data standards specialist Published10 October 2026 Last checked10 October 2026 Reading time7 minutes Sources11 peer-reviewed

Clinical research has no single method for obtaining medical information from patients and their records; it has a small set of distinct approaches, each with its own failure modes, and a recurring problem common to all of them. Richesson and Nadkarni, surveying clinical research data collection forms, state this problem directly: no universal standard exists for designing a case report form, and the data elements collected by different studies are rarely defined in the same way, which limits the extent to which data from one study can be compared, pooled or reused in another.1 This article surveys the approaches in current use — direct extraction from electronic health records, manual chart abstraction, patient-reported data collection, registries and, increasingly, device and sensor data — and then addresses the standardization layer, built around metadata registry standards such as ISO/IEC 11179, that determines whether any of them produces data that other researchers can reuse.

Learning objectives

  • The distinct data collection approaches used in clinical research, as opposed to different names for the same process
  • Why Richesson and Nadkarni found no universal standard for case report form design, and the consequences for researchers
  • What a metadata registry standard like ISO/IEC 11179 does and does not solve
  • Where the strengths and failure modes of each approach lie, so that the appropriate approach is chosen for each question

Common misconceptions

  • “Electronic health record extraction and chart abstraction are the same thing, just automated versus manual.” EHR extraction pulls discrete, already-coded fields (lab values, diagnosis codes, medication orders) directly from the database; chart abstraction is the manual reading and interpretation of free-text clinical narrative that was never coded in the first place. They fail in different ways — extraction fails when the needed data was never coded, abstraction fails on inter-rater disagreement — and conflating them hides which failure mode a given study is actually exposed to.
  • “Using a standard like ISO/IEC 11179 makes data from different studies automatically comparable.” Richesson and Nadkarni note that ISO/IEC 11179 captures the semantics of individual data elements reasonably well but cannot represent constraints between elements, does not model groupings like CRFs, and lacks element ordering or skip logic — so adopting it does not by itself make two studies' CRFs comparable; it only standardizes how each element is described.1

Distinct data collection approaches in current use

Clinical research draws patient data from a small number of genuinely distinct sources, and the choice between them is rarely interchangeable: it is set by what question is being asked, what already exists in structured form, and what the study can afford to collect prospectively. The main approaches are direct extraction from electronic health records, manual abstraction from clinical charts, data reported directly by patients, structured registries built to track a defined population over time, and, increasingly, data streamed from wearable or implanted sensors. Each has a different relationship to the underlying clinical encounter, and each fails in a different way when it goes wrong.

Electronic health record extraction

EHR extraction pulls fields that are already discrete and coded within the record — laboratory results, diagnosis and procedure codes, medication orders, vital signs — directly into a research dataset, generally through a query against the EHR's database or a clinical data warehouse built on top of it. Its strength is scale: a query can retrieve data for thousands of patients in the time a manual review would take for dozens. Its weakness is that it can only retrieve what was coded in the first place; a clinical detail that exists only as free text in a progress note is invisible to a structured extraction, no matter how large the query.

Manual chart abstraction

Chart abstraction is the manual reading of clinical narrative — progress notes, discharge summaries, operative reports — by a trained person who records the relevant findings onto a structured abstraction form. It recovers information EHR extraction cannot reach, at the cost of being slow, expensive, and dependent on the abstractor's judgment, which is why abstraction-specific methodology (operational definitions, pilot testing, inter-abstractor reliability checks) exists as its own area of practice distinct from EHR extraction methodology.

Patient-reported data and registries

Patient-reported data collection — symptom diaries, quality-of-life questionnaires, patient-reported outcome measures — captures information that does not exist in the chart at all, because it reflects the patient's own experience rather than a clinician's observation. Registries are a distinct, longer-horizon approach: a structured, ongoing data collection effort built around a defined population (a disease, a device, a procedure) and designed to track outcomes over years rather than over the course of a single study, typically combining several of the other approaches — extraction, abstraction, and patient-reported data — under one governance structure.

The standardization problem described by Richesson and Nadkarni

Richesson and Nadkarni's central finding, surveying the state of clinical research data collection forms, is that no universal standard governs how a case report form should be designed, and that CDISC's standardization efforts — while real and useful — are concentrated on regulated studies rather than research broadly.1 The practical consequence is that the same clinical concept — a symptom severity grade, a comorbidity definition — is very often defined slightly differently by every study that collects it, which is what makes pooling or comparing data across studies difficult even when every individual study collected its data carefully.

Scope and limitations of ISO/IEC 11179

ISO/IEC 11179 is the metadata registry standard most directly relevant here: it defines a data element as a unit of data for which the definition, identification, representation, and permissible values are specified, giving researchers a shared vocabulary for describing what a single field means and what values it can take.2 Richesson and Nadkarni note, however, that it captures individual elements well while failing to represent constraints between elements, groupings such as CRFs, element ordering, or skip logic — meaning it standardizes the description of a single field without standardizing the form the field sits inside.1 Choosing a data collection approach and adopting a metadata standard are therefore two separate decisions, and a study needs both: the right approach for the question being asked, and a standards-based description of its data elements if that data is ever meant to be reused or compared.

Reporting the data collection approach with STROBE

Whichever approach is chosen, most of the designs described above — EHR extraction, chart abstraction, registry-based collection — fall under the observational study umbrella, and the STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) Statement is the relevant checklist for describing that choice transparently in a published paper. STROBE, developed by von Elm, Altman, Egger, Pocock, Gøtzsche and Vandenbroucke and published simultaneously in 2007 across journals including Annals of Internal Medicine, The Lancet, and BMJ, asks authors to state explicitly how participants were identified, how key variables were measured, and what sources of bias were addressed.3 Applied to the taxonomy in this article, that means a paper should name which of the distinct approaches supplied each variable — not simply state that "data were collected from medical records," which conflates extraction, abstraction, and patient-reported sources into a single vague phrase a reader cannot evaluate.

The approaches also differ in what data-governance route is available for using them without full patient authorization. HHS's guidance on HIPAA and research describes de-identification (stripping data of the identifiers specified at 45 CFR 164.514 so it falls outside the Privacy Rule entirely), IRB or Privacy Board waiver of authorization, and the limited data set shared under a data use agreement as the three main routes.4 EHR extraction and chart abstraction, which draw on records that already exist, are the approaches most often run under a waiver or a limited data set, since re-contacting every patient for authorization would often defeat the purpose of a retrospective design; prospective patient-reported data collection, by contrast, typically obtains authorization directly from the participant at the point of collection, because the data does not exist until the patient actively provides it.4 Choosing an approach, in other words, is not purely a measurement decision — it also determines which governance pathway is realistic.

This has a direct bearing on registries in particular, since a registry typically combines several approaches under one long-running governance structure. A registry that mixes retrospective EHR extraction with prospective patient-reported follow-up may need to run two different authorization pathways concurrently — a waiver or limited data set for the historical extraction, and direct participant authorization for the ongoing patient-reported component — rather than a single governance plan covering the whole effort. Treating "the registry's data collection method" as one undifferentiated approach, instead of naming each component separately, is the same conflation error Richesson and Nadkarni identify at the level of data elements, just one level up, at the level of governance.

Supporting evidence across these data collection approaches

Several of the broader methodological sources underlying this comparison are worth naming directly. Weiskopf and Weng's framework for electronic health record data quality assessment gives any of the EHR-based approaches described above a shared vocabulary — completeness, correctness, concordance, plausibility, currency — for describing exactly what can go wrong with routinely collected data before it reaches an analysis file.5 Cowie and colleagues extend this to the clinical-research use case specifically, documenting how electronic health records can support patient identification, outcome ascertainment and even pragmatic trial conduct, provided the underlying data quality issues are addressed first.6

On tooling, Harris and colleagues' description of REDCap documents the metadata-driven electronic data capture platform widely used across all of the structured-form approaches discussed here, with built-in validation rules that reduce transcription error relative to a paper-based form.7 Gregory and Radovinsky's research strategies for optimal data collection from the patient medical record speak directly to the manual chart-abstraction end of this spectrum, covering abstractor training and source-field precedence rules. 8 And where structured data collection feeds a formal patient registry rather than a single study, AHRQ's Registries for Evaluating Patient Outcomes guide sets out the data-element selection and governance practices that sit above any one approach's form design.9

Two further sources address validity and extraction specifically. Harris, McGregor and colleagues' guidance on quasi-experimental studies in medical informatics describes the threats to internal validity that apply whenever any of these approaches is used to evaluate an intervention rather than simply describe a population.10 And where free text has to be converted into structured fields, Wieland-Jorna and colleagues' systematic review of natural language processing systems for extracting information from electronic health records found performance varying widely by clinical domain, a reason to validate any automated extraction step against the same kind of reliability check this article recommends for manual abstraction.11

Key points

Key points

EHR extraction, chart abstraction, patient-reported data and registries are distinct approaches with different failure modes and different legal and governance routes. In the absence of a universal case report form (CRF) standard, their outputs are not automatically comparable without deliberate use of metadata standards such as ISO/IEC 11179 and transparent STROBE-style reporting.

Practical application

Practical application

  1. The approach should be matched to the research question rather than chosen for convenience. EHR extraction is appropriate only for variables that are reliably coded; abstraction is required when the necessary detail exists only in narrative text.
  2. The failure mode of each approach should be addressed explicitly. The protocol should state whether the principal risk is incomplete coding (extraction), rater disagreement (abstraction) or recall bias (patient-reported data).
  3. A metadata standard should be adopted deliberately, not by default. ISO/IEC 11179 or a domain-specific CDISC standard should be used to describe data elements if cross-study comparison or reuse is an objective.
  4. Standardization should not be assumed to resolve form design. Element-level standards do not define CRF structure, skip logic or inter-element constraints; these still require study-specific design work.

Frequently asked questions

None is uniformly more accurate; each is accurate for what it is built to capture and unreliable outside that scope — EHR extraction for coded fields, abstraction for narrative detail, patient-reported data for subjective experience.

Partially. Richesson and Nadkarni note CDISC's efforts are substantial but concentrated on regulated studies, leaving much of academic and investigator-initiated research without an equivalent universal standard.

It is not a legal requirement; it is a voluntary metadata registry standard that improves the describability and potential reuse of data elements when adopted deliberately.

No. STROBE is a reporting checklist for observational studies generally; it asks authors to state clearly which approach was used and how, but it does not recommend EHR extraction, abstraction, or any other specific approach over another.

Not typically. Extraction and abstraction from existing records commonly proceed under a de-identification standard, an IRB waiver, or a limited data set, since the data already exists; prospective patient-reported collection usually obtains the participant's authorization directly at the point of collection.

References

  1. Richesson RL, Nadkarni P. Data standards for clinical research data collection forms: current status and challenges. J Am Med Inform Assoc. 2011;18(3):341-346. read 10 October 2026. View on PubMed Central
  2. International Organization for Standardization. ISO/IEC 11179-1:2023 Information technology — Metadata registries (MDR) — Part 1: Framework. read 10 October 2026. View source
  3. von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP; STROBE Initiative. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Ann Intern Med. 2007;147(8):573-577. read 10 October 2026. View on EQUATOR Network
  4. U.S. Department of Health and Human Services, Office for Civil Rights. HIPAA Privacy Rule and research: de-identification, authorization waivers, and limited data sets. read 10 October 2026. View source
  5. Weiskopf NG, Weng C. Methods and dimensions of electronic health record data quality assessment: enabling reuse for clinical research. Journal of the American Medical Informatics Association. 2013;20(1):144-151. read 10 October 2026. View on PubMed Central
  6. Cowie MR, Blomster JI, Curtis LH, et al. Electronic health records to facilitate clinical research. Clinical Research in Cardiology. 2017;106(1):1-9. read 10 October 2026. View on PubMed Central
  7. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap): a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377-381. read 10 October 2026. View on PubMed
  8. Gregory KE, Radovinsky L. Research strategies that result in optimal data collection from the patient medical record. Appl Nurs Res. 2012;25(2):108-116. read 10 October 2026. View on PubMed Central
  9. Gliklich RE, Leavy MB, Dreyer NA, eds. Registries for Evaluating Patient Outcomes: A User's Guide. 4th ed. Rockville, MD: Agency for Healthcare Research and Quality; 2020. AHRQ Publication No. 19(20)-EHC020. read 10 October 2026. View source
  10. Harris AD, McGregor JC, Perencevich EN, Furuno JP, Zhu J, Peterson DE, Finkelstein J. The use and interpretation of quasi-experimental studies in medical informatics. J Am Med Inform Assoc. 2006;13(1):16-23. read 10 October 2026. View on PubMed Central
  11. Wieland-Jorna Y, van Kooten D, Verheij RA, de Man Y, Francke AL, Oosterveld-Vlug MG. Natural language processing systems for extracting information from electronic health records about activities of daily living: a systematic review. JAMIA Open. 2024;7(2):ooae044. read 10 October 2026. View source

Every source above is a peer-reviewed article, a publisher’s record of one, or a standard issued by the body that owns it. Each carries the date it was last read. Where a standard moves, this article is revised rather than left up with a note.

How to cite this article

Pick your referencing style and copy it. Your university or target journal may vary the punctuation — check its guide if it has one.

Pubrica Academy. (2026, October 10). Common Medical Data Collection Approaches in Clinical Research and the Standardization of Data Elements. Pubrica. https://pubrica.com/academy/concepts-definitions/commonly-used-medical-data-collection-approaches-in-clinical-research/

This article is written to be learned from. Cite it if you are referring to what it says; if you are using the evidence behind it, cite the sources listed above instead.

Data collection approach selection checklist

This tool supports matching a study’s data collection approach to its research question.

[ ] Confirmed which variables are reliably coded in the EHR vs. only in narrative text
[ ] Decided whether patient-reported outcomes are needed for subjective measures
[ ] Assessed whether a registry-level, multi-year approach is warranted
[ ] Identified each approach's main failure mode and documented mitigation
[ ] Decided whether a metadata standard (ISO/IEC 11179, CDISC) will be applied to data elements
[ ] Confirmed CRF structure and skip logic are separately designed, not assumed from the standard

Pubrica AcademyWritten and maintained by Pubrica’s subject matter experts. Articles are revised when the underlying standards change, and each article states the date on which it was last reviewed.

How Pubrica can help

Pubrica helps research teams select and combine data collection approaches — EHR extraction, abstraction, patient-reported data, registries — and apply data-element standards for studies intended for comparison or reuse.

Find something else

Search the Pubrica Academy

Access is free and requires no account. Enter a topic, or open the full library to filter by category, sub-topic or resource type.

Selection of a data collection method appropriate to the study

A scoping assessment is free and takes two business days. It provides a written assessment of what the work requires and which specialist would undertake it.

Get a free quote

Regulations

Check your project against this rule

  1. 1Your work
  2. 2Your details
  3. 3Send
What do you need?

Material you send is held under signed confidentiality, with access limited to the assigned team and encrypted transfer. It is not used to train anything and is not passed to third-party tools. Ask for the agreement before you send anything you would rather cover first.