Contents
On this page
- Prospective and retrospective collection from electronic health records (EHRs)
- Structured fields and free-text extraction
- Performance of natural language processing (NLP)-assisted extraction
- Five dimensions of data quality
- The governance basis for reuse of EHR data in research
- Reporting EHR-based methodology for critical appraisal
Academy · Research Data, Analytics & AI · EHR Data Collection Methodology
Research Methodologies for Adequate Data Collection from the Patient Medical Record
The choice between prospective and retrospective collection of data from electronic health records (EHRs), and between structured fields and natural language processing (NLP)-assisted text extraction, is a methodological decision with documented trade-offs rather than a matter of convenience.
Patient medical records appear to constitute a single data source, but the extraction of research-grade data from them involves a substantive methodological choice: prospective collection designed for the study versus retrospective reuse of records collected for care; structured fields versus natural language processing applied to free text. Cowie and colleagues' widely cited review of EHR use in clinical research, Weiskopf and Weng's data-quality framework, and Wieland-Jorna and colleagues' 2024 systematic review of NLP extraction methods each document a different part of this trade-off, and together they describe the requirements of adequate data collection from a medical record.
Learning objectives
- The trade-offs between structured, coded EHR fields and free-text data requiring NLP extraction.
- Five documented dimensions of EHR data quality, and those that researchers assess most and least frequently.
- Areas in which EHR use is methodologically well established, and those in which its use in prospective research remains limited.
Common misconceptions
- “Structured, coded fields in an EHR are always more reliable than the free text written by a clinician.” Cowie et al. note that structured, coded EHR data may originate as free text subsequently coded for billing or registry purposes, and that this coding can lag the clinical event by weeks or months; consequently, a coded field is not necessarily more current or complete than the narrative note from which it was derived.
- “If NLP can extract information from clinical free text, it is a straightforward substitute for structured data collection.” Wieland-Jorna et al.’s 2024 systematic review of 22 studies (2017-2022) found that NLP performance varied widely with the dataset, research question and specific activity being extracted, and that studies were difficult to compare because of inconsistent datasets and designs; NLP extraction is therefore a method with measured variation in performance rather than a uniform solution.
Prospective and retrospective collection from electronic health records (EHRs)
Cowie, Blomster, Curtis, Duclaux, Ford, Fritz, Goldman, Janmohamed, Kreuzer, Leenay, Michel, Ong, Pell, Southworth, Stough, Thoenes, Zannad and Zalewski reviewed the use of electronic health records in clinical research in Clinical Research in Cardiology in 20161. In their assessment, EHRs are methodologically well established for observational research, safety surveillance and retrospective registry-style studies, whereas their use in prospective clinical research — feasibility assessment, patient recruitment, pragmatic or registry-based randomised trials — remains comparatively limited1.
For prospective uses specifically, the authors recommend validating EHR-derived endpoints rather than accepting them without verification — for example, by running an electronic case report form in parallel with EHR capture, or by using a hybrid approach in which candidate events are identified in the EHR and then clinically adjudicated1. They also note that EHR data access can lag real time, which is particularly relevant to time-sensitive prospective uses such as serious adverse event reporting1.
Structured fields and free-text extraction
Cowie et al. frame the structured/unstructured distinction in terms of coding rather than format: EHR fields that appear structured may have originated as free text subsequently coded for the record, so the coded version may not be available when a researcher requires it, and the coding itself can occur weeks or months after the clinical event1. They note that coded events linked to reimbursement or new diagnoses, such as hospitalisations and myocardial infarction, tend to be more reliable, but add that coding driven by billing incentives is itself a documented concern rather than a neutral administrative detail1.
Broader data heterogeneity compounds this problem. The authors identify differing vendor systems, non-standardised terminology, the absence of common data fields and the inability to link patient identifiers reliably across systems as structural barriers to complete capture, and recommend common minimum data sets and prespecified endpoint definitions in response1.
Performance of natural language processing (NLP)-assisted extraction
Wieland-Jorna, van Kooten, Verheij, de Man, Francke and Oosterveld-Vlug published a systematic review of NLP systems for extracting information about activities of daily living from EHRs in JAMIA Open in 2024, covering 22 studies published between 2017 and 20221. Their principal conclusion is that “NLP systems are valuable for the extraction of unstructured EHR data” on activities of daily living, converting free-text clinical notes into structured, analysable information1. The most common technical approach combined deep learning with rule-based or machine-learning methods, although pre-processing and algorithm choice varied widely across the included studies1.
The review is equally explicit about the limitations: most studies (65%) addressed only one or two activities of daily living rather than a broad set, performance was difficult to compare across studies because datasets and designs differed, and restricted data access, inconsistent documentation and small sample sizes were recurring constraints1. The authors conclude that the best-performing NLP approach depends on the specific dataset, research question and activity type being extracted, rather than there being a single universally superior method1.
Five dimensions of data quality
Weiskopf and Weng published “Methods and dimensions of electronic health record data quality assessment: enabling reuse for clinical research” in the Journal of the American Medical Informatics Association in 2013, reviewing 95 articles to identify how EHR data quality is assessed in practice1. They identify five dimensions: completeness (whether a true fact about the patient is present in the record), correctness (whether a present element is true), concordance (whether elements agree with each other or with another source), plausibility (whether an element is consistent with other medical knowledge), and currency (whether the element reflects the patient’s state at the relevant time)1. Completeness and correctness were by far the most frequently assessed dimensions (61 and 57 of the 95 articles respectively), while currency was assessed in only 41.
Their central argument is that data quality is task-dependent — a “fitness for use” framing — and that concordance and plausibility often serve as practical proxies for correctness and completeness when these cannot be assessed directly. Their stated recommendation is that “researchers should adopt validated, systematic methods of EHR data quality assessment” rather than informal spot-checks1.
The governance basis for reuse of EHR data in research
Everything above concerns measurement quality; a separate question is what legal basis allows a researcher to reuse EHR data for a purpose — research — different from the one it was originally recorded for, patient care. HHS's guidance on HIPAA and research sets out the main routes: de-identification under the standard at 45 CFR 164.514, which removes the data from the Privacy Rule's scope entirely; an IRB or Privacy Board waiver or alteration of individual authorization, granted only after a finding that the research poses no more than minimal privacy risk and could not practicably be done without the identifiable data; and a limited data set, shared under a data use agreement, with specified direct identifiers removed.4 Which route applies constrains what a study can actually extract. A fully de-identified extraction, for instance, cannot retain exact admission or discharge dates if those fall under the identifiers that must be removed — which can directly conflict with Weiskopf and Weng's "currency" dimension, since judging whether a data element reflects the patient's state at the relevant time often depends on exactly the kind of precise date a strict de-identification standard requires removing.1 A study's data-quality plan and its data-governance route have to be designed together, not decided independently and reconciled afterward.
Reporting EHR-based methodology for critical appraisal
Most EHR-based research — observational cohort studies, registry analyses, retrospective extractions — falls under the reporting scope of the STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) Statement. STROBE, developed by von Elm, Altman, Egger, Pocock, Gøtzsche and Vandenbroucke and published simultaneously in 2007 across several journals including Annals of Internal Medicine and The Lancet, asks authors to describe how key variables were defined, measured, and validated, and what steps were taken to address bias and missing data.5 Applied to the methodological choices in this article, a STROBE-compliant EHR study should state explicitly: whether the collection was prospective or retrospective in Cowie et al.'s sense; whether any NLP extraction was used and, if so, on which specific data elements, following Wieland-Jorna et al.'s finding that NLP performance varies by task and dataset; and which of Weiskopf and Weng's five data-quality dimensions were assessed, including — given how rarely it is checked — whether currency was addressed at all. A methods section that states only "data were extracted from the EHR" answers none of these, and leaves a reader unable to judge which of this article's documented trade-offs the study actually ran into.
Data standards, tools and registries supporting adequate collection
Richesson and Nadkarni's review of data standards for clinical research data collection forms found that most projects still define ad hoc variable sets rather than reusing standardized common data elements, which is precisely the gap that keeps individually "adequate" collection efforts from being easily comparable across studies.6 On tooling, Harris and colleagues' description of REDCap documents a metadata-driven electronic data capture platform with built-in validation designed to reduce exactly the transcription and missing-data problems that undermine adequacy in manually collected datasets.7
Gregory and Radovinsky's research strategies for optimal data collection from the patient medical record speak directly to the manual chart-abstraction methods this article's title references, covering abstractor training and source-field precedence rules when different parts of a record disagree.8 Where collection methodology feeds a structured registry rather than a single study, AHRQ's Registries for Evaluating Patient Outcomes guide sets the governance and data-element standards that sit above any individual study's form.9 And ISO/IEC 11179, the international metadata registry standard, gives the formal specification of a properly defined data element — name, definition, value domain, classification — that "adequate" data collection is implicitly reaching for even where a study does not cite the standard directly.10
Key points
Key points
No single methodology is correct for collecting research data from the patient medical record. The appropriate methodology depends on whether the use is prospective or retrospective, and on whether structured fields or NLP-assisted free-text extraction better suits the specific data being collected; each option carries documented, measurable trade-offs.
Practical application
Practical application
- The collection method should be matched to the research timeline. Established retrospective or observational designs should be used where EHRs are well validated; for prospective uses, EHR-derived endpoints should be validated against a parallel data source or an adjudication step, as recommended by Cowie et al.
- A coded field should not be assumed to be more current than the note from which it was derived. The timing of coding relative to the clinical event should be checked; Cowie et al. note that this lag can extend from weeks to months, which is relevant to any time-sensitive analysis.
- NLP extraction should be limited to validated applications rather than those that appear feasible. Wieland-Jorna et al. found that most NLP studies addressed only one or two specific data elements, with performance that varied by dataset; an NLP approach should be piloted and validated on the study data rather than assuming that published performance transfers.
- All five data-quality dimensions should be assessed, not completeness alone. Weiskopf and Weng found that currency was checked in only 4 of 95 reviewed studies despite being one of five core dimensions; a currency check should be built explicitly into the protocol rather than relying on a completeness-only audit.
Frequently asked questions
Not necessarily. Cowie et al. note that structured, coded fields often originate as free text coded later for billing or registry purposes, and that coding can lag the clinical event by weeks or months; a ‘structured’ field is therefore not necessarily more current or complete than the narrative note on which it is based.
Wieland-Jorna et al.’s 2024 systematic review found that NLP systems are valuable for extracting this kind of unstructured EHR data, but performance varied by dataset and research question, and two-thirds of the reviewed studies covered only one or two specific activities. NLP should therefore be treated as a tool to be validated for each use case rather than as a universal replacement.
Weiskopf and Weng’s review of 95 articles found that currency — whether a data element reflects the patient’s true state at the relevant time — was assessed in only 4 studies, far less often than completeness (61) or correctness (57), making it the most commonly overlooked dimension.
Not always. If the de-identification standard requires the removal of exact dates, the study loses the precise timing information on which Weiskopf and Weng’s currency dimension depends; for this reason, the data-governance route and the data-quality plan should be decided together rather than separately.
Whether collection was prospective or retrospective, whether and how NLP extraction was used, and which specific data-quality dimensions (completeness, correctness, concordance, plausibility, currency) were assessed — not simply that data were drawn from the EHR.
References
- Cowie MR, Blomster JI, Curtis LH, et al. Electronic health records to facilitate clinical research. Clinical Research in Cardiology. 2017;106(1):1-9. read 10 October 2026. View on PubMed Central
- Weiskopf NG, Weng C. Methods and dimensions of electronic health record data quality assessment: enabling reuse for clinical research. Journal of the American Medical Informatics Association. 2013;20(1):144-151. read 10 October 2026. View on PubMed Central
- Wieland-Jorna Y, van Kooten D, Verheij RA, de Man Y, Francke AL, Oosterveld-Vlug MG. Natural language processing systems for extracting information from electronic health records about activities of daily living: a systematic review. JAMIA Open. 2024;7(2):ooae044. read 10 October 2026. View source
- U.S. Department of Health and Human Services, Office for Civil Rights. HIPAA Privacy Rule and research: de-identification, authorization waivers, and limited data sets. read 10 October 2026. View source
- von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP; STROBE Initiative. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Ann Intern Med. 2007;147(8):573-577. read 10 October 2026. View on EQUATOR Network
- Richesson RL, Nadkarni P. Data standards for clinical research data collection forms: current status and challenges. J Am Med Inform Assoc. 2011;18(3):341-346. read 10 October 2026. View on PubMed Central
- Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap): a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377-381. read 10 October 2026. View on PubMed
- Gregory KE, Radovinsky L. Research strategies that result in optimal data collection from the patient medical record. Appl Nurs Res. 2012;25(2):108-116. read 10 October 2026. View on PubMed Central
- Gliklich RE, Leavy MB, Dreyer NA, eds. Registries for Evaluating Patient Outcomes: A User's Guide. 4th ed. Rockville, MD: Agency for Healthcare Research and Quality; 2020. AHRQ Publication No. 19(20)-EHC020. read 10 October 2026. View source
- International Organization for Standardization. ISO/IEC 11179-1:2023 Information technology — Metadata registries (MDR) — Part 1: Framework. read 10 October 2026. View source
Every source above is a peer-reviewed article, a publisher’s record of one, or a standard issued by the body that owns it. Each carries the date it was last read. Where a standard moves, this article is revised rather than left up with a note.
How to cite this article
Pick your referencing style and copy it. Your university or target journal may vary the punctuation — check its guide if it has one.
Pubrica Academy. (2026, October 10). Research Methodologies for Adequate Data Collection from the Patient Medical Record. Pubrica. https://pubrica.com/academy/data-collection/research-methodologies-that-result-in-adequate-data-collecting-from-the-patient-medical-record/
Pubrica Academy. “Research Methodologies for Adequate Data Collection from the Patient Medical Record.” Pubrica, 10 Oct. 2026, pubrica.com/academy/data-collection/research-methodologies-that-result-in-adequate-data-collecting-from-the-patient-medical-record/.
Pubrica Academy (2026) Research Methodologies for Adequate Data Collection from the Patient Medical Record. Pubrica. Available at: https://pubrica.com/academy/data-collection/research-methodologies-that-result-in-adequate-data-collecting-from-the-patient-medical-record/ (Accessed: 15 October 2026).
Pubrica Academy. “Research Methodologies for Adequate Data Collection from the Patient Medical Record.” Pubrica. October 10, 2026. https://pubrica.com/academy/data-collection/research-methodologies-that-result-in-adequate-data-collecting-from-the-patient-medical-record/
Pubrica Academy, ‘Research Methodologies for Adequate Data Collection from the Patient Medical Record’ (Pubrica, 10 October 2026) <https://pubrica.com/academy/data-collection/research-methodologies-that-result-in-adequate-data-collecting-from-the-patient-medical-record/> accessed 15 October 2026.
Pubrica Academy. Research Methodologies for Adequate Data Collection from the Patient Medical Record. [Internet]. Pubrica; 2026 Oct 10 [cited 2026 Oct 15]. Available from: https://pubrica.com/academy/data-collection/research-methodologies-that-result-in-adequate-data-collecting-from-the-patient-medical-record/
Pubrica Academy, “Research Methodologies for Adequate Data Collection from the Patient Medical Record,” Pubrica, Oct. 10, 2026. [Online]. Available: https://pubrica.com/academy/data-collection/research-methodologies-that-result-in-adequate-data-collecting-from-the-patient-medical-record/
@misc{/la_research_methodologies_adequate_data_collecting_patient_medical_record_2026,
author = {{Pubrica Academy}},
title = {Research Methodologies for Adequate Data Collection from the Patient Medical Record},
howpublished = {Pubrica},
year = {2026},
month = {oct},
url = {https://pubrica.com/academy/data-collection/research-methodologies-that-result-in-adequate-data-collecting-from-the-patient-medical-record/},
note = {Accessed: <span data-cite-acc="iso">2026-10-15</span>}
}This article is written to be learned from. Cite it if you are referring to what it says; if you are using the evidence behind it, cite the sources listed above instead.
EHR-based data collection methodology checklist
This checklist is intended for scoping a chart-review or EHR-based research data collection plan.
[ ] Decided prospective vs retrospective collection and matched it to validated EHR use cases [ ] Checked timing lag between clinical event and field coding [ ] Scoped any NLP extraction to a validated use case, not an assumed transfer [ ] Assessed completeness, correctness, concordance, plausibility AND currency [ ] Documented data-source heterogeneity (vendor, terminology, identifier linkage) [ ] Confirmed data-governance route (de-identification, waiver, limited data set) compatible with the data-quality plan [ ] Methods section states collection type, NLP use, and data-quality dimensions assessed, per STROBE
Pubrica AcademyWritten and maintained by Pubrica’s subject matter experts. Articles are revised when the underlying standards change, and each article states the date on which it was last reviewed.
How Pubrica can help
Pubrica’s data methodologists design EHR-based data collection plans that match the method — prospective or retrospective, structured or NLP-assisted — to the requirements of the specific research question, with documented data-quality checks incorporated.