Contents
On this page
- Selection of abstractors on the basis of clinical experience
- Development and use of the coding manual
- Blinding of abstractors to hypotheses and case status
- Training abstractors to manage missing and conflicting data
- Ongoing audits as an alternative to a single reliability check
- Institutional review board status of abstractors as a data-governance issue
- Reporting abstractor reliability to permit critical appraisal
Academy · Research Data, Analytics & AI · Abstractor Training & Calibration
Optimal Data Gathering from Patient Medical Records for Research: Abstractor Selection, Training and Quality Assurance
The accuracy of data extracted from a patient medical record depends less on the record itself than on who reads it and how that reader was trained.
When a chart abstraction study produces unreliable data, the commonly suspected causes, such as a poorly designed form, an inconsistent electronic health record (EHR) or an underpowered sample, are not always the source of the problem. Gregory and Radovinsky, writing on research strategies for optimal data collection from the patient medical record, locate much of the risk more specifically: in who performs the abstraction, how abstractors are selected and trained, and how their work is checked while the study is under way.1 Their recommendations concern the person operating the tool more than the tool itself. This article examines that perspective directly, addressing abstractor selection, training against a coding manual, blinding and the ongoing audits that detect drift before it compromises a dataset, rather than the form design or technology choices covered elsewhere.
Learning objectives
- Who should be selected as a chart abstractor, and why clinical experience matters more than general research experience
- The required content of a coding manual that effectively guides abstractor behavior
- Why Gregory and Radovinsky recommend blinding abstractors to study hypotheses and, in case-control designs, to case/control status
- How periodic audits, rather than a single reliability check, are used to catch abstractor drift during a study
Common misconceptions
- “Any detail-oriented staff member can be trained to abstract charts accurately.” Gregory and Radovinsky specifically recommend selecting abstractors who are clinically experienced and interested in research, because clinical experience enables an abstractor to interpret ambiguous or conflicting documentation correctly rather than transcribing it literally.1
- “A single inter-rater reliability check at the start of a study is sufficient quality control.” Gregory and Radovinsky recommend periodic audits of data accuracy and reliability at multiple points during the study, together with ongoing monitoring and regular meetings between the principal investigator and abstractors, rather than a one-time check, because abstractor interpretation can drift over a data collection period of several months even after thorough initial training.1
Selection of abstractors on the basis of clinical experience
Gregory and Radovinsky recommend selecting abstractors who are clinically experienced, interested in research, and willing to learn the specific demands of a data collection protocol — not simply whichever staff member has spare capacity.1 The reasoning is practical: a clinically experienced abstractor can recognize when a chart's documentation is internally inconsistent or incomplete and knows what to do with that recognition, where a non-clinical abstractor is more likely to transcribe whatever text is present without flagging that it may not reflect the actual clinical picture. Before any abstraction begins, Gregory and Radovinsky also specify that abstractors must complete human subjects protection training and be formally added to the IRB-approved protocol.1
Development and use of the coding manual
Gregory and Radovinsky's central recommendation is a written coding manual, separate from the data collection tool itself, that lists every variable, explains exactly how it should be recorded on the abstraction tool, and specifies the best source location for that variable within the medical record.1 They recommend the manual be pilot tested and revised based on that test, and — importantly for multi-site studies — that site-specific versions of the manual be used where sites differ in how they document care, since a single manual written for one site's documentation conventions will not transfer cleanly to another's.1 The manual, not the abstractor's memory of a training session, is what should be consulted whenever a borderline case arises during actual data collection.
Blinding of abstractors to hypotheses and case status
Abstractors should be kept blinded to the study hypothesis, and in case-control designs specifically, to whether the chart they are reading belongs to a case or a control.1 The purpose is to prevent an abstractor who knows what result the study expects from unconsciously reading ambiguous documentation in the direction that confirms it — a risk that exists independent of how careful or well-intentioned the abstractor is, simply because knowledge of the hypothesis changes how ambiguous information gets resolved. Operationally, this means abstraction forms and chart assignment should be structured so that group membership is not visible to the person abstracting.
Training abstractors to manage missing and conflicting data
Gregory and Radovinsky include explicit abstractor training on how to handle missing and conflicting record data as a core part of preparation, rather than leaving it to individual judgment in the moment.1 They also note a related, often-overlooked risk in the mechanics of data entry itself: paper-based entry is, in their words, fraught with opportunity for error, and direct electronic entry at the point of abstraction reduces that specific risk by removing a transcription step.1 This is a narrower and more concrete recommendation than general advice to "be careful with data entry" — it is a specific argument for collecting data directly into an electronic tool rather than onto paper that is transcribed later.
Ongoing audits as an alternative to a single reliability check
Rather than a single inter-rater reliability statistic computed once near the start of a study, Gregory and Radovinsky recommend a team of two to four abstractors with periodic audits of data accuracy and reliability conducted at multiple points across the study period, combined with ongoing monitoring and regular meetings between the principal investigator and the abstraction team.1 They also observe that most chart review studies do not report inter-rater reliability at all, which they flag as a limitation worth naming explicitly in a study's own methods and limitations sections rather than leaving unaddressed.1 The practical takeaway is that abstractor quality control is a continuous activity for the duration of data collection, not a milestone that is passed once and then forgotten.
Institutional review board status of abstractors as a data-governance issue
Gregory and Radovinsky's requirement that abstractors be formally added to the IRB-approved protocol before touching a single chart connects directly to how U.S. regulators frame access to identifiable patient data for research. HHS's guidance on the HIPAA Privacy Rule and research describes an IRB or Privacy Board's authority to approve an alteration or waiver of individual patient authorization only after finding, among other things, that the research could not practicably be conducted without access to the identifiable information in question, and that a plan is in place to protect those identifiers and destroy them as early as practical.2 An abstractor who has not been added to that approved protocol is not simply out of compliance with an internal checklist — they are accessing identifiable patient information outside the specific legal basis the IRB or Privacy Board actually approved, which is a materially different problem from, say, an untrained abstractor misreading a dosage field. The two failure modes are often discussed together under "abstractor readiness," but only one of them is a data breach risk.
Reporting abstractor reliability to permit critical appraisal
Gregory and Radovinsky's observation that most chart review studies fail to report inter-rater reliability at all is echoed by the structure of the STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) checklist, the standard reporting guideline for the observational designs chart abstraction studies belong to. STROBE, developed by von Elm, Altman, Egger, Pocock, Gøtzsche and Vandenbroucke and published simultaneously in 2007 in Annals of Internal Medicine, The Lancet, BMJ and other journals, asks authors to describe the methods used to address potential sources of bias in the "Methods" section of a report.3 Abstractor training, blinding, and inter-rater reliability are exactly the kind of bias-mitigation detail that item is asking for; a methods section that states only "data were abstracted from the medical record" leaves a STROBE-literate reviewer with no way to judge whether Gregory and Radovinsky's recommendations were followed at all. Naming the number of abstractors, how they were trained, whether they were blinded, and what reliability was measured is the concrete way a chart-abstraction paper satisfies that STROBE expectation.
It is also worth separating two different "transparency" failures that often get collapsed into one complaint about chart review studies: a study can report its reliability statistic in full and still have selected, trained, or blinded its abstractors poorly, and a study can have followed every one of Gregory and Radovinsky's recommendations correctly and still fail to say so in the published methods. The first is a methodology failure; the second is a reporting failure. STROBE addresses the second, Gregory and Radovinsky's recommendations address the first, and a chart-abstraction study needs to get both right — a well-run but under-reported study and a poorly-run but well-reported one are both, in different ways, incomplete evidence for a reader trying to judge the data.
Methodological and data-standards literature underlying this objective
Gregory and Radovinsky's own recommendations sit inside a wider methodological literature worth naming directly. Gearing and colleagues' chart-review methodology paper, written for child and adolescent psychiatry but applicable far more broadly, supplies the coding-manual and inter-abstractor reliability framework that operationalizes "optimal data collection" into specific, checkable steps.4 And where structured abstraction feeds a formal registry rather than a single study, AHRQ's Registries for Evaluating Patient Outcomes guide sets out the data-element selection and governance decisions that sit above any one study's abstraction form.5
On data standards specifically, Richesson and Nadkarni's review of clinical research data collection forms found that most projects still define ad hoc variable sets rather than reusing standardized common data elements, which limits exactly the kind of cross-study comparison optimal data gathering is meant to support.6 ISO/IEC 11179, the international metadata registry standard, gives the formal definition of a properly specified data element that this recommendation is, in effect, scaling down to project level.7
Where the record being abstracted is electronic rather than paper, Weiskopf and Weng's framework for EHR data quality assessment and Cowie and colleagues' review of how electronic health records facilitate clinical research both describe the same completeness, concordance, and currency issues that "optimal" gathering from a patient medical record has to resolve before the resulting dataset can be trusted for analysis.89
Finally, on tooling and validity: Harris and colleagues' description of REDCap documents the metadata-driven electronic data capture platform many research teams use to operationalize structured abstraction with built-in validation rules.10 And Harris, McGregor and colleagues' guidance on quasi-experimental studies in medical informatics describes the validity threats that apply whenever medical-record data collected for one purpose are repurposed to evaluate an intervention, a caution relevant to any research initiative built on "optimal" record-based data gathering.11
Key points
Key points
Optimal data gathering from medical records depends on who abstracts the data and how they are trained, blinded, and audited throughout the study — not primarily on the form or the technology — and on reporting that process (IRB status, training, reliability) transparently enough for a reader to judge it.
Practical application
Practical application
- Abstractors should be selected for clinical experience rather than availability. Abstractors with a clinical background and a genuine interest in the research question should be prioritized over staff selected solely on the basis of availability.
- Institutional review board (IRB) and human-subjects training should be required before abstraction begins. Each abstractor should be confirmed as formally added to the IRB-approved protocol and as having completed the required training beforehand.
- A standalone coding manual should be developed and pilot tested. The coding manual should be kept separate from the data collection tool, tested, and adapted into site-specific versions where documentation practices differ.
- Abstractors should be blinded to the hypothesis and to group status. Chart assignment and forms should be structured so that abstractors cannot infer case/control status or the expected direction of results.
- Audits should be continuous rather than one-time. Reliability and accuracy audits should be scheduled at multiple points during the study, with regular meetings between the principal investigator (PI) and abstractors, rather than a single check at the outset.
Frequently asked questions
The rationale of Gregory and Radovinsky is that clinically experienced staff interpret ambiguous or conflicting documentation, which is the central judgement that abstraction requires, more reliably than staff without that clinical background.
Gregory and Radovinsky note that this omission is common but identify it as a limitation rather than a best practice; they recommend that inter-rater reliability be measured and reported rather than omitted in accordance with common practice in the field.
No. Electronic entry specifically reduces transcription error; it does not resolve the interpretive judgements that a coding manual is designed to standardise.
They represent different risks: the first is primarily a data-quality problem (probable misinterpretation of the chart), whereas the second is a data-governance problem, because that person is accessing identifiable patient data outside the specific basis approved by an IRB or Privacy Board.
At a minimum, the number of abstractors, the training they received, whether they were blinded to the hypothesis or group status, and the inter-rater reliability measured and the method by which disagreements were resolved should be reported, rather than only the statement that data were abstracted from the medical record.
References
- Gregory KE, Radovinsky L. Research strategies that result in optimal data collection from the patient medical record. Appl Nurs Res. 2012;25(2):108-116. read 10 October 2026. View on PubMed Central
- U.S. Department of Health and Human Services, Office for Civil Rights. HIPAA Privacy Rule and research: IRB/Privacy Board waiver and alteration of authorization. read 10 October 2026. View source
- von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP; STROBE Initiative. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Ann Intern Med. 2007;147(8):573-577. read 10 October 2026. View on EQUATOR Network
- Gearing RE, Mian IA, Barber J, Ickowicz A. A methodology for conducting retrospective chart review research in child and adolescent psychiatry. J Can Acad Child Adolesc Psychiatry. 2006;15(3):126-134. read 10 October 2026. View on PubMed Central
- Gliklich RE, Leavy MB, Dreyer NA, eds. Registries for Evaluating Patient Outcomes: A User's Guide. 4th ed. Rockville, MD: Agency for Healthcare Research and Quality; 2020. AHRQ Publication No. 19(20)-EHC020. read 10 October 2026. View source
- Richesson RL, Nadkarni P. Data standards for clinical research data collection forms: current status and challenges. J Am Med Inform Assoc. 2011;18(3):341-346. read 10 October 2026. View on PubMed Central
- International Organization for Standardization. ISO/IEC 11179-1:2023 Information technology — Metadata registries (MDR) — Part 1: Framework. read 10 October 2026. View source
- Weiskopf NG, Weng C. Methods and dimensions of electronic health record data quality assessment: enabling reuse for clinical research. Journal of the American Medical Informatics Association. 2013;20(1):144-151. read 10 October 2026. View on PubMed Central
- Cowie MR, Blomster JI, Curtis LH, et al. Electronic health records to facilitate clinical research. Clinical Research in Cardiology. 2017;106(1):1-9. read 10 October 2026. View on PubMed Central
- Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap): a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377-381. read 10 October 2026. View on PubMed
- Harris AD, McGregor JC, Perencevich EN, Furuno JP, Zhu J, Peterson DE, Finkelstein J. The use and interpretation of quasi-experimental studies in medical informatics. J Am Med Inform Assoc. 2006;13(1):16-23. read 10 October 2026. View on PubMed Central
Every source above is a peer-reviewed article, a publisher’s record of one, or a standard issued by the body that owns it. Each carries the date it was last read. Where a standard moves, this article is revised rather than left up with a note.
How to cite this article
Pick your referencing style and copy it. Your university or target journal may vary the punctuation — check its guide if it has one.
Pubrica Academy. (2026, October 10). Optimal Data Gathering from Patient Medical Records for Research: Abstractor Selection, Training and Quality Assurance. Pubrica. https://pubrica.com/academy/concepts-definitions/optimal-data-gathering-from-the-patient-medical-record-is-the-goal-of-research-initiatives/
Pubrica Academy. “Optimal Data Gathering from Patient Medical Records for Research: Abstractor Selection, Training and Quality Assurance.” Pubrica, 10 Oct. 2026, pubrica.com/academy/concepts-definitions/optimal-data-gathering-from-the-patient-medical-record-is-the-goal-of-research-initiatives/.
Pubrica Academy (2026) Optimal Data Gathering from Patient Medical Records for Research: Abstractor Selection, Training and Quality Assurance. Pubrica. Available at: https://pubrica.com/academy/concepts-definitions/optimal-data-gathering-from-the-patient-medical-record-is-the-goal-of-research-initiatives/ (Accessed: 15 October 2026).
Pubrica Academy. “Optimal Data Gathering from Patient Medical Records for Research: Abstractor Selection, Training and Quality Assurance.” Pubrica. October 10, 2026. https://pubrica.com/academy/concepts-definitions/optimal-data-gathering-from-the-patient-medical-record-is-the-goal-of-research-initiatives/
Pubrica Academy, ‘Optimal Data Gathering from Patient Medical Records for Research: Abstractor Selection, Training and Quality Assurance’ (Pubrica, 10 October 2026) <https://pubrica.com/academy/concepts-definitions/optimal-data-gathering-from-the-patient-medical-record-is-the-goal-of-research-initiatives/> accessed 15 October 2026.
Pubrica Academy. Optimal Data Gathering from Patient Medical Records for Research: Abstractor Selection, Training and Quality Assurance. [Internet]. Pubrica; 2026 Oct 10 [cited 2026 Oct 15]. Available from: https://pubrica.com/academy/concepts-definitions/optimal-data-gathering-from-the-patient-medical-record-is-the-goal-of-research-initiatives/
Pubrica Academy, “Optimal Data Gathering from Patient Medical Records for Research: Abstractor Selection, Training and Quality Assurance,” Pubrica, Oct. 10, 2026. [Online]. Available: https://pubrica.com/academy/concepts-definitions/optimal-data-gathering-from-the-patient-medical-record-is-the-goal-of-research-initiatives/
@misc{/la_optimal_data_gathering_from_the_patient_medical_record_is_the_goal_of_research_initiatives_2026,
author = {{Pubrica Academy}},
title = {Optimal Data Gathering from Patient Medical Records for Research: Abstractor Selection, Training and Quality Assurance},
howpublished = {Pubrica},
year = {2026},
month = {oct},
url = {https://pubrica.com/academy/concepts-definitions/optimal-data-gathering-from-the-patient-medical-record-is-the-goal-of-research-initiatives/},
note = {Accessed: <span data-cite-acc="iso">2026-10-15</span>}
}This article is written to be learned from. Cite it if you are referring to what it says; if you are using the evidence behind it, cite the sources listed above instead.
Abstractor training and calibration checklist
These items should be confirmed before and during chart abstraction.
[ ] Abstractors selected for clinical experience and research interest [ ] Human subjects protection training completed and documented [ ] Abstractors formally added to IRB-approved protocol [ ] Standalone coding manual written, pilot tested, and revised [ ] Site-specific coding manual versions created for multi-site studies [ ] Abstractors blinded to hypothesis and, in case-control designs, to group status [ ] Training provided specifically on handling missing/conflicting data [ ] Periodic audits of accuracy/reliability scheduled across the study period [ ] Regular PI-abstractor meetings scheduled [ ] Methods section states abstractor count, training, blinding, and reliability per STROBE
Pubrica AcademyWritten and maintained by Pubrica’s subject matter experts. Articles are revised when the underlying standards change, and each article states the date on which it was last reviewed.
How Pubrica can help
Pubrica trains and calibrates chart abstraction teams, builds coding manuals, and designs ongoing reliability audits for medical record-based research studies.