Free scoping read. A written assessment of your manuscript in two business days, at no cost.

Start one
Contents

Academy · Research Data Analytics & AI · Data collection challenges

Existing Challenges in Medical Data Collection Processes

Medical data collection fails in a small number of predictable ways: the burden falls on staff with little time to bear it, variables are not validated against any reference, missingness is informative, codes do not mean what the analysis assumes, and records are dispersed across systems that cannot interpret one another. Each of these problems has been measured, and each requires a different remedy.

Pubrica AcademyWritten by the medical writing and biostatistics teams Reviewed byPubrica biostatistics team Published9 October 2026 Last checked9 October 2026 Reading time15 minutes Sources40 peer-reviewed

The phrase “data collection challenges” usually introduces a list of complaints. It is more useful as a diagnostic category, because the five or six distinct failure modes have different causes and different remedies, and projects routinely apply the wrong one, for example retraining staff to address a terminology problem, or imputing values to address an ascertainment problem. This article sets out what has been measured about each failure mode, including areas in which the measurement literature is itself limited.

Learning objectives

  • Separate the distinct failure modes in medical data collection rather than treating them as one problem
  • Quantify the collection burden at the point of care, in both high-income and low-resource settings
  • Validate a variable against a reference standard, and recognise the validation statistics that result
  • Recognise informative missingness and avoid the analysis choices that convert it into bias
  • Account for terminology coverage and mapping loss as part of your method rather than as preprocessing
  • Write a data collection specification that another team could execute identically

Common misconceptions

  • “Routinely collected data is free data.” Collection is paid for by someone, usually a clinician. In a time and motion study, electronic health record (EHR) and desk work consumed 49.2% of the physician office day and 37.0% of time even within the examination room.
  • “Coded diagnoses are accurate enough for research.” Accuracy varies by condition, often considerably. Among conditions validated by medical chart review, the median positive predictive value of ICD-9 coding was 85%, with a range of 39-100%, and validation statistics existed for only 51 of 81 conditions examined.
  • “There is a standard way to assess data quality.” There is not. Weiskopf and Weng identified five dimensions of EHR data quality — completeness, correctness, concordance, plausibility and currency — alongside seven broad categories of assessment method, with little consistency or potential generalisability across studies.
  • “Digitising collection removes the paperwork burden.” Not by itself. Across five low- and lower-middle income countries, the average primary care facility held 34 registers and 35 monthly reporting forms, with recording taking about one-third of total consultation time and monthly reporting about 9 hours per health worker per month.
  • “De-identified data carries no re-identification risk.” Risk scales with cohort size. For commonly used privacy-preserving linkage encodings, re-identification risk rose from under 1% for datasets below one million individuals to roughly 10-20% at 250 million.

The burden of data collection and who bears it

Every data field has a cost, and the cost is usually borne by a clinician in a consultation. Sinsky, Colligan, Li and colleagues, in the Annals of Internal Medicine, measured it directly: EHR and desk work consumed 49.2% of the physician office day, and 37.0% of time even inside the examination room 23. This represents the upper limit of what can be required of point-of-care collection in a high-income setting, and it is already being exceeded.

In lower-resource settings the burden is structurally similar and differently distributed. Siyam, Ir, York, Antwi, Amponsah and colleagues, in BMC Health Services Research, surveyed primary care facilities across five low- and lower-middle income countries: the average facility held 34 registers, ranging from 16 to 48, and 35 monthly reporting forms, ranging from 19 to 52 16,19. Only 77% of registers and 78% of forms could be verified as actually in use. Mean time spent on recording was about one-third of total consultation time, and completing monthly reporting forms took an estimated 9 hours per health worker per month, ranging from 4 to 15 hours.

Two consequences follow. First, roughly a quarter of the collection instruments in use could not be verified as in use at all, which indicates that a meaningful share of the burden produces no output 16. An audit of which forms are actually read is a less costly intervention than any data quality programme. Second, a new study-specific collection instrument is added to this existing workload; a case report form that takes ten minutes per patient competes with a documented existing load equivalent to one-third of consultation time.

One commonly asserted driver of collection behaviour deserves correction. Adler-Milstein and Jha, in Health Affairs, examined whether hospitals used new electronic records to increase reimbursement: Medicare payment per discharge rose by $849 among 393 hospitals that newly adopted a basic EHR versus $945 among 782 non-adopting controls, a difference-in-difference of $96 that was not statistically significant, with a p-value of 0.673 31. The up-coding hypothesis was not supported in that analysis, which should be taken into account before it is repeated 31.

The paper and recording burden in primary care facilities across five low- and lower-middle income countries

MeasureMeanRange
Registers held per facility3416 to 48
Monthly reporting forms per facility3519 to 52
Registers verified as actually in use77%—
Reporting forms verified as actually in use78%—
Time spent on recording, as a share of total consultation timeabout one-third—
Hours per health worker per month completing monthly reporting forms94 to 15

Siyam A, Ir P, York D, Antwi J, Amponsah F, Rambique O, Funzamo C, Azeez A, Mboera L, Kumalija CJ, Rumisha SF, Mremi I and colleagues. BMC Health Services Research. Roughly a quarter of instruments in use could not be verified as in use, which indicates that a meaningful share of the recorded burden produces no output.

Five dimensions of data quality without an agreed assessment method

The most useful single framework in this literature is that of Weiskopf and Weng, in the Journal of the American Medical Informatics Association. They identified five dimensions of EHR data quality — completeness, correctness, concordance, plausibility and currency — alongside seven broad categories of assessment method, and found little consistency or potential generalisability across studies 33. The second part of that finding has the greater practical consequence: there is no standard assessment that can simply be cited.

The size of the surrounding literature provides less assistance than might be expected. An, Lim and Lee, in the Journal of Medical Internet Research, conducted a PubMed search retrieving 82,346 articles, narrowed to 851 after title and abstract review, with 105 papers reviewed on clinical data quality management across the data life cycle 4. The narrowing of more than eighty thousand articles to approximately one hundred usable ones suggests that much of this literature is descriptive rather than methodological 4.

In practical terms, “we assessed data quality” is not a method statement. Authors should name the dimension assessed, the method used, the threshold applied and the handling of records that failed it. If only completeness was assessed — the most common case — this should be stated, together with the fact that correctness and concordance were not assessed. The page on handling discrepancies in collected data covers the resolution process.

The data collection and quality section of a study protocol

Not executable by anyone else

Data will be collected from the electronic health records of participating sites using a standardised extraction template. Data quality will be assessed and any discrepancies resolved by the study team. Variables will be mapped to a common data model. Missing data will be handled using multiple imputation. Patient identifiers will be removed prior to analysis in accordance with applicable regulations.

“Standardised extraction template” across different vendor systems. The template can be standardised while the underlying fields are not, which produces apparent differences between patients that in fact reflect differences between systems.

“Data quality will be assessed” names no dimension and no method. With five recognised dimensions and no agreed assessment standard, this sentence specifies no commitment.

Imputation without a stated mechanism. Multiple imputation under a missing-at-random assumption is probably inappropriate for routine clinical data, in which a value exists because a clinician requested it.

Mapping with no rate. The proportion of records that failed to map, and what happened to them, is a selection step inside preprocessing.

“In accordance with applicable regulations” is not a de-identification method. The reviewer requires two facts: which of the two permitted methods was used, and at what cohort size the risk was assessed.

Executable, and its weaknesses are visible

Extraction specifications are given per site in Appendix 1, because the three sites run two vendor systems and the structured fields are not equivalent; the same named field was confirmed to carry different content at two of the sites, and the variable definitions differ accordingly. Data quality assessment is specified by dimension rather than in general: we assess completeness for all variables, plausibility through the range and cross-field rules listed in Appendix 2, and concordance for the subset of variables available from two independent sources. Correctness is assessed only for exposure and outcome, through blinded chart review on a random sample, with sensitivity and positive predictive value to be reported; we state here that correctness is not assessed for the remaining covariates. Missingness will be reported by variable, site and period before any imputation, and the imputation model is specified in Appendix 3 together with the assumed mechanism; because missingness in laboratory variables in these systems is known not to be at random, a sensitivity analysis under a plausible alternative mechanism is pre-specified. Mapping to the common data model will report the model version, the vocabulary release, the achieved mapping rate and the disposition of unmapped records; nursing flowsheet variables are excluded in advance because standard terminology coverage of flowsheet concepts and values is incomplete. De-identification follows the Safe Harbor method with all eighteen identifier categories removed, and the re-identification risk assessment is documented at the planned cohort size, to be repeated if the cohort is extended.

Per-site specifications make the non-equivalence of fields explicit and reportable instead of silently pooling it.

Quality assessment is specified dimension by dimension, including an explicit statement of which dimension is not assessed for which variables.

Correctness is measured on a sample for the variables that carry the result, converting the usual assumption into a reported property.

The imputation mechanism is stated and a sensitivity analysis under an alternative is pre-specified, which is the appropriate response to known informative missingness.

The mapping rate and the fate of unmapped records are reportable outputs, and a poorly covered variable class is excluded in advance with the reason given.

The de-identification method is named and the risk assessment is tied to a cohort size, with a trigger for repeating it.

The first version could describe almost any study, which is why a reviewer cannot evaluate it. The second can be executed by a team not involved in its preparation, and each weakness is stated rather than concealed — which is the only durable way to handle data collected for another purpose.

Validation against a reference standard and its findings

A variable derived from routine data is a measurement with properties, and those properties are knowable. The studies that have measured them give a consistent picture: specificity and negative predictive value are high, sensitivity and positive predictive value are variable and sometimes poor 1,17.

Campbell, Tyler, Shuttleworth, Pearce and colleagues, in JAMIA Open, examined studies validating administrative data against clinical records and reported interquartile ranges of 44.0-91.4 for sensitivity and 59.4-91.1 for positive predictive value, while specificity ranged 90.0-99.6 and negative predictive value 90.0-99.5 1. The asymmetry reflects how administrative coding works: if a code is present the condition is usually present, whereas the absence of a code is considerably less informative than researchers often assume.

Wei, Luster, Chan and Min, in BMC Health Services Research, found that among conditions validated by medical chart review, the median positive predictive value of ICD-9 coding was 85% with a range of 39-100%, and median negative predictive value 91% with a range of 41-100% 10,12,17. The more important finding in that paper is the denominator: validation statistics existed for only 51 of 81 conditions. For more than a third of the conditions examined, the performance of the coding has not been established 17.

Shaw, Harron, Pescarini and colleagues cover the biases arising from linked administrative data, and Gianfrancesco and Goldstein’s narrative review addresses the validity of EHR-based epidemiological research 11,13. The practical rule is clear: the exposure and the outcome should be validated on a sample against chart review, sensitivity and positive predictive value reported, and, if the condition lacks published validation, this should be stated in the limitations rather than a figure borrowed from a different condition.

What validation studies find when routine data variables are checked against clinical records

MeasureReported valueWhat it means for your analysis
Sensitivity, administrative data against clinical recordsInterquartile range 44.0-91.4Case ascertainment is the weak point. A substantial share of true cases may carry no code
Positive predictive value, administrative dataInterquartile range 59.4-91.1A present code is usually but not always correct; the lower quartile is poor enough to matter
Specificity90.0-99.6Absence of the condition is reliably reflected by absence of a code
Negative predictive value90.0-99.5High, which is why the sensitivity problem is easy to overlook in a low-prevalence condition
Median positive predictive value, ICD-9 coding against chart review85% (range 39-100%)The range is the key consideration. A condition-specific figure is required; a median is not a substitute
Median negative predictive value, ICD-991% (range 41-100%)Also condition-dependent, with a lower bound that undermines any blanket assumption
Conditions with any published validation statistics51 of 81For over a third of conditions examined, the performance of the coding is unknown

Administrative data interquartile ranges from Campbell AC, Tyler J, Shuttleworth RR, Pearce LA and colleagues, JAMIA Open. ICD-9 figures and the 51 of 81 denominator from Wei MY, Luster JE, Chan CL, Min L, BMC Health Services Research.

Informative missingness

In purpose-built research data collection, a missing value usually means something went wrong. In routine clinical data it usually means a clinician decided the measurement was not needed — which is a statement about the patient 20,34. Treating the two in the same way is the most common analytical error in this field.

Li, Yan, Chaudhary, Avula and colleagues, in npj Digital Medicine, showed that the pattern of missingness in EHR laboratory variables was not at random, and that a multi-level imputation algorithm produced smaller imputation error than a cross-sectional method — the mechanism being that a patient’s own earlier and later values carry information that a cross-sectional model ignores 15. Groenwold’s treatment of informative missingness in these systems makes the conceptual point: the fact that a value is known is itself data 20.

Three consequences follow for design. Complete-case analysis selects on care intensity, so it systematically over-represents patients who were investigated more thoroughly, who are usually sicker or more closely followed 15,24. Imputation under a missing-at-random assumption is often biased in a predictable direction, so the assumption should be stated and a sensitivity analysis conducted under a plausible alternative. A variable with very high missingness is often better treated as a binary indicator of whether the test was performed, since that indicator may be the clinically meaningful quantity.

Coding, terminology mapping and unmapped data

Standard terminologies are the mechanism by which collected data becomes comparable, and their coverage is uneven in ways that determine which research questions are answerable.

Chang and Mostafa, in the Journal of the American Medical Informatics Association, reviewed 651 articles on SNOMED CT use published between October 2020 and June 2025, with 188 from the United States and 92 from the United Kingdom 3. The most prominent use case was retrieving patient data from national and commercial registries, at 186 articles. Implementation accounted for 26.4% of the literature and evaluation of the terminology as a commodity for 32.0%. This distribution indicates where practical knowledge is concentrated — registry retrieval — and, by implication, where it is not 3.

Coverage is markedly incomplete for some data classes. Austin, Lalich, Stewart, Zarbano and colleagues, in JAMIA Open, found only 65.5% of nursing flowsheet concepts (n=1,170) and 56.0% of flowsheet values (n=1,831) could be mapped to SNOMED CT and LOINC target codes 5. Roughly a third of nursing-recorded concepts and nearly half of the recorded values have no standard representation, which means that a study depending on them relies on local, non-portable definitions 5.

In contrast, Oja, Tamm, Mooses and colleagues converted Estonian national health data to the OMOP common data model and achieved an average mapping rate of 95% across more than 100 million transformed entries 10. Kent, Burn, Dawoud and colleagues discuss common data model solutions to common problems. The gap between 95% and 56% is not a difference in diligence; it is a difference in which data classes have mature standard representations 5,10.

The mapping rate should be reported as a result. A study stating “data were mapped to a common data model” without a rate and without the disposition of unmapped records has concealed a selection step within preprocessing.

Fragmentation and loss of meaning across systems

A patient’s record is rarely in one place, and moving it between places loses meaning as well as risking loss of content. About 4.5% of expenditure-weighted individual Medicare beneficiaries had meaningful-use records associated with a single EHR vendor, while about 19.8% had records stored across 8 or more vendors 38. Concentration is therefore the exception, and wide dispersion is common.

The semantic consequence has been measured: only 22% of data was understood when exchanged between different EHR vendors, against 68% between sites sharing a vendor 38. Walker, Tarver, Jonnalagadda, Ranbom, Ford and Rahurkar, in JMIR Medical Informatics, examine the challenges and opportunities for interoperability from the stakeholder side and report, citing other sources, that only 45% of US hospitals engaged in all four core interoperability elements despite near-universal certified EHR implementation.

For a data collection plan, this has a specific and often overlooked implication. If a cohort is assembled from multiple systems, variable definitions must be site-specific and between-site comparability must be demonstrated rather than assumed — because a field with the same name in two vendor systems commonly contains different content, and in a pooled analysis that difference appears as a patient characteristic 9,39. Our page on the limitations of digital health records covers the interoperability evidence in detail.

4.5%of expenditure-weighted Medicare beneficiaries had meaningful-use records at a single EHR vendor
19.8%had records stored across 8 or more vendors
68% against 22%of data understood when exchanged within one vendor, against between different vendors

Fragmentation and semantic loss. Vendor distribution and comprehension figures as reported in the cited analysis of meaningful-use record distribution and cross-vendor exchange across oncology sites. A field with the same name in two vendor systems commonly carries different content.

Social determinants data as a concentrated problem area

Social and economic variables are the data class most in demand and least reliably collected, and the pattern of their failure is specific rather than general. Cook, Sachs and Weiskopf, in the Journal of the American Medical Informatics Association, systematically reviewed the quality of social determinants data in the electronic health record: plausibility was the most commonly represented quality issue, in 31 articles (41%), and 47 articles (62%) linked bias or validity problems to data quality issues, with misclassification addressed in 23 articles (30%) 12.

The predominance of plausibility problems is informative. It indicates that the recorded values frequently do not make sense — implausible combinations, out-of-range entries, values that cannot be true given other fields — which suggests collection under time pressure into fields with no validation and no accountable owner, rather than a conceptual difficulty in defining the variables 12.

The implication for research is unwelcome but clear. If a social determinant is central to the research question, prospective collection with a validated instrument is usually necessary, and retrospective extraction from routine fields should be treated as a measurement with unknown and probably poor properties. Where the extracted variable is nevertheless used, the plausibility screen should be reported: how many records had impossible values, and how they were handled.

De-identification, linkage and scaling re-identification risk

The governance challenge in data collection is usually framed as a compliance task, which obscures the quantitative part. Under HIPAA, de-identification rests on two permitted methods — Expert Determination and Safe Harbor — with the Safe Harbor method requiring removal of 18 categories of identifier 35. That is a defined procedure, and meeting it is not the same as achieving a low re-identification risk 2.

Eliazar, Brown, Cinamon, Kantarcioglu and Malin, in the Journal of the American Medical Informatics Association, examined re-identification risk for commonly used privacy-preserving linkage encodings and found it rose from under 1% for datasets below one million individuals to roughly 10-20% at 250 million individuals 2,19. The relationship with cohort size is the principal finding: a linkage approach that is safe for a regional study is not necessarily safe for a national one, and the risk assessment must be repeated when the cohort grows rather than inherited from the pilot.

For a collection plan, this requires recording which de-identification method was applied, stating the cohort size at which the risk assessment was performed, and re-examining it if the scale changes. The compliance and data management plan pages cover the documentation, and our page on blockchain for biomedical data sharing covers one much-discussed and still largely unproven architectural response.

Routine health information systems in low- and middle-income settings

The challenges in low- and middle-income settings are not simply a less-resourced version of the same problem, and the literature addressing them is small. Lemma, Janson, Persson, Wickremasinghe and colleagues, in PLOS ONE, scoped the field and identified 20 articles on data quality and 16 on data use in routine health information systems in low- and middle-income countries 2,19. This is a very small evidence base for a question affecting the majority of the world’s health systems, and it should be described as such rather than summarised as though the question were settled 19.

The survey by Siyam and colleagues adds a description of the structure of the burden: 34 registers and 35 monthly reporting forms on average per facility, roughly a quarter of which could not be verified as in use, consuming about a third of consultation time for recording plus around 9 hours per health worker per month for reporting 16. The separation of data quality from data use in the Lemma review is also notable: the two literatures barely overlap, which suggests that collected data frequently goes unused, and that improving quality without addressing use may simply increase the cost of data that are never read 19.

The practical implication for those designing collection in these settings is that the binding constraint is usually the proliferation of instruments rather than their design. Removing a form that is not read yields a greater improvement than redesigning one that is.

Key points

Key points

Medical data collection fails in distinguishable ways, and the remedies are not interchangeable. The collection burden is real and already at its limit: EHR and desk work takes 49.2% of the physician office day and 37.0% of time in the examination room, while primary care facilities across five low- and lower-middle income countries hold an average of 34 registers and 35 reporting forms, about a quarter of which cannot be verified as in use, consuming roughly a third of consultation time plus around 9 hours per worker per month. Data quality has five dimensions (completeness, correctness, concordance, plausibility and currency) and no agreed assessment method, so the method used should be stated. Exposure and outcome should be validated: comparisons of administrative data with clinical records show interquartile sensitivity of 44.0-91.4 and positive predictive value of 59.4-91.1, with a median ICD-9 positive predictive value of 85% but a range of 39-100% and validation statistics available for only 51 of 81 conditions. Missingness should be treated as informative. The mapping rate should be reported, noting that only 65.5% of nursing flowsheet concepts and 56.0% of values map to SNOMED CT and LOINC, while national-scale OMOP conversion has reached 95%. Finally, the re-identification risk assessment should be repeated when the cohort grows, since risk rose from under 1% below a million individuals to roughly 10-20% at 250 million.

Application to your study

Application to your study

  1. Before a collection instrument is added, existing instruments should be audited to establish which are actually read. About a quarter of registers and forms in the five-country survey could not be verified as in use; removing an unread form is preferable to redesigning one that is read.
  2. The quality dimension and the assessment method should be named. Completeness, correctness, concordance, plausibility and currency: the report should state which are assessed, how and at what threshold, and which are not assessed.
  3. Exposure and outcome should be validated against chart review in a sample. Sensitivity and positive predictive value should be reported. If no published validation exists for the condition, this should be stated rather than a figure imported from another condition.
  4. Missingness should be reported before imputation, and its mechanism stated. A sensitivity analysis under a plausible alternative mechanism should be pre-specified, because missing at random is usually an inappropriate assumption in routine data.
  5. Very high missingness should be treated as a variable in its own right. Whether a test was ordered is often the clinically meaningful quantity, and modelling it directly is preferable to imputing the value.
  6. Variables should be defined per site when systems differ. Whether an identically named field carries the same content should be confirmed empirically, and the comparison reported rather than data pooled on assumption.
  7. The re-identification risk assessment should be linked to a specified cohort size. The cohort size at which it was conducted, and the trigger for repeating it if the cohort grows, should be documented.

Frequently asked questions

Six recur, and they have different remedies. Burden: collection consumes clinician time that is already fully committed. Quality: five recognised dimensions — completeness, correctness, concordance, plausibility, currency — with no agreed assessment method. Validity: derived variables have measurable sensitivity and predictive value that are frequently poor and frequently unmeasured. Missingness: in routine data it carries information about the patient rather than occurring at random. Terminology: coverage is incomplete for whole data classes, so some content has no portable representation. Fragmentation: records are distributed across systems with poor mutual interpretability. Projects most often fail by applying the wrong remedy, such as retraining staff to address a terminology problem.

Sufficiently accurate to be useful, but not sufficiently accurate to be assumed correct. Campbell and colleagues examined studies validating administrative data against clinical records and reported interquartile ranges of 44.0-91.4 for sensitivity and 59.4-91.1 for positive predictive value, with specificity at 90.0-99.6 and negative predictive value at 90.0-99.5. Wei, Luster, Chan and Min found a median positive predictive value for ICD-9 coding of 85% against chart review, with a range of 39-100%, and a median negative predictive value of 91%, range 41-100%. The most important figure in that paper is that validation statistics existed for only 51 of 81 conditions — for over a third, performance is unknown.

No, and a claim to the contrary is likely to attract criticism from reviewers. Weiskopf and Weng identified five dimensions of EHR data quality — completeness, correctness, concordance, plausibility and currency — alongside seven broad categories of assessment method, with little consistency or potential generalisability across studies. An, Lim and Lee retrieved 82,346 articles on clinical data quality, narrowed to 851 after screening, and reviewed 105 on data quality management across the life cycle; the attrition indicates how much of the literature is descriptive rather than methodological. The appropriate response is to specify the assessment by dimension, method, threshold and action, and to state which dimensions were not assessed.

Often, but not under the default assumption. In routine clinical data a value typically exists because a clinician decided the measurement was needed, so its presence carries information about the patient — which Groenwold describes as informative missingness, the curse of knowing. Li, Yan, Chaudhary, Avula and colleagues showed the pattern of missingness in EHR laboratory variables was not at random, and that a multi-level imputation algorithm using a patient’s own earlier and later values produced smaller imputation error than a cross-sectional approach. The assumed mechanism should therefore be stated, a sensitivity analysis under a plausible alternative pre-specified, and consideration given to whether the indicator of whether a test was performed is itself the variable of interest.

It remains local and non-portable, which limits the questions that can be addressed. Austin, Lalich, Stewart and colleagues found only 65.5% of nursing flowsheet concepts (n=1,170) and 56.0% of flowsheet values (n=1,831) could be mapped to SNOMED CT and LOINC target codes — so roughly a third of nursing-recorded concepts and nearly half the values have no standard representation. By contrast, Oja and colleagues achieved an average 95% mapping rate converting Estonian national health data to the OMOP common data model across more than 100 million entries. The difference reflects which data classes have mature standard representations, not differences in effort. The mapping rate and the disposition of unmapped records should be reported.

Because of how they are collected rather than how they are defined. Cook, Sachs and Weiskopf systematically reviewed the quality of social determinants data in the electronic health record and found plausibility the most commonly represented quality issue, in 31 articles (41%), with 47 articles (62%) linking bias or validity problems to data quality issues and 23 (30%) addressing misclassification. The predominance of plausibility problems indicates that the recorded values frequently cannot be true — impossible combinations, out-of-range entries — which suggests entry under time pressure into unvalidated fields with no accountable owner. If a social determinant is central to the research question, prospective collection with a validated instrument is usually necessary.

The answer depends on scale, which is why a single answer is misleading. Under HIPAA, de-identification rests on two permitted methods — Expert Determination and Safe Harbor — with Safe Harbor requiring removal of 18 categories of identifier. Compliance with that procedure is not equivalent to achieving low risk. Eliazar, Brown, Cinamon, Kantarcioglu and Malin examined commonly used privacy-preserving record linkage encodings and found re-identification risk rising from under 1% for datasets below one million individuals to roughly 10-20% at 250 million. The operational implication is that a risk assessment is specific to a cohort size: an approach safe for a regional study may not be safe for a national one.

The binding constraint is usually the proliferation of instruments rather than their design, and the evidence base is small. Siyam and colleagues surveyed primary care facilities across five low- and lower-middle income countries and found an average of 34 registers (range 16 to 48) and 35 monthly reporting forms (range 19 to 52), with only 77% of registers and 78% of forms verifiable as actually in use; recording took about a third of consultation time and monthly reporting about 9 hours per health worker per month (range 4 to 15). The scoping review by Lemma and colleagues identified only 20 articles on data quality and 16 on data use in routine health information systems in these settings — a very limited literature, and the near-absence of overlap between the two suggests that collected data often go unread.

References

  1. Campbell AC, Tyler J, Shuttleworth RR, Pearce LA, Heerde JA, Borschmann R, Sawyer SM, Preen DB, Kinner SA, Calais-Ferreira L. Accuracy of administrative data in ascertaining health conditions: a systematic review. JAMIA Open. 2026;9(3):ooag109. doi:10.1093/jamiaopen/ooag109 read 9 October 2026. doi.org/10.1093/jamiaopen/ooag109
  2. Eliazar A, Brown JT, Cinamon S, Kantarcioglu M, Malin B. Re-identification risk for common privacy preserving patient matching strategies when shared with de-identified demographics. Journal of the American Medical Informatics Association. 2026;33(2):336-346. doi:10.1093/jamia/ocaf183 read 9 October 2026. doi.org/10.1093/jamia/ocaf183
  3. Chang E, Mostafa J. Use of SNOMED CT, 2020-2025: literature review. Journal of the American Medical Informatics Association. 2026;33(9):1798-1810. doi:10.1093/jamia/ocag094 read 9 October 2026. doi.org/10.1093/jamia/ocag094
  4. An D, Lim M, Lee S. Challenges for Data Quality in the Clinical Data Life Cycle: Systematic Review. Journal of Medical Internet Research. 2025;27(1):e60709. doi:10.2196/60709 read 9 October 2026. doi.org/10.2196/60709
  5. Austin R, Lalich MB, Stewart K, Zarbano J, Byrne M, Pinto MD, Umberfield EE. Exploring common data model coverage of nursing flowsheet data: a pilot study using SNOMED CT and LOINC mapping. JAMIA Open. 2025;8(6):ooaf168. doi:10.1093/jamiaopen/ooaf168 read 9 October 2026. doi.org/10.1093/jamiaopen/ooaf168
  6. Cheng AC, Shyr C, Lewis A, Delacqua F, Bosler T, Banasiewicz MK, Taylor R, Lindsell CJ, Harris PA. Opportunities, barriers, and remedies for implementing REDCap integration with electronic health records via Fast Healthcare Interoperability Resources (FHIR). JAMIA Open. 2025;8(5):ooaf111. doi:10.1093/jamiaopen/ooaf111 read 9 October 2026. doi.org/10.1093/jamiaopen/ooaf111
  7. Parker G. Future of Electronic Health Records: A Challenge to Maximize Their Utility. SSRN Electronic Journal. 2023. doi:10.2139/ssrn.4457214 read 9 October 2026. doi.org/10.2139/ssrn.4457214
  8. Tsiampalis T, Panagiotakos D. Methodological issues of the electronic health records' use in the context of epidemiological investigations, in light of missing data: a review of the recent literature. BMC Medical Research Methodology. 2023;23(1):180. doi:10.1186/s12874-023-02004-5 read 9 October 2026. doi.org/10.1186/s12874-023-02004-5
  9. Walker DM, Tarver WL, Jonnalagadda P, Ranbom L, Ford EW, Rahurkar S. Perspectives on Challenges and Opportunities for Interoperability: Findings From Key Informant Interviews With Stakeholders in Ohio. JMIR Medical Informatics. 2023;11(1):e43848. doi:10.2196/43848 read 9 October 2026. doi.org/10.2196/43848
  10. Oja M, Tamm S, Mooses K, Pajusalu M, Talvik HA, Ott A, Laht M, Malk M, et al. Transforming Estonian health data to the Observational Medical Outcomes Partnership (OMOP) Common Data Model: lessons learned. JAMIA Open. 2023;6(4):ooad100. doi:10.1093/jamiaopen/ooad100 read 9 October 2026. doi.org/10.1093/jamiaopen/ooad100
  11. Shaw RJ, Harron KL, Pescarini JM, Pereira Pinto Junior E, Allik M, Siroky AN, et al. Biases arising from linked administrative data for epidemiological research: a conceptual framework from registration to analyses. European Journal of Epidemiology. 2022;37(12):1215-1224. doi:10.1007/s10654-022-00934-w read 9 October 2026. doi.org/10.1007/s10654-022-00934-w
  12. Cook LA, Sachs J, Weiskopf NG. The quality of social determinants data in the electronic health record: a systematic review. Journal of the American Medical Informatics Association. 2022;29(1):187-196. doi:10.1093/jamia/ocab199 read 9 October 2026. doi.org/10.1093/jamia/ocab199
  13. Gianfrancesco MA, Goldstein ND. A narrative review on the validity of electronic health record-based research in epidemiology. BMC Medical Research Methodology. 2021;21(1):234. doi:10.1186/s12874-021-01416-5 read 9 October 2026. doi.org/10.1186/s12874-021-01416-5
  14. Kent S, Burn E, Dawoud D, Jonsson P, Ostby JT, Hughes N, Rijnbeek P, et al. Common Problems, Common Data Model Solutions: Evidence Generation for Health Technology Assessment. PharmacoEconomics. 2021;39(3):275-285. doi:10.1007/s40273-020-00981-9 read 9 October 2026. doi.org/10.1007/s40273-020-00981-9
  15. Li J, Yan XS, Chaudhary D, Avula V, Mudiganti S, Husby H, Shahjouei S, Afshar A, Stewart WF, Yeasin M, Zand R, et al., Abedi V. Imputation of missing values for electronic health record laboratory data. npj Digital Medicine. 2021;4(1):147. doi:10.1038/s41746-021-00518-0 read 9 October 2026. doi.org/10.1038/s41746-021-00518-0
  16. Siyam A, Ir P, York D, Antwi J, Amponsah F, Rambique O, Funzamo C, Azeez A, Mboera L, Kumalija CJ, Rumisha SF, Mremi I, Boerma T, O'Neill K. The burden of recording and reporting health data in primary health care facilities in five low- and lower-middle income countries. BMC Health Services Research. 2021;21(1):691. doi:10.1186/s12913-021-06652-5 read 9 October 2026. doi.org/10.1186/s12913-021-06652-5
  17. Wei MY, Luster JE, Chan CL, Min L. Comprehensive review of ICD-9 code accuracies to measure multimorbidity in administrative data. BMC Health Services Research. 2020;20(1):489. doi:10.1186/s12913-020-05207-4 read 9 October 2026. doi.org/10.1186/s12913-020-05207-4
  18. Peloquin D, DiMaio M, Bierer B, Barnes M. Disruptive and avoidable: GDPR challenges to secondary research uses of data. European Journal of Human Genetics. 2020;28(6):697-705. doi:10.1038/s41431-020-0596-x read 9 October 2026. doi.org/10.1038/s41431-020-0596-x
  19. Lemma S, Janson A, Persson LA, Wickremasinghe D, Kallestal C. Improving quality and use of routine health information system data in low- and middle-income countries: A scoping review. PLOS ONE. 2020;15(10):e0239683. doi:10.1371/journal.pone.0239683 read 9 October 2026. doi.org/10.1371/journal.pone.0239683
  20. Groenwold RHH. Informative missingness in electronic health record systems: the curse of knowing. Diagnostic and Prognostic Research. 2020;4(1):8. doi:10.1186/s41512-020-00077-0 read 9 October 2026. doi.org/10.1186/s41512-020-00077-0
  21. Doidge JC, Harron KL. Reflections on modern methods: linkage error bias. International Journal of Epidemiology. 2019;48(6):2050-2060. doi:10.1093/ije/dyz203 read 9 October 2026. doi.org/10.1093/ije/dyz203
  22. Gilbert R, Lafferty R, Hagger-Johnson G, Harron K, Zhang LC, Smith P, Dibben C, Goldstein H. GUILD: GUidance for Information about Linking Data sets. Journal of Public Health. 2018;40(1):191-198. doi:10.1093/pubmed/fdx037 read 9 October 2026. doi.org/10.1093/pubmed/fdx037
  23. Sinsky CA, Colligan L, Li L, et al. Allocation of Physician Time in Ambulatory Practice: A Time and Motion Study in 4 Specialties. Annals of Internal Medicine. 2016;165(11):753-760. doi:10.7326/M16-0961 read 9 October 2026. doi.org/10.7326/M16-0961
  24. Goldstein BA, Bhavsar NA, Phelan M, Pencina MJ. Controlling for Informed Presence Bias Due to the Number of Health Encounters in an Electronic Health Record. American Journal of Epidemiology. 2016;184(11):847-855. doi:10.1093/aje/kww112 read 9 October 2026. doi.org/10.1093/aje/kww112
  25. Wilkinson MD, Dumontier M, Aalbersberg IJ, et al. (53 authors; Mons B last). The FAIR Guiding Principles for scientific data management and stewardship. Scientific Data. 2016;3:160018. doi:10.1038/sdata.2016.18 read 9 October 2026. doi.org/10.1038/sdata.2016.18
  26. Adler-Milstein J, DesRoches CM, Kralovec P, Foster G, Worzala C, Charles D, Searcy T, Jha AK. Electronic Health Record Adoption in US Hospitals: Progress Continues, But Challenges Persist. Health Affairs. 2015. doi:10.1377/hlthaff.2015.0992 read 9 October 2026. doi.org/10.1377/hlthaff.2015.0992
  27. Benchimol EI, Smeeth L, Guttmann A, Harron K, Moher D, Petersen I, Sorensen HT, von Elm E, Langan SM, RECORD Working Committee. The REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) Statement. PLOS Medicine. 2015;12(10):e1001885. doi:10.1371/journal.pmed.1001885 read 9 October 2026. doi.org/10.1371/journal.pmed.1001885
  28. Furukawa MF, King J, Patel V, Hsiao CJ, Adler-Milstein J, Jha AK. Despite Substantial Progress In EHR Adoption, Health Information Exchange And Patient Engagement Remain Low In Office Settings. Health Affairs (Millwood). 2014;33(9):1672-1679. read 9 October 2026. help.senate.gov
  29. Adler-Milstein J, Jha A. Health information exchange among U.S. hospitals: who's in, who's out and why? Healthcare. 2014;2(1):26-32. read 9 October 2026. help.senate.gov
  30. Adler-Milstein J, DesRoches CM, Furukawa MF, Worzala C, Charles D, Kralovec P, Stalley S, Jha A. More than half of US hospitals have at least a basic EHR, but stage 2 criteria remain challenging for most. Health Affairs (Millwood). 2014;33(9):1664-71. read 9 October 2026. help.senate.gov
  31. Adler-Milstein J, Jha AK. No Evidence That Hospitals Are Using New Electronic Health Records to Increase Medicare Reimbursements. Health Affairs. 2014;33(7):1271. doi:10.1377/hlthaff.2014.0023 read 9 October 2026. doi.org/10.1377/hlthaff.2014.0023
  32. DesRoches CM, Charles D, Furukawa MF, et al. Adoption of Electronic Health Records Grows Rapidly, But Fewer Than Half of US Hospitals Had At Least A Basic System in 2012. Health Affairs (Millwood). 2013;32(8). read 9 October 2026. healthaffairs.org
  33. Weiskopf NG, Weng C. Methods and dimensions of electronic health record data quality assessment: enabling reuse for clinical research. Journal of the American Medical Informatics Association. 2013;20(1):144-151. doi:10.1136/amiajnl-2011-000681 read 9 October 2026. doi.org/10.1136/amiajnl-2011-000681
  34. Hripcsak G, Albers DJ. Next-generation phenotyping of electronic health records. Journal of the American Medical Informatics Association. 2013;20(1):117-121. doi:10.1136/amiajnl-2012-001145 read 9 October 2026. doi.org/10.1136/amiajnl-2012-001145
  35. Office for Civil Rights, U.S. Department of Health and Human Services. Guidance Regarding Methods for De-identification of Protected Health Information in Accordance with the Health Insurance Portability and Accountability Act (HIPAA) Privacy Rule. U.S. Department of Health and Human Services. 2012. read 9 October 2026. hhs.gov
  36. DesRoches CM, Campbell EG, Rao SR, Donelan K, Ferris T, Jha A, Kaushal R, Levy D, Rosenbaum S, Shields A, Blumenthal D. Electronic health records in ambulatory care - a national survey of physicians. New England Journal of Medicine. 2008;359(1):50-60. read 9 October 2026. help.senate.gov
  37. Ammenwerth E, Schnell-Inderst P, Machan C, Siebert U. The effect of electronic prescribing on medication errors and adverse drug events: a systematic review. Journal of the American Medical Informatics Association. 2008;15(5):585-600. read 9 October 2026. help.senate.gov
  38. Kuraitis V, McNicoll I. read 9 October 2026. e-caremanagement.com
  39. Journal of the American Medical Informatics Association. doi:10.1093/jamia/ocab289 read 9 October 2026. doi.org/10.1093/jamia/ocab289
  40. Assistant Secretary for Technology Policy (ASTP). Interoperable Exchange of Patient Health Information Among US Hospitals, 2023. read 9 October 2026. techtarget.com

Every source above is a peer-reviewed article, a publisher’s record of one, or a standard issued by the body that owns it. Each carries the date it was last read. Where a standard moves, this article is revised rather than left up with a note.

How to cite this article

Pick your referencing style and copy it. Your university or target journal may vary the punctuation — check its guide if it has one.

Pubrica Academy. (2026, October 9). Existing Challenges in Medical Data Collection Processes. Pubrica. https://pubrica.com/academy/latest-topics/what-are-the-existing-challenges-in-the-medical-data-collection-processes/

This article is written to be learned from. Cite it if you are referring to what it says; if you are using the evidence behind it, cite the sources listed above instead.

Data collection specification, reviewer-ready

This specification should be completed before extraction or enrolment. The test is whether a team not involved in the planning could execute it and obtain the same dataset.

MEDICAL DATA COLLECTION SPECIFICATION

1. BURDEN CHECK  (do this first)
   New fields being added at the point of care: ______
   Estimated minutes per patient ..............: ______
   Existing instruments that could be retired instead: ______
   [ ] Confirmed that each existing form we rely on is read
       by someone. Forms with no reader listed: ___________

2. SOURCES AND SITES
   Sites .....................................: ____________
   Systems / vendors at each site ............: ____________
   [ ] Same-named fields CONFIRMED to carry the same content
   [ ] They do not - per-site definitions in Appendix ____

3. VARIABLE DEFINITIONS  (one block per key variable)
   Variable ..................................: ____________
     Field(s) / code list ....................: ____________
     Source type: [ ] billed/coded [ ] interfaced result
                  [ ] clinician structured [ ] nurse flowsheet
                  [ ] free text [ ] prospectively collected
     Standard terminology coverage: [ ] good [ ] partial [ ] none

4. QUALITY ASSESSMENT  (by dimension - do not write "assessed")
   Completeness: variables ______  threshold ______ action ____
   Correctness .: variables ______  method ______________
     [ ] blinded chart review, sample n ____
     Reported statistics: sensitivity / PPV / other: ________
   Concordance .: variables available from 2 sources: _______
   Plausibility : range and cross-field rules in Appendix ____
   Currency ....: acceptable lag ______
   DIMENSIONS NOT ASSESSED, and for which variables: ________

5. MISSINGNESS
   Reported before imputation, by: [ ] variable [ ] site
                                   [ ] period [ ] patient group
   Assumed mechanism .........................: ____________
   Imputation model ..........................: Appendix ____
   Pre-specified sensitivity analysis under an alternative
     mechanism ...............................: ____________
   [ ] Any variable better modelled as "test done / not done"?
       which: _______________________________________________

6. MAPPING
   Common data model and version .............: ____________
   Vocabulary release ........................: ____________
   Expected / achieved mapping rate ..........: ______ %
   Disposition of unmapped records ...........: ____________
   Variable classes excluded in advance, and why: ___________

7. VALIDATION STATUS OF THE CONDITION ITSELF
   [ ] Published validation statistics exist for this condition
       in this data type - cite them
   [ ] They do not - stated as a limitation, and our own
       validation is: _______________________________________

8. GOVERNANCE AND RE-IDENTIFICATION
   De-identification method: [ ] Expert Determination
                             [ ] Safe Harbor (all 18 categories)
   Cohort size at which risk was assessed ....: ____________
   Trigger for reassessment ..................: ____________
   Linkage performed?  [ ] no  [ ] yes - encoding: __________

PHRASES TO DELETE IF PRESENT
  [ ] "a standardised extraction template" across different systems
  [ ] "data quality will be assessed"
  [ ] "discrepancies will be resolved by the study team"
  [ ] "missing data will be handled using multiple imputation"
      with no mechanism stated
  [ ] "in accordance with applicable regulations" as the whole
      de-identification method

Pubrica AcademyWritten and maintained by Pubrica’s subject matter experts. Articles are revised when the underlying standards change, and each article states the date on which it was last reviewed.

How Pubrica can help

Pubrica reviews collection specifications and protocols for the issues reviewers query: whether variable definitions are site-specific where systems differ, whether quality assessment names dimensions and methods rather than merely asserting that assessment occurred, whether exposure and outcome are validated against a reference, and whether missingness is characterised before imputation. For retrospective studies, Pubrica checks whether published validation statistics exist for the condition in the relevant data type, and states explicitly when they do not.

Find something else

Search the Pubrica Academy

Access is free and requires no account. Enter a topic, or open the full library to filter by category, sub-topic or resource type.

Pubrica reviews data collection plans to identify variables unlikely to withstand methods review.

A scoping assessment is free and takes two business days. It provides a written assessment of what the work requires and which specialist would undertake it.

Get a free quote

Regulations

Check your project against this rule

  1. 1Your work
  2. 2Your details
  3. 3Send
What do you need?

Material you send is held under signed confidentiality, with access limited to the assigned team and encrypted transfer. It is not used to train anything and is not passed to third-party tools. Ask for the agreement before you send anything you would rather cover first.