Free scoping read. A written assessment of your manuscript in two business days, at no cost.

Start one
Contents

Academy · Research Data Analytics & AI · big data and AI analytics

What Big Data Means in Health, and How Machine Learning Reads It

Health big data is defined by volume, velocity and variety — and veracity, which Raghupathi and Raghupathi were careful to describe as a goal rather than a current property. That caveat is the whole subject. Machine learning applied to large health datasets inherits every defect in how those datasets were assembled, and scale does not dilute bias; it makes small biases statistically significant.

Pubrica AcademyWritten by the medical writing and biostatistics teams Reviewed byPubrica biostatistics team Published9 October 2026 Last checked9 October 2026 Reading time14 minutes Sources50 peer-reviewed

The useful definition of health big data is not about size. It is about the fact that the data were generated for some other purpose — billing, care documentation, recruitment into a cohort — and are being reused for a question nobody had in mind when they were recorded. That is what makes harmonisation necessary, what makes counting hard, and what makes a large-sample result capable of being precisely wrong. Raghupathi and Raghupathi’s framing in Health Information Science and Systems captures it in one move: three characteristics describe the data as it is, and the fourth — veracity — describes what we would like it to be.

What you will learn

  • What the characteristics of health big data commit you to analytically
  • What the major resources actually contain, at their reported scale
  • Why a common data model is a precondition rather than an optimisation
  • Why counting patients in a federated dataset is genuinely difficult, and how to report a count
  • What machine learning adds on large health data, and what the big data paradox means for interpretation
  • How to describe a large-scale data source so a reviewer can judge what it can support

What to unlearn

  • “Big data means veracity is one of its properties.” Raghupathi and Raghupathi describe health big data as characterised by volume, velocity and variety, with veracity added as a fourth property that is a goal rather than current reality. Treating accuracy as a feature of a dataset because the dataset is large inverts the actual problem.
  • “With a large enough sample, bias stops mattering.” The opposite: a large sample narrows the confidence interval around a biased estimate. Msaouel describes this as the big data paradox in Cancer Investigation, and Verheij, Curcin, Delaney and McGilchrist catalogue the specific sources of bias in primary care electronic health record data in the Journal of Medical Internet Research. Precision is not accuracy.
  • “Patient counts in large federated studies are simple facts.” They are definitional choices, and they can disagree within a single paper. Voss and colleagues’ account of building the European Health Data & Evidence Network in the Journal of the American Medical Informatics Association describes data standardised from 25 partners across 11 European countries, and reports more than 133 million patients in the abstract against more than 67 million patient records in the results — a discrepancy worth surfacing rather than smoothing, because patients, records and encounters are different units.
  • “Machine learning finds signal that conventional analysis misses.” Sometimes, and it finds artefacts conventional analysis would also have found if anyone had looked. The MicroArray Quality Control II study, reported by Shi and colleagues in Nature Biotechnology, examined common practices for developing and validating high-dimensional predictive models and remains the clearest demonstration that modelling choices, not just data, determine the result.
  • “Integrating data sources is mostly an engineering problem.” The engineering is the visible part. The hard part is semantic: deciding that a code in one system means the same clinical thing as a different code in another. That is why the OHDSI standardised vocabularies, described by Reich and colleagues in the same journal, exist as a large-scale centralised reference ontology rather than a mapping table.

The definition, and what each characteristic costs you

Raghupathi and Raghupathi, in Health Information Science and Systems, characterise health big data by volume, velocity and variety, and add veracity as a fourth property which they describe as a goal rather than current reality 47. That ordering is the most useful thing in the definition, and it is routinely lost when the characteristics are recited as a list of four equivalent features.

Read as analytical commitments rather than adjectives, each characteristic has a cost. Volume means that statistical significance stops being informative, because everything is significant at a hundred million records; effect size, calibration and clinical relevance have to carry the interpretive load instead. Velocity means the dataset you analysed is not the dataset that exists now, so any model fitted to it is fitted to a moment. Variety means the fields were defined by different systems for different purposes, which is a semantic problem rather than a formatting one. And veracity, as a goal, means that establishing what the data mean is part of the analysis, not a preliminary to it.

Badawy, Ramadan and Hefny, in the Journal of Electrical Systems and Information Technology, survey the data sources, tools and challenges in the same terms, and Dash and colleagues, in the Journal of Big Data, cover management and analysis 4,29. Shahzad and colleagues’ 2025 systematic literature review in the same journal covered 35 peer-reviewed papers published between 2014 and 2025 on current practices, innovations and future prospects in healthcare big-data analytics — a useful indication that the methodological literature here is still modest in size relative to how often the term is used 1. Agrawal and Prabakaran, in Heredity, draw lessons specifically for general practice 24. See significant medical data collection sources in healthcare for where these data come from.

The characteristics of health big data, read as analytical commitments

CharacteristicWhat it describesWhat it costs you analytically
VolumeRecord counts into the hundreds of millionsStatistical significance becomes uninformative; effect size, calibration and clinical relevance must carry the interpretation
VelocityData accruing and changing continuouslyAny fitted model is fitted to a moment; the analysed dataset is not the live one
VarietyStructured codes, free text, images, waveforms, device streams, omicsA semantic problem, not a formatting one — fields were defined by different systems for different purposes
VeracityAccuracy and trustworthiness — described by Raghupathi and Raghupathi as a goal rather than current realityEstablishing what the data mean is part of the analysis, not a preliminary step

Characteristics as set out in Raghupathi W, Raghupathi V. Big data analytics in healthcare: promise and potential. Health Information Science and Systems. The analytical consequences are our summary, not theirs.

What health big data actually looks like

Abstraction is unhelpful here, so it is worth naming the kinds of resource and their reported scale. They differ in ways that determine what questions each can answer.

Deeply phenotyped prospective cohorts. UK Biobank, described by Bycroft and colleagues in Nature, is a prospective cohort of approximately 500,000 UK participants with genetic data available for 488,377 of them 39. The value is depth and consent for research use; the limitation is that participants volunteered, so the cohort is not a population sample.

Programme-scale sequencing with linked records. The All of Us Research Program genomics release, reported in Nature, included 245,388 clinical-grade genome sequences, with genome-wide genotyping on 312,925 participants 6. Explicit recruitment of under-represented populations is part of the design, which matters for every downstream model trained on it 6.

Routine claims and electronic health records at administrative scale. Voss and colleagues, in the Journal of the American Medical Informatics Association, applied a common data model across six disparate claims and EHR databases, the largest of which — Premier — held 100,092,900 patients 45. This is where volume becomes extreme, and also where the data were generated for reimbursement and documentation rather than research.

Federated international networks. The European Health Data & Evidence Network, described by Voss and colleagues in the same journal, standardised data from 25 partners across 11 European countries 5. Pinto Junior and colleagues extended the same approach well beyond Europe, integrating real-world data from Brazil and Pakistan into the OMOP common data model to characterise COVID-19 in the Global South — which is directly relevant to whether a model trained on North American and Western European data generalises 13.

The three resource types answer different questions. Cohorts support aetiological and genomic work with good measurement and poor representativeness; claims and EHR data support real-world effectiveness and safety with excellent coverage and poor measurement; federated networks support replication across health systems at the cost of lowest-common-denominator variables. Electronic health records and the limitations of digital health records cover the second category in detail.

Health big data resources and their reported scale

Resource typeExample and reported scaleBest suited toPrincipal weakness
Deeply phenotyped prospective cohortUK Biobank — approximately 500,000 UK participants, genetic data for 488,377 (Bycroft and colleagues, Nature)Aetiological and genomic questions with good measurementVolunteer participation, so not a population sample
Programme-scale sequencing with linked recordsAll of Us — 245,388 clinical-grade genome sequences, genome-wide genotyping on 312,925 participants (Nature)Genomic questions across a deliberately broader populationStill a recruited cohort; linkage depth varies
Routine claims and EHR data at administrative scaleSix disparate claims and EHR databases, the largest (Premier) holding 100,092,900 patients (Voss and colleagues, Journal of the American Medical Informatics Association)Real-world effectiveness, safety and utilisationGenerated for reimbursement and documentation, not research — measurement is weak
Federated international networkEHDEN — data standardised from 25 partners across 11 European countries (Voss and colleagues, same journal)Replication across health systems and populationsHarmonised variable set is narrower than any single contributing source
Federated network beyond high-income settingsReal-world data from Brazil and Pakistan integrated into the OMOP common data model (Pinto Junior and colleagues, same journal)Testing whether findings hold outside the usual data geographiesMapping quality and coverage differ substantially between partners

Scale figures exactly as reported in each publication. The counts are not comparable across rows: participants, patients and records are different units, which is the point made in the section on counting.

488,377UK Biobank participants with genetic data, of approximately 500,000 enrolled
245,388All of Us clinical-grade genome sequences
312,925All of Us participants with genome-wide genotyping

Two programme-scale genomic resources at their reported scale. UK Biobank figures from Bycroft C, Freeman C, Petkova D, et al. The UK Biobank resource with deep phenotyping and genomic data. Nature. All of Us figures from The All of Us Research Program Genomics Investigators. Genomic data in the All of Us Research Program. Nature.

Harmonisation is the precondition, not the optimisation

Nothing in the previous section can be analysed together until the records mean the same thing, and that is a harder problem than file formats. A diagnosis code in one country’s billing system and a problem-list entry in another country’s EHR may refer to the same clinical state, a narrower one, a broader one, or an administrative artefact with no clinical referent.

This is why a common data model is infrastructure rather than tidying. Voss and colleagues demonstrated feasibility by applying the model across six disparate claims and EHR databases; Reich and colleagues describe the OHDSI standardised vocabularies as a large-scale centralised reference ontology for international data harmonisation, which is the semantic layer that makes the structural layer meaningful 9,45. The extensions are informative about what remains unsolved: Park and colleagues incorporated the DICOM imaging standard into the OMOP common data model to enable multimodal research, explicitly to break data silos between imaging and structured clinical data; Künnapuu and colleagues, in JAMIA Open, built a framework for detecting temporal clinical event sequences from standardised data, because a common data model gives you comparable fields and not comparable trajectories 2,19.

Two practical points for anyone writing this up. Mapping is lossy, and the losses are not random: a variable that maps cleanly in one partner’s data and approximately in another’s introduces differential measurement error between sites, which is exactly the structure that produces spurious between-site heterogeneity. And the harmonised variable set is narrower than any contributing source, so a federated analysis answers a slightly different question from the one a single-site analysis of the richest partner could answer. State which. AI data preparation and CDISC SDTM and ADaM standards are the practical end of this work.

Counting is a definitional choice, and papers disagree with themselves

A number that looks like a simple fact about a dataset is usually the result of several decisions: whether the unit is a patient, a record, an encounter or an observation period; whether patients appearing in two contributing databases are counted once or twice; whether a minimum observation window is required; and whether the figure is at extraction, after quality filtering, or after applying eligibility criteria.

The consequence is visible in the literature. The EHDEN account by Voss and colleagues reports more than 133 million patients in the abstract and more than 67 million patient records in the results. Those are different units and may both be correct, which is precisely why the discrepancy matters: a reader cannot tell from the abstract which quantity the study is about, and a reader who takes the larger figure forward into a power calculation or a generalisability argument will be wrong by roughly a factor of two 5.

Three rules make a count usable. Define the unit explicitly in the same sentence as the number — patients, records, person-years, encounters. State the point in the pipeline at which it was taken, and give the figures at each subsequent stage, which is the same discipline a review’s flow diagram imposes. And handle overlap between contributing sources explicitly: say whether de-duplication across sources was possible, and if it was not, say that the total is a sum of sources that may share patients. None of this is onerous, and all of it is what a reviewer checks first in a large-scale observational paper. Our page on handling data discrepancies covers the reconciliation habit in a review context, and it transfers directly.

25data partners across 11 European countries
>133 millionpatients, as reported in the abstract
>67 millionpatient records, as reported in the results

Patients, records and encounters are different units, and a single paper can report more than one. Figures as reported in Voss EA, Blacketer C, van Sandijk S, et al. European Health Data & Evidence Network — learnings from building out a standardized international health data network. Journal of the American Medical Informatics Association. We surface rather than reconcile the difference, because a reader cannot tell from the abstract which quantity the study concerns.

What machine learning adds, and the paradox it runs into

Machine learning on large health data does three things conventional analysis does less well. It handles high-dimensional inputs where the number of candidate predictors is large relative to any prior hypothesis. It captures interactions and non-linearities without them being pre-specified. And it operates on unstructured data — notes, images, waveforms — that structured analysis cannot use at all. Topol, in Nature Medicine, surveys the clinical applications; Shaban-Nejad, Michalowski and Buckeridge, in npj Digital Medicine, frame the population-health version 31,36.

What it does not do is repair the data. Msaouel’s big data paradox in Cancer Investigation is the essential corrective: as sample size grows, the variance of an estimate shrinks while its bias does not, so a large biased dataset produces a narrow confidence interval around the wrong answer, and the apparent precision makes the error harder to notice 18. Verheij, Curcin, Delaney and McGilchrist enumerate where those biases come from in primary care EHR data specifically — recording practice, consultation behaviour, coding incentives, differential completeness — and none of them is reduced by having more of the same data 38.

Two consequences for interpretation. First, the design question outranks the model question. Hernán and Robins’ target trial framework, in the American Journal of Epidemiology, is the discipline worth importing: specify the randomised trial you would have run, then say explicitly how your observational analysis deviates from it 44. A machine learning model fitted without that specification is an exceptionally good estimator of an unclear quantity. Second, generalisability has to be argued rather than assumed — Degtiar and Rose review generalisability and transportability in the Annual Review of Statistics and Its Application, and the distinction between the two is the distinction between your sample and your target population 11. Car, Sheikh, Wicks and Williams put the general case in BMC Medicine under a title that is hard to improve on: building foundations for knowledge and wisdom beyond the hype. See the challenges of delivering clinical impact with machine learning for the validation side.

Multi-omic and multimodal integration

The most distinctive thing about health big data is not the row count but the number of different measurement modalities that can now be attached to one person: genome, transcriptome, imaging, longitudinal clinical record, and increasingly device and sensor streams.

Karczewski and Snyder, in Nature Reviews Genetics, set out integrative omics for health and disease 37. Curtis and colleagues’ METABRIC analysis in Nature is the canonical demonstration that combining genomic and transcriptomic architecture across a large tumour series reveals subgroups neither modality identifies alone 48. On the analysis side, Toro-Domínguez and colleagues survey gene expression meta-analysis methods and applications in Briefings in Bioinformatics, and Zhou and colleagues describe NetworkAnalyst as a visual analytics platform for expression profiling and meta-analysis in Nucleic Acids Research — covered further in our page on NetworkAnalyst as a bioinformatics tool 20,34.

The bottleneck is linkage to the clinical record, and it has been a known bottleneck for a long time. Ritchie, de Andrade and Kuivaniemi, in Frontiers in Genetics, describe integrating electronic health records with genomics as the foundation of precision medicine; Scheuner and colleagues asked whether electronic health records were ready for genomic medicine in Genetics in Medicine, and the question has not been fully answered since 46,50. Park and colleagues’ work bringing DICOM into the OMOP common data model is the current state of the imaging equivalent. The honest summary for a methods section: multimodal integration is technically demonstrated and operationally rare, and a study that achieves it should describe how, because readers will assume the easier version. Current developments in bioinformatics and computational biology covers the analytic toolchain.

Describing a large-scale data source so a reviewer can judge it

The data section of a big-data paper does more work than the methods section in a conventional study, because the reader’s entire assessment of validity depends on understanding what the data are. Four things have to be in it.

Provenance and purpose. What system generated these records, for what purpose, and over what period. A claims database exists to settle payments; a registry exists to count cases; a cohort exists to be studied. Each produces characteristic distortions, and naming the purpose is the shortest way to signal that you know yours.

Composition and representativeness. Who is in and who is not, and how the included population differs from the target population. Arora and colleagues, in Nature Medicine, argue for standards for health datasets used in AI applications precisely because provenance, composition and known limitations are what make fitness for purpose assessable 15.

Counts with units and stages. As above: unit defined, pipeline stage stated, overlap between sources addressed.

Measurement. How each key variable was ascertained, and whether that differed across sites or over time. This is the item most often missing and the one that most often explains an anomalous result.

Then separate description from inference. A sentence about what the dataset contains and a sentence about what the analysis therefore shows are different claims, and the common failure is to let the size of the first do the work of the second. Data management planning and research data services sit on the upstream side of this, and biostatistics support on the analytic side.

The data-source paragraph of a large-scale observational study

Size as credential

We analysed a large real-world database containing over 100 million patients from multiple health systems. The size of the dataset allowed us to detect associations with high statistical power, and all associations reported were highly significant (p < 0.001).

“Over 100 million patients” with no unit discipline. Patients, records and encounters are different quantities, and a reader cannot tell which this is or at which pipeline stage it was taken.

No provenance or purpose. Claims data, EHR data and registry data distort in different directions, and nothing here says which this is or what it was collected for.

No pipeline. The analysed sample is almost certainly orders of magnitude smaller than the headline figure, and the attrition is where all the selection happens.

Power treated as a virtue in itself. At this scale everything is significant, so citing significance is uninformative at best and misleading at worst.

No measurement statement. How exposure and outcome were ascertained is the item most likely to explain an anomalous result, and it is absent.

Overlap between sources unaddressed. “Multiple health systems” raises the question of double counting and does not answer it.

Size with provenance and units

We analysed de-identified administrative claims and hospital electronic health record data mapped to the OMOP common data model, covering care delivered between January 2016 and December 2022. At extraction the dataset held 100,092,900 distinct patients; after requiring at least 365 days of continuous observation before the index date, 14.2 million patients were eligible, and after applying the diagnostic and age criteria, 318,400 entered the analysis. These are counts of distinct patients within a single contributing database, so no cross-source de-duplication was required; where we report the pooled total across sources in Table 1, patients may appear in more than one source and the total is a sum rather than a distinct count. The data were generated for reimbursement and clinical documentation rather than research, so exposure is ascertained from dispensing or prescribing records and not from administration, and outcomes are ascertained from coded diagnoses whose recording may vary across contributing systems. Because statistical significance is uninformative at this sample size, we report absolute risk differences with confidence intervals and pre-specified minimum clinically important differences, and we do not interpret p values.

The unit is named in the same sentence as every number, and the figure at each pipeline stage is given, so the selection is visible.

Provenance and original purpose are stated, which tells a reader which distortions to expect before they read the results.

Cross-source overlap is addressed explicitly in both directions — where a distinct count is possible and where the figure is a sum.

Measurement limitations are stated concretely: dispensing rather than administration, coded diagnoses with variable recording. These are the real limits on what the study can conclude.

The inference strategy is adapted to the sample size rather than exploiting it, with absolute differences and a pre-specified clinically important difference instead of p values.

Everything claimed is checkable, and a reader can decide for themselves whether the dataset supports the question — which is the only purpose of this paragraph.

A reviewer reading the first paragraph has to assume the worst about selection, measurement and double counting, because nothing rules it out. The second paragraph answers all three questions before they are asked, which is also the cheapest way to make the results section believable.

Take-home message

Take-home message

Health big data is characterised by volume, velocity and variety, with veracity as a fourth property that Raghupathi and Raghupathi were explicit in calling a goal rather than current reality — and that distinction governs everything downstream. The resources differ in kind: UK Biobank holds approximately 500,000 participants with genetic data for 488,377; All of Us released 245,388 clinical-grade genome sequences with genotyping on 312,925 participants; a single administrative database in a six-database common-data-model study held 100,092,900 patients; and EHDEN standardised data from 25 partners across 11 European countries. Harmonisation through a common data model and a shared vocabulary is the precondition for using any of them together, and it is lossy in ways that differ by site. Counts are definitional choices, and they can disagree inside one paper — the EHDEN account reports more than 133 million patients in its abstract and more than 67 million patient records in its results. Machine learning adds high-dimensional, non-linear and unstructured-data capability, and adds nothing to data quality: as Msaouel’s big data paradox has it, scale shrinks variance and leaves bias intact, so the result is a narrow interval around a possibly wrong answer.

How to apply this to your study

How to apply this to your own study

  1. Write the provenance sentence before the sample-size sentence. What system generated these data, for what original purpose, over what period. Every distortion a reviewer will suspect follows from the answer.
  2. Give every count a unit and a pipeline stage. Patients, records, encounters or person-years; at extraction, after observation-window requirements, after eligibility criteria. Report the figure at each stage.
  3. Address source overlap explicitly. Say whether cross-source de-duplication was possible, and where it was not, label the total as a sum of sources that may share patients.
  4. Specify a target trial before fitting anything. State the randomised trial you would have run — eligibility, treatment strategies, assignment, outcome, follow-up — and say where your analysis departs from it.
  5. Replace significance with magnitude. At these sample sizes p values are uninformative, so pre-specify a minimum clinically important difference and report absolute effects with intervals and calibration.
  6. Document the mapping where data were harmonised. Which variables mapped cleanly, which approximately, which were dropped, and whether mapping quality differed between contributing sites — differential mapping error is a common source of spurious between-site heterogeneity.

Common questions

Data assembled at a scale and variety that conventional single-purpose analysis cannot handle, and — more importantly — usually generated for a purpose other than the research question being asked of it. Raghupathi and Raghupathi, in Health Information Science and Systems, characterise it by volume, velocity and variety, adding veracity as a fourth property they describe as a goal rather than current reality. In practice the term covers prospective cohorts with deep phenotyping, programme-scale genomic resources, routine claims and electronic health record data at administrative scale, federated international networks built on a common data model, and increasingly imaging and device streams linked to those records. The practical definition that matters for interpretation is secondary use: these data were recorded for billing, documentation or a different study.

Volume, velocity and variety are the characteristics of the data as it is. Veracity is usually listed as a fourth, but Raghupathi and Raghupathi were careful about this: they add it as a property that is a goal rather than current reality. The distinction is worth preserving, because treating accuracy as an attribute a dataset possesses by virtue of being large inverts the actual problem. Read as commitments, volume means statistical significance stops being informative; velocity means your model is fitted to a moment that has passed; variety means the fields carry different semantics because different systems defined them for different purposes; and veracity, as a goal, means that establishing what the data mean is part of the analysis rather than a step before it.

More precise ones, not more accurate ones, and conflating those is the central error in this field. Msaouel describes the big data paradox in Cancer Investigation: increasing sample size reduces the variance of an estimate and does nothing to its bias, so a large biased dataset yields a narrow confidence interval around the wrong answer — and the apparent precision makes the error harder to detect. Verheij, Curcin, Delaney and McGilchrist catalogue the specific bias sources in primary care EHR data in the Journal of Medical Internet Research: recording practice, consultation behaviour, coding incentives, differential completeness. None of these shrinks with more of the same data. Design and measurement decide reliability; sample size decides precision.

Because records from different systems do not mean the same thing until someone decides they do. A diagnosis code in one country’s billing system and a problem-list entry in another country’s EHR may refer to the same clinical state, a narrower one, a broader one, or an administrative artefact. Voss and colleagues demonstrated the feasibility of applying a common data model across six disparate claims and EHR databases, and Reich and colleagues describe the OHDSI standardised vocabularies as the large-scale centralised reference ontology that makes the structural mapping semantically meaningful. Two things to report: mapping is lossy and the losses differ by site, which can manufacture between-site heterogeneity; and the harmonised variable set is narrower than any single contributing source, so the federated question is slightly different.

With the unit named in the same sentence as the number, the pipeline stage stated, and overlap between sources addressed. “Over 100 million patients” is not a usable figure on its own: patients, records, encounters and person-years are different quantities, and a count at extraction is usually orders of magnitude larger than the analysed sample. Report the figure at each stage — extraction, observation-window requirement, eligibility criteria, final analysis — which is the same discipline a systematic review flow diagram imposes. And say whether patients can appear in more than one contributing source. The need for this is not hypothetical: the EHDEN account by Voss and colleagues reports more than 133 million patients in the abstract and more than 67 million patient records in the results.

Three specific capabilities. It handles high-dimensional inputs where candidate predictors greatly outnumber any prior hypothesis. It captures interactions and non-linearities without them being pre-specified, which matters when the functional form is genuinely unknown. And it uses unstructured data — clinical notes, images, waveforms — that structured analysis cannot touch. What it does not add is data quality, causal identification or generalisability. For causal questions the design has to come first: Hernán and Robins’ target trial framework in the American Journal of Epidemiology is the discipline to import, specifying the trial you would have run and stating where the observational analysis departs from it. Degtiar and Rose’s review of generalisability and transportability covers the second gap.

Demonstrably yes, and operationally much less often than the literature’s enthusiasm implies. Karczewski and Snyder set out integrative omics in Nature Reviews Genetics, and Curtis and colleagues’ METABRIC analysis in Nature showed that combining genomic and transcriptomic architecture across a large tumour series reveals subgroups neither modality identifies alone. The bottleneck is linkage to the clinical record: Ritchie, de Andrade and Kuivaniemi describe integrating EHRs with genomics as the foundation of precision medicine in Frontiers in Genetics, while Scheuner and colleagues’ question in Genetics in Medicine — whether EHRs are ready for genomic medicine — has not been fully answered. Park and colleagues’ incorporation of DICOM into the OMOP common data model is the imaging equivalent, and it is recent.

The limitations specific to secondary use, named concretely. Typically: that the data were generated for billing or documentation rather than research, so exposure may be ascertained from prescribing rather than administration and outcomes from codes whose recording varies; that the population is defined by who appears in these systems rather than by a sampling frame, with a statement of who is absent; that observation is censored by enrolment and disenrolment rather than by the clinical course; that mapping to a common data model was lossy and may have been differentially lossy across sites; that the analysed period predates known changes in practice or coding; and that statistical significance is uninformative at this scale, so magnitude and calibration carry the interpretation.

References and sources

  1. Shahzad K, Khan SA, Latif M, Javeed AMD, et al. , Iqbal A, et al. Big data analytics in healthcare: current practices, innovations, and future prospects. Journal of Big Data. 2025;12(1):242. doi:10.1186/s40537-025-01288-2 read 9 October 2026. doi.org/10.1186/s40537-025-01288-2
  2. Park WY, Sippel Schmidt T, Salvador G, O'Donnell K, Genereaux B, Jeon K, You SC, Dewey BE, Nagy P; Alzheimer's Disease Neuroimaging Initiative. Breaking data silos: incorporating the DICOM imaging standard into the OMOP CDM to enable multimodal research. Journal of the American Medical Informatics Association. 2025;32(10):1533-1541. doi:10.1093/jamia/ocaf091 read 9 October 2026. doi.org/10.1093/jamia/ocaf091
  3. Coveney P, Highfield R, Stahlberg E, et al. , Vazquez M, et al. Digital twins and Big AI: the future of truly individualised healthcare. npj Digital Medicine. 2025;8(1):494. doi:10.1038/s41746-025-01874-x read 9 October 2026. doi.org/10.1038/s41746-025-01874-x
  4. Badawy M, Ramadan N, Hefny HA. Big data analytics in healthcare: data sources, tools, challenges, and opportunities. Journal of Electrical Systems and Information Technology. 2024;11(1):63. doi:10.1186/s43067-024-00190-w read 9 October 2026. doi.org/10.1186/s43067-024-00190-w
  5. Voss EA, Blacketer C, van Sandijk S, Moinat M, Kallfelz M, van Speybroeck M, Prieto-Alhambra D, Schuemie MJ, Rijnbeek PR. European Health Data & Evidence Network-learnings from building out a standardized international health data network. Journal of the American Medical Informatics Association. 2024;31(1):209-219. doi:10.1093/jamia/ocad214 read 9 October 2026. doi.org/10.1093/jamia/ocad214
  6. The All of Us Research Program Genomics Investigators. Genomic data in the All of Us Research Program. Nature. 2024;627(8003):340-346. doi:10.1038/s41586-023-06957-x read 9 October 2026. doi.org/10.1038/s41586-023-06957-x
  7. Li YH, Li YL, Wei MY, Li GY, et al, et al. Innovation and challenges of artificial intelligence technology in personalized healthcare. Scientific Reports. 2024;14(1):18994. doi:10.1038/s41598-024-70073-7 read 9 October 2026. doi.org/10.1038/s41598-024-70073-7
  8. Becker R, Chokoshvili D, Dove ES. Legal bases for effective secondary use of health and genetic data in the EU: time for new legislative solutions to better harmonize data for cross-border sharing? International Data Privacy Law. 2024;14(3):223-246. doi:10.1093/idpl/ipae014 read 9 October 2026. doi.org/10.1093/idpl/ipae014
  9. Reich C, Ostropolets A, Ryan P, Rijnbeek P, Schuemie M, Davydov A, Dymshyts D, Hripcsak G. OHDSI Standardized Vocabularies-a large-scale centralized reference ontology for international data harmonization. Journal of the American Medical Informatics Association. 2024;31(3):583-590. doi:10.1093/jamia/ocad247 read 9 October 2026. doi.org/10.1093/jamia/ocad247
  10. Lasko TA, Strobl EV, Stead WW. Why do probabilistic clinical models fail to transport between sites. npj Digital Medicine. 2024;7. read 9 October 2026. arxiv.org
  11. Degtiar I, Rose S. A review of generalizability and transportability. Annual Review of Statistics and Its Application. 2023;10:501-24. read 9 October 2026. arxiv.org
  12. Kamal S, Lazaar M, Bennani Othmani M, Bourzgui F, Mahboub O. Deep Learning Models for Medical Image Analysis in Smart Healthcare System: A Review. Lecture Notes in Networks and Systems (Proceedings of the 6th International Conference on Big Data and Internet of Things, BDIoT 2022), Springer. 2023;625:376-387. doi:10.1007/978-3-031-28387-1_32 read 9 October 2026. doi.org/10.1007/978-3-031-28387-1_32
  13. Pinto Junior EP, Normando P, Flores-Ortiz R, Afzal MU, Jamil MA, Fernandez Bertolin S, de Araujo Oliveira V, Martufi V, de Sousa F, Bashir A, Burn E, Ichihara MY, Barreto ML, Duarte Salles T, Prieto-Alhambra D, Hafeez H, Khalid S. Integrating real-world data from Brazil and Pakistan into the OMOP common data model and standardized health analytics framework to characterize COVID-19 in the Global South. Journal of the American Medical Informatics Association. 2023;30(4):643-655. doi:10.1093/jamia/ocac180 read 9 October 2026. doi.org/10.1093/jamia/ocac180
  14. Subbiah V. The next generation of evidence-based medicine. Nature Medicine. 2023;29:49-58. doi:10.1038/s41591-022-02160-z read 9 October 2026. doi.org/10.1038/s41591-022-02160-z
  15. Arora A, Alderman JE, Palmer J, Ganapathi S, Laws E, McCradden MD, Oakden-Rayner L, Pfohl SR, Ghassemi M, McKay F, et al. , Liu X, et al. The value of standards for health datasets in artificial intelligence-based applications. Nature Medicine. 2023;29(11):2929-2938. doi:10.1038/s41591-023-02608-w read 9 October 2026. doi.org/10.1038/s41591-023-02608-w
  16. Quazi S. Artificial intelligence and machine learning in precision and genomic medicine. 2022;39(8):120. doi:10.1007/s12032-022-01711-1 read 9 October 2026. doi.org/10.1007/s12032-022-01711-1
  17. Reps JM, Williams RD, Schuemie MJ, Ryan PB, Rijnbeek PR. Learning patient-level prediction models across multiple healthcare databases: evaluation of ensembles for increasing model transportability. BMC Medical Informatics and Decision Making. 2022;22:142. read 9 October 2026. arxiv.org
  18. Msaouel P. The big data paradox in clinical practice. Cancer Investigation. 2022;40:567-76. doi:10.1080/07357907.2022.2084621 read 9 October 2026. doi.org/10.1080/07357907.2022.2084621
  19. Kunnapuu K, Ioannou S, Ligi K, Kolde R, Laur S, Vilo J, Rijnbeek PR, Reisberg S. Trajectories: a framework for detecting temporal clinical event sequences from health data standardized to the Observational Medical Outcomes Partnership (OMOP) Common Data Model. JAMIA Open. 2022;5(1):ooac021. doi:10.1093/jamiaopen/ooac021 read 9 October 2026. doi.org/10.1093/jamiaopen/ooac021
  20. Toro-Dominguez D, Villatoro-Garcia JA, Martorell-Marugan J, Roman-Montoya Y, Alarcon-Riquelme ME, Carmona-Saez P. A survey of gene expression meta-analysis: methods and applications. Briefings in Bioinformatics. 2021;22(2):1694-1705. doi:10.1093/bib/bbaa019 read 9 October 2026. doi.org/10.1093/bib/bbaa019
  21. de Jong VMT, Moons KGM, Eijkemans MJC, Riley RD, Debray TPA. Developing more generalizable prediction models from pooled studies and large clustered data sets. Statistics in Medicine. 2021;40:3533-59. read 9 October 2026. arxiv.org
  22. Kaissis G, Ziller A, Passerat-Palmbach J, Ryffel T, Usynin D, Trask A, Lima I, Mancuso J, Jungmann F, Steinborn MM, Saleh A, Makowski M, Rueckert D, Braren R. End-to-end privacy preserving deep learning on multi-institutional medical imaging. Nature Machine Intelligence. 2021;3(6):473-484. doi:10.1038/s42256-021-00337-8 read 9 October 2026. doi.org/10.1038/s42256-021-00337-8
  23. Dockes J, Varoquaux G, Poline JB. Preventing dataset shift from breaking machine-learning biomarkers. GigaScience. 2021;10:giab055. read 9 October 2026. arxiv.org
  24. Agrawal R, Prabakaran S. Big data in digital healthcare: lessons learnt and recommendations for general practice. Heredity. 2020;124(4):525-534. doi:10.1038/s41437-020-0303-2 read 9 October 2026. doi.org/10.1038/s41437-020-0303-2
  25. Wouters OJ, McKee M, Luyten J. Estimated research and development investment needed to bring a new medicine to market, 2009-2018. JAMA. 2020;323:844-53. doi:10.1001/jama.2020.1166 read 9 October 2026. doi.org/10.1001/jama.2020.1166
  26. McKinney SM, Sieniek M, Godbole V, Godwin J, Antropova N, Ashrafian H, Back T, Chesus M, Corrado GS, Darzi A, Etemadi M, Garcia-Vicente F, Gilbert FJ, Halling-Brown M, Hassabis D, Jansen S, Karthikesalingam A, Kelly CJ, King D, Ledsam JR, Melnick D, Mostofi H, Peng L, Reicher JJ, Romera-Paredes B, Sidebottom R, Suleyman M, Tse D, Young KC, De Fauw J, Shetty S. International evaluation of an AI system for breast cancer screening. Nature. 2020;577:89-94. doi:10.1038/s41586-019-1799-6 read 9 October 2026. doi.org/10.1038/s41586-019-1799-6
  27. Liu X, Faes L, Kale AU, Wagner SK, Fu DJ, Bruynseels A, Mahendiran T, Moraes G, Shamdas M, Kern C, Ledsam JR, Schmid MK, Balaskas K, Topol EJ, Bachmann LM, Keane PA, Denniston AK. A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis. The Lancet Digital Health. 2019;1(6):e271-e297. doi:10.1016/S2589-7500(19)30123-2 read 9 October 2026. doi.org/10.1016/S2589-7500(19)30123-2
  28. Car J, Sheikh A, Wicks P, Williams MS. Beyond the hype of big data and artificial intelligence: building foundations for knowledge and wisdom. BMC Medicine. 2019;17:143. doi:10.1186/s12916-019-1382-x read 9 October 2026. doi.org/10.1186/s12916-019-1382-x
  29. Dash S, Shakyawar SK, Sharma M, et al. , Kaushik S, et al. Big data in healthcare: management, analysis and future prospects. Journal of Big Data. 2019;6(1):54. doi:10.1186/s40537-019-0217-0 read 9 October 2026. doi.org/10.1186/s40537-019-0217-0
  30. Wong CH, Siah KW, Lo AW. Estimation of clinical trial success rates and related parameters. Biostatistics. 2019;20:273-86. doi:10.1093/biostatistics/kxx069 read 9 October 2026. doi.org/10.1093/biostatistics/kxx069
  31. Topol EJ. High-performance medicine: the convergence of human and artificial intelligence. Nature Medicine. 2019;25(1):44-56. doi:10.1038/s41591-018-0300-7 read 9 October 2026. doi.org/10.1038/s41591-018-0300-7
  32. Bourassa Forcier M, Gallois H, Mullan S, Joly Y. Integrating artificial intelligence into health care through data access: can the GDPR act as a beacon for policymakers? Journal of Law and the Biosciences. 2019;6(1):317-335. doi:10.1093/jlb/lsz013 read 9 October 2026. doi.org/10.1093/jlb/lsz013
  33. Kelly CJ, Karthikesalingam A, Suleyman M, Corrado G, King D. Key challenges for delivering clinical impact with artificial intelligence. BMC Medicine. 2019. doi:10.1186/s12916-019-1426-2 read 9 October 2026. doi.org/10.1186/s12916-019-1426-2
  34. Zhou G, Soufan O, Ewald J, Hancock REW, Basu N, Xia J. NetworkAnalyst 3.0: a visual analytics platform for comprehensive gene expression profiling and meta-analysis. Nucleic Acids Research. 2019;47(W1):W234-W241. doi:10.1093/nar/gkz240 read 9 October 2026. doi.org/10.1093/nar/gkz240
  35. Miller DD. The medical AI insurgency: what physicians must know about data to practice with intelligent machines. npj Digital Medicine. 2019;2(1):62. doi:10.1038/s41746-019-0138-5 read 9 October 2026. doi.org/10.1038/s41746-019-0138-5
  36. Shaban-Nejad A, Michalowski M, Buckeridge DL. Health intelligence: how artificial intelligence transforms population and personalized health. npj Digital Medicine. 2018;1(1):53. doi:10.1038/s41746-018-0058-9 read 9 October 2026. doi.org/10.1038/s41746-018-0058-9
  37. Karczewski KJ, Snyder MP. Integrative omics for health and disease. Nature Reviews Genetics. 2018;19(5):299-310. doi:10.1038/nrg.2018.4 read 9 October 2026. doi.org/10.1038/nrg.2018.4
  38. Verheij RA, Curcin V, Delaney BC, McGilchrist MM. Possible sources of bias in primary care electronic health record data use and reuse. Journal of Medical Internet Research. 2018;20:e185. doi:10.2196/jmir.9134 read 9 October 2026. doi.org/10.2196/jmir.9134
  39. Bycroft C, Freeman C, Petkova D, Band G, Elliott LT, Sharp K, Motyer A, Vukcevic D, Delaneau O, O'Connell J, et al. , Marchini J, et al. The UK Biobank resource with deep phenotyping and genomic data. Nature. 2018;562(7726):203-209. doi:10.1038/s41586-018-0579-z read 9 October 2026. doi.org/10.1038/s41586-018-0579-z
  40. Ballantyne A. Where is the human in the data? A guide to ethical data use. GigaScience. 2018;7(7):giy076. doi:10.1093/gigascience/giy076 read 9 October 2026. doi.org/10.1093/gigascience/giy076
  41. Burton PR, Banner N, Elliot MJ, Knoppers BM, Banks J. Policies and strategies to facilitate secondary use of research data in the health sciences. International Journal of Epidemiology. 2017;46(6):1729-1733. doi:10.1093/ije/dyx195 read 9 October 2026. doi.org/10.1093/ije/dyx195
  42. Riley RD, Ensor J, Snell KIE, Debray TPA, Altman DG, Moons KGM, Collins GS. External validation of clinical prediction models using big datasets from e-health records or IPD meta-analysis: opportunities and challenges. BMJ. 2016;353:i3140. read 9 October 2026. arxiv.org
  43. Beckmann JS, Lew D. Reconciling evidence-based medicine and precision medicine in the era of big data: challenges and opportunities. Genome Medicine. 2016;8:134. doi:10.1186/s13073-016-0388-7 read 9 October 2026. doi.org/10.1186/s13073-016-0388-7
  44. Hernan MA, Robins JM. Using big data to emulate a target trial when a randomized trial is not available. American Journal of Epidemiology. 2016;183:758-64. read 9 October 2026. arxiv.org
  45. Voss EA, Makadia R, Matcho A, Ma Q, Knoll C, Schuemie M, DeFalco FJ, Londhe A, Zhu V, Ryan PB. Feasibility and utility of applications of the common data model to multiple, disparate observational health databases. Journal of the American Medical Informatics Association. 2015;22(3):553-564. doi:10.1093/jamia/ocu023 read 9 October 2026. doi.org/10.1093/jamia/ocu023
  46. Ritchie MD, de Andrade M, Kuivaniemi H. The foundation of precision medicine: integrating electronic health records with genomics through basic, clinical, and translational research. Frontiers in Genetics. 2015;6:104. doi:10.3389/fgene.2015.00104 read 9 October 2026. doi.org/10.3389/fgene.2015.00104
  47. Raghupathi W, Raghupathi V. Big data analytics in healthcare: promise and potential. Health Information Science and Systems. 2014;2(1):3. doi:10.1186/2047-2501-2-3 read 9 October 2026. doi.org/10.1186/2047-2501-2-3
  48. Curtis C, Shah SP, Chin SF, Turashvili G, Rueda OM, Dunning MJ, Speed D, Lynch AG, Samarajiwa S, Yuan Y, Graf S, Ha G, Haffari G, Bashashati A, Russell R, McKinney S, METABRIC Group, Langerod A, Green A, Provenzano E, Wishart G, Pinder S, Watson P, Markowetz F, Murphy L, Ellis I, Purushotham A, Borresen-Dale AL, Brenton JD, Tavare S, Caldas C, Aparicio S. The genomic and transcriptomic architecture of 2,000 breast tumours reveals novel subgroups. Nature. 2012;486(7403):346-52. doi:10.1038/nature10983 read 9 October 2026. doi.org/10.1038/nature10983
  49. Shi L, Campbell G, Jones WD, Campagne F, Wen Z, Walker SJ, Su Z, Chu TM, Goodsaid FM, Pusztai L, Shaughnessy JD Jr, Oberthuer A, Thomas RS, Paules RS, Fielden M, Barlogie B, Chen W, Du P, Fischer M, Furlanello C, Gallas BD, Ge X, Megherbi DB, Symmans WF, Wang MD, Zhang J, Bitter H, Brors B, Bushel PR, Bylesjo M, Chen M, Cheng J, Cheng J, Chou J, Davison TS, Delorenzi M, Deng Y, Devanarayan V, Dix DJ, Dopazo J, Dorff KC, Elloumi F, Fan J, Fan S, Fan X, Fang H, Gonzaludo N, Hess KR, Hong H, Huan J, Irizarry RA, Judson R, Juraeva D, Lababidi S, Lambert CG, Li L, Li Y, Li Z, Lin SM, Liu G, Lobenhofer EK, Luo J, Luo W, McCall MN, Nikolsky Y, Pennello GA, Perkins RG, Philip R, Popovici V, Price ND, Qian F, Scherer A, Shi T, Shi W, Sung J, Thierry-Mieg D, Thierry-Mieg J, Thodima V, Trygg J, Vishnuvajjala L, Wang SJ, Wilson M, Winslow RL, Wu Q, Xie Z, Yousef GM, Zhang L, Zhao Y, Zhou J, Zhu M, Zhu S, Zollanvari A, Tong W, Dragan Y, Shaughnessy JD. The MicroArray Quality Control (MAQC)-II study of common practices for the development and validation of microarray-based predictive models. Nature Biotechnology. 2010;28(8):827-38. doi:10.1038/nbt.1665 read 9 October 2026. doi.org/10.1038/nbt.1665
  50. Scheuner MT, de Vries H, Kim B, Meili RC, Olmstead SH, Teleki S. Are electronic health records ready for genomic medicine? Genetics in Medicine. 2009;11:510-17. doi:10.1097/GIM.0b013e3181a53331 read 9 October 2026. doi.org/10.1097/GIM.0b013e3181a53331

Every source above is a peer-reviewed article, a publisher’s record of one, or a standard issued by the body that owns it. Each carries the date it was last read. Where a standard moves, this article is revised rather than left up with a note.

How to cite this article

Pick your referencing style and copy it. Your university or target journal may vary the punctuation — check its guide if it has one.

Pubrica Academy. (2026, October 9). What Big Data Means in Health, and How Machine Learning Reads It. Pubrica. https://pubrica.com/academy/medical-writing/what-is-big-data-discuss-the-interpretation-of-artificial-intelligence-machine-learning-in-big-data-analytics/

This article is written to be learned from. Cite it if you are referring to what it says; if you are using the evidence behind it, cite the sources listed above instead.

Big-data study data-description stub

Paste this into the data section of a large-scale observational or machine learning paper and complete the blanks. It is organised so that a reviewer’s first four questions — where did these data come from, who is in them, how many exactly, and how was anything measured — are answered before the analysis is described.

LARGE-SCALE DATA SOURCE DESCRIPTION

1. PROVENANCE AND PURPOSE
   Data type: [ ] claims  [ ] EHR  [ ] registry  [ ] cohort
              [ ] genomic programme  [ ] imaging  [ ] device/sensor
              [ ] federated network across ____ partners, ____ countries
   Generating system(s): ______________________
   ORIGINAL purpose these data were collected for: ______________
   Period covered: ______ to ______
   Date of extraction: ______

2. COMPOSITION AND REPRESENTATIVENESS
   Population captured by this system: ______________________
   Target population for my question: ______________________
   Groups absent or under-represented: ______________________
   Entry and exit into the data (enrolment, registration, coverage):
   ______________________________________________________

3. COUNTS  (state the UNIT every time)
   Unit: [ ] distinct patients  [ ] records  [ ] encounters
         [ ] person-years      [ ] other: ______
   At extraction ................................. ____________
   After observation-window requirement (____ days) ____________
   After eligibility criteria ..................... ____________
   Entering the analysis .......................... ____________
   Cross-source overlap:
     [ ] single source - no de-duplication needed
     [ ] de-duplicated across sources using ______________
     [ ] NOT de-duplicable - totals are sums and patients may recur

4. HARMONISATION  (if a common data model was used)
   Model and version: ______________________
   Vocabulary/ontology: ______________________
   Variables that mapped cleanly: ______________________
   Variables that mapped approximately: ______________________
   Variables dropped: ______________________
   Did mapping quality differ between sites?  [ ] yes  [ ] no
     If yes, which and how: ______________________

5. MEASUREMENT  (one line per key variable)
   Exposure ascertained from: ______________  Varies by site? [ ]
   Outcome ascertained from: _______________  Varies by site? [ ]
   Covariates ascertained from: ____________  Varies by site? [ ]
   Known changes in coding or practice during the period: ________

6. TARGET TRIAL  (for any causal question)
   Eligibility: ______________________
   Treatment strategies compared: ______________________
   Assignment and how it is emulated: ______________________
   Outcome and follow-up window: ______________________
   Departures from the target trial: ______________________

7. INFERENCE AT SCALE
   [ ] Minimum clinically important difference pre-specified: ______
   [ ] Absolute effects reported with confidence intervals
   [ ] Calibration reported for any predictive model
   [ ] p values NOT used as evidence of importance

8. LIMITATIONS DRAWN FROM THE ABOVE  (one sentence each)
   Secondary use: ______________________
   Population coverage: ______________________
   Measurement: ______________________
   Harmonisation loss: ______________________
   Time period: ______________________

Pubrica AcademyWritten and maintained by the subject matter experts of Pubrica who do this work. Articles are revised when the standard underneath them moves, and each one carries the date it was last checked.

How Pubrica helps

Pubrica’s data analytics and biostatistics teams write and review the data sections that decide whether a large-scale study is believable: provenance and original purpose, composition against the target population, counts with their units and pipeline stages, and how each key variable was actually ascertained. We specify target trials for causal questions on observational data, check that mapping losses in a harmonised dataset are documented and not differential between sites, and replace significance-driven interpretation with magnitude and calibration where the sample size makes p values meaningless.

Find something else

Search the rest of the Academy.

Open with no account. Type what you are working on, or open the full library to filter by category, sub-topic or resource type.

Send us your dataset description and analysis plan and we will tell you which questions it can answer and which it cannot.

A scoping read costs nothing and takes two business days. You get a written view of what the work needs and who would hold it.

Get a free quote

Regulations

Check your project against this rule

  1. 1Your work
  2. 2Your details
  3. 3Send
What do you need?

Material you send is held under signed confidentiality, with access limited to the assigned team and encrypted transfer. It is not used to train anything and is not passed to third-party tools. Ask for the agreement before you send anything you would rather cover first.