Contents
Academy · Research Data, Analytics & AI · Mixed-Methods Study Quality
The Design Quality of Mixed-Methods Studies in Health Care Research: Evidence from an Empirical Audit
A review of 75 mixed-methods proposals and reports commissioned by England's Department of Health found only about a third explicitly justified combining quantitative and qualitative methods, and researchers rarely attempted to integrate the two sets of findings into a single conclusion.
A mixed-methods study combines quantitative data collection and statistical analysis with qualitative methods, such as interviews, focus groups or thematic analysis, within a single research project, with the intention that the two components address complementary questions or strengthen each other’s conclusions. Whether a given mixed-methods study achieves this intention, rather than merely conducting two separate studies under one title, depends on specific, verifiable design choices: a stated rationale for combining methods, a clear priority and sequence between the components, and a genuine attempt to integrate their findings. The most detailed empirical audit of these design choices in health services research found that most commissioned mixed-methods studies were deficient on precisely these points.
Learning objectives
- What the most detailed empirical audit of mixed-methods health research measured and found
- The specific design elements, rationale, priority, sequence, and integration, that distinguish genuine mixed-methods research from two parallel studies
- Why qualitative components were reported less adequately than quantitative ones in the audited sample
Common misconceptions
- “Conducting a quantitative strand and a qualitative strand within the same project automatically constitutes a well-designed mixed-methods study.” O’Cathain, Murphy and Nicholl’s review of mixed-methods research commissioned by England’s Department of Health found that only around a third of the sampled proposals and final reports explicitly justified the use of a mixed-methods design, and that most studies made little or no attempt to integrate their quantitative and qualitative findings into a combined conclusion; the presence of both methods therefore did not by itself indicate the integrated design that the term is intended to describe.
Scope and methods of the review
Alicia O’Cathain, Elizabeth Murphy and Jon Nicholl reviewed mixed-methods studies commissioned by England’s Department of Health, examining both research proposals and final reports to assess the design and conduct of each study against a structured set of quality questions that they developed specifically for this purpose1. The research team obtained study proposals, final reports and related publications directly from the original investigators and assessed each against criteria covering whether the mixed-methods design was explicitly justified, how the quantitative and qualitative components were described, and whether and how the two sets of findings were integrated12. The authors stated explicitly that this was an initial, exploratory effort intended to begin a discussion of quality in mixed-methods health research, based on the portfolio of a single national commissioner, rather than a validated, universally applicable quality-assessment instrument1.
Reporting of rationale, priority and sequence
The review found that only around a third of the sampled proposals and reports explicitly justified why a mixed-methods design, rather than a single-method study, was appropriate for the research question1. Few studies clearly stated the priority given to each component (whether the quantitative or qualitative strand was primary, or whether they carried equal weight), the sequence in which components were conducted (concurrently or sequentially), or the specific point at which findings from the two components were meant to be brought together1. These design choices are not cosmetic labels: Creswell and Plano Clark’s foundational framework for mixed-methods design treats priority, sequence and point of integration as defining structural decisions that determine the type of mixed-methods design (convergent, explanatory sequential, exploratory sequential or embedded) a study represents, and a study that cannot state these choices clearly has, in effect, not specified its own design3.
Less complete reporting of qualitative components
Within the studies reviewed, the quantitative components were generally better described in terms of their stated purpose, though the review found that details of the statistical analysis itself were frequently missing even where the purpose was clear1. The qualitative components were described less adequately: the review reported that qualitative methods were inadequately described in a substantially larger share of proposals than quantitative methods, a difference that the authors found to be statistically significant at the proposal stage1. This asymmetry is consistent with a broader pattern noted in the methodological literature on mixed methods: because many health researchers are trained primarily in quantitative methods, qualitative components are sometimes added to a study design without the level of methodological planning, sampling rationale or analytical detail given to the quantitative component, even when the qualitative strand is intended to carry equal weight in answering the research question34.
Integration of quantitative and qualitative findings
The review’s most consequential finding concerned integration, the step in which quantitative and qualitative findings are explicitly combined to produce a conclusion that neither component could reach alone. O’Cathain and colleagues found that researchers rarely attempted such integration in the studies reviewed; in most cases, therefore, the quality of integration could not be assessed, because no integrative step existed1. This finding anticipated a concern that later became central to mixed-methods methodology: the widely cited framework of Fetters, Curry and Creswell for achieving integration in mixed-methods designs argues that integration, at the levels of design, methods, and interpretation and reporting, is the step that distinguishes mixed-methods research from the publication of a quantitative study and a qualitative study side by side under one title, and that many self-described mixed-methods studies fail to achieve it at any of these three levels4. Taken together, these two analyses, one an empirical audit of commissioned health research and the other a later methodological framework, support the same practical conclusion: the quality of a mixed-methods study depends on whether it integrates its components on the basis of a stated rationale, not on whether it merely contains both a quantitative and a qualitative element14.
Developments in mixed-methods practice since the audit
Methodological work published since the audit by O’Cathain and colleagues has focused on making integration a concrete, verifiable step rather than an aspiration. Bryman’s analysis of how integration is undertaken in practice, based on a review of published mixed-methods studies across social science and health research, found a similarly wide gap between studies that claimed to combine methods and studies that visibly integrated their findings, and identified specific techniques that distinguish genuine integration from parallel reporting, including the use of qualitative findings to help interpret unexpected quantitative results and the use of quantitative findings to identify cases for more detailed qualitative follow-up5. The later methodological contribution of Guetterman, Fetters and Creswell operationalized this further through the “joint display”, a table or figure that places corresponding quantitative and qualitative findings side by side so that a reader can see where the two components agree, disagree or add distinct dimensions to the same research question; this converts integration from a claim made in a discussion section into a visible, reviewable element of the paper itself6. Pluye and Hong’s review of mixed-methods research and mixed-studies reviews situates these integration techniques within a broader argument that the distinctive value of mixed-methods research, the combination of the explanatory depth of qualitative data with the generalizability of quantitative data, is realized only when integration is performed. This reinforces the view that the central finding of the original health-services audit, that integration was the rarest and most consequential missing step, remains the core methodological challenge of the field rather than a problem specific to the Department of Health portfolio first studied7. O’Cathain’s later editorial reflection on the state of mixed-methods research in the health sciences, written approximately two years after the original audit, describes the subsequent period as “a quiet revolution”, noting a marked increase in funders’ willingness to support mixed-methods proposals and in journals’ willingness to publish them, but cautioning that the quantity of output was increasing faster than the methodological sophistication, particularly with regard to integration, that the earlier audit had identified as the principal gap10. Read against the specific figures from the original audit (roughly a third of proposals explicitly justifying the choice of design, and qualitative components reported markedly less completely than quantitative ones), this later reflection suggests that the problem of volume improved considerably while the problem of quality measured by the audit improved more slowly; for this reason, current methodological guidance continues to treat explicit planning of integration, rather than the mere inclusion of both a quantitative and a qualitative strand, as the test of the design quality of a mixed-methods study. Pluye and Hong’s review of mixed-methods research argues that the distinctive contribution of the approach, the combination of the explanatory depth of qualitative data with the generalizability of quantitative data, is precisely the capability that remains unrealized when integration is omitted, since a study that reports both components without integrating them provides neither the depth nor the generalizability advantage in connected form, but only two separate, smaller findings under one title8. The methodological contribution of Curry, Nembhard and Bradley, directed at outcomes researchers in cardiovascular and health services research, makes the implications concrete for clinical and policy-relevant work: they argue that qualitative and mixed-methods components can reveal why an intervention succeeded or failed for particular patients or in particular settings, information that a quantitative outcome measure alone cannot provide, but only if the qualitative findings are genuinely integrated with the quantitative results rather than reported as a disconnected addendum; this is precisely the integration that the original health-services audit found to be absent from most of the studies it reviewed9.
Key points
Key points
Combining quantitative and qualitative methods in one study does not by itself make it well-designed mixed-methods research. The empirical evidence indicates that researchers frequently fail to justify the design, specify priority and sequence, and, most consequentially, integrate the two sets of findings into a single conclusion.
Practical application
Practical application
- State the rationale for a mixed-methods design explicitly in the proposal. The proposal should explain specifically why neither a purely quantitative nor a purely qualitative approach alone could answer the research question.
- Specify the priority and sequence of the design using a recognized typology. The design should be specified as convergent, explanatory sequential, exploratory sequential or embedded, following a framework such as that of Creswell and Plano Clark, rather than leaving the structure implicit.
- Plan the integration step before data collection begins. How and when the quantitative and qualitative findings will be combined should be decided at the design stage, using the levels identified by Fetters, Curry and Creswell (design, methods, interpretation and reporting) as a checklist.
Frequently asked questions
O’Cathain, Murphy and Nicholl found that only around a third of the sampled proposals and reports gave an explicit rationale for choosing a mixed-methods design over a single-method approach.
The review found that qualitative methods were inadequately described in a significantly larger share of proposals than quantitative methods, a pattern consistent with a broader tendency for researchers trained mainly in quantitative methods to give qualitative components less detailed methodological planning.
Integration refers to the explicit combination of quantitative and qualitative findings at the level of design, methods or interpretation, so that the two components inform a single conclusion rather than being reported as separate, parallel results.
No; the authors describe it as an initial, exploratory effort based on the portfolio of studies of one national commissioner, intended to initiate a discussion of mixed-methods quality rather than to serve as a validated universal instrument.
The widely used framework of Creswell and Plano Clark identifies convergent, explanatory sequential, exploratory sequential and embedded designs, distinguished by the priority and timing given to the quantitative and qualitative components.
References
- O'Cathain A, Murphy E, Nicholl J. Why, and how, mixed methods research is undertaken in health services research in England: a mixed methods study. BMC Health Services Research. 2007;7:85. read 10 October 2026. View on BMC Health Services Research
- O'Cathain A, Murphy E, Nicholl J. The quality of mixed methods studies in health services research. Journal of Health Services Research & Policy. 2008;13(2):92-98. read 10 October 2026. View on Journal of Health Services Research & Policy
- Creswell JW, Plano Clark VL. Designing and Conducting Mixed Methods Research. 3rd ed. SAGE Publications, 2017. read 10 October 2026. View publisher record
- Fetters MD, Curry LA, Creswell JW. Achieving integration in mixed methods designs-principles and practices. Health Services Research. 2013;48(6 Pt 2):2134-2156. read 10 October 2026. View on Health Services Research
- O'Cathain A, Murphy E, Nicholl J. Three techniques for integrating data in mixed methods studies. BMJ. 2010;341:c4587. read 10 October 2026. View on BMJ
- Bryman A. Integrating quantitative and qualitative research: how is it done? Qualitative Research. 2006;6(1):97-113. read 10 October 2026. View on Qualitative Research
- Guetterman TC, Fetters MD, Creswell JW. Integrating Quantitative and Qualitative Results in Health Science Mixed Methods Research Through Joint Displays. Annals of Family Medicine. 2015;13(6):554-561. read 10 October 2026. View on Annals of Family Medicine
- Pluye P, Hong QN. Combining the Power of Stories and the Power of Numbers: Mixed Methods Research and Mixed Studies Reviews. Annual Review of Public Health. 2014;35:29-45. read 10 October 2026. View on Annual Review of Public Health
- Curry LA, Nembhard IM, Bradley EH. Qualitative and Mixed Methods Provide Unique Contributions to Outcomes Research. Circulation. 2009;119(10):1442-1452. read 10 October 2026. View on Circulation
- O'Cathain A. Mixed Methods Research in the Health Sciences: A Quiet Revolution. Journal of Mixed Methods Research. 2009;3(1):3-6. read 10 October 2026. View on Journal of Mixed Methods Research
Every source above is a peer-reviewed article, a publisher’s record of one, or a standard issued by the body that owns it. Each carries the date it was last read. Where a standard moves, this article is revised rather than left up with a note.
How to cite this article
Pick your referencing style and copy it. Your university or target journal may vary the punctuation — check its guide if it has one.
Pubrica Academy. (2026, October 10). The Design Quality of Mixed-Methods Studies in Health Care Research: Evidence from an Empirical Audit. Pubrica. https://pubrica.com/academy/research-services/effectiveness-of-mixed-method-studies-in-health-care-research/
Pubrica Academy. “The Design Quality of Mixed-Methods Studies in Health Care Research: Evidence from an Empirical Audit.” Pubrica, 10 Oct. 2026, pubrica.com/academy/research-services/effectiveness-of-mixed-method-studies-in-health-care-research/.
Pubrica Academy (2026) The Design Quality of Mixed-Methods Studies in Health Care Research: Evidence from an Empirical Audit. Pubrica. Available at: https://pubrica.com/academy/research-services/effectiveness-of-mixed-method-studies-in-health-care-research/ (Accessed: 15 October 2026).
Pubrica Academy. “The Design Quality of Mixed-Methods Studies in Health Care Research: Evidence from an Empirical Audit.” Pubrica. October 10, 2026. https://pubrica.com/academy/research-services/effectiveness-of-mixed-method-studies-in-health-care-research/
Pubrica Academy, ‘The Design Quality of Mixed-Methods Studies in Health Care Research: Evidence from an Empirical Audit’ (Pubrica, 10 October 2026) <https://pubrica.com/academy/research-services/effectiveness-of-mixed-method-studies-in-health-care-research/> accessed 15 October 2026.
Pubrica Academy. The Design Quality of Mixed-Methods Studies in Health Care Research: Evidence from an Empirical Audit. [Internet]. Pubrica; 2026 Oct 10 [cited 2026 Oct 15]. Available from: https://pubrica.com/academy/research-services/effectiveness-of-mixed-method-studies-in-health-care-research/
Pubrica Academy, “The Design Quality of Mixed-Methods Studies in Health Care Research: Evidence from an Empirical Audit,” Pubrica, Oct. 10, 2026. [Online]. Available: https://pubrica.com/academy/research-services/effectiveness-of-mixed-method-studies-in-health-care-research/
@misc{/la_mixed_methods_studies_health_care_research_quality_2026,
author = {{Pubrica Academy}},
title = {The Design Quality of Mixed-Methods Studies in Health Care Research: Evidence from an Empirical Audit},
howpublished = {Pubrica},
year = {2026},
month = {oct},
url = {https://pubrica.com/academy/research-services/effectiveness-of-mixed-method-studies-in-health-care-research/},
note = {Accessed: <span data-cite-acc="iso">2026-10-15</span>}
}This article is written to be learned from. Cite it if you are referring to what it says; if you are using the evidence behind it, cite the sources listed above instead.
Mixed-Methods Design Quality Checklist
This checklist is intended for use when planning or appraising a mixed-methods health research study.
[ ] Written an explicit rationale for using a mixed-methods design over a single-method approach [ ] Named the design type (convergent, explanatory sequential, exploratory sequential, embedded) [ ] Specified the priority between quantitative and qualitative components [ ] Specified the sequence (concurrent or sequential) of data collection [ ] Planned the specific integration step and point at which findings will be combined [ ] Given the qualitative component the same level of methodological planning detail as the quantitative component
Pubrica AcademyWritten and maintained by Pubrica’s subject matter experts. Articles are revised when the underlying standards change, and each article states the date on which it was last reviewed.
How Pubrica can help
Pubrica’s mixed-methods research support team assists in structuring study designs with an explicit rationale, sequencing and a planned integration step, and reviews draft reports against the same quality criteria.