Free scoping read. A written assessment of your manuscript in two business days, at no cost.

Start one
Contents

Academy · Research Data, Analytics & AI · QUADAS-2 Applied Examples

Examples of QUADAS-2 Risk-of-Bias Assessments in Published Diagnostic Accuracy Reviews

A meta-research study of diagnostic accuracy systematic reviews in artificial intelligence found that most reviews reporting the use of QUADAS-2 applied it inconsistently, with the patient-selection and flow-and-timing domains being the most frequently and most unevenly assessed.

Pubrica AcademyWritten by a diagnostic accuracy research writer Reviewed bya systematic review methodologist Published10 October 2026 Last checked10 October 2026 Reading time7 minutes Sources11 peer-reviewed

QUADAS-2 (Quality Assessment of Diagnostic Accuracy Studies, version 2) is applied by producing a risk-of-bias and applicability table for every study included in a diagnostic accuracy systematic review, rating each study as at low, high or unclear risk across four domains (patient selection, index test, reference standard, and flow and timing) and then typically presenting that table as a graphical summary alongside the pooled findings of the review. Examination of how reviews have applied the tool in practice, rather than only of its specification, demonstrates both its practical value and the points at which consistent application is more difficult than it appears, particularly for newer study types such as those evaluating artificial intelligence diagnostic systems.

QUADAS-2 in practice at a glance

Tool
QUADAS-2, the 2011 revision of the 2003 QUADAS tool for diagnostic accuracy studies1
Four domains
Patient selection, index test, reference standard, and flow and timing1
Judgements
Risk of bias in all four domains; concerns about applicability in the first three1
Four phases
Summarise the review question, tailor the tool, draw a flow diagram for each study, then judge1
Published examples
A meta-research audit of 50 AI diagnostic reviews, and a Cochrane review of 155 rapid antigen test cohorts211
Reading time
About 11 minutes, with 3 figures and 2 tables

Learning objectives

  • The presentation and use of a completed QUADAS-2 assessment within a systematic review
  • The findings of a meta-research audit of AI diagnostic accuracy reviews on the consistency with which QUADAS-2 was applied
  • The QUADAS-2 domains that review teams found most difficult to apply consistently in practice

Common misconceptions

  • “Citing QUADAS-2 in a systematic review’s methods section means the tool was applied rigorously and consistently.” A meta-research study of diagnostic accuracy systematic reviews evaluating artificial intelligence tools found that, among reviews reporting the use of QUADAS-2, application was frequently inconsistent, with reviewers varying in which signalling questions they answered, how they resolved disagreements between assessors and how thoroughly they justified their domain-level risk-of-bias judgments.

Components of a completed QUADAS-2 assessment

A properly completed QUADAS-2 assessment produces, for every included study, a set of answers to the signalling questions and a resulting risk-of-bias judgment (low, high or unclear) for each of the four domains, together with applicability judgments for the first three domains; systematic reviews typically present this as a tabular or graphical summary (a “traffic light” plot is a common format) displaying the judgments for all included studies side by side1. The original QUADAS-2 development paper illustrates this with worked examples from the piloting of the tool, showing how specific signalling questions, such as whether the patient sample was consecutive or random and whether the results of the reference standard were interpreted without knowledge of the results of the index test, are answered and then used to justify a “high risk of bias” or “low risk of bias” judgment at domain level, rather than the judgment being made directly without that intermediate step1. This structured format enables readers of the completed review to see not only a summary judgment that a study was at high risk, but also the specific reasoning (through the answered signalling questions) underlying that judgment for each included study1.

1Patient selectionHow participants were recruited, for example whether the sample was consecutive or random. Bias and applicability are both judged.
2Index testHow the test under evaluation was conducted and interpreted. Bias and applicability are both judged.
3Reference standardWhether the reference standard classifies the condition correctly and was interpreted without knowledge of the index test result. Bias and applicability are both judged.
4Flow and timingWhich patients received which tests, the interval between them, and who was left out of the analysis. Bias only.
Figure 1. The four QUADAS-2 domains. Each is rated low, high or unclear risk of bias; the first three are also rated for applicability to the review question.1

What each QUADAS-2 domain asks, and where judgements commonly stall

DomainRated forWhat the signalling questions probeCommon cause of “unclear”
Patient selectionBias and applicabilityWhether the sample was consecutive or randomRecruitment method often not reported3
Index testBias and applicabilityHow the test was conducted and whether its interpretation was independent of the reference standardFor AI tools: overlap between training and test data2
Reference standardBias and applicabilityWhether the reference standard was interpreted without knowledge of the index test resultA single imperfect reference test11
Flow and timingBias onlyWhether all patients received the reference standard, the interval between tests, and exclusions from analysisSequence and interval between tests not reported3

Domain structure from Whiting et al. (2011). The middle column paraphrases the focus of each domain, including the two worked examples discussed in this article; it does not quote the tool. Review teams are expected to tailor the questions to their topic.1

A documented case: QUADAS-2 in artificial intelligence (AI) diagnostic accuracy reviews

A 2022 meta-research study published in npj Digital Medicine examined how systematic reviews of artificial intelligence-based diagnostic tools assessed study quality, including their use of QUADAS-2, across a sample of published systematic reviews of AI diagnostic accuracy2. The study found that quality assessment practices in this emerging literature were inconsistent: some reviews reported using QUADAS-2 but did not report results by domain, some applied it without adapting the signalling questions to AI-specific features (such as the overlap between a model’s training and test datasets and the study’s own evaluation population), and the reporting of inter-rater agreement, where multiple assessors were involved, was often incomplete2. This example is instructive because it shows what inconsistent application involves in practice: citing QUADAS-2 in a methods section does not in itself demonstrate that every domain was assessed, justified and reported with equal rigour2.

Where the concerns sat in two published applications of QUADAS-2

AI reviews: high or unclear risk (of 423 studies)Cochrane antigen review: high risk (of 152 studies)Counts
Patient selection
57.5%
26%
243 · 40
Index test
26%
4%
110 · 6
Reference standard
28.6%
78%
121 · 119
Flow and timing
37.1%
27%
157 · 41

The two columns use different definitions: the AI audit pooled high and unclear ratings; the Cochrane review counted high risk only. Bars are on a 0–100% scale.

Figure 2. Jayakumar et al. (2022) pooled QUADAS-2 ratings from 27 AI diagnostic accuracy reviews; Dinnes et al. (2022) rated 152 evaluations of rapid antigen tests for SARS-CoV-2. Patient selection dominated in the AI literature; the reference standard dominated in the antigen review.211

Two published uses of QUADAS-2, side by side

FeatureAI diagnostic accuracy reviewsRapid antigen tests for SARS-CoV-2
ReviewJayakumar et al., npj Digital Medicine (2022)2Dinnes et al., Cochrane Database of Systematic Reviews (2022)11
TypeMeta-research study of systematic reviewsDiagnostic test accuracy review (second update)
Sample50 AI diagnostic accuracy reviews published 2000 to December 2020155 study cohorts; 152 evaluations of single test applications, 100,462 samples
How QUADAS-2 was used36 of 50 reviews assessed quality; 27 used QUADAS-2, covering 423 primary studiesTwo people independently carried out quality assessment with QUADAS-2
Main concern foundPatient selection: 243 of 423 studies (57.5%) at high or unclear riskReference standard for absence of infection: 119 studies (78%) at high risk; 91% used a single RT-PCR result
Applicability concernsNot summarised in the abstractParticipants differed from intended use in 45 (30%); index test delivery in 47 (31%)
Authors’ conclusionIncomplete uptake and inconsistent reporting; an AI-specific extension may helpAntigen tests vary in sensitivity and are more suitable as triage to RT-PCR testing

All figures as reported in the two abstracts.

Domains most difficult to apply consistently

Methodological commentary on the practical use of QUADAS-2 repeatedly identifies the patient selection domain and the flow and timing domain as those that assessors find most difficult to judge consistently, because both require detailed information about recruitment methods and about the sequence of and interval between the index test and the reference standard; such information is frequently reported incompletely in primary studies, so reviewers judge the risk of bias to be “unclear” more often than definitively “low” or “high”3. This constitutes one of the direct, documented links between the reporting quality of a primary study (as specified by STARD, the Standards for Reporting of Diagnostic Accuracy Studies) and the reliability with which it can later be appraised using QUADAS-2: a primary study that reports its patient flow and recruitment method completely, in STARD-compliant detail, provides reviewers using QUADAS-2 with sufficient information to reach a confident judgment of low or high risk, whereas an incompletely reported study more often necessitates an “unclear” judgment by default34.

Why “unclear” is a reporting signal, not a failure

In the AI audit, ratings of high or unclear risk were most common for patient selection (57.5%) and flow and timing (37.1%), the two domains that depend most on how completely primary studies describe recruitment and test sequence. Primary studies written to STARD 2015 give reviewers what they need to reach a definite low or high rating.23

Implications for the application of QUADAS-2 in systematic reviews

The practical lesson of these documented cases is that sound application of QUADAS-2 entails reporting, and not only performing, the full assessment: publishing the domain-level judgments for every included study, reporting how disagreements between independent assessors were resolved, and adapting or supplementing the signalling questions explicitly when a study type (such as an AI-based diagnostic tool) raises considerations that the original tool was not designed to address21. Where primary studies are reported incompletely, reviewers should record “unclear” rather than conjecture a definitive judgment, and should note in the discussion of the review that this reflects a limitation in the reporting of the underlying literature, not an ambiguity in the criteria of QUADAS-23.

Weak QUADAS-2 reporting

“Study quality was assessed using QUADAS-2. Most studies were of good quality.”

An overall verdict, no domain counts, no word on who assessed or how disagreements were settled.

Stronger QUADAS-2 reporting

“Two reviewers independently applied QUADAS-2 with review-specific guidance. Risk of bias was high for the reference standard in 119 of 152 studies (78%), mainly because a single RT-PCR result defined absence of infection.”

Illustrative wording modelled on the Cochrane antigen review. Domain, count, denominator and reason are all stated.

Reporting checklist for a QUADAS-2 assessment

  • Publish the domain-level judgement for every included study, not an overall score.
  • Make the answered signalling questions available, for example in a supplementary table.
  • State how many assessors rated each study independently and how disagreements were resolved.
  • Document any tailoring of the signalling questions, especially for AI or other non-standard index tests.
  • Record “unclear” where primary reporting is incomplete, and name this as a limitation of the evidence.
  • Report applicability concerns separately from risk of bias for the first three domains.

Contribution of QUADAS-2 ratings to the conclusions of a review

A completed QUADAS-2 table is not intended to stand alone as a reporting formality; well-conducted reviews use it to inform the conclusions they draw. The PRISMA-DTA statement, the diagnostic test accuracy extension of the PRISMA reporting guideline, requires reviews to report how risk-of-bias findings influenced the interpretation of pooled accuracy estimates, for example by indicating whether sensitivity and specificity differed meaningfully between studies rated at low and at high risk of bias in a given domain5. The Cochrane methods group for diagnostic test accuracy reviews recommends investigating risk of bias as a potential source of heterogeneity, treating domain-level QUADAS-2 ratings as candidate explanatory variables in meta-regression or subgroup analyses rather than only as a descriptive table presented separately from the statistical results6. The widely used guidance of Mallett and colleagues on interpreting diagnostic accuracy studies for patient care makes the same point from the clinician’s perspective, advising readers of a systematic review to check whether the authors examined whether studies at higher risk of bias reported systematically different accuracy estimates from studies at lower risk, since a review that pools all studies without this check may report an estimate biased upwards or downwards by precisely the kind of bias that QUADAS-2 was designed to identify7.

An early methodological debate in this literature, concerning whether risk-of-bias information should be condensed into a single overall quality score for each study, was resolved firmly against that approach: the analysis by Whiting, Harbord and Kleijnen found that summary quality scores obscure precisely the domain-specific information that a reader requires to judge which kind of bias, if any, is influencing a given result, which partly explains why QUADAS-2 deliberately reports separate domain judgments rather than a single combined score8.

A further example of QUADAS-2 ratings informing a formal downstream judgment is provided by the GRADE working group’s guidance on assessing the certainty of evidence for diagnostic test accuracy, which instructs guideline developers to treat the proportion of included studies rated at high or unclear risk of bias in the QUADAS-2 domains as one of several factors that may lower the certainty assigned to a body of diagnostic accuracy evidence, alongside inconsistency, imprecision and indirectness9. This example is significant because it demonstrates a concrete, specified downstream use of the QUADAS-2 table beyond its presentation: a guideline panel using GRADE is expected to examine the pattern of domain ratings across included studies and decide, with recorded reasons, whether that pattern is serious enough to downgrade the certainty of a recommendation based on that evidence, which is a substantially more demanding use of the assessment than the inclusion of a traffic-light figure in a results section9.

The original empirical demonstration by Lijmer and colleagues that specific design features predict inflated accuracy estimates is the historical example that first established the credibility of this link between domain and conclusion, since it showed with pooled data, rather than by theoretical argument alone, that the features addressed by QUADAS and subsequently QUADAS-2 are not arbitrary checklist items but documented, quantified predictors of biased results10.

Considered together, these examples (the worked examples from the piloting of the tool, the documented inconsistency in reviews of AI diagnostics and the formal use of domain ratings by the GRADE working group to downgrade certainty) indicate a consistent pattern: QUADAS-2 was developed from empirical evidence of bias, is applied unevenly in current practice and is most effective when its output is treated as an input to further judgment rather than as a final verdict.

1999Design features predict biasLijmer et al., JAMA: specific design features linked to inflated accuracy estimates10
2005No role for quality scoresWhiting, Harbord and Kleijnen, BMC Medical Research Methodology: summary scores hide domain-specific bias8
2011QUADAS-2 publishedWhiting et al., Annals of Internal Medicine: four domains, signalling questions, applicability1
2018PRISMA-DTAMcInnes et al., JAMA: reviews must report how risk of bias affected interpretation5
2020GRADE for test accuracySchünemann et al., Journal of Clinical Epidemiology: QUADAS-2 ratings can lower certainty9
2022Two applications audited and reportedJayakumar et al. (AI reviews) and Dinnes et al. (Cochrane antigen tests)211
Figure 3. From the empirical evidence of bias that QUADAS-2 was built on to the downstream uses of its ratings discussed in this section.

Takeaway

The same tool can surface very different weak points. In AI diagnostic reviews the main problem was who was recruited; in the antigen-test review it was how absence of infection was confirmed. Read the domain pattern, not the headline verdict, before trusting a pooled accuracy estimate.211

Key points

Key points

A completed QUADAS-2 assessment should present, for every included study, the signalling-question answers underlying each domain judgment, not merely an overall rating. Documented practice indicates that the tool is frequently applied inconsistently, particularly for the patient selection and flow and timing domains and for newer study types such as AI-based diagnostics.

Practical application

Practical application

  1. Domain-level judgments and the underlying signalling-question answers should be reported for every included study. An overall risk rating without supporting signalling-question detail does not allow readers to verify the reasoning.
  2. Signalling questions should be adapted explicitly for non-standard study types. When AI-based or other non-traditional diagnostic studies are assessed, the adaptation or supplementation of the base QUADAS-2 questions to address study-specific concerns should be documented.
  3. “Unclear” should be recorded where the reporting of the primary study is incomplete. An unclear judgment should be treated as a reflection of a reporting gap in the primary study rather than as a failure of the assessment.

Frequently asked questions

For each included study: answers to the relevant signalling questions, a risk-of-bias judgment (low, high or unclear) for each of the four domains, and applicability judgments for the first three domains.

It found inconsistent application across the sampled reviews, including incomplete reporting by domain, signalling questions not adapted to AI-specific features, and incomplete reporting of inter-rater agreement.

Patient selection and flow and timing are commonly identified as the most difficult, because they depend on details of recruitment and of test interval and sequence that primary studies often report incompletely.

Independent assessment by at least two reviewers, with a documented method for resolving disagreements, is standard good practice and is one of the elements that meta-research has found to be inconsistently reported.

Yes, reviewers may supplement or adapt the signalling questions for study types that the original tool was not designed to address, provided that the adaptation is documented explicitly in the methods of the review.

References

  1. Whiting PF, Rutjes AWS, Westwood ME, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Annals of Internal Medicine. 2011;155(8):529-536. read 10 October 2026. View on Annals of Internal Medicine
  2. Jayakumar S, Sounderajah V, Normahani P, et al. Quality assessment standards in artificial intelligence diagnostic accuracy systematic reviews: a meta-research study. npj Digital Medicine. 2022;5(1):11. read 10 October 2026. View on npj Digital Medicine
  3. Bossuyt PM, Reitsma JB, Bruns DE, et al. STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies. BMJ. 2015;351:h5527. read 10 October 2026. View on BMJ
  4. Whiting PF, Rutjes AWS, Westwood ME, et al. Sources of variation and bias in studies of diagnostic accuracy: a systematic review. Annals of Internal Medicine. 2004;140(3):189-202. read 10 October 2026. View on Annals of Internal Medicine
  5. McInnes MDF, Moher D, Thombs BD, et al. Preferred Reporting Items for a Systematic Review and Meta-analysis of Diagnostic Test Accuracy Studies: The PRISMA-DTA Statement. JAMA. 2018;319(4):388-396. read 10 October 2026. View on JAMA
  6. Leeflang MMG, Deeks JJ, Takwoingi Y, Macaskill P. Cochrane diagnostic test accuracy reviews. Systematic Reviews. 2013;2:82. read 10 October 2026. View on Systematic Reviews
  7. Mallett S, Halligan S, Thompson M, Collins GS, Altman DG. Interpreting diagnostic accuracy studies for patient care. BMJ. 2012;345:e3999. read 10 October 2026. View on BMJ
  8. Whiting P, Harbord R, Kleijnen J. No role for quality scores in systematic reviews of diagnostic accuracy studies. BMC Medical Research Methodology. 2005;5:19. read 10 October 2026. View on BMC Medical Research Methodology
  9. Schünemann HJ, Mustafa RA, Brozek J, et al. GRADE guidelines: 21 part 1. Study design, risk of bias, and indirectness in rating the certainty across a body of evidence for test accuracy. Journal of Clinical Epidemiology. 2020;122:129-141. read 10 October 2026. View on Journal of Clinical Epidemiology
  10. Lijmer JG, Mol BW, Heisterkamp S, et al. Empirical evidence of design-related bias in studies of diagnostic tests. JAMA. 1999;282(11):1061-1066. read 10 October 2026. View on JAMA
  11. Dinnes J, Sharma P, Berhane S, et al. Rapid, point-of-care antigen tests for diagnosis of SARS-CoV-2 infection. Cochrane Database Syst Rev. 2022;7(7):CD013705. read 13 November 2026. View on the Cochrane Library

Every source above is a peer-reviewed article, a publisher’s record of one, or a standard issued by the body that owns it. Each carries the date it was last read. Where a standard moves, this article is revised rather than left up with a note.

How to cite this article

Pick your referencing style and copy it. Your university or target journal may vary the punctuation — check its guide if it has one.

Pubrica Academy. (2026, October 10). Examples of QUADAS-2 Risk-of-Bias Assessments in Published Diagnostic Accuracy Reviews. Pubrica. https://pubrica.com/academy/original-research-article/give-an-example-of-studies-that-used-the-quadas-2-tool/

This article is written to be learned from. Cite it if you are referring to what it says; if you are using the evidence behind it, cite the sources listed above instead.

QUADAS-2 Reporting Checklist for Reviews

This checklist may be used to confirm that a QUADAS-2 assessment has been reported and not merely performed.

[ ] Reported signalling-question answers, not only bottom-line judgments, for each included study
[ ] Published a full risk-of-bias/applicability table or graphical summary
[ ] Documented number of independent assessors and how disagreements were resolved
[ ] Noted any adaptation of signalling questions for non-standard study types
[ ] Distinguished "unclear" judgments caused by incomplete primary-study reporting from genuine ambiguity in the tool

Pubrica AcademyWritten and maintained by Pubrica’s subject matter experts. Articles are revised when the underlying standards change, and each article states the date on which it was last reviewed.

How Pubrica can help

Pubrica’s systematic review team conducts and fully documents QUADAS-2 assessments, including domain-level signalling-question detail and the resolution of inter-rater disagreement, for diagnostic accuracy reviews.

Find something else

Search the Pubrica Academy

Access is free and requires no account. Enter a topic, or open the full library to filter by category, sub-topic or resource type.

Transparency of QUADAS-2 assessments

A scoping assessment is free and takes two business days. It provides a written assessment of what the work requires and which specialist would undertake it.

Get a free quote

Regulations

Check your project against this rule

  1. 1Your work
  2. 2Your details
  3. 3Send
What do you need?

Material you send is held under signed confidentiality, with access limited to the assigned team and encrypted transfer. It is not used to train anything and is not passed to third-party tools. Ask for the agreement before you send anything you would rather cover first.