Finding What Works in Health Care: Standards for Systematic Reviews

Systematic reviews (SRs) of comparative effectiveness research (CER) have become a resource for healthcare decision-makers who are looking for the best evidence to support clinical judgments. They help them understand what is known and what is unknown about the potential advantages and disadvantages of alternative drugs, devices, and other healthcare services. An SR is scientific research that focuses on a specific subject and uses clear, predefined scientific processes to discover, select, appraise, and synthesize the findings of related but distinct investigations. Depending on the data available, it may involve a quantitative synthesis (meta-analysis). Although the relevance of SRs is growing, the quality of published SRs is diverse and frequently inadequate. In many circumstances, the reader cannot assess the quality of an SR because the techniques are inadequately disclosed, and even if methodologies are provided, they may be misused, such as in meta-analysis. However, many assessments fail to analyze the underlying research’s quality and reveal funding sources. Finally, many contradictory approaches to evidence hierarchies and grading methodologies for bodies of evidence add to the uncertainty.

Purpose of setting standards

Organizations create standards to define performance expectations and enhance accountability for fulfilling those objectives. Setting standards, particularly for SRs, has the primary goal of minimizing bias in locating, choosing, and interpreting evidence. The committee defined an SR “standard” for this study as a method, action, or technique necessary for creating scientifically sound, transparent, and repeatable SRs. A standard may be backed by scientific evidence, a reasonable assumption that the standard would assist in achieving the expected level of quality in an SR, or widespread acceptance of the practice among SR writers.

Fundamentals of Systematic Reviews

Many of the main characteristics of a high-quality SR are agreed upon by experts. An SR’s goal is to address a specific research question by following a predefined methodology to locate, select, analyze, and synthesize the findings of related but distinct investigations. A quantitative synthesis is frequently included in SRs but is not required (meta-analysis). The SR process may be broken down into six steps:

Step 1: Begin the process by organizing the review team, developing a strategy for obtaining user and stakeholder feedback, formulating the research question, and implementing procedures to reduce the influence of bias and conflict of interest.

Step 2: Create the review protocol, which includes the context and justification for the review, as well as precise protocols for the search strategy, data collection and extraction, qualitative and quantitative data synthesis (if a meta-analysis is performed), reporting, and peer review.

Step 3: Locate, screen, and select studies for evaluation systematically.

Step 4: Evaluate the possibility of bias in particular research and collect data for analysis.

Step 5: Synthesize the results and evaluate the body of evidence’s overall quality.

Step 6: Prepare a final report and submit it for peer review.

Reference:

1. Morton, Sally, et al., eds. “Finding what works in health care: standards for systematic reviews.” (2011).