What is the risk of bias assessment and different tools used to assess systematic review?

In Brief:

• The risk of bias is defined as the risk of systematic error or a deviation from reporting the truth or an appropriate evidence finding.
• It occurs when there is a systematic flaw or limitations in the study design or the conduct of the study.
• This article describes different assessment tools and the types of study designs for which the tool is applicable along with the flow of how to avoid the risk of bias.

A systematic review guideline will often determine the study design to answer the formulated question, and it is not enough in trusting the evidence of systematic review over observational studies. There are several terms like quality assessment, critical appraisal, or internal validity, which are used for the evaluation of studies supported by the guideline. Whereas in the risk of bias, the bias determines the factor that can affect the observations and findings of the study systematically and report it to be different from the actual conclusion. In other words, a study affected by bias can be inaccurate and thus leads to an inappropriate guideline recommendation. Therefore, inadequate study design or the conduct of the study will give false findings which result in wasting time and resources, and missing opportunity for effective intervention.

Quality assessment or risk of bias assessment helps in regulating and establishing transparency of evidence synthesis (data collection methods, search strategies, etc.) and findings and it is often performed for each study in a review; thus the collected evidence eliminate bias in the outcomes. Generally, a quality assessment outside of a systematic review does not require evidence synthesis method; however, it completely depends upon the utilization of evidence synthesis method. The risk of bias assessment can be done by professionals who may be conducted by the guideline development group or requires a methodological expert or by experiences systematic review researchers. Once the evaluation is done, then it can be used for the synthesis of study results and findings and combined into the complete assessment of the evidence. There are certain common factors which can introduce bias in many research areas, that include

• Selection bias – Participant or Population comparability in a study
• Performance bias – The influence of estimate effect (aspects other than the intervention or exposure of interest)
• Reporting bias – Problems with measurements or classification of outcome (missing information)

The risk of bias is assessed frequently by evaluating the study design and the conduct of the study because it is impossible to determine a specific study has been affected by bias or not.

Different tools used to assess the risk of bias in the systematic review:

1. AMSTAR 2 – A MeaSurement Tool to Assess systematic Reviews
2. AHRQ RRB – AHRQ tool for evaluating the Risk of Reporting Bias
3. AHRQ ORB – AHRQ Outcome and analysis Reporting Bias framework
4. GRADE – Grading of Recommendations Assessment, Development and Evaluation
5. GRADE-CERQual – GRADE Confidence in Evidence from Reviews of Qualitative research
6. SAQAT – Semi-Automated Quality Assessment Tool
7. NMA Quality – Framework for evaluating the quality of evidence from a Network Meta-Analysis
8. NOS – Newcastle-Ottawa Scale
9. AXIS – Appraisal tool for Cross-Sectional Studies
10. QUIPS – QUality In Prognosis Studies tool
11. RoB 1.0 – Cochrane Risk of Bias tool for randomized trials
12. RoB 2.0 – Revised tool for Assessing Risk of Bias in randomized trials
13. SYRCLE RoB – SYstematic Review Centre for Laboratory animal Experimentation Risk of Bias tool
14. SciRAP – Science in Risk Assessment and Policy
15. SIGN – The Scottish Intercollegiate Guidelines Network
16. ROBIS – Risk Of Bias in Systematic Reviews
17. RoBANS – Risk of Bias Assessment Tool for Nonrandomized Studies
18. RoBINS I – Risk of Bias In Non-randomized Studies of Interventions tool
19. ORBIT 1 – Outcome Reporting Bias In Trials classification system for benefit outcomes
20. QUADAS 2 – Quality Assessment of Diagnostic Accuracy Studies
21. PROBAST – Prediction model Risk Of Bias ASsessment Tool
22. ORBIT 2 – Outcome Reporting Bias In Trials classification system for harm outcomes
23. Reid – Selective reporting bias algorithm
24. RTI 2012 – RTI Item Bank for Assessment of Risk of Bias and Precision for Observational Studies of Interventions or Exposures
25. RTI 2013 – RTI Item Bank for Assessing Risk of Bias and Confounding for Observational Studies of Interventions or Exposures
26. JBI – the Joanna Briggs Institute
27. OHAT – Office of Health Assessment and Translation
28. Downs Black – Downs Black tool

How to assess the risk of bias?

1. Plan your approach
2. Identifying an appropriate risk of bias assessment tool
3. Be aware of related issues
4. Appraise each study
5. Report the assessment process
6. Use your appraisals to inform the guideline

Determining an appropriate risk of bias assessment tool:

Study type	Assessment tools	Year	Source
Systematic reviews	AMSTAR – 2	2017	https://amstar.ca/Amstar-2.php
ROBIS	2016	www.bristol.ac.uk/population-health-sciences/projects/robis/
SIGN checklist	2014	https://www.sign.ac.uk/checklists-and-notes.html
Randomized trials	Cochrane RoB 2.0 Tool	2016	www.riskofbias.info
SIGN checklist	2014	https://www.sign.ac.uk/checklists-and-notes.html
Non-randomised studies of interventions (case-control, cohort, etc.)	ROBINS-I	2016	www.riskofbias.info
SIGN checklist	2014	https://www.sign.ac.uk/checklists-and-notes.html
Prognostic: Prognostic factorsRisk prediction modellingOverall prognosis/baseline risk/prevalence/incidence	JBI checklist for prevalence studies	2017	http://joannabriggs.org/research/critical-appraisal-tools.html
PROBAST	2014	www.systematic-reviews.com/probast
QUIPS	2013	https://methods.cochrane.org/prognosis/welcome
Diagnostic	SIGN checklist	2014	https://www.sign.ac.uk/checklists-and-notes.html
QUADAS-2	2011	www.bristol.ac.uk/population-health-sciences/projects/quadas/quadas-2
Qualitative	CASP Qualitative Checklist	2018	https://casp-uk.net/wp-content/uploads/2018/01/CASP-Qualitative-Checklist-2018.pdf
GRADE-CERQual	2017	https://www.cerqual.org/
JBI Checklist for Qualitative Research	2017	http://joannabriggs.org/research/critical-appraisal-tools.html
Observational studies of exposures (human epidemiology, wildlife)	Navigation Guide risk of bias checklist	2018	https://www.cosmin.nl/
OHAT tool	2015	https://ntp.niehs.nih.gov/ntp/ohat/pubs/riskofbiastool_508.pdf
In vivo animal studies	Navigation Guide risk of bias checklist	2018	https://www.cosmin.nl/
SciRAP tool	2018	http://www.scirap.org/
OHAT tool	2015	https://ntp.niehs.nih.gov/ntp/ohat/pubs/riskofbiastool_508.pdf
CRED	2016	www.ecotoxcentre.ch/projects/risk-assessment/cred
SYRCLE tool	2014	Systematic Review Centre for Laboratory Animal Experimentation
In vitro studies	OHAT tool	2015	https://ntp.niehs.nih.gov/ntp/ohat/pubs/riskofbiastool_508.pdf
Obtained from:  https://www.nhmrc.gov.au/guidelinesforguidelines/develop/assessing-risk-bias#

In summary, risk of bias assessment is a tool used to regulate findings which are accurate and appropriate, and it is essential to select the risk of bias tool rightly. There are many researches available to help, and it makes things easier to find appropriate tools for assessing the risk of bias. This article describes different assessment tools for a systematic review and the types of study designs for which the tool is applicable along with the flow of how to avoid the risk of bias.

Pubrica provides support in writing a systematic review and offers you complete support across a variety of journals, publications, and books. We provide complete support from translating your concept to incisive report, and our experts team will be with you through the entire publication stages.

Reference:

1. Bero, L. A. Why the Cochrane risk of bias tool should include funding source as a standard item. Cochrane Database of Systematic Reviews, 2013.
2. Higgins, J. P., J. Savović, et al., Eds. Revised Cochrane the risk of bias tool for randomized trials, 20 October 2016.
3. Marshall, I. J., A. Noel-Storr, et al. Machine learning for identifying Randomized Controlled Trials: An evaluation and practitioner’s guide. Research Synthesis Methods, 2016.
4. Reeves, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomized or non-randomised studies of healthcare interventions, or both. British Medical Journal 358, 2017.
5. Whiting, P., J. Savović, et al. ROBIS: A new tool to assess the risk of bias in systematic reviews was developed. Journal of Clinical Epidemiology 69: 225-234, 2016.