How to evaluate bias in meta-analysis within meta-epidemiological studies?

Introduction

Meta-analysis is a type of statistical approach which synthesizes results from different studies and the final result serves as a much stronger evidence than the one collected from an individual study. It gives an estimate of the success of a newly introduced treatment/ intervention or the risk factors associated with a disease/ line of treatment(Hayden et al., 2021). Thus, it can serve as the best source for evidence-based clinical studies. The studies used in meta-analysis can combine results from systematic review, randomised controlled trials (RCT) etc. Meta epidemiological studies is a new type of method which helps in closing the gap between trials and practice and is a much improved version of systematic review(Page, 2020). They adopt either systematic review or meta-analysis approach and aims to understand the impact of certain factors on the outcome. Thus, they try to confirm or nullify the hypothesis in question. The object of analysis is a study and not a patient or an individual. Results of meta-epidemiological study might be directly related to exposure but can also be a result of an alternative effect that might have impacted the overall study outcome. These alternative effects can be a random error, a bias that can produce incorrect results(Steenland et al., 2020). Due to these effects, sometimes an association is falsely accounted for in the outcome when it is not present and on the other hand, sometimes an association is overlooked even in its presence.

Bias in meta-analysis within meta-epidemiological studies

In some meta epidemiological studies, the effect of interventions in RCT’s (Randomised Controlled Trials) can be misunderstood leading to underestimation or overestimation of the intervention (Christensen and Berthelsen, 2020). There can be several reasons which have been elaborated bellow-

Bias arising due to randomisation- The procedure of sequence generation or allocation concealment might vary the effects of the introduced interventions. These two factors also affects the in between heterogeneity.

Bias arising due to opting for unintended interventions- This type of bias arises when the participant opts for an intervention different from which they have been randomly allotted for.

Bias arising due to lack of proper outcome data- The exaggeration of the intervention effect can arise when the data of outcomes are either not completely/ falsely reported. There are some examples when there is overestimation and underestimation of the intervention effect even when the outcome has been properly recorded This is caused due to attrition, but the average bias reported due to attrition could not be combined as the definition of attrition differs across studies.

Bias arising due to improper result selection- There has been reports of bias when the outcomes are not properly generated due to discrepancies between results and methods.

Bias arising due to incorrectly measuring outcomes- Due to lack of proper outcome accessors, bias arises in properly measuring the outcomes. This results in improper estimation of intervention effects.

In most meta-epidemiological studies, a written protocol for selecting the studies need to be framed before conducting the meta analysis. It is important to include all the related studies as missing out on one can introduce bias and makes the study less effective (Pan et al., 2020). The protocol must focus on the selection criteria (eligibility criteria, type of studies to be included, etc.) of the studies to reduce section bias. Fig 1 depicts a flowchart of selecting studies.

Fig 1: Flowchart for selection of studies

Alongside these, the other important points to be included in the protocol are objectives of the study, hypothesis to be tested etc.(Steenland et al., 2020). According to some authors, it can be quite tricky to combine different study designs of meta-epidemiological studies in a meta-analysis and thus have stated “a meta-analysis may give a precise estimate of average bias, rather than an estimate of the intervention’s effect” and that “heterogeneity between study results may reflect differential biases rather than true differences in an intervention’s effect”. In order to understand the amount of bias that might have impacted the study outcome, it has been unanimously agreed upon that all the non-randomized and observational studies included in the meta-analysis should be assessed(Puljak et al., 2020). But there has been no proper agreement on the guidelines of assessing the risk of bias in different meta-analyses(Mathur and VanderWeele, 2021).

Meta epidemiological studies helps in overcoming the challenges of systematic reviews. Out of all, it focuses to get rid of publication bias. Publication bias is also an important type of bias that stresses upon the fact that the data used in meta-epidemiological studies should also be drawn upon from unpublished study sources(Lin, 2020). It is sometimes observed that few studies are not accepted for publishing as they report negative results. Thus, missing out on these can enhance the risk of bias and can give a false impression about the effectiveness of the interpretation(Tan et al., 2021).

Conclusion

The bias which arises during different steps of the meta-analysis must be addressed as this might report contradictory results. It must be noted that false reports can impact medical research which can be fatal in few aspects. The problem with meta-epidemiological study lies in the fact that when the number of studies reduces, the statistical power also reduces.

References

1. Hayden, J.A., Ellis, J., Ogilvie, R., Boulos, L., Stanojevic, S., 2021. Meta-epidemiological study of publication integrity, and quality of conduct and reporting of randomized trials included in a systematic review of low back pain. J. Clin. Epidemiol. 134, 65–78. https://doi.org/10.1016/j.jclinepi.2021.01.020
2. Lin, L., 2020. Hybrid test for publication bias in meta-analysis. Stat. Methods Med. Res. 29, 2881–2899. https://doi.org/10.1177/0962280220910172
3. Mathur, M.B., VanderWeele, T.J., 2021. Estimating publication bias in meta-analyses of peer-reviewed studies: A meta-meta-analysis across disciplines and journal tiers. Res. Synth. Methods 12, 176–191. https://doi.org/10.1002/jrsm.1464
4. Page, M.J., 2020. Controversy and Debate on Meta-epidemiology. Paper 4: Confounding and other concerns in meta-epidemiological studies of bias. J. Clin. Epidemiol. https://doi.org/10.1016/j.jclinepi.2020.03.022
5. Puljak, L., Makaric, Z.L., Buljan, I., Pieper, D., 2020. What is a meta-epidemiological study? Analysis of published literature indicated heterogeneous study designs and definitions. J. Comp. Eff. Res. 9, 497–508. https://doi.org/10.2217/cer-2019-0201
6. Steenland, K., Schubauer-Berigan, M.K., Vermeulen, R., Lunn, R.M., Straif, K., Zahm, S., Stewart, P., Arroyave, W.D., Mehta, S.S., Pearce, N., 2020. Risk of bias assessments and evidence syntheses for observational epidemiologic studies of environmental and occupational exposures: Strengths and limitations. Environ. Health Perspect. https://doi.org/10.1289/EHP6980
7. Tan, R.Q.X., Li, W.T.V., Shum, W.Z., Chu, S.C., Li, H.L., Shea, Y.F., Chung, T.W.H., 2021. A systematic review and meta-analysis protocol examining the clinical characteristics and epidemiological features of olfactory dysfunction (OD) in coronavirus disease 2019 (COVID-19). Syst. Rev. 10, 1–6. https://doi.org/10.1186/s13643-021-01624-6
8. Christensen, R., & Berthelsen, D. B. (2020). Controversy and Debate on Meta-epidemiology. Paper 3: Causal inference from meta-epidemiology: a reasonable goal, or wishful thinking?. Journal of clinical epidemiology, 123, 131.
9. Pan, Y. C., Chiu, Y. C., & Lin, Y. H. (2020). Systematic review and meta-analysis of epidemiology of internet addiction. Neuroscience & Biobehavioral Reviews.