Are Retracted Studies Hiding in Your Systematic Review? A Practical Guide to Screening Against Retraction Watch

High-Impact Journals

Interesting topics

Are Retracted Studies Hiding in Your Systematic Review? A Practical Guide to Screening Against Retraction Watch

Systematic reviews and meta-analyses are meant to give us dependable information from existing literature. Nevertheless, conclusions of such reviews can be compromised by the retraction of one of the studies included in the review. Retracted studies in systematic review can introduce concerns about systematic review quality, particularly when the retracted article contributed data to the evidence synthesis. A retracted article usually contains information that can be considered either unreliable, erroneous or even unethical. According to the Committee on Publication Ethics (COPE), retraction represents a procedure that allows correcting mistakes in the scientific record and notifying readers about the possible unreliability of previously published findings.[1]

It is particularly critical since the paper may be retracted after the systematic review is completed. A systematic review and meta-analysis completed in 2025 showed that retracted studies were included in evidence syntheses and their removal changed results or interpretation in some cases. Hence, it means that systematic review authors need to continue screening included studies for retraction information not only during inclusion but also after.

1. Why Retracted Studies Matter in Systematic Reviews

Retraction of a research study does not mean that it becomes irrelevant for every review. The key factor here is whether the retracted paper contributed any data or conclusions to the evidence synthesis. Effective citation screening is therefore important for maintaining literature review accuracy and ensuring that unreliable findings do not influence the review.[2]

Possible consequences are:

Biased pooled effects
Incorrect confidence intervals
Biased conclusions
Loss of confidence in the evidence
Persistence of citations of unreliable results

Depending on the contribution of the paper to the analysis, the consequences can be significant or insignificant. In the case where retracted paper is the only one proving certain outcomes, its exclusion from the analysis can significantly affect the overall conclusion. Explain that implications depend on reason for retraction and affected study components.

For example, recently Cochrane investigated and discovered that about 1% of their 9,500 systematic reviews contained retracted papers in the analysis.

2. What Is the Retraction Watch Database?

Retraction Watch Database (RWDB) is one of the primary databases for locating retracted publications. The database has been made available publicly via Crossref and includes data on retracted publications and related publication notices. The search can be performed with the help of identifiers or bibliographic data such as DOI, PMID, title, author, journal, publisher, and date range.[3]

Nevertheless, it would be advisable to regard Retraction Watch as an element of the verification process rather than a sole source.

The researchers are to check:

Original publisher/journal website;
Metadata from Crossref;
Publications on PubMed or similar databases for different disciplines;
The retraction notice itself.

According to a comparison of databases performed in 2025, the Retraction Watch Database had better overall coverage of retracted publications compared to PubMed and Web of Science in the medical disciplines considered. Such verification supports systematic review quality and helps researchers avoid citing retracted papers in evidence synthesis.

3. When Should Researchers Screen for Retractions?

Retraction screening should be incorporated into several stages of the review lifecycle rather than performed only once.

Review Stage	Retraction Screening Action	Purpose
Initial searching	Check retrieved records and identifiers	Detect already-retracted studies
Full-text screening	Verify publication status	Confirm eligibility
Data extraction	Recheck included studies	Identify newly issued notices
Statistical analysis	Verify studies contributing to models	Protect pooled estimates
Pre-publication	Perform final retraction search	Capture recent changes
Review update	Repeat database and publisher checks	Maintain evidence currency

Rewrite to say retraction screening can be incorporated and documented, not that PRISMA mandates it. A retraction-screening procedure can therefore be incorporated into the review’s search and study-selection documentation. This approach supports PRISMA guidelines, citation screening, and overall evidence synthesis quality.

4. A Practical Retraction Screening Workflow

Step 1: Prepare a Master List

Prepare a master list that contains the following elements:

Retrieval/deduplication → full text → extraction → synthesis → pre-publication → update

The unique identifier is especially important because the title and name of authors can differ between databases.

Step 2: Perform a Retraction Watch Search: Perform a search at the Retraction Watch Database using the information like DOI, PMID, title, author, and journal. Note whether an article is flagged as retracted and identify the relevant retraction notice.

Step 3: Check Publisher’s Webpage: It is not enough to look for the retraction information in the database. Visit the publisher’s page to see whether a retraction notice exists.

Step 4: Establish Contribution of the Study

If an article was retracted, determine whether it:

Contributes data to meta-analysis
Affected individual outcome
Is cited only as background information
Was excluded from quantitative synthesis already.

This step is particularly important for maintaining meta-analysis quality control and determining whether the retracted study systematic review issue has an actual analytical impact.[4]

5. Assessing the Impact of a Retracted Study

Once a retracted study is identified, researchers should conduct an impact assessment rather than simply deleting the reference.

Assessment Question	Why It Matters
Was the study included in the meta-analysis?	Determines direct analytical impact
Where was the paper used?	Identifies whether it was used in the meta-analysis, narrative synthesis, background, methods, or elsewhere
Did its data enter the synthesis?	Confirms whether the paper contributed data to a pooled estimate or other formal synthesis
Which outcomes used its data?	Identifies affected findings
What statistical weight did it receive?	Estimates influence on pooled results

A sensitivity analysis excluding the retracted study can help determine whether the pooled estimate, confidence interval, heterogeneity, or overall interpretation changes.

The 2025 JAMA Internal Medicine investigation actual 50 reviews / 62 retracted studies / 74% post-publication / 11% significance-change / 42% ≥10% primary-effect change findings. It also reinforces the importance of systematic review quality, literature review accuracy, and meta-analysis quality control when evaluating evidence affected by retractions.

6. Current Trend: Automated Retraction Screening

Retraction screening is becoming increasingly automated and semi-automated, rather than manual. There have been recent attempts to develop automation techniques for detecting problematic citations and incorporating retraction data into evidence-synthesis processes. One such attempt was Problematic Paper Screener used in conjunction with Retraction Watch and publisher confirmation to find problematic or retracted citations.

Cochrane Update with Cochrane’s 2026 framework, tagging and post-publication management. [5]

It seems clear that, in the future, systematic review processes may include:

Bibliographic database → Retraction database → Automated alert → Manual confirmation → Impact assessment monitoring + automated alerts + Problematic Paper Screener + Cochrane 2026

7. Recommended Retraction-Screening Checklist

Check	Recommended Action
DOI/PMID verification	Confirm the exact publication
Retraction Watch search	Check current retraction status
Publisher verification	Locate the official notice
Data contribution	Identify affected outcomes
Sensitivity analysis	Recalculate where appropriate
Conclusion review	Determine whether interpretation changes
Documentation	Record the decision and rationale
Final screening	Repeat before publication/update

This checklist provides a practical approach to maintaining systematic review quality, meta-analysis quality control, and literature review accuracy throughout the review lifecycle.

Connect with us to explore how we can support you in maintaining academic integrity and enhancing the visibility of your research across the world!

Conclusion

Retracted studies may pose an indirect threat to the quality of systematic reviews, especially in cases where retraction takes place after the completion of evidence synthesis. It is evident that there is a real issue at stake, which is further proved by new workflows from institutions like Cochrane.

There is an online service called Retraction Watch, which helps in determining the existence of retracted articles. However, screening must involve the use of both databases searching and the verification of publishers and the influence of the study on the review. Integrating these steps into the systematic-review workflow will assist researchers in recognizing unreliable evidence and carrying out sensitivity analysis.

Are Retracted Studies Hiding in Your Systematic Review? A Practical Guide to Screening Against Retraction Watch. Our Pubrica consultants are here to guide you. [Get Expert Publishing Support] or [Schedule a Free Consultation]

Frequently Asked Questions (FAQs)

1. What are retracted studies in a systematic review?

Retracted studies are published research papers that have been officially withdrawn or marked unreliable by the journal or publisher. If such studies are included in a systematic review or meta-analysis, they may affect the reliability of the evidence synthesis.

2. Why should researchers screen systematic reviews for retracted studies?

Retraction screening helps protect systematic review quality, identify unreliable evidence, and prevent retracted findings from influencing pooled estimates, conclusions, or interpretations.

3. What is the Retraction Watch Database?

The Retraction Watch Database is a resource used to identify retracted publications and related notices. Researchers can search using information such as DOI, PMID, title, author, journal, publisher, or date range.

4. When should researchers check for retractions?

Retraction screening should be performed at multiple stages, including initial searching, full-text screening, data extraction, statistical analysis, before publication, and during systematic-review updates. This helps identify studies that may have been retracted after the review began.

5. What should researchers do if a study included in a meta-analysis is retracted?

Researchers should assess the study’s contribution to the analysis rather than simply deleting the reference. They should determine which outcomes used its data and conduct a sensitivity analysis where appropriate to evaluate whether the pooled estimate, confidence interval, heterogeneity, or conclusions change.

References

Graña Possamai, C., Cabanac, G., Perrodeau, E., Ghosn, L., Ravaud, P., & Boutron, I. (2025). Inclusion of Retracted Studies in Systematic Reviews and Meta-Analyses of Interventions: A Systematic Review and Meta-Analysis. JAMA internal medicine, 185(6), 702–709. https://doi.org/10.1001/jamainternmed
Shahraki-Mohammadi, A., Keikha, L., & Zahedi, R. (2024). Investigate the relationship between the retraction reasons and the quality of methodology in non-Cochrane retracted systematic reviews: a systematic review. Systematic reviews, 13(1), 24. https://doi.org/10.1186/s13643-023-02439-3
Barton, A. T., Cordero-Coma, M., Gallagher, M. J., & Yilmaz, T. (2021). The application of the retraction watch database in eye research-based studies. Indian journal of ophthalmology, 69(12), 3767. https://doi.org/10.4103/ijo.IJO1
Schneider, J., Woods, N. D., Proescholdt, R., & RISRS Team (2022). Reducing the Inadvertent Spread of Retracted Science: recommendations from the RISRS report. Research integrity and peer review, 7(1), 6. https://doi.org/10.1186/s41073-022-00125-x
Fletcher, A. H. A., & Stevenson, M. (2025). Predicting retracted research: a dataset and machine learning approaches. Research integrity and peer review, 10(1), 9. https://doi.org/10.1186/s41073-025-00168-w