Grey Literature in Systematic Reviews: What Counts, Where to Search and How to Report It (PRISMA 2020 / PRISMA-S)

The Luxembourg definition (1997, expanded in New York in 2004) describes grey literature as material produced at all levels of government, academia, business and industry, in print or electronic form, that is not controlled by commercial publishers. The Prague definition (2010) adds that it is protected by intellectual property rights and of sufficient quality to be collected by libraries or repositories [4]. Common examples:

• Technical reports and white papers
• Government reports and HTA reports from agencies
• Clinical trial registries: records and results
• Regulatory documents, including clinical study reports
• Conference abstracts and posters
• Theses, dissertations and preprints

Grey literature is not low quality, but it is usually not peer reviewed, so it needs appraisal. Cochrane states that a search of MEDLINE alone is not adequate and warns that restricted searches introduce publication and language bias [5].

A Cochrane methodology review of five studies found that published trials showed on average a 9% greater treatment effect than trials in the grey literature (ratio of odds ratios 1.09, 95% CI 1.03 to 1.16). The most common grey types were abstracts (55%) and unpublished data (30%) [1]. A Health Technology Assessment monograph found that studies with significant results tended to be published earlier, and that published studies tended to report greater effects than unpublished studies in the grey literature [6]. A systematic review limited to journal articles can overstate benefit.

• Is the question about intervention effects? Unpublished trials are a real risk, and registries are the first stop [1, 6].
• Is the topic fast-moving or policy-led? Reports and preprints may be the main evidence.
• Does the review feed a regulatory or HTA dossier? Regulatory and agency documents are then likely sources; see our guide to the EU HTA Regulation.
• Record the decision. Write the reasoning into the protocol, including why if you decide not to search grey sources.

Match the source to the gap you are trying to close (Table 1). Registries show which trials started, even when no paper followed; regulatory documents show what was submitted to authorities; agency reports show how national bodies judged the evidence. OpenGrey was discontinued as a live service on 1 December 2020 and archived [7].

Suppose your question is whether an intervention reduces a clinical outcome. Start with registries, then regulatory documents for authorised products, then conference abstracts and preprints. Match each registry record to a publication, or note that none was found: trials with no results are themselves a finding about selective publication. For a policy-led question, start with agency reports and domain-limited web searches. Write the plan into the protocol before searching, so the choice of sources is defensible.

Keep a search log with one row per source: the source and its platform or URL, the date searched, the terms or browsing path, any limits, the number of records or hits screened, and who ran it. These fields map onto PRISMA-S items 1 to 4, 7 to 9, 13 and 15 [3]. Three habits save time:

• Set a stopping rule in the protocol for web searches, such as how many results you will screen per search.
• Save what you found. Web pages change, so keep a dated copy or PDF.
• Peer review the strategy with the PRESS checklist where you can [8].

Example row: DRKS; registry search page; date; German and English condition and intervention terms; no limits; hits screened; searcher’s initials. One row per source documents a systematic literature search.

Screen grey records against the same eligibility criteria as database records. When the item is a study, such as a trial in a registry, appraise it with the tool you use for published studies. For reports and web documents, use the AACODS checklist: Authority, Accuracy, Coverage, Objectivity, Date and Significance [9]. Watch for double counting: one trial can appear as a registry record, a conference abstract and a paper, so link all reports to one study [2]. Record each AACODS judgement in the extraction sheet.

PRISMA 2020 does not make grey literature searching compulsory. It asks you to report every information source and the search strategy (items 6 and 7) and to show records from databases and registers separately from those found by other methods in the flow diagram [2]. PRISMA-S asks for the detail [3]:

• Study registries searched (item 3) and websites or proceedings browsed (item 4)
• Citation searching (item 5) and contacts with authors or manufacturers (item 6)
• Other methods (item 7), full strategies (item 8), dates of searches (item 13) and total records (item 15)

Cite included grey items in the reference list like any source: author or organisation, year, title, URL and access date.

Table 2 lists the standards that apply to grey literature in systematic reviews.

Across a review, the literature search and study retrieval take less time than study selection, data extraction and appraisal [10], but grey searching still needs its own budget, because no single database does for grey literature what MEDLINE does for journals. Registry searches are quick to run but slow to reconcile with publications.

• Citing OpenGrey as a live database.
• Searching registries without recording the registry name and date (PRISMA-S items 3 and 13).
• Keeping no search log, so the search cannot be reproduced.
• Counting one trial several times because registry, abstract and paper were not linked.
• Including grey items without appraisal.
• Writing “Google” as a source without the search string, domain limits and date.

• DRKS. The German Clinical Trials Register is the WHO primary register for Germany, recognised since October 2008 and now run by the BfArM [11].
• G-BA and IQWiG. Benefit assessments and dossier assessments under the German early benefit assessment are national HTA documents worth searching; see our IQWiG General Methods guide.
• ZB MED. The LIVIVO portal searches life-sciences literature and can supplement database searches.
• Language. Use German search terms as well as English ones, because restricting by language can introduce bias.