Contents
Academy · Concepts & Definitions · Before you start a review
Before You Start a Systematic Review: The Four Things That Cannot Be Retrofitted
Most systematic review problems trace back to one of four decisions made — or skipped — before the search ever ran. Fix these four first; nothing after them can undo a mistake made here.
Four specific decisions need to be settled before a systematic review's search begins — the question framed with explicit criteria, the protocol registered, the search strategy peer-reviewed, and the screening rules agreed between reviewers — because each one shapes work that follows it and cannot be meaningfully corrected after the fact.
What you will learn
- Why these specific four steps need to happen before the search, not alongside or after it.
- What 'peer-reviewed' means for a search strategy, specifically.
- Why screening rules need to be agreed in advance, not worked out as disagreements arise.
What to unlearn
- “The protocol can be finalised and registered while the search is already running.” Registration is meant to record the protocol before results are known; finalising or registering after the search has started undermines that purpose and can be flagged by some registries.
- “Screening disagreements can be resolved case by case as they come up.” Pre-agreeing the screening rules (how disagreements between two screeners are resolved, what counts as a borderline case) before screening starts keeps the process consistent, rather than having early disagreements resolved differently from later ones simply because the rule wasn't yet settled.
1. Question framed with explicit criteria
The research question needs to be broken into explicit, operational eligibility criteria (commonly via PICO or a design-appropriate variant) before the search strategy can be meaningfully built, since the search terms and the screening criteria both derive from this framing.
2. Protocol registered
The protocol — eligibility criteria, search strategy, and planned analysis approach — should be registered (commonly via PROSPERO for health-related reviews) before the search runs, consistent with the registry's prospective purpose1.
3. Search strategy peer-reviewed
Having the search strategy reviewed by a second information specialist or librarian before it's run — checking the Boolean logic, controlled-vocabulary terms, and database-specific syntax — catches errors that are otherwise expensive to fix (requiring the whole search to be rerun) once screening has already begun on the original result set.
4. Screening rules agreed
Before screening starts, reviewers should agree in advance how disagreements between two independent screeners will be resolved (a third reviewer, discussion to consensus, or another pre-specified rule) and how borderline or ambiguous cases will be handled, so the same kind of case is treated consistently throughout screening rather than resolved differently depending on when it came up.
Take-home message
Take-home message
Framing the question, registering the protocol, having the search strategy peer-reviewed, and agreeing screening rules all need to happen before the search runs — each shapes everything that follows it, and none can be meaningfully retrofitted once screening is underway.
How to apply this to your work
How to apply this to your own work
- Finalise eligibility criteria before building the search strategy. The search terms should derive from the already-settled criteria, not be built in parallel with them.
- Register the protocol before, not during or after, the search. This preserves the prospective purpose of registration.
- Have a second information specialist peer-review the search strategy before running it. This catches errors before they propagate into the full screening workload.
- Agree screening-disagreement rules in writing before screening starts. So every reviewer applies the same resolution process throughout.
Common questions
Minor, justified refinements can sometimes be accommodated and documented transparently, but a substantial change to eligibility criteria after screening has begun generally means the search needs to be reconsidered, which is exactly what this checklist is meant to avoid needing.
An external peer reviewer, a librarian from another institution, or a dedicated search-strategy peer-review service can fill this role when an internal second reviewer isn't available.
PRISMA 2020's reporting items expect the protocol and registration to be reported, and the search methods to be documented in detail, which in practice requires these steps to have genuinely happened beforehand.
References and sources
- PROSPERO International Prospective Register of Systematic Reviews. University of York, Centre for Reviews and Dissemination. read 10 October 2026. View PROSPERO
Every source above is a peer-reviewed article, a publisher’s record of one, or a standard issued by the body that owns it. Each carries the date it was last read. Where a standard moves, this article is revised rather than left up with a note.
How to cite this article
Pick your referencing style and copy it. Your university or target journal may vary the punctuation — check its guide if it has one.
Pubrica Academy. (2026, October 10). Before You Start a Systematic Review: The Four Things That Cannot Be Retrofitted. Pubrica. https://pubrica.com/insights/checklist/before-you-start-a-review-checklist/
Pubrica Academy. “Before You Start a Systematic Review: The Four Things That Cannot Be Retrofitted.” Pubrica, 10 Oct. 2026, pubrica.com/insights/checklist/before-you-start-a-review-checklist/.
Pubrica Academy (2026) Before You Start a Systematic Review: The Four Things That Cannot Be Retrofitted. Pubrica. Available at: https://pubrica.com/insights/checklist/before-you-start-a-review-checklist/ (Accessed: 15 October 2026).
Pubrica Academy. “Before You Start a Systematic Review: The Four Things That Cannot Be Retrofitted.” Pubrica. October 10, 2026. https://pubrica.com/insights/checklist/before-you-start-a-review-checklist/
Pubrica Academy, ‘Before You Start a Systematic Review: The Four Things That Cannot Be Retrofitted’ (Pubrica, 10 October 2026) <https://pubrica.com/insights/checklist/before-you-start-a-review-checklist/> accessed 15 October 2026.
Pubrica Academy. Before You Start a Systematic Review: The Four Things That Cannot Be Retrofitted. [Internet]. Pubrica; 2026 Oct 10 [cited 2026 Oct 15]. Available from: https://pubrica.com/insights/checklist/before-you-start-a-review-checklist/
Pubrica Academy, “Before You Start a Systematic Review: The Four Things That Cannot Be Retrofitted,” Pubrica, Oct. 10, 2026. [Online]. Available: https://pubrica.com/insights/checklist/before-you-start-a-review-checklist/
@misc{before_you_start_a_review_checklist_2026,
author = {{Pubrica Academy}},
title = {Before You Start a Systematic Review: The Four Things That Cannot Be Retrofitted},
howpublished = {Pubrica},
year = {2026},
month = {oct},
url = {https://pubrica.com/insights/checklist/before-you-start-a-review-checklist/},
note = {Accessed: <span data-cite-acc="iso">2026-10-15</span>}
}This article is written to be learned from. Cite it if you are referring to what it says; if you are using the evidence behind it, cite the sources listed above instead.
Four-point pre-search check
Confirm all four before running the search.
[ ] 1. Question framed with explicit, operational eligibility criteria [ ] 2. Protocol finalised and registered (e.g. PROSPERO) [ ] 3. Search strategy peer-reviewed by a second information specialist [ ] 4. Screening-disagreement resolution rule agreed and documented in advance
Pubrica AcademyWritten and maintained by the subject matter experts of Pubrica who do this work. Articles are revised when the standard underneath them moves, and each one carries the date it was last checked.
How Pubrica helps
We confirm all four of these steps are genuinely complete before a systematic review's search is run, including arranging independent peer review of the search strategy.