Free scoping read. A written assessment of your manuscript in two business days, at no cost.

Start one
Contents

Academy · Research Data Analytics & AI · abstract screening

Title and Abstract Screening in Systematic Reviews: Planning, Calibration and Reporting

Title and abstract screening is an exercise in sensitivity rather than precision: at this stage, reviewers exclude only the records they are confident are ineligible and promote all others, because a wrongly excluded study is not reconsidered. The Cochrane Handbook’s indicative rate is 60 to 120 records an hour, roughly 500 to 1,000 over an eight-hour period, which makes the workload calculable in advance and an undisciplined search an expensive one.

Pubrica AcademyWritten by the medical writing and biostatistics teams Reviewed byPubrica biostatistics team Published9 October 2026 Last checked9 October 2026 Reading time12 minutes Sources29 peer-reviewed

Title and abstract screening typically involves two reviewers, several thousand records and a set of eligibility criteria that appeared unambiguous in the protocol but may cease to function by the fortieth abstract. Most of these difficulties can be prevented by three steps taken before screening begins rather than during it: estimating the workload, writing the criteria as operational decision rules, and calibrating both screeners on a shared sample until their disagreements concern the records rather than the meaning of the criteria.

Learning objectives

  • Estimation of screening workload in person-hours before a search strategy is finalized
  • Conversion of eligibility criteria into screening rules with an explicit default for silent records
  • Conduct of a calibration round that corrects the rules rather than attributing disagreement to the screeners
  • Handling of record types that challenge the rules: conference abstracts, companion reports, duplicates, non-English records and retractions
  • Transition to full-text screening with exclusion reasons recorded where they are required
  • Assessment of what priority screening and machine classifiers do and do not change

Common misconceptions

  • “Screening is about excluding irrelevant records efficiently.” The primary aim is to avoid losing eligible records. The two aims conflict, and at the title and abstract stage sensitivity takes priority, because an excluded record is not reviewed again.
  • “One screener is enough if they are experienced.” Cochrane requires at least two people working independently at the study selection and data collection stages. Experience changes which records a single screener misses, not whether they miss any.
  • “Several thousand records can be screened in two days.” The Handbook’s indicative rate is 60 to 120 records per hour, or approximately 500 to 1,000 records over an eight-hour period, per screener, and two screeners are required.
  • “Disagreement between screeners means poor training.” Early disagreement is usually evidence that a criterion is ambiguous. Retraining the screener leaves the ambiguity in place for every subsequent record.
  • “Searching additional databases carries no cost.” Embase alone holds more than 10 million records from approximately 3,000 journals that are not indexed in MEDLINE, so multi-database searching increases the screening load as well as the yield. This trade-off is usually justified, but it remains a trade-off.

Purpose of the screen and the priority of sensitivity

Screening has a single purpose: to reduce the retrieved records to a set small enough to assess at full text without losing any eligible record. The consequences of the two possible errors are not symmetrical. A record wrongly promoted to full text costs ten minutes of reviewer time. A record wrongly excluded at title and abstract is lost — the excluded records are not re-read, and the absence of the study is invisible in the final review.

This asymmetry should govern every decision rule. The operative question for each record is not “is this study eligible?” but “can it be determined with confidence from the title and abstract alone that it is not?” If it cannot, for any reason including that the abstract does not report the relevant information, the record proceeds to full text. This approach is deliberately permissive: the screen is intended to be permissive, and precision is intended to come at the next stage, when the full paper is available 1.

It follows that a screen which excludes a large share of records on criteria that abstracts rarely report is functioning incorrectly. If the eligibility criteria depend on information that abstracts do not carry — a biomarker threshold, an allocation method, a specific outcome timepoint — those criteria belong at full text, and the screening rules should state this explicitly rather than leaving each screener to improvise. The criteria themselves derive from the review question; see framing the question with PICO.

Estimation of screening workload

Screening effort is one of the few elements of a review that can be forecast, and forecasting it informs decisions about the search rather than merely preparing the team for the consequences.

The Handbook gives an indicative screening rate of 60 to 120 records per hour, or approximately 500 to 1,000 records over an eight-hour period 1. Two consequences follow. The rate is per screener, and Cochrane requires at least two people working independently at study selection, so the person-hours are double the single-screener figure. In addition, the range spans a factor of two, depending on the clarity of the rules, the specificity of the topic and the proportion of the record set that is obviously irrelevant; a plan based on the optimistic end of the range therefore has no margin.

The arithmetic should be completed before the search is finalised. A deduplicated set of several thousand records implies several working days of screening per screener, doubled for independence, plus reconciliation. At this point the database decision becomes concrete: Embase holds more than 10 million records from approximately 3,000 journals that MEDLINE does not index, so adding it increases both yield and load 1. Whether that trade-off is worthwhile is a judgement specific to the review question, discussed on our page on sources of review data; what is not defensible is making it inadvertently and discovering the workload afterwards.

If the arithmetic does not fit the team’s capacity, the defensible responses are to narrow the question, to restrict the search and declare the restriction, or to choose a design that permits an abbreviated search — a rapid review or a scoping review rather than a systematic review conducted at half strength.

What the Handbook’s indicative rate implies for a two-screener team

Deduplicated recordsScreener-days at 1,000 per eight-hour dayScreener-days at 500 per eight-hour dayWith two independent screeners
1,0001 day2 days2 to 4 screener-days, plus reconciliation
3,0003 days6 days6 to 12 screener-days, plus reconciliation
5,0005 days10 days10 to 20 screener-days, plus reconciliation
10,00010 days20 days20 to 40 screener-days, plus reconciliation

Arithmetic applied to the Cochrane Handbook’s indicative screening rate of 60 to 120 records per hour, or approximately 500 to 1,000 records over an eight-hour period. These are planning figures, not an empirical finding about any particular review: actual rates depend on the clarity of the rules and the specificity of the topic.

60–120records screened per hour, indicative rate
500–1,000records over an eight-hour period, per screener
2independent screeners required at study selection
10+ millionEmbase records from journals MEDLINE does not index

Screening rates and the multi-database load as given in the Cochrane Handbook for Systematic Reviews of Interventions, Chapter 4, on searching for and selecting studies; the independence requirement is Cochrane’s MECIR standard.

Conversion of eligibility criteria into screening rules

Eligibility criteria are written for the review. Screening rules are written for a fatigued screener assessing a 200-word abstract late in the working day. The two are distinct documents, and the second must be derived deliberately from the first.

A screening rule set has, for each criterion, four parts. The include signal: what in a title or abstract indicates the criterion is met. The exclude signal: what indicates it is definitely not met, phrased strictly enough that a screener will not over-apply it. The stage: whether this criterion is assessable at title and abstract or deferred to full text. And the default: what to do when the record is silent, which at this stage should almost always be to promote.

Two further structural rules are useful. The ordered list should be kept short, since a screener who must consult a long document for every record will cease to consult it. The exclusion categories should also be mutually exclusive and ordered, so that a record excluded for the wrong population is recorded consistently rather than according to whichever criterion the screener happened to check first; this ordering is what makes exclusion counts meaningful later.

A one-line definition should be written for each ambiguous term before screening begins. “Adult”, “chronic”, “usual care”, “multicomponent” and “high risk” are the terms most commonly affected. A five-line glossary prevents prolonged inconsistency.

A screening decision rule, as usually written and as it needs to be

Leaves every borderline call to the screener

Screen titles and abstracts against the inclusion criteria. Include if the study is a randomised trial of exercise in older adults with chronic pain reporting pain or function outcomes. Exclude if not relevant. If unsure, discuss.

“Exclude if not relevant” is not a rule. Relevance is precisely the judgement the rule is intended to constrain, so each screener applies an individual standard.

Inclusion is framed positively. Asking “is this eligible?” makes the screener exclude anything that does not positively demonstrate eligibility — the opposite of what the stage requires.

No stage assignment. Criteria that abstracts rarely report, such as pain duration and comparator detail, will be applied silently and inconsistently at screening.

No default for silent records. Two screeners facing an abstract with no age stated will resolve it differently, repeatedly.

No ordering of exclusion reasons. Two screeners excluding the same record will record different reasons, so the exclusion counts mean nothing.

“If unsure, discuss” scales badly. On several thousand records this converts the screen into a continuous meeting, and the resulting shared understanding is tacit and unwritable.

No glossary. “Older adults” and “chronic” are undefined, so the effective eligibility criteria are whatever each screener assumes.

Decides borderline calls before they arise

Decide each record by working down this list in order and recording the first reason that applies. 1. Not a primary study: exclude as review, editorial, commentary, guideline or case report. Protocols and conference abstracts are NOT excluded here — promote them. 2. Population: exclude only if the record states that participants were all under 60, or that the condition was not musculoskeletal pain. If age is not stated, promote. If pain duration is not stated, promote — duration is a full-text criterion. 3. Intervention: exclude only if the record states that no exercise or physical activity component was delivered. If the intervention is multicomponent, promote; whether the comparator received the same non-exercise components is a full-text criterion. 4. Comparator: do not exclude on comparator at this stage; if no comparator is mentioned, promote. 5. Design: exclude only if the record explicitly states a non-randomised design. If design is not stated, promote. 6. Outcomes: do not exclude on outcomes at this stage. Promote any record you cannot exclude for a reason above, including records with no abstract, records in a language you cannot read, and records whose abstract is uninformative. Terms: “older adults” means 60 years or over; “exercise” includes supervised, prescribed and home-based physical activity programmes; a record “states” something only if it is in the title or abstract — inference from the journal, the authors or your knowledge of the field is not permitted.

Every rule is phrased as a reason to exclude, which keeps the stage sensitive by construction rather than by reminder.

The default for a silent record is stated at every criterion, so the two screeners behave identically without conferring.

Criteria that cannot be assessed from an abstract are explicitly deferred to full text, so they are applied once, properly, with the paper in hand.

Exclusion reasons are ordered and recorded, so the pattern of reasons becomes a diagnostic on whether a criterion is working.

Awkward record types — protocols, conference abstracts, records with no abstract, unreadable languages — have pre-decided destinations instead of ad hoc ones.

The glossary fixes the three terms that would otherwise drift, and the prohibition on inference stops a knowledgeable screener from quietly applying information the record does not contain.

The whole rule set is short enough to keep beside the screen, which determines whether it is still consulted at the four-hundredth record.

The second version is what makes two screeners interchangeable, which is the only sense in which a screen can be called reproducible. It also converts reconciliation from a negotiation into a short list of genuinely borderline records.

Calibration of screening rules

Before the full screen, both screeners assess the same sample of records independently and compare 1. The purpose of this exercise is frequently misunderstood: it is a test of the rule set, not an examination of the screeners.

The sample should be drawn from the actual record set; a few dozen records is usually sufficient to expose the ambiguities, and some records expected to be borderline should be deliberately included. Both screeners decide independently and then compare their decisions one by one. For every disagreement, the screeners should identify which criterion produced it and whether clearer wording of that criterion would have prevented it. In most rounds, the majority of disagreements trace back to two or three criteria, and amending those criteria is the principal benefit of the exercise.

The rules should then be revised, and the exercise repeated on a fresh sample if the first round produced substantial disagreement. The full screen should begin only when the disagreements concern genuinely borderline records rather than the meaning of the criteria. The calibration should be recorded: the number of records, the number of disagreements and the rules changed as a result. This requires three lines in the methods section and constitutes the most persuasive evidence available that the screen was consistent.

Calibration should not be conducted by having the screeners discuss records as they proceed without changing the rules. That approach produces convergence on a shared tacit understanding that cannot be written down or transferred to a third screener, and it leaves no trace for a reader. Disagreements that persist into the main screen are handled by the procedure described on our page on handling discrepancies during data collection.

Conduct of the screen

Cochrane requires at least two people working independently at the study selection stage 1. Independence means that each screener sees the record and the rules, not the other screener’s decision. Screening software should be configured to enforce this rather than relying on discipline in a shared spreadsheet.

Four practical habits make a measurable difference. First, sessions should be short enough that the screening rate does not deteriorate; the Handbook’s own figures imply roughly 500 to 1,000 records in a full working day, which is a sustained pace, and accuracy declines before speed does 1. Second, a reason should be recorded for every exclusion from a fixed, ordered list, even at the title and abstract stage, where reporting does not require it, because the pattern of reasons indicates whether a criterion is being applied as intended 9. Third, a parked category should be kept for records that cannot be decided at all, rather than forcing a binary decision on a record whose abstract is missing or uninformative. Fourth, disagreements should be resolved in batches rather than individually, which is faster and makes patterns visible.

Reconciliation should proceed by promotion rather than debate: at the title and abstract stage, the most efficient defensible rule is that any record included by either screener proceeds to full text. This rule removes most of the reconciliation workload at the cost of a slightly larger full-text set, and it is consistent with the sensitivity logic of the stage. Substantive discussion should be reserved for the full-text stage, where decisions are final.

Record types that challenge screening rules

Every screen encounters a small number of record types that a simple include-or-exclude rule handles poorly. Each should be decided in advance, in writing.

Conference abstracts. They identify studies that may never be published in full, which is precisely why they matter, but they carry too little detail to assess properly 14. The workable rule is to treat an abstract as an indication that a study exists rather than as a study: it should be promoted, a full report and a registry entry should be sought, and a separate decision should be made on whether an abstract-only study may contribute to the synthesis 15.

Companion reports and duplicate publications. One study reported across several papers, and one paper reporting several studies, are both common. The unit of inclusion is the study, not the record, so the screen requires a linking step: records should be grouped by study, a primary report selected, and the other reports kept attached 1,11. Completing this step at screening rather than at extraction prevents double-counting in the flow diagram.

Non-English records. If the protocol applies no language restriction, a route is required for records the team cannot read — a colleague, a translation service or, at minimum, an assessment of the English abstract where one exists. If a language restriction is applied, it is a declared limitation, not a silent one.

Retracted and corrected articles. Retraction status should be checked before inclusion rather than after. A retracted study included in a synthesis is likely to necessitate a correction notice, and the check is quick.

Preprints and records with no abstract. Both go to the parked category by default. A record with no abstract cannot be excluded on the basis of an abstract.

Pre-agreed handling for the awkward record types

Record typeDefault decision at title and abstractWhat has to happen next
Conference abstractPromote — treat as a signal that a study existsSearch for a full report and a registry entry; decide separately whether abstract-only data may enter the synthesis
Companion reports of one studyPromote all, then linkGroup records by study, nominate a primary report, keep the others attached so the flow diagram counts studies not papers
Record with no abstractPark, then promoteCannot be excluded on an abstract that does not exist; assess at full text
Non-English recordPromote if no language restriction appliesRoute to a reader or translation; if the protocol restricts language, record it as a declared limitation
PreprintPromoteCheck for a published version and link the two; decide in the protocol whether preprint-only evidence is eligible
Retracted or corrected articleCheck status before inclusionExclude retracted studies and say so; incorporate corrections into extraction

Compiled as a working rule set; the unit-of-inclusion principle follows the Cochrane Handbook’s treatment of selecting studies.

Transition to full-text screening

Full-text screening is a different exercise with a different standard: here precision matters, decisions are final, and reasons have to be recorded because this is the stage a reader needs to see.

Three things change. The criteria deferred at the title and abstract stage are now applied, which is why the deferral had to be documented. Exclusion reasons must be recorded from the ordered list for every excluded full text, because the flow diagram requires them at this stage and because an unexplained full-text exclusion is the first item a methods reviewer queries 9. In addition, unretrievable reports form a separate category: a record that could not be obtained is neither included nor excluded, and the number of such records should be reported.

The records of the two stages should be kept distinct. The flow diagram needs records screened, records excluded at title and abstract, full texts sought, full texts not retrieved, full texts assessed, full texts excluded with reasons, and studies included — and the arithmetic must reconcile down the whole diagram 9. Reconciling the counts once at the end is more difficult than maintaining them throughout. Reporting requirements for the whole sequence are covered on our PRISMA page.

Priority screening and machine classifiers

Two distinct practices are described as screening automation, and they carry different risks.

Priority screening reorders the record set so that likely-eligible records are seen first 6,8. It does not change which records are assessed, since every record is still screened; it changes when information becomes available. Identifying most of the included studies in the first fifth of the set indicates early whether the search performed adequately and allows extraction to begin sooner. This approach is low risk and generally worth using.

Classification — letting a model exclude records without human assessment, or stopping the screen early on the basis of a model’s prediction — is a different proposition, because it means some records are excluded without a person reading them 4. At least 26 studies have tested natural language processing and machine learning approaches to automating extraction from the records that survive screening, so this is an active research area rather than speculation, but the decision for a given review is one of risk tolerance and transparency rather than of capability in the abstract 5.

If either approach is used, it should be reported as a method: which tool, which version, what it was trained or primed on, whether any record was excluded without human assessment, and if the screen was stopped early, on what criterion 9. A review that used a classifier and does not say so has an undocumented exclusion step. A review that says so, with the criterion stated, has a method a reader can judge.

Key points

Key points

Title and abstract screening is an exercise in sensitivity: the question is whether a record can confidently be judged ineligible, not whether it can be confirmed as eligible, and any record that cannot be excluded for a stated reason should be promoted. The workload should be planned first. The Handbook’s indicative rate is 60 to 120 records an hour, approximately 500 to 1,000 over an eight-hour period, per screener, and Cochrane requires at least two screeners working independently at study selection, so a set of several thousand records represents several weeks of screener time. The eligibility criteria should be written as an ordered set of exclusion rules with a stated default for silent records and an explicit note of the criteria deferred to full text, supplemented by a five-line glossary for terms prone to drift, and screeners should calibrate on a shared sample until disagreements concern records rather than meanings. Awkward record types should be decided in advance: conference abstracts and companion reports are promoted and linked, records without an abstract cannot be excluded on the basis of an abstract, and retraction status is checked before inclusion. At full text, a reason should be recorded for every exclusion, and the flow diagram arithmetic should be reconciled throughout.

Application to your study

Application to your study

  1. The workload should be calculated before the search is finalised. Deduplicated records divided by 500 to 1,000 per eight-hour day, doubled for two independent screeners, plus reconciliation time.
  2. Eligibility criteria should be rewritten as ordered exclusion rules. Each criterion should state one reason, phrased as the condition that permits exclusion, in a fixed order so that the reasons are comparable.
  3. Each rule should include a default. The default specifies the action to take when the record is silent; at this stage, it is almost always to promote.
  4. Deferred criteria should be marked explicitly. Anything an abstract rarely reports is a full-text criterion and should say so on the rule sheet.
  5. A five-line glossary should be prepared. The glossary defines the terms that would otherwise drift: the age band, the duration threshold, what counts as the intervention and what counts as a comparator.
  6. Calibration should be conducted on a shared sample, and the rules amended accordingly. The number of records, the number of disagreements and the rules amended as a result should be recorded.
  7. Disagreements at this stage should be reconciled by promotion. Any record included by either screener proceeds to full text; substantive discussion is reserved for the full-text decisions, which are final.
  8. Flow counts should be maintained throughout. Screened, excluded, sought, not retrieved, assessed, excluded with reasons, included — reconciled at every step rather than at submission.

Frequently asked questions

The Cochrane Handbook gives an indicative rate of 60 to 120 records per hour, or approximately 500 to 1,000 records over an eight-hour period. That rate is per screener, and Cochrane requires at least two people working independently at study selection, so the person-hours are roughly double the single-screener figure, plus reconciliation. A deduplicated set of 5,000 records therefore implies approximately 10 to 20 screener-days before any reconciliation. The factor-of-two spread in the rate is not random variation: it depends on how clearly the rules are written, how specific the topic is, and how much of the set is obviously irrelevant. Planning should assume the slower end of the range, and the arithmetic should be completed before the search is finalised rather than after.

Cochrane requires at least two people working independently at the study selection and data collection stages, and independence means each screener sees the record and the rules rather than the other screener’s decision. Where capacity does not permit this, the defensible options are to declare partial double screening explicitly — stating what proportion of records were screened twice and how the subset was chosen — or to choose a design that permits a lighter process, such as a rapid review, and label it as that. What is not defensible is single screening presented as double, or a second screener who sees and effectively ratifies the first screener’s calls in a shared spreadsheet.

Yes. At the title and abstract stage, a record that cannot be confidently excluded proceeds to full text, including records with no abstract, records with uninformative abstracts and records in a language the team cannot read. The rationale is the asymmetry of cost: a record wrongly promoted costs ten minutes at full text, whereas a record wrongly excluded is lost and its absence is invisible in the finished review. The default should be built into the rule sheet explicitly, criterion by criterion — “if age is not stated, promote” — rather than relying on screeners to remember the principle at the four-hundredth record of the day.

At the title and abstract stage, the simplest defensible rule is that any record included by either screener proceeds to full text, which is consistent with the sensitivity logic of the stage and removes most of the reconciliation workload. Substantive discussion should be reserved for full-text decisions, which are final. Separately, early disagreements should be treated as evidence about the rules rather than about the screeners: if two careful reviewers reading the same abstract reach different conclusions, the usual cause is an ambiguous criterion, and amending the criterion resolves every future instance, whereas retraining the screener resolves none. The disagreement count and the rules changed should be recorded; this requires three lines in the methods section and is the best available evidence that the screen was consistent.

They should be promoted and treated as an indication that a study exists rather than as a study that can be assessed. They matter precisely because they identify work that may never be published in full, and they cannot be assessed at screening because they carry too little methodological detail. Once they are promoted, a full report, a trial registry entry and any regulatory record should be sought, and all records of the same study linked so that the flow diagram counts studies rather than papers. A decision should then be made — in the protocol, not afterwards — on whether an abstract-only study may contribute data to the synthesis and, if so, with what limitation noted, since results reported in an abstract do not always match the eventual full report.

They should be linked at screening rather than identified as a problem at extraction. The unit of inclusion in a systematic review is the study, not the record, so the screen requires an explicit grouping step: all related records are promoted and grouped by study, a primary report is nominated, and the companion reports are kept attached to it. Completing this step early prevents double-counting in the flow diagram and in the synthesis, and it also identifies companion papers that may resolve an ambiguity in the primary report. Identifiers assist this process: a trial registration number, a matching sample size and recruitment window, or an overlapping author list are the usual indications that two records describe one study.

The answer depends on which of two distinct practices is meant. Priority screening reorders the record set so that likely-eligible records are seen first; every record is still assessed by a person, no record is excluded by the model, and the benefit is early information on whether the search performed adequately. This approach is low risk and generally worth using. Classification — allowing a model to exclude records without human assessment, or stopping the screen early on the basis of a model’s prediction — means that some records are excluded unread, which is a different decision concerning risk and transparency. At least 26 studies have tested natural language processing and machine learning approaches for automating extraction from the records that survive screening, so this is an active field of research rather than commercial speculation. In either case, the tool, its version, whether any record was excluded without human assessment, and any early-stopping criterion should be reported.

At full text, yes, and from a fixed ordered list, because the flow diagram requires exclusion reasons at that stage and an unexplained full-text exclusion is the first item a methods reviewer queries. At the title and abstract stage it is not required, but recording a reason from the same ordered list is advisable, because the pattern of reasons serves as a diagnostic. A criterion that excludes far more records than expected is usually being over-applied, and this is detectable only if the reasons were captured. The counts for the two stages should be kept separate, with a distinct category for reports that were sought but could not be retrieved; these are neither included nor excluded, and their number should be reported.

References

  1. Chapter 4: Searching for and selecting studies. Cochrane Handbook for Systematic Reviews of Interventions, version 6.5. Cochrane. 2024;6.5. read 9 October 2026. cochrane.org
  2. Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions, version 6.5. Cochrane. 2024;6.5. read 9 October 2026. cochrane.org
  3. Rethlefsen ML, Kirtley S, Waffenschmidt S, Ayala AP, Moher D, Page MJ, Koffel JB; PRISMA-S Group. PRISMA-S: an extension to the PRISMA Statement for Reporting Literature Searches in Systematic Reviews. Systematic Reviews. 2021;10(1):1-19. read 9 October 2026. doaj.org
  4. Reddy SM, Patel S, Weyrich M, Fenton J, Viswanathan M. Comparison of a traditional systematic review approach with review-of-reviews and semi-automation as strategies to update the evidence. Systematic Reviews. 2020;9(1):243. doi:10.1186/s13643-020-01450-2 read 9 October 2026. doi.org/10.1186/s13643-020-01450-2
  5. Jonnalagadda SR, Goyal P, Huffman MD. Automating data extraction in systematic reviews: a systematic review. Systematic Reviews. 2015;4:78. read 9 October 2026. cochrane.org
  6. Tsafnat G, Glasziou P, Choong MK, Dunn A, Galgani F, Coiera E. Systematic review automation technologies. Systematic Reviews. 2014;3(1):74. doi:10.1186/2046-4053-3-74 read 9 October 2026. doi.org/10.1186/2046-4053-3-74
  7. Horsley T, Dingwall O, Sampson M. Checking reference lists to find additional studies for systematic reviews. Cochrane Database of Systematic Reviews. 2011;2011(8):MR000026. doi:10.1002/14651858.MR000026.pub2 read 9 October 2026. doi.org/10.1002/14651858.MR000026.pub2
  8. Bolanos F, Salatino A, Osborne F, Motta E. Artificial intelligence for literature reviews: opportunities and challenges. Artificial Intelligence Review. 2024;57(10):259. doi:10.1007/s10462-024-10902-3 read 9 October 2026. doi.org/10.1007/s10462-024-10902-3
  9. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71. doi:10.1136/bmj.n71 read 9 October 2026. doi.org/10.1136/bmj.n71
  10. Bramer WM, de Jonge GB, Rethlefsen ML, Mast F, Kleijnen J. A systematic approach to searching: an efficient and complete method to develop literature searches. Journal of the Medical Library Association. 2018;106(4):531-541. doi:10.5195/jmla.2018.283 read 9 October 2026. doi.org/10.5195/jmla.2018.283
  11. Mayo-Wilson E, Li T, Fusco N, Dickersin K. Practical guidance for using multiple data sources in systematic reviews and meta-analyses (with examples from the MUDS study). Research Synthesis Methods. 2018;9:2-12. read 9 October 2026. cochrane.org
  12. Page MJ, Altman DG, Shamseer L, McKenzie JE, Ahmadzai N, et al. Reproducible research practices are underused in systematic reviews of biomedical interventions. Journal of Clinical Epidemiology. 2018;94:8-18. doi:10.1016/j.jclinepi.2017.10.017 read 9 October 2026. doi.org/10.1016/j.jclinepi.2017.10.017
  13. Munn Z, Peters MD, Stern C, Tufanaru C, McArthur A, Aromataris E. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Medical Research Methodology. 2018;18(1):143. doi:10.1186/s12874-018-0611-x read 9 October 2026. doi.org/10.1186/s12874-018-0611-x
  14. Paez A. Gray literature: An important resource in systematic reviews. Journal of Evidence-Based Medicine. 2017;10(3):233-240. doi:10.1111/jebm.12266 read 9 October 2026. doi.org/10.1111/jebm.12266
  15. Baudard M, Yavchitz A, Ravaud P, Perrodeau E, Boutron I. Impact of searching clinical trial registries in systematic reviews of pharmaceutical treatments: methodological systematic review and reanalysis of meta-analyses. BMJ. 2017;356:j448. read 9 October 2026. cochrane.org
  16. Bramer WM, Giustini D, de Jonge GB, Holland L, Bekhuis T. De-duplication of database search results for systematic reviews in EndNote. Journal of the Medical Library Association. 2016;104(3):240-243. doi:10.3163/1536-5050.104.3.014 read 9 October 2026. doi.org/10.3163/1536-5050.104.3.014
  17. Millard LAC, Flach PA, Higgins JPT. Machine learning to assist risk-of-bias assessments in systematic reviews. International Journal of Epidemiology. 2016;45:266-277. read 9 October 2026. cochrane.org
  18. Marshall IJ, Kuiper J, Wallace BC. RobotReviewer: evaluation of a system for automatically assessing bias in clinical trials. Journal of the American Medical Informatics Association. 2016;23:193-201. read 9 October 2026. cochrane.org
  19. Rethlefsen ML, Farrell AM, Osterhaus Trzasko LC, Brigham TJ. Librarian co-authors correlated with higher quality reported search strategies in general internal medicine systematic reviews. Journal of Clinical Epidemiology. 2015;68(6):617-626. doi:10.1016/j.jclinepi.2014.11.025 read 9 October 2026. doi.org/10.1016/j.jclinepi.2014.11.025
  20. Koffel JB. Use of recommended search strategies in systematic reviews and the impact of librarian involvement: a cross-sectional survey of recent authors. PLOS ONE. 2015;10(5):e0125931. doi:10.1371/journal.pone.0125931 read 9 October 2026. doi.org/10.1371/journal.pone.0125931
  21. Rethlefsen ML, Murad MH, Livingston EH. Engaging medical librarians to improve the quality of review articles. JAMA. 2014;312(10):999-1000. doi:10.1001/jama.2014.9263 read 9 October 2026. doi.org/10.1001/jama.2014.9263
  22. Elliott JH, Turner T, Clavisi O, Thomas J, Higgins JP, Mavergames C, Gruen RL. Living systematic reviews: an emerging opportunity to narrow the evidence-practice gap. PLOS Medicine. 2014;11(2):e1001603. doi:10.1371/journal.pmed.1001603 read 9 October 2026. doi.org/10.1371/journal.pmed.1001603
  23. Gross A, Schirm S, Scholz M. Ycasd - a tool for capturing and scaling data from graphical representations. BMC Bioinformatics. 2014;15(1):219. doi:10.1186/1471-2105-15-219 read 9 October 2026. doi.org/10.1186/1471-2105-15-219
  24. Zarin DA, Tse T, Williams RJ, Califf RM, Ide NC. The ClinicalTrials.gov results database: update and key issues. New England Journal of Medicine. 2011;364:852-860. read 9 October 2026. cochrane.org
  25. Papaioannou D, Sutton A, Carroll C, Booth A, Wong R. Literature searching for social science systematic reviews: consideration of a range of search techniques. Health Information and Libraries Journal. 2010;27(2):114-122. doi:10.1111/j.1471-1842.2009.00863.x read 9 October 2026. doi.org/10.1111/j.1471-1842.2009.00863.x
  26. Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information and Libraries Journal. 2009;26(2):91-108. doi:10.1111/j.1471-1842.2009.00848.x read 9 October 2026. doi.org/10.1111/j.1471-1842.2009.00848.x
  27. Schardt C, Adams MB, Owens T, Keitz S, Fontelo P. Utilization of the PICO framework to improve searching PubMed for clinical questions. BMC Medical Informatics and Decision Making. 2007;7:16. doi:10.1186/1472-6947-7-16 read 9 October 2026. doi.org/10.1186/1472-6947-7-16
  28. Buscemi N, Hartling L, Vandermeer B, Tjosvold L, Klassen TP. Single data extraction generated more errors than double data extraction in systematic reviews. Journal of Clinical Epidemiology. 2006;59:697-703. read 9 October 2026. cochrane.org
  29. Jones AP, Remmington T, Williamson PR, Ashby D, Smyth RL. High prevalence but low impact of data extraction and reporting errors were found in Cochrane systematic reviews. Journal of Clinical Epidemiology. 2005;58:741-742. read 9 October 2026. cochrane.org

Every source above is a peer-reviewed article, a publisher’s record of one, or a standard issued by the body that owns it. Each carries the date it was last read. Where a standard moves, this article is revised rather than left up with a note.

How to cite this article

Pick your referencing style and copy it. Your university or target journal may vary the punctuation — check its guide if it has one.

Pubrica Academy. (2026, October 9). Title and Abstract Screening in Systematic Reviews: Planning, Calibration and Reporting. Pubrica. https://pubrica.com/academy/systematic-review/how-to-conduct-abstract-screening-for-systemic-review-from-beginning-to-end/

This article is written to be learned from. Cite it if you are referring to what it says; if you are using the evidence behind it, cite the sources listed above instead.

Screening rule sheet, calibration record and flow counts

The rule sheet should be printed and kept beside the screen. The calibration block should be completed before the full screen starts, and the flow counts maintained throughout rather than at submission.

SCREENING RULE SHEET — REVIEW: __________________
Rule sheet version: ____  Frozen on: __________
Screeners: ______________  Independence enforced how: __________

=== WORKLOAD ESTIMATE (do this before finalising the search) ===
  Records retrieved, all sources: ______
  After deduplication: ______
  At 1,000 per eight-hour day: ______ screener-days
  At 500 per eight-hour day:   ______ screener-days
  x 2 for independent screening: ______ to ______ screener-days
  Reconciliation allowance: ______
  Does this fit the team and the deadline?  [ ] yes
  If no:  [ ] narrow the question  [ ] restrict the search AND declare it
          [ ] change design (rapid / scoping review) AND label it

=== GLOSSARY (define the terms that will otherwise drift) ===
  ____________ means ____________________________________
  ____________ means ____________________________________
  ____________ means ____________________________________
  A record "states" something only if it appears in the title or
  abstract. Inference from the journal, authors or your own knowledge
  of the field is NOT permitted.

=== ORDERED EXCLUSION RULES ===
Work down the list. Record the FIRST reason that applies.
Phrase every rule as a reason to EXCLUDE, never as a test of eligibility.

  1. NOT A PRIMARY STUDY
     Exclude if: ____________________________________
     Do NOT exclude: protocols, conference abstracts (promote these)
     If silent: [ ] promote
  2. POPULATION
     Exclude only if the record states: ______________________
     Deferred to full text: ________________________________
     If silent: [ ] promote  [ ] exclude (justify: __________)
  3. INTERVENTION
     Exclude only if the record states: ______________________
     Deferred to full text: ________________________________
     If silent: [ ] promote
  4. COMPARATOR
     Exclude at this stage?  [ ] no (recommended)  [ ] yes, if: ______
     If silent: [ ] promote
  5. DESIGN
     Exclude only if the record explicitly states: ____________
     If silent: [ ] promote
  6. OUTCOMES
     Exclude at this stage?  [ ] no (recommended)  [ ] yes, if: ______
  7. OTHERWISE: PROMOTE.

=== PRE-DECIDED HANDLING ===
  Conference abstract: promote, then seek full report + registry entry
  Companion reports: promote all, group by study, nominate primary
  No abstract: promote
  Language you cannot read: [ ] promote and route to ______
                            [ ] excluded by protocol — declared limitation
  Preprint: [ ] promote  Eligible for synthesis? [ ] yes [ ] no
  Retraction check done before inclusion?  [ ] yes

=== CALIBRATION RECORD (before the full screen) ===
  Round 1: records ____  disagreements ____
    Rules amended as a result: ____________________________
  Round 2 (if needed): records ____  disagreements ____
    Rules amended: ________________________________________
  Full screen started on: __________
  Reconciliation rule at this stage:
    [ ] any record either screener includes goes to full text
    [ ] discussion  [ ] third screener: ______________

=== FLOW COUNTS (keep current; reconcile at every step) ===
  Records identified, by source:  ______________________
  Duplicates removed: ______
  Records screened: ______
  Records excluded at title/abstract: ______
    by reason 1 ____  2 ____  3 ____  4 ____  5 ____  6 ____
  Full texts sought: ______
  Full texts NOT retrieved: ______
  Full texts assessed: ______
  Full texts excluded WITH REASONS: ______
    by reason: ____________________________________________
  Studies included: ______   Reports of those studies: ______
  [ ] Every stage adds up

=== AUTOMATION, IF USED ===
  Priority screening (reordering only)?  [ ] yes, tool: __________
  Any record excluded WITHOUT human assessment?  [ ] no  [ ] yes: ____
  Screen stopped early?  [ ] no  [ ] yes, on this criterion: ________
  Reported in the methods?  [ ] yes

Pubrica AcademyWritten and maintained by Pubrica’s subject matter experts. Articles are revised when the underlying standards change, and each article states the date on which it was last reviewed.

How Pubrica can help

Pubrica converts eligibility criteria into an ordered screening rule sheet with defaults and deferrals, conducts the calibration round on a sample of the client’s own records, and screens with two independent reviewers in software that enforces blinding. The deliverables comprise the rule sheet, the calibration record, the exclusion-reason breakdown and reconciled flow counts, which are the elements that teams most often have to reconstruct at submission.

Find something else

Search the Pubrica Academy

Access is free and requires no account. Enter a topic, or open the full library to filter by category, sub-topic or resource type.

On receipt of search results and eligibility criteria, Pubrica provides a screening rule sheet and a calibrated workload estimate.

A scoping assessment is free and takes two business days. It provides a written assessment of what the work requires and which specialist would undertake it.

Get a free quote

Regulations

Check your project against this rule

  1. 1Your work
  2. 2Your details
  3. 3Send
What do you need?

Material you send is held under signed confidentiality, with access limited to the assigned team and encrypted transfer. It is not used to train anything and is not passed to third-party tools. Ask for the agreement before you send anything you would rather cover first.