| GROUP | PRIMARY RESPONSIBILITY | NOTES |
|---|---|---|
| SR Team | X |
The Librarian searches multiple databases and forwards results per database to the SR Team as EndNote ready files and WORD documents (upon request). The SR Team uses the results from the Librarian's searches and the SR Team's searches to begin the PRISMA Flow Diagram.
The EndNote ready files allows the SR Team to import the results into EndNote and consolidate the results from the multiple databases. Next, the SR Team can use a combination of the software's de-duplication algorithm and manual review to eliminate citation duplicates from the consolidated results.
By default, EndNote identifies duplicate references in the EndNote library as references of the same reference type with matching Author, Year, and Title fields. The SR Team can customize the criteria that EndNote uses to deduplicate citations. To access these settings
UT Southwestern has an institutional license to EndNote. To download the software and get started, visit the following link:
| GROUP | PRIMARY RESPONSIBILITY | NOTES |
|---|---|---|
| SR Team | X |
To identify the relevant studies, SR searches aim to be comprehensive and seek to maximize recall (sensitivity) while balancing precision (specificity). It is not uncommon for these searches to retrieve thousands of studies that need to be reviewed. Sampson, et. al., found the average percent of studies retrieved making it into a final systematic review is around 3%, with a range of 3-6 articles meeting inclusion criteria per 100 articles examined. (Sampson, Tetzlaff, & Urquhart, 2011)
The objective of the screening process is to exclude the irrelevant citations retrieved from the searches conducted in the Retrieval phase. This task involves careful – sometimes subjective – judgments and thorough documentation. If the study inclusion criteria outlined in the SR protocol are too narrow, critical data may be missed. If the inclusion criteria outlined in the SR protocol are too broad, irrelevant studies may overburden the process. (IOM (Institute of Medicine), 2011; Page, 2021)
Typically the screening or study selection process is a multi-stage process and includes:
Two SR Team members must independently examine the titles and abstracts in the de-duplicated results (Task 6) versus the inclusion criteria to remove obvious irrelevant results. If a citation lacks an abstract and only has the title, the citation should be included in the full-text review. A third SR team member will resolve any differences in their decisions. The SR Team should generally be over-inclusive at this step.
The screening steps can be time and labor intensive and logistically challenging. In addition to using EndNote and Excel to help organize the screening process, software applications have been developed to support the process. For more information, see Screening Tools (for Tasks 7 and 9).
(Higgins JPT, 2020; IOM (Institute of Medicine), 2011; Muka, 2020; Page, 2021; Tsafnat et al., 2014; Sampson, Tetzlaff, & Urquhart, 2011)
| GROUP | PRIMARY RESPONSIBILITY | NOTES |
|---|---|---|
| SR Team | X |
Search results are forwarded to the SR Team as EndNote ready files or (upon request) Microsoft Word documents. After the SR Team de-duplicates the consolidated results (Task 6) and screens the title/abstracts (Task 7), the SR Team can use EndNote's full-text retrieval feature. For more information on how to activate and retrieve available full text in UT Southwestern's online journal collection using EndNote, review Find Full Text Using EndNote.
For those citations where EndNote does not retrieve the full text, the SR Team can search the Library's Ejournals collection. To minimize the introduction of bias, it is important to not ignore citations outside of the Library's collection – the SR Team should utilize the UT Southwestern Library's Interlibrary Loan (ILLiad) system by submitting a separate request for each article citation. Note: It is not the responsibility of the Librarian to provide full-text retrieval.
Questions? Use the Ask Us form.
| GROUP | PRIMARY RESPONSIBILITY | NOTES |
|---|---|---|
| SR Team | X |
Two SR Team members must independently examine the retrieved full text of the potentially relevant citations and evaluate it against the inclusion/eligibility criteria. The SR Team may need to correspond with the researchers to clarify whether the study should be included. A third SR team member will resolve any differences in their decisions.
The screening steps can be time- and labor-intensive, as well as logistically challenging. In addition to using EndNote and Excel to help organize the screening process, software applications have been developed to support the process. For more information, see Screening Tools (for Tasks 7 and 9).
| GROUP | PRIMARY RESPONSIBILITY | NOTES |
|---|---|---|
| SR Team | X |
Citation searching – also known as "handsearching" or "snowballing" – is frequently recommended by systematic review manuals and is undertaken by the SR Team after full-text screening (Task 9). It supplements searching bibliographic databases to identify studies for a systematic review.
Assuming the retained citations are likely to have similar content, these reference lists are reviewed for relevant studies cited in them (backward search) or articles that have cited the retained full-text articles (forward search). Also, the reference lists of published related systematic reviews should be reviewed. Horsley, et. al., performed a systematic review to examine the effectiveness of checking reference lists as a method to supplement electronic searching. The data suggests that in situations where researchers have difficulty locating information, checking the reference lists of included articles may be an important way to reduce the risk of missing relevant information.(Briscoe, Bethel, & Rogers, 2020; Higgins JPT, 2020; Horsley, Dingwall, & Sampson, 2011; Muka et al., 2020)
Citation searching can also include the page-by-page examination of the entire contents of a print or online journal issue or conference proceedings to identify all eligible reports of trials. When reporting, it is important to specify whether the full text of a journal or conference proceedings has been searched electronically or using the print version. Some journals omit sections of the print version – for example, letters – from the electronic version, and some include supplementary information such as extra articles in the electronic format only. (Higgins JPT, 2020)
Articles identified from citation searching will need to undergo the de-duplicating and screening steps (Tasks 6 - 9).
Cochrane Handbook for Systematic Reviews of Interventions, 2nd ed., 2019 (Wiley Online Library) by 
| GROUP | PRIMARY RESPONSIBILITY | NOTES |
|---|---|---|
| SR Team | X |
Study appraisal systematically examines factors, such as:
It is important for the SR Team to report the quality of the evidence of the included studies. The quality encompasses the study's methodological quality (i.e., how the study was conducted) and reporting quality and reproducibility (how it was described). Poor methodological and reporting quality may introduce bias and questionable conclusions (Muka et al., 2020).
The following table includes definitions of some types of non-reporting biases (Higgins JPT, 2020). For more information on bias and risk of bias assessment, see Chapters 7, 8, 13 and 25 in Cochrane Handbook for Systematic Reviews of Interventions (Higgins JPT, 2020).
|
TYPE |
DEFINITION |
|---|---|
|
Publication bias |
The publication or non-publication of research findings, depending on the nature and direction of the results. |
|
Time-lag bias |
The rapid or delayed publication of research findings, depending on the nature and direction of the results. |
|
Language bias |
The publication of research findings in a particular language, depending on the nature and direction of the results. |
|
Citation bias |
The citation or non-citation of research findings, depending on the nature and direction of the results. |
|
Multiple (duplicate) publication bias |
The multiple or singular publication of research findings, depending on the nature and direction of the results. |
|
Location bias |
The selective reporting of some outcomes or analyses, but not others, depending on the nature and direction of the results. |
Currently Covidence's quality assessment is limited to Risk of Bias only. The Cochrane Collaboration's tool, Risk-of-Bias Tool (RoB2), and the Newcastle-Ottawa Scale are commonly used to evaluate the risk of bias in randomized controlled trials and prospective observational studies, respectively (Muka, 2020).
The screening steps can be time and labor intensive and logistically challenging. In addition to using EndNote and Excel to help organize the screening process, tools have been developed to support the process.
First time users need to register; existing accounts can proceed to Covidence.
The following flowchart summarizes the Covidence screening process.
While the Library does not provide training, Covidence provides excellent video tutorials and instructions detailing the screening process from title/abstract screening to full-text review to resolving conflicts. For more information, see:
Additional tools can be found by searching the Systematic Review Toolbox (SR Toolbox).
