Skip to Main Content

Systematic Reviews and Other Evidence Synthesis Types Guide

Systematic Reviews and Other Evidence Synthesis Types Guide

Task 5 - Search

GROUP PRIMARY RESPONSIBILITY NOTES
Librarian Librarian performs the searches as part of the fee-based service.
ES/SR Team   Important: If a Librarian is not part of the ES/SR Team, then the ES/SR Team is responsible for search tasks 5a. – 5f.

Overview

The literature search is essential – it generates the data available for analysis. The other tasks of the systematic review process (i.e., screening, appraisal, data extraction, and synthesis) are dependent on the identification of eligible studies. As a result, the literature search must be comprehensive, objective, and reproducible to identify as many relevant studies as possible to minimize bias (i.e., citation bias, publication bias, language bias, etc.). (Higgins JPT, 2020; Rethlefsen et al., 2021)

Note: Refer to Chapters 7, 8, 13, and 25 in Cochrane Handbook for Systematic Reviews of Interventions, 2nd ed., 2019, for more information on assessing bias.

As a ES/SR Team Member/co-author, the Librarian:

  1. Suggests databases/resources for searches.
  2. Translates approved initial strategy (Task 5a) and searches across multiple databases. Note: The ES/SR Team reviews and approves the search strategy in the remaining core databases.
  3. Documents the search methods for databases and non-database sources. Note: The ES/SR Team may choose to search additional Grey Literature sources and perform Citation Searching (Task 10).
  4. Organizes the search documentation.
  5. Delivers citations as citation management software importable files (e.g., EndNote, which is the UT Southwestern standard) and, upon request, as Microsoft Word documents.

Task 5a – Initial Search

It is important to locate and identify all relevant studies using a wide range of resources and search methods for the systematic review, scoping review and other select evidence synthesis types. Developing the search strategy is different from conducting the search. A comprehensive search strategy will help to reduce the risk of bias by identifying as much relevant evidence as possible. (Tsafnat et al., 2014; Higgins JPT, 2020)

A well-defined ES/SR question is essential to develop a comprehensive search strategy. As a member of the ES/SR team, the Librarian:

  • Collaborates with the ES/SR Team primary contact to develop the search strategy.
  • Uses the ES/SR question to identify key concepts. The PICO mnemonic or equivalent is helpful to organize concepts.
  • Further defines key concepts using a combination of subject headings and keywords.
  • Requests ES/SR Team to review the preliminary list of search terms and decide whether terms need to be added or deleted.
  • Performs a preliminary search using Boolean Operators (OR, AND, NOT), search field queries and adjacency/proximity commands. Subject headings and keywords for each concept are combined using OR. The results for each concept are then combined using AND to make sure that each concept is included in the final search results.
  • Checks whether the preliminary search strategy retrieved the benchmark articles identified by the ES/SR Team.
  • Peer librarian performs a review of the initial search strategy following the PRESS (Peer Review of Electronic Search Strategies) 2015 Evidence-Based Checklist. Research indicates using a structured tool, such as the PRESS 2015 Evidence-Based Checklist, can improve the quality and comprehensiveness of the search and reduce. (McGowan et al 2016)
  • Requests ES/SR Team to review/approve the initial search strategy in the core database(s). This step takes time and may require multiple revisions. Once approved by the ES/SR team, this search strategy will become the master search strategy which will then be translated for each database.
  • Seeks to maximize recall (sensitivity) while striving for reasonable precision (specificity). See Limits and Filters.
  • Recommends which bibliographic databases to search – the most commonly searched core bibliographic databases are Medline/PubMed, Embase and Cochrane Central.  It is recommended to set up an account in each database so that you can save your search strategy.  For Cochrane Central, you need to use all lowercase characters for your email address/login. 
  • Database selection should be guided by the review topic. When topics are specialized, cross-disciplinary, or involve emerging technologies, additional databases may need to be identified and searched.
  • Discusses grey literature, different sources and ES/SR Team and Librarian respective search responsibilities.

Note:

  • The ES/SR Team also reviews and approves each translated search strategy in the remaining databases.
  • Review the search guidelines specific to those listed at Evidence Synthesis – Guidelines and Resources. For questions regarding guidelines to an evidence synthesis not listed, contact the Library using the Ask Us form.

Task 5b – Databases

There is no single database that is able to provide a complete and accurate list of all studies that meet the various ES/SR guidelines’ criteria due to the differences in the articles included, controlled vocabulary/subject headings, syntax, and the indexing methods used between databases.

As noted in the respective ES/SR guidelines, for a search to be comprehensive, it is important to perform a search using a combination of controlled vocabulary/subject headings and keywords in multiple databases, grey literature, and other resources – the searches must be reproducible. This approach helps to minimize selection bias, publication bias and language bias and results in more reliable estimates of effects and uncertainties. The Technical Supplement to Chapter 4: Searching for and selecting studies to the Cochrane Handbook for Systematic Reviews of Interventions Version 6.3, includes additional information on bibliographic databases other than MEDLINE, Embase, and Cochrane CENTRAL. (Higgins JPT, 2020; Rethlefsen et al., 2021)

Sources to search include:

  • Bibliographic Databases – Searches of health-related bibliographic databases are usually the most efficient way to identify an initial set of results.

    Database selection should be guided by the type of ES/SR question. If the ES/SR question is specialized, multi-disciplinary or involves new technologies, specialty databases (i.e., CINAHL, PsycINFO, ERIC). multidisciplinary (i.e., Scopus, Web of Science), and other resources, including the grey literature, may need to be identified and searched (Higgins JPT, 2020). Additionally, it is recommended to review the requirements or recommendations specified in the various guidelines and recommended best practices for the type of review being conducted.

    The three bibliographic databases generally considered the most important sources to search for biomedical studies are PubMed/MEDLINE, Embase and the Cochrane Central Register of Controlled Trials (Higgins JPT, 2020).  It is recommended to set up an account in each database so that you can save your search strategy.  For Cochrane Central, you need to use all lowercase characters for your email address/login.

  • Ongoing Studies and Unpublished Data Sources – While there is not a single easy and reliable way to obtain information about studies that have been completed but never published, it is important identify and include results when appropriate to minimize bias. Possible sources include trial registers, regulatory agencies, correspondences, etc. The ES/SR Team may choose to search these data sources.
  • Trial Registries and Trial Results Registries – Study registries allow researchers to find ongoing clinical trials and studies that may have gone unpublished. Key sources include ClinicalTrials.gov and WHO International Clinical Trials Registry Platform (ICTRP) portal. The ES/SR Team may choose to search these data sources.
  • Regulatory Agencies and Clinical Study Reports – These include, but are not limited to, Drugs@FDA, and OpenTrialsFDA. Clinical study reports are the reports of clinical trials providing detailed information on the methods an results of clinical trials submitted in support of marketing authorization applications. The ES/SR Team may choose to search these data sources.

Other sources to be considered by the SR Team:

  • Grey Literature – Reports and information published outside traditional commercial publishing, including dissertations/theses, conference abstracts/papers, preprints, regulatory agencies, statistics, etc. For more information, see Grey Literature.
  • Citation Searching, also known as Handsearching or Snowballing – This is the responsibility of the ES/SR Team. It can involve checking reference lists of included articles and/or a page-by-page examination of the entire contents of a print or online journal issue or conference proceedings to identify all eligible reports of trials. For more information, see Citation Searching (Task 10).

Task 5c – Translate Searches

ES/SR guidelines require the primary investigator/project lead/researcher to search multiple databases. Even if the database is on the same platform (i.e., OVID, Elsevier, EBSCO, etc.) or its source of records is the same (i.e., National Library of Medicine is the source for MEDLINE records on applicable platforms, i.e., PubMed, Ovid MEDLINE, Web of Science MEDLINE, EBSCO MEDLINE, etc.), the database must be searched using its specialized search syntax. An Ovid MEDLINE search cannot be simply cut and pasted into Ovid Embase. Likewise, a PubMed search cannot be simply cut and pasted into Ovid MEDLINE. The search strategy must be 'translated' between databases to take into account the differences in the articles included, controlled vocabulary/subject headings, syntax, and the indexing methods.

The Library’s Database Comparison Chart summarizes key features, search syntax, search tips, proximity & adjacency operators, and common field tags of popular bibliographic databases. This tool is helpful when translating a search from one database to another database.

As a member of the ES/SR Team, the Librarian translates the approved master strategy in each of the remaining databases following the approach detailed in Task 5b. Before conducting each translated search, the Librarian submits the translated search strategy to the ES/SR Team to review/approve.

Task 5d – Document Methods

The intent of PRISMA-S (Preferred Reporting Items for Systematic reviews and Meta-Analyses literature search extension) is to complement the PRISMA 2020 Statement and its extensions. The checklist provides the ES/SR Team the framework that helps ensure transparency and maximum reproducibility of the search component of the ES/SR. It covers multiple aspects of the search process for systematic reviews and is intended to guide reporting, not conduct, of the search. Rethlefsen et. al.'s article includes explanation and elaboration for each checklist item. (Rethlefsen, 2021)

For the other ES/SR in this Guide, review the related search guidelines and reporting guidelines at Evidence Synthesis – Guidelines and Resources and/or use the PRISMA-S to the extent possible, and adapt it appropriately.

The Librarian:

  • Lists all databases (including platform), registries, websites, organizations, and other sources searched or consulted to identify studies.
  • Includes the date when each source was last searched or consulted.
  • Presents the full search strategies for all databases, registries and websites, including any filters and limits used. The searches of all the databases and other sources should be reproducible.
  • Describes any additional information sources or search methods used

The ES/SR Team:

  • Indicates whether cited references or citing references were examined, and describes any methods used for locating cited/citing references (Task 10).
  • Specifies whether additional studies or data were sought by contacting authors, experts, manufacturers, or others.
  • Describes any additional information sources or search methods used

Task 5e – Organize Documentation

Recording the flow of citations through the systematic review process is a key component of the PRISMA Statement. It is helpful to include the dates searched and identify how many records were identified within each database and additional source. Knowing the number of records from each source also helps with reproducibility. (Reflethsen, 2021)

For the other ES/SR in this Guide, review the related search guidelines and reporting guidelines at Evidence Synthesis – Guidelines and Resources and/or use the PRISMA-S to the extent possible, and adapt it according.

Task 5f – Deliver Citations

EndNote is the UT Southwestern subscribed citation management program from Clarivate. This citation management program saves time: you can consolidate and organize the citations from your multiple searches, de-duplicate the results and automatically format in-text citations and bibliographies when drafting your manuscript.

The Librarian searches multiple databases and forwards results from each searched database to the ES/SR Team as EndNote ready files and, upon request, Microsoft Word documents.

Exporting Search Results – Recommended Settings by Database:

Grey Literature

Grey literature (or gray literature) includes information or materials that are not published or indexed in the traditional databases, indexes, peer-reviewed journals, and books. These materials are referred to as grey or fugitive because they can be difficult to identify and obtain. Examples include dissertations, theses, government reports/publications, academic reports/publications, internal company reports, policy documents, and poster and paper presentations. (Adorno, Garbee, & Marix, 2016; Booth, 2012; Health Sciences Library McMaster University, 2019)

To minimize publication bias, it is recommended to search the grey literature as noted in the following select guidelines and standards:

  • Finding What Works in Health Care: Standards for Systematic Reviews
    • Standard 3.2 Take action to address potentially biased reporting of research results
  • Cochrane Handbook for Systematic Reviews of Interventions
    • Chapter 4: Searching for and selecting studies in Higgins J.P.T., Thomas, J. et al (editors). See Section 4.3, Sources to Search.
    • Technical Supplement to Chapter 4: Searching for and selecting studies
    • Chapter 13: Assessing risk of bias due to missing results in a synthesis

There are numerous sources of grey literature. The Grey Literature Network Service (GreyNET) has a comprehensive list of the different grey literature document types. The Canadian Agency for Drugs and Technologies in Health (CADTH) has developed and made available a grey literature checklist, Grey Matters: A Practical Tool for Searching Health-Related Grey Literature.

While the Librarian typically searches 1-2 grey literature resources, the ES/SR Team may need to conduct their own grey literature searching. It is important to identify which grey literature types and sources are appropriate to search based on the research question and discipline.

Some suggested resources are included in the following sub-sections.

Abstracts and Conferences

Interventions - Trials Registers

Drugs and Devices

Relevant professional associations may also be searched for additional information.

Theses and Dissertations

To search in ProQuest™ Dissertations & Theses Citation Index, select the database from the “Search In” dropdown on the Documents tab on the Web of Science home page.

Preprints

Preprints are drafts of research manuscripts that are intended to be shared publicly. There are numerous preprint servers which can be searched using a combination of keywords. As preprint servers maybe subject/discipline focused, it may be necessary to search multiple preprint servers. Additionally, it is important to remember that preprints have not been peer reviewed. (Bonato 2022)

Statistical Data

Statistical data can be found on:

  • Government agencies at the Federal, state, and local levels, i.e., National Center for Health Statistics, Texas Department of State Health Services, Dallas County Health and Human Services
  • International organizations, i.e., World Health Organization
  • Professional Associations, i.e., American Heart Association
  • Other nongovernment organization

More Information on Grey Literature

Limits and Filters

To minimize bias and perform a comprehensive, exhaustive search, it is usually recommended not to apply limits to the search. For example, database limits for language, publication date or publication format may introduce bias, reduce search sensitivity and unintentionally exclude relevant citations.

Another approach in lieu of applying filters or limits to the search strategy, the SR/ES Team may apply the limits or filters when screening the retrieved results as part of the inclusion/exclusion criteria.

If filters or limits are used, they should be applied at the end of the search strategy. Their application and reason for use must be documented in the methods section.

Search Strategy Checklists

PRISMA for Searching

PRISMA for searching – Published in 2021, the checklist includes 16 reporting items, each of which is detailed with exemplar reporting and Rationale. The intent of PRISMA-S is to complement the PRISMA Statement and its extensions by providing a checklist that could be used by interdisciplinary authors, editors, and peer reviewers to verify that each component of a search is completely reported and therefore reproducible.

PRESS (Peer Review of Electronic Search Strategies) Checklist

The PRESS (Peer Review of Electronic Search Strategies) Checklist provides a list of questions organized in six categories to consider when reviewing the SR search strategy. The categories encompass:

  • Translation of the research question
  • Boolean and proximity operators
  • Subject headings (database specific)
  • Text word searching (free text)
  • Spelling, syntax and line numbers
  • Limits and filters

The full guideline statement and checklist document may be accessed via the open access article from the Journal of Clinical Epidemiology (McGowan et al., 2016b).

References

  • Adorno, M., Garbee, D., & Marix, M. L. (2016). Improving literature searches. Clinical nurse specialist CNS, 30(2), 74–80. https://doi.org/10.1097/NUR.0000000000000187
  • Kugley, S., Wade, A., Thomas, J., Mahood, Q., Jørgensen, A. K., Hammerstrøm, K., & Sathe, N. (2017). Searching for studies: A guide to information retrieval for Campbell Systematic Reviews. Campbell Systematic Reviews, 13(1), 1–73. https://doi.org/10.4073/cmg.2016.1
  • McGowan, J., Sampson, M., Salzwedel, D. M., Cogo, E., Foerster, V., & Lefebvre, C. (2016). PRESS peer review of electronic search strategies: 2015 guideline statement. Journal of clinical epidemiology, 75, 40–46. https://doi.org/10.1016/j.jclinepi.2016.01.021
  • Rethlefsen, M. L., Kirtley, S., Waffenschmidt, S., Ayala, A. P., Moher, D., Page, M. J., Koffel, J. B., & PRISMA-S Group (2021). PRISMA-S: an extension to the PRISMA Statement for Reporting Literature Searches in Systematic Reviews. Systematic reviews, 10(1), 39. https://doi.org/10.1186/s13643-020-01542-z
  • Tsafnat, G., Glasziou, P., Choong, M. K., Dunn, A., Galgani, F., & Coiera, E. (2014). Systematic review automation technologies. Systematic reviews, 3, 74. https://doi.org/10.1186/2046-4053-3-74

The Technical Supplement to Chapter 4: Searching for and selecting studies to the Cochrane Handbook for Systematic Reviews of Interventions Version 6.3, includes additional information on bibliographic databases other than MEDLINE, Embase and Cochrane CENTRAL.