Literature search in systematic reviews: How much is good enough?

Abstract

Background Systematic reviews focus on performing exhaustive literature searches to address specific research questions. However, in practice, many reviews search a limited number of databases. Objective This analysis was undertaken to identify the potential impact of searching a limited number of databases in systematic reviews. Methods Two recently published systematic reviews addressing the same research question, but using different search strategies, were utilized as a case study. Both were published in 2023, less than 6 months apart. One of these searched Pubmed and Embase; whereas the other searched through Pubmed and the Cochrane Library. Information on the review characteristics, detailed methodology, search strategy, reviewers' results and conclusions; were extracted. Both reviews were appraised for methodological quality using the AMSTAR-2 checklist. Results The search output of the two reviews was vastly different. Using a step-wise screening process, the two reviews finally included 16 and 11 studies. Among these, only 4 studies were included in both reviews, suggesting that both reviews missed multiple eligible studies. The data on the primary outcomes were also very different in the two reviews, suggesting that neither could be considered completely reliable for decision-making by stakeholders. Conclusion This analysis highlights the risk of bias arising in systematic reviews undertaking limited literature searches, especially limited number of databases for published literature. The commonly employed practice of searching two databases in systematic reviews, needs to be rectified urgently.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 16 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.