This guide is derived from the information in chapter 6 of The Cochrane Handbook (1, 2). Carol Lefebvre, Eric Manheimer and Julie Glanville kindly gave permission to the original Campbell Collaboration (Campbell) Information Retrieval Guide authors to use the chapter and chapter updates as the basis for this guide. In 2015 the Campbell Information Retrieval Methods Group (IRMG) revised this guide to reflect current Campbell Collaboration areas of practice and recommendations in the Methodological Expectations of Campbell Collaboration Intervention Reviews (MEC2IR), capture evolving practice and strategies for searching, and update links and descriptions of individual bibliographic and other resources. This guide presents key considerations on the information retrieval process and provides examples of strategies and resources for review authors and Trial Search coordinators (TSC) to reference in the planning and conduct of Campbell Systematic Reviews. The guide provides anoverview of information retrieval principles (Ch.2); covers sources of literature including specific subject databases and bibliographic indexes (Ch. 3); describes steps for planning and executing searches (Ch.4 and 5); reviews tools for managing retrievals (Ch. 6); introduces the role of text mining (Ch. 7); and outlines key elements of the search process and search strategies for documentation (Ch.8). The Guide Appendices include subject specific databases and other sources of literature (Appendix I), grey literature sources (Appendix II), a search strategy template (Appendix III), a checklist for information retrieval activity (Appendix IV), and the literature search-specific items from the MEC2IR document (Appendix V) and list of abbreviations used in the guide (Appendix VI). This guide provides high-level, overview information on information retrieval principles and is not a substitute for the Help sections of individual databases. Individuals who wish to search particular sources should familiarize themselves with the resource before beginning a search. Campbell Coordinating Groups (CGs) are responsible for providing review authors with help locating references to studies that are possibly relevant to their review. The majority of CGs employ a dedicated Trials Search Co-ordinator (TSC) to provide this service (see Section 2.1.2). Thus, the information in this guide is designed to assist authors wishing to undertake supplementary searches for studies and to provide background information so that they can better understand the search process. In all cases review authors should contact the TSC of their CG before starting to search in order to find out the level of support they provide. This guide will also be useful to TSCs who are new to their post, as well those who are more experienced, who may wish to consult this guide as a reference source. The document outlines some general issues in searching for studies; describes the main sources of potential studies; and discusses how to plan the search process, design and carry out search strategies, manage references found during the search process and correctly document and report the search process. This document is meant to provide general guidance to reviewers and to establish minimum standards for key information retrieval tasks. Although the guide speaks specifically to individuals planning to conduct a Campbell review, the policies, procedures and guidelines are applicable to anyone interested in implementing information retrieval methods that maximise coverage and minimise bias during the information retrieval process. For a more comprehensive discussion of information retrieval for systematic reviews see White's chapter entitled Scientific Communication and Literature Retrieval(3) and other reports that have described searching challenges for systematic reviews of various topics (4-8). Thisguide's fundamental premise is that information retrieval is an essential component of the systematic review process, analogous to the data collection phase of a primary research study, and requires the expertise of TSC, an information specialist (IS) or a librarian. A thorough and unbiased compilation of all potentially relevant studies is one of the key characteristics of a systematic review and if the literature located is unrepresentative of the population of completed studies, the remainder of the review process will be compromised (9). Given the diverse nature of the research questions addressed in the social, behavioural, and educational sciences that encompass the Campbell Collaboration's core areas of focus (crime and justice, education, international development, and social welfare), potentially relevant studies are likely to be widely distributed and unreliably categorised. While retrieval of information from the literature is a critical concern for any systematic reviewer, retrieval of information about complex social, behavioural, and educational interventions is likely to be especially challenging. Systematic reviews of interventions require a thorough, objective and reproducible search of a range of sources to identify as many relevant studies as possible (within resource limits). This is one factor in distinguishing systematic reviews from traditional narrative reviews and helps to minimize bias and therefore assist in achieving reliable estimates of effects. A search of one database alone is typically not considered adequate. A systematic review showed that in the medical sciences, only 30% - 80% of all known published randomized trials were identifiable using MEDLINE (depending on the area or specific question) (10). Even if relevant records are in MEDLINE, it can be difficult to retrieve them (11, 12). For the social sciences, this is even more pronounced. Firstly, there is no apparent main source of randomized trials and secondly, it seems that studies in the social sciences more often lack a structured abstract and methodology terms compared to those of the medical sciences (13). Going beyond the main subject database (such as ERIC) is important not only for ensuring that as many relevant studies as possible are identified but also to minimize selection bias for those that are found. Relying exclusively on one database search may retrieve a set of studies unrepresentative of all studies that would have been identified through a comprehensive search of several sources. Time and budget restraints require the review author to balance the thoroughness of the search with efficiency in use of time and funds and the best way of achieving this balance is to be aware of, and try to minimize, the biases such as publication bias and language bias that can result from restricting searches in different ways. The Trials Search Co-ordinator (TSC) for each CG is responsible for providing assistance to authors with searching for studies for inclusion in their reviews. The range of assistance varies according to the resources available to individual CCGs but may include designing search strategies for the main bibliographic databases, running these searches in databases available to the CG, saving search results and sending them to authors, advising authors on how to run searches in other databases (including web search engines) and how to download results into their reference management software (see Section 6). Contact your TSC before you start searching to find out the level of assistance offered. If a CG is currently without a TCG, authors should seek the guidance of an academic librarian or information specialist. In the latter two instances, ideally these individuals should have experience conducting searches for systematic reviews. Systematic reviews typically identify studies as the unit of analysis. Individual studies maybe reported in multiple publications or be associated with other studies (e.g., post hoc analyses or surveillance studies). Related publications may report unique information or be a source of potentially unwanted duplicate data. Every possible effort should be made to flag associated or related publications for inspection by the review authors. This guide focuses on searching for studies on the effectiveness of interventions. Some reviews may cover other types of studies and hence need a different approach to searching, but the sources listed in this document may nevertheless be equally relevant for these reviews. For reviews concerning cost effectiveness specifically, authors may seek guidance on searching from chapter 7 in Evidence-Based Decisions and Economics(13) or methodologic reports (13-18). It is Campbellpolicy that all review authors and others involved in the Collaboration should adhere to copyright legislation. With respect to searching for studies, this refers in particular to adhering to the terms and conditions of use when searching databases and downloading records and adhering to copyright legislation when obtaining copies of articles. Review authors should seek guidance on this from their TSC or local academic librarian, as copyright legislation varies across jurisdictions. Social science-related subject databases are generally the best way to identify an initial set of relevant reports of studies within a specific field. The majority of field specific databases, such as ERIC and PsycINFO®, include abstracts, and may include links to the full-text of the scholarly literature within a given field or the full text of materials. Access to full text literature will vary based on a user's institutional affiliations. These databases usually index journal and non-journal sources, as well as non-English material. A key advantage of these databases is that they can be searched for keywords in the title or abstract and/or by using controlled vocabulary assigned to each record (see Section 5.4). Decisions related to which subject-specific databases are to be searched, in addition to the main field-related database, will be influenced by the topic of the review, access to specific databases, and budget considerations. Most of thesubject-specific databases are available only on a subscription or “pay-as-you-go” basis. Databases also may be available at no cost to the individual through national provisions, site-wide licences at institutions such as universities or hospitals, through professional organizations as part of their membership packages, or free of charge on the internet. Access to databases is therefore likely to be limited to those databases that are available to the TSC and those that are available at the institutions of the review authors. A selection of the main subject-specific databases that are more likely to be available through institutional subscriptions (and therefore available at no cost to the individual) or are available free of charge on the internet are listed in Appendix I, together with web addresses for further information. Access details vary according to institution. Review authors should seek advice from their local librarian for access at their institution. In addition to field-specific databases, there are also a variety of multi-disciplinary databases (Appendix I) which can beworth while to search. A comprehensive list of the databases in the social, behavioural, education and health sciencesis in Appendix I, however some of the more frequently used subject databases include: ASSIA provides access to literature in the health, social services, psychology, sociology, economics, politics, race relations and education. ASSIA is only available by subscription. CENTRAL serves as a comprehensive source of reports of controlled trials in medicine. CENTRAL is published as part of The Cochrane Library and is updated quarterly. CENTRAL contains citations to reports of trials and other studies potentially eligible for inclusion in Cochrane reviews from MEDLINE, EMBASE and other sources such as other databases and handsearching. CENTRAL also includes citations to reports of controlled trials that are not indexed in MEDLINE, EMBASE or other bibliographic databases; citations published in many languages; and citations that are available only in conference proceedings or other sources that are difficult to access (19). It also includes records from trials registers and registers of trials results. CENTRAL is available through The Cochrane Library. Many health and academic institutions and organizations provide access to their members, and in many countries there is free access for the whole population (for example through funded national licences or arrangements for low-income countries). Information about access to The Cochrane Library for specific countries can be found under Access to Cochrane at the top of The Cochrane Library home page: ERIC provides free access to millions of bibliographic records of journal articles and other education-related materials, such as journal articles, books, research syntheses, conference papers, technical reports, policy papers and other education-related materials. ERIC has special collections from the What Works Clearinghouse and the Regional Educational Laboratories, and also provides full-text access to grey literature, ranging from informational materials to researched and reviewed documents, including research syntheses, conference papers, and policy reports. IBSS is funded by the Economic and Social Research Council and has been produced by The London School of Economics and Political Science (LSE) since 1989. The IBSS database has a main focus on the social science subjects of anthropology, economics, politics and sociology. IBSS covers current social science journals from every region of the world, plus journals which are indexed on an irregular basis. In addition to printed journals IBSS also indexes electronic journals. Journal coverage includes articles, reviews, research notes, responses and short essays. MEDLINE MEDLINE indexes health sciences journal titles from numerous languages from the 1950s onwards. To see the list of currently indexed MEDLINE titles, from the Limit function, select the subset titled “Journals Currently indexed in MEDLINE” and click the Search button. PubMed provides access to a free version of MEDLINE and MEDLINE is available on subscription from numerous database vendors, such as Ovid. The US National Library of Medicine (NLM) has developed the NLM Gateway, which allows users to search MEDLINE or PubMed together with other NLM resources. PsycINFO®is a database of psychological literature from the 1800s to the present, containing bibliographic citations, abstracts and cited references across a wide variety of scholarly publications in the behavioral and social sciences. Sociological Abstracts indexes the international literature in sociology and related disciplines in the social and behavioural sciences. The database provides abstracts of journal articles and citations to book reviews, and also provides abstracts of books, book chapters, dissertations, and conference papers. Many countries and regions produce databases that concentrate on the literature produced in those regions, and which often include journals and other literature not indexed elsewhere. Access to many of these databases is available free of charge on the internet. Others are only available by subscription or on a pay-as-you-go basis. Indexing complexity and consistency varies, as does the sophistication of the search interface, but they can be an important source of additional studies from journal articles not indexed in other international databases. Some examples are listed in Appendix I. As well as searching the references cited in existing systematic reviews and meta analyses, reference lists of identified studies may also be searched for additional studies (20). This approach can complement the searches in the subject databases as it provides an alternative way of accessing potentially relevant studies. A recent study found that 7% of the included studies in a social science systematic review were located through citation searching alone (21). Many of the bibliographic databases allow for citation searches (known as cited reference searches), as does Google Scholar. See also 3.2.3. Web of Science is a database that lists published articles from over 10,000 major journals across scientific disciplines and links them to the articles in which they have been cited (a feature known as cited reference searching). This large database can be used to identify studies for a review by identifying a known relevant source article, and checking each of the articles citing the source article, to see if they are also relevant to the review. Records also include the listed references from the original record, which in turn are another possible source of relevant trial reports. Citation or reference searching is a way of searching forward in time from the publication of an important relevant article to identify additional relevant articles published since then. It is an important adjunct to database searching and handsearching (20), and may prove very effective.Web of Science can also be searched for articles indexed by author, title, topic, etc. Web of Science subscriptions may include access to the Science Citation Index Expanded, Arts and Humanities Citation Index, and Social Sciences Citation Index databases as well as other resources. This resource indexes leading international arts and humanities journals and relevant items from major science and social science journals. SCIE indexes major journals across multiple scientific disciplines including the agricultural, biological, and environmental sciences, engineering, technology, applied science, medical and life sciences, and physical and chemical sciences. Social Sciences Citation Index (SSCI) SSCI indexes journals spanning multiple disciplines related to the social sciences, as well as covering individually selected, relevant items from the world's leading scientific and technical journals. Scopus is an abstract and citations database which covers thousands of journals and conference proceedings, and results from scientific web pages and patent records. The full-text of an increasing number of journals is available electronically on a subscription basis or free of charge on the internet. In addition to providing a convenient method for retrieving the full article of already identified records, full-text journals can also be searched electronically, depending on the search interface, similarly to the way database records can be searched in a subject database. It is important to specify if the full text of a journal has been searched electronically. Some electronic journals omit sections of the print version, for example letters, and some include extra articles in electronic format only. Most academic institutions subscribe to a wide range of electronic journals and these are therefore available free of charge to members of those institutions. Review authors should seek advice about electronic journal access from the library service at their local institution. Some professional organizations provide access to a range of journals as part of their membership package. In some countries similar arrangements exist through national licences. There are also a number of international initiatives to provide free or low-cost online access to full-text journals (and databases) over the internet, including the Health InterNetwork Access to Research Initiative (HINARI), the International Network for the Availability of Scientific Publications (INASP) and Electronic Information for Libraries (eIFL). For further information on these initiatives see Section 3.1.1. Examples of some full-text journal sources that are available free of charge without subscription are given in Appendix I. It is recommended that a local electronic or print copy be maintained for any possibly relevant article found electronically in subscription journals, as the subscription to that journal may not be in
Stored only when both papers are in the corpus — 371 citations exist upstream.