How is AMSTAR applied by authors – a call for better reporting

Dawid Pieper, Nadja Könsgen, Jessica Breuing, Long Ge, Uta E Wegewitz

2018-06-18articleBMC Medical Research Methodology78 citationsgold

No readable copy stored yet — this record is metadata only.

Meta-analysis and systematic reviews Clinical practice guidelines implementation Delphi Technique in Research

Abstract

BACKGROUND: The assessment of multiple systematic reviews (AMSTAR) tool is widely used for investigating the methodological quality of systematic reviews (SR). Originally, AMSTAR was developed for SRs of randomized controlled trials (RCTs). Its applicability to SRs of other study designs remains unclear. Our objectives were to: 1) analyze how AMSTAR is applied by authors and (2) analyze whether the authors pay attention to the original purpose of AMSTAR and for what it has been validated. METHODS: We searched MEDLINE (via PubMed) from inception through October 2016 to identify studies that applied AMSTAR. Full-text studies were sought for all retrieved hits and screened by one reviewer. A second reviewer verified the excluded studies (liberal acceleration). Data were extracted into structured tables by one reviewer and were checked by a second reviewer. Discrepancies at any stage were resolved by consensus or by consulting a third person. We analyzed the data descriptively as frequencies or medians and interquartile ranges (IQRs). Associations were quantified using the risk ratio (RR), with 95% confidence intervals. RESULTS: We identified 247 studies. They included a median of 17 reviews (interquartile range (IQR): 8 to 47) per study. AMSTAR was modified in 23% (57/247) of studies. In most studies, an AMSTAR score was calculated (200/247; 81%). Methods for calculating an AMSTAR score varied, with summing up all yes answers (yes = 1) being the most frequent option (102/200; 51%). More than one third of the authors failed to report how the AMSTAR score was obtained (71/200; 36%). In a subgroup analysis, we compared overviews of reviews (n = 154) with the methodological publications (n = 93). The overviews of reviews were much less likely to mention both limitations with respect to study designs (if other studies other than RCTs were included in the reviews) (RR 0.27, 95% CI 0.09 to 0.75) and overall score (RR 0.08, 95% CI 0.02 to 0.35). CONCLUSIONS: Authors, peer reviewers, and editors should pay more attention to the correct use and reporting of assessment tools in evidence synthesis. Authors of overviews of reviews should ensure to have a methodological expert in their review team.

Citations in this library

Stored only when both papers are in the corpus — 78 citations exist upstream.

References

2009Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement

2009Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.

2016ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions

2009Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement

2017AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both

2007Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews

2001Assessing the quality of controlled clinical trials

2011Measurement in Medicine

Cited by

2020Use of AMSTAR-2 in the methodological assessment of systematic reviews: protocol for a methodological study

2020Authors should clearly report how they derived the overall rating when applying AMSTAR 2—a cross-sectional study

2023AMSTAR 2 is only partially applicable to systematic reviews of non-intervention studies: a meta-research study

2021Methodological quality was critically low in 9/10 systematic reviews in advanced cancer patients—A methodological study

2019How best to reduce unhealthy risk-taking behaviours? A meta-review of evidence syntheses of interventions using self-regulation principles