Can We Trust the Results of Meta-Analyses?: A Systematic Approach to Sensitivity Analysis in Meta-Analyses

Abstract

Every meta-analysis involves a number of choices made by the analyst. These choices may refer to, for example, estimator of effect, model for analysis (fixed-effects or random effects), or the treatment of varying study quality. The choices made can affect the results of the analysis. Every meta-analysis should therefore include a sensitivity analysis, designed to probe how choices made as part of the analysis affects its results. This paper describes a systematic approach to sensitivity analysis in meta-analyses. An index intended to summarize the results of a sensitivity analysis, the robustness score, is developed. The robustness score varies from 0 to 1. A value of 1 indicates that the results of a metaanalysis are very robust, meaning that they are not at all affected by the choices made by the analyst. It is proposed that every meta-analysis should include a sensitivity analysis with respect to: (1) The potential presence of publication bias, (2) Choice of estimator of effect (if relevant), (3) The possible presence of outlier bias (a single result having decisive influence on the summary estimate), (4) Statistical weighting of individual estimates of effect, (5) Assessment of study quality. A recently reported meta-analysis of studies that have evaluated the effects on road safety of daytime running lights for cars is used as a case in order to explain the approach to sensitivity analysis proposed in the paper.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 18 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.