Simulation-Based Comparison of Methods for Meta-Analysis of Proportions and Rates

Abstract

Background In many systematic reviews it is appropriate to summarize proportions and rates (e.g., incidence rates) using meta-analysis. For example, researchers commonly perform meta-analyses of sensitivity and specificity to summarize medical test performance, or of adverse or harmful events. Many statistical methods can be used for meta-analysis of rates and proportions. Purpose To help provide guidance for meta-analysts, we performed an extensive simulation study to assess the statistical properties of alternative approaches to meta-analysis of proportions and incidence rates. Methods We simulated a large number of scenarios for meta-analyses of proportions and incidence rates (n=792 scenarios for each). The distinct scenarios were defined by combinations of various factors, including the distributional form for the true summary proportion or rate and its defining parameters (mean, variance), the number of studies per meta-analysis, and the number of patients per study. For each scenario we generated 1000 random meta-analyses, on which we applied fixed and random effects analyses for two families of methods: (1) methods that approximate within-study variability with a normal distribution—not using a transformation, using a canonical transformation (logit and logarithmic for proportions and rates, respectively), or using a variance stabilizing transformation (arcsine and square root for proportions and rates, respectively); and (2) “discrete likelihood” methods that use the theoretically motivated binomial or Poisson distribution to model within study variability. We measured the performance of each method relative to the true values set in the simulation by their mean squared error, bias, and coverage. Results In general, and for both proportions and rates, the discrete likelihood approaches performed better than the approximate methods in terms of the three metrics. Of the approximate methods, the variance stabilizing variants (arcsine transformation for proportions and square root transformation for rates) performed better than the untransformed methods or the methods using a canonical link. Continuity correction factors are necessary to calculate real-valued means or variances for some approximate methods. The bias, mean square error and coverage of these approximate methods are very sensitive to the choice of continuity correction factors. Conclusions Discrete likelihood methods are preferable for the meta-analyses of proportions and rates. We discourage the use of approximate methods that require continuity corrections, as the arbitrary choice of the correction factor can greatly impact on the performance of the method. If software for fitting the discrete likelihood methods is unavailable and expected counts are large enough that normal approximations are adequate, we recommend use of a variance stabilizing transformation.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 55 citations exist upstream.