Comment on: heterogeneity in meta-analysis should be expected and appropriately quantified

Abstract

Higgins1 in his commentary on the study by Patsopoulos and coworkers2 outlines some of the limitations of I2 in assessing heterogeneity in a meta-analysis. These limitations have perhaps not been as explicitly explained before. They have particular implications for assessing heterogeneity in meta-analyses of cohort and database studies and this aspect is worthy of further comment. As pointed out by Higgins, I2 is not a point estimate of the between-study heterogeneity.1 It is the proportion of total variation in the point estimates that is attributable to between-study heterogeneity. The total variation is comprised of between-study variation and within-study variation. If the individual studies have large sample size, such that the within-study variation is small, then I2 could be large (say >75%), even if the between-study variation is small. This is particularly a problem for meta-analyses of cohort and database studies, which typically have large sample sizes. The advantages I2 are that it does not depend on the number of studies in the meta-analysis or the effect measure used (e.g. odds ratio, risk ratio, hazard ratio, mean).3 However, it does depend on the size (precision) of the individual studies.4 It might be sensible to compare I2 across different meta-analyses, where the component randomised controlled trials (RCTs) all have about the same sample size. However, it is not sensible to compare I2 across meta-analyses of RCTs (sample size in the 100s) and meta-analyses of cohort or database studies (sample size in the 1000s).

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 145 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.