Quantifying the risk of error when interpreting funnel plots

Abstract

BACKGROUND: Funnel plots are widely used to investigate possible publication bias in meta-analyses. There has, however, been little formal assessment of whether a visual inspection of a funnel plot is sufficient to identify publication bias. METHODS: Visual assessment of bias in a funnel plot is quantified using two new statistics: the Imbalance and the Asymmetry Distance, both intended to replicate how a funnel plot is typically assessed. A simulation study was performed to assess the performance of these two statistics for identifying publication bias. RESULTS: The two statistics both have high type I error and low statistical power, unless the number of studies in the meta-analysis is very large. These results suggest that visual inspection of a funnel plot is unlikely to lead to a valid assessment of publication bias. CONCLUSIONS: In most systematic reviews, visual inspection of a funnel plot may give a misleading impression of the presence or absence of publication bias. Formal statistical tests for bias should generally be preferred.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 251 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.