A tutorial for calculating field-specific effect size distributions

Abstract

Effect sizes are useful for understanding the magnitude of study results and for planning new studies via power analysis. However, despite their wide usage, effect sizes are often misinterpreted. This is mostly due to an over-reliance on general effect size benchmarks that were not intended for broad application across diverse research fields. Inaccurate effect size interpretations can lead to incorrect conclusions about the magnitude of study results and incorrect sample size estimates, thereby increasing the likelihood of false-positive results. This article introduces the ESDist R package, which is designed to calculate empirically derived effect-size benchmarks or a range of reliably detectable empirical effect sizes for a specific research question or field of interest by computing effect size distributions (ESDs). This package can be used on data that can be easily extracted from pre-existing meta-analyses to help researchers more accurately plan new studies or to better understand how an individual study might relate to other studies in their field. ESDist includes a set of features that make it easy to use in a priori power analysis. Moreover, the package includes a feature for estimating effect size benchmarks that account for publication bias and are weighted by effect sizes' variances, which addresses existing limitations of using ESDs for study planning or interpretation.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 7 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.