The File Drawer Problem in Reliability Generalization

Abstract

Meta-analytic reliability generalizations (RGs) are limited by the scarcity of reliability reporting in primary articles, and currently, RG investigators lack a method to quantify the impact of such nonreporting. This article introduces a stepwise procedure to address this challenge. First, the authors introduce a formula that allows researchers to estimate the lower bound population average reliability for a desired instrument. Second, they present an equation to determine the Fail-Safe N for RG. This equation estimates the number of ``file drawer'' studies required to drop the aggregate score reliability of an instrument below a specified criterion value. Finally, the authors demonstrate the utility of these equations using published RG studies. Comments on the conclusions drawn from each RG application are provided.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 30 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.