Exploring Small, Confirming Big: An alternative system to The New Statistics for advancing cumulative and replicable psychological research

Abstract

While outlining his vision of The New Statistics, Cumming (2014) proposes that a more rigorous and cumulative psychological science will be built, in part, by having psychologists abandon traditional null-hypothesis significance testing (NHST) approaches, and conducting small-scale meta-analyses on their data whenever possible. In the present paper, I propose an alternative system for conducting rigorous and replicable psychological investigations, which I describe as Exploring Small, Confirming Big. I begin with a critical evaluation of the merits of NHST and small-scale meta-analyses, and argue that NHST does have a valuable role in the scientific process, whereas small-scale meta-analyses will do little to advance a cumulative science. I then present an overview of an alternative system for producing cumulative and replicable psychological research: Exploring Small, Confirming Big. It involves a two-step process to psychological research, consisting of (1) small N investigation(s), in which psychologists use NHST to develop exploratory models; and (2) strong, confirmatory tests of exploratory models, by analyzing new and/or existing large N datasets with variables that capture the effect(s) of interest from the Exploring Small stage. I conclude by discussing several anticipated benefits and challenges of adopting the Exploring Small, Confirming Big approach.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 71 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.