Not all effects are indispensable: Psychological science requires verifiable lines of reasoning for whether an effect matters.

Abstract

Psychological researchers currently lack guidance for how to make claims about and evaluate the practical relevance and significance of observed effect sizes, i.e. whether a finding will have impact when translated to a different context of application. Although psychologists have recently highlighted theoretical justifications for why small effect sizes might be practically relevant, such justifications fail to provide the information necessary for evaluation and falsification. Claims about whether an observed effect size is practically relevant need to consider both the mechanisms amplifying and counteracting practical relevance, as well as the assumptions underlying each mechanism at play. To provide guidance for making claims about whether an observed effect size is practically relevant in such a way that the claims can be systematically evaluated, we present examples of widely applicable mechanisms and the key assumptions needed for justifying whether an observed effect size can be expected to generalize to different contexts. Routine use of these mechanisms to justify claims about practical relevance has the potential to make researchers’ claims about generalizability substantially more transparent. This transparency can help move psychological science towards a more rigorous assessment of when psychological findings can be applied in the world.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 18 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.