Benefits of open and high-powered research outweigh costs.

Etienne P. LeBel · Lorne Campbell · Timothy J. Loving

61 citations0/0 files storedJournal of Personality and Social Psychology

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

Academic Publishing and Open Access Research Data Management Practices Psychology Research and Bibliometrics

Abstract

Several researchers recently outlined unacknowledged costs of open science practices, arguing these costs may outweigh benefits and stifle discovery of novel findings. We scrutinize these researchers' (a) statistical concern that heightened stringency with respect to false-positives will increase false-negatives and (b) metascientific concern that larger samples and executing direct replications engender opportunity costs that will decrease the rate of making novel discoveries. We argue their statistical concern is unwarranted given open science proponents recommend such practices to reduce the inflated Type I error rate from .35 down to .05 and simultaneously call for high-powered research to reduce the inflated Type II error rate. Regarding their metaconcern, we demonstrate that incurring some costs is required to increase the rate (and frequency) of making true discoveries because distinguishing true from false hypotheses requires a low Type I error rate, high statistical power, and independent direct replications. We also examine pragmatic concerns raised regarding adopting open science practices for relationship science (preregistration, open materials, open data, direct replications, sample size); while acknowledging these concerns, we argue they are overstated given available solutions. We conclude benefits of open science practices outweigh costs for both individual researchers and the collective field in the long run, but that short term costs may exist for researchers because of the currently dysfunctional academic incentive structure. Our analysis implies our field's incentive structure needs to change whereby better alignment exists between researcher's career interests and the field's cumulative progress. We delineate recent proposals aimed at such incentive structure realignment. (PsycINFO Database Record

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 61 citations exist upstream.

References

1979The file drawer problem and tolerance for null results.1990The case for motivated reasoning.2015Estimating the reproducibility of psychological science 2013Power failure: why small sample size undermines the reliability of neuroscience 1998Confirmation Bias: A Ubiquitous Phenomenon in Many Guises 2011False-Positive Psychology 2005Why Most Published Research Findings Are False 2012Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling

Cited by

2022Construct validity and the validity of replication studies: A systematic review.2019No Replication, No Trust? How Low Replicability Influences Trust in Psychology 2017Falsifiability is not optional.2016A Call for Openness in Research Reporting: How to Turn Covert Practices Into Helpful Tools 2021Friendship Importance Around the World: Links to Cultural Factors, Health, and Well-Being 2017Replicability and other features of a high-quality science: Toward a balanced and empirical approach.

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.