Statistical Rigor and the Perils of Chance

Abstract

There is increasing awareness of the problem of unreliable findings across biomedical sciences (Ioannidis, 2005).Many "landmark" findings could not be replicated (Scott et al., 2008;Begley and Ellis, 2012;Steward et al., 2012) and many promising preclinical findings have failed to translate into clinical application (Perel et al., 2007;Prinz et al., 2011), leading many to question whether science is broken (Economist,2013).Central to this problem is a poor appreciation of the role of chance in the scientific process.As neuroscience has developed over the past 50 years, many of the large, easily observable effects have been found, and the field is likely pursuing smaller and more subtle effects.The corresponding growth in computational capabilities (Moore, 1998) means that researchers can run numerous tests on a single dataset in a matter of minutes.The human brain processes randomness poorly, and the huge potential for undisclosed analytical flexibility in modern datamanagement packages leaves researchers increasingly vulnerable to being fooled by chance.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 12 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.