Lady Justice v. Cult of Statistical Significance: Oomph-Less Science and the New Rule of Law

Abstract

We have an ethical problem in economics and other sciences using null hypothesis statistical significance testing without a loss function — a test that avoids asking, How Big is a Big Loss or Gain? Statistical significance is not equivalent to economic significance, nor to medical, clinical, nor any other kind of scientific significance — those functions of gain and loss. The mistake in the falsely made equation is evident when one reflects that the estimated payoff from a lottery is not the same object as the odds of winning that lottery. Yet a widespread failure to make the distinction between an estimate of human consequence and an estimate of its probability — between the meaning of an estimated average and the random variance around it — is killing people in medicine and impoverishing people in economics (Ziliak and McCloskey 2008). The ethical problem created by a test of statistical significance is made worse by the method’s blatant illogic at the foundational level, a fact that is unacknowledged by the bulk of decision makers depending upon it. Several changes to the scientific paper — and a recent decision by the Supreme Court of the United States — could help.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 3 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.