Separating the Brier Score into Calibration and Refinement Components: A Graphical Exposition

Abstract

Proper scoring rules of subjective probability assessments have been shown to be separable into distinct calibration and refinement components. This article presents a graphical description of this separation theorem as applied to the Brier score (quadratic loss) of assessed probabilities for a sequence of observable events. Configurations of achievable calibration, refinement, and Brier scores are exhibited in three-dimensional space and by projection into interpretable subspaces. Relationships of calibration and refinement to the usual sum-of-squares partition in analysis of variance are denoted. Controversy concerning the implications of long-run calibration for probability theory based solely on the principle of coherence is outlined.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 74 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.