Aggregating multiple probability intervals to improve their calibration

Abstract

Many empirical studies have shown that interval probability estimates are too narrow (overconfident). We show that the 'Wisdom of Crowds' can mitigate the bias and improve the accuracy of the estimates by combining individual intervals. We re-analyzed data from the studies of Glaser, Langer, and Weber (2012) and Soll and Klayman (2004) . We applied 5- Averaging, Median, Enveloping, Probability averaging, and Quartiles- to combine the upper and lower bounds of the individual judges. Several measures were implemented for evaluating the methods. All methods were able to correct for the common miscalibration at different degrees and Quartiles was the most beneficial securing accuracy and informativeness.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 14 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.