Different methods elicit different belief distributions.

Abstract

When eliciting people’s forecasts or beliefs, you can ask for a point estimate – e.g., what is the most likely state of the world? – or you can ask for an entire distribution of beliefs – e.g., how likely is every possible state of the world? Eliciting belief distributions potentially yields more information, and researchers have increasingly tried to do so. In this paper, we show that different elicitation methods elicit different belief distributions. We compare two popular methods used to elicit belief distributions: Distribution Builder and Sliders. In ten pre-registered studies (N = 14,553), we find that Distribution Builder elicits more accurate belief distributions than Sliders, except when true distributions are right-skewed, for which the results are mixed. This result holds when we assess accuracy (1) relative to a normative benchmark and (2) relative to participants’ own beliefs. Our evidence suggests that participants approach these two methods differently: Sliders users are more likely to start with the lowest bins in the interface, which in turn leads them to put excessive mass in those bins. Our research sheds light on the process by which people construct belief distributions while offering a practical recommendation for future research: All else equal, Distribution Builder yields more accurate belief distributions.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 3 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.