Hypothetical nudges provide directional but noisy estimates of real behavior change

Abstract

Hypothetical scenarios provide an extremely useful alternative to field experiments for scholars interested in nudging behavior change, comprising a substantial proportion of the literature. Yet the extent to which hypotheticals accurately estimate real-world treatment effects is not well understood. To investigate, we identified five recent field studies of real-world nudges in distinct domains and designed four styles of hypothetical scenarios to approximate each one. This setup allows for clear comparison of old field data with new hypothetical data. Across our 20 experiments (N = 16,114), hypothetical scenarios nearly always estimated the correct direction of treatment effects. However, they varied widely in estimating magnitudes, making them unreliable inputs to real-world policy applications such as cost-benefit analyses. Our findings underscore the promising value of hypotheticals, but also the need for greater investigation into strategies to calibrate their estimates. Twenty experiments show that hypothetical scenarios (across four styles) correctly estimated whether or not behavioral interventions (i.e., nudges) would encourage behaviors in five field settings, but unreliably estimated the size of those effects.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 1 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.