Attention Checks in Decisions From Experience: The Case of Checking Experiments

Abstract

Can online participants be compared to laboratory subjects. In the five studies (Ntotal = 1519) reported in this article, we comprehensively compared the behavior of online attentive and inattentive participants; i.e., those who passed or failed a simple attention check. The findings show that in a decisions from experience paradigm (i.e., a multi-trial repeated choice task), even one simple attention check is sufficient to differentiate between attentive and inattentive participants. The validity of this separation is further evidenced in the significant difference in the reported conscientiousness measure.The results fully replicated three previously run lab studies for the attentive participants, but not for the inattentive participants. This highlights the importance of using attention checks to avoid spurious conclusions. There are three explanations for the behavioral differences between the attentive and inattentive participants. Attentive participants exhibited a pre-disposition towards checking. They also had a greater likelihood of noticing distinct outcomes. Finally, the inattentive participants tended to behave more randomly. When checking was beneficial, these three effects pointed in the same direction, leading to a major difference between participants. When checking was detrimental, these effects cancelled each other out, resulting in similar checking rates and indistinguishable behavior.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 2 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.