Respondent Behavior in Magnitude Estimation

Abstract

Some researchers argue that data obtained using magnitude estimation have metric properties superior to those of data obtained using traditional survey items. The analysis in this article examines evidence relevant to the assumptions that respondents can express ratios using numbers, can locate an internal continuum, and can use the standard stimulus effectively. The data are answers to 5 practice items designed to teach magnitude estimation using line lengths and up to 18 items about stress experienced in caring for a sick person. The five practice items provide evidence of respondents' ability to give numeric estimates: 28% gave at least one answer that falls outside fairly tolerant boundaries that define a reasonable answer. Other results indicate that respondents do not improve with practice, that accuracy on the practice items does not predict good performance on magnitude estimates of a subjective property, and that respondents for whom the standard stress stimulus was hypothetical were more likely than others to answer inappropriately. Scale values calculated as medians or geometric means are accurate for the practice items and at least sensible for the care-giving items. Evaluating individual-level scores is more difficult. On the one hand, using a single point estimate entails considerable error, and unscalable answers may have disproportionate effects on individual level statistics. Furthermore, older, less educated, and male respondents find providing magnitude estimates difficult, possibly biasing analysis of individual scores. On the other hand, magnitude estimates may preserve important distinctions among individuals and may be less susceptible to bias arising from group differences in the use of the phrases attached to category ratings.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 27 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.