Detection and Determinants of Bias in Subjective Measures

Abstract

Many concepts in sociology are difficult or impossible to objectively measure. This limitation forces a reliance on subjective measures that typically contain both systematic and random measurement errors. Systematic errors, or biases, are the focus of this paper. Donald T. Campbell and Donald W. Fiske's multitrait-multimethod (MTMM) research design is the best known social scientific procedure for uncovering systematic errors, but the data requirements for classical MTMM designs are too demanding for many areas of sociology in which secondary data are the norm. The authors show that the benefits of the MTMM design are available under more relaxed conditions. In addition, they illustrate how researchers can examine the determinants of systematic errors and gain insights into the potential for confounding or spurious effects caused by systematic errors. They demonstrate the usefulness of these methods using the subjective measures of liberal democracy used in several recent «American Sociological Review» papers and provide additional examples, including measures of the reputational quality of graduate programs and job evaluations for comparable-worth investigations. They conclude that sociologists can do far more to understand the systematic error present in their subjective variables

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 140 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.