Does aggregation produce spuriously high estimates of behavior stability?

Abstract

A number of arguments as to why aggregation produces spurious correlations are considered and shown to be specious. A series of computer-simulated studies, claimed by H. D. Day and his colleagues to demonstrate that aggregation produces artifactually high stability coefficients, was shown to involve bizarre distributions and inappropriate conclusions. Nevertheless, their data actually indicated the potency of aggregation in detecting whatever empirical relations they built into their data. This was not true of their analyses of their unaggregated data, which, as a result of a high level of noise of measurement, failed to enable them to detect anomalies in their data. No amount of aggregation produced evidence of stability in its absence. It is concluded that aggregation provides a powerful tool for reducing error of measurement and for establishing and enhancing generality as well as stability in measurement. Although there is nothing intrinsic in aggregation that fosters spurious correlations, it like any other technique can be used wisely or foolishly. In order to achieve the former, aggregation must be guided by psychometric principles and theoretical considerations. A procedure guaranteed to achieve the latter is to simply aggregate whatever can be assigned a common label.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 90 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.