A Monte Carlo Comparison of Measures of Relative and Absolute Monitoring Accuracy

Abstract

Researchers studying monitoring accuracy currently use two different indexes to estimate accuracy: relative accuracy and absolute accuracy. The authors compared the distributional properties of two measures of monitoring accuracy using Monte Carlo procedures that fit within these categories. They manipulated the accuracy of judgments (i.e., chance level or 60% and above) and the number of items per test (i.e., 20, 50, or 1,000) using 10,000 computer-generated cases. Gamma, an estimate of relative accuracy, yielded a skewed, leptokurtic distribution under the 50-item, 60% accuracy conditions. The Hamann coefficient, an estimate of absolute accuracy, yielded a normal distribution under the same conditions. Both statistics yielded normal distributions under the 1,000item, 60% conditions, although parameter estimates differed widely. The two statistics were similar, and normally distributed, under the 50and 1,000-item, chance conditions. Recommendations are made regarding the use of each measure in applied monitoring accuracy research.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 49 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.