Estimating the Accuracy of Dichotomous Judgments

Abstract

A reliability study is assumed to be carried out with each of a number of observers making a dichotomous judgment concerning each of a sample of subjects. A nonparametric model is proposed for the errors underlying such judgments, and conditions are given under which Cochran's Q statistic is valid for testing the hypothesis of no systematic differences among the judgments of the different observers. Inferences concerning the probabilities of error are shown to be possible in terms of the intraclass correlation coefficient. A numerical example is given.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 58 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.