Intrarater Reliability

Abstract

Abstract Intrarater reliability refers to the ability of a rater or a measurement system to reproduce quantitative or qualitative outcomes under the same experimental conditions. In this article, we review two statistical measures often used in the literature for quantifying intrarater reliability. The first measure is the intraclass correlation coefficient (ICC), which is the preferred measure for continuous data. The second measure is the kappa coefficient and its many kappa‐like variants recommended for nominal data. Also included in this article are guidelines for optimizing the number of subjects and number of replicates per subject when designing an intrarater reliability study for quantitative measurements. We also explore important extensions of the kappa coefficient to ordinal and interval data, which may also be used to analyze multivariate ratings obtained by rating subjects on multiple characteristics. For the intraclass correlation coefficient and kappa‐like statistics, we discuss a framework for statistical inference within which to evaluate the precision of estimations.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 85 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.