Interrater reliability in scoring the Wisconsin card sorting test

Abstract

Neuropsychological instruments continue to proliferate the field, while studies systematically addressing their accuracy and standardization have lagged behind. This paper examines the standard scoring criteria on a widely used neuropsychological test, the Wisconsin Card Sorting Test (WCST). Specifically, the scoring criteria for the perseveration variables are complex and interrater reliability had not yet been established. Study 1 revealed excellent interrater reliability between three individuals experienced with the WCST who had independently learned the scoring procedures. Study 2 demonstrated excellent interrater agreement among six novice raters. Some of the novice scorers used supplemental scoring instructions that were developed in an attempt to improve understanding of the manual. The supplemental scoring instructions reduced training time by 41% without affecting the high reliability between the raters. This paper also illustrates an appropriate use of generalizability theory, a powerful statistical method to examine reliability.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 61 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.