Validation of an automated scoring program for a digital complex figure copy task within healthy aging and stroke.

Abstract

OBJECTIVE: Complex Figure Copy Tasks are one of the most commonly employed neuropsychological tests. However, manual scoring of this test is time-consuming, requires training, and can then still be inconsistent between different examiners. We aimed to develop and evaluate a novel, automated method for scoring a tablet-based Figure Copy Task. METHOD: A cohort of 261 healthy adults and 203 stroke survivors completed the digital Oxford Cognitive Screen-Plus (OCS-Plus) Figure Copy Task. Responses were independently scored by two trained human raters and by a novel automated scoring program. RESULTS: Overall, the Automated Scoring Program was able to reliably extract and identify the separate figure elements (average sensitivity and specificity of 92.10% and 90.20%, respectively) and assigned total scores which agreed well with manual scores (intraclass correlation coefficient [ICC] = .83). Receiver Operating Curve analysis demonstrated that, compared to overall impairment categorizations based on manual scores, the Automated Scoring Program had an overall sensitivity and specificity of 80% and 93.40%, respectively (Area Under the Curve; AUC = 86.70%). Automated total scores also reliably distinguished between different clinical impairment groups with subacute stroke survivors scoring significantly worse than longer-term survivors, which in turn scored worse than neurologically healthy adults. CONCLUSIONS: These results demonstrate that the novel Automated Scoring Program was able to reliably extract and accurately score Figure Copy Task data, even in cases where drawings were highly distorted due to comorbid fine-motor deficits. This represents a significant advancement as this novel technology can be employed to produce immediate, unbiased, and reproducible scores for Figure Copy Task responses in clinical and research environments. Key Points-Question: We aimed to develop and evaluate a novel, automated method for scoring a tablet-based Figure Copy Task. FINDINGS: The novel Automated Scoring Program was able to reliably extract and accurately score Figure Copy Task data, even in cases where drawings were highly distorted due to comorbid fine-motor deficits. IMPORTANCE: This represents a significant advancement as this novel technology can be employed to produce immediate, unbiased, and reproducible scores for Figure Copy Task responses in clinical and research environments. Next Steps: Trialing the Automated Scoring Program in clinical environments. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 16 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.