The reliability of recorded text test scores: widespread inconsistent intelligibility testing in minority languages

Abstract

The recorded text test (RTT) is commonly used to test dialect intelligibility, often to inform language development decisions. More than 25 papers using the RTT method were published on www.sil.org/silesr from January 2009 to March 2013. As introduced by Casad [1974. Dialect Intelligibility Testing. Summer Institute of Linguistics Publications in Linguistics and Related Fields, publication no. 38. Norman: Summer Institute of Linguistics of the University of Oklahoma], the method involves recording a narrative text, developing, and piloting an audio test based on the text, and using this test with speakers of speech varieties related to the text’s speech variety. This is the first study to explore the reliability of RTT scores in a generalisable way. This study demonstrates that the selection of a text to be developed into an RTT has significant implications on the results of the RTT which could lead to incorrect recommendations about how well material produced in one speech variety could be used by speakers of related speech varieties. The 10 questions Casad (1974, 117) recommends may not be sufficient to obtain a consistent measure of intelligibility.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 2 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.