Raters’ L2 background as a potential source of bias in rating oral performance

Paula M. Winke · Susan M. Gass · Carol M. Myford

236 citations0/0 files storedLanguage Testing

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

EFL/ESL Teaching and Learning Phonetics and Phonology Research Psychometric Methodologies and Testing

Abstract

Based on evidence that listeners may favor certain foreign accents over others (Gass & Varonis, 1984; Major, Fitzmaurice, Bunta, & Balasubramanian, 2002; Tauroza & Luk, 1997) and that language-test raters may better comprehend and/or rate the speech of test takers whose native languages (L1s) are more familiar on some level (Carey, Mannell, & Dunn, 2011; Fayer & Krasinski, 1987; Scales, Wennerstrom, Richard, & Wu, 2006), we investigated whether accent familiarity (defined as having learned the test takers’ L1) leads to rater bias. We examined 107 raters’ ratings on 432 TOEFL iBT TM speech samples from 72 test takers. The raters of interest were L2 speakers of Spanish, Chinese, or Korean, while the test takers comprised three native-speaker groups (24 each) of Spanish, Chinese, and Korean. We analyzed the ratings using a multifaceted Rasch measurement approach. Results indicated that L2 Spanish raters were significantly more lenient with L1 Spanish test takers, as were L2 Chinese raters with L1 Chinese test takers. We conclude by concurring with Xi and Mollaun (2009, 2011) and Carey et al. that rater training should address raters’ linguistic background as a potential rater effect. Furthermore, we discuss the importance of recognizing rater L2 as a possible source of bias.

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 236 citations exist upstream.

References

2003Applying the Rasch Model: Fundamental Measurement in the Human Sciences 2007Standards for Educational and Psychological Testing 2004Applying the Rasch Model: Fundamental Measurement in the Human Sciences 1995Foreign Accent, Comprehensibility, and Intelligibility in the Speech of Second Language Learners 1985Standards for educational and psychological testing 2005Second Language Accent and Pronunciation Teaching: A Research-Based Approach 2007Perceptual adaptation to non-native speech 2004Assessing Speaking

Cited by

2015Pronunciation Fundamentals 2015Using Listener Judgments to Investigate Linguistic Influences on L2 Comprehensibility and Accentedness: A Validation and Generalization Study 2014Second Language Comprehensibility Revisited: Investigating the Effects of Learner Background 2016An A–Z of Applied Linguistics Research Methods 2015Does a Speaking Task Affect Second Language Comprehensibility?2017

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.