HOW DO RATERS FROM INDIA PERFORM IN SCORING THE TOEFL IBT™ SPEAKING SECTION AND WHAT KIND OF TRAINING HELPS?

Abstract

ABSTRACT This study investigated the scoring of the Test of English as a Foreign Language™ Internet‐based Test (TOEFL iBT™) Speaking section by bilingual or multilingual speakers of English and 1 or more Indian languages. We explored the extent to which raters from India, after being trained and certified, were able to score the Speaking section for TOEFL iBT examinees with mixed first language (L1) backgrounds, especially those speaking an Indian language, accurately and consistently. The effectiveness of a special training package designed for scoring Indian examinees was examined as well. A total of 26 trained and certified raters from India were randomly divided into 2 groups and participated in 2 on‐site scoring sessions in Mumbai. In the first session, both groups received regular training for scoring the TOEFL iBT Speaking section, which was largely similar to that received by raters in North America. In the second scoring session, 1 group continued to receive the regular training while the second group was trained using a special training package. Rater feedback surveys were also given to the raters. It was found that with training similar to that which operational U.S.‐based raters receive, the raters from India performed as well as the operational raters in scoring both Indian and non‐Indian examinees. In addition, the special training helped the raters score Indian examinees more consistently, leading to increased score reliability estimates. It also boosted raters' levels of confidence in scoring Indian examinees.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 26 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.