Evaluating English language learners’ conversations: Man vs. Machine

Abstract

This study investigated a new conversation-based assessment for English language learners. In this assessment, students converse with animated agents in natural language conversations to assess various aspects of English language learning. In a between subjects design, 31 students (N = 31) were asked questions by either the newly created system or human interviewers. Results revealed that there was no difference in answers in regards to impacting the constructs under investigation. However, there were other differences including higher word count and potentially more complex language usage when speaking to a human. Furthermore, students liked interacting with the system. These results suggest that the newly created system can assess students’ English capabilities comparably to human interviewers and implementing such a formative assessment is feasible.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 30 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.