Have your Cake and Eat it Too

Abstract

Learning from captioned foreign language videos is highly effective, but the availability of such videos is limited. By using speech-to-text technology to generate partially correct transcripts as a starting point, we see an opportunity for learners to build accurate foreign language captions while learning at the same time. We present a system where learners correct captions using automatic transcription and machine-generated suggested alternative words for scaffolding. In a lab study of 49 participants, we found that compared to watching the video with accurate caption, learning and quality of experience were not significantly impaired by the secondary caption correction task using interface designs either with or without scaffolding from speech-to-text generated alternative words. Nevertheless, aggregating corrections reduced word error rate from 19% to 5.5% without scaffolding from suggested-alternatives, and 1.8% with scaffolding. Feedback from participants suggest that emphasizing the learning community contribution aspect is important for motivating learners and reducing frustration.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 30 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.