Score generalizability of academic writing tasks: Does one test method fit it all?

Abstract

Generalizability of writing scores has always been a longstanding concern in L2 writing assessment. A number of studies have been conducted to investigate this topic during the last two decades. However, with the introduction of new test methods, such as reading-to-write tasks, generalizability studies need to focus on the score accuracy of different task types. The current study attempts to examine how reading-to-write tasks are compara ble to independent tasks in terms of score generalizability. To look into this issue, 115 Egyptian university students were asked to write on two independent tasks and two reading-to-write tasks. The essays were holistica lly scored by three raters, who received training prior to scoring. The study design followed a fully crossed univariate design (p × t × r), and consequently GENOVA (Crock & Brennan, 1983) was used to analyze the results. Results showed that the reading-to-write tasks yielded as reliable scores as independent tasks. In addition, the result indicated that score generalizability is very low when using one writing task due to the large (pt) variance component. Implications and limitations of the study are presented.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 96 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.