Article2006

Dependability of scores for a new ESL speaking assessment consisting of integrated and independent tasks

115 citations0/0 files storedLanguage Testing

No readable copy stored yet — this record is metadata only.

Psychometric Methodologies and Testing EFL/ESL Teaching and Learning Student Assessment and Feedback

Abstract

A multitask speaking measure consisting of both integrated and independent tasks is expected to be an important component of a new version of the TOEFL test. This study considered two critical issues concerning score dependability of the new speaking measure: How much would the score dependability be impacted by (1) combining scores on different task types into a composite score and (2) rating each task only once? To answer these questions, generalizability theory (G-theory) procedures were used to examine the impact of the numbers of tasks and raters per speech sample and of subsection lengths on the dependability of speaking scores. Univariate and multivariate G-theory analyses were conducted on rating data collected for 261 examinees for the study. The finding in the univariate analyses was that it would be more efficient to increase the number of tasks rather than the number of ratings per speech sample in maximizing the score dependability. The multivariate G-theory analyses also revealed that (1) the universe (or true) scores among the task-type subsections were very highly correlated and that (2) slightly larger gains in composite score reliability would result from increasing the number of listening - speaking tasks for the fixed section lengths.

Citations in this library

Stored only when both papers are in the corpus — 115 citations exist upstream.

References

1991Fundamental Considerations in Language Testing 1992Generalizability theory: a primer 2001Generalizability Theory 2014Testing Second Language Speaking 1992Elements of Generalizability Theory 1993Sampling Variability of Performance Assessments 1994Understanding and Developing Language Tests 1999Validating Measures of Performance

Cited by

2014What accounts for integrated reading-to-write task scores?2015Improving the Validity of Quantitative Measures in Applied Linguistics Research1 2020Assessing Speaking Proficiency: A Narrative Review of Speaking Assessment Research Within the Argument-Based Validation Framework 2019Is anybody listening? The nature of second language listening in integrated listening-to-summarize tasks 2018Assessing L2 Listening 2010Bringing reading-to-write and writing-only assessment tasks together: A generalizability analysis 2016Exploring the Relationships Between Independent Listening and Listening-Reading-Writing Tasks in Chinese Language Testing: Toward a Better Understanding of the Construct Underlying Integrated Writing Tasks

Imported from OpenAlex (CC0) — nothing here is generated.