A comparative study of the variables used to measure syntactic complexity and accuracy in task-based research

Chihiro Inoue

70 citations0/0 files storedLanguage Learning Journal

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

EFL/ESL Teaching and Learning Second Language Acquisition and Learning Linguistics and Language Analysis

Abstract

The constructs of complexity, accuracy and fluency (CAF) have been used extensively to investigate learner performance on second language tasks. However, a serious concern is that the variables used to measure these constructs are sometimes used conventionally without any empirical justification. It is crucial for researchers to understand how results might be different depending on which measurements are used, and accordingly, choose the most appropriate variables for their research aims. The first strand of this article examines the variables conventionally used to measure syntactic complexity in order to identify which may be the best indicators of different proficiency levels, following suggestions by Norris and Ortega. The second strand compares the three variables used to measure accuracy in order to identify which one is most valid. The data analysed were spoken performances by 64 Japanese EFL students on two picture-based narrative tasks, which were rated at Common European Framework of Reference for Languages (CEFR) A2 to B2 according to Rasch-adjusted ratings by seven human judges. The tasks performed were very similar, but had different degrees of what Loschky and Bley-Vroman term ‘task-essentialness’ for subordinate clauses. It was found that the variables used to measure syntactic complexity yielded results that were not consistent with suggestions by Norris and Ortega. The variable found to be the most valid for measuring accuracy was errors per 100 words. Analysis of transcripts revealed that results were strongly influenced by the differing degrees of task-essentialness for subordination between the two tasks, as well as the spread of errors across different units of analysis. This implies that the characteristics of test tasks need to be carefully scrutinised, followed by careful piloting, in order to ensure greater validity and reliability in task-based research.

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 70 citations exist upstream.

References

2009Common European Framework of Reference for Languages: learning, teaching, assessment 2004Applying the Rasch Model: Fundamental Measurement in the Human Sciences 1996A Framework for the Implementation of Task-based Instruction 2001Applying the Rasch Model 2001Researching Pedagogic Tasks: Second Language Learning, Teaching, and Testing 2001Task complexity, task difficulty, and task production: exploring interactions in a componential framework 2009Towards an Organic Approach to Investigating CAF in Instructed SLA: The Case of Complexity 2009Modelling Second Language Performance: Integrating Complexity, Accuracy, Fluency, and Lexis

Cited by

2017“I think that's what he's doing”: Effects of intentional reasoning on second language (L2) speech performance 2017L2 development in an intensive Study Abroad EAP context 2019The effects of hybrid online planning and L2 proficiency on video-based speaking task performance

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.