Inferential statistics in Language Teaching Research : A review and ways forward

Abstract

This article reviews all (quasi)experimental studies appearing in the first 19 volumes (1997–2015) of Language Teaching Research ( LTR). Specifically, it provides an overview of how statistical analyses were conducted in these studies and of how the analyses were reported. The overall conclusion is that there has been a tight adherence to traditional methods and practices, some of which are suboptimal. Accordingly, a number of improvements are recommended. Topics covered include the implications of small average sample sizes, the unsuitability of p values as indicators of replicability, statistical power and implications of low power, the non-robustness of the most commonly used significance tests, the benefits of reporting standardized effect sizes such as Cohen’s d, options regarding control of the familywise Type I error rate, analytic options in pretest–posttest designs, ‘meta-analytic thinking’ and its benefits, and the mistaken use of a significance test to show that treatment groups are equivalent at pretest. An online companion article elaborates on some of these topics plus a few additional ones and offers guidelines, recommendations, and additional background discussion for researchers intending to submit to LTR an article reporting a (quasi)experimental study.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 75 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.