ESTIMATING THE EFFECTS OF TEST LENGTH AND TEST TIME ON PARAMETER ESTIMATION USING THE HYBRID MODEL

Kentaro Yamamoto

78 citations0/0 files storedETS Research Report Series

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

Psychometric Methodologies and Testing Educational Assessment and Pedagogy Educational Technology and Assessment

Abstract

ABSTRACT When individuals perform tasks, they differ from each other not only in their ability to perform the tasks correctly, but also in their speed. Even though the traditional indicator of test speededness, missing responses, clearly indicates a lack of time to respond (thereby indicating the speededness of the test), it is inadequate for evaluating speededness in a multiple‐choice test scored as number correct and underestimates test speededness. Conventional IRT parameter estimation ignores the mixture of random responses during calibration; consequently, estimated parameters are biased. The HYBRID model (Yamamoto, 1989) was extended (Yamamoto, 1990) to characterize when each examinee switches from an ability‐based response strategy to a strategy of responding randomly. The model has allowed us to evaluate test speededness by estimating the proportions of examinees who switch strategies at any possible point in the test. The estimated IRT parameters based on the HYBRID model were more accurate than the ordinary IRT‐only analysis. With the extended HYBRID model applied to the data taken from an experimental form of TOEFL®, we found that 1) the test length had a small impact on the proportion of the examinees affected by the speededness of the test, 2) a greater proportion of examinees were affected by speededness of a test with a 50‐minute time limit than a test with a 55‐ or 60‐minute time limit, and 3) the difference in the proportions of examinees affected by speededness of tests under 55‐ and 60‐minute time limits was small. However, nearly 20 percent of the examinees were affected by speededness after completing 80 percent of the test. In other words, the last 20 percent of the responses of 20 percent of the examinees did not represent their true ability.

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 78 citations exist upstream.

References

1987Factor Analysis and AIC 2012Applications of Item Response Theory To Practical Testing Problems 1981Marginal Maximum Likelihood Estimation of Item Parameters: Application of an EM Algorithm 1990Modeling Item Responses When Different Subjects Employ Different Solution Strategies 1986Bayes Modal Estimation in Item Response Models 1985BAYES MODAL ESTIMATION IN ITEM RESPONSE MODELS*1982HYBRID MODEL OF IRT AND LATENT CLASS MODELS

Cited by

2017Conditioning factors of test-taking engagement in PIAAC: an exploratory IRT modelling approach considering person and item characteristics 2007A Speeded Item Response Model with Gradual Process Change 2007Bayesian IRT Guessing Models for Partial Guessing Behaviors 2003A Method for Maintaining Scale Stability in the Presence of Test Speededness 2015Detection of Test Speededness Using Change-Point Analysis 2019Using Response Times to Model Not-Reached Items due to Time Limits 2006

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.