Taking the Time to Improve the Validity of Low‐Stakes Tests: The Effort‐Monitoring CBT

Steven L. Wise · Dennison S. Bhola · Sheng‐Ta Yang

70 citations0/0 files storedEducational Measurement Issues and Practice

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

Psychometric Methodologies and Testing Psychological Testing and Assessment Educational Technology and Assessment

Abstract

The attractiveness of computer‐based tests (CBTs) is due largely to their capability to expand the ways we conduct testing. A relatively unexplored application, however, is actively using the computer to reduce construct‐irrelevant variance while a test is being administered. This investigation introduces the effort‐monitoring CBT, in which the computer monitors examinee effort (based on item response time) in a low‐stakes test and displays warning messages to those exhibiting rapid‐guessing behavior. The results of an experimental study are presented, which showed that an effort‐monitoring CBT increased examinee effort and yielded more valid test scores than a conventional CBT. Thus, unlike previous research that has focused on identifying rapid‐guessing behavior after it has occurred, the effort‐monitoring CBT proactively attempts to suppress rapid‐guessing behavior. This innovative testing procedure extends the capabilities of measurement practitioners to manage the psychometric challenges posed by unmotivated examinees.

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 70 citations exist upstream.

References

2005Response Time Effort: A New Measure of Examinee Motivation in Computer-Based Tests 2005Low Examinee Effort in Low-Stakes Assessment: Problems and Potential Solutions 2004Construct‐Irrelevant Variance in High‐Stakes Testing 2006An Application of Item Response Time: The Effort‐Moderated IRT Model 1997Modeling Item Response Times With a Two‐State Mixture Model: A New Method of Measuring Speededness 2006An Investigation of the Differential Effort Received by Items on a Low-Stakes Computer-Based Test 2007Setting the Response Time Threshold Parameter to Differentiate Solution Behavior From Rapid-Guessing Behavior

Cited by

2020The relationship between performance and test-taking effort when measured with self-report or time-based instruments: A meta-analytic review 2011A model of examinee test-taking effort.2015The Effects of Motivational Instruction on College Students' Performance on Low-Stakes Assessment 2016Change in test-taking motivation and its relationship to test performance in low-stakes assessments 2019The Onset of Rapid-Guessing Behavior Over the Course of Testing Time: A Matter of Motivation and Cognitive Resources

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.