Practical issues in developing and maintaining acomputerized adaptive testing program

Abstract

The basic principles of computerized adaptive testing are relatively straightforward. The practice of implementing and maintaining an adaptive testing program, however, is far more complex. A number of practical challenges await measurement professionals; we discuss four general types: establishing and maintaining item pools, choosing test administration procedures, protecting test security, and responding to examinee issues. The success of an adaptive testing program will largely depend on how well the measurement practitioner deals with these challenges. The computerized adaptive test (CAT) has become increasingly common in large-scale testing programs. The primary advantage of a CAT to test developers and administrators is its promise of efficient testing. In theory, examinee testing times can be dramatically reduced while maintaining the quality of measurement provided by conventional (i.e., fixed-item) tests. This advantage is particularly attractive to testing programs that have traditionally required lengthy tests. In such testing contexts, the potential problem of examinee fatigue and, consequently, diminished effort can be alleviated by use of a CAT. Virtually all operational CATs use measurement methods based on item response theory (IRT; Lord & Novick, 1968) to select test items to administer and to estimate examinee proficiency. The invariance principle of IRT allows one to administer different sets of items drawn from an item pool to different examinees, yet estimate their relative levels of proficiency on a common scale of measurement. A CAT’s efficiency is realized through the targeting of item difficulty to examinee proficiency. IRT principles suggest that items targeted in this manner provide maximal information in the estimation of examinee proficiency.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 46 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.