Item Selection in Computerized Adaptive Testing: Improving the a-Stratified Design with the Sympson-Hetter Algorithm

Abstract

Item exposure control, test-overlap minimization, and the efficient use of item pool are some of the important issues in computerized adaptive testing (CAT) designs. The overexposure of some items and high test-overlap rate may cause both item and test security problems. Previously these problems associated with the maximum information (Max-I) item selection method have been partially solved by incorporating the Sympson-Hetter (SH) probabilistic procedure that attempts to control the actual usage of the popular high-discrimination items. On a different line of thought, Chang and Ying proposed the a-stratified design (STR) and advocated the use of less discriminating items in the earlier stages of testing. The method has been demonstrated to be effective in improving the utilization of the entire pool without sacrificing the efficiency in ability estimation. Nevertheless, the problem of item overexposure still persists when the ratio of pool size to test length is small. In four simulation studies, the possible benefits of incorporating the SH procedure in the STR method were investigated. The performance of such an enhanced stratified method (STR-SH) was compared with that of STR as well as Max-I-SH. The results indicated the potential advantages of the STR-SH design over the original STR in yielding a more balanced item exposure distribution, lowering the test-overlap rate, and remarkably reducing the number of overexposed items.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 42 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.