SIMULATION STUDIES APPLYING POSTERIOR PREDICTIVE MODEL CHECKING FOR ASSESSING FIT OF THE COMMON ITEM RESPONSE THEORY MODELS

Abstract

ABSTRACT Model checking in item response theory (IRT) is an underdeveloped area. No universal model‐checking tool exists in this field. Theposterior predictive model‐checkingmethod (Guttman, 1967; Rubin, 1981, 1984) is a popular Bayesian model‐checking tool because of its simplicity, strong theoretical basis, obvious intuitive appeal, and ability to provide graphical evidence. This study applies the posterior predictive model‐checking method to assess the fit of the popular IRT models (Rasch model, 2PL model, and 3PL model) for tests with dichotomous items. A series of simulation studies applying the method yields promising results. An important issue with the application of the posterior predictive model‐checking method is the choice ofdiscrepancy measures(which are like thetest statisticsin classical hypothesis testing). This article examines the performance of a number of discrepancy measures for assessing different aspects of fit of the common IRT models and suggests simple summarization of the results. Simple graphical summaries provide useful insight about the fit of the models. Theodds ratios, corresponding to the responses of the examinees to pairs of items, appear to be powerful discrepancy measures in detecting several types of misfits of the IRT models. The observed score distribution and biserial correlation coefficients are also found useful as discrepancy measures on a number of occasions. A companion article applying the posterior predictive method to real data examples shows that the method has the potential to become very useful in psychometrics.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 34 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.