Descriptive and Inferential Procedures for Assessing Differential Item Functioning in Polytomous Items

Rebecca Zwick · Dorothy T. Thayer · John Mazzeo

74 citations0/0 files storedApplied Measurement in Education

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

Psychometric Methodologies and Testing Educational Assessment and Pedagogy Educational Technology and Assessment

Abstract

Differential item functioning (DIF) assessment procedures for items with more than 2 ordered score categories were evaluated. Three descriptive statistics—the standardized mean difference (SMD; Dorans & Schmitt, 1991) and 2 procedures based on SIBTEST (Shealy & Stout, 1993)—were considered, along with 5 inferential procedures: 2 based on SMD, 2 based on SIBTEST, and the Mantel (1963) method. A simulation showed that, when the 2 examinee groups had the same distribution, the descriptive index that performed best was the SMD. When the group means differed by 1 SD, a modified form of the SIBTEST DIF effect size measure tended to perform best. The 5 inferential procedures performed almost indistinguishably when the 2 groups had identical distributions. When the groups had different distributions and the studied item was highly discriminating, the SIBTEST procedures showed much better Type I error control than did the SMD and Mantel methods, particularly in short tests. The power ranking of the 5 procedures was inconsistent; it depended on the direction of DIF and other factors. Routine application of these polytomous DIF methods seems feasible when a reliable test is available for matching examinees. The Type I error rates of the Mantel and SMD methods may be a concern under certain conditions. The current version of SIBTEST cannot easily accommodate matching tests that do not use number-right scoring. Additional research in these areas would be useful.

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 74 citations exist upstream.

References

1969Statistical Theories of Mental Test Scores 1982A Rasch Model for Partial Credit Scoring 1992A Generalized Partial Credit Model: Application of an EM Algorithm 2013Differential Item Performance and the Mantel–Haenszel Procedure 1993A Model-Based Standardization Approach that Separates True Bias/DIF from Group Ability Differences and Detects Test Bias/DTF as well as Item Bias/DIF 1986DEMONSTRATING THE UTILITY OF THE STANDARDIZATION APPROACH TO ASSESSING UNEXPECTED DIFFERENTIAL ITEM PERFORMANCE ON THE SCHOLASTIC APTITUDE TEST 1993Assessment of Differential Item Functioning for Performance Tasks 1992On the Misuse of Manifest Variables in the Detection of Measurement Bias

Cited by

2018Structural validation of the Self-Compassion Scale with a German general population sample 2003Applying the Liu‐Agresti Estimator of the Cumulative Common Odds Ratio to DIF Detection in Polytomous Items 2013Decisions that make a difference in detecting differential item functioning 2000Using Loss Functions for DIF Detection: An Empirical Bayes Approach

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.