Accounting for Statistical Artifacts in Item Bias Research

Lorrie A. Shepard · Gregory Camilli

129 citations0/0 files storedJournal of Educational Statistics

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

Psychometric Methodologies and Testing Educational Assessment and Pedagogy Educational Technology and Assessment

Abstract

Theoretically preferred IRT bias detection procedures were applied to both a mathematics achievement and vocabulary test. The data were from black and white seniors on the High School and Beyond data files. To account for statistical artifacts, each analysis was repeated on randomly equivalent samples of blacks and whites ( n’s = 1,500). Furthermore, to establish a baseline for judging bias indices that might be attributable only to sampling fluctuations, bias analyses were conducted comparing randomly selected groups of whites. To assess the effect of mean group differences on the appearance of bias, pseudo-ethnic groups were created, that is, samples of whites were selected to simulate the average black-white difference. The validity and sensitivity of the IRT bias indices was supported by several findings. A relatively large number of items (10 of 29) on the math test were found to be consistently biased; they were replicated in parallel analyses. The bias indices were substantially smaller in white-white analyses. Furthermore, the indices (with the possible exception of χ 2 ) did not find bias in the pseudo-ethnic comparison. The pattern of between-study correlations showed high consistency for parallel ethnic analyses where bias was plausibly present. Also, the indices met the discriminant validity test—the correlations were low between conditions where bias should not be present. For the math test, where a substantial number of items appeared biased, the results were interpretable. Verbal math problems were systematically more difficult for blacks. Overall, the sums-of-squares statistics (weighted by the inverse of the variance errors) were judged to be the best indices for quantifying ICC differences between groups. Not only were these statistics the most consistent in detecting bias in the ethnic comparisons, but they also intercorrelated the least in situations of no bias.

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 129 citations exist upstream.

References

1960Modern factor analysis 2012Applications of Item Response Theory To Practical Testing Problems 1968Modern Factor Analysis 1973The American Heritage Word Frequency Book 1979Unifactor Latent Trait Models Applied to Multifactor Tests: Results and Implications 1968Modern Factor Analysis 1982Handbook of methods for detecting test bias 1976AN EVALUATION OF SOME MODELS FOR CULTURE‐FAIR SELECTION

Cited by

2011Road Maps for Learning: A Guide to the Navigation of Learning Progressions 2012The Rasch Rating Model and the Disordered Threshold Controversy 2002A Monte Carlo Comparison of Parametric and Nonparametric Polytomous DIF Detection Methods 1989Detecting Potentially Biased Test Items: Comparison of IRT Area and Mantel-Haenszel Methods 1989Two New Approaches to Assessing Differential Item Functioning: Standardization and the Mantel--Haenszel Method 1993Detection of Differential Item Functioning in the Graded Response Model

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.