Population Invariance and the Equatability of Tests: Basic Theory and The Linear Case

Neil J. Dorans · Paul W. Holland

240 citations0/0 files storedJournal of Educational Measurement

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

Psychometric Methodologies and Testing Educational Assessment and Pedagogy Educational Technology and Assessment

Abstract

How does the fact that two tests should not be equated manifest itself? This paper addresses this question through the study of the degree to which equating functions fail to exhibit population invariance across subpopulations. Equating fimctions are supposed to be population invariant by definition. But, when two tests are not equatable, it is possible that the linking functions, used to connect the scores of one to the scores of the other, are not invariant across different populations of examinees. While no acceptable equating function is ever completely population invariant, in the situations where equating is usually performed we believe that the dependence of the equating function on the population used to compute it is usually small enough to be ignored. We introduce two root‐mean‐square difference measures of the degree to which the functions used to link two tests computed on different subpopulations differ from the linking function computed for the whole population. We also introduce the system of “parallel‐linear” linking functions for multiple subpopulations and show that, for this system, our measure of population invariance can be computed easily from the standardized mean differences between the scores of the subpopulations on the two tests. For the parallel‐linear case, we develop a correlation‐based upper bound on our measure that holds for all systems of subpopulations. We illustrate these ideas using data from the SAT I and from a concordance study of several combinations of ACT and SAT I scores, In the appendices, we give some theoretical results bearing on the other equating “requirements” of “same construct,”“same reliability” and one aspect of Lord's concept of equity.

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 240 citations exist upstream.

References

2012Applications of Item Response Theory To Practical Testing Problems 2015Educational measurement.2012Test Equating: Methods and Practices 1995Test Equating

Cited by

2003A Discussion of Modern Versus Traditional Psychometrics As Applied to Personality Assessment Scales 2007Linking scores from multiple health outcome instruments 2004Equating, Concordance, and Expectation 2004Using Subpopulation Invariance to Assess Test Score Equity 2004The Chain and Post‐Stratification Methods for Observed‐Score Equating: Their Relationship to Population Invariance 2015Establishing a common metric for self-reported pain: linking BPI Pain Interference and SF-36 Bodily Pain Subscale scores to the PROMIS Pain Interference metric 2003Classical Test Theory as a First-Order Item Response Theory: Application to True-Score Prediction from a Possibly Nonparallel Test 2021Linking Scores with Patient-Reported Health Outcome Instruments: A Validation Study and Comparison of Three Linking Methods

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.