The influence of equating methodology on reported trends in PISA.

Eveline Gebhardt · Raymond J. Adams

40 citations0/0 files storedPubMed

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

Psychometric Methodologies and Testing Global Educational Policies and Reforms Educational Assessment and Pedagogy

Abstract

In 2005 PISA published trend indicators that compared the results of PISA 2000 and PISA 2003. In this paper we explore the extent to which the outcomes of these trend analyses are sensitive to the choice of test equating methodologies, the choice of regression models and the choice of linking items. To establish trends PISA equated its 2000 and 2003 tests using a methodology based on Rasch Modelling that involved estimating linear transformations that mapped 2003 Rasch-scaled scores to the previously established PISA 2000 Rasch-scaled scores. In this paper we compare the outcomes of this approach with an alternative, which involves the joint Rasch scaling of the PISA 2000 and PISA 2003 data separately for each country. Note that under this approach the item parameters are estimated separately for each country, whereas the linear transformation approach used a common set of item parameter estimates for all countries. Further, as its primary trend indicators, PISA reported changes in mean scores between 2000 and 2003. These means are not adjusted for changes in the background characteristics of the PISA 2000 and PISA 2003 samples - that is, they are marginal rather than conditional means. The use of conditional rather than marginal means results in some differing conclusions regarding trends at both the country and within-country level.

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 40 citations exist upstream.

References

1961Probabilistic Models for Some Intelligence and Attainment Tests 1982A Rasch Model for Partial Credit Scoring 1992A standard international socio-economic index of occupational status 1994Many-Facet Rasch Measurement 1998A Comparison of Linking and Concurrent Calibration Under Item Response Theory 2007The computation of equating errors in international surveys in education.

Cited by

2014Performance status and change – measuring education system effectiveness with data from PISA 2000–2009 2020Reanalysis of the German PISA Data: A Comparison of Different Approaches for Trend Estimation With a Particular Emphasis on Mode Effects 2022Some thoughts on analytical choices in the scaling model for test scores in international large-scale assessment studies 2020Lp Loss Functions in Invariance Alignment and Haberman Linking with Few or Many Groups 2016A Comparison of Linking Methods for Estimating National Trends in International Comparative Large‐Scale Assessments in the Presence of Cross‐National DIF

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.