Validating process variables of sourcing in an assessment of multiple document comprehension

Carolin Hahnel · Ulf Kroehne · Frank Goldhammer · Cornelia Schoor · Nina Mahlow · Cordula Artelt

38 citations0/0 files storedBritish Journal of Educational Psychology

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

Educational Strategies and Epistemologies Psychometric Methodologies and Testing Online Learning and Analytics

Abstract

BACKGROUND: With digital technologies, competence assessments can provide process data, such as mouse clicks with corresponding timestamps, as additional information about the skills and strategies of test takers. However, in order to use variables generated from process data sensibly for educational purposes, their interpretation needs to be validated with regard to their intended meaning. AIMS: This study seeks to demonstrate how process data from an assessment of multiple document comprehension can be used to represent sourcing, which summarizes activities for the consideration of the origin and intention of documents. The investigated process variables were created according to theoretical assumptions about sourcing, and systematically tested for differences between persons, units (i.e., documents and items), and properties of the test administration. SAMPLE: The sample included 310 German university students (79.4% female), enrolled in several bachelor's or master's programmes of the social sciences and humanities. METHODS: Regarding the hierarchical data structure, the hypotheses were analysed with generalized linear mixed models (GLMM). RESULTS: The results mostly revealed expected differences between individuals and units. However, unexpected effects of the administered order of units and documents were detected. CONCLUSIONS: The study demonstrates the theory-informed construction of process variables from log-files and an approach for empirical validation of their interpretation. The results suggest that students apply sourcing for different reasons, but also stress the need of further validation studies and refinements in the operationalization of the indicators investigated.

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 38 citations exist upstream.

References

2010Heuristic Decision Making 2017Thanks coefficient alpha, we’ll take it from here.2013Validating the Interpretations and Uses of Test Scores 1991Historical problem solving: A study of the cognitive processes used in the evaluation of documentary and pictorial evidence.1989Weighted Likelihood Estimation of Ability in Item Response Theory 2011Handbook of Test Development 2003Handbook of Discourse Processes 2002Improving Students' Ability to Identify and Use Source Information

Cited by

2020Web-based and mixed-mode cognitive large-scale assessments in higher education: An evaluation of selection bias, measurement bias, and prediction bias 2020More Than (Single) Text Comprehension? – On University Students’ Understanding of Multiple Documents

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.