Interpreting Kappa in Observational Research: Baserate Matters

Cornelia Taylor Bruckner · Paul J. Yoder

40 citations0/0 files storedAmerican Journal on Mental Retardation

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

Reliability and Agreement in Measurement Psychometric Methodologies and Testing Behavioral and Psychological Studies

Abstract

Kappa (Cohen, 1960) is a popular agreement statistic used to estimate the accuracy of observers. The response of kappa to differing baserates was examined and methods for estimating the accuracy of observers presented. Results suggest that setting a single value of kappa as "minimally acceptable" (i.e., a criterion value) is not useful in ensuring adequate accuracy of observers. Instead, researchers should use the best estimate of the true baserate of the target behavior and the obtained kappa to estimate the accuracy of observers. Investigators can then compare the estimated accuracy of observers to a preselected criterion level. Guidelines are given for selecting a criterion accuracy level.

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 40 citations exist upstream.

References

1960A Coefficient of Agreement for Nominal Scales 2005The Use of Single-Subject Research to Identify Evidence-Based Practice in Special Education 1990High agreement but low kappa: II. Resolving the paradoxes 1981Coefficient Kappa: Some Uses, Misuses, and Alternatives 2002Kappa coefficients in medical research 1985A Proposed Solution to the Base Rate Problem in the Kappa Statistic 1981Reliability Studies of Psychiatric Diagnosis 2000Play diagnosis and assessment

Cited by

2010Language, gesture, action! A test of the Gesture as Simulated Action framework 2014Preschool children's joint block building during a guided play activity 2017Parenting Assessed by Observation versus Parent-report: Moderation by Parent Distress and Family Socioeconomic Status 2017Can credibility criteria be assessed reliably? A meta-analysis of criteria-based content analysis.2012Influence of a Playful, Child-Directed Context on Preschool Children’s Peer Cooperation

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.