Statistical properties of the rWG(J) index of agreement.

Ayala Cohen · Etti Doveh · Uri Eick

167 citations0/0 files storedPsychological Methods

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

Reliability and Agreement in Measurement Psychometric Methodologies and Testing Delphi Technique in Research

Abstract

L. R. James, R. G. Demaree, and G. Wolf (1984) introduced rWG(J) to estimate interrater agreement for a group. This index is calculated by comparing an observed group variance with an expected random variance. As researchers have gained experience using this index, several questions have arisen. What are the consequences of replacing values beyond the unit interval by 0? What is the dependence of rWG(J) on the group size? The authors' simulations show that a positive bias is caused by the truncation, but for large population values of rWG(J) it is negligible. Also, in this case, the group size has no effect on the expected value of rWG(J). For inference on rWG(J), researchers can exploit the availability of computers to simulate data from the hypothesized distribution and then compare the simulation results for rWG(J) with the actual values. In addition, it is shown how the bootstrap method can be used for comparing the indices of 2 groups.

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 167 citations exist upstream.

References

1979Intraclass correlations: Uses in assessing rater reliability.1996Forming inferences about some intraclass correlation coefficients.2000Multilevel Theory, Research, and Methods in Organizations: Foundations, Extensions, and New Directions 1984Estimating within-group interrater reliability with and without response bias.1993rwg: An assessment of within-group interrater agreement.1982Aggregation bias in estimates of perceptual agreement.1990Personality, affect, and behavior in groups.1975Interrater reliability and agreement of subjective judgments.

Cited by

2007Team Cognition: Development and Evolution in Software Project Teams 2008Composing Group-Level Constructs From Individual-Level Survey Data 2008Research Note —A Model of Conflict, Leadership, and Performance in Virtual Teams 2007Reliability and agreement of student ratings of the classroom environment: A reanalysis of TIMSS data 2010Diversity and organizational innovation: The role of employee involvement

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.