No open-access location is known for this record yet.
No open-access location is known for this record yet.
Abstract The National Institutes of Health refused to switch to disaggregated ratings as a method for evaluating proposals, because no contest between disaggregated and holistic ratings had ever used scientific materials as the to‐be‐rated stimuli. We designed two studies to fill this research void. Participants rated scientific convention presentations either using a holistic procedure in which one overall rating was given or a disaggregated procedure in which one rating was given to each of five criteria. In four of the six convention sessions the disaggregated ratings led to higher inter‐rater reliability than did the holistic ratings; three of these differences were statistically significant. The inter‐rater reliabilities between the two types of ratings collapsed across all sessions differed significantly. In a second experiment, participants rated posters using either disaggregated or holistic ratings. The disaggregated ratings again led to higher inter‐rater reliability, but not significantly so. In 35 of the 43 sessions in which disaggregated and holistic ratings were compared, the variance of the disaggregated ratings was smaller. Copyright © 2006 John Wiley & Sons, Ltd.
Loading…
Stored only when both papers are in the corpus — 22 citations exist upstream.
Papers that cite the same references — the relational fallback while embeddings are still backfilling.
Imported from OpenAlex (CC0) — nothing here is generated.