The Reliability of Peer Reviews of Papers on Information Systems

Michael C. Wood · Martyn Roberts · Barbara Howell

31 citations0/0 files storedJournal of Information Science

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

Publishing and Scholarly Communication scientometrics and bibliometrics research Meta-analysis and systematic reviews

Abstract

This paper analyses the reliability of the double-blind peer review systems used for submissions to the 2001 and 2002 UK Academy for Information Systems (UKAIS) conferences. The level of reliability found in the first conference was marginally lower than would be expected from a model based on chance. In the second conference the reliability level was significantly better, but still low. The paper explores some of the implications of this for the reviewing system, and suggests a model for assessing the impact of low levels of reliability.

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 31 citations exist upstream.

References

1960A Coefficient of Agreement for Nominal Scales 1982Peer-review practices of psychological journals: The fate of published articles, submitted again 2016The Effects of Double-Blind versus Single-Blind Reviewing: Experimental Evidence from The American Economic Review 2000Reproducibility of peer review in clinical neuroscience: Is agreement between reviewers any greater than would be expected by chance alone?2001Editorial peer review : its strengths and weaknesses 1999Development of the Review Quality Instrument (RQI) for Assessing Peer Reviews of Manuscripts 1990But the reviewers are making different criticisms of my paper! Diversity and uniqueness in reviewer comments.1993Guardians of Science

Cited by

2014How good is a model based on bibliometric indicators in predicting the final decisions made by peers?2014Definition of a model based on bibliometric indicators for assessing applicants to academic positions 2015Context-Aware Reviewer Assignment for Trust Enhanced Peer Review 2006The basis for evidence‐based practice: evaluating the research evidence 2011

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.