A randomized controlled study of reviewer bias against an unconventional therapy

K I Resch · E Ernst · John S Garrow

92 citations0/0 files storedJournal of the Royal Society of Medicine

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

Academic Publishing and Open Access Publishing and Scholarly Communication Meta-analysis and systematic reviews

Abstract

A study was designed to test the hypothesis that experts who review papers for publication are prejudiced against an unconventional form of therapy. Two versions were produced (A and B) of a 'short report' that related to treatments of obesity, identical except for the nature of the intervention. Version A related to an orthodox treatment, version B to an unconventional treatment. 398 reviewers were randomized to receive one or the other version for peer review. The primary outcomes were the reviewers' rating of 'importance' on a scale of 1-5 and their verdict regarding rejection or acceptance of the paper. Reviewers were unaware that they were taking part in a study. The overall response rate was 41.7%, and 141 assessment forms were suitable for statistical evaluation. After dichotomization of the rating scale, a significant difference in favour of the orthodox version with an odds ratio of 3.01 (95% confidence interval, 1.03 to 8.25), was found. This observation mirrored that of the visual analogue scale for which the respective medians and interquartile ranges were 67% (51% to 78.5%) for version A and 57% (29.7% to 72.6%) for version B. Reviewers showed a wide range of responses to both versions of the paper, with a significant bias in favour of the orthodox version. Authors of technically good unconventional papers may therefore be at a disadvantage in the peer review process. Yet the effect is probably too small to preclude publication of their work in peer-reviewed orthodox journals.

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 92 citations exist upstream.

References

1996Improving the Quality of Reporting of Randomized Controlled Trials 1997Language bias in randomised controlled trials published in English and German 1998Effect on the Quality of Peer Review of Blinding Reviewers and Asking Them to Sign Their Reports 1998US and Non-US Submissions 1998Peer review of grant applications: what do we know?1999Evidence on peer review—scientific quality control or smokescreen?

Cited by

2006Editorial peer review for improving the quality of reports of biomedical studies 2008Perfect Study, Poor Evidence: Interpretation of Biases Preceding Study Design 2006Peering at peer review revealed high degree of chance associated with funding of grant applications 2011Journal peer review in context: A qualitative study of the social and subjective dimensions of manuscript review in biomedical publishing 2015Catastrophism, Uniformitarianism, and a Scientific Realism Debate That Makes a Difference 2016Peer Review and Scholarly Originality 2017Peer review in forensic science

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.