Influence of prior acquaintance with the ratee on rater accuracy and halo

Abstract

Purpose The purpose of this paper is to examine the effect of acquaintance on performance rating accuracy and halo. Design/methodology/approach After expert ratings were obtained, US Air Force Officers (n=104) with an average of six years experience rated the performance of four officers who delivered 6‐7 minute briefings on their research projects; 26 raters reported being acquainted with one or more of the briefers. Raters were randomly assigned to use a rating format designed to encourage between‐ratee comparisons on each dimension or a format in which each ratee was separately rated on all dimensions. Findings Ratings made by acquainted raters were more accurate than ratings by unacquainted raters. Accuracy was positively correlated with halo for both sets of ratings. Rating format had no discernible effect on accuracy or halo. Research limitations/implications One limitation of this study is that the measure of acquaintance was not designed as a surrogate for familiarity. Development of a multi‐item, psychometrically‐valid measure of acquaintance should be the first step in pursuing this research. The use of a laboratory design where only a small percentage of the sample was acquainted with those being rated also limits the study's generalizability. Practical implications The results show that prior acquaintance with the ratee results in more accurate ratings. Ratings were also more positive when raters had prior contact with the person they rated. Originality/value The hypothesis is that the cognitive processes used to produce ratings are different for raters who have had no prior contact with a ratee and raters who are in some manner acquainted with a ratee. In the past, a positive halo effect from acquaintance between raters and ratees has been a concern. However, this limited study indicates that acquaintance may actually result in more accurate ratings.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 12 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.