JUST BECAUSE IT'S RELIABLE DOESN'T MEAN IT'S BELIEVABLE: A Commentary on Two Articles by Birkimer and Brown

Abstract

prompted a somewhat unsettled state of affairs in the area of observer agreement in applied behavioral re- search.Although there was general consensus among these authors that the conventional percentage method of calculating "reliability" esti- mates did not adequately control for chance correspondence between observers' data, there was considerable disagreement among at least some of these authors over the most effective way to solve the problems.Birkimer and Brown (1979a, 1979b) have done an admirable job in providing alternatives to some of the previous recommendations in reporting observer agreement data.Their pro- cedures have much to recommend them and, if followed, will likely have an impact on future applied behavioral research.Despite many positive aspects of both these reports, some words of caution are necessary.In the 1979b article, the authors demonstrate that although whole percentage agreement estimates are adequate as reliability measures (with due consideration for their limitations), tables developed to determine if obtained disagreements are unlikely due to chance are more easily em- ployed.Alternatively, a rule developed in their

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 10 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.