Repeatability and reproducibility of comparison decisions by firearms examiners

Keith L. Monson · Erich D. Smith · Eugene M. Peters

16 citations0/0 files storedJournal of Forensic Sciences

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

Reliability and Agreement in Measurement Legal processes and jurisprudence Gun Ownership and Violence Research

Abstract

In a comprehensive study to assess various aspects of the performance of qualified forensic firearms examiners, volunteer examiners compared both bullets and cartridge cases fired from three different types of firearms. They rendered opinions on each comparison according to the Association of Firearm & Tool Mark Examiners (AFTE) Range of Conclusions, as Identification, Inconclusive (A, B, or C), Elimination, or Unsuitable. In this part of the study, comparison sets used previously to characterize the overall accuracy of examiners were blindly resubmitted to examiners to assess the repeatability (105 examiners; 5700 comparisons of bullets and cartridge cases) and reproducibility (191 examiners of bullets, 193 of cartridge cases; 5790 comparisons) of firearms examinations. Data gathered using the prevailing AFTE Range were also recategorized into two hypothetical scoring systems. Consistently positive differences between observed agreement and expected agreement indicate that the repeatability and reproducibility of examiners exceed chance agreement. When averaged over bullets and cartridge cases, the repeatability of comparison decisions (involving all five levels of the AFTE Range) was 78.3% for known matches and 64.5% for known nonmatches. Similarly averaged reproducibility was 67.3%% for known matches and 36.5% for known nonmatches. For both repeatability and reproducibility, many of the observed disagreements were between a definitive and inconclusive category. Examiner decisions are reliable and trustworthy in the sense that identifications are unlikely when examiners are comparing non-matching items, and eliminations are unlikely when they are comparing matching items.

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 16 citations exist upstream.

References

1977The Measurement of Observer Agreement for Categorical Data 1971Measuring nominal scale agreement among many raters.2012Computing Inter-Rater Reliability for Observational Data: An Overview and Tutorial 2008Inter-Coder Agreement for Computational Linguistics 1981Coefficient Kappa: Some Uses, Misuses, and Alternatives 2013A comparison of Cohen’s Kappa and Gwet’s AC1 when calculating inter-rater reliability coefficients: a study conducted with personality disorder samples 2008Reliability, repeatability and reproducibility: analysis of measurement errors in continuous variables 2012Communication Yearbook 10

Cited by

2024Methodological problems in every black-box study of forensic firearm comparisons

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.