Properties of statistical tests appropriate for the analysis of data in disparate impact cases

Abstract

The article reviews the use of statistical tests to establish a prima facie case that an exam has a disparate impact on minorities. Two common scenarios are discussed. The first is that promotions are made in accordance with the ‘rank-order’ of the exam scores or a composite of the exam scores and some other factors. Courts have used several statistical tests in this situation, which may lead to conflicting conclusions from similar data. It will be shown that when the spreads of the exam scores in both groups are close to each other, the modified Wilcoxon test has desirable statistical properties. When the spreads of the exam scores of the two groups are noticeably different, a two-test procedure is proposed and shown to have higher power, especially when the spread of the minority scores is less than that of the majority. The second situation occurs when once an applicant passes the exam, s/he is eligible for further consideration and the actual exam score no longer matters. Courts may need to consider both the practical and statistical significance of the difference in pass rates. Small, unimportant differences may reach statistically significance when the numbers of applicants are large. In contrast, large differences in pass rates may not be detected as statistically significant in small samples. Two tables are provided to assist courts in reaching more consistent decisions when statistical and practical significance may not agree.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 9 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.