How Biased Are Culture-Loaded Tests?.

Abstract

The culture-loaded Peabody Picture Vocabulary Test (PPVT) and the culture-reduced Raven's Progressive Matrices(Colored and Standard forms) were examined and compared in terms of various internal criteria of culture bias in large representative samples of white, Negro, and Mexican-American school children, from kindergarten through 8th grade, in three California school districts. On both the PPVT and the Raven the three ethnic groups, which show large mean differences, show very littld difference in the rank order of item difficulties, the relative difficulty of adjacent items, the loadings of items on the first principal component, and the clAce of distractors for incorrect responses. Analysis of variance revealed very small Ethnic Group X Items interaction, but a sensitive index of item bias derived from ANOVA indicates that the Raven is considerably less biased than the PPVT, especially in the Mexican group. The Groups X Items interaction was shown to be attributable largely to differences in mental mathrity. On both tests groups of culturally homogeneous younger and old r white children (separated by 2 years) perfectly simulated the White/Negro differences in Group X Item interactions and choice of error distractors in the Raven. Certain expectations from a culture bias hypothesis were borne out only for the PPVT in the Mexican group. Unless the unlikely and empirically unsubstantiated assumption is made that culture bias affects all kinds of test items about equally, the various item analyses of the present studies lend no support to the proposition that either the PPVT or the Raven is a culturally biased test for Negroes. How Biased Are Culture-Loaded Tests? Arthur R. Jensen University of California, Berkeley Standard tests of intelligence and scholastic aptitude, it is often claimed, are culturally biased so as to favor white subjects of middle and upper-middle-class backgrounds and to disfavor subjects of lower socioeconomic status, especially certain ethnic and racial minorities. Such culture bias is often regarded as the main explanation for mean test score differences between particular subpopulations within the United States. In researching the validity of these claims, investigators have had to establish various objective criteria of culture bias in tests, so that its existence and magnitude might be assessed. It seems to be agreed upon by nearly all psychometric researchers that the presence of population differences in the distribution of test scores is by itself not a proper criterion for judging test bias. The argument that any test which shows group mean differences is therefore biased obviously begs the question. The psychometrically defensible criteria of test bias that have been proposed in the literature fall into two classes: external and internal. The first is certainly the more important from the standpoint of practical prediction. The second, however, may be even more directly relevant to many current popular criticisms of mental tests on the grounds that they are culturally loaded, therefore culturally biased. The fact that they may meet certain criteria of external validity may be attributed to culture 2 bias in the criterion. Whether such bias is fair or unfair to the members of one or another group is another matter which must be argued on still other grounds, usually involving matters of social policy rather than psychometrics. The external criteria of test bias have been the most thoroughly discussed and studied (e.g., Cleary, 1968, Darlington, 1971; Humphreys, 1973; Jensen, 1968, Linn, 1973; Thorndike, 1971). External evidence for bias is based essentially on the regression of a criterion measure on test scores in the two (or more) groups under consideration. If the intercepts and slopes of the regressions in the two goups do not differ significantly (or by more than some predetermined magnitude), the test is regarded as fair or unbiased with respect to its predictive validity for the criterion in question. The above cited references all explicate this approach and its variations and interpretations. The bulk of related empirical findings involve comparisons of white and Negro samples. Concerning these studies, Humphreys (1973, p. 59) stated: When the literature reporting regression comparisons is summarized, the following conclusion seems warranted: there is relatively little difference in the slopes or intercepts of regression lines as a function of the demographic groups that have been studied. Use of a single regression equation for these groups leads to no substantial degree of unfairness in drawing inferences concerning the criteria measured. The criteria have generally been scholastic and job performance. Internal criteria of test bias involve item analyses and particularly evidence of Groups x Items interaction. One kind of evidence of such interaction is seen when the rank order of difficulty of items (as indicated by the percent passing each item) is significantly different in two populations. Another evidence of interactions is seen even when the rank order

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 144 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.