Empirically Based Criteria for Determining Meaningful Effect Size.

Abstract

The purpose of this study was to determine: (1) the extent to which effect sizes vary by chance; (2) the proportion of standardized effect sizes that achieve or exceed commonly used criteria for small, medium, and large effect sizes; (3) whether standardized effect sizes are random or systematic across numbers of groups and sample z:izes; and (4) whether it is possible to predict standardized effect sizes using degrees of freedom, number of groups, and sample sizes. Monte Carlo procedures were used to generate standardized effect sizes in a one-way analysis of variance situation with 2 through 10 groups with samples sizes from 5 to 100 in steps of 5. Within each of the 180 configurations, 5,000 replications were done. It was found that standardized effect size variation was systematic rather than random. Numbers of groups and sample sizes were highly predictive of standardized effect size, but error degrees of freedom was not predictive. Equations were developed that could be used to predict standardized effect sizes that could be expected by chance, using number of groups and sample size as the predictor variables. The prediction equations were extremely accurate. This research provides a better alternative for the evaluation of empirical standardized effect sizes than the somewhat arbitrary and fixed criteria often used to classify standardized effect sizes as small, medium, or large. (Contains 3 tables, 10 figures, and 34 references.) (SLD) Reproductions supplied by EDRS are the best that can be made from the original document. (1). O ce) O 00 e Points of view or opinions stated in this U.S. DEPARTMENT OF EDUCATION Office of Educational Research and Improvement EDUCATIONAL RESOURCES INFORMATION CENTER (ERIC) 1:3/1his document has been reproduced as received from the person or organization originating it. Minor changes have been made to improve reproduction quality. document do not necessarily represent Ctofficial OERI position or policy. 1 PERMISSION TO REPRODUCE AND DISSEMINATE THIS MATERIAL HAS BEEN GRANTED BY TO THE EDUCATIONAL RESOURCES INFORMATION CENTER (ERIC) Empirically Based Criteria for Determining Meaningful Effect Size J. Jackson Barnette Univ,ersity of Iowa and James E. McLean University of Alabama at Birmingham A Paper Presented at the 1999 Annual Meeting of the Mid-South Educational Research Association Point Clear, Alabama November 19, 1999 For further information, contact: Dr. Jack Barnette College of Public Health 2811 Steindler Bldg. University of Iowa Iowa City, IA 52242 (319) 335 8905 jack-bantette@uiowa.edu BESTCOPYAVAILABLE

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 7 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.