Effect Sizes in Cross-Cultural Research

Abstract

Effect sizes in cross-cultural research Comparison* is a cornerstone of cross-cultural research. A standard methodology in conducting these comparisons begins with the selection of measures of psychological constructs that produce quantitative data from two or more cultures or countries. The usual statistical analysis of data from two cultures involves testing a null hypothesis (H 0 ) that conflicts with the research hypothesis either by positing that a correlation between two variables is zero in the population or that there is no difference between the means of two populations. Differences are tested by comparing variance among the culture means relative to the variance within the cultures, typically using t or F tests. When the chance probability of obtaining t or F values is sufficiently low (≤5%), the result is considered statistically significant. The p level represents the probability that a result at least as extreme as the obtained result would occur if the H 0 were true. This attained p value primarily indicates the strength of the evidence that the H 0 is wrong (but the p value does not by itself indicate sufficiently how wrong H 0 is). Statistical significance does not necessarily reflect differences among people of the different cultures, however. The sole computation of t s or F s precludes our ability to interpret meaningful differences among people, because p values merely indicate the strength of the evidence against the null hypothesis of no difference between population means. Statistical significance, assuming no Type I error, only reflects some unknown, nonzero difference between the population means. Furthermore, the larger the sample sizes, the easier it is for smaller differences to become statistically significant. Therefore, a statistically significant difference may actually reflect a trivially small difference between population means. Interpretations of cultural differences among people based on “statistically significant” findings may be based on “practically insignificant” differences between means. “Practically insignificant” means that the nonzero difference between culture means is so small that it is of little or no practical significance. A synonymous phrase would be “substantively insignificant.”

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 13 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.