Avoiding Methodological Biases in Meta-Analysis

Abstract

Abstract. Individual participant data (IPD) meta-analysis is the gold standard of meta-analyses. This paper points out several advantages of IPD meta-analysis over classical meta-analysis, such as avoiding aggregation bias (e.g., ecological fallacy or Simpson’s paradox) and shows how its two main disadvantages (time and cost) can be overcome through Internet-based research. Ideally, we recommend carrying out IPD meta-analyses that consider online versus offline data gathering processes and examine data quality. Through a comprehensive literature search, we investigated whether IPD meta-analyses published in the field of educational psychology already follow these recommendations; this was not the case. For this reason, the paper demonstrates characteristics of ideal meta-analysis on teachers’ judgment accuracy and links it to recent meta-analyses on that topic. The recommendations are important for meta-analysis researchers and for readers and reviewers of meta-analyses. Our paper is also relevant to current discussions within the psychological community on study replication.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 31 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.