Replications in Comparative Cognition: What Should We Expect and How Can We Improve?

Benjamin George Farrar · Markus Boeckle · Nicola Susan Clayton

89 citations0/0 files storedAnimal Behavior and Cognition

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

Meta-analysis and systematic reviews Animal testing and alternatives Psychology Research and Bibliometrics

Abstract

Direct replication studies follow an original experiment's methods as closely as possible.They provide information about the reliability and validity of an original study's findings.The present paper asks what comparative cognition should expect if its studies were directly replicated, and how researchers can use this information to improve the reliability of future research.Because published effect sizes are likely overestimated, comparative cognition researchers should not expect findings with p-values just below the significance level to replicate consistently.Nevertheless, there are several statistical and design features that can help researchers identify reliable research.However, researchers should not simply aim for maximum replicability when planning studies; comparative cognition faces strong replicability-validity and replicability-resource trade-offs.Next, the paper argues that it may not even be possible to perform truly direct replication studies in comparative cognition because of: 1) a lack of access to the species of interest; 2) real differences in animal behavior across sites; and 3) sample size constraints producing very uncertain statistical estimates, meaning that it will often not be possible to detect statistical differences between original and replication studies.These three reasons suggest that many claims in the comparative cognition literature are practically unfalsifiable, and this presents a challenge for cumulative science in comparative cognition.To address this challenge, comparative cognition can begin to formally assess the replicability of its findings, improve its statistical thinking and explore new infrastructures that can help to form a field that can create and combine the data necessary to understand how cognition evolves.

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 89 citations exist upstream.

References

2015Estimating the reproducibility of psychological science 2011False-Positive Psychology 20161,500 scientists lift the lid on reproducibility 2017Redefine statistical significance 2012Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling 2018The preregistration revolution 2018Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015 2017The reliability paradox: Why robust cognitive tasks do not produce reliable individual differences

Cited by

2023The replication crisis has led to positive structural, procedural, and community changes 2022Bridging Neurodiversity and Open Scholarship: How Shared Values Can Guide Best Practices for Research Integrity, Social Justice, and Principled Education 2020Validity of Cognitive Tests for Non-human Animals: Pitfalls and Prospects 2023Teaching open and reproducible scholarship: a critical review of the evidence base for current pedagogical methods and their outcomes 2021Replications in Comparative Psychology

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.