Replication of Quantitative Work in Development Studies: Experiences and Suggestions

Abstract

Replication 1 is seen as a key characteristic of natural science (Collins, 1985; Jasny et al., 2011); observations, especially those employing complex instruments, and experiments need to be repeated, and statistical analyses be scrutinized, before results gain credibility. This is not the case in social sciences; social science data are seldom re-produced, 2 or re-analysed to check the original calculations, or analysed using alternative perspectives or frameworks. 3 Hence, it is not clear that quantitative social science can advance in the same way as natural science. Nevertheless, there have often been calls among quantitative social scientists for replication (Frisch, 1933; Dewald et al., 1986; King, 1995; Gleditsch and Metelits, 2003; McCullough and Vinod, 2003; Pesaran, 2003; Bernanke, 2004; Freeze, 2007; McCullough and McKitrick, 2009; Burman et al., 2010), especially of computational 4 studies (Peng, 2011). The proposed benefits include: full understanding of the computations 5 and estimations, including for pedagogy; credibility; a basis for further work either assessing the robustness of the study to alternative variable constructions, model specifications, estimation methods or software, or extending or building on it; and an audit function — to identify and deter fraud and/or over-interpretation of the data (McCullough et al., 2006). These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 13 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.