Are disruption index indicators convergently valid? The comparison of several indicator variants with assessments by peers

Lutz Bornmann · Alexander Tekles · George W. Chacko

99 citations0/0 files storedQuantitative Science Studies

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

scientometrics and bibliometrics research Psychology Research and Bibliometrics Web visibility and informetrics

Abstract

Recently, Wu, Wang, and Evans (2019) proposed a new family of indicators, which measure whether a scientific publication is disruptive to a field or tradition of research. Such disruptive influences are characterized by citations to a focal paper, but not its cited references. In this study, we are interested in the question of convergent validity. We used external criteria of newness to examine convergent validity: In the postpublication peer review system of F1000Prime, experts assess papers whether the reported research fulfills these criteria (e.g., reports new findings). This study is based on 120,179 papers from F1000Prime published between 2000 and 2016. In the first part of the study we discuss the indicators. Based on the insights from the discussion, we propose alternate variants of disruption indicators. In the second part, we investigate the convergent validity of the indicators and the (possibly) improved variants. Although the results of a factor analysis show that the different variants measure similar dimensions, the results of regression analyses reveal that one variant ( DI 5 ) performs slightly better than the others.

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 99 citations exist upstream.

References

2015Bibliometrics: The Leiden Manifesto for research metrics 2008What do citation counts measure? A review of studies on citing behavior 2013Atypical Combinations and Scientific Impact 2019Large teams develop and small teams disrupt science and technology 2013On exploratory factor analysis: A review of recent evidence, an assessment of current practice, and recommendations for future use 2015Tradition and Innovation in Scientists’ Research Strategies 2011Scientific peer review 2017Bias against novelty in science: A cautionary tale for users of bibliometric indicators

Cited by

2023What do we know about the disruption index in scientometrics? An overview of the literature 2021Rethinking the disruption index as a measure of scientific and technological advances 2021A multidimensional framework for characterizing the citation impact of scientific publications 2022Quantifying scientific breakthroughs by a novel disruption indicator based on knowledge entities 2020Disruptive papers published in Scientometrics: meaningful results by using an improved variant of the disruption index originally proposed by Wu, Wang, and Evans (2019)

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.