Robustness of personal rankings: the Handelsblatt example

Abstract

Abstract In the last years,Handelsblatthas published several rankings of business economists from German, Swiss and Austrian research institutions based on their journal publication output. These rankings have a strong influence on the academic profession. We scrutinize theHandelsblattmethodology by examining the effect the rankings’ underlying algorithms and assumptions have on the scores and ranks of individual researchers. In doing so, we clarify how robust the result is with respect to these internal parameters. Since the parameters used byHandelsblattare not scientifically substantiated but defined ad hoc, this question is of great importance. For each parameter variation, we provide several robustness measures for both theHandelsblattlife’s work ranking and theHandelsblattrecent research performance ranking. E.g., if one applies a weighting scheme that lays more emphasis on first tier journal publications such that the weight of a particular category is always double of the weight of the next lower category, rank correlations based on all researchers in both personal rankings exceed 80 %. However, if one solely considers the top 25 performing researchers rank correlations fall below 50 and 20 % of researchers even drop out of this top group. Further research as well as the discussion in the academic community should clarify whether these correlations verify the robustness of the ranking or manifest the opposite.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 10 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.