A High-Quality Gold Standard for Citation-based Tasks

Abstract

Analyzing and recommending citations within their specific citation contexts has recently received much attention due to the growing number of available publications.Although data sets such as CiteSeerX have been created for evaluating approaches for such tasks, those data sets exhibit striking defects.This is understandable when one considers that both information extraction and entity linking, as well as entity resolution, need to be performed.In this paper, we propose a new evaluation data set for citation-dependent tasks based on arXiv.orgpublications.Our data set is characterized by the fact that it exhibits almost zero noise in its extracted content and that all citations are linked to their correct publications.Besides the pure content, available on a sentence-by-sentence basis, cited publications are annotated directly in the text via global identifiers.As far as possible, referenced publications are further linked to the DBLP Computer Science Bibliography.Our data set consists of over 15 million sentences and is freely available for research purposes.It can be used for training and testing citation-based tasks, such as recommending citations, determining the functions or importance of citations, and summarizing documents based on their citations.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 14 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.