Towards Parameter-free Blocking for Scalable Record Linkage

Abstract

Linking or matching databases is becoming increasingly important in\nmany data mining projects, as linked data can contain information that\nis not available otherwise, or that would be too expensive to collect.\nA main challenge when linking large databases is the complexity of the\nlinkage process: potentially each record in one database has to be\ncompared with all records in the other database. Various techniques,\ncollectively know as `blocking', have been developed to deal with this\nquadratic complexity. Most of these techniques require several\nparameters to be set by the user in order to achieve good results. In\nthis paper we evaluate six blocking techniques within a common\nframework with regard to the number and quality of the candidate\nrecord pairs generated. We propose a modification to two existing\ntechniques that reduces the variance in the quality of the blocking\nresults over a range of parameter values, enabling more robust,\npractical record linkage without the need of time consuming manual\nparameter tuning.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 34 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.