Humans-in-the-loop: Gamifying activity label repair in process event logs

Abstract

A key challenge in data mining, machine learning and artificial intelligence concerns data quality. Process mining is not an exception. A range of data quality problems exists in process data, some of which caused by activity labels. While automated techniques can be used for remedying data quality problems, they are effective up to a point and the aid of domain experts is required (cf. the “human-in-the-loop” paradigm in artificial intelligence). They are eminently suited to fix incorrect labels, but hard to engage as the repair task can be time-consuming and tedious. Gamification may offer a promising solution to this challenge. In this paper, we examine what motivational drives can be exploited to gamify activity-label repair, specifically those with identical semantics but different syntax. We conducted two experiments. First we recruited experts from the insurance domain to repair labels using our gamified system. Results show that log quality was improved, and participants had a positive experience interacting with the system. To validate the generalisability of the approach, our second experiment involved 30 participants who repaired a real-life medical log. We found promising results on quality improvement of the log and comparison with automatic approaches. This work contributes to process mining by improving activity-label quality in event logs and by turning data cleaning from the least enjoyable task to a fun experience for users. By bringing together elements from gamification, crowdsourcing (“humans-in-the-loop”), and process mining, this study contributes to improving the reliability of data-driven decision making in machine learning and artificial intelligence.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 8 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.