Algorithmic decision-making and system destructiveness: A case of automatic debt recovery

Tapani Rinta-Kahila · Ida Someh · Nicole A. Gillespie · Marta Indulska · Shirley Diane Gregor

110 citations0/0 files storedEuropean Journal of Information Systems

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

Ethics and Social Impacts of AI E-Government and Public Services Information Systems Theories and Implementation

Abstract

Governments are increasingly relying on algorithmic decision-making (ADM) to deliver public services. Recent information systems literature has raised concerns regarding ADM’s negative unintended consequences, such as widespread discrimination, which in extreme cases can be destructive to society. The extant empirical literature, however, has not sufficiently examined the destructive effects of governmental ADM. In this paper, we report on a case study of the Australian government’s “Robodebt” programme that was designed to automatically calculate and collect welfare overpayment debts from citizens but ended up causing severe distress to citizens and welfare agency staff. Employing perspectives from systems thinking and organisational limits, we develop a research model that explains how a socially destructive government ADM programme was initiated, sustained, and delegitimized. The model offers a set of generalisable mechanisms that can benefit investigations of ADM’s consequences. Our findings contribute to the literature of unintended consequences of ADM and demonstrate to practitioners the importance of setting up robust governance infrastructures for ADM programmes.

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 110 citations exist upstream.

References

1979Prospect Theory: An Analysis of Decision under Risk 2012Seeking Qualitative Rigor in Inductive Research 1999Strategies for Theorizing from Process Data 2012CRITICAL QUESTIONS FOR BIG DATA 2015Big other: Surveillance Capitalism and the Prospects of an Information Civilization 1956General Systems Theory—The Skeleton of Science 2017Fair Prediction with Disparate Impact: A Study of Bias in Recidivism Prediction Instruments 2017European Union Regulations on Algorithmic Decision Making and a “Right to Explanation”

Cited by

2025Complementarity in human-AI collaboration: concept, sources, and evidence 2023The dynamics of AI capability and its influence on public value creation of AI within public administration 2024Design principles for artificial intelligence-augmented decision making: An action design research study 2022AI-deploying organizations are key to addressing ‘perfect storm’ of AI risks 2023Managing unintended consequences of algorithmic decision-making: The case of Robodebt

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.