How Platform-User Power Relations Shape Algorithmic Accountability

Divya Ramesh · Vaishnav Kameswaran · Ding Wang · Nithya Sambasivan

47 citations0/0 files storedACM Conference on Fairness, Accountability, and Transparency (FAccT)

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

Ethics and Social Impacts of AI Digitalization, Law, and Regulation Explainable Artificial Intelligence (XAI)

Abstract

Accountability, a requisite for responsible AI, can be facilitated through transparency mechanisms such as audits and explainability. However, prior work suggests that the success of these mechanisms may be limited to Global North contexts; understanding the limitations of current interventions in varied socio-political conditions is crucial to help policymakers facilitate wider accountability. To do so, we examined the mediation of accountability in the existing interactions between vulnerable users and a ‘high-risk’ AI system in a Global South setting. We report on a qualitative study with 29 financially-stressed users of instant loan platforms in India. We found that users experienced intense feelings of indebtedness for the ‘boon’ of instant loans, and perceived huge obligations towards loan platforms. Users fulfilled obligations by accepting harsh terms and conditions, over-sharing sensitive data, and paying high fees to unknown and unverified lenders. Users demonstrated a dependence on loan platforms by persisting with such behaviors despite risks of harms such as abuse, recurring debts, discrimination, privacy harms, and self-harm to them. Instead of being enraged with loan platforms, users assumed responsibility for their negative experiences, thus releasing the high-powered loan platforms from accountability obligations. We argue that accountability is shaped by platform-user power relations, and urge caution to policymakers in adopting a purely technical approach to fostering algorithmic accountability. Instead, we call for situated interventions that enhance agency of users, enable meaningful transparency, reconfigure designer-user relations, and prompt a critical reflection in practitioners towards wider accountability. We conclude with implications for responsibly deploying AI in FinTech applications in India and beyond.

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 47 citations exist upstream.

References

2022The Coding Manual for Qualitative Researchers 2012Thematic analysis.2021A Survey on Bias and Fairness in Machine Learning 2012CRITICAL QUESTIONS FOR BIG DATA 2018Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor 2016How the machine ‘thinks’: Understanding opacity in machine learning algorithms 2018Algorithms of Oppression 2019Model Cards for Model Reporting

Cited by

2024Attitudes toward artificial intelligence: combining three theoretical perspectives on technology acceptance 2022The methodology of studying fairness perceptions in Artificial Intelligence: Contrasting CHI and FAccT 2022Effects of algorithmic control on power asymmetry and inequality within organizations 2024Understanding Contestability on the Margins: Implications for the Design of Algorithmic Decision-making in Public Services 2023A Sociotechnical Audit: Assessing Police Use of Facial Recognition 2023

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.