Anticipating Safety Issues in E2E Conversational AI: Framework and Tooling

Emily Dinan · Gavin Abercrombie · A. Stevie Bergman · Shannon Lydia Spruit · Dirk Hovy · Y-Lan Boureau +1 more

44 citations0/0 files storedarXiv (Cornell University)

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

Ethics and Social Impacts of AI Adversarial Robustness in Machine Learning Hate Speech and Cyberbullying Detection

Abstract

Over the last several years, end-to-end neural conversational agents have vastly improved in their ability to carry a chit-chat conversation with humans. However, these models are often trained on large datasets from the internet, and as a result, may learn undesirable behaviors from this data, such as toxic or otherwise harmful language. Researchers must thus wrestle with the issue of how and when to release these models. In this paper, we survey the problem landscape for safety for end-to-end conversational AI and discuss recent and related work. We highlight tensions between values, potential positive impact and potential harms, and provide a framework for making decisions about whether and how to release these models, following the tenets of value-sensitive design. We additionally provide a suite of tools to enable researchers to make better-informed decisions about training and releasing end-to-end conversational AI models.

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 44 citations exist upstream.

References

1988Prospect theory: An analysis of decision under risk 2011Thinking, Fast and Slow 2008Nudge: improving decisions about health, wealth, and happiness 1987Perception of Risk 2001Bad is Stronger than Good 1991Loss Aversion in Riskless Choice: A Reference-Dependent Model 1991Anomalies: The Endowment Effect, Loss Aversion, and Status Quo Bias 2004Risk as Analysis and Risk as Feelings: Some Thoughts about Affect, Reason, Risk, and Rationality

Cited by

2024Future Shock: Generative AI and the International AI Policy and Governance Crisis 2022Towards Identifying Social Bias in Dialog Systems: Framework, Dataset, and Benchmark 2021A Word on Machine Ethics: A Response to Jiang et al. (2021)2025Cheap Learning: Maximizing Performance of Language Models for Social Data Science Using Minimal Data

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.