Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems

Svetlana Kiritchenko · Saif M. Mohammad

37 citations0/0 files storedarXiv (Cornell University)

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

Ethics and Social Impacts of AI Sentiment Analysis and Opinion Mining Hate Speech and Cyberbullying Detection

Abstract

Automatic machine learning systems can inadvertently accentuate and perpetuate inappropriate human biases. Past work on examining inappropriate biases has largely focused on just individual systems. Further, there is no benchmark dataset for examining inappropriate biases in systems. Here for the first time, we present the Equity Evaluation Corpus (EEC), which consists of 8,640 English sentences carefully chosen to tease out biases towards certain races and genders. We use the dataset to examine 219 automatic sentiment analysis systems that took part in a recent shared task, SemEval-2018 Task 1 'Affect in Tweets'. We find that several of the systems show statistically significant bias; that is, they consistently provide slightly higher sentiment intensity predictions for one race or one gender. We make the EEC freely available.

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 37 citations exist upstream.

References

2018Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification 2017Semantics derived automatically from language corpora contain human-like biases 2017Fair Prediction with Disparate Impact: A Study of Bias in Recidivism Prediction Instruments 2018Mitigating Unwanted Biases with Adversarial Learning 2013Discrimination in online ad delivery 2008Discrimination-aware data mining 2015Automated Experiments on Ad Privacy Settings 2017Avoiding Discrimination through Causal Reasoning

Cited by

2022Divergent semantic integration (DSI): Extracting creativity from narratives with distributional semantic modeling 2022The importance of humanizing AI: using a behavioral lens to bridge the gaps between humans and machines 2020Towards Understanding Gender Bias in Relation Extraction 2020Artificial mental phenomena

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.