Understanding Undesirable Word Embedding Associations

Kawin Ethayarajh · David Duvenaud · Graeme Hirst

110 citations0/0 files storedAnnual Meeting of the Association for Computational Linguistics (ACL)

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

Ethics and Social Impacts of AI Topic Modeling Hate Speech and Cyberbullying Detection

Abstract

Word embeddings are often criticized for capturing undesirable word associations such as gender stereotypes.However, methods for measuring and removing such biases remain poorly understood.We show that for any embedding model that implicitly does matrix factorization, debiasing vectors post hoc using subspace projection (Bolukbasi et al., 2016) is, under certain conditions, equivalent to training on an unbiased corpus.We also prove that WEAT, the most common association test for word embeddings, systematically overestimates bias.Given that the subspace projection method is provably effective, we use it to derive a new measure of association called the relational inner product association (RIPA).Experiments with RIPA reveal that, on average, skipgram with negative sampling (SGNS) does not make most words any more gendered than they are in the training corpus.However, for gender-stereotyped words, SGNS actually amplifies the gender association in the corpus.

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 110 citations exist upstream.

References

2017Semantics derived automatically from language corpora contain human-like biases 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings 2019Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them 2019Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them

Cited by

2021RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Models 2019Examining Gender Bias in Languages with Grammatical Gender 2024What’s in a Name? Experimental Evidence of Gender Bias in Recommendation Letters Generated by ChatGPT 2021Bad Seeds: Evaluating Lexical Methods for Bias Measurement 2022Unmasking the Mask – Evaluating Social Biases in Masked Language Models 2021Towards a Comprehensive Understanding and Accurate Evaluation of Societal Biases in Pre-Trained Transformers 2020Double-Hard Debias: Tailoring Word Embeddings for Gender Bias Mitigation 2023Fairness in Language Models Beyond English: Gaps and Challenges

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.