Multi-Modal Bias: Introducing a Framework for Stereotypical Bias Assessment beyond Gender and Race in Vision–Language Models

Sepehr Janghorbani · Gerard de Melo

18 citations0/0 files storedConference of the European Chapter of the Association for Computational Linguistics

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

Ethics and Social Impacts of AI Multimodal Machine Learning Applications Hate Speech and Cyberbullying Detection

Abstract

Recent breakthroughs in self-supervised training have led to a new class of pretrained vision-language models.While there have been investigations of bias in multimodal models, they have mostly focused on gender and racial bias, giving much less attention to other relevant groups, such as minorities with regard to religion, nationality, sexual orientation, or disabilities.This is mainly due to lack of suitable benchmarks for such groups.We seek to address this gap by providing a visual and textual bias benchmark called MMBias, consisting of around 3,800 images and phrases covering 14 population subgroups.We utilize this dataset to assess bias in several prominent selfsupervised multimodal models, including CLIP, ALBEF, and ViLT.Our results show that these models demonstrate meaningful bias favoring certain groups.Finally, we introduce a debiasing method designed specifically for such large pretrained models that can be applied as a post-processing step to mitigate bias, while preserving the remaining accuracy of the model.

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 18 citations exist upstream.

References

1998Measuring individual differences in implicit cognition: The implicit association test.1995Implicit social cognition: Attitudes, self-esteem, and stereotypes.2003Core affect and the psychological construction of emotion.2017Semantics derived automatically from language corpora contain human-like biases 2021StereoSet: Measuring stereotypical bias in pretrained language models 2002Formal and Interpersonal Discrimination: A Field Study of Bias Toward Homosexual Applicants 2019Balanced Datasets Are Not Enough: Estimating and Mitigating Gender Bias in Deep Image Representations 2020Social Biases in NLP Models as Barriers for Persons with Disabilities

Cited by

2024SocialCounterfactuals: Probing and Mitigating Intersectional Social Biases in Vision-Language Models with Counterfactual Examples

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.