X-Vectors Meet Emotions: A Study On Dependencies Between Emotion and Speaker Recognition

Raghavendra Reddy Pappagari · Tianzi Wang · Jesús Villalba · Nanxin Chen · Najim Dehak

111 citations0/0 files storedIEEE International Conference on Acoustics Speech and Signal Processing

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

Emotion and Mood Recognition Speech Recognition and Synthesis Speech and Audio Processing

Abstract

In this work, we explore the dependencies between speaker recognition and emotion recognition. We first show that knowledge learned for speaker recognition can be reused for emotion recognition through transfer learning. Then, we show the effect of emotion on speaker recognition. For emotion recognition, we show that using a simple linear model is enough to obtain good performance on the features extracted from pre-trained models such as the x-vector model. Then, we improve emotion recognition performance by finetuning for emotion classification. We evaluated our experiments on three different types of datasets: IEMOCAP, MSP-Podcast, and Crema-D. By fine-tuning, we obtained 30.40%, 7.99%, and 8.61% absolute improvement on IEMOCAP, MSP-Podcast, and Crema-D respectively over baseline model with no pre-training. Finally, we present results on the effect of emotion on speaker verification. We observed that speaker verification performance is prone to changes in test speaker emotions. We found that trials with angry utterances performed worst in all three datasets. We hope our analysis will initiate a new line of research in the speaker recognition community.

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 111 citations exist upstream.

References

2008IEMOCAP: interactive emotional dyadic motion capture database 2015The Geneva Minimalistic Acoustic Parameter Set (GeMAPS) for Voice Research and Affective Computing 2018Speech emotion recognition using deep 1D & 2D CNN LSTM networks 2016Adieu features? End-to-end speech emotion recognition using a deep convolutional recurrent network 2017End-to-End Multimodal Emotion Recognition Using Deep Neural Networks 2016Speech emotion recognition using convolutional and Recurrent Neural Networks 2014Speech Emotion Recognition Using CNN 2017Building Naturalistic Emotionally Balanced Speech Corpus by Retrieving Emotional Speech from Existing Podcast Recordings

Cited by

2021Emotion Recognition by Fusing Time Synchronous and Time Asynchronous Representations 2022Multimodal Emotion Recognition Using Transfer Learning from Speaker Recognition and BERT-Based Models 2021Improved Speech Emotion Recognition using Transfer Learning and Spectrogram Augmentation 2022Evaluating Self-Supervised Speech Representations for Speech Emotion Recognition 2021CopyPaste: An Augmentation Method for Speech Emotion Recognition

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.