Head Fusion: Improving the Accuracy and Robustness of Speech Emotion Recognition on the IEMOCAP and RAVDESS Dataset

Mingke Xu · Fan Zhang · Wei Zhang

125 citations0/0 files storedIEEE Access

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

Emotion and Mood Recognition Speech Recognition and Synthesis Speech and Audio Processing

Abstract

Speech Emotion Recognition (SER) refers to the use of machines to recognize the emotions of a speaker from his (or her) speech. SER benefits Human-Computer Interaction(HCI). But there are still many problems in SER research, e.g., the lack of high-quality data, insufficient model accuracy, little research under noisy environments, etc. In this paper, we proposed a method called Head Fusion based on the multi-head attention mechanism to improve the accuracy of SER. We implemented an attention-based convolutional neural network(ACNN) model and conducted experiments on the Interactive Emotional Dyadic Motion Capture (IEMOCAP) data set. The accuracy is improved to 76.18% (weighted accuracy, WA) and 76.36% (unweighted accuracy, UA). To the best of our knowledge, compared with the state-of-the-art result on this dataset (76.4% of WA and 70.1% of WA), we achieved a UA improvement of about 6% absolute while achieving a similar WA. Furthermore, We conducted empirical experiments by injecting speech data with 50 types of common noises. We inject the noises by altering the noise intensity, time-shifting the noises, and mixing different noise types, to identify their varied impacts on the SER accuracy and verify the robustness of our model. This work will also help researchers and engineers properly add their training data by using speech data with the appropriate types of noises to alleviate the problem of insufficient high-quality data.

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 125 citations exist upstream.

References

2008IEMOCAP: interactive emotional dyadic motion capture database 2010Survey on speech emotion recognition: Features, classification schemes, and databases 1977Evidence for a three-factor theory of emotions 2018The Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS): A dynamic, multimodal set of facial and vocal expressions in North American English 2014Speech emotion recognition using deep neural network and extreme learning machine 1961Intonation of colloquial English 2020Speech emotion recognition with deep convolutional neural networks 2003Hidden Markov model-based speech emotion recognition

Cited by

2024Using transformers for multimodal emotion recognition: Taxonomies and state of the art review 2022Two-Way Feature Extraction for Speech Emotion Recognition Using Deep Learning 2023Speech Emotion Recognition Using Attention Model 2022Learning multi-scale features for speech emotion recognition with connection attention mechanism 2023A review on speech emotion recognition: A survey, recent advances, challenges, and the influence of noise

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.