Speech Emotion Recognition Using 3D Convolutions and Attention-Based Sliding Recurrent Networks With Auditory Front-Ends

Zhichao Peng · Xingfeng Li · Zhi Zhu · Masashi Unoki · Jianwu Dang · Masato Akagi

82 citations0/0 files storedIEEE Access

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

Emotion and Mood Recognition Speech Recognition and Synthesis Speech and Audio Processing

Abstract

Emotion information from speech can effectively help robots understand speaker's intentions in natural human-robot interaction. The human auditory system can easily track temporal dynamics of emotion by perceiving the intensity and fundamental frequency of speech, and focus on the salient emotion regions. Therefore, speech emotion recognition combined with the auditory mechanism and attention mechanism may be an effective way. Some previous studies used auditory-based static features to identify emotion while ignoring the emotion dynamics. Some other studies used attention models to capture the salient regions of emotion while ignoring cognitive continuity. To fully utilize the auditory and attention mechanism, we first investigate temporal modulation cues from auditory front-ends and then propose a joint deep learning model that combines 3D convolutions and attention-based sliding recurrent neural networks (ASRNNs) for emotion recognition. Our experiments on the IEMOCAP and MSP-IMPROV datasets indicate that the proposed method can be effectively used to recognize the emotions of speech from temporal modulation cues. The subjective evaluation shows that the attention patterns of the attention model are basically consistent with human behaviors in recognizing the emotions.

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 82 citations exist upstream.

References

2008IEMOCAP: interactive emotional dyadic motion capture database 2010Survey on speech emotion recognition: Features, classification schemes, and databases 1992Temporal information in speech: acoustic, auditory and linguistic aspects 2009The INTERSPEECH 2009 emotion challenge 2017Automatic speech emotion recognition using recurrent neural networks with local attention 2013The INTERSPEECH 2013 computational paralinguistics challenge: social signals, conflict, emotion, autism 2014Learning Salient Features for Speech Emotion Recognition Using Convolutional Neural Networks 20183-D Convolutional Recurrent Neural Networks With Attention Model for Speech Emotion Recognition

Cited by

2021Bangla Speech Emotion Recognition and Cross-Lingual Study Using Deep CNN and BLSTM Networks 2022Modulation spectral features for speech emotion recognition using deep neural networks 2023Multichannel CNN-BLSTM Architecture for Speech Emotion Recognition System by Fusion of Magnitude and Phase Spectral Features Using DCCA for Consumer Applications 2021Speech Emotion Recognition Using Clustering Based GA-Optimized Feature Set

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.