Recognizing Human Emotional State From Audiovisual Signals*

Yongjin Wang · Ling Guan

202 citations0/0 files storedIEEE Transactions on Multimedia

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

Emotion and Mood Recognition Speech and Audio Processing Face and Expression Recognition

Abstract

Machine recognition of human emotional state is an important component for efficient human-computer interaction. The majority of existing works address this problem by utilizing audio signals alone, or visual information only. In this paper, we explore a systematic approach for recognition of human emotional state from audiovisual signals. The audio characteristics of emotional speech are represented by the extracted prosodic, Mel-frequency Cepstral Coefficient (MFCC), and formant frequency features. A face detection scheme based on HSV color model is used to detect the face from the background. The visual information is represented by Gabor wavelet features. We perform feature selection by using a stepwise method based on Mahalanobis distance. The selected audiovisual features are used to classify the data into their corresponding emotions. Based on a comparative study of different classification algorithms and specific characteristics of individual emotion, a novel multiclassifier scheme is proposed to boost the recognition performance. The feasibility of the proposed system is tested over a database that incorporates human subjects from different languages and cultural backgrounds. Experimental results demonstrate the effectiveness of the proposed system. The multiclassifier scheme achieves the best overall recognition rate of 82.14%.

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 202 citations exist upstream.

References

1997Affective Computing 2002Recognizing emotion in speech 2003Emotion recognition by speech signals 2002Facial emotion recognition using multi-modal information 2004Automatic emotional speech classification

Cited by

2014Survey on audiovisual emotion recognition: databases, features, and data fusion strategies 2012Kernel Cross-Modal Factor Analysis for Information Fusion With Application to Bimodal Emotion Recognition 20193D CNN-Based Speech Emotion Recognition Using K-Means Clustering and Spectrograms 2018Audio-visual emotion fusion (AVEF): A deep efficient weighted approach 2023A review of multimodal emotion recognition from datasets, preprocessing, features, and fusion methods 2020Impact of Feature Selection Algorithm on Speech Emotion Recognition Using Deep Convolutional Neural Network 2019An Active Learning Paradigm for Online Audio-Visual Emotion Recognition 2011Error Weighted Semi-Coupled Hidden Markov Model for Audio-Visual Emotion Recognition

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.