AU-aware Deep Networks for facial expression recognition

Mengyi Liu, Shaoxin Li, Shiguang Shan, Xilin Chen

2013-04-01otherInternational Conference on Automatic Face and Gesture Recognition229 citationsclosed

No readable copy stored yet — this record is metadata only.

Emotion and Mood Recognition Face recognition and analysis Face and Expression Recognition

Abstract

In this paper, we propose to construct a deep architecture, AU-aware Deep Networks (AUDN), for facial expression recognition by elaborately utilizing the prior knowledge that the appearance variations caused by expression can be decomposed into a batch of local facial Action Units (AUs). The proposed AUDN is composed of three sequential modules: the first module consists of two layers, i.e., a convolution layer and a max-pooling layer, which aim to generate an over-complete representation encoding all expression-specific appearance variations over all possible locations; In the second module, an AU-aware receptive field layer is designed to search subsets of the over-complete representation, each of which aims at best simulating the combination of AUs; In the last module, multilayer Restricted Boltzmann Machines (RBM) are exploited to learn hierarchical features, which are then concatenated for final expression recognition. Experiments on three expression databases CK+, MMI and SFEW demonstrate the effectiveness of AUDN in both lab-controlled and wild environments. All our results are better than or at least competitive to the best known results.

Citations in this library

Stored only when both papers are in the corpus — 229 citations exist upstream.

References

1978Facial Action Coding System

2010The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression

2002Comprehensive database for facial expression analysis

2011Static facial expression analysis in tough conditions: Data, evaluation protocol and benchmark

2006Dynamics of facial expression extracted automatically from video

Cited by

2015AU-inspired Deep Networks for Facial Expression Feature Learning

2014Combining Multiple Kernel Methods on Riemannian Manifold for Emotion Recognition in the Wild

2016Selective Transfer Machine for Personalized Facial Expression Analysis

2019Three convolutional neural network models for facial expression recognition in the wild

2022Impact of Deep Learning Approaches on Facial Expression Recognition in Healthcare Industries

2018Blended Emotion in-the-Wild: Multi-label Facial Expression Recognition Using Crowdsourced Annotations and Deep Locality Feature Learning

2017Deep spatial-temporal feature fusion for facial expression recognition in static images

2016How deep neural networks can improve emotion recognition on video data