CNN and LSTM Based Facial Expression Analysis Model for a Humanoid Robot

Tzuu‐Hseng S. Li · Ping‐Huan Kuo · Ting-Nan Tsai · Po‐Chien Luan

108 citations0/0 files storedIEEE Access

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

Emotion and Mood Recognition Face recognition and analysis Face and Expression Recognition

Abstract

Robots must be able to recognize human emotions to improve the human-robot interaction (HRI). This study proposes an emotion recognition system for a humanoid robot. The robot is equipped with a camera to capture users' facial images, and it uses this system to recognize users' emotions and responds appropriately. The emotion recognition system, based on a deep neural network, learns six basic emotions: happiness, anger, disgust, fear, sadness, and surprise. First, a convolutional neural network (CNN) is used to extract visual features by learning on a large number of static images. Second, a long short-term memory (LSTM) recurrent neural network is used to determine the relationship between the transformation of facial expressions in image sequences and the six basic emotions. Third, CNN and LSTM are combined to exploit their advantages in the proposed model. Finally, the performance of the emotion recognition system is improved by using transfer learning, that is, by transferring knowledge of related but different problems. The performance of the proposed system is verified through leave-one-out cross-validation and compared with that of other models. The system is applied to a humanoid robot to demonstrate its practicability for improving the HRI.

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 108 citations exist upstream.

References

2010The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression 2005What are emotions? And how can they be measured?2017AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild 2016Facial expression recognition with Convolutional Neural Networks: Coping with few data and the training sample order 2006The eNTERFACE’05 Audio-Visual Emotion Database 2015Image based Static Facial Expression Recognition with Multiple Deep Network Learning 2018Emotion recognition using deep learning approach from audio–visual emotional big data 2011Continuous Prediction of Spontaneous Affect from Multiple Cues and Modalities in Valence-Arousal Space

Cited by

2019Deep Learning Classification of Neuro-Emotional Phase Domain Complexity Levels Induced by Affective Video Film Clips 2023Facial expression recognition using bidirectional LSTM - CNN 2021Dual integrated convolutional neural network for real-time facial expression recognition in the wild 2024Detection of human emotions through facial expressions using hybrid convolutional neural network-recurrent neural network algorithm

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.