Work2019

Multimodal Emotion Recognition from Eye Image, Eye Movement and EEG Using Deep Neural Networks

Jiang-Jian Guo · Rong Feng Zhou · Li-Ming Zhao · Bao‐Liang Lu

73 citations0/0 files storedAnnual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

Emotion and Mood Recognition EEG and Brain-Computer Interfaces Gaze Tracking and Assistive Technology

Abstract

In consideration of the complexity of recording electroencephalography(EEG), some researchers are trying to find new features of emotion recognition. In order to investigate the potential of eye tracking glasses for multimodal emotion recognition, we collect and use eye images to classify five emotions along with eye movements and EEG. We compare four combinations of the three different types of data and two kinds of fusion methods, feature level fusion and Bimodal Deep AutoEncoder (BDAE). According to the three-modality fusion features generated by BDAE, the best mean accuracy of 79.63% is achieved. By analyzing the confusion matrices, we find that the three modalities can provide complementary information for recognizing five emotions. Meanwhile, the experimental results indicate that the classifiers with eye image and eye movement fusion features can achieve a comparable classification accuracy of 71.99%.

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 73 citations exist upstream.

References

2015Investigating Critical Frequency Bands and Channels for EEG-Based Emotion Recognition with Deep Neural Networks 2013Differential entropy feature for EEG-based emotion classification 2011EEG-based emotion recognition during watching movies 2016Emotion Recognition Using Multimodal Deep Learning 2014Multimodal emotion recognition using EEG and eye tracking data 2015Combining eye movements and EEG to enhance emotion recognition 2019Eyemotion: Classifying Facial Expressions in VR Using Eye-Tracking Cameras 2019Classification of Five Emotions from EEG and Eye Movement Signals: Discrimination Ability and Stability over Time

Cited by

2022A new approach for product evaluation based on integration of EEG and eye-tracking 2024Research Progress of EEG-Based Emotion Recognition: A Survey 2023EmotionKD: A Cross-Modal Knowledge Distillation Framework for Emotion Recognition Based on Physiological Signals 2022Emerging Wearable Biosensor Technologies for Stress Monitoring and Their Real-World Applications 2022An Emotion Recognition Method Based on Eye Movement and Audiovisual Features in MOOC Learning Environment

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.