Continuous emotion detection using EEG signals and facial expressions

Mohammad Soleymani · Sadjad Asghari-Esfeden · Maja Pantić · Yun Fu

97 citations0/0 files storedIEEE International Conference on Multimedia and Expo (ICME)

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

Emotion and Mood Recognition EEG and Brain-Computer Interfaces Sentiment Analysis and Opinion Mining

Abstract

Emotions play an important role in how we select and consume multimedia. Recent advances on affect detection are focused on detecting emotions continuously. In this paper, for the first time, we continuously detect valence from electroencephalogram (EEG) signals and facial expressions in response to videos. Multiple annotators provided valence levels continuously by watching the frontal facial videos of participants who watched short emotional videos. Power spectral features from EEG signals as well as facial fiducial points are used as features to detect valence levels for each frame continuously. We study the correlation between features from EEG and facial expressions with continuous valence. We have also verified our model's performance for the emotional highlight detection using emotion recognition from EEG signals. Finally the results of multimodal fusion between facial expression and EEG signals are presented. Having such models we will be able to detect spontaneous and subtle affective responses over time and use them for video highlight detection.

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 97 citations exist upstream.

References

2011DEAP: A Database for Emotion Analysis ;Using Physiological Signals 2005What are emotions? And how can they be measured?1977Evidence for a three-factor theory of emotions 2011A Multimodal Database for Affect Recognition and Implicit Tagging 2011The SEMAINE Database: Annotated Multimodal Records of Emotionally Colored Conversations between a Person and a Limited Agent 2000FEELTRACE: an instrument for recording perceived emotion in real time 2011Continuous Prediction of Spontaneous Affect from Multiple Cues and Modalities in Valence-Arousal Space 2012Categorical and dimensional affect analysis in continuous input: Current trends and future directions

Cited by

2016Deep Neural Networks with Relativity Learning for facial expression recognition 2021Effects of Data Augmentation Method Borderline-SMOTE on Emotion Recognition of EEG Signals Based on Convolutional Neural Network 2019Modeling Emotion in Complex Stories: The Stanford Emotional Narratives Dataset 2019Recognizing Induced Emotions of Movie Audiences from Multimodal Information 2020

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.