Multimodal Hierarchical CNN Feature Fusion for Stress Detection

Ramanathan Subramanian · O.V. Ramana Murthy

80 citations0/0 files storedIEEE Access

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

Emotion and Mood Recognition Heart Rate Variability and Autonomic Control ECG Monitoring and Analysis

Abstract

Stress is one of the most severe concerns in modern life. High-level stress can create various diseases or loss of focus and productivity at work. Being under stress prevents people from recognizing their stress levels, so early stress detection is essential. Recently, multimodal fusion has enhanced the performance of stress detection models using Deep Learning (DL) techniques. The low, mid, and high-level features of a Convolutional Neural Network (CNN) are discriminative. A comprehensive feature representation can be obtained by fusing all three levels of CNN’s features. This study mainly focuses on detecting stress by exploiting these advantages using a multimodal hierarchical CNN feature fusion. The two multimodal physiological signals used in this study are Electrodermal activity (EDA) and Electrocardiogram (ECG). We develop a hierarchical feature set by concatenating multi-level CNN features for each modality. Multimodal fusion on both hierarchical feature sets is performed using the Multimodal Transfer Module (MMTM). The experiments are carried out with raw frequency domain data and the features from the frequency bands to study the effectiveness of both. The model’s performance is compared to the different combinations of hierarchical features from low, mid, and high levels. To verify the generalizability, the proposed approach has been evaluated on four benchmark datasets - ASCERTAIN, CLAS, MAUS, and WAUC. The proposed method showed its effectiveness by outperforming existing models by 1-2%, respectively, on frequency band features. It is observed that the hierarchical feature set from all three levels performed better than all other combinations by 2-4%. As a result, this strategy can be a useful addition to stress detection.

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 80 citations exist upstream.

References

2011DEAP: A Database for Emotion Analysis ;Using Physiological Signals 2018A Review of Emotion Recognition Using Physiological Signals 2016ASCERTAIN: Emotion and Personality Recognition Using Commercial Sensors 2015DECAF: MEG-Based Multimodal Database for Decoding Affective Physiological Responses 2019Continuous Stress Detection Using Wearable Sensors in Real Life: Algorithmic Programming Contest Case Study 2020Stress Detection with Machine Learning and Deep Learning using Multimodal Physiological Data 2018Emotion Based Music Recommendation System Using Wearable Physiological Sensors 2019Feature Extraction and Selection for Emotion Recognition from Electrodermal Activity

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.