Evaluating Multimodal Wearable Sensors for Quantifying Affective States and Depression With Neural Networks

Abdullah Ahmed · Jayroop Ramesh · Sandipan Ganguly · Raafat Aburukba · Assim A. Sagahyroon · Fadi Aloul

40 citations0/0 files storedIEEE Sensors Journal

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

Emotion and Mood Recognition Context-Aware Activity Recognition Systems Non-Invasive Vital Sign Monitoring

Abstract

With the increasing proliferation of embedded sensors in wearable devices, there is potential for modeling individual emotional and mental state variations. The popular measure for the quantification of emotions outlines the affective states of arousal and valences, with high and low being the discrete categories of interest. Recent works explore the discernability of digital behavior differences between groups with and without mental disorders. However, the interaction between physiological states and affective states within a predominantly depressive population remains to be studied with the aid of wearables. Despite the pervasiveness of emotional state inference through the tracking of ubiquitous physiological trackers such as heart rate, blood volume pulse, skin conductance, and motion, a dearth of work is noted in the exploration of physiological markers in single and multimodal settings. This work provides an extensive evaluation of a convolutional neural network with an attention mechanism ensembled with a random forest algorithm to effectively leverage multiple raw signal-to-image transformations as feature inputs to predict depression severity and affective state. The proposed models are assessed on the Daily Ambulatory Psychological and Physiological recording for Emotion Research (DAPPER) dataset, and achieve the sensitivity: specificity scores of 58.75%:45.59%, 62.34%:43.41%, 49.43%:51.70% for predicting depression, valence, and arousal with a mixture of uni- and bi- modality applying Continuous Wavelet Transforms and Short-time Fourier Transform to motion skin-conductance readings, respectively. This work is envisioned as a preliminary study to contribute towards the monitoring of affective states among a depressed population by utilizing low-frequency sensor recordings with the DAPPER dataset.

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 40 citations exist upstream.

References

2018Introducing WESAD, a Multimodal Dataset for Wearable Stress and Affect Detection 2012The relation between valence and arousal in subjective experience.2020Human Emotion Recognition: Review of Sensors and Methods 2013Variation of a test’s sensitivity and specificity with disease prevalence 2011Continuous Prediction of Spontaneous Affect from Multiple Cues and Modalities in Valence-Arousal Space 2018Mental health monitoring with multimodal sensing and machine learning: A survey 2021Smart Devices and Wearable Technologies to Detect and Monitor Mental Health Conditions and Stress: A Systematic Review 2012Activity-Aware Mental Stress Detection Using Physiological Sensors

Cited by

2024Advancements in Affective Disorder Detection: Using Multimodal Physiological Signals and Neuromorphic Computing Based on SNNs

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.