Machine Learning and End-to-End Deep Learning for Monitoring Driver Distractions From Physiological and Visual Signals

Martin Gjoreski · Matjaž Gams · Mitja Luštrek · Pelin Genc · Jens-U. Garbas · Teena Hassan

87 citations0/0 files storedIEEE Access

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

Human-Automation Interaction and Safety Emotion and Mood Recognition Sleep and Work-Related Fatigue

Abstract

It is only a matter of time until autonomous vehicles become ubiquitous; however, human driving supervision will remain a necessity for decades. To assess the driver's ability to take control over the vehicle in critical scenarios, driver distractions can be monitored using wearable sensors or sensors that are embedded in the vehicle, such as video cameras. The types of driving distractions that can be sensed with various sensors is an open research question that this study attempts to answer. This study compared data from physiological sensors (palm electrodermal activity (pEDA), heart rate and breathing rate) and visual sensors (eye tracking, pupil diameter, nasal EDA (nEDA), emotional activation and facial action units (AUs)) for the detection of four types of distractions. The dataset was collected in a previous driving simulation study. The statistical tests showed that the most informative feature/modality for detecting driver distraction depends on the type of distraction, with emotional activation and AUs being the most promising. The experimental comparison of seven classical machine learning (ML) and seven end-to-end deep learning (DL) methods, which were evaluated on a separate test set of 10 subjects, showed that when classifying windows into distracted or not distracted, the highest F1-score of 79% was realized by the extreme gradient boosting (XGB) classifier using 60-second windows of AUs as input. When classifying complete driving sessions, XGB's F1-score was 94%. The best-performing DL model was a spectro-temporal ResNet, which realized an F1-score of 75% when classifying segments and an F1-score of 87% when classifying complete driving sessions. Finally, this study identified and discussed problems, such as label jitter, scenario overfitting and unsatisfactory generalization performance, that may adversely affect related ML approaches.

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 87 citations exist upstream.

References

1978Facial Action Coding System 1995A desicion-theoretic generalization of on-line learning and an application to boosting 2005Detecting Stress During Real-World Driving Tasks Using Physiological Sensors 2006Real-Time System for Monitoring Driver Vigilance 2018Driver Fatigue Detection Systems: A Review 2010Driver Drowsiness Classification Using Fuzzy Wavelet-Packet-Based Feature-Extraction Algorithm 2011Driver distraction and driver inattention: Definition, relationship and taxonomy 2007Observer-Based Measurement of Facial Expression with the Facial Action Coding System

Cited by

2022Bringing Emotion Recognition Out of the Lab into Real Life: Recent Advances in Sensors and Machine Learning 2021Sensitivity of Electrodermal Activity Features for Driver Arousal Measurement in Cognitive Load: The Application in Automated Driving Systems 2020Can We Ditch Feature Engineering? End-to-End Deep Learning for Affect Recognition from Physiological Sensor Data 2021Robust Deep Learning-Based Driver Distraction Detection and Classification

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.