Towards multimodal emotion recognition in e-learning environments

Kiavash Bahreini · Rob J. Nadolski · Wim Westera

137 citations0/0 files storedInteractive Learning Environments

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

Emotion and Mood Recognition Online Learning and Analytics Intelligent Tutoring Systems and Adaptive Learning

Abstract

This paper presents a framework (FILTWAM (Framework for Improving Learning Through Webcams And Microphones)) for real-time emotion recognition in e-learning by using webcams. FILTWAM offers timely and relevant feedback based upon learner's facial expressions and verbalizations. FILTWAM's facial expression software module has been developed and tested in a proof-of-concept study. The main goal of this study was to validate the use of webcam data for a real-time and adequate interpretation of facial expressions into extracted emotional states. The software was calibrated with 10 test persons. They received the same computer-based tasks in which each of them were requested 100 times to mimic specific facial expressions. All sessions were recorded on video. For the validation of the face emotion recognition software, two experts annotated and rated participants’ recorded behaviours. Expert findings were contrasted with the software results and showed an overall value of kappa of 0.77. An overall accuracy of our software based on the requested emotions and the recognized emotions is 72%. Whereas existing software only allows not-real time, discontinuous and obtrusive facial detection, our software allows to continuously and unobtrusively monitor learners' behaviours and converts these behaviours directly into emotional states. This paves the way for enhancing the quality and efficacy of e-learning by including the learner's emotional states.

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 137 citations exist upstream.

References

1977The Measurement of Observer Agreement for Categorical Data 2008A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions 1992The Impact of Emotions on Learning and Achievement: Towards a Theory of Cognitive/Motivational Mediators 2005Affective Computing: A Review 2012Face Expression Recognition and Analysis: The State of the Art 2005Affective multimodal human-computer interaction 2013Empathic reactions of younger and older adults: No age related decline in affective responding

Cited by

2024A Systematic Review on Multimodal Emotion Recognition: Building Blocks, Current State, Applications, and Challenges 2025A review on EEG-based multimodal learning for emotion recognition 2025A Comprehensive Review of Multimodal Emotion Recognition: Techniques, Challenges, and Future Directions 2019Comparative study on emotions analysis from facial expressions in children with and without learning disabilities in virtual learning environment 2024Bridging computer and education sciences: A systematic review of automated emotion recognition in online learning environments

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.