Facial Emotion Detection to Assess Learner's State of Mind in an Online Learning System

Abstract

Despite the success and the popularity of the online learning system, it still lacks in dynamically adapting suitable pedagogical methods according to the changing emotions and behaviour of the learner, as can be done in the face-to-face mode of learning. This makes the learning process mechanized, which significantly affects the learning outcome. To resolve this, the first and necessary step is to assess the emotion of a learner and identify the change of emotions during a learning session. Usually, images of facial expressions are analysed to assess one's state of mind. However, human emotions are far more complex, and these psychological states may not be reflected only through the basic emotion of a learner (i.e. analysing a single image), but a combination of two or more emotions which may be reflected on the face over a period of time. From a real survey, we derived four complex emotions that are a combination of basic human emotions often experienced by a learner, in concert, during a learning session. To capture these combined emotions correctly, we considered a fixed set of continuous image frames, instead of discrete images. We built a CNN model to classify the basic emotions and then identify the states of mind of the learners. The outcome is verified mathematically as well as surveying the learners. The results show a 65% and 62% accuracy respectively, for emotion classification and state of mind identification.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 99 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.