Predicting Emotion in Spoken Dialogue from Multiple Knowledge Sources

Katherine Forbes-Riley · Diane J. Litman

94 citations0/0 files stored

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

Emotion and Mood Recognition Intelligent Tutoring Systems and Adaptive Learning Speech and dialogue systems

Abstract

We examine the utility of multiple types of turn-level and contextual linguistic features for automatically predicting student emotions in human-human spoken tutoring dialogues. We first annotate student turns in our corpus for negative, neutral and positive emotions. We then automatically extract features representing acoustic-prosodic and other linguistic information from the speech signal and associated transcriptions. We compare the results of machine learning experiments using different feature sets to predict the annotated emotions. Our best performing feature set contains both acoustic-prosodic and other types of linguistic features, extracted from both the current turn and a context of previous student turns, and yields a prediction accuracy of 84.75%, which is a 44 % relative improvement in error reduction over a baseline. Our results suggest that the intelligent tutoring spoken dialogue system we are developing can be enhanced to automatically predict and adapt to student emotions. 1

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 94 citations exist upstream.

References

2001Emotion recognition in human-computer interaction 1996Assessing agreement on classification tasks: the kappa statistic 2003The production and recognition of emotions in speech: features and algorithms 2002Prosody-based automatic detection of annoyance and frustration in human-computer dialog 2003How to find trouble in communication 2002Combining acoustic and language information for emotion recognition 2005Recognition of negative emotions from the speech signal

Cited by

2020HiTrans: A Transformer-Based Context- and Speaker-Sensitive Model for Emotion Detection in Conversations 2008Influence of contextual information in emotion annotation for spoken dialogue systems 2004Co-training for predicting emotions with spoken dialogue data 2011Pain monitoring: A dynamic and context-sensitive system 2013Predicting and Eliciting Addressee's Emotion in Online Dialogue 2011Real-Time Recognition of Affective States from Nonverbal Features of Speech and Its Application for Public Speaking Skill Analysis

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.