EmoNet: A Transfer Learning Framework for Multi-Corpus Speech Emotion Recognition

Abstract

In this manuscript, the topic of multi-corpus Speech Emotion Recognition (SER) is approached from a deep transfer learning perspective. A large corpus of emotional speech data,EmoSet, is assembled from a number of existing Speech Emotion Recognition (SER) corpora. In total,EmoSetcontains84 181 audio recordingsfrom26 SER corporawith a total duration of over65 hours. The corpus is then utilised to create a novel framework for multi-corpus SER and general audio recognition, namelyEmoNet. A combination of a deep ResNet architecture and residual adapters is transferred from the field of multi-domain visual recognition to multi-corpus SER onEmoSet. The introduced residual adapter approach enables parameter efficient training of a multi-domain SER model on all 26 corpora. A shared model with only 3.5 times the number of parameters of a model trained on a single database leads to increased performance for 21 of the 26 corpora inEmoSet. Using repeated training runs and Almost Stochastic Order with significance level of$\alpha = 0.05$, these improvements are further significant for 15 datasets while there are just three corpora that see only significant decreases across the residual adapter transfer experiments. Finally, we make ourEmoNetframework publicly available for users and developers athttps://github.com/EIHW/EmoNet.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 76 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.