SpeechFormer: A Hierarchical Efficient Framework Incorporating the Characteristics of Speech

Weidong Chen · Xiaofen Xing · Xiangmin Xu · Jianxin Pang

69 citations0/0 files storedInterspeech (USB)

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

Emotion and Mood Recognition Speech Recognition and Synthesis Speech and Audio Processing

Abstract

Transformer has obtained promising results on cognitive speech signal processing field, which is of interest in various applications ranging from emotion to neurocognitive disorder analysis. However, most works treat speech signal as a whole, leading to the neglect of the pronunciation structure that is unique to speech and reflects the cognitive process. Meanwhile, Transformer has heavy computational burden due to its full attention operation. In this paper, a hierarchical efficient framework, called SpeechFormer, which considers the structural characteristics of speech, is proposed and can be served as a general-purpose backbone for cognitive speech signal processing. The proposed SpeechFormer consists of frame, phoneme, word and utterance stages in succession, each performing a neighboring attention according to the structural pattern of speech with high computational efficiency. SpeechFormer is evaluated on speech emotion recognition (IEMOCAP & MELD) and neurocognitive disorder detection (Pitt & DAIC-WOZ) tasks, and the results show that SpeechFormer outperforms the standard Transformer-based framework while greatly reducing the computational cost. Furthermore, our SpeechFormer achieves comparable results to the state-of-the-art approaches.

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 69 citations exist upstream.

References

2008IEMOCAP: interactive emotional dyadic motion capture database 2019MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations 2017Automatic speech emotion recognition using recurrent neural networks with local attention 2014The Distress Analysis Interview Corpus of human and computer interviews 2015High-level feature representation using recurrent neural network for speech emotion recognition 2021CTNet: Conversational Transformer Network for Emotion Recognition 2011Deep neural networks for acoustic emotion recognition: Raising the benchmarks 1985A Stochastic Approximation Method

Cited by

2024Multimodal Sensing for Depression Risk Detection: Integrating Audio, Video, and Text Data 2024Text-guided multimodal depression detection via cross-modal feature reconstruction and decomposition 2023DST: Deformable Speech Transformer for Emotion Recognition 2023Enhancing accuracy and privacy in speech-based depression detection through speaker disentanglement

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.