Multisensory perception of the six basic emotions is modulated by attentional instruction and unattended modality

Sachiko Takagi · Saori Hiramatsu · Ken‐ichi Tabei · Akihiro Tanaka

99 citations0/0 files storedFrontiers in Integrative Neuroscience

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

Multisensory perception and integration Tactile and Sensory Interactions Face Recognition and Perception

Abstract

Previous studies have shown that the perception of facial and vocal affective expressions interacts with each other. Facial expressions usually dominate vocal expressions when we perceive the emotions of face-voice stimuli. In most of these studies, participants were instructed to pay attention to the face or voice. Few studies compared the perceived emotions with and without specific instructions regarding the modality to which attention should be directed. Also, these studies used combinations of the face and voice which expresses two opposing emotions, which limits the generalizability of the findings. The purpose of this study is to examine whether the emotion perception is modulated by instructions to pay attention to the face or voice using the six basic emotions. Also we examine the modality dominance between the face and voice for each emotion category. Before the experiment, we recorded faces and voices which expresses the six basic emotions and orthogonally combined these faces and voices. Consequently, the emotional valence of visual and auditory information was either congruent or incongruent. In the experiment, there were unisensory and multisensory sessions. The multisensory session was divided into three blocks according to whether an instruction was given to pay attention to a given modality (face attention, voice attention, and no instruction). Participants judged whether the speaker expressed happiness, sadness, anger, fear, disgust, or surprise. Our results revealed that instructions to pay attention to one modality and congruency of the emotions between modalities modulated the modality dominance, and the modality dominance is differed for each emotion category. In particular, the modality dominance for anger changed according to each instruction. Analyses also revealed that the modality dominance suggested by the congruency effect can be explained in terms of the facilitation effect and the interference effect.

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 99 citations exist upstream.

References

1978Facial Action Coding System 1971Constants across cultures in the face and emotion.2003Vocal communication of emotion: A review of research paradigms 2010The multifaceted interplay between attention and multisensory integration 2003Selective attention to threat in the dot probe paradigm: differentiating vigilance and difficulty to disengage 2000The perception of emotions by ear and by eye 1989A cross-cultural study of a circumplex model of affect.2006Selective Attention and Audiovisual Integration: Is Attending to Both Modalities a Prerequisite for Early Integration?

Cited by

2017A multisensory perspective on object memory 2018Sensory dominance and multisensory integration as screening tools in aging 2016Predicting the sensory consequences of one’s own action: First evidence for multisensory facilitation 2021Paraverbal Expression of Verbal Irony: Vocal Cues Matter and Facial Cues Even More 2006The Interior Design Course: Principles, Practices and Techniques for the Aspiring Designer

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.