The time-course of intermodal binding between seeing and hearing affective information

Gilles Pourtois · Béatrice de Gelder · Jean Vroomen · Bruno Rossion · Marc Crommelinck

197 citations0/0 files storedNeuroreport

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

Multisensory perception and integration Neuroscience and Music Perception Tactile and Sensory Interactions

Abstract

Intermodal binding between affective information that is seen as well as heard triggers a mandatory process of audiovisual integration. In order to track the time course of this audiovisual binding, event related brain potentials were recorded while subjects saw facial expression and concurrently heard auditory fragment. The results suggest that the combination of the two inputs is early in time (110 ms post-stimulus) and translates as a specific enhancement in amplitude of the auditory NI component. These findings are compatible with previous functional neuroimaging results of audiovisual speech showing strong audiovisual interactions in auditory cortex in the form of magnetic response amplifications, as well as with electrophysiological studies demonstrating early audiovisual interactions (before 200 ms post-stimulus). Moreover, our results show that the informational content present in the two modalities plays a crucial role in triggering the intermodal binding process.

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 197 citations exist upstream.

References

1994The merging of the senses 1976Measuring facial movement 1999Auditory-Visual Integration during Multimodal Object Recognition in Humans: A Behavioral and Electrophysiological Study

Cited by

2011The temporal dynamics of processing emotions from vocal, facial, and bodily expressions 2002Processing of changes in visual speech in the human auditory cortex 2001Is cross-modal integration of emotional expressions independent of attentional resources?2006Crossmodal integration of emotional information from face and voice in the infant brain 2009Association of trait emotional intelligence and individual fMRI‐activation patterns during the perception of social signals from voice and face 2004Multisensory integration of emotional faces and voices in schizophrenics 2002Fear recognition in the voice is modulated by unconsciously recognized facial expressions but not by unconsciously recognized affective pictures 2006Combined perception of emotion in pictures and musical sounds

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.