Crossmodal integration of object features: Voxel-based correlations in brain-damaged patients

Abstract

How does the brain bind together the different sensory features of objects to form meaningful, multimodal object representations? Human functional imaging findings implicate the left posterior superior temporal sulcus/middle temporal gyrus (pSTS/MTG) in crossmodal integration, while animal ablation findings support a hierarchical object processing model in which outputs from each sensory stream are integrated in perirhinal cortex (PRc) of the anteromedial temporal lobe. To determine which neural regions are necessary for integrating audiovisual object features, and which regions are necessary for understanding the meaning of crossmodal objects, we administered crossmodal (audio-visual) and unimodal (auditory, visual) integration tasks to 16 brain-damaged patients. We correlated patients' behavioural performance with measures of neural integrity (signal intensity) of each voxel across the brains of each patient. The integrity of bilateral anteromedial and temporopolar regions, but not pSTS/MTG, was significantly correlated with poorer crossmodal compared with unimodal integration performance, and with meaningful aspects of crossmodal integration. Additional analyses confirmed the negative crossmodal integration findings in the pSTS/MTG: performance on a sentence-picture matching control task was significantly correlated with MTG/STG voxel signal intensities, suggesting that a truncated range of signals in this region could not have been responsible for the lack of a significant correlation between integrity and crossmodal integration performance, and individual analyses of three patients with lesions in pSTS/MTG but spared anteromedial temporal cortex revealed equivalent unimodal and crossmodal integration performance. These results extend findings from the non-human primate literature into the human domain by demonstrating that anteromedial temporal cortex is critically involved in crossmodal integration of object features. However, pSTS/MTG appears to play a supportive but non-essential role during crossmodal integration. Taken together, the present findings are consistent with a neurocognitive account of object representations which claims that anteromedial temporal lobe is critically involved in the formation and processing of complex, multimodal object representations.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 63 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.