Decoding Sound and Imagery Content in Early Visual Cortex

Petra Vetter · Fraser W. Smith · Lars F. Muckli

323 citations0/0 files storedCurrent Biology

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

Multisensory perception and integration Visual perception and processing mechanisms Tactile and Sensory Interactions

Abstract

Human early visual cortex was traditionally thought to process simple visual features such as orientation, contrast, and spatial frequency via feedforward input from the lateral geniculate nucleus (e.g., [1]). However, the role of nonretinal influence on early visual cortex is so far insufficiently investigated despite much evidence that feedback connections greatly outnumber feedforward connections [2-5]. Here, we explored in five fMRI experiments how information originating from audition and imagery affects the brain activity patterns in early visual cortex in the absence of any feedforward visual stimulation. We show that category-specific information from both complex natural sounds and imagery can be read out from early visual cortex activity in blindfolded participants. The coding of nonretinal information in the activity patterns of early visual cortex is common across actual auditory perception and imagery and may be mediated by higher-level multisensory areas. Furthermore, this coding is robust to mild manipulations of attention and working memory but affected by orthogonal, cognitively demanding visuospatial processing. Crucially, the information fed down to early visual cortex is category specific and generalizes to sound exemplars of the same category, providing evidence for abstract information feedback rather than precise pictorial feedback. Our results suggest that early visual cortex receives nonretinal input from other brain areas when it is generated by auditory perception and/or imagery, and this input carries common abstract information. Our findings are compatible with feedback of predictive information to the earliest visual input level (e.g., [6]), in line with predictive coding models [7-10].

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 323 citations exist upstream.

References

2003Multisensory convergence in calcarine visual areas in macaque monkey 2002Deactivation of Sensory-Specific Cortex by Cross-Modal Stimuli

Cited by

2014Varieties of cognitive penetration in visual perception 2015Predictive coding and multisensory integration: an attentional account of the multisensory mind 2015The multisensory function of the human primary visual cortex 2015Motor imagery of hand actions: Decoding the content of motor imagery from brain activity in frontal and parietal motor areas 2015Words Jump-Start Vision: A Label Advantage in Object Recognition 2016Distinct Computational Principles Govern Multisensory Integration in Primary Sensory and Association Cortices 2016Imagined and Executed Actions in the Human Motor System: Testing Neural Similarity Between Execution and Imagery of Actions with a Multivariate Approach 2015A multisensory perspective of working memory

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.