Responses to visual speech in human posterior superior temporal gyrus examined with iEEG deconvolution

Brian A. Metzger, John F. Magnotti, Zhengjia Wang, Elizabeth Nesbitt, Patrick J. Karas, Daniel Yoshor, Michael S. Beauchamp

Research output: Contribution to journalReview articlepeer-review


Experimentalists studying multisensory integration compare neural responses to multisensory stimuli with responses to the component modalities presented in isolation. This procedure is problematic for multisensory speech perception since audiovisual speech and auditory-only speech are easily intelligible but visual-only speech is not. To overcome this confound, we developed intracranial encephalography (iEEG) deconvolution. Individual stimuli always contained both auditory and visual speech, but jittering the onset asynchrony between modalities allowed for the time course of the unisensory responses and the interaction between them to be independently estimated. We applied this procedure to electrodes implanted in human epilepsy patients (both male and female) over the posterior superior temporal gyrus (pSTG), a brain area known to be important for speech perception. iEEG deconvolution revealed sustained positive responses to visual-only speech and larger, phasic responses to auditory-only speech. Confirming results from scalp EEG, responses to audiovisual speech were weaker than responses to auditory-only speech, demonstrating a subadditive multisensory neural computation. Leveraging the spatial resolution of iEEG, we extended these results to show that subadditivity is most pronounced in more posterior aspects of the pSTG. Across electrodes, subadditivity correlated with visual responsiveness, supporting a model in which visual speech enhances the efficiency of auditory speech processing in pSTG. The ability to separate neural processes may make iEEG deconvolution useful for studying a variety of complex cognitive and perceptual tasks.

Original languageEnglish (US)
Pages (from-to)6938-6948
Number of pages11
JournalJournal of Neuroscience
Issue number36
StatePublished - Sep 2 2020
Externally publishedYes


  • Audiovisual
  • Human
  • Intracranial
  • Multisensory
  • Speech perception
  • Superior temporal gyrus

ASJC Scopus subject areas

  • General Neuroscience


Dive into the research topics of 'Responses to visual speech in human posterior superior temporal gyrus examined with iEEG deconvolution'. Together they form a unique fingerprint.

Cite this