Inverse Effectiveness and Multisensory Interactions in Visual Event-Related Potentials with Audiovisual Speech
Identifieur interne : 001696 ( Pmc/Checkpoint ); précédent : 001695; suivant : 001697Inverse Effectiveness and Multisensory Interactions in Visual Event-Related Potentials with Audiovisual Speech
Auteurs : Ryan A. Stevenson ; Maxim Bushmakin ; Sunah Kim ; Mark T. Wallace ; Aina Puce ; Thomas W. JamesSource :
- Brain topography [ 0896-0267 ] ; 2012.
Abstract
In recent years, it has become evident that neural responses previously considered to be unisensory can be modulated by sensory input from other modalities. In this regard, visual neural activity elicited to viewing a face is strongly influenced by concurrent incoming auditory information, particularly speech. Here, we applied an additive-factors paradigm aimed at quantifying the impact that auditory speech has on visual event-related potentials (ERPs) elicited to visual speech. These multisensory interactions were measured across parametrically varied stimulus salience, quantified in terms of signal to noise, to provide novel insights into the neural mechanisms of audiovisual speech perception. First, we measured a monotonic increase of the amplitude of the visual P1-N1-P2 ERP complex during a spoken-word recognition task with increases in stimulus salience. ERP component amplitudes varied directly with stimulus salience for visual, audiovisual, and summed unisensory recordings. Second, we measured changes in multisensory gain across salience levels. During audiovisual speech, the P1 and P1-N1 components exhibited less multisensory gain relative to the summed unisensory components with reduced salience, while N1-P2 amplitude exhibited greater multisensory gain as salience was reduced, consistent with the principle of inverse effectiveness. The amplitude interactions were correlated with behavioral measures of multisensory gain across salience levels as measured by response times, suggesting that change in multisensory gain associated with unisensory salience modulations reflects an increased efficiency of visual speech processing.
Url:
DOI: 10.1007/s10548-012-0220-7
PubMed: 22367585
PubMed Central: 3789520
Affiliations:
Links toward previous steps (curation, corpus...)
Links to Exploration step
PMC:3789520Le document en format XML
<record><TEI><teiHeader><fileDesc><titleStmt><title xml:lang="en">Inverse Effectiveness and Multisensory Interactions in Visual Event-Related Potentials with Audiovisual Speech</title>
<author><name sortKey="Stevenson, Ryan A" sort="Stevenson, Ryan A" uniqKey="Stevenson R" first="Ryan A." last="Stevenson">Ryan A. Stevenson</name>
</author>
<author><name sortKey="Bushmakin, Maxim" sort="Bushmakin, Maxim" uniqKey="Bushmakin M" first="Maxim" last="Bushmakin">Maxim Bushmakin</name>
</author>
<author><name sortKey="Kim, Sunah" sort="Kim, Sunah" uniqKey="Kim S" first="Sunah" last="Kim">Sunah Kim</name>
</author>
<author><name sortKey="Wallace, Mark T" sort="Wallace, Mark T" uniqKey="Wallace M" first="Mark T." last="Wallace">Mark T. Wallace</name>
</author>
<author><name sortKey="Puce, Aina" sort="Puce, Aina" uniqKey="Puce A" first="Aina" last="Puce">Aina Puce</name>
</author>
<author><name sortKey="James, Thomas W" sort="James, Thomas W" uniqKey="James T" first="Thomas W." last="James">Thomas W. James</name>
</author>
</titleStmt>
<publicationStmt><idno type="wicri:source">PMC</idno>
<idno type="pmid">22367585</idno>
<idno type="pmc">3789520</idno>
<idno type="url">http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3789520</idno>
<idno type="RBID">PMC:3789520</idno>
<idno type="doi">10.1007/s10548-012-0220-7</idno>
<date when="2012">2012</date>
<idno type="wicri:Area/Pmc/Corpus">001A40</idno>
<idno type="wicri:Area/Pmc/Curation">001A40</idno>
<idno type="wicri:Area/Pmc/Checkpoint">001696</idno>
</publicationStmt>
<sourceDesc><biblStruct><analytic><title xml:lang="en" level="a" type="main">Inverse Effectiveness and Multisensory Interactions in Visual Event-Related Potentials with Audiovisual Speech</title>
<author><name sortKey="Stevenson, Ryan A" sort="Stevenson, Ryan A" uniqKey="Stevenson R" first="Ryan A." last="Stevenson">Ryan A. Stevenson</name>
</author>
<author><name sortKey="Bushmakin, Maxim" sort="Bushmakin, Maxim" uniqKey="Bushmakin M" first="Maxim" last="Bushmakin">Maxim Bushmakin</name>
</author>
<author><name sortKey="Kim, Sunah" sort="Kim, Sunah" uniqKey="Kim S" first="Sunah" last="Kim">Sunah Kim</name>
</author>
<author><name sortKey="Wallace, Mark T" sort="Wallace, Mark T" uniqKey="Wallace M" first="Mark T." last="Wallace">Mark T. Wallace</name>
</author>
<author><name sortKey="Puce, Aina" sort="Puce, Aina" uniqKey="Puce A" first="Aina" last="Puce">Aina Puce</name>
</author>
<author><name sortKey="James, Thomas W" sort="James, Thomas W" uniqKey="James T" first="Thomas W." last="James">Thomas W. James</name>
</author>
</analytic>
<series><title level="j">Brain topography</title>
<idno type="ISSN">0896-0267</idno>
<idno type="eISSN">1573-6792</idno>
<imprint><date when="2012">2012</date>
</imprint>
</series>
</biblStruct>
</sourceDesc>
</fileDesc>
<profileDesc><textClass></textClass>
</profileDesc>
</teiHeader>
<front><div type="abstract" xml:lang="en"><p id="P1">In recent years, it has become evident that neural responses previously considered to be unisensory can be modulated by sensory input from other modalities. In this regard, visual neural activity elicited to viewing a face is strongly influenced by concurrent incoming auditory information, particularly speech. Here, we applied an additive-factors paradigm aimed at quantifying the impact that auditory speech has on visual event-related potentials (ERPs) elicited to visual speech. These multisensory interactions were measured across parametrically varied stimulus salience, quantified in terms of signal to noise, to provide novel insights into the neural mechanisms of audiovisual speech perception. First, we measured a monotonic increase of the amplitude of the visual P1-N1-P2 ERP complex during a spoken-word recognition task with increases in stimulus salience. ERP component amplitudes varied directly with stimulus salience for visual, audiovisual, and summed unisensory recordings. Second, we measured changes in multisensory gain across salience levels. During audiovisual speech, the P1 and P1-N1 components exhibited less multisensory gain relative to the summed unisensory components with reduced salience, while N1-P2 amplitude exhibited greater multisensory gain as salience was reduced, consistent with the principle of inverse effectiveness. The amplitude interactions were correlated with behavioral measures of multisensory gain across salience levels as measured by response times, suggesting that change in multisensory gain associated with unisensory salience modulations reflects an increased efficiency of visual speech processing.</p>
</div>
</front>
</TEI>
<pmc article-type="research-article"><pmc-comment>The publisher of this article does not allow downloading of the full text in XML form.</pmc-comment>
<pmc-dir>properties manuscript</pmc-dir>
<front><journal-meta><journal-id journal-id-type="nlm-journal-id">8903034</journal-id>
<journal-id journal-id-type="pubmed-jr-id">1738</journal-id>
<journal-id journal-id-type="nlm-ta">Brain Topogr</journal-id>
<journal-id journal-id-type="iso-abbrev">Brain Topogr</journal-id>
<journal-title-group><journal-title>Brain topography</journal-title>
</journal-title-group>
<issn pub-type="ppub">0896-0267</issn>
<issn pub-type="epub">1573-6792</issn>
</journal-meta>
<article-meta><article-id pub-id-type="pmid">22367585</article-id>
<article-id pub-id-type="pmc">3789520</article-id>
<article-id pub-id-type="doi">10.1007/s10548-012-0220-7</article-id>
<article-id pub-id-type="manuscript">NIHMS512732</article-id>
<article-categories><subj-group subj-group-type="heading"><subject>Article</subject>
</subj-group>
</article-categories>
<title-group><article-title>Inverse Effectiveness and Multisensory Interactions in Visual Event-Related Potentials with Audiovisual Speech</article-title>
</title-group>
<contrib-group><contrib contrib-type="author" corresp="yes"><name><surname>Stevenson</surname>
<given-names>Ryan A.</given-names>
</name>
<aff id="A1">Department of Psychological and Brain Sciences, Indiana University, Bloomington, IN, USA. Program in Neuroscience, Indiana University, Bloomington, IN, USA. Department of Hearing and Speech Sciences, Vanderbilt University Medical Center, MRB 3, Room 7110, Nashville, TN 37232, USA</aff>
<email>ryan.andrew.stevenson@gmail.com</email>
</contrib>
<contrib contrib-type="author"><name><surname>Bushmakin</surname>
<given-names>Maxim</given-names>
</name>
<aff id="A2">Department of Psychological and Brain Sciences, Indiana University, Bloomington, IN, USA. Cognitive Science Program, Indiana University, Bloomington, IN, USA</aff>
</contrib>
<contrib contrib-type="author"><name><surname>Kim</surname>
<given-names>Sunah</given-names>
</name>
<aff id="A3">Program in Neuroscience, Indiana University, Bloomington, IN, USA. Cognitive Science Program, Indiana University, Bloomington, IN, USA. Vision Science Program, School of Optometry, University of California, Berkeley, CA, USA</aff>
</contrib>
<contrib contrib-type="author"><name><surname>Wallace</surname>
<given-names>Mark T.</given-names>
</name>
<aff id="A4">Department of Hearing and Speech Sciences, Vanderbilt University Medical Center, MRB 3, Room 7110, Nashville, TN 37232, USA</aff>
</contrib>
<contrib contrib-type="author"><name><surname>Puce</surname>
<given-names>Aina</given-names>
</name>
<aff id="A5">Department of Psychological and Brain Sciences, Indiana University, Bloomington, IN, USA. Program in Neuroscience, Indiana University, Bloomington, IN, USA. Cognitive Science Program, Indiana University, Bloomington, IN, USA</aff>
</contrib>
<contrib contrib-type="author"><name><surname>James</surname>
<given-names>Thomas W.</given-names>
</name>
<aff id="A6">Department of Psychological and Brain Sciences, Indiana University, Bloomington, IN, USA. Program in Neuroscience, Indiana University, Bloomington, IN, USA. Cognitive Science Program, Indiana University, Bloomington, IN, USA</aff>
</contrib>
</contrib-group>
<pub-date pub-type="nihms-submitted"><day>17</day>
<month>9</month>
<year>2013</year>
</pub-date>
<pub-date pub-type="epub"><day>25</day>
<month>2</month>
<year>2012</year>
</pub-date>
<pub-date pub-type="ppub"><month>7</month>
<year>2012</year>
</pub-date>
<pub-date pub-type="pmc-release"><day>03</day>
<month>10</month>
<year>2013</year>
</pub-date>
<volume>25</volume>
<issue>3</issue>
<fpage>308</fpage>
<lpage>326</lpage>
<permissions><copyright-statement>© Springer Science+Business Media, LLC 2012</copyright-statement>
<copyright-year>2012</copyright-year>
</permissions>
<abstract><p id="P1">In recent years, it has become evident that neural responses previously considered to be unisensory can be modulated by sensory input from other modalities. In this regard, visual neural activity elicited to viewing a face is strongly influenced by concurrent incoming auditory information, particularly speech. Here, we applied an additive-factors paradigm aimed at quantifying the impact that auditory speech has on visual event-related potentials (ERPs) elicited to visual speech. These multisensory interactions were measured across parametrically varied stimulus salience, quantified in terms of signal to noise, to provide novel insights into the neural mechanisms of audiovisual speech perception. First, we measured a monotonic increase of the amplitude of the visual P1-N1-P2 ERP complex during a spoken-word recognition task with increases in stimulus salience. ERP component amplitudes varied directly with stimulus salience for visual, audiovisual, and summed unisensory recordings. Second, we measured changes in multisensory gain across salience levels. During audiovisual speech, the P1 and P1-N1 components exhibited less multisensory gain relative to the summed unisensory components with reduced salience, while N1-P2 amplitude exhibited greater multisensory gain as salience was reduced, consistent with the principle of inverse effectiveness. The amplitude interactions were correlated with behavioral measures of multisensory gain across salience levels as measured by response times, suggesting that change in multisensory gain associated with unisensory salience modulations reflects an increased efficiency of visual speech processing.</p>
</abstract>
<kwd-group><kwd>Multisensory integration</kwd>
<kwd>Inverse effectiveness</kwd>
<kwd>ERPs</kwd>
<kwd>P1-N1-P2</kwd>
<kwd>N170</kwd>
<kwd>Speech perception</kwd>
<kwd>Face perception</kwd>
</kwd-group>
<funding-group><award-group><funding-source country="United States">National Institute on Deafness and Other Communication Disorders : NIDCD</funding-source>
<award-id>T32 DC000012 || DC</award-id>
</award-group>
<award-group><funding-source country="United States">National Institute on Deafness and Other Communication Disorders : NIDCD</funding-source>
<award-id>R34 DC010927 || DC</award-id>
</award-group>
<award-group><funding-source country="United States">National Institute on Deafness and Other Communication Disorders : NIDCD</funding-source>
<award-id>F32 DC011993 || DC</award-id>
</award-group>
</funding-group>
</article-meta>
</front>
</pmc>
<affiliations><list></list>
<tree><noCountry><name sortKey="Bushmakin, Maxim" sort="Bushmakin, Maxim" uniqKey="Bushmakin M" first="Maxim" last="Bushmakin">Maxim Bushmakin</name>
<name sortKey="James, Thomas W" sort="James, Thomas W" uniqKey="James T" first="Thomas W." last="James">Thomas W. James</name>
<name sortKey="Kim, Sunah" sort="Kim, Sunah" uniqKey="Kim S" first="Sunah" last="Kim">Sunah Kim</name>
<name sortKey="Puce, Aina" sort="Puce, Aina" uniqKey="Puce A" first="Aina" last="Puce">Aina Puce</name>
<name sortKey="Stevenson, Ryan A" sort="Stevenson, Ryan A" uniqKey="Stevenson R" first="Ryan A." last="Stevenson">Ryan A. Stevenson</name>
<name sortKey="Wallace, Mark T" sort="Wallace, Mark T" uniqKey="Wallace M" first="Mark T." last="Wallace">Mark T. Wallace</name>
</noCountry>
</tree>
</affiliations>
</record>
Pour manipuler ce document sous Unix (Dilib)
EXPLOR_STEP=$WICRI_ROOT/Ticri/CIDE/explor/HapticV1/Data/Pmc/Checkpoint
HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 001696 | SxmlIndent | more
Ou
HfdSelect -h $EXPLOR_AREA/Data/Pmc/Checkpoint/biblio.hfd -nk 001696 | SxmlIndent | more
Pour mettre un lien sur cette page dans le réseau Wicri
{{Explor lien |wiki= Ticri/CIDE |area= HapticV1 |flux= Pmc |étape= Checkpoint |type= RBID |clé= PMC:3789520 |texte= Inverse Effectiveness and Multisensory Interactions in Visual Event-Related Potentials with Audiovisual Speech }}
Pour générer des pages wiki
HfdIndexSelect -h $EXPLOR_AREA/Data/Pmc/Checkpoint/RBID.i -Sk "pubmed:22367585" \ | HfdSelect -Kh $EXPLOR_AREA/Data/Pmc/Checkpoint/biblio.hfd \ | NlmPubMed2Wicri -a HapticV1
![]() | This area was generated with Dilib version V0.6.23. | ![]() |