The interaction between stimulus factors and cognitive factors during multisensory integration of audiovisual speech
Identifieur interne : 000C29 ( Pmc/Curation ); précédent : 000C28; suivant : 000C30The interaction between stimulus factors and cognitive factors during multisensory integration of audiovisual speech
Auteurs : Ryan A. Stevenson [Canada] ; Mark T. Wallace [États-Unis] ; Nicholas Altieri [États-Unis]Source :
- Frontiers in Psychology [ 1664-1078 ] ; 2014.
Url:
DOI: 10.3389/fpsyg.2014.00352
PubMed: 24817856
PubMed Central: 4013471
Links toward previous steps (curation, corpus...)
- to stream Pmc, to step Corpus: Pour aller vers cette notice dans l'étape Curation :000C29
Links to Exploration step
PMC:4013471Le document en format XML
<record><TEI><teiHeader><fileDesc><titleStmt><title xml:lang="en">The interaction between stimulus factors and cognitive factors during multisensory integration of audiovisual speech</title><author><name sortKey="Stevenson, Ryan A" sort="Stevenson, Ryan A" uniqKey="Stevenson R" first="Ryan A." last="Stevenson">Ryan A. Stevenson</name><affiliation wicri:level="1"><nlm:aff id="aff1"><institution>Psychology Department, University of Toronto</institution><country>Toronto, ON, Canada</country></nlm:aff><country xml:lang="fr">Canada</country><wicri:regionArea></wicri:regionArea><wicri:regionArea># see nlm:aff region in country</wicri:regionArea></affiliation></author><author><name sortKey="Wallace, Mark T" sort="Wallace, Mark T" uniqKey="Wallace M" first="Mark T." last="Wallace">Mark T. Wallace</name><affiliation wicri:level="1"><nlm:aff id="aff2"><institution>Vanderbilt Brain Institute, Vanderbilt University Medical Center</institution><country>Nashville, TN, USA</country></nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea></wicri:regionArea><wicri:regionArea># see nlm:aff region in country</wicri:regionArea></affiliation><affiliation wicri:level="1"><nlm:aff id="aff3"><institution>Department of Hearing and Speech Sciences, Vanderbilt University Medical Center</institution><country>Nashville, TN, USA</country></nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea></wicri:regionArea><wicri:regionArea># see nlm:aff region in country</wicri:regionArea></affiliation><affiliation wicri:level="1"><nlm:aff id="aff4"><institution>Vanderbilt Kennedy Center, Vanderbilt University Medical Center</institution><country>Nashville, TN, USA</country></nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea></wicri:regionArea><wicri:regionArea># see nlm:aff region in country</wicri:regionArea></affiliation><affiliation wicri:level="1"><nlm:aff id="aff5"><institution>Department of Psychology, Vanderbilt University</institution><country>Nashville, TN, USA</country></nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea></wicri:regionArea><wicri:regionArea># see nlm:aff region in country</wicri:regionArea></affiliation><affiliation wicri:level="1"><nlm:aff id="aff6"><institution>Department of Psychiatry, Vanderbilt University Medical Center</institution><country>Nashville, TN, USA</country></nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea></wicri:regionArea><wicri:regionArea># see nlm:aff region in country</wicri:regionArea></affiliation></author><author><name sortKey="Altieri, Nicholas" sort="Altieri, Nicholas" uniqKey="Altieri N" first="Nicholas" last="Altieri">Nicholas Altieri</name><affiliation wicri:level="1"><nlm:aff id="aff7"><institution>Department of Communication Sciences and Disorders, Idaho State University</institution><country>Pocatello, ID, USA</country></nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea></wicri:regionArea><wicri:regionArea># see nlm:aff region in country</wicri:regionArea></affiliation></author></titleStmt><publicationStmt><idno type="wicri:source">PMC</idno><idno type="pmid">24817856</idno><idno type="pmc">4013471</idno><idno type="url">http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4013471</idno><idno type="RBID">PMC:4013471</idno><idno type="doi">10.3389/fpsyg.2014.00352</idno><date when="2014">2014</date><idno type="wicri:Area/Pmc/Corpus">000C29</idno><idno type="wicri:Area/Pmc/Curation">000C29</idno></publicationStmt><sourceDesc><biblStruct><analytic><title xml:lang="en" level="a" type="main">The interaction between stimulus factors and cognitive factors during multisensory integration of audiovisual speech</title><author><name sortKey="Stevenson, Ryan A" sort="Stevenson, Ryan A" uniqKey="Stevenson R" first="Ryan A." last="Stevenson">Ryan A. Stevenson</name><affiliation wicri:level="1"><nlm:aff id="aff1"><institution>Psychology Department, University of Toronto</institution><country>Toronto, ON, Canada</country></nlm:aff><country xml:lang="fr">Canada</country><wicri:regionArea></wicri:regionArea><wicri:regionArea># see nlm:aff region in country</wicri:regionArea></affiliation></author><author><name sortKey="Wallace, Mark T" sort="Wallace, Mark T" uniqKey="Wallace M" first="Mark T." last="Wallace">Mark T. Wallace</name><affiliation wicri:level="1"><nlm:aff id="aff2"><institution>Vanderbilt Brain Institute, Vanderbilt University Medical Center</institution><country>Nashville, TN, USA</country></nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea></wicri:regionArea><wicri:regionArea># see nlm:aff region in country</wicri:regionArea></affiliation><affiliation wicri:level="1"><nlm:aff id="aff3"><institution>Department of Hearing and Speech Sciences, Vanderbilt University Medical Center</institution><country>Nashville, TN, USA</country></nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea></wicri:regionArea><wicri:regionArea># see nlm:aff region in country</wicri:regionArea></affiliation><affiliation wicri:level="1"><nlm:aff id="aff4"><institution>Vanderbilt Kennedy Center, Vanderbilt University Medical Center</institution><country>Nashville, TN, USA</country></nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea></wicri:regionArea><wicri:regionArea># see nlm:aff region in country</wicri:regionArea></affiliation><affiliation wicri:level="1"><nlm:aff id="aff5"><institution>Department of Psychology, Vanderbilt University</institution><country>Nashville, TN, USA</country></nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea></wicri:regionArea><wicri:regionArea># see nlm:aff region in country</wicri:regionArea></affiliation><affiliation wicri:level="1"><nlm:aff id="aff6"><institution>Department of Psychiatry, Vanderbilt University Medical Center</institution><country>Nashville, TN, USA</country></nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea></wicri:regionArea><wicri:regionArea># see nlm:aff region in country</wicri:regionArea></affiliation></author><author><name sortKey="Altieri, Nicholas" sort="Altieri, Nicholas" uniqKey="Altieri N" first="Nicholas" last="Altieri">Nicholas Altieri</name><affiliation wicri:level="1"><nlm:aff id="aff7"><institution>Department of Communication Sciences and Disorders, Idaho State University</institution><country>Pocatello, ID, USA</country></nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea></wicri:regionArea><wicri:regionArea># see nlm:aff region in country</wicri:regionArea></affiliation></author></analytic><series><title level="j">Frontiers in Psychology</title><idno type="eISSN">1664-1078</idno><imprint><date when="2014">2014</date></imprint></series></biblStruct></sourceDesc></fileDesc><profileDesc><textClass></textClass></profileDesc></teiHeader><back><div1 type="bibliography"><listBibl><biblStruct><analytic><author><name sortKey="Calvert, G A" uniqKey="Calvert G">G. A. Calvert</name></author><author><name sortKey="Campbell, R" uniqKey="Campbell R">R. Campbell</name></author><author><name sortKey="Brammer, M J" uniqKey="Brammer M">M. J. Brammer</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Dixon, N F" uniqKey="Dixon N">N. F. Dixon</name></author><author><name sortKey="Spitz, L" uniqKey="Spitz L">L. Spitz</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="James, T W" uniqKey="James T">T. W. James</name></author><author><name sortKey="Stevenson, R A" uniqKey="Stevenson R">R. A. Stevenson</name></author><author><name sortKey="Kim, S" uniqKey="Kim S">S. Kim</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Kim, S" uniqKey="Kim S">S. Kim</name></author><author><name sortKey="Stevenson, R A" uniqKey="Stevenson R">R. A. Stevenson</name></author><author><name sortKey="James, T W" uniqKey="James T">T. W. James</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Lachs, L" uniqKey="Lachs L">L. Lachs</name></author><author><name sortKey="Pisoni, D B" uniqKey="Pisoni D">D. B. Pisoni</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Laurienti, P J" uniqKey="Laurienti P">P. J. Laurienti</name></author><author><name sortKey="Kraft, R A" uniqKey="Kraft R">R. A. Kraft</name></author><author><name sortKey="Maldjian, J A" uniqKey="Maldjian J">J. A. Maldjian</name></author><author><name sortKey="Burdette, J H" uniqKey="Burdette J">J. H. Burdette</name></author><author><name sortKey="Wallace, M T" uniqKey="Wallace M">M. T. Wallace</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Macaluso, E" uniqKey="Macaluso E">E. Macaluso</name></author><author><name sortKey="George, N" uniqKey="George N">N. George</name></author><author><name sortKey="Dolan, R" uniqKey="Dolan R">R. Dolan</name></author><author><name sortKey="Spence, C" uniqKey="Spence C">C. Spence</name></author><author><name sortKey="Driver, J" uniqKey="Driver J">J. Driver</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Miller, L M" uniqKey="Miller L">L. M. Miller</name></author><author><name sortKey="D Esposito, M" uniqKey="D Esposito M">M. D'Esposito</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Royal, D W" uniqKey="Royal D">D. W. Royal</name></author><author><name sortKey="Carriere, B N" uniqKey="Carriere B">B. N. Carriere</name></author><author><name sortKey="Wallace, M T" uniqKey="Wallace M">M. T. Wallace</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Spence, C" uniqKey="Spence C">C. Spence</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Stevenson, R A" uniqKey="Stevenson R">R. A. Stevenson</name></author><author><name sortKey="Fister, J K" uniqKey="Fister J">J. K. Fister</name></author><author><name sortKey="Barnett, Z P" uniqKey="Barnett Z">Z. P. Barnett</name></author><author><name sortKey="Nidiffer, A R" uniqKey="Nidiffer A">A. R. Nidiffer</name></author><author><name sortKey="Wallace, M T" uniqKey="Wallace M">M. T. Wallace</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Stevenson, R A" uniqKey="Stevenson R">R. A. Stevenson</name></author><author><name sortKey="Vanderklok, R M" uniqKey="Vanderklok R">R. M. VanDerKlok</name></author><author><name sortKey="Pisoni, D B" uniqKey="Pisoni D">D. B. Pisoni</name></author><author><name sortKey="James, T W" uniqKey="James T">T. W. James</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Stevenson, R A" uniqKey="Stevenson R">R. A. Stevenson</name></author><author><name sortKey="Wallace, M T" uniqKey="Wallace M">M. T. Wallace</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Stevenson, R A" uniqKey="Stevenson R">R. A. Stevenson</name></author><author><name sortKey="Zemtsov, R K" uniqKey="Zemtsov R">R. K. Zemtsov</name></author><author><name sortKey="Wallace, M T" uniqKey="Wallace M">M. T. Wallace</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Ten Oever, S" uniqKey="Ten Oever S">S. Ten Oever</name></author><author><name sortKey="Sack, A T" uniqKey="Sack A">A. T. Sack</name></author><author><name sortKey="Wheat, K L" uniqKey="Wheat K">K. L. Wheat</name></author><author><name sortKey="Bien, N" uniqKey="Bien N">N. Bien</name></author><author><name sortKey="Van Atteveldt, N" uniqKey="Van Atteveldt N">N. van Atteveldt</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Treisman, A" uniqKey="Treisman A">A. Treisman</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Vatakis, A" uniqKey="Vatakis A">A. Vatakis</name></author><author><name sortKey="Spence, C" uniqKey="Spence C">C. Spence</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Vatakis, A" uniqKey="Vatakis A">A. Vatakis</name></author><author><name sortKey="Spence, C" uniqKey="Spence C">C. Spence</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Wallace, M T" uniqKey="Wallace M">M. T. Wallace</name></author><author><name sortKey="Roberson, G E" uniqKey="Roberson G">G. E. Roberson</name></author><author><name sortKey="Hairston, W D" uniqKey="Hairston W">W. D. Hairston</name></author><author><name sortKey="Stein, B E" uniqKey="Stein B">B. E. Stein</name></author><author><name sortKey="Vaughan, J W" uniqKey="Vaughan J">J. W. Vaughan</name></author><author><name sortKey="Schirillo, J A" uniqKey="Schirillo J">J. A. Schirillo</name></author></analytic></biblStruct></listBibl></div1></back></TEI><pmc article-type="article-commentary"><pmc-dir>properties open_access</pmc-dir>
<front><journal-meta><journal-id journal-id-type="nlm-ta">Front Psychol</journal-id><journal-id journal-id-type="iso-abbrev">Front Psychol</journal-id><journal-id journal-id-type="publisher-id">Front. Psychol.</journal-id><journal-title-group><journal-title>Frontiers in Psychology</journal-title></journal-title-group><issn pub-type="epub">1664-1078</issn><publisher><publisher-name>Frontiers Media S.A.</publisher-name></publisher></journal-meta><article-meta><article-id pub-id-type="pmid">24817856</article-id><article-id pub-id-type="pmc">4013471</article-id><article-id pub-id-type="doi">10.3389/fpsyg.2014.00352</article-id><article-categories><subj-group subj-group-type="heading"><subject>Psychology</subject><subj-group><subject>General Commentary Article</subject></subj-group></subj-group></article-categories><title-group><article-title>The interaction between stimulus factors and cognitive factors during multisensory integration of audiovisual speech</article-title></title-group><contrib-group><contrib contrib-type="author"><name><surname>Stevenson</surname><given-names>Ryan A.</given-names></name><xref ref-type="aff" rid="aff1"><sup>1</sup></xref><xref ref-type="author-notes" rid="fn001"><sup>*</sup></xref><uri xlink:type="simple" xlink:href="http://community.frontiersin.org/people/u/48934"></uri></contrib><contrib contrib-type="author"><name><surname>Wallace</surname><given-names>Mark T.</given-names></name><xref ref-type="aff" rid="aff2"><sup>2</sup></xref><xref ref-type="aff" rid="aff3"><sup>3</sup></xref><xref ref-type="aff" rid="aff4"><sup>4</sup></xref><xref ref-type="aff" rid="aff5"><sup>5</sup></xref><xref ref-type="aff" rid="aff6"><sup>6</sup></xref><uri xlink:type="simple" xlink:href="http://community.frontiersin.org/people/u/1255"></uri></contrib><contrib contrib-type="author"><name><surname>Altieri</surname><given-names>Nicholas</given-names></name><xref ref-type="aff" rid="aff7"><sup>7</sup></xref><uri xlink:type="simple" xlink:href="http://community.frontiersin.org/people/u/27496"></uri></contrib></contrib-group><aff id="aff1"><sup>1</sup><institution>Psychology Department, University of Toronto</institution><country>Toronto, ON, Canada</country></aff><aff id="aff2"><sup>2</sup><institution>Vanderbilt Brain Institute, Vanderbilt University Medical Center</institution><country>Nashville, TN, USA</country></aff><aff id="aff3"><sup>3</sup><institution>Department of Hearing and Speech Sciences, Vanderbilt University Medical Center</institution><country>Nashville, TN, USA</country></aff><aff id="aff4"><sup>4</sup><institution>Vanderbilt Kennedy Center, Vanderbilt University Medical Center</institution><country>Nashville, TN, USA</country></aff><aff id="aff5"><sup>5</sup><institution>Department of Psychology, Vanderbilt University</institution><country>Nashville, TN, USA</country></aff><aff id="aff6"><sup>6</sup><institution>Department of Psychiatry, Vanderbilt University Medical Center</institution><country>Nashville, TN, USA</country></aff><aff id="aff7"><sup>7</sup><institution>Department of Communication Sciences and Disorders, Idaho State University</institution><country>Pocatello, ID, USA</country></aff><author-notes><corresp id="fn001">*Correspondence: <email xlink:type="simple">ryan.andrew.stevenson@gmail.com</email></corresp><fn fn-type="other" id="fn002"><p>This article was submitted to Language Sciences, a section of the journal Frontiers in Psychology.</p></fn><fn fn-type="edited-by"><p>Edited by: Bruce D. McCandliss, Vanderbilt University, USA</p></fn><fn fn-type="edited-by"><p>Reviewed by: Urs Maurer, University of Zurich, Switzerland</p></fn></author-notes><pub-date pub-type="epub"><day>01</day><month>5</month><year>2014</year></pub-date><pub-date pub-type="collection"><year>2014</year></pub-date><volume>5</volume><elocation-id>352</elocation-id><history><date date-type="received"><day>05</day><month>1</month><year>2014</year></date><date date-type="accepted"><day>03</day><month>4</month><year>2014</year></date></history><permissions><copyright-statement>Copyright © 2014 Stevenson, Wallace and Altieri.</copyright-statement><copyright-year>2014</copyright-year><license license-type="open-access" xlink:href="http://creativecommons.org/licenses/by/3.0/"><license-p>This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.</license-p></license></permissions><related-article id="d35e176" related-article-type="commentary-article" xlink:href="23805110" ext-link-type="pubmed" journal-id="Front Psychol" journal-id-type="nlm-ta" vol="4" page="331">A commentary on <article-title>Audio-visual onset differences are used to determine syllable identity for ambiguous audio-visual stimulus pairs</article-title> by Ten Oever, S., Sack, A. T., Wheat, K. L., Bien, N., and van Atteveldt, N. (2013). Front. Psychol. 4:331. doi: 10.3389/fpsyg.2013.00331</related-article><kwd-group><kwd>multisensory processing</kwd><kwd>audiovisual integration</kwd><kwd>speech perception</kwd><kwd>temporal processing</kwd><kwd>sensory processing</kwd><kwd>crossmodal</kwd><kwd>perceptual binding</kwd><kwd>speech integration</kwd></kwd-group><counts><fig-count count="1"></fig-count><table-count count="0"></table-count><equation-count count="0"></equation-count><ref-count count="19"></ref-count><page-count count="3"></page-count><word-count count="1886"></word-count></counts></article-meta></front><body><p>The amount of research focused on multisensory speech perception has expanded considerably in recent years. Much of this research has focused on which factors influence whether or not an auditory and a visual speech input are “integrated” (i.e., perceptually bound); a special case of how our perceptual systems solve the “binding problem” (Treisman, <xref rid="B16" ref-type="bibr">1996</xref>). The factors that have been identified as influencing multisensory integration can be roughly divided into two groups. First are the low-level stimulus factors that include the physical characteristics of the sensory signals. The most commonly studied of these include the spatial (e.g., Macaluso et al., <xref rid="B7" ref-type="bibr">2004</xref>; Wallace et al., <xref rid="B19" ref-type="bibr">2004</xref>) and temporal (e.g., Miller and D'Esposito, <xref rid="B8" ref-type="bibr">2005</xref>; Stevenson et al., <xref rid="B12" ref-type="bibr">2011</xref>) relationship of the two inputs, and their relative effectiveness (e.g., James et al., <xref rid="B3" ref-type="bibr">2012</xref>; Kim et al., <xref rid="B4" ref-type="bibr">2012</xref>) in driving a neural, perceptual, or behavioral response. The second group of factors can be considered more higher-order or cognitive, and include factors such as the semantic congruence of the auditory and visual signals (Laurienti et al., <xref rid="B6" ref-type="bibr">2004</xref>) or whether or not the gender of the speaker's voice is matched to the face (Lachs and Pisoni, <xref rid="B5" ref-type="bibr">2004</xref>).</p><p>While these two categories can be considered conceptually distinct, they are related because of their mutual dependence upon the natural statistics of signals in the environment. When auditory and visual speech signals are closely proximate in time (low-level), they are more likely to have originated from the same speaker, and thus should be integrated (Dixon and Spitz, <xref rid="B2" ref-type="bibr">1980</xref>; Stevenson et al., <xref rid="B14" ref-type="bibr">2012b</xref>). Likewise, if an auditory and a visual speech signal are semantically congruent (high-level), they are more likely to have originated from the same speaker and thus should be integrated (Calvert et al., <xref rid="B1" ref-type="bibr">2000</xref>). Given that these low- and high-level factors are each reflective of the natural statistics of the environmental signals, they will generally co-vary. Taking speech as an example, in a natural setting, the temporally-coincident auditory and visual components of a syllable or word are also semantically congruent (Spence, <xref rid="B10" ref-type="bibr">2007</xref>).</p><p>To date, most research has investigated these low- and high-level factors independently. These studies have been highly informative, providing descriptions as to how each of these factors contributes to the process of multisensory integration. What has not received a great deal of focus is the interplay between these factors. A handful of experiments have investigated how low-level factors interact with one another and influence multisensory integration (Macaluso et al., <xref rid="B7" ref-type="bibr">2004</xref>; Royal et al., <xref rid="B9" ref-type="bibr">2009</xref>; Stevenson et al., <xref rid="B11" ref-type="bibr">2012a</xref>), but few have attempted to bridge between low-level stimulus-characteristics and high-level cognitive factors (Vatakis and Spence, <xref rid="B18" ref-type="bibr">2007</xref>). A recent article by Ten Oever et al. (<xref rid="B15" ref-type="bibr">2013</xref>), <italic>Audio-visual onset differences are used to determine syllable identity for ambiguous audio-visual stimulus pairs</italic> addresses this gap in our understanding by investigating the interaction between stimulus timing and semantic congruency modulated by changes in place of articulation or voicing.</p><p>In this study, participants were presented with single-syllable stimuli, with auditory, visual, and audiovisual syllables systematically manipulated according to place of articulation and voicing. In addition, the temporal alignment of the audiovisual presentations was also parametrically varied. Hence, semantic content was varied through changes both in the auditory (voicing) and visual (place of articulation) signals, while at the same time, the relative timing of the auditory and visual stimuli were systematically varied. While the results specific to these factors are interesting on their own, most germane to this commentary is how these two factors interacted. The authors measured the window of time within which the visual cue influenced the syllable that was heard. This probabilistic construct, referred to as the “time window of integration” or the “temporal binding window,” has been shown to vary greatly according the type of stimulus being integrated (Vatakis and Spence, <xref rid="B17" ref-type="bibr">2006</xref>; Stevenson and Wallace, <xref rid="B13" ref-type="bibr">2013</xref>). In the Ten Oever et al. study, semantically congruent stimuli were found to be associated with a wider temporal binding window than semantically incongruent stimuli. That is, stimulus components that are semantically matched have higher rates of integration at more temporally disparate offsets.</p><p>The result is surprising in that it runs counter to predictions generated by hierarchical serial models. In such models, lower-level properties such as stimulus timing are processed initially, and are then followed by the processing of the linguistic (i.e., semantic) content in the auditory and visual signals. However, the current results, by illustrating an interaction between timing and congruency, suggest that hierarchical models are insufficient to explain the data. Rather, we posit that these results are better interpreted within a “parallel accumulation of evidence” framework (Figure <xref ref-type="fig" rid="F1">1</xref>). In this model, the temporal relationship of two sensory inputs provides important information about the likelihood that those two inputs originated from the same speaker and should be integrated. In addition, the semantic congruence of these inputs also provides information as to whether or not the two sensory inputs should be bound. Importantly, these two types of evidence are pooled into a single decision criterion. Thus, within such a framework, when stimuli are semantically congruent, a decreased amount of temporal alignment is needed in order to cross a decision bound that would result in these two inputs being integrated, manifesting in a broader temporal binding window for semantically congruent speech stimulus pairs.</p><fig id="F1" position="float"><label>Figure 1</label><caption><p><bold>The left panel shows a “parallel accumulator” model with auditory and visual evidence racing toward threshold (γ).</bold> The amount of visual influence on the auditory signal is a function, <italic>f</italic>, of parameters δ, and θ which represent temporal coincidence detection and phonetic congruence respectively, both of which contribute evidence to a single accumulator. The serial model on the right shows two separate stages where integration is affected first by temporal, then by semantic processing. Hence, in stage 1, visual information influences auditory processing only as a function <italic>f</italic> of temporal coincidence. In stage 2, visual information influences auditory processing solely as a function,<italic>g</italic>, of phonemic compatibility.</p></caption><graphic xlink:href="fpsyg-05-00352-g0001"></graphic></fig><p>Through this interaction between stimulus timing and semantic congruence, Ten Oever and colleagues provided compelling evidence that low-level stimulus and high-level cognitive factors are not processed in a completely serial manner, but rather interact with one another in the formation of a perceptual decision. These results have significant implications in informing our view as to the neurobiological substrates involved in real-world multisensory perceptual processes. Most importantly, the work suggests that significant feedforward and feedback circuits are engaged in the processing of naturalistic multisensory stimuli, and that these circuits work in a parallel and cooperative fashion in evaluating the statistical relations of the stimuli to one another on both their low-level (i.e., stimulus feature) and high-level (i.e., learned semantic) correspondences.</p><sec><title>Conflict of interest statement</title><p>The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.</p></sec></body><back><ref-list><title>References</title><ref id="B1"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Calvert</surname><given-names>G. A.</given-names></name><name><surname>Campbell</surname><given-names>R.</given-names></name><name><surname>Brammer</surname><given-names>M. J.</given-names></name></person-group> (<year>2000</year>). <article-title>Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex</article-title>. <source>Curr. Biol</source>. <volume>10</volume>, <fpage>649</fpage>–<lpage>657</lpage><pub-id pub-id-type="doi">10.1016/S0960-9822(00)00513-3</pub-id><pub-id pub-id-type="pmid">10837246</pub-id></mixed-citation></ref><ref id="B2"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Dixon</surname><given-names>N. F.</given-names></name><name><surname>Spitz</surname><given-names>L.</given-names></name></person-group> (<year>1980</year>). <article-title>The detection of auditory visual desynchrony</article-title>. <source>Perception</source><volume>9</volume>, <fpage>719</fpage>–<lpage>721</lpage><pub-id pub-id-type="doi">10.1068/p090719</pub-id><pub-id pub-id-type="pmid">7220244</pub-id></mixed-citation></ref><ref id="B3"><mixed-citation publication-type="book"><person-group person-group-type="author"><name><surname>James</surname><given-names>T. W.</given-names></name><name><surname>Stevenson</surname><given-names>R. A.</given-names></name><name><surname>Kim</surname><given-names>S.</given-names></name></person-group> (<year>2012</year>). <article-title>Inverse effectiveness in multisensory processing</article-title>, in <source>The New Handbook of Multisensory Processes</source>, ed B. E. Stein (<publisher-loc>Cambridge, MA</publisher-loc>: <publisher-name>MIT Press</publisher-name>), <fpage>207</fpage>–<lpage>221</lpage></mixed-citation></ref><ref id="B4"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Kim</surname><given-names>S.</given-names></name><name><surname>Stevenson</surname><given-names>R. A.</given-names></name><name><surname>James</surname><given-names>T. W.</given-names></name></person-group> (<year>2012</year>). <article-title>Visuo-haptic neuronal convergence demonstrated with an inversely effective pattern of BOLD activation</article-title>. <source>J. Cogn. Neurosci</source>. <volume>24</volume>, <fpage>830</fpage>–<lpage>842</lpage><pub-id pub-id-type="doi">10.1162/jocn_a_00176</pub-id><pub-id pub-id-type="pmid">22185495</pub-id></mixed-citation></ref><ref id="B5"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Lachs</surname><given-names>L.</given-names></name><name><surname>Pisoni</surname><given-names>D. B.</given-names></name></person-group> (<year>2004</year>). <article-title>Crossmodal source identification in speech perception</article-title>. <source>Ecol. Psychol</source>. <volume>16</volume>, <fpage>159</fpage>–<lpage>187</lpage><pub-id pub-id-type="doi">10.1207/s15326969eco1603_1</pub-id><pub-id pub-id-type="pmid">21544262</pub-id></mixed-citation></ref><ref id="B6"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Laurienti</surname><given-names>P. J.</given-names></name><name><surname>Kraft</surname><given-names>R. A.</given-names></name><name><surname>Maldjian</surname><given-names>J. A.</given-names></name><name><surname>Burdette</surname><given-names>J. H.</given-names></name><name><surname>Wallace</surname><given-names>M. T.</given-names></name></person-group> (<year>2004</year>). <article-title>Semantic congruence is a critical factor in multisensory behavioral performance</article-title>. <source>Exp. Brain Res</source>. <volume>158</volume>, <fpage>405</fpage>–<lpage>414</lpage><pub-id pub-id-type="doi">10.1007/s00221-004-1913-2</pub-id><pub-id pub-id-type="pmid">15221173</pub-id></mixed-citation></ref><ref id="B7"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Macaluso</surname><given-names>E.</given-names></name><name><surname>George</surname><given-names>N.</given-names></name><name><surname>Dolan</surname><given-names>R.</given-names></name><name><surname>Spence</surname><given-names>C.</given-names></name><name><surname>Driver</surname><given-names>J.</given-names></name></person-group> (<year>2004</year>). <article-title>Spatial and temporal factors during processing of audiovisual speech: a PET study</article-title>. <source>Neuroimage</source><volume>21</volume>, <fpage>725</fpage>–<lpage>732</lpage><pub-id pub-id-type="doi">10.1016/j.neuroimage.2003.09.049</pub-id><pub-id pub-id-type="pmid">14980575</pub-id></mixed-citation></ref><ref id="B8"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Miller</surname><given-names>L. M.</given-names></name><name><surname>D'Esposito</surname><given-names>M.</given-names></name></person-group> (<year>2005</year>). <article-title>Perceptual fusion and stimulus coincidence in the cross-modal integration of speech</article-title>. <source>J. Neurosci</source>. <volume>25</volume>, <fpage>5884</fpage>–<lpage>5893</lpage><pub-id pub-id-type="doi">10.1523/JNEUROSCI.0896-05.2005</pub-id><pub-id pub-id-type="pmid">15976077</pub-id></mixed-citation></ref><ref id="B9"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Royal</surname><given-names>D. W.</given-names></name><name><surname>Carriere</surname><given-names>B. N.</given-names></name><name><surname>Wallace</surname><given-names>M. T.</given-names></name></person-group> (<year>2009</year>). <article-title>Spatiotemporal architecture of cortical receptive fields and its impact on multisensory interactions</article-title>. <source>Exp. Brain Res</source>. <volume>198</volume>, <fpage>127</fpage>–<lpage>136</lpage><pub-id pub-id-type="doi">10.1007/s00221-009-1772-y</pub-id><pub-id pub-id-type="pmid">19308362</pub-id></mixed-citation></ref><ref id="B10"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Spence</surname><given-names>C.</given-names></name></person-group> (<year>2007</year>). <article-title>Audiovisual multisensory integration</article-title>. <source>Acoust. Sci. Technol</source>. <volume>28</volume>, <fpage>61</fpage>–<lpage>70</lpage><pub-id pub-id-type="doi">10.1250/ast.28.61</pub-id></mixed-citation></ref><ref id="B11"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Stevenson</surname><given-names>R. A.</given-names></name><name><surname>Fister</surname><given-names>J. K.</given-names></name><name><surname>Barnett</surname><given-names>Z. P.</given-names></name><name><surname>Nidiffer</surname><given-names>A. R.</given-names></name><name><surname>Wallace</surname><given-names>M. T.</given-names></name></person-group> (<year>2012a</year>). <article-title>Interactions between the spatial and temporal stimulus factors that influence multisensory integration in human performance</article-title>. <source>Exp. Brain Res</source>. <volume>219.1</volume>, <fpage>121</fpage>–<lpage>137</lpage><pub-id pub-id-type="doi">10.1007/s00221-012-3072-1</pub-id><pub-id pub-id-type="pmid">22447249</pub-id></mixed-citation></ref><ref id="B12"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Stevenson</surname><given-names>R. A.</given-names></name><name><surname>VanDerKlok</surname><given-names>R. M.</given-names></name><name><surname>Pisoni</surname><given-names>D. B.</given-names></name><name><surname>James</surname><given-names>T. W.</given-names></name></person-group> (<year>2011</year>). <article-title>Discrete neural substrates underlie complementary audiovisual speech integration processes</article-title>. <source>Neuroimage</source><volume>55</volume>, <fpage>1339</fpage>–<lpage>1345</lpage><pub-id pub-id-type="doi">10.1016/j.neuroimage.2010.12.063</pub-id><pub-id pub-id-type="pmid">21195198</pub-id></mixed-citation></ref><ref id="B13"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Stevenson</surname><given-names>R. A.</given-names></name><name><surname>Wallace</surname><given-names>M. T.</given-names></name></person-group> (<year>2013</year>). <article-title>Multisensory temporal integration: task and stimulus dependencies</article-title>. <source>Exp. Brain Res</source>. <volume>227</volume>, <fpage>249</fpage>–<lpage>261</lpage><pub-id pub-id-type="doi">10.1007/s00221-013-3507-3</pub-id><pub-id pub-id-type="pmid">23604624</pub-id></mixed-citation></ref><ref id="B14"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Stevenson</surname><given-names>R. A.</given-names></name><name><surname>Zemtsov</surname><given-names>R. K.</given-names></name><name><surname>Wallace</surname><given-names>M. T.</given-names></name></person-group> (<year>2012b</year>). <article-title>Individual differences in the multisensory temporal binding window predict susceptibility to audiovisual illusions</article-title>. <source>J. Exp. Psychol. Hum. Percept. Perform</source>. <volume>38.6</volume>, <fpage>1517</fpage><pub-id pub-id-type="doi">10.1037/a0027339</pub-id><pub-id pub-id-type="pmid">22390292</pub-id></mixed-citation></ref><ref id="B15"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Ten Oever</surname><given-names>S.</given-names></name><name><surname>Sack</surname><given-names>A. T.</given-names></name><name><surname>Wheat</surname><given-names>K. L.</given-names></name><name><surname>Bien</surname><given-names>N.</given-names></name><name><surname>van Atteveldt</surname><given-names>N.</given-names></name></person-group> (<year>2013</year>). <article-title>Audio-visual onset differences are used to determine syllable identity for ambiguous audio-visual stimulus pairs</article-title>. <source>Front. Psychol</source>. <volume>4</volume>:<issue>331</issue><pub-id pub-id-type="doi">10.3389/fpsyg.2013.00331</pub-id><pub-id pub-id-type="pmid">23805110</pub-id></mixed-citation></ref><ref id="B16"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Treisman</surname><given-names>A.</given-names></name></person-group> (<year>1996</year>). <article-title>The binding problem</article-title>. <source>Curr. Opin. Neurobiol</source>. <volume>6</volume>, <fpage>171</fpage>–<lpage>178</lpage><pub-id pub-id-type="doi">10.1016/S0959-4388(96)80070-5</pub-id><pub-id pub-id-type="pmid">8725958</pub-id></mixed-citation></ref><ref id="B17"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Vatakis</surname><given-names>A.</given-names></name><name><surname>Spence</surname><given-names>C.</given-names></name></person-group> (<year>2006</year>). <article-title>Audiovisual synchrony perception for music, speech, and object actions</article-title>. <source>Brain Res</source>. <volume>1111</volume>, <fpage>134</fpage>–<lpage>142</lpage><pub-id pub-id-type="doi">10.1016/j.brainres.2006.05.078</pub-id><pub-id pub-id-type="pmid">16876772</pub-id></mixed-citation></ref><ref id="B18"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Vatakis</surname><given-names>A.</given-names></name><name><surname>Spence</surname><given-names>C.</given-names></name></person-group> (<year>2007</year>). <article-title>Crossmodal binding: evaluating the “unity assumption” using audiovisual speech stimuli</article-title>. <source>Percept. Psychophys</source>. <volume>69</volume>, <fpage>744</fpage>–<lpage>756</lpage><pub-id pub-id-type="doi">10.3758/BF03193776</pub-id><pub-id pub-id-type="pmid">17929697</pub-id></mixed-citation></ref><ref id="B19"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Wallace</surname><given-names>M. T.</given-names></name><name><surname>Roberson</surname><given-names>G. E.</given-names></name><name><surname>Hairston</surname><given-names>W. D.</given-names></name><name><surname>Stein</surname><given-names>B. E.</given-names></name><name><surname>Vaughan</surname><given-names>J. W.</given-names></name><name><surname>Schirillo</surname><given-names>J. A.</given-names></name></person-group> (<year>2004</year>). <article-title>Unifying multisensory signals across time and space</article-title>. <source>Exp. Brain Res</source>. <volume>158</volume>, <fpage>252</fpage>–<lpage>258</lpage><pub-id pub-id-type="doi">10.1007/s00221-004-1899-9</pub-id><pub-id pub-id-type="pmid">15112119</pub-id></mixed-citation></ref></ref-list></back></pmc></record>Pour manipuler ce document sous Unix (Dilib)
EXPLOR_STEP=$WICRI_ROOT/Ticri/CIDE/explor/HapticV1/Data/Pmc/Curation
HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 000C29 | SxmlIndent | more
Ou
HfdSelect -h $EXPLOR_AREA/Data/Pmc/Curation/biblio.hfd -nk 000C29 | SxmlIndent | more
Pour mettre un lien sur cette page dans le réseau Wicri
{{Explor lien
|wiki= Ticri/CIDE
|area= HapticV1
|flux= Pmc
|étape= Curation
|type= RBID
|clé= PMC:4013471
|texte= The interaction between stimulus factors and cognitive factors during multisensory integration of audiovisual speech
}}
Pour générer des pages wiki
HfdIndexSelect -h $EXPLOR_AREA/Data/Pmc/Curation/RBID.i -Sk "pubmed:24817856" \
| HfdSelect -Kh $EXPLOR_AREA/Data/Pmc/Curation/biblio.hfd \
| NlmPubMed2Wicri -a HapticV1
| This area was generated with Dilib version V0.6.23. |  |