The sound of your lips: electrophysiological cross-modal interactions during hand-to-face and face-to-face speech perception
Identifieur interne : 000D43 ( Main/Exploration ); précédent : 000D42; suivant : 000D44The sound of your lips: electrophysiological cross-modal interactions during hand-to-face and face-to-face speech perception
Auteurs : Avril Treille ; Coriandre Vilain ; Marc SatoSource :
- Frontiers in Psychology [ 1664-1078 ] ; 2014.
Abstract
Recent magneto-encephalographic and electro-encephalographic studies provide evidence for cross-modal integration during audio-visual and audio-haptic speech perception, with speech gestures viewed or felt from manual tactile contact with the speaker’s face. Given the temporal precedence of the haptic and visual signals on the acoustic signal in these studies, the observed modulation of N1/P2 auditory evoked responses during bimodal compared to unimodal speech perception suggest that relevant and predictive visual and haptic cues may facilitate auditory speech processing. To further investigate this hypothesis, auditory evoked potentials were here compared during auditory-only, audio-visual and audio-haptic speech perception in live dyadic interactions between a listener and a speaker. In line with previous studies, auditory evoked potentials were attenuated and speeded up during both audio-haptic and audio-visual compared to auditory speech perception. Importantly, the observed latency and amplitude reduction did not significantly depend on the degree of visual and haptic recognition of the speech targets. Altogether, these results further demonstrate cross-modal interactions between the auditory, visual and haptic speech signals. Although they do not contradict the hypothesis that visual and haptic sensory inputs convey predictive information with respect to the incoming auditory speech input, these results suggest that, at least in live conversational interactions, systematic conclusions on sensory predictability in bimodal speech integration have to be taken with caution, with the extraction of predictive cues likely depending on the variability of the speech stimuli.
Url:
DOI: 10.3389/fpsyg.2014.00420
PubMed: 24860533
PubMed Central: 4026678
Affiliations:
Links toward previous steps (curation, corpus...)
- to stream Pmc, to step Corpus: 001F07
- to stream Pmc, to step Curation: 001F07
- to stream Pmc, to step Checkpoint: 000871
- to stream PubMed, to step Corpus: 000632
- to stream PubMed, to step Curation: 000632
- to stream PubMed, to step Checkpoint: 000504
- to stream Ncbi, to step Merge: 003024
- to stream Ncbi, to step Curation: 003024
- to stream Ncbi, to step Checkpoint: 003024
- to stream Hal, to step Corpus: 000524
- to stream Hal, to step Curation: 000524
- to stream Hal, to step Checkpoint: 000097
- to stream Main, to step Merge: 000D42
- to stream Main, to step Curation: 000D43
Le document en format XML
<record><TEI><teiHeader><fileDesc><titleStmt><title xml:lang="en">The sound of your lips: electrophysiological cross-modal interactions during hand-to-face and face-to-face speech perception</title>
<author><name sortKey="Treille, Avril" sort="Treille, Avril" uniqKey="Treille A" first="Avril" last="Treille">Avril Treille</name>
</author>
<author><name sortKey="Vilain, Coriandre" sort="Vilain, Coriandre" uniqKey="Vilain C" first="Coriandre" last="Vilain">Coriandre Vilain</name>
</author>
<author><name sortKey="Sato, Marc" sort="Sato, Marc" uniqKey="Sato M" first="Marc" last="Sato">Marc Sato</name>
</author>
</titleStmt>
<publicationStmt><idno type="wicri:source">PMC</idno>
<idno type="pmid">24860533</idno>
<idno type="pmc">4026678</idno>
<idno type="url">http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4026678</idno>
<idno type="RBID">PMC:4026678</idno>
<idno type="doi">10.3389/fpsyg.2014.00420</idno>
<date when="2014">2014</date>
<idno type="wicri:Area/Pmc/Corpus">001F07</idno>
<idno type="wicri:Area/Pmc/Curation">001F07</idno>
<idno type="wicri:Area/Pmc/Checkpoint">000871</idno>
<idno type="wicri:source">PubMed</idno>
<idno type="wicri:Area/PubMed/Corpus">000632</idno>
<idno type="wicri:Area/PubMed/Curation">000632</idno>
<idno type="wicri:Area/PubMed/Checkpoint">000504</idno>
<idno type="wicri:Area/Ncbi/Merge">003024</idno>
<idno type="wicri:Area/Ncbi/Curation">003024</idno>
<idno type="wicri:Area/Ncbi/Checkpoint">003024</idno>
<idno type="wicri:source">HAL</idno>
<idno type="RBID">Hal:hal-00982092</idno>
<idno type="url">https://hal.archives-ouvertes.fr/hal-00982092</idno>
<idno type="wicri:Area/Hal/Corpus">000524</idno>
<idno type="wicri:Area/Hal/Curation">000524</idno>
<idno type="wicri:Area/Hal/Checkpoint">000097</idno>
<idno type="wicri:doubleKey">1664-1078:2014:Treille A:the:sound:of</idno>
<idno type="wicri:Area/Main/Merge">000D42</idno>
<idno type="wicri:Area/Main/Curation">000D43</idno>
<idno type="wicri:Area/Main/Exploration">000D43</idno>
</publicationStmt>
<sourceDesc><biblStruct><analytic><title xml:lang="en" level="a" type="main">The sound of your lips: electrophysiological cross-modal interactions during hand-to-face and face-to-face speech perception</title>
<author><name sortKey="Treille, Avril" sort="Treille, Avril" uniqKey="Treille A" first="Avril" last="Treille">Avril Treille</name>
</author>
<author><name sortKey="Vilain, Coriandre" sort="Vilain, Coriandre" uniqKey="Vilain C" first="Coriandre" last="Vilain">Coriandre Vilain</name>
</author>
<author><name sortKey="Sato, Marc" sort="Sato, Marc" uniqKey="Sato M" first="Marc" last="Sato">Marc Sato</name>
</author>
</analytic>
<series><title level="j">Frontiers in Psychology</title>
<idno type="eISSN">1664-1078</idno>
<imprint><date when="2014">2014</date>
</imprint>
</series>
</biblStruct>
</sourceDesc>
</fileDesc>
<profileDesc><textClass></textClass>
</profileDesc>
</teiHeader>
<front><div type="abstract" xml:lang="en"><p>Recent magneto-encephalographic and electro-encephalographic studies provide evidence for cross-modal integration during audio-visual and audio-haptic speech perception, with speech gestures viewed or felt from manual tactile contact with the speaker’s face. Given the temporal precedence of the haptic and visual signals on the acoustic signal in these studies, the observed modulation of N1/P2 auditory evoked responses during bimodal compared to unimodal speech perception suggest that relevant and predictive visual and haptic cues may facilitate auditory speech processing. To further investigate this hypothesis, auditory evoked potentials were here compared during auditory-only, audio-visual and audio-haptic speech perception in live dyadic interactions between a listener and a speaker. In line with previous studies, auditory evoked potentials were attenuated and speeded up during both audio-haptic and audio-visual compared to auditory speech perception. Importantly, the observed latency and amplitude reduction did not significantly depend on the degree of visual and haptic recognition of the speech targets. Altogether, these results further demonstrate cross-modal interactions between the auditory, visual and haptic speech signals. Although they do not contradict the hypothesis that visual and haptic sensory inputs convey predictive information with respect to the incoming auditory speech input, these results suggest that, at least in live conversational interactions, systematic conclusions on sensory predictability in bimodal speech integration have to be taken with caution, with the extraction of predictive cues likely depending on the variability of the speech stimuli.</p>
</div>
</front>
<back><div1 type="bibliography"><listBibl><biblStruct><analytic><author><name sortKey="Alcorn, S" uniqKey="Alcorn S">S. Alcorn</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Arnal, L H" uniqKey="Arnal L">L. H. Arnal</name>
</author>
<author><name sortKey="Giraud, A L" uniqKey="Giraud A">A. L. Giraud</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Arnal, L H" uniqKey="Arnal L">L. H. Arnal</name>
</author>
<author><name sortKey="Morillon, B" uniqKey="Morillon B">B. Morillon</name>
</author>
<author><name sortKey="Kell, C A" uniqKey="Kell C">C. A. Kell</name>
</author>
<author><name sortKey="Giraud, A L" uniqKey="Giraud A">A. L. Giraud</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Baart, M" uniqKey="Baart M">M. Baart</name>
</author>
<author><name sortKey="Stekelenburg, J J" uniqKey="Stekelenburg J">J. J. Stekelenburg</name>
</author>
<author><name sortKey="Vroomen, J" uniqKey="Vroomen J">J. Vroomen</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Benoit, C" uniqKey="Benoit C">C. Benoît</name>
</author>
<author><name sortKey="Mohamadi, T" uniqKey="Mohamadi T">T. Mohamadi</name>
</author>
<author><name sortKey="Kandel, S D" uniqKey="Kandel S">S. D. Kandel</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Besle, J" uniqKey="Besle J">J. Besle</name>
</author>
<author><name sortKey="Fort, A" uniqKey="Fort A">A. Fort</name>
</author>
<author><name sortKey="Delpuech, C" uniqKey="Delpuech C">C. Delpuech</name>
</author>
<author><name sortKey="Giard, M H" uniqKey="Giard M">M. H. Giard</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Boersma, P" uniqKey="Boersma P">P. Boersma</name>
</author>
<author><name sortKey="Weenink, D" uniqKey="Weenink D">D. Weenink</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Campbell, C S" uniqKey="Campbell C">C. S. Campbell</name>
</author>
<author><name sortKey="Massaro, D W" uniqKey="Massaro D">D. W. Massaro</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Chandrasekaran, C" uniqKey="Chandrasekaran C">C. Chandrasekaran</name>
</author>
<author><name sortKey="Trubanova, A" uniqKey="Trubanova A">A. Trubanova</name>
</author>
<author><name sortKey="Stillittano, S" uniqKey="Stillittano S">S. Stillittano</name>
</author>
<author><name sortKey="Caplier, A" uniqKey="Caplier A">A. Caplier</name>
</author>
<author><name sortKey="Ghazanfar, A A" uniqKey="Ghazanfar A">A. A. Ghazanfar</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Delorme, A" uniqKey="Delorme A">A. Delorme</name>
</author>
<author><name sortKey="Makeig, S" uniqKey="Makeig S">S. Makeig</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Fowler, C" uniqKey="Fowler C">C. Fowler</name>
</author>
<author><name sortKey="Dekle, D" uniqKey="Dekle D">D. Dekle</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Gick, B" uniqKey="Gick B">B. Gick</name>
</author>
<author><name sortKey="J Hannsd Ttir, K M" uniqKey="J Hannsd Ttir K">K. M. Jóhannsdóttir</name>
</author>
<author><name sortKey="Gibraiel, D" uniqKey="Gibraiel D">D Gibraiel</name>
</author>
<author><name sortKey="Muhlbauer, M" uniqKey="Muhlbauer M">M. Mühlbauer</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Grant, K" uniqKey="Grant K">K. Grant</name>
</author>
<author><name sortKey="Walden, B E" uniqKey="Walden B">B. E. Walden</name>
</author>
<author><name sortKey="Seitz, P F" uniqKey="Seitz P">P. F. Seitz</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Green, K P" uniqKey="Green K">K. P. Green</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Hertrich, I" uniqKey="Hertrich I">I. Hertrich</name>
</author>
<author><name sortKey="Mathiak, K" uniqKey="Mathiak K">K. Mathiak</name>
</author>
<author><name sortKey="Lutzenberger, W" uniqKey="Lutzenberger W">W. Lutzenberger</name>
</author>
<author><name sortKey="Menning, H" uniqKey="Menning H">H. Menning</name>
</author>
<author><name sortKey="Ackermann, H" uniqKey="Ackermann H">H. Ackermann</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Jones, J A" uniqKey="Jones J">J. A. Jones</name>
</author>
<author><name sortKey="Munhall, K G" uniqKey="Munhall K">K. G. Munhall</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Klucharev, V" uniqKey="Klucharev V">V. Klucharev</name>
</author>
<author><name sortKey="Mottonen, R" uniqKey="Mottonen R">R. Möttönen</name>
</author>
<author><name sortKey="Sams, M" uniqKey="Sams M">M. Sams</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Lebib, R" uniqKey="Lebib R">R. Lebib</name>
</author>
<author><name sortKey="Papo, D" uniqKey="Papo D">D. Papo</name>
</author>
<author><name sortKey="De Bode, S" uniqKey="De Bode S">S de Bode</name>
</author>
<author><name sortKey="Baudonniere, P M" uniqKey="Baudonniere P">P. M. Baudonnière</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Mcgurk, H" uniqKey="Mcgurk H">H. McGurk</name>
</author>
<author><name sortKey="Macdonald, J" uniqKey="Macdonald J">J. MacDonald</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="N T Nen, R" uniqKey="N T Nen R">R. Näätänen</name>
</author>
<author><name sortKey="Picton, T W" uniqKey="Picton T">T. W. Picton</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Navarra, J" uniqKey="Navarra J">J. Navarra</name>
</author>
<author><name sortKey="Soto Faraco, S" uniqKey="Soto Faraco S">S. Soto-Faraco</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Norton, S J" uniqKey="Norton S">S. J. Norton</name>
</author>
<author><name sortKey="Schultz, M C" uniqKey="Schultz M">M. C. Schultz</name>
</author>
<author><name sortKey="Reed, C M" uniqKey="Reed C">C. M. Reed</name>
</author>
<author><name sortKey="Braida, L D" uniqKey="Braida L">L. D. Braida</name>
</author>
<author><name sortKey="Durlach, N I" uniqKey="Durlach N">N. I. Durlach</name>
</author>
<author><name sortKey="Rabinowitz, W M" uniqKey="Rabinowitz W">W. M. Rabinowitz</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Pilling, M" uniqKey="Pilling M">M. Pilling</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Reisberg, D" uniqKey="Reisberg D">D. Reisberg</name>
</author>
<author><name sortKey="Mclean, J" uniqKey="Mclean J">J. McLean</name>
</author>
<author><name sortKey="Goldfield, A" uniqKey="Goldfield A">A. Goldfield</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Sams, M" uniqKey="Sams M">M. Sams</name>
</author>
<author><name sortKey="Aulanko, R" uniqKey="Aulanko R">R. Aulanko</name>
</author>
<author><name sortKey="H M L Inen, M" uniqKey="H M L Inen M">M. Hämäläinen</name>
</author>
<author><name sortKey="Hari, R" uniqKey="Hari R">R. Hari</name>
</author>
<author><name sortKey="Lounasmaa, O V" uniqKey="Lounasmaa O">O. V. Lounasmaa</name>
</author>
<author><name sortKey="Lu, S T" uniqKey="Lu S">S. T. Lu</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Sato, M" uniqKey="Sato M">M. Sato</name>
</author>
<author><name sortKey="Cave, C" uniqKey="Cave C">C. Cavé</name>
</author>
<author><name sortKey="Menard, L" uniqKey="Menard L">L. Ménard</name>
</author>
<author><name sortKey="Brasseur, L" uniqKey="Brasseur L">L. Brasseur</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Scherg, M" uniqKey="Scherg M">M Scherg</name>
</author>
<author><name sortKey="Von Cramon, D" uniqKey="Von Cramon D">D. Von Cramon</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Schwartz, J L" uniqKey="Schwartz J">J. L. Schwartz</name>
</author>
<author><name sortKey="Berthommier, F" uniqKey="Berthommier F">F. Berthommier</name>
</author>
<author><name sortKey="Savariaux, C" uniqKey="Savariaux C">C. Savariaux</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Schwartz, J L" uniqKey="Schwartz J">J. L. Schwartz</name>
</author>
<author><name sortKey="Menard, L" uniqKey="Menard L">L. Ménard</name>
</author>
<author><name sortKey="Basirat, A" uniqKey="Basirat A">A. Basirat</name>
</author>
<author><name sortKey="Sato, M" uniqKey="Sato M">M. Sato</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Schwartz, J L" uniqKey="Schwartz J">J. L. Schwartz</name>
</author>
<author><name sortKey="Savariaux, C" uniqKey="Savariaux C">C. Savariaux</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Stein, B E" uniqKey="Stein B">B. E. Stein</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Stein, B E" uniqKey="Stein B">B. E. Stein</name>
</author>
<author><name sortKey="Meredith, M A" uniqKey="Meredith M">M. A. Meredith</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Stekelenburg, J J" uniqKey="Stekelenburg J">J. J. Stekelenburg</name>
</author>
<author><name sortKey="Vroomen, J" uniqKey="Vroomen J">J. Vroomen</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Sumby, W H" uniqKey="Sumby W">W. H. Sumby</name>
</author>
<author><name sortKey="Pollack, I" uniqKey="Pollack I">I. Pollack</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Treille, A" uniqKey="Treille A">A. Treille</name>
</author>
<author><name sortKey="Cordeboeuf, C" uniqKey="Cordeboeuf C">C. Cordeboeuf</name>
</author>
<author><name sortKey="Vilain, C" uniqKey="Vilain C">C. Vilain</name>
</author>
<author><name sortKey="Sato, M" uniqKey="Sato M">M. Sato</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Van Wassenhove, V" uniqKey="Van Wassenhove V">V. van Wassenhove</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Van Wassenhove, V" uniqKey="Van Wassenhove V">V. van Wassenhove</name>
</author>
<author><name sortKey="Grant, K W" uniqKey="Grant K">K. W. Grant</name>
</author>
<author><name sortKey="Poeppel, D" uniqKey="Poeppel D">D. Poeppel</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Van Wassenhove, V" uniqKey="Van Wassenhove V">V. van Wassenhove</name>
</author>
<author><name sortKey="Grant, K W" uniqKey="Grant K">K. W. Grant</name>
</author>
<author><name sortKey="Poeppel, D" uniqKey="Poeppel D">D. Poeppel</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Vroomen, J" uniqKey="Vroomen J">J. Vroomen</name>
</author>
<author><name sortKey="Stekelenburg, J J" uniqKey="Stekelenburg J">J. J. Stekelenburg</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Winneke, A H" uniqKey="Winneke A">A. H. Winneke</name>
</author>
<author><name sortKey="Phillips, N A" uniqKey="Phillips N">N. A. Phillips</name>
</author>
</analytic>
</biblStruct>
</listBibl>
</div1>
</back>
</TEI>
<affiliations><list></list>
<tree><noCountry><name sortKey="Sato, Marc" sort="Sato, Marc" uniqKey="Sato M" first="Marc" last="Sato">Marc Sato</name>
<name sortKey="Treille, Avril" sort="Treille, Avril" uniqKey="Treille A" first="Avril" last="Treille">Avril Treille</name>
<name sortKey="Vilain, Coriandre" sort="Vilain, Coriandre" uniqKey="Vilain C" first="Coriandre" last="Vilain">Coriandre Vilain</name>
</noCountry>
</tree>
</affiliations>
</record>
Pour manipuler ce document sous Unix (Dilib)
EXPLOR_STEP=$WICRI_ROOT/Ticri/CIDE/explor/HapticV1/Data/Main/Exploration
HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 000D43 | SxmlIndent | more
Ou
HfdSelect -h $EXPLOR_AREA/Data/Main/Exploration/biblio.hfd -nk 000D43 | SxmlIndent | more
Pour mettre un lien sur cette page dans le réseau Wicri
{{Explor lien |wiki= Ticri/CIDE |area= HapticV1 |flux= Main |étape= Exploration |type= RBID |clé= PMC:4026678 |texte= The sound of your lips: electrophysiological cross-modal interactions during hand-to-face and face-to-face speech perception }}
Pour générer des pages wiki
HfdIndexSelect -h $EXPLOR_AREA/Data/Main/Exploration/RBID.i -Sk "pubmed:24860533" \ | HfdSelect -Kh $EXPLOR_AREA/Data/Main/Exploration/biblio.hfd \ | NlmPubMed2Wicri -a HapticV1
![]() | This area was generated with Dilib version V0.6.23. | ![]() |