Multimodal perceptual organization of speech: Evidence from tone analogs of spoken utterances
Identifieur interne : 008524 ( Main/Exploration ); précédent : 008523; suivant : 008525Multimodal perceptual organization of speech: Evidence from tone analogs of spoken utterances
Auteurs : Robert E. Remez [États-Unis] ; Jennifer M. Fellowes [États-Unis] ; David B. Pisoni [États-Unis] ; Winston D. Goh [États-Unis] ; Philip E. Rubin [États-Unis]Source :
- Speech Communication [ 0167-6393 ] ; 1998.
Abstract
Theoretical and practical motives alike have prompted recent investigations of multimodal speech perception. Theoretically, multimodal studies have extended the conceptualization of perceptual organization beyond the familiar modality-bound accounts deriving from Gestalt psychology. Practically, such investigations have been driven by a need to understand the proficiency of multimodal speech perception using an electrocochlear prosthesis for hearing. In each domain, studies have shown that perceptual organization of speech can occur even when the perceiver's auditory experience departs from natural speech qualities. Accordingly, our research examined auditor-visual multimodal integration of videotaped faces and selected acoustic constituents of speech signals, each realized as a single sinewave tone accompanying a video image of an articulating face. The single tone reproduced the frequency and amplitude of the phonatory cycle or of one of the lower three oral formants. Our results showed a distinct advantage for the condition pairing the video image of the face with a sinewave replicating the second formant, despite its unnatural timbre and its presentation in acoustic isolation from the rest of the speech signal. Perceptual coherence of multimodal speech in these circumstances is established when the two modalities concurrently specify the same underlying phonetic attributes.
Url:
- https://api.istex.fr/document/B53758DA350391BEA639015B1DC43F7038D7078B/fulltext/pdf
- http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3060793
DOI: 10.1016/S0167-6393(98)00050-8
Affiliations:
Links toward previous steps (curation, corpus...)
- to stream Istex, to step Corpus: 001540
- to stream Istex, to step Curation: 001540
- to stream Istex, to step Checkpoint: 003714
- to stream Pmc, to step Corpus: 001004
- to stream Pmc, to step Curation: 001004
- to stream Pmc, to step Checkpoint: 002683
- to stream Ncbi, to step Merge: 001954
- to stream Ncbi, to step Curation: 001954
- to stream Ncbi, to step Checkpoint: 001954
- to stream Main, to step Merge: 008D31
- to stream Main, to step Curation: 008524
Le document en format XML
<record><TEI wicri:istexFullTextTei="biblStruct"><teiHeader><fileDesc><titleStmt><title>Multimodal perceptual organization of speech: Evidence from tone analogs of spoken utterances</title>
<author><name sortKey="Remez, Robert E" sort="Remez, Robert E" uniqKey="Remez R" first="Robert E" last="Remez">Robert E. Remez</name>
</author>
<author><name sortKey="Fellowes, Jennifer M" sort="Fellowes, Jennifer M" uniqKey="Fellowes J" first="Jennifer M" last="Fellowes">Jennifer M. Fellowes</name>
</author>
<author><name sortKey="Pisoni, David B" sort="Pisoni, David B" uniqKey="Pisoni D" first="David B" last="Pisoni">David B. Pisoni</name>
</author>
<author><name sortKey="Goh, Winston D" sort="Goh, Winston D" uniqKey="Goh W" first="Winston D" last="Goh">Winston D. Goh</name>
</author>
<author><name sortKey="Rubin, Philip E" sort="Rubin, Philip E" uniqKey="Rubin P" first="Philip E" last="Rubin">Philip E. Rubin</name>
</author>
</titleStmt>
<publicationStmt><idno type="wicri:source">ISTEX</idno>
<idno type="RBID">ISTEX:B53758DA350391BEA639015B1DC43F7038D7078B</idno>
<date when="1998" year="1998">1998</date>
<idno type="doi">10.1016/S0167-6393(98)00050-8</idno>
<idno type="url">https://api.istex.fr/document/B53758DA350391BEA639015B1DC43F7038D7078B/fulltext/pdf</idno>
<idno type="wicri:Area/Istex/Corpus">001540</idno>
<idno type="wicri:Area/Istex/Curation">001540</idno>
<idno type="wicri:Area/Istex/Checkpoint">003714</idno>
<idno type="wicri:doubleKey">0167-6393:1998:Remez R:multimodal:perceptual:organization</idno>
<idno type="wicri:source">PMC</idno>
<idno type="url">http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3060793</idno>
<idno type="RBID">PMC:3060793</idno>
<idno type="wicri:Area/Pmc/Corpus">001004</idno>
<idno type="wicri:Area/Pmc/Curation">001004</idno>
<idno type="wicri:Area/Pmc/Checkpoint">002683</idno>
<idno type="wicri:Area/Ncbi/Merge">001954</idno>
<idno type="wicri:Area/Ncbi/Curation">001954</idno>
<idno type="wicri:Area/Ncbi/Checkpoint">001954</idno>
<idno type="wicri:doubleKey">0167-6393:1998:Remez R:multimodal:perceptual:organization</idno>
<idno type="wicri:Area/Main/Merge">008D31</idno>
<idno type="wicri:Area/Main/Curation">008524</idno>
<idno type="wicri:Area/Main/Exploration">008524</idno>
</publicationStmt>
<sourceDesc><biblStruct><analytic><title level="a">Multimodal perceptual organization of speech: Evidence from tone analogs of spoken utterances</title>
<author><name sortKey="Remez, Robert E" sort="Remez, Robert E" uniqKey="Remez R" first="Robert E" last="Remez">Robert E. Remez</name>
<affiliation wicri:level="2"><country xml:lang="fr">États-Unis</country>
<wicri:regionArea>Department of Psychology, Barnard College, 3009 Broadway, New York, NY 10027-6598</wicri:regionArea>
<placeName><region type="state">État de New York</region>
</placeName>
</affiliation>
<affiliation wicri:level="1"><country wicri:rule="url">États-Unis</country>
</affiliation>
</author>
<author><name sortKey="Fellowes, Jennifer M" sort="Fellowes, Jennifer M" uniqKey="Fellowes J" first="Jennifer M" last="Fellowes">Jennifer M. Fellowes</name>
<affiliation wicri:level="2"><country xml:lang="fr">États-Unis</country>
<wicri:regionArea>Columbia University College of Physicians and Surgeons, 630 West 168th Street, New York, NY 10032</wicri:regionArea>
<placeName><region type="state">État de New York</region>
</placeName>
</affiliation>
</author>
<author><name sortKey="Pisoni, David B" sort="Pisoni, David B" uniqKey="Pisoni D" first="David B" last="Pisoni">David B. Pisoni</name>
<affiliation wicri:level="2"><country xml:lang="fr">États-Unis</country>
<wicri:regionArea>Speech Research Laboratory, Department of Psychology, Indiana University, Bloomington, IN 47405</wicri:regionArea>
<placeName><region type="state">Indiana</region>
</placeName>
</affiliation>
</author>
<author><name sortKey="Goh, Winston D" sort="Goh, Winston D" uniqKey="Goh W" first="Winston D" last="Goh">Winston D. Goh</name>
<affiliation wicri:level="2"><country xml:lang="fr">États-Unis</country>
<wicri:regionArea>Speech Research Laboratory, Department of Psychology, Indiana University, Bloomington, IN 47405</wicri:regionArea>
<placeName><region type="state">Indiana</region>
</placeName>
</affiliation>
</author>
<author><name sortKey="Rubin, Philip E" sort="Rubin, Philip E" uniqKey="Rubin P" first="Philip E" last="Rubin">Philip E. Rubin</name>
<affiliation wicri:level="2"><country xml:lang="fr">États-Unis</country>
<wicri:regionArea>Haskins Laboratories and Department of Surgery, Yale University School of Medicine, 270 Crown Street, New Haven, CT 06511</wicri:regionArea>
<placeName><region type="state">Connecticut</region>
</placeName>
</affiliation>
</author>
</analytic>
<monogr></monogr>
<series><title level="j">Speech Communication</title>
<title level="j" type="abbrev">SPECOM</title>
<idno type="ISSN">0167-6393</idno>
<imprint><publisher>ELSEVIER</publisher>
<date type="published" when="1998">1998</date>
<biblScope unit="volume">26</biblScope>
<biblScope unit="issue">1–2</biblScope>
<biblScope unit="page" from="65">65</biblScope>
<biblScope unit="page" to="73">73</biblScope>
</imprint>
<idno type="ISSN">0167-6393</idno>
</series>
<idno type="istex">B53758DA350391BEA639015B1DC43F7038D7078B</idno>
<idno type="DOI">10.1016/S0167-6393(98)00050-8</idno>
<idno type="PII">S0167-6393(98)00050-8</idno>
</biblStruct>
</sourceDesc>
<seriesStmt><idno type="ISSN">0167-6393</idno>
</seriesStmt>
</fileDesc>
<profileDesc><textClass></textClass>
<langUsage><language ident="en">en</language>
</langUsage>
</profileDesc>
</teiHeader>
<front><div type="abstract" xml:lang="en">Theoretical and practical motives alike have prompted recent investigations of multimodal speech perception. Theoretically, multimodal studies have extended the conceptualization of perceptual organization beyond the familiar modality-bound accounts deriving from Gestalt psychology. Practically, such investigations have been driven by a need to understand the proficiency of multimodal speech perception using an electrocochlear prosthesis for hearing. In each domain, studies have shown that perceptual organization of speech can occur even when the perceiver's auditory experience departs from natural speech qualities. Accordingly, our research examined auditor-visual multimodal integration of videotaped faces and selected acoustic constituents of speech signals, each realized as a single sinewave tone accompanying a video image of an articulating face. The single tone reproduced the frequency and amplitude of the phonatory cycle or of one of the lower three oral formants. Our results showed a distinct advantage for the condition pairing the video image of the face with a sinewave replicating the second formant, despite its unnatural timbre and its presentation in acoustic isolation from the rest of the speech signal. Perceptual coherence of multimodal speech in these circumstances is established when the two modalities concurrently specify the same underlying phonetic attributes.</div>
</front>
</TEI>
<affiliations><list><country><li>États-Unis</li>
</country>
<region><li>Connecticut</li>
<li>Indiana</li>
<li>État de New York</li>
</region>
</list>
<tree><country name="États-Unis"><region name="État de New York"><name sortKey="Remez, Robert E" sort="Remez, Robert E" uniqKey="Remez R" first="Robert E" last="Remez">Robert E. Remez</name>
</region>
<name sortKey="Fellowes, Jennifer M" sort="Fellowes, Jennifer M" uniqKey="Fellowes J" first="Jennifer M" last="Fellowes">Jennifer M. Fellowes</name>
<name sortKey="Goh, Winston D" sort="Goh, Winston D" uniqKey="Goh W" first="Winston D" last="Goh">Winston D. Goh</name>
<name sortKey="Pisoni, David B" sort="Pisoni, David B" uniqKey="Pisoni D" first="David B" last="Pisoni">David B. Pisoni</name>
<name sortKey="Remez, Robert E" sort="Remez, Robert E" uniqKey="Remez R" first="Robert E" last="Remez">Robert E. Remez</name>
<name sortKey="Rubin, Philip E" sort="Rubin, Philip E" uniqKey="Rubin P" first="Philip E" last="Rubin">Philip E. Rubin</name>
</country>
</tree>
</affiliations>
</record>
Pour manipuler ce document sous Unix (Dilib)
EXPLOR_STEP=$WICRI_ROOT/Ticri/CIDE/explor/HapticV1/Data/Main/Exploration
HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 008524 | SxmlIndent | more
Ou
HfdSelect -h $EXPLOR_AREA/Data/Main/Exploration/biblio.hfd -nk 008524 | SxmlIndent | more
Pour mettre un lien sur cette page dans le réseau Wicri
{{Explor lien |wiki= Ticri/CIDE |area= HapticV1 |flux= Main |étape= Exploration |type= RBID |clé= ISTEX:B53758DA350391BEA639015B1DC43F7038D7078B |texte= Multimodal perceptual organization of speech: Evidence from tone analogs of spoken utterances }}
![]() | This area was generated with Dilib version V0.6.23. | ![]() |