Cue Integration in Categorical Tasks: Insights from Audio-Visual Speech Perception
Identifieur interne : 001A71 ( Ncbi/Checkpoint ); précédent : 001A70; suivant : 001A72Cue Integration in Categorical Tasks: Insights from Audio-Visual Speech Perception
Auteurs : Vikranth Rao Bejjanki ; Meghan Clayards ; David C. Knill ; Richard N. AslinSource :
- PLoS ONE [ 1932-6203 ] ; 2011.
Abstract
Previous cue integration studies have examined continuous perceptual dimensions (e.g., size) and have shown that human cue integration is well described by a normative model in which cues are weighted in proportion to their sensory reliability, as estimated from single-cue performance. However, this normative model may not be applicable to categorical perceptual dimensions (e.g., phonemes). In tasks defined over categorical perceptual dimensions, optimal cue weights should depend not only on the sensory variance affecting the perception of each cue but also on the environmental variance inherent in each task-relevant category. Here, we present a computational and experimental investigation of cue integration in a categorical audio-visual (articulatory) speech perception task. Our results show that human performance during audio-visual phonemic labeling is qualitatively consistent with the behavior of a Bayes-optimal observer. Specifically, we show that the participants in our task are sensitive, on a trial-by-trial basis, to the sensory uncertainty associated with the auditory and visual cues, during phonemic categorization. In addition, we show that while sensory uncertainty is a significant factor in determining cue weights, it is not the only one and participants' performance is consistent with an optimal model in which environmental, within category variability also plays a role in determining cue weights. Furthermore, we show that in our task, the sensory variability affecting the visual modality during cue-combination is not well estimated from single-cue performance, but can be estimated from multi-cue performance. The findings and computational principles described here represent a principled first step towards characterizing the mechanisms underlying human cue integration in categorical tasks.
Url:
DOI: 10.1371/journal.pone.0019812
PubMed: 21637344
PubMed Central: 3102664
Affiliations:
Links toward previous steps (curation, corpus...)
- to stream Pmc, to step Corpus: 002407
- to stream Pmc, to step Curation: 002407
- to stream Pmc, to step Checkpoint: 001B98
- to stream Ncbi, to step Merge: 001A71
- to stream Ncbi, to step Curation: 001A71
Links to Exploration step
PMC:3102664Le document en format XML
<record><TEI><teiHeader><fileDesc><titleStmt><title xml:lang="en">Cue Integration in Categorical Tasks: Insights from Audio-Visual Speech Perception</title>
<author><name sortKey="Bejjanki, Vikranth Rao" sort="Bejjanki, Vikranth Rao" uniqKey="Bejjanki V" first="Vikranth Rao" last="Bejjanki">Vikranth Rao Bejjanki</name>
<affiliation><nlm:aff id="aff1"></nlm:aff>
</affiliation>
</author>
<author><name sortKey="Clayards, Meghan" sort="Clayards, Meghan" uniqKey="Clayards M" first="Meghan" last="Clayards">Meghan Clayards</name>
<affiliation><nlm:aff id="aff1"></nlm:aff>
</affiliation>
</author>
<author><name sortKey="Knill, David C" sort="Knill, David C" uniqKey="Knill D" first="David C." last="Knill">David C. Knill</name>
<affiliation><nlm:aff id="aff1"></nlm:aff>
</affiliation>
</author>
<author><name sortKey="Aslin, Richard N" sort="Aslin, Richard N" uniqKey="Aslin R" first="Richard N." last="Aslin">Richard N. Aslin</name>
<affiliation><nlm:aff id="aff1"></nlm:aff>
</affiliation>
</author>
</titleStmt>
<publicationStmt><idno type="wicri:source">PMC</idno>
<idno type="pmid">21637344</idno>
<idno type="pmc">3102664</idno>
<idno type="url">http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3102664</idno>
<idno type="RBID">PMC:3102664</idno>
<idno type="doi">10.1371/journal.pone.0019812</idno>
<date when="2011">2011</date>
<idno type="wicri:Area/Pmc/Corpus">002407</idno>
<idno type="wicri:Area/Pmc/Curation">002407</idno>
<idno type="wicri:Area/Pmc/Checkpoint">001B98</idno>
<idno type="wicri:Area/Ncbi/Merge">001A71</idno>
<idno type="wicri:Area/Ncbi/Curation">001A71</idno>
<idno type="wicri:Area/Ncbi/Checkpoint">001A71</idno>
</publicationStmt>
<sourceDesc><biblStruct><analytic><title xml:lang="en" level="a" type="main">Cue Integration in Categorical Tasks: Insights from Audio-Visual Speech Perception</title>
<author><name sortKey="Bejjanki, Vikranth Rao" sort="Bejjanki, Vikranth Rao" uniqKey="Bejjanki V" first="Vikranth Rao" last="Bejjanki">Vikranth Rao Bejjanki</name>
<affiliation><nlm:aff id="aff1"></nlm:aff>
</affiliation>
</author>
<author><name sortKey="Clayards, Meghan" sort="Clayards, Meghan" uniqKey="Clayards M" first="Meghan" last="Clayards">Meghan Clayards</name>
<affiliation><nlm:aff id="aff1"></nlm:aff>
</affiliation>
</author>
<author><name sortKey="Knill, David C" sort="Knill, David C" uniqKey="Knill D" first="David C." last="Knill">David C. Knill</name>
<affiliation><nlm:aff id="aff1"></nlm:aff>
</affiliation>
</author>
<author><name sortKey="Aslin, Richard N" sort="Aslin, Richard N" uniqKey="Aslin R" first="Richard N." last="Aslin">Richard N. Aslin</name>
<affiliation><nlm:aff id="aff1"></nlm:aff>
</affiliation>
</author>
</analytic>
<series><title level="j">PLoS ONE</title>
<idno type="eISSN">1932-6203</idno>
<imprint><date when="2011">2011</date>
</imprint>
</series>
</biblStruct>
</sourceDesc>
</fileDesc>
<profileDesc><textClass></textClass>
</profileDesc>
</teiHeader>
<front><div type="abstract" xml:lang="en"><p>Previous cue integration studies have examined continuous perceptual dimensions (e.g., size) and have shown that human cue integration is well described by a normative model in which cues are weighted in proportion to their sensory reliability, as estimated from single-cue performance. However, this normative model may not be applicable to categorical perceptual dimensions (e.g., phonemes). In tasks defined over categorical perceptual dimensions, optimal cue weights should depend not only on the sensory variance affecting the perception of each cue but also on the environmental variance inherent in each task-relevant category. Here, we present a computational and experimental investigation of cue integration in a categorical audio-visual (articulatory) speech perception task. Our results show that human performance during audio-visual phonemic labeling is qualitatively consistent with the behavior of a Bayes-optimal observer. Specifically, we show that the participants in our task are sensitive, on a trial-by-trial basis, to the sensory uncertainty associated with the auditory and visual cues, during phonemic categorization. In addition, we show that while sensory uncertainty is a significant factor in determining cue weights, it is not the only one and participants' performance is consistent with an optimal model in which environmental, within category variability also plays a role in determining cue weights. Furthermore, we show that in our task, the sensory variability affecting the visual modality during cue-combination is not well estimated from single-cue performance, but can be estimated from multi-cue performance. The findings and computational principles described here represent a principled first step towards characterizing the mechanisms underlying human cue integration in categorical tasks.</p>
</div>
</front>
<back><div1 type="bibliography"><listBibl><biblStruct><analytic><author><name sortKey="Ernst, Mo" uniqKey="Ernst M">MO Ernst</name>
</author>
<author><name sortKey="Bulthoff, Hh" uniqKey="Bulthoff H">HH Bülthoff</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Jacobs, Ra" uniqKey="Jacobs R">RA Jacobs</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Campbell, R" uniqKey="Campbell R">R Campbell</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Ma, Wj" uniqKey="Ma W">WJ Ma</name>
</author>
<author><name sortKey="Zhou, X" uniqKey="Zhou X">X Zhou</name>
</author>
<author><name sortKey="Ross, La" uniqKey="Ross L">LA Ross</name>
</author>
<author><name sortKey="Foxe, Jj" uniqKey="Foxe J">JJ Foxe</name>
</author>
<author><name sortKey="Parra, Lc" uniqKey="Parra L">LC Parra</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Massaro, Dw" uniqKey="Massaro D">DW Massaro</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Massaro, Dw" uniqKey="Massaro D">DW Massaro</name>
</author>
<author><name sortKey="Cohen, Mm" uniqKey="Cohen M">MM Cohen</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Mcgurk, H" uniqKey="Mcgurk H">H McGurk</name>
</author>
<author><name sortKey="Macdonald, J" uniqKey="Macdonald J">J MacDonald</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Ross, La" uniqKey="Ross L">LA Ross</name>
</author>
<author><name sortKey="Saint Amour, D" uniqKey="Saint Amour D">D Saint-Amour</name>
</author>
<author><name sortKey="Leavitt, Vm" uniqKey="Leavitt V">VM Leavitt</name>
</author>
<author><name sortKey="Javitt, Dc" uniqKey="Javitt D">DC Javitt</name>
</author>
<author><name sortKey="Foxe, Jj" uniqKey="Foxe J">JJ Foxe</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Sumby, Wh" uniqKey="Sumby W">WH Sumby</name>
</author>
<author><name sortKey="Pollack, I" uniqKey="Pollack I">I Pollack</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Summerfield, Q" uniqKey="Summerfield Q">Q Summerfield</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Desjardins, Rn" uniqKey="Desjardins R">RN Desjardins</name>
</author>
<author><name sortKey="Rogers, J" uniqKey="Rogers J">J Rogers</name>
</author>
<author><name sortKey="Werker, Jf" uniqKey="Werker J">JF Werker</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Erber, Np" uniqKey="Erber N">NP Erber</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Massaro, Dw" uniqKey="Massaro D">DW Massaro</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Massaro, Dw" uniqKey="Massaro D">DW Massaro</name>
</author>
<author><name sortKey="Cohen, Mm" uniqKey="Cohen M">MM Cohen</name>
</author>
<author><name sortKey="Gesi, A" uniqKey="Gesi A">A Gesi</name>
</author>
<author><name sortKey="Heredia, R" uniqKey="Heredia R">R Heredia</name>
</author>
<author><name sortKey="Tsuzaki, M" uniqKey="Tsuzaki M">M Tsuzaki</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Sekiyama, K" uniqKey="Sekiyama K">K Sekiyama</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Sekiyama, K" uniqKey="Sekiyama K">K Sekiyama</name>
</author>
<author><name sortKey="Burnham, D" uniqKey="Burnham D">D Burnham</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Desjardins, Rn" uniqKey="Desjardins R">RN Desjardins</name>
</author>
<author><name sortKey="Werker, Jf" uniqKey="Werker J">JF Werker</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Rosenblum, Ld" uniqKey="Rosenblum L">LD Rosenblum</name>
</author>
<author><name sortKey="Schmuckler, Ma" uniqKey="Schmuckler M">MA Schmuckler</name>
</author>
<author><name sortKey="Johnson, Ja" uniqKey="Johnson J">JA Johnson</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Teinonen, T" uniqKey="Teinonen T">T Teinonen</name>
</author>
<author><name sortKey="Aslin, Rn" uniqKey="Aslin R">RN Aslin</name>
</author>
<author><name sortKey="Alku, P" uniqKey="Alku P">P Alku</name>
</author>
<author><name sortKey="Csibra, G" uniqKey="Csibra G">G Csibra</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Ernst, Mo" uniqKey="Ernst M">MO Ernst</name>
</author>
<author><name sortKey="Banks, Ms" uniqKey="Banks M">MS Banks</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Knill, Dc" uniqKey="Knill D">DC Knill</name>
</author>
<author><name sortKey="Pouget, A" uniqKey="Pouget A">A Pouget</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Yuille, Al" uniqKey="Yuille A">AL Yuille</name>
</author>
<author><name sortKey="Bulthoff, Hh" uniqKey="Bulthoff H">HH Bulthoff</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Landy, Ms" uniqKey="Landy M">MS Landy</name>
</author>
<author><name sortKey="Maloney, Lt" uniqKey="Maloney L">LT Maloney</name>
</author>
<author><name sortKey="Johnston, Eb" uniqKey="Johnston E">EB Johnston</name>
</author>
<author><name sortKey="Young, M" uniqKey="Young M">M Young</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Alais, D" uniqKey="Alais D">D Alais</name>
</author>
<author><name sortKey="Burr, D" uniqKey="Burr D">D Burr</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Battaglia, Pw" uniqKey="Battaglia P">PW Battaglia</name>
</author>
<author><name sortKey="Jacobs, Ra" uniqKey="Jacobs R">RA Jacobs</name>
</author>
<author><name sortKey="Aslin, Rn" uniqKey="Aslin R">RN Aslin</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Hillis, Jm" uniqKey="Hillis J">JM Hillis</name>
</author>
<author><name sortKey="Ernst, Mo" uniqKey="Ernst M">MO Ernst</name>
</author>
<author><name sortKey="Banks, Ms" uniqKey="Banks M">MS Banks</name>
</author>
<author><name sortKey="Landy, Ms" uniqKey="Landy M">MS Landy</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Jacobs, Ra" uniqKey="Jacobs R">RA Jacobs</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Knill, Dc" uniqKey="Knill D">DC Knill</name>
</author>
<author><name sortKey="Saunders, Ja" uniqKey="Saunders J">JA Saunders</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Kording, Kp" uniqKey="Kording K">KP Kording</name>
</author>
<author><name sortKey="Wolpert, Dm" uniqKey="Wolpert D">DM Wolpert</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Van Beers, Rj" uniqKey="Van Beers R">RJ van Beers</name>
</author>
<author><name sortKey="Sittig, Ac" uniqKey="Sittig A">AC Sittig</name>
</author>
<author><name sortKey="Gon, Jjdvd" uniqKey="Gon J">JJDvd Gon</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Wolpert, Dm" uniqKey="Wolpert D">DM Wolpert</name>
</author>
<author><name sortKey="Ghahramani, Z" uniqKey="Ghahramani Z">Z Ghahramani</name>
</author>
<author><name sortKey="Jordan, Mi" uniqKey="Jordan M">MI Jordan</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Feldman, Nh" uniqKey="Feldman N">NH Feldman</name>
</author>
<author><name sortKey="Griffiths, Tl" uniqKey="Griffiths T">TL Griffiths</name>
</author>
<author><name sortKey="Morgan, Jl" uniqKey="Morgan J">JL Morgan</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Papoulis, A" uniqKey="Papoulis A">A Papoulis</name>
</author>
<author><name sortKey="Pillai, Su" uniqKey="Pillai S">SU Pillai</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Klatt, Dh" uniqKey="Klatt D">DH Klatt</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Cohen, Mm" uniqKey="Cohen M">MM Cohen</name>
</author>
<author><name sortKey="Massaro, Dw" uniqKey="Massaro D">DW Massaro</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Remez, Re" uniqKey="Remez R">RE Remez</name>
</author>
<author><name sortKey="Pardo, Js" uniqKey="Pardo J">JS Pardo</name>
</author>
<author><name sortKey="Piorkowski, Rl" uniqKey="Piorkowski R">RL Piorkowski</name>
</author>
<author><name sortKey="Rubin, Pe" uniqKey="Rubin P">PE Rubin</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Massaro, Dw" uniqKey="Massaro D">DW Massaro</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Swanson, Wh" uniqKey="Swanson W">WH Swanson</name>
</author>
<author><name sortKey="Birch, Ee" uniqKey="Birch E">EE Birch</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Kording, Kp" uniqKey="Kording K">KP Körding</name>
</author>
<author><name sortKey="Beierholm, U" uniqKey="Beierholm U">U Beierholm</name>
</author>
<author><name sortKey="Ma, Wj" uniqKey="Ma W">WJ Ma</name>
</author>
<author><name sortKey="Quartz, S" uniqKey="Quartz S">S Quartz</name>
</author>
<author><name sortKey="Tenenbaum, Jb" uniqKey="Tenenbaum J">JB Tenenbaum</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Shams, L" uniqKey="Shams L">L Shams</name>
</author>
<author><name sortKey="Ma, Wj" uniqKey="Ma W">WJ Ma</name>
</author>
<author><name sortKey="Beierholm, U" uniqKey="Beierholm U">U Beierholm</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Wallace, Mt" uniqKey="Wallace M">MT Wallace</name>
</author>
<author><name sortKey="Roberson, Ge" uniqKey="Roberson G">GE Roberson</name>
</author>
<author><name sortKey="Hairston, Wd" uniqKey="Hairston W">WD Hairston</name>
</author>
<author><name sortKey="Stein, Be" uniqKey="Stein B">BE Stein</name>
</author>
<author><name sortKey="Vaughan, Jw" uniqKey="Vaughan J">JW Vaughan</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Banks, Ms" uniqKey="Banks M">MS Banks</name>
</author>
<author><name sortKey="Backus, Bt" uniqKey="Backus B">BT Backus</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Knill, Dc" uniqKey="Knill D">DC Knill</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Peterson, Ge" uniqKey="Peterson G">GE Peterson</name>
</author>
<author><name sortKey="Barney, Hl" uniqKey="Barney H">HL Barney</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Schwartz, J L" uniqKey="Schwartz J">J-L Schwartz</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Schwartz, J L" uniqKey="Schwartz J">J-L Schwartz</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Holt, Ll" uniqKey="Holt L">LL Holt</name>
</author>
<author><name sortKey="Lotto, Aj" uniqKey="Lotto A">AJ Lotto</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Clayards, M" uniqKey="Clayards M">M Clayards</name>
</author>
<author><name sortKey="Tanenhaus, Mk" uniqKey="Tanenhaus M">MK Tanenhaus</name>
</author>
<author><name sortKey="Aslin, Rn" uniqKey="Aslin R">RN Aslin</name>
</author>
<author><name sortKey="Jacobs, Ra" uniqKey="Jacobs R">RA Jacobs</name>
</author>
</analytic>
</biblStruct>
</listBibl>
</div1>
</back>
</TEI>
<affiliations><list></list>
<tree><noCountry><name sortKey="Aslin, Richard N" sort="Aslin, Richard N" uniqKey="Aslin R" first="Richard N." last="Aslin">Richard N. Aslin</name>
<name sortKey="Bejjanki, Vikranth Rao" sort="Bejjanki, Vikranth Rao" uniqKey="Bejjanki V" first="Vikranth Rao" last="Bejjanki">Vikranth Rao Bejjanki</name>
<name sortKey="Clayards, Meghan" sort="Clayards, Meghan" uniqKey="Clayards M" first="Meghan" last="Clayards">Meghan Clayards</name>
<name sortKey="Knill, David C" sort="Knill, David C" uniqKey="Knill D" first="David C." last="Knill">David C. Knill</name>
</noCountry>
</tree>
</affiliations>
</record>
Pour manipuler ce document sous Unix (Dilib)
EXPLOR_STEP=$WICRI_ROOT/Ticri/CIDE/explor/HapticV1/Data/Ncbi/Checkpoint
HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 001A71 | SxmlIndent | more
Ou
HfdSelect -h $EXPLOR_AREA/Data/Ncbi/Checkpoint/biblio.hfd -nk 001A71 | SxmlIndent | more
Pour mettre un lien sur cette page dans le réseau Wicri
{{Explor lien |wiki= Ticri/CIDE |area= HapticV1 |flux= Ncbi |étape= Checkpoint |type= RBID |clé= PMC:3102664 |texte= Cue Integration in Categorical Tasks: Insights from Audio-Visual Speech Perception }}
Pour générer des pages wiki
HfdIndexSelect -h $EXPLOR_AREA/Data/Ncbi/Checkpoint/RBID.i -Sk "pubmed:21637344" \ | HfdSelect -Kh $EXPLOR_AREA/Data/Ncbi/Checkpoint/biblio.hfd \ | NlmPubMed2Wicri -a HapticV1
This area was generated with Dilib version V0.6.23. |