Inferring Relevance in a Changing World
Identifieur interne : 002389 ( Main/Merge ); précédent : 002388; suivant : 002390Inferring Relevance in a Changing World
Auteurs : Robert C. Wilson [États-Unis] ; Yael Niv [États-Unis]Source :
- Frontiers in Human Neuroscience [ 1662-5161 ] ; 2012.
Abstract
Reinforcement learning models of human and animal learning usually concentrate on how we learn the relationship between different stimuli or actions and rewards. However, in real-world situations “stimuli” are ill-defined. On the one hand, our immediate environment is extremely multidimensional. On the other hand, in every decision making scenario only a few aspects of the environment are relevant for obtaining reward, while most are irrelevant. Thus a key question is how do we learn these relevant dimensions, that is, how do we learn what to learn about? We investigated this process of “representation learning” experimentally, using a task in which one stimulus dimension was relevant for determining reward at each point in time. As in real life situations, in our task the relevant dimension can change without warning, adding ever-present uncertainty engendered by a constantly changing environment. We show that human performance on this task is better described by a suboptimal strategy based on selective attention and serial-hypothesis-testing rather than a normative strategy based on probabilistic inference. From this, we conjecture that the problem of inferring relevance in general scenarios is too computationally demanding for the brain to solve optimally. As a result the brain utilizes approximations, employing these even in simplified scenarios in which optimal representation learning is tractable, such as the one in our experiment.
Url:
DOI: 10.3389/fnhum.2011.00189
PubMed: 22291631
PubMed Central: 3264902
Links toward previous steps (curation, corpus...)
- to stream Pmc, to step Corpus: 001E07
- to stream Pmc, to step Curation: 001E07
- to stream Pmc, to step Checkpoint: 001711
- to stream Ncbi, to step Merge: 001E34
- to stream Ncbi, to step Curation: 001E34
- to stream Ncbi, to step Checkpoint: 001E34
Links to Exploration step
PMC:3264902Le document en format XML
<record><TEI><teiHeader><fileDesc><titleStmt><title xml:lang="en">Inferring Relevance in a Changing World</title>
<author><name sortKey="Wilson, Robert C" sort="Wilson, Robert C" uniqKey="Wilson R" first="Robert C." last="Wilson">Robert C. Wilson</name>
<affiliation wicri:level="1"><nlm:aff id="aff1"><institution>Department of Psychology, Neuroscience Institute, Princeton University</institution>
<country>Princeton, NJ, USA</country>
</nlm:aff>
<country xml:lang="fr">États-Unis</country>
<wicri:regionArea></wicri:regionArea>
<wicri:regionArea># see nlm:aff region in country</wicri:regionArea>
</affiliation>
</author>
<author><name sortKey="Niv, Yael" sort="Niv, Yael" uniqKey="Niv Y" first="Yael" last="Niv">Yael Niv</name>
<affiliation wicri:level="1"><nlm:aff id="aff1"><institution>Department of Psychology, Neuroscience Institute, Princeton University</institution>
<country>Princeton, NJ, USA</country>
</nlm:aff>
<country xml:lang="fr">États-Unis</country>
<wicri:regionArea></wicri:regionArea>
<wicri:regionArea># see nlm:aff region in country</wicri:regionArea>
</affiliation>
</author>
</titleStmt>
<publicationStmt><idno type="wicri:source">PMC</idno>
<idno type="pmid">22291631</idno>
<idno type="pmc">3264902</idno>
<idno type="url">http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3264902</idno>
<idno type="RBID">PMC:3264902</idno>
<idno type="doi">10.3389/fnhum.2011.00189</idno>
<date when="2012">2012</date>
<idno type="wicri:Area/Pmc/Corpus">001E07</idno>
<idno type="wicri:Area/Pmc/Curation">001E07</idno>
<idno type="wicri:Area/Pmc/Checkpoint">001711</idno>
<idno type="wicri:Area/Ncbi/Merge">001E34</idno>
<idno type="wicri:Area/Ncbi/Curation">001E34</idno>
<idno type="wicri:Area/Ncbi/Checkpoint">001E34</idno>
<idno type="wicri:Area/Main/Merge">002389</idno>
</publicationStmt>
<sourceDesc><biblStruct><analytic><title xml:lang="en" level="a" type="main">Inferring Relevance in a Changing World</title>
<author><name sortKey="Wilson, Robert C" sort="Wilson, Robert C" uniqKey="Wilson R" first="Robert C." last="Wilson">Robert C. Wilson</name>
<affiliation wicri:level="1"><nlm:aff id="aff1"><institution>Department of Psychology, Neuroscience Institute, Princeton University</institution>
<country>Princeton, NJ, USA</country>
</nlm:aff>
<country xml:lang="fr">États-Unis</country>
<wicri:regionArea></wicri:regionArea>
<wicri:regionArea># see nlm:aff region in country</wicri:regionArea>
</affiliation>
</author>
<author><name sortKey="Niv, Yael" sort="Niv, Yael" uniqKey="Niv Y" first="Yael" last="Niv">Yael Niv</name>
<affiliation wicri:level="1"><nlm:aff id="aff1"><institution>Department of Psychology, Neuroscience Institute, Princeton University</institution>
<country>Princeton, NJ, USA</country>
</nlm:aff>
<country xml:lang="fr">États-Unis</country>
<wicri:regionArea></wicri:regionArea>
<wicri:regionArea># see nlm:aff region in country</wicri:regionArea>
</affiliation>
</author>
</analytic>
<series><title level="j">Frontiers in Human Neuroscience</title>
<idno type="eISSN">1662-5161</idno>
<imprint><date when="2012">2012</date>
</imprint>
</series>
</biblStruct>
</sourceDesc>
</fileDesc>
<profileDesc><textClass></textClass>
</profileDesc>
</teiHeader>
<front><div type="abstract" xml:lang="en"><p>Reinforcement learning models of human and animal learning usually concentrate on how we learn the relationship between different stimuli or actions and rewards. However, in real-world situations “stimuli” are ill-defined. On the one hand, our immediate environment is extremely multidimensional. On the other hand, in every decision making scenario only a few aspects of the environment are relevant for obtaining reward, while most are irrelevant. Thus a key question is how do we learn these relevant dimensions, that is, how do we learn what to learn about? We investigated this process of “representation learning” experimentally, using a task in which one stimulus dimension was relevant for determining reward at each point in time. As in real life situations, in our task the relevant dimension can change without warning, adding ever-present uncertainty engendered by a constantly changing environment. We show that human performance on this task is better described by a suboptimal strategy based on selective attention and serial-hypothesis-testing rather than a normative strategy based on probabilistic inference. From this, we conjecture that the problem of inferring relevance in general scenarios is too computationally demanding for the brain to solve optimally. As a result the brain utilizes approximations, employing these even in simplified scenarios in which optimal representation learning is tractable, such as the one in our experiment.</p>
</div>
</front>
<back><div1 type="bibliography"><listBibl><biblStruct><analytic><author><name sortKey="Adams, R P" uniqKey="Adams R">R. P. Adams</name>
</author>
<author><name sortKey="Mackay, D J" uniqKey="Mackay D">D. J. MacKay</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Alais, D" uniqKey="Alais D">D. Alais</name>
</author>
<author><name sortKey="Burr, D" uniqKey="Burr D">D. Burr</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Bahrami, B" uniqKey="Bahrami B">B. Bahrami</name>
</author>
<author><name sortKey="Olsen, K" uniqKey="Olsen K">K. Olsen</name>
</author>
<author><name sortKey="Latham, P E" uniqKey="Latham P">P. E. Latham</name>
</author>
<author><name sortKey="Roepstorff, A" uniqKey="Roepstorff A">A. Roepstorff</name>
</author>
<author><name sortKey="Rees, G" uniqKey="Rees G">G. Rees</name>
</author>
<author><name sortKey="Frith, C D" uniqKey="Frith C">C. D. Frith</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Barry, D" uniqKey="Barry D">D. Barry</name>
</author>
<author><name sortKey="Hartigan, J A" uniqKey="Hartigan J">J. A. Hartigan</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Behrens, T E J" uniqKey="Behrens T">T. E. J. Behrens</name>
</author>
<author><name sortKey="Woolrich, M W" uniqKey="Woolrich M">M. W. Woolrich</name>
</author>
<author><name sortKey="Walton, M E" uniqKey="Walton M">M. E. Walton</name>
</author>
<author><name sortKey="Rushworth, M F S" uniqKey="Rushworth M">M. F. S. Rushworth</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Braun, D A" uniqKey="Braun D">D. A. Braun</name>
</author>
<author><name sortKey="Mehring, C" uniqKey="Mehring C">C. Mehring</name>
</author>
<author><name sortKey="Wolpert, D M" uniqKey="Wolpert D">D. M. Wolpert</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Broadbent, D" uniqKey="Broadbent D">D. Broadbent</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Brown, S D" uniqKey="Brown S">S. D. Brown</name>
</author>
<author><name sortKey="Steyvers, M" uniqKey="Steyvers M">M. Steyvers</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Bundesen, C" uniqKey="Bundesen C">C. Bundesen</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Daw, N D" uniqKey="Daw N">N. D. Daw</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Daw, N D" uniqKey="Daw N">N. D. Daw</name>
</author>
<author><name sortKey="Courville, A C" uniqKey="Courville A">A. C. Courville</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Dayan, P" uniqKey="Dayan P">P. Dayan</name>
</author>
<author><name sortKey="Kakade, S" uniqKey="Kakade S">S. Kakade</name>
</author>
<author><name sortKey="Montague, P R" uniqKey="Montague P">P. R. Montague</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Dehaene, S" uniqKey="Dehaene S">S. Dehaene</name>
</author>
<author><name sortKey="Changeux, J P" uniqKey="Changeux J">J. P. Changeux</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Ernst, M O" uniqKey="Ernst M">M. O. Ernst</name>
</author>
<author><name sortKey="Banks, M S" uniqKey="Banks M">M. S. Banks</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Fearnhead, P" uniqKey="Fearnhead P">P. Fearnhead</name>
</author>
<author><name sortKey="Liu, Z" uniqKey="Liu Z">Z. Liu</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Gershman, S J" uniqKey="Gershman S">S. J. Gershman</name>
</author>
<author><name sortKey="Cohen, J D" uniqKey="Cohen J">J. D. Cohen</name>
</author>
<author><name sortKey="Niv, Y" uniqKey="Niv Y">Y. Niv</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Kahneman, D" uniqKey="Kahneman D">D. Kahneman</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Kass, R" uniqKey="Kass R">R. Kass</name>
</author>
<author><name sortKey="Raftery, A" uniqKey="Raftery A">A. Raftery</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Kemp, C" uniqKey="Kemp C">C. Kemp</name>
</author>
<author><name sortKey="Tenenbaum, J B" uniqKey="Tenenbaum J">J. B. Tenenbaum</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Kording, K P" uniqKey="Kording K">K. P. Kording</name>
</author>
<author><name sortKey="Wolpert, D M" uniqKey="Wolpert D">D. M. Wolpert</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Krechevsky, I" uniqKey="Krechevsky I">I. Krechevsky</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Lashley, K" uniqKey="Lashley K">K. Lashley</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Mackintosh, N" uniqKey="Mackintosh N">N. Mackintosh</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Milner, B" uniqKey="Milner B">B. Milner</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Munuera, J" uniqKey="Munuera J">J. Munuera</name>
</author>
<author><name sortKey="Morel, P" uniqKey="Morel P">P. Morel</name>
</author>
<author><name sortKey="Duhamel, J R" uniqKey="Duhamel J">J. R. Duhamel</name>
</author>
<author><name sortKey="Deneve, S" uniqKey="Deneve S">S. Deneve</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Nassar, M R" uniqKey="Nassar M">M. R. Nassar</name>
</author>
<author><name sortKey="Wilson, R C" uniqKey="Wilson R">R. C. Wilson</name>
</author>
<author><name sortKey="Heasly, B" uniqKey="Heasly B">B. Heasly</name>
</author>
<author><name sortKey="Gold, J I" uniqKey="Gold J">J. I. Gold</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Niv, Y" uniqKey="Niv Y">Y. Niv</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Steyvers, M" uniqKey="Steyvers M">M. Steyvers</name>
</author>
<author><name sortKey="Lee, M D" uniqKey="Lee M">M. D. Lee</name>
</author>
<author><name sortKey="Wagenmakers, E J" uniqKey="Wagenmakers E">E. J. Wagenmakers</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Steyvers, M" uniqKey="Steyvers M">M. Steyvers</name>
</author>
<author><name sortKey="Tenenbaum, J B" uniqKey="Tenenbaum J">J. B. Tenenbaum</name>
</author>
<author><name sortKey="Wagenmakers, E J" uniqKey="Wagenmakers E">E. J. Wagenmakers</name>
</author>
<author><name sortKey="Blum, B" uniqKey="Blum B">B. Blum</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Sutherland, N S" uniqKey="Sutherland N">N. S. Sutherland</name>
</author>
<author><name sortKey="Mackintosh, J" uniqKey="Mackintosh J">J. Mackintosh</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Sutton, R S" uniqKey="Sutton R">R. S. Sutton</name>
</author>
<author><name sortKey="Barto, A G" uniqKey="Barto A">A. G. Barto</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Sutton, R S" uniqKey="Sutton R">R. S. Sutton</name>
</author>
<author><name sortKey="Barto, A G" uniqKey="Barto A">A. G. Barto</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Wilder, M" uniqKey="Wilder M">M. Wilder</name>
</author>
<author><name sortKey="Jones, M" uniqKey="Jones M">M. Jones</name>
</author>
<author><name sortKey="Mozer, M" uniqKey="Mozer M">M. Mozer</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Wilson, R C" uniqKey="Wilson R">R. C. Wilson</name>
</author>
<author><name sortKey="Nassar, M R" uniqKey="Nassar M">M. R. Nassar</name>
</author>
<author><name sortKey="Gold, J I" uniqKey="Gold J">J. I. Gold</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Wunderlich, K" uniqKey="Wunderlich K">K. Wunderlich</name>
</author>
<author><name sortKey="Beierholm, U R" uniqKey="Beierholm U">U. R. Beierholm</name>
</author>
<author><name sortKey="Bossaerts, P" uniqKey="Bossaerts P">P. Bossaerts</name>
</author>
<author><name sortKey="O Oherty, J P" uniqKey="O Oherty J">J. P. O’Doherty</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Yu, A J" uniqKey="Yu A">A. J. Yu</name>
</author>
<author><name sortKey="Cohen, J D" uniqKey="Cohen J">J. D. Cohen</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Yu, A J" uniqKey="Yu A">A. J. Yu</name>
</author>
<author><name sortKey="Dayan, P" uniqKey="Dayan P">P. Dayan</name>
</author>
</analytic>
</biblStruct>
</listBibl>
</div1>
</back>
</TEI>
</record>
Pour manipuler ce document sous Unix (Dilib)
EXPLOR_STEP=$WICRI_ROOT/Ticri/CIDE/explor/HapticV1/Data/Main/Merge
HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 002389 | SxmlIndent | more
Ou
HfdSelect -h $EXPLOR_AREA/Data/Main/Merge/biblio.hfd -nk 002389 | SxmlIndent | more
Pour mettre un lien sur cette page dans le réseau Wicri
{{Explor lien |wiki= Ticri/CIDE |area= HapticV1 |flux= Main |étape= Merge |type= RBID |clé= PMC:3264902 |texte= Inferring Relevance in a Changing World }}
Pour générer des pages wiki
HfdIndexSelect -h $EXPLOR_AREA/Data/Main/Merge/RBID.i -Sk "pubmed:22291631" \ | HfdSelect -Kh $EXPLOR_AREA/Data/Main/Merge/biblio.hfd \ | NlmPubMed2Wicri -a HapticV1
![]() | This area was generated with Dilib version V0.6.23. | ![]() |