A Comprehensive Reinforcement Learning Framework for Dialogue Management Optimisation
Identifieur interne : 001835 ( Main/Exploration ); précédent : 001834; suivant : 001836A Comprehensive Reinforcement Learning Framework for Dialogue Management Optimisation
Auteurs : Lucie Daubigney [France] ; Matthieu Geist [France] ; Senthilkumar Chandramohan [France] ; Olivier Pietquin [France]Source :
Abstract
Reinforcement learning is now an acknowledged approach for optimising the interaction strategy of spoken dialogue systems. If the first considered algorithms were quite basic (like SARSA), recent works concentrated on more sophisticated methods. More attention has been paid to off-policy learning, dealing with the exploration-exploitation dilemma, sample efficiency or handling non-stationarity. New algorithms have been proposed to address these issues and have been applied to dialogue management. However, each algorithm often solves a single issue at a time, while dialogue systems exhibit all the problems at once. In this paper, we propose to apply the Kalman Temporal Differences (KTD) framework to the problem of dialogue strategy optimisation so as to address all these issues in a comprehensive manner with a single framework. Our claims are illustrated by experiments led on two real-world goal-oriented dialogue management frameworks, DIPPER and HIS.
Url:
DOI: 10.1109/JSTSP.2012.2229257
Affiliations:
- France
- Franche-Comté, Grand Est, Lorraine (région)
- Besançon, Metz, Nancy
- Université Paul Verlaine - Metz, Université de Bourgogne Franche-Comté, Université de Franche-Comté, Université de Lorraine
Links toward previous steps (curation, corpus...)
- to stream Hal, to step Corpus: 000125
- to stream Hal, to step Curation: 000125
- to stream Hal, to step Checkpoint: 001449
- to stream Main, to step Merge: 001862
- to stream Main, to step Curation: 001835
Le document en format XML
<record><TEI><teiHeader><fileDesc><titleStmt><title xml:lang="en">A Comprehensive Reinforcement Learning Framework for Dialogue Management Optimisation</title><author><name sortKey="Daubigney, Lucie" sort="Daubigney, Lucie" uniqKey="Daubigney L" first="Lucie" last="Daubigney">Lucie Daubigney</name><affiliation wicri:level="1"><hal:affiliation type="researchteam" xml:id="struct-205124" status="OLD"><idno type="RNSR">200218290B</idno><orgName>Autonomous intelligent machine</orgName><orgName type="acronym">MAIA</orgName><date type="end">2014-12-31</date><desc><address><country key="FR"></country></address><ref type="url">http://www.inria.fr/equipes/maia</ref></desc><listRelation><relation active="#struct-129671" type="direct"></relation><relation active="#struct-300009" type="indirect"></relation><relation active="#struct-423090" type="direct"></relation><relation active="#struct-206040" type="indirect"></relation><relation active="#struct-413289" type="indirect"></relation><relation name="UMR7503" active="#struct-441569" type="indirect"></relation></listRelation><tutelles><tutelle active="#struct-129671" type="direct"><org type="laboratory" xml:id="struct-129671" status="VALID"><idno type="RNSR">198618246Y</idno><orgName>INRIA Nancy - Grand Est</orgName><desc><address><addrLine>615 rue du Jardin Botanique 54600 Villers-lès-Nancy</addrLine><country key="FR"></country></address><ref type="url">http://www.inria.fr/nancy</ref></desc><listRelation><relation active="#struct-300009" type="direct"></relation></listRelation></org></tutelle><tutelle active="#struct-300009" type="indirect"><org type="institution" xml:id="struct-300009" status="VALID"><orgName>Institut National de Recherche en Informatique et en Automatique</orgName><orgName type="acronym">Inria</orgName><desc><address><addrLine>Domaine de VoluceauRocquencourt - BP 10578153 Le Chesnay Cedex</addrLine><country key="FR"></country></address><ref type="url">http://www.inria.fr/en/</ref></desc></org></tutelle><tutelle active="#struct-423090" type="direct"><org type="department" xml:id="struct-423090" status="VALID"><orgName>Department of Complex Systems, Artificial Intelligence & Robotics</orgName><orgName type="acronym">LORIA - AIS</orgName><desc><address><country key="FR"></country></address><ref type="url">http://www.loria.fr/la-recherche-en/departements/complex-system-and-artificial-intelligence</ref></desc><listRelation><relation active="#struct-206040" type="direct"></relation><relation active="#struct-300009" type="indirect"></relation><relation active="#struct-413289" type="indirect"></relation><relation name="UMR7503" active="#struct-441569" type="indirect"></relation></listRelation></org></tutelle><tutelle active="#struct-206040" type="indirect"><org type="laboratory" xml:id="struct-206040" status="VALID"><idno type="IdRef">067077927</idno><idno type="RNSR">198912571S</idno><idno type="IdUnivLorraine">[UL]RSI--</idno><orgName>Laboratoire Lorrain de Recherche en Informatique et ses Applications</orgName><orgName type="acronym">LORIA</orgName><date type="start">2012-01-01</date><desc><address><addrLine>Campus Scientifique BP 239 54506 Vandoeuvre-lès-Nancy Cedex</addrLine><country key="FR"></country></address><ref type="url">http://www.loria.fr</ref></desc><listRelation><relation active="#struct-300009" type="direct"></relation><relation active="#struct-413289" type="direct"></relation><relation name="UMR7503" active="#struct-441569" type="direct"></relation></listRelation></org></tutelle><tutelle active="#struct-413289" type="indirect"><org type="institution" xml:id="struct-413289" status="VALID"><idno type="IdRef">157040569</idno><idno type="IdUnivLorraine">[UL]100--</idno><orgName>Université de Lorraine</orgName><orgName type="acronym">UL</orgName><date type="start">2012-01-01</date><desc><address><addrLine>34 cours Léopold - CS 25233 - 54052 Nancy cedex</addrLine><country key="FR"></country></address><ref type="url">http://www.univ-lorraine.fr/</ref></desc></org></tutelle><tutelle name="UMR7503" active="#struct-441569" type="indirect"><org type="institution" xml:id="struct-441569" status="VALID"><idno type="IdRef">02636817X</idno><idno type="ISNI">0000000122597504</idno><orgName>Centre National de la Recherche Scientifique</orgName><orgName type="acronym">CNRS</orgName><date type="start">1939-10-19</date><desc><address><country key="FR"></country></address><ref type="url">http://www.cnrs.fr/</ref></desc></org></tutelle></tutelles></hal:affiliation><country>France</country><placeName><settlement type="city">Nancy</settlement><settlement type="city">Metz</settlement><region type="region" nuts="2">Grand Est</region><region type="old region" nuts="2">Lorraine (région)</region></placeName><orgName type="university">Université de Lorraine</orgName></affiliation></author><author><name sortKey="Geist, Matthieu" sort="Geist, Matthieu" uniqKey="Geist M" first="Matthieu" last="Geist">Matthieu Geist</name><affiliation wicri:level="1"><hal:affiliation type="researchteam" xml:id="struct-394500" status="INCOMING"><orgName>IMS - Equipe Information, Multimodalité et Signal</orgName><desc><address><country key="FR"></country></address></desc><listRelation><relation active="#struct-24541" type="direct"></relation><relation active="#struct-242365" type="indirect"></relation><relation active="#struct-411575" type="indirect"></relation><relation active="#struct-301991" type="indirect"></relation><relation active="#struct-301990" type="indirect"></relation><relation active="#struct-300812" type="indirect"></relation><relation active="#struct-300413" type="indirect"></relation><relation active="#struct-300289" type="indirect"></relation><relation name="UMI2958" active="#struct-441569" type="indirect"></relation><relation active="#struct-26305" type="direct"></relation></listRelation><tutelles><tutelle active="#struct-24541" type="direct"><org type="laboratory" xml:id="struct-24541" status="VALID"><idno type="RNSR">200619366D</idno><orgName>Georgia Tech - CNRS [Metz]</orgName><orgName type="acronym">UMI2958</orgName><desc><address><addrLine>Metz Technopôle 2-3 rue Marconi 57070 METZ</addrLine><country key="FR"></country></address><ref type="url">http://www.umi2958.eu</ref></desc><listRelation><relation active="#struct-242365" type="direct"></relation><relation active="#struct-411575" type="direct"></relation><relation active="#struct-301991" type="direct"></relation><relation active="#struct-301990" type="direct"></relation><relation active="#struct-300812" type="direct"></relation><relation active="#struct-300413" type="direct"></relation><relation active="#struct-300289" type="direct"></relation><relation name="UMI2958" active="#struct-441569" type="direct"></relation></listRelation></org></tutelle><tutelle active="#struct-242365" type="indirect"><org type="institution" xml:id="struct-242365" status="VALID"><idno type="IdRef">026403188</idno><idno type="ISNI">0000 0001 2188 3779 </idno><orgName>Université de Franche-Comté</orgName><orgName type="acronym">UFC</orgName><desc><address><country key="FR"></country></address><ref type="url">http://www.univ-fcomte.fr</ref></desc></org></tutelle><tutelle active="#struct-411575" type="indirect"><org type="institution" xml:id="struct-411575" status="VALID"><orgName>CentraleSupélec</orgName><desc><address><addrLine>3, rue Joliot Curie,Plateau de Moulon,91192 GIF-SUR-YVETTE Cedex</addrLine><country key="FR"></country></address><ref type="url">http://www.centralesupelec.fr</ref></desc></org></tutelle><tutelle active="#struct-301991" type="indirect"><org type="institution" xml:id="struct-301991" status="VALID"><orgName>Georgia Tech Lorraine</orgName><desc><address><country key="FR"></country></address></desc></org></tutelle><tutelle active="#struct-301990" type="indirect"><org type="institution" xml:id="struct-301990" status="VALID"><orgName>Georgia Institute of Technology [Atlanta]</orgName><desc><address><addrLine>North Avenue, Atlanta, GA 30332</addrLine><country key="US"></country></address><ref type="url">http://www.gatech.edu/</ref></desc></org></tutelle><tutelle active="#struct-300812" type="indirect"><org type="institution" xml:id="struct-300812" status="VALID"><orgName>SUPELEC</orgName><desc><address><country key="FR"></country></address></desc></org></tutelle><tutelle active="#struct-300413" type="indirect"><org type="institution" xml:id="struct-300413" status="VALID"><orgName>Ecole Nationale Supérieure des Arts et Metiers Metz</orgName><desc><address><country key="FR"></country></address></desc></org></tutelle><tutelle active="#struct-300289" type="indirect"><org type="institution" xml:id="struct-300289" status="OLD"><orgName>Université Paul Verlaine - Metz</orgName><orgName type="acronym">UPVM</orgName><date type="end">2011-12-31</date><desc><address><country key="FR"></country></address></desc></org></tutelle><tutelle name="UMI2958" active="#struct-441569" type="indirect"><org type="institution" xml:id="struct-441569" status="VALID"><idno type="IdRef">02636817X</idno><idno type="ISNI">0000000122597504</idno><orgName>Centre National de la Recherche Scientifique</orgName><orgName type="acronym">CNRS</orgName><date type="start">1939-10-19</date><desc><address><country key="FR"></country></address><ref type="url">http://www.cnrs.fr/</ref></desc></org></tutelle><tutelle active="#struct-26305" type="direct"><org type="laboratory" xml:id="struct-26305" status="VALID"><orgName>SUPELEC-Campus Metz</orgName><desc><address><addrLine>2 rue Edouard Belin 57070 Metz</addrLine><country key="FR"></country></address><ref type="url">http://www.metz.supelec.fr/metz/</ref></desc><listRelation><relation active="#struct-300812" type="direct"></relation></listRelation></org></tutelle></tutelles></hal:affiliation><country>France</country><placeName><settlement type="city" wicri:auto="siege">Besançon</settlement><region type="region" nuts="2">Franche-Comté</region></placeName><orgName type="university">Université de Franche-Comté</orgName><orgName type="institution" wicri:auto="newGroup">Université de Bourgogne Franche-Comté</orgName><placeName><settlement type="city">Metz</settlement><region type="region" nuts="2">Grand Est</region><region type="old region" nuts="2">Lorraine (région)</region></placeName><orgName type="university">Université Paul Verlaine - Metz</orgName><orgName type="institution" wicri:auto="newGroup">Université de Lorraine</orgName></affiliation></author><author><name sortKey="Chandramohan, Senthilkumar" sort="Chandramohan, Senthilkumar" uniqKey="Chandramohan S" first="Senthilkumar" last="Chandramohan">Senthilkumar Chandramohan</name><affiliation wicri:level="1"><hal:affiliation type="researchteam" xml:id="struct-394500" status="INCOMING"><orgName>IMS - Equipe Information, Multimodalité et Signal</orgName><desc><address><country key="FR"></country></address></desc><listRelation><relation active="#struct-24541" type="direct"></relation><relation active="#struct-242365" type="indirect"></relation><relation active="#struct-411575" type="indirect"></relation><relation active="#struct-301991" type="indirect"></relation><relation active="#struct-301990" type="indirect"></relation><relation active="#struct-300812" type="indirect"></relation><relation active="#struct-300413" type="indirect"></relation><relation active="#struct-300289" type="indirect"></relation><relation name="UMI2958" active="#struct-441569" type="indirect"></relation><relation active="#struct-26305" type="direct"></relation></listRelation><tutelles><tutelle active="#struct-24541" type="direct"><org type="laboratory" xml:id="struct-24541" status="VALID"><idno type="RNSR">200619366D</idno><orgName>Georgia Tech - CNRS [Metz]</orgName><orgName type="acronym">UMI2958</orgName><desc><address><addrLine>Metz Technopôle 2-3 rue Marconi 57070 METZ</addrLine><country key="FR"></country></address><ref type="url">http://www.umi2958.eu</ref></desc><listRelation><relation active="#struct-242365" type="direct"></relation><relation active="#struct-411575" type="direct"></relation><relation active="#struct-301991" type="direct"></relation><relation active="#struct-301990" type="direct"></relation><relation active="#struct-300812" type="direct"></relation><relation active="#struct-300413" type="direct"></relation><relation active="#struct-300289" type="direct"></relation><relation name="UMI2958" active="#struct-441569" type="direct"></relation></listRelation></org></tutelle><tutelle active="#struct-242365" type="indirect"><org type="institution" xml:id="struct-242365" status="VALID"><idno type="IdRef">026403188</idno><idno type="ISNI">0000 0001 2188 3779 </idno><orgName>Université de Franche-Comté</orgName><orgName type="acronym">UFC</orgName><desc><address><country key="FR"></country></address><ref type="url">http://www.univ-fcomte.fr</ref></desc></org></tutelle><tutelle active="#struct-411575" type="indirect"><org type="institution" xml:id="struct-411575" status="VALID"><orgName>CentraleSupélec</orgName><desc><address><addrLine>3, rue Joliot Curie,Plateau de Moulon,91192 GIF-SUR-YVETTE Cedex</addrLine><country key="FR"></country></address><ref type="url">http://www.centralesupelec.fr</ref></desc></org></tutelle><tutelle active="#struct-301991" type="indirect"><org type="institution" xml:id="struct-301991" status="VALID"><orgName>Georgia Tech Lorraine</orgName><desc><address><country key="FR"></country></address></desc></org></tutelle><tutelle active="#struct-301990" type="indirect"><org type="institution" xml:id="struct-301990" status="VALID"><orgName>Georgia Institute of Technology [Atlanta]</orgName><desc><address><addrLine>North Avenue, Atlanta, GA 30332</addrLine><country key="US"></country></address><ref type="url">http://www.gatech.edu/</ref></desc></org></tutelle><tutelle active="#struct-300812" type="indirect"><org type="institution" xml:id="struct-300812" status="VALID"><orgName>SUPELEC</orgName><desc><address><country key="FR"></country></address></desc></org></tutelle><tutelle active="#struct-300413" type="indirect"><org type="institution" xml:id="struct-300413" status="VALID"><orgName>Ecole Nationale Supérieure des Arts et Metiers Metz</orgName><desc><address><country key="FR"></country></address></desc></org></tutelle><tutelle active="#struct-300289" type="indirect"><org type="institution" xml:id="struct-300289" status="OLD"><orgName>Université Paul Verlaine - Metz</orgName><orgName type="acronym">UPVM</orgName><date type="end">2011-12-31</date><desc><address><country key="FR"></country></address></desc></org></tutelle><tutelle name="UMI2958" active="#struct-441569" type="indirect"><org type="institution" xml:id="struct-441569" status="VALID"><idno type="IdRef">02636817X</idno><idno type="ISNI">0000000122597504</idno><orgName>Centre National de la Recherche Scientifique</orgName><orgName type="acronym">CNRS</orgName><date type="start">1939-10-19</date><desc><address><country key="FR"></country></address><ref type="url">http://www.cnrs.fr/</ref></desc></org></tutelle><tutelle active="#struct-26305" type="direct"><org type="laboratory" xml:id="struct-26305" status="VALID"><orgName>SUPELEC-Campus Metz</orgName><desc><address><addrLine>2 rue Edouard Belin 57070 Metz</addrLine><country key="FR"></country></address><ref type="url">http://www.metz.supelec.fr/metz/</ref></desc><listRelation><relation active="#struct-300812" type="direct"></relation></listRelation></org></tutelle></tutelles></hal:affiliation><country>France</country><placeName><settlement type="city" wicri:auto="siege">Besançon</settlement><region type="region" nuts="2">Franche-Comté</region></placeName><orgName type="university">Université de Franche-Comté</orgName><orgName type="institution" wicri:auto="newGroup">Université de Bourgogne Franche-Comté</orgName><placeName><settlement type="city">Metz</settlement><region type="region" nuts="2">Grand Est</region><region type="old region" nuts="2">Lorraine (région)</region></placeName><orgName type="university">Université Paul Verlaine - Metz</orgName><orgName type="institution" wicri:auto="newGroup">Université de Lorraine</orgName></affiliation></author><author><name sortKey="Pietquin, Olivier" sort="Pietquin, Olivier" uniqKey="Pietquin O" first="Olivier" last="Pietquin">Olivier Pietquin</name><affiliation wicri:level="1"><hal:affiliation type="researchteam" xml:id="struct-394500" status="INCOMING"><orgName>IMS - Equipe Information, Multimodalité et Signal</orgName><desc><address><country key="FR"></country></address></desc><listRelation><relation active="#struct-24541" type="direct"></relation><relation active="#struct-242365" type="indirect"></relation><relation active="#struct-411575" type="indirect"></relation><relation active="#struct-301991" type="indirect"></relation><relation active="#struct-301990" type="indirect"></relation><relation active="#struct-300812" type="indirect"></relation><relation active="#struct-300413" type="indirect"></relation><relation active="#struct-300289" type="indirect"></relation><relation name="UMI2958" active="#struct-441569" type="indirect"></relation><relation active="#struct-26305" type="direct"></relation></listRelation><tutelles><tutelle active="#struct-24541" type="direct"><org type="laboratory" xml:id="struct-24541" status="VALID"><idno type="RNSR">200619366D</idno><orgName>Georgia Tech - CNRS [Metz]</orgName><orgName type="acronym">UMI2958</orgName><desc><address><addrLine>Metz Technopôle 2-3 rue Marconi 57070 METZ</addrLine><country key="FR"></country></address><ref type="url">http://www.umi2958.eu</ref></desc><listRelation><relation active="#struct-242365" type="direct"></relation><relation active="#struct-411575" type="direct"></relation><relation active="#struct-301991" type="direct"></relation><relation active="#struct-301990" type="direct"></relation><relation active="#struct-300812" type="direct"></relation><relation active="#struct-300413" type="direct"></relation><relation active="#struct-300289" type="direct"></relation><relation name="UMI2958" active="#struct-441569" type="direct"></relation></listRelation></org></tutelle><tutelle active="#struct-242365" type="indirect"><org type="institution" xml:id="struct-242365" status="VALID"><idno type="IdRef">026403188</idno><idno type="ISNI">0000 0001 2188 3779 </idno><orgName>Université de Franche-Comté</orgName><orgName type="acronym">UFC</orgName><desc><address><country key="FR"></country></address><ref type="url">http://www.univ-fcomte.fr</ref></desc></org></tutelle><tutelle active="#struct-411575" type="indirect"><org type="institution" xml:id="struct-411575" status="VALID"><orgName>CentraleSupélec</orgName><desc><address><addrLine>3, rue Joliot Curie,Plateau de Moulon,91192 GIF-SUR-YVETTE Cedex</addrLine><country key="FR"></country></address><ref type="url">http://www.centralesupelec.fr</ref></desc></org></tutelle><tutelle active="#struct-301991" type="indirect"><org type="institution" xml:id="struct-301991" status="VALID"><orgName>Georgia Tech Lorraine</orgName><desc><address><country key="FR"></country></address></desc></org></tutelle><tutelle active="#struct-301990" type="indirect"><org type="institution" xml:id="struct-301990" status="VALID"><orgName>Georgia Institute of Technology [Atlanta]</orgName><desc><address><addrLine>North Avenue, Atlanta, GA 30332</addrLine><country key="US"></country></address><ref type="url">http://www.gatech.edu/</ref></desc></org></tutelle><tutelle active="#struct-300812" type="indirect"><org type="institution" xml:id="struct-300812" status="VALID"><orgName>SUPELEC</orgName><desc><address><country key="FR"></country></address></desc></org></tutelle><tutelle active="#struct-300413" type="indirect"><org type="institution" xml:id="struct-300413" status="VALID"><orgName>Ecole Nationale Supérieure des Arts et Metiers Metz</orgName><desc><address><country key="FR"></country></address></desc></org></tutelle><tutelle active="#struct-300289" type="indirect"><org type="institution" xml:id="struct-300289" status="OLD"><orgName>Université Paul Verlaine - Metz</orgName><orgName type="acronym">UPVM</orgName><date type="end">2011-12-31</date><desc><address><country key="FR"></country></address></desc></org></tutelle><tutelle name="UMI2958" active="#struct-441569" type="indirect"><org type="institution" xml:id="struct-441569" status="VALID"><idno type="IdRef">02636817X</idno><idno type="ISNI">0000000122597504</idno><orgName>Centre National de la Recherche Scientifique</orgName><orgName type="acronym">CNRS</orgName><date type="start">1939-10-19</date><desc><address><country key="FR"></country></address><ref type="url">http://www.cnrs.fr/</ref></desc></org></tutelle><tutelle active="#struct-26305" type="direct"><org type="laboratory" xml:id="struct-26305" status="VALID"><orgName>SUPELEC-Campus Metz</orgName><desc><address><addrLine>2 rue Edouard Belin 57070 Metz</addrLine><country key="FR"></country></address><ref type="url">http://www.metz.supelec.fr/metz/</ref></desc><listRelation><relation active="#struct-300812" type="direct"></relation></listRelation></org></tutelle></tutelles></hal:affiliation><country>France</country><placeName><settlement type="city" wicri:auto="siege">Besançon</settlement><region type="region" nuts="2">Franche-Comté</region></placeName><orgName type="university">Université de Franche-Comté</orgName><orgName type="institution" wicri:auto="newGroup">Université de Bourgogne Franche-Comté</orgName><placeName><settlement type="city">Metz</settlement><region type="region" nuts="2">Grand Est</region><region type="old region" nuts="2">Lorraine (région)</region></placeName><orgName type="university">Université Paul Verlaine - Metz</orgName><orgName type="institution" wicri:auto="newGroup">Université de Lorraine</orgName></affiliation></author></titleStmt><publicationStmt><idno type="wicri:source">HAL</idno><idno type="RBID">Hal:hal-00771646</idno><idno type="halId">hal-00771646</idno><idno type="halUri">https://hal-supelec.archives-ouvertes.fr/hal-00771646</idno><idno type="url">https://hal-supelec.archives-ouvertes.fr/hal-00771646</idno><idno type="doi">10.1109/JSTSP.2012.2229257</idno><date when="2012-12">2012-12</date><idno type="wicri:Area/Hal/Corpus">000125</idno><idno type="wicri:Area/Hal/Curation">000125</idno><idno type="wicri:Area/Hal/Checkpoint">001449</idno><idno type="wicri:explorRef" wicri:stream="Hal" wicri:step="Checkpoint">001449</idno><idno type="wicri:Area/Main/Merge">001862</idno><idno type="wicri:Area/Main/Curation">001835</idno><idno type="wicri:Area/Main/Exploration">001835</idno></publicationStmt><sourceDesc><biblStruct><analytic><title xml:lang="en">A Comprehensive Reinforcement Learning Framework for Dialogue Management Optimisation</title><author><name sortKey="Daubigney, Lucie" sort="Daubigney, Lucie" uniqKey="Daubigney L" first="Lucie" last="Daubigney">Lucie Daubigney</name><affiliation wicri:level="1"><hal:affiliation type="researchteam" xml:id="struct-205124" status="OLD"><idno type="RNSR">200218290B</idno><orgName>Autonomous intelligent machine</orgName><orgName type="acronym">MAIA</orgName><date type="end">2014-12-31</date><desc><address><country key="FR"></country></address><ref type="url">http://www.inria.fr/equipes/maia</ref></desc><listRelation><relation active="#struct-129671" type="direct"></relation><relation active="#struct-300009" type="indirect"></relation><relation active="#struct-423090" type="direct"></relation><relation active="#struct-206040" type="indirect"></relation><relation active="#struct-413289" type="indirect"></relation><relation name="UMR7503" active="#struct-441569" type="indirect"></relation></listRelation><tutelles><tutelle active="#struct-129671" type="direct"><org type="laboratory" xml:id="struct-129671" status="VALID"><idno type="RNSR">198618246Y</idno><orgName>INRIA Nancy - Grand Est</orgName><desc><address><addrLine>615 rue du Jardin Botanique 54600 Villers-lès-Nancy</addrLine><country key="FR"></country></address><ref type="url">http://www.inria.fr/nancy</ref></desc><listRelation><relation active="#struct-300009" type="direct"></relation></listRelation></org></tutelle><tutelle active="#struct-300009" type="indirect"><org type="institution" xml:id="struct-300009" status="VALID"><orgName>Institut National de Recherche en Informatique et en Automatique</orgName><orgName type="acronym">Inria</orgName><desc><address><addrLine>Domaine de VoluceauRocquencourt - BP 10578153 Le Chesnay Cedex</addrLine><country key="FR"></country></address><ref type="url">http://www.inria.fr/en/</ref></desc></org></tutelle><tutelle active="#struct-423090" type="direct"><org type="department" xml:id="struct-423090" status="VALID"><orgName>Department of Complex Systems, Artificial Intelligence & Robotics</orgName><orgName type="acronym">LORIA - AIS</orgName><desc><address><country key="FR"></country></address><ref type="url">http://www.loria.fr/la-recherche-en/departements/complex-system-and-artificial-intelligence</ref></desc><listRelation><relation active="#struct-206040" type="direct"></relation><relation active="#struct-300009" type="indirect"></relation><relation active="#struct-413289" type="indirect"></relation><relation name="UMR7503" active="#struct-441569" type="indirect"></relation></listRelation></org></tutelle><tutelle active="#struct-206040" type="indirect"><org type="laboratory" xml:id="struct-206040" status="VALID"><idno type="IdRef">067077927</idno><idno type="RNSR">198912571S</idno><idno type="IdUnivLorraine">[UL]RSI--</idno><orgName>Laboratoire Lorrain de Recherche en Informatique et ses Applications</orgName><orgName type="acronym">LORIA</orgName><date type="start">2012-01-01</date><desc><address><addrLine>Campus Scientifique BP 239 54506 Vandoeuvre-lès-Nancy Cedex</addrLine><country key="FR"></country></address><ref type="url">http://www.loria.fr</ref></desc><listRelation><relation active="#struct-300009" type="direct"></relation><relation active="#struct-413289" type="direct"></relation><relation name="UMR7503" active="#struct-441569" type="direct"></relation></listRelation></org></tutelle><tutelle active="#struct-413289" type="indirect"><org type="institution" xml:id="struct-413289" status="VALID"><idno type="IdRef">157040569</idno><idno type="IdUnivLorraine">[UL]100--</idno><orgName>Université de Lorraine</orgName><orgName type="acronym">UL</orgName><date type="start">2012-01-01</date><desc><address><addrLine>34 cours Léopold - CS 25233 - 54052 Nancy cedex</addrLine><country key="FR"></country></address><ref type="url">http://www.univ-lorraine.fr/</ref></desc></org></tutelle><tutelle name="UMR7503" active="#struct-441569" type="indirect"><org type="institution" xml:id="struct-441569" status="VALID"><idno type="IdRef">02636817X</idno><idno type="ISNI">0000000122597504</idno><orgName>Centre National de la Recherche Scientifique</orgName><orgName type="acronym">CNRS</orgName><date type="start">1939-10-19</date><desc><address><country key="FR"></country></address><ref type="url">http://www.cnrs.fr/</ref></desc></org></tutelle></tutelles></hal:affiliation><country>France</country><placeName><settlement type="city">Nancy</settlement><settlement type="city">Metz</settlement><region type="region" nuts="2">Grand Est</region><region type="old region" nuts="2">Lorraine (région)</region></placeName><orgName type="university">Université de Lorraine</orgName></affiliation></author><author><name sortKey="Geist, Matthieu" sort="Geist, Matthieu" uniqKey="Geist M" first="Matthieu" last="Geist">Matthieu Geist</name><affiliation wicri:level="1"><hal:affiliation type="researchteam" xml:id="struct-394500" status="INCOMING"><orgName>IMS - Equipe Information, Multimodalité et Signal</orgName><desc><address><country key="FR"></country></address></desc><listRelation><relation active="#struct-24541" type="direct"></relation><relation active="#struct-242365" type="indirect"></relation><relation active="#struct-411575" type="indirect"></relation><relation active="#struct-301991" type="indirect"></relation><relation active="#struct-301990" type="indirect"></relation><relation active="#struct-300812" type="indirect"></relation><relation active="#struct-300413" type="indirect"></relation><relation active="#struct-300289" type="indirect"></relation><relation name="UMI2958" active="#struct-441569" type="indirect"></relation><relation active="#struct-26305" type="direct"></relation></listRelation><tutelles><tutelle active="#struct-24541" type="direct"><org type="laboratory" xml:id="struct-24541" status="VALID"><idno type="RNSR">200619366D</idno><orgName>Georgia Tech - CNRS [Metz]</orgName><orgName type="acronym">UMI2958</orgName><desc><address><addrLine>Metz Technopôle 2-3 rue Marconi 57070 METZ</addrLine><country key="FR"></country></address><ref type="url">http://www.umi2958.eu</ref></desc><listRelation><relation active="#struct-242365" type="direct"></relation><relation active="#struct-411575" type="direct"></relation><relation active="#struct-301991" type="direct"></relation><relation active="#struct-301990" type="direct"></relation><relation active="#struct-300812" type="direct"></relation><relation active="#struct-300413" type="direct"></relation><relation active="#struct-300289" type="direct"></relation><relation name="UMI2958" active="#struct-441569" type="direct"></relation></listRelation></org></tutelle><tutelle active="#struct-242365" type="indirect"><org type="institution" xml:id="struct-242365" status="VALID"><idno type="IdRef">026403188</idno><idno type="ISNI">0000 0001 2188 3779 </idno><orgName>Université de Franche-Comté</orgName><orgName type="acronym">UFC</orgName><desc><address><country key="FR"></country></address><ref type="url">http://www.univ-fcomte.fr</ref></desc></org></tutelle><tutelle active="#struct-411575" type="indirect"><org type="institution" xml:id="struct-411575" status="VALID"><orgName>CentraleSupélec</orgName><desc><address><addrLine>3, rue Joliot Curie,Plateau de Moulon,91192 GIF-SUR-YVETTE Cedex</addrLine><country key="FR"></country></address><ref type="url">http://www.centralesupelec.fr</ref></desc></org></tutelle><tutelle active="#struct-301991" type="indirect"><org type="institution" xml:id="struct-301991" status="VALID"><orgName>Georgia Tech Lorraine</orgName><desc><address><country key="FR"></country></address></desc></org></tutelle><tutelle active="#struct-301990" type="indirect"><org type="institution" xml:id="struct-301990" status="VALID"><orgName>Georgia Institute of Technology [Atlanta]</orgName><desc><address><addrLine>North Avenue, Atlanta, GA 30332</addrLine><country key="US"></country></address><ref type="url">http://www.gatech.edu/</ref></desc></org></tutelle><tutelle active="#struct-300812" type="indirect"><org type="institution" xml:id="struct-300812" status="VALID"><orgName>SUPELEC</orgName><desc><address><country key="FR"></country></address></desc></org></tutelle><tutelle active="#struct-300413" type="indirect"><org type="institution" xml:id="struct-300413" status="VALID"><orgName>Ecole Nationale Supérieure des Arts et Metiers Metz</orgName><desc><address><country key="FR"></country></address></desc></org></tutelle><tutelle active="#struct-300289" type="indirect"><org type="institution" xml:id="struct-300289" status="OLD"><orgName>Université Paul Verlaine - Metz</orgName><orgName type="acronym">UPVM</orgName><date type="end">2011-12-31</date><desc><address><country key="FR"></country></address></desc></org></tutelle><tutelle name="UMI2958" active="#struct-441569" type="indirect"><org type="institution" xml:id="struct-441569" status="VALID"><idno type="IdRef">02636817X</idno><idno type="ISNI">0000000122597504</idno><orgName>Centre National de la Recherche Scientifique</orgName><orgName type="acronym">CNRS</orgName><date type="start">1939-10-19</date><desc><address><country key="FR"></country></address><ref type="url">http://www.cnrs.fr/</ref></desc></org></tutelle><tutelle active="#struct-26305" type="direct"><org type="laboratory" xml:id="struct-26305" status="VALID"><orgName>SUPELEC-Campus Metz</orgName><desc><address><addrLine>2 rue Edouard Belin 57070 Metz</addrLine><country key="FR"></country></address><ref type="url">http://www.metz.supelec.fr/metz/</ref></desc><listRelation><relation active="#struct-300812" type="direct"></relation></listRelation></org></tutelle></tutelles></hal:affiliation><country>France</country><placeName><settlement type="city" wicri:auto="siege">Besançon</settlement><region type="region" nuts="2">Franche-Comté</region></placeName><orgName type="university">Université de Franche-Comté</orgName><orgName type="institution" wicri:auto="newGroup">Université de Bourgogne Franche-Comté</orgName><placeName><settlement type="city">Metz</settlement><region type="region" nuts="2">Grand Est</region><region type="old region" nuts="2">Lorraine (région)</region></placeName><orgName type="university">Université Paul Verlaine - Metz</orgName><orgName type="institution" wicri:auto="newGroup">Université de Lorraine</orgName></affiliation></author><author><name sortKey="Chandramohan, Senthilkumar" sort="Chandramohan, Senthilkumar" uniqKey="Chandramohan S" first="Senthilkumar" last="Chandramohan">Senthilkumar Chandramohan</name><affiliation wicri:level="1"><hal:affiliation type="researchteam" xml:id="struct-394500" status="INCOMING"><orgName>IMS - Equipe Information, Multimodalité et Signal</orgName><desc><address><country key="FR"></country></address></desc><listRelation><relation active="#struct-24541" type="direct"></relation><relation active="#struct-242365" type="indirect"></relation><relation active="#struct-411575" type="indirect"></relation><relation active="#struct-301991" type="indirect"></relation><relation active="#struct-301990" type="indirect"></relation><relation active="#struct-300812" type="indirect"></relation><relation active="#struct-300413" type="indirect"></relation><relation active="#struct-300289" type="indirect"></relation><relation name="UMI2958" active="#struct-441569" type="indirect"></relation><relation active="#struct-26305" type="direct"></relation></listRelation><tutelles><tutelle active="#struct-24541" type="direct"><org type="laboratory" xml:id="struct-24541" status="VALID"><idno type="RNSR">200619366D</idno><orgName>Georgia Tech - CNRS [Metz]</orgName><orgName type="acronym">UMI2958</orgName><desc><address><addrLine>Metz Technopôle 2-3 rue Marconi 57070 METZ</addrLine><country key="FR"></country></address><ref type="url">http://www.umi2958.eu</ref></desc><listRelation><relation active="#struct-242365" type="direct"></relation><relation active="#struct-411575" type="direct"></relation><relation active="#struct-301991" type="direct"></relation><relation active="#struct-301990" type="direct"></relation><relation active="#struct-300812" type="direct"></relation><relation active="#struct-300413" type="direct"></relation><relation active="#struct-300289" type="direct"></relation><relation name="UMI2958" active="#struct-441569" type="direct"></relation></listRelation></org></tutelle><tutelle active="#struct-242365" type="indirect"><org type="institution" xml:id="struct-242365" status="VALID"><idno type="IdRef">026403188</idno><idno type="ISNI">0000 0001 2188 3779 </idno><orgName>Université de Franche-Comté</orgName><orgName type="acronym">UFC</orgName><desc><address><country key="FR"></country></address><ref type="url">http://www.univ-fcomte.fr</ref></desc></org></tutelle><tutelle active="#struct-411575" type="indirect"><org type="institution" xml:id="struct-411575" status="VALID"><orgName>CentraleSupélec</orgName><desc><address><addrLine>3, rue Joliot Curie,Plateau de Moulon,91192 GIF-SUR-YVETTE Cedex</addrLine><country key="FR"></country></address><ref type="url">http://www.centralesupelec.fr</ref></desc></org></tutelle><tutelle active="#struct-301991" type="indirect"><org type="institution" xml:id="struct-301991" status="VALID"><orgName>Georgia Tech Lorraine</orgName><desc><address><country key="FR"></country></address></desc></org></tutelle><tutelle active="#struct-301990" type="indirect"><org type="institution" xml:id="struct-301990" status="VALID"><orgName>Georgia Institute of Technology [Atlanta]</orgName><desc><address><addrLine>North Avenue, Atlanta, GA 30332</addrLine><country key="US"></country></address><ref type="url">http://www.gatech.edu/</ref></desc></org></tutelle><tutelle active="#struct-300812" type="indirect"><org type="institution" xml:id="struct-300812" status="VALID"><orgName>SUPELEC</orgName><desc><address><country key="FR"></country></address></desc></org></tutelle><tutelle active="#struct-300413" type="indirect"><org type="institution" xml:id="struct-300413" status="VALID"><orgName>Ecole Nationale Supérieure des Arts et Metiers Metz</orgName><desc><address><country key="FR"></country></address></desc></org></tutelle><tutelle active="#struct-300289" type="indirect"><org type="institution" xml:id="struct-300289" status="OLD"><orgName>Université Paul Verlaine - Metz</orgName><orgName type="acronym">UPVM</orgName><date type="end">2011-12-31</date><desc><address><country key="FR"></country></address></desc></org></tutelle><tutelle name="UMI2958" active="#struct-441569" type="indirect"><org type="institution" xml:id="struct-441569" status="VALID"><idno type="IdRef">02636817X</idno><idno type="ISNI">0000000122597504</idno><orgName>Centre National de la Recherche Scientifique</orgName><orgName type="acronym">CNRS</orgName><date type="start">1939-10-19</date><desc><address><country key="FR"></country></address><ref type="url">http://www.cnrs.fr/</ref></desc></org></tutelle><tutelle active="#struct-26305" type="direct"><org type="laboratory" xml:id="struct-26305" status="VALID"><orgName>SUPELEC-Campus Metz</orgName><desc><address><addrLine>2 rue Edouard Belin 57070 Metz</addrLine><country key="FR"></country></address><ref type="url">http://www.metz.supelec.fr/metz/</ref></desc><listRelation><relation active="#struct-300812" type="direct"></relation></listRelation></org></tutelle></tutelles></hal:affiliation><country>France</country><placeName><settlement type="city" wicri:auto="siege">Besançon</settlement><region type="region" nuts="2">Franche-Comté</region></placeName><orgName type="university">Université de Franche-Comté</orgName><orgName type="institution" wicri:auto="newGroup">Université de Bourgogne Franche-Comté</orgName><placeName><settlement type="city">Metz</settlement><region type="region" nuts="2">Grand Est</region><region type="old region" nuts="2">Lorraine (région)</region></placeName><orgName type="university">Université Paul Verlaine - Metz</orgName><orgName type="institution" wicri:auto="newGroup">Université de Lorraine</orgName></affiliation></author><author><name sortKey="Pietquin, Olivier" sort="Pietquin, Olivier" uniqKey="Pietquin O" first="Olivier" last="Pietquin">Olivier Pietquin</name><affiliation wicri:level="1"><hal:affiliation type="researchteam" xml:id="struct-394500" status="INCOMING"><orgName>IMS - Equipe Information, Multimodalité et Signal</orgName><desc><address><country key="FR"></country></address></desc><listRelation><relation active="#struct-24541" type="direct"></relation><relation active="#struct-242365" type="indirect"></relation><relation active="#struct-411575" type="indirect"></relation><relation active="#struct-301991" type="indirect"></relation><relation active="#struct-301990" type="indirect"></relation><relation active="#struct-300812" type="indirect"></relation><relation active="#struct-300413" type="indirect"></relation><relation active="#struct-300289" type="indirect"></relation><relation name="UMI2958" active="#struct-441569" type="indirect"></relation><relation active="#struct-26305" type="direct"></relation></listRelation><tutelles><tutelle active="#struct-24541" type="direct"><org type="laboratory" xml:id="struct-24541" status="VALID"><idno type="RNSR">200619366D</idno><orgName>Georgia Tech - CNRS [Metz]</orgName><orgName type="acronym">UMI2958</orgName><desc><address><addrLine>Metz Technopôle 2-3 rue Marconi 57070 METZ</addrLine><country key="FR"></country></address><ref type="url">http://www.umi2958.eu</ref></desc><listRelation><relation active="#struct-242365" type="direct"></relation><relation active="#struct-411575" type="direct"></relation><relation active="#struct-301991" type="direct"></relation><relation active="#struct-301990" type="direct"></relation><relation active="#struct-300812" type="direct"></relation><relation active="#struct-300413" type="direct"></relation><relation active="#struct-300289" type="direct"></relation><relation name="UMI2958" active="#struct-441569" type="direct"></relation></listRelation></org></tutelle><tutelle active="#struct-242365" type="indirect"><org type="institution" xml:id="struct-242365" status="VALID"><idno type="IdRef">026403188</idno><idno type="ISNI">0000 0001 2188 3779 </idno><orgName>Université de Franche-Comté</orgName><orgName type="acronym">UFC</orgName><desc><address><country key="FR"></country></address><ref type="url">http://www.univ-fcomte.fr</ref></desc></org></tutelle><tutelle active="#struct-411575" type="indirect"><org type="institution" xml:id="struct-411575" status="VALID"><orgName>CentraleSupélec</orgName><desc><address><addrLine>3, rue Joliot Curie,Plateau de Moulon,91192 GIF-SUR-YVETTE Cedex</addrLine><country key="FR"></country></address><ref type="url">http://www.centralesupelec.fr</ref></desc></org></tutelle><tutelle active="#struct-301991" type="indirect"><org type="institution" xml:id="struct-301991" status="VALID"><orgName>Georgia Tech Lorraine</orgName><desc><address><country key="FR"></country></address></desc></org></tutelle><tutelle active="#struct-301990" type="indirect"><org type="institution" xml:id="struct-301990" status="VALID"><orgName>Georgia Institute of Technology [Atlanta]</orgName><desc><address><addrLine>North Avenue, Atlanta, GA 30332</addrLine><country key="US"></country></address><ref type="url">http://www.gatech.edu/</ref></desc></org></tutelle><tutelle active="#struct-300812" type="indirect"><org type="institution" xml:id="struct-300812" status="VALID"><orgName>SUPELEC</orgName><desc><address><country key="FR"></country></address></desc></org></tutelle><tutelle active="#struct-300413" type="indirect"><org type="institution" xml:id="struct-300413" status="VALID"><orgName>Ecole Nationale Supérieure des Arts et Metiers Metz</orgName><desc><address><country key="FR"></country></address></desc></org></tutelle><tutelle active="#struct-300289" type="indirect"><org type="institution" xml:id="struct-300289" status="OLD"><orgName>Université Paul Verlaine - Metz</orgName><orgName type="acronym">UPVM</orgName><date type="end">2011-12-31</date><desc><address><country key="FR"></country></address></desc></org></tutelle><tutelle name="UMI2958" active="#struct-441569" type="indirect"><org type="institution" xml:id="struct-441569" status="VALID"><idno type="IdRef">02636817X</idno><idno type="ISNI">0000000122597504</idno><orgName>Centre National de la Recherche Scientifique</orgName><orgName type="acronym">CNRS</orgName><date type="start">1939-10-19</date><desc><address><country key="FR"></country></address><ref type="url">http://www.cnrs.fr/</ref></desc></org></tutelle><tutelle active="#struct-26305" type="direct"><org type="laboratory" xml:id="struct-26305" status="VALID"><orgName>SUPELEC-Campus Metz</orgName><desc><address><addrLine>2 rue Edouard Belin 57070 Metz</addrLine><country key="FR"></country></address><ref type="url">http://www.metz.supelec.fr/metz/</ref></desc><listRelation><relation active="#struct-300812" type="direct"></relation></listRelation></org></tutelle></tutelles></hal:affiliation><country>France</country><placeName><settlement type="city" wicri:auto="siege">Besançon</settlement><region type="region" nuts="2">Franche-Comté</region></placeName><orgName type="university">Université de Franche-Comté</orgName><orgName type="institution" wicri:auto="newGroup">Université de Bourgogne Franche-Comté</orgName><placeName><settlement type="city">Metz</settlement><region type="region" nuts="2">Grand Est</region><region type="old region" nuts="2">Lorraine (région)</region></placeName><orgName type="university">Université Paul Verlaine - Metz</orgName><orgName type="institution" wicri:auto="newGroup">Université de Lorraine</orgName></affiliation></author></analytic><idno type="DOI">10.1109/JSTSP.2012.2229257</idno></biblStruct></sourceDesc></fileDesc><profileDesc><textClass></textClass></profileDesc></teiHeader><front><div type="abstract" xml:lang="en">Reinforcement learning is now an acknowledged approach for optimising the interaction strategy of spoken dialogue systems. If the first considered algorithms were quite basic (like SARSA), recent works concentrated on more sophisticated methods. More attention has been paid to off-policy learning, dealing with the exploration-exploitation dilemma, sample efficiency or handling non-stationarity. New algorithms have been proposed to address these issues and have been applied to dialogue management. However, each algorithm often solves a single issue at a time, while dialogue systems exhibit all the problems at once. In this paper, we propose to apply the Kalman Temporal Differences (KTD) framework to the problem of dialogue strategy optimisation so as to address all these issues in a comprehensive manner with a single framework. Our claims are illustrated by experiments led on two real-world goal-oriented dialogue management frameworks, DIPPER and HIS.</div></front></TEI><affiliations><list><country><li>France</li></country><region><li>Franche-Comté</li><li>Grand Est</li><li>Lorraine (région)</li></region><settlement><li>Besançon</li><li>Metz</li><li>Nancy</li></settlement><orgName><li>Université Paul Verlaine - Metz</li><li>Université de Bourgogne Franche-Comté</li><li>Université de Franche-Comté</li><li>Université de Lorraine</li></orgName></list><tree><country name="France"><region name="Grand Est"><name sortKey="Daubigney, Lucie" sort="Daubigney, Lucie" uniqKey="Daubigney L" first="Lucie" last="Daubigney">Lucie Daubigney</name></region><name sortKey="Chandramohan, Senthilkumar" sort="Chandramohan, Senthilkumar" uniqKey="Chandramohan S" first="Senthilkumar" last="Chandramohan">Senthilkumar Chandramohan</name><name sortKey="Geist, Matthieu" sort="Geist, Matthieu" uniqKey="Geist M" first="Matthieu" last="Geist">Matthieu Geist</name><name sortKey="Pietquin, Olivier" sort="Pietquin, Olivier" uniqKey="Pietquin O" first="Olivier" last="Pietquin">Olivier Pietquin</name></country></tree></affiliations></record>Pour manipuler ce document sous Unix (Dilib)
EXPLOR_STEP=$WICRI_ROOT/Wicri/Lorraine/explor/InforLorV4/Data/Main/Exploration
HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 001835 | SxmlIndent | more
Ou
HfdSelect -h $EXPLOR_AREA/Data/Main/Exploration/biblio.hfd -nk 001835 | SxmlIndent | more
Pour mettre un lien sur cette page dans le réseau Wicri
{{Explor lien
|wiki= Wicri/Lorraine
|area= InforLorV4
|flux= Main
|étape= Exploration
|type= RBID
|clé= Hal:hal-00771646
|texte= A Comprehensive Reinforcement Learning Framework for Dialogue Management Optimisation
}}
| This area was generated with Dilib version V0.6.33. |  |