The development of an adaptive upper-limb stroke rehabilitation robotic system
Identifieur interne : 001A92 ( Ncbi/Merge ); précédent : 001A91; suivant : 001A93The development of an adaptive upper-limb stroke rehabilitation robotic system
Auteurs : Patricia Kan [Canada] ; Rajibul Huq [Canada] ; Jesse Hoey [Royaume-Uni] ; Robby Goetschalckx [Royaume-Uni] ; Alex Mihailidis [Canada]Source :
- Journal of NeuroEngineering and Rehabilitation [ 1743-0003 ] ; 2011.
Abstract
Stroke is the primary cause of adult disability. To support this large population in recovery, robotic technologies are being developed to assist in the delivery of rehabilitation. This paper presents an automated system for a rehabilitation robotic device that guides stroke patients through an upper-limb reaching task. The system uses a decision theoretic model (a partially observable Markov decision process, or POMDP) as its primary engine for decision making. The POMDP allows the system to automatically modify exercise parameters to account for the specific needs and abilities of different individuals, and to use these parameters to take appropriate decisions about stroke rehabilitation exercises.
The performance of the system was evaluated by comparing the decisions made by the system with those of a human therapist. A single patient participant was paired up with a therapist participant for the duration of the study, for a total of six sessions. Each session was an hour long and occurred three times a week for two weeks. During each session, three steps were followed: (A) after the system made a decision, the therapist either agreed or disagreed with the decision made; (B) the researcher had the device execute the decision made by the therapist; (C) the patient then performed the reaching exercise. These parts were repeated in the order of A-B-C until the end of the session. Qualitative and quantitative question were asked at the end of each session and at the completion of the study for both participants.
Overall, the therapist agreed with the system decisions approximately 65% of the time. In general, the therapist thought the system decisions were believable and could envision this system being used in both a clinical and home setting. The patient was satisfied with the system and would use this system as his/her primary method of rehabilitation.
The data collected in this study can only be used to provide insight into the performance of the system since the sample size was limited. The next stage for this project is to test the system with a larger sample size to obtain significant results.
Url:
DOI: 10.1186/1743-0003-8-33
PubMed: 21679457
PubMed Central: 3152889
Links toward previous steps (curation, corpus...)
- to stream Pmc, to step Corpus: 002047
- to stream Pmc, to step Curation: 002047
- to stream Pmc, to step Checkpoint: 001952
Links to Exploration step
PMC:3152889Le document en format XML
<record><TEI><teiHeader><fileDesc><titleStmt><title xml:lang="en">The development of an adaptive upper-limb stroke rehabilitation robotic system</title><author><name sortKey="Kan, Patricia" sort="Kan, Patricia" uniqKey="Kan P" first="Patricia" last="Kan">Patricia Kan</name><affiliation wicri:level="4"><nlm:aff id="I1">Institute of Biomaterials and Biomedical Engineering, Rosebrugh Building, 164 College Street, Room 407, University of Toronto, Toronto, M5T 1P7, Canada</nlm:aff><country xml:lang="fr">Canada</country><wicri:regionArea>Institute of Biomaterials and Biomedical Engineering, Rosebrugh Building, 164 College Street, Room 407, University of Toronto, Toronto, M5T 1P7</wicri:regionArea><orgName type="university">Université de Toronto</orgName><placeName><settlement type="city">Toronto</settlement><region type="state">Ontario</region></placeName></affiliation></author><author><name sortKey="Huq, Rajibul" sort="Huq, Rajibul" uniqKey="Huq R" first="Rajibul" last="Huq">Rajibul Huq</name><affiliation wicri:level="4"><nlm:aff id="I1">Institute of Biomaterials and Biomedical Engineering, Rosebrugh Building, 164 College Street, Room 407, University of Toronto, Toronto, M5T 1P7, Canada</nlm:aff><country xml:lang="fr">Canada</country><wicri:regionArea>Institute of Biomaterials and Biomedical Engineering, Rosebrugh Building, 164 College Street, Room 407, University of Toronto, Toronto, M5T 1P7</wicri:regionArea><orgName type="university">Université de Toronto</orgName><placeName><settlement type="city">Toronto</settlement><region type="state">Ontario</region></placeName></affiliation></author><author><name sortKey="Hoey, Jesse" sort="Hoey, Jesse" uniqKey="Hoey J" first="Jesse" last="Hoey">Jesse Hoey</name><affiliation wicri:level="1"><nlm:aff id="I2">School of Computing, University of Dundee, Dundee, DD1 4HN, UK</nlm:aff><country xml:lang="fr">Royaume-Uni</country><wicri:regionArea>School of Computing, University of Dundee, Dundee, DD1 4HN</wicri:regionArea><wicri:noRegion>DD1 4HN</wicri:noRegion></affiliation></author><author><name sortKey="Goetschalckx, Robby" sort="Goetschalckx, Robby" uniqKey="Goetschalckx R" first="Robby" last="Goetschalckx">Robby Goetschalckx</name><affiliation wicri:level="1"><nlm:aff id="I2">School of Computing, University of Dundee, Dundee, DD1 4HN, UK</nlm:aff><country xml:lang="fr">Royaume-Uni</country><wicri:regionArea>School of Computing, University of Dundee, Dundee, DD1 4HN</wicri:regionArea><wicri:noRegion>DD1 4HN</wicri:noRegion></affiliation></author><author><name sortKey="Mihailidis, Alex" sort="Mihailidis, Alex" uniqKey="Mihailidis A" first="Alex" last="Mihailidis">Alex Mihailidis</name><affiliation wicri:level="4"><nlm:aff id="I1">Institute of Biomaterials and Biomedical Engineering, Rosebrugh Building, 164 College Street, Room 407, University of Toronto, Toronto, M5T 1P7, Canada</nlm:aff><country xml:lang="fr">Canada</country><wicri:regionArea>Institute of Biomaterials and Biomedical Engineering, Rosebrugh Building, 164 College Street, Room 407, University of Toronto, Toronto, M5T 1P7</wicri:regionArea><orgName type="university">Université de Toronto</orgName><placeName><settlement type="city">Toronto</settlement><region type="state">Ontario</region></placeName></affiliation><affiliation wicri:level="4"><nlm:aff id="I3">Department of Occupational Science and Occupational Therapy, University of Toronto, 160-500 University Avenue, Toronto, M5G 1V7, Canada</nlm:aff><country xml:lang="fr">Canada</country><wicri:regionArea>Department of Occupational Science and Occupational Therapy, University of Toronto, 160-500 University Avenue, Toronto, M5G 1V7</wicri:regionArea><orgName type="university">Université de Toronto</orgName><placeName><settlement type="city">Toronto</settlement><region type="state">Ontario</region></placeName></affiliation><affiliation wicri:level="1"><nlm:aff id="I4">Toronto Rehabilitation Institute, 550 University Avenue, M5G 2A2, Toronto, Canada</nlm:aff><country xml:lang="fr">Canada</country><wicri:regionArea>Toronto Rehabilitation Institute, 550 University Avenue, M5G 2A2, Toronto</wicri:regionArea><wicri:noRegion>Toronto</wicri:noRegion></affiliation></author></titleStmt><publicationStmt><idno type="wicri:source">PMC</idno><idno type="pmid">21679457</idno><idno type="pmc">3152889</idno><idno type="url">http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3152889</idno><idno type="RBID">PMC:3152889</idno><idno type="doi">10.1186/1743-0003-8-33</idno><date when="2011">2011</date><idno type="wicri:Area/Pmc/Corpus">002047</idno><idno type="wicri:Area/Pmc/Curation">002047</idno><idno type="wicri:Area/Pmc/Checkpoint">001952</idno><idno type="wicri:Area/Ncbi/Merge">001A92</idno></publicationStmt><sourceDesc><biblStruct><analytic><title xml:lang="en" level="a" type="main">The development of an adaptive upper-limb stroke rehabilitation robotic system</title><author><name sortKey="Kan, Patricia" sort="Kan, Patricia" uniqKey="Kan P" first="Patricia" last="Kan">Patricia Kan</name><affiliation wicri:level="4"><nlm:aff id="I1">Institute of Biomaterials and Biomedical Engineering, Rosebrugh Building, 164 College Street, Room 407, University of Toronto, Toronto, M5T 1P7, Canada</nlm:aff><country xml:lang="fr">Canada</country><wicri:regionArea>Institute of Biomaterials and Biomedical Engineering, Rosebrugh Building, 164 College Street, Room 407, University of Toronto, Toronto, M5T 1P7</wicri:regionArea><orgName type="university">Université de Toronto</orgName><placeName><settlement type="city">Toronto</settlement><region type="state">Ontario</region></placeName></affiliation></author><author><name sortKey="Huq, Rajibul" sort="Huq, Rajibul" uniqKey="Huq R" first="Rajibul" last="Huq">Rajibul Huq</name><affiliation wicri:level="4"><nlm:aff id="I1">Institute of Biomaterials and Biomedical Engineering, Rosebrugh Building, 164 College Street, Room 407, University of Toronto, Toronto, M5T 1P7, Canada</nlm:aff><country xml:lang="fr">Canada</country><wicri:regionArea>Institute of Biomaterials and Biomedical Engineering, Rosebrugh Building, 164 College Street, Room 407, University of Toronto, Toronto, M5T 1P7</wicri:regionArea><orgName type="university">Université de Toronto</orgName><placeName><settlement type="city">Toronto</settlement><region type="state">Ontario</region></placeName></affiliation></author><author><name sortKey="Hoey, Jesse" sort="Hoey, Jesse" uniqKey="Hoey J" first="Jesse" last="Hoey">Jesse Hoey</name><affiliation wicri:level="1"><nlm:aff id="I2">School of Computing, University of Dundee, Dundee, DD1 4HN, UK</nlm:aff><country xml:lang="fr">Royaume-Uni</country><wicri:regionArea>School of Computing, University of Dundee, Dundee, DD1 4HN</wicri:regionArea><wicri:noRegion>DD1 4HN</wicri:noRegion></affiliation></author><author><name sortKey="Goetschalckx, Robby" sort="Goetschalckx, Robby" uniqKey="Goetschalckx R" first="Robby" last="Goetschalckx">Robby Goetschalckx</name><affiliation wicri:level="1"><nlm:aff id="I2">School of Computing, University of Dundee, Dundee, DD1 4HN, UK</nlm:aff><country xml:lang="fr">Royaume-Uni</country><wicri:regionArea>School of Computing, University of Dundee, Dundee, DD1 4HN</wicri:regionArea><wicri:noRegion>DD1 4HN</wicri:noRegion></affiliation></author><author><name sortKey="Mihailidis, Alex" sort="Mihailidis, Alex" uniqKey="Mihailidis A" first="Alex" last="Mihailidis">Alex Mihailidis</name><affiliation wicri:level="4"><nlm:aff id="I1">Institute of Biomaterials and Biomedical Engineering, Rosebrugh Building, 164 College Street, Room 407, University of Toronto, Toronto, M5T 1P7, Canada</nlm:aff><country xml:lang="fr">Canada</country><wicri:regionArea>Institute of Biomaterials and Biomedical Engineering, Rosebrugh Building, 164 College Street, Room 407, University of Toronto, Toronto, M5T 1P7</wicri:regionArea><orgName type="university">Université de Toronto</orgName><placeName><settlement type="city">Toronto</settlement><region type="state">Ontario</region></placeName></affiliation><affiliation wicri:level="4"><nlm:aff id="I3">Department of Occupational Science and Occupational Therapy, University of Toronto, 160-500 University Avenue, Toronto, M5G 1V7, Canada</nlm:aff><country xml:lang="fr">Canada</country><wicri:regionArea>Department of Occupational Science and Occupational Therapy, University of Toronto, 160-500 University Avenue, Toronto, M5G 1V7</wicri:regionArea><orgName type="university">Université de Toronto</orgName><placeName><settlement type="city">Toronto</settlement><region type="state">Ontario</region></placeName></affiliation><affiliation wicri:level="1"><nlm:aff id="I4">Toronto Rehabilitation Institute, 550 University Avenue, M5G 2A2, Toronto, Canada</nlm:aff><country xml:lang="fr">Canada</country><wicri:regionArea>Toronto Rehabilitation Institute, 550 University Avenue, M5G 2A2, Toronto</wicri:regionArea><wicri:noRegion>Toronto</wicri:noRegion></affiliation></author></analytic><series><title level="j">Journal of NeuroEngineering and Rehabilitation</title><idno type="eISSN">1743-0003</idno><imprint><date when="2011">2011</date></imprint></series></biblStruct></sourceDesc></fileDesc><profileDesc><textClass></textClass></profileDesc></teiHeader><front><div type="abstract" xml:lang="en"><sec><title>Background</title><p>Stroke is the primary cause of adult disability. To support this large population in recovery, robotic technologies are being developed to assist in the delivery of rehabilitation. This paper presents an automated system for a rehabilitation robotic device that guides stroke patients through an upper-limb reaching task. The system uses a decision theoretic model (a partially observable Markov decision process, or POMDP) as its primary engine for decision making. The POMDP allows the system to automatically modify exercise parameters to account for the specific needs and abilities of different individuals, and to use these parameters to take appropriate decisions about stroke rehabilitation exercises.</p></sec><sec><title>Methods</title><p>The performance of the system was evaluated by comparing the decisions made by the system with those of a human therapist. A single patient participant was paired up with a therapist participant for the duration of the study, for a total of six sessions. Each session was an hour long and occurred three times a week for two weeks. During each session, three steps were followed: (A) after the system made a decision, the therapist either agreed or disagreed with the decision made; (B) the researcher had the device execute the decision made by the therapist; (C) the patient then performed the reaching exercise. These parts were repeated in the order of A-B-C until the end of the session. Qualitative and quantitative question were asked at the end of each session and at the completion of the study for both participants.</p></sec><sec><title>Results</title><p>Overall, the therapist agreed with the system decisions approximately 65% of the time. In general, the therapist thought the system decisions were believable and could envision this system being used in both a clinical and home setting. The patient was satisfied with the system and would use this system as his/her primary method of rehabilitation.</p></sec><sec><title>Conclusions</title><p>The data collected in this study can only be used to provide insight into the performance of the system since the sample size was limited. The next stage for this project is to test the system with a larger sample size to obtain significant results.</p></sec></div></front><back><div1 type="bibliography"><listBibl><biblStruct></biblStruct><biblStruct></biblStruct><biblStruct></biblStruct><biblStruct><analytic><author><name sortKey="Fasoli, Se" uniqKey="Fasoli S">SE Fasoli</name></author><author><name sortKey="Krebs, Hi" uniqKey="Krebs H">HI Krebs</name></author><author><name sortKey="Hogan, N" uniqKey="Hogan N">N Hogan</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Hidler, J" uniqKey="Hidler J">J Hidler</name></author><author><name sortKey="Nichols, D" uniqKey="Nichols D">D Nichols</name></author><author><name sortKey="Pelliccio, M" uniqKey="Pelliccio M">M Pelliccio</name></author><author><name sortKey="Brady, K" uniqKey="Brady K">K Brady</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Caplan, Lr" uniqKey="Caplan L">LR Caplan</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Reinkensmeyer, Dj" uniqKey="Reinkensmeyer D">DJ Reinkensmeyer</name></author><author><name sortKey="Kahn, Le" uniqKey="Kahn L">LE Kahn</name></author><author><name sortKey="Averbuch, M" uniqKey="Averbuch M">M Averbuch</name></author><author><name sortKey="Mckenna Cole, A" uniqKey="Mckenna Cole A">A McKenna-Cole</name></author><author><name sortKey="Schmit, Bd" uniqKey="Schmit B">BD Schmit</name></author><author><name sortKey="Rymer, Wz" uniqKey="Rymer W">WZ Rymer</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Lum, Ps" uniqKey="Lum P">PS Lum</name></author><author><name sortKey="Burgar, Cg" uniqKey="Burgar C">CG Burgar</name></author><author><name sortKey="Shor, Pc" uniqKey="Shor P">PC Shor</name></author><author><name sortKey="Majmundar, M" uniqKey="Majmundar M">M Majmundar</name></author><author><name sortKey="Van Der Loos, M" uniqKey="Van Der Loos M">M Van der Loos</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Amirabdollahian, F" uniqKey="Amirabdollahian F">F Amirabdollahian</name></author><author><name sortKey="Loureiro, R" uniqKey="Loureiro R">R Loureiro</name></author><author><name sortKey="Gradwell, E" uniqKey="Gradwell E">E Gradwell</name></author><author><name sortKey="Collin, C" uniqKey="Collin C">C Collin</name></author><author><name sortKey="Harwin, W" uniqKey="Harwin W">W Harwin</name></author><author><name sortKey="Johnson, G" uniqKey="Johnson G">G Johnson</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Krebs, Hi" uniqKey="Krebs H">HI Krebs</name></author><author><name sortKey="Hogan, N" uniqKey="Hogan N">N Hogan</name></author><author><name sortKey="Aisen, Ml" uniqKey="Aisen M">ML Aisen</name></author><author><name sortKey="Volpe, Bt" uniqKey="Volpe B">BT Volpe</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Ferraro, M" uniqKey="Ferraro M">M Ferraro</name></author><author><name sortKey="Palazzolo, Jj" uniqKey="Palazzolo J">JJ Palazzolo</name></author><author><name sortKey="Krol, J" uniqKey="Krol J">J Krol</name></author><author><name sortKey="Krebs, Hi" uniqKey="Krebs H">HI Krebs</name></author><author><name sortKey="Hogan, N" uniqKey="Hogan N">N Hogan</name></author><author><name sortKey="Volpe, Bt" uniqKey="Volpe B">BT Volpe</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Fasoli, Se" uniqKey="Fasoli S">SE Fasoli</name></author><author><name sortKey="Krebs, Hi" uniqKey="Krebs H">HI Krebs</name></author><author><name sortKey="Stein, J" uniqKey="Stein J">J Stein</name></author><author><name sortKey="Frontera, Wr" uniqKey="Frontera W">WR Frontera</name></author><author><name sortKey="Hughes, R" uniqKey="Hughes R">R Hughes</name></author><author><name sortKey="Hogan, N" uniqKey="Hogan N">N Hogan</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Macclellan, Lr" uniqKey="Macclellan L">LR MacClellan</name></author><author><name sortKey="Bradham, Dd" uniqKey="Bradham D">DD Bradham</name></author><author><name sortKey="Whitall, J" uniqKey="Whitall J">J Whitall</name></author><author><name sortKey="Volpe, B" uniqKey="Volpe B">B Volpe</name></author><author><name sortKey="Wilson, Pd" uniqKey="Wilson P">PD Wilson</name></author><author><name sortKey="Ohlhoff, J" uniqKey="Ohlhoff J">J Ohlhoff</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Lo, Al" uniqKey="Lo A">AL Lo</name></author><author><name sortKey="Guarino, Pd" uniqKey="Guarino P">PD Guarino</name></author><author><name sortKey="Richards, Lg" uniqKey="Richards L">LG Richards</name></author><author><name sortKey="Haselkorn, Jk" uniqKey="Haselkorn J">JK Haselkorn</name></author><author><name sortKey="Wittenberg, Gf" uniqKey="Wittenberg G">GF Wittenberg</name></author><author><name sortKey="Federman, Dg" uniqKey="Federman D">DG Federman</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Reinkensmeyer, D" uniqKey="Reinkensmeyer D">D Reinkensmeyer</name></author><author><name sortKey="Pang, C" uniqKey="Pang C">C Pang</name></author><author><name sortKey="Nessler, J" uniqKey="Nessler J">J Nessler</name></author><author><name sortKey="Painter, C" uniqKey="Painter C">C Painter</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Sucar, Le" uniqKey="Sucar L">LE Sucar</name></author><author><name sortKey="Leder, R" uniqKey="Leder R">R Leder</name></author><author><name sortKey="Reinkensmeyer, D" uniqKey="Reinkensmeyer D">D Reinkensmeyer</name></author><author><name sortKey="Hernandez, J" uniqKey="Hernandez J">J Hernández</name></author><author><name sortKey="Azcarate, G" uniqKey="Azcarate G">G Azcárate</name></author><author><name sortKey="Casta Eda, N" uniqKey="Casta Eda N">N Castañeda</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Qiu, Q" uniqKey="Qiu Q">Q Qiu</name></author><author><name sortKey="Ramirez, Da" uniqKey="Ramirez D">DA Ramirez</name></author><author><name sortKey="Saleh, S" uniqKey="Saleh S">S Saleh</name></author><author><name sortKey="Fluet, Gg" uniqKey="Fluet G">GG Fluet</name></author><author><name sortKey="Parikh, Hd" uniqKey="Parikh H">HD Parikh</name></author><author><name sortKey="Kelly, D" uniqKey="Kelly D">D Kelly</name></author><author><name sortKey="Adamovich, S" uniqKey="Adamovich S">S Adamovich</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Ju, Ms" uniqKey="Ju M">MS Ju</name></author><author><name sortKey="Lin, Cck" uniqKey="Lin C">CCK Lin</name></author><author><name sortKey="Lin, Dh" uniqKey="Lin D">DH Lin</name></author><author><name sortKey="Hwang, Is" uniqKey="Hwang I">IS Hwang</name></author><author><name sortKey="Chen, Sm" uniqKey="Chen S">SM Chen</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Erol, D" uniqKey="Erol D">D Erol</name></author><author><name sortKey="Mallapragada, V" uniqKey="Mallapragada V">V Mallapragada</name></author><author><name sortKey="Sarkar, N" uniqKey="Sarkar N">N Sarkar</name></author><author><name sortKey="Uswatte, G" uniqKey="Uswatte G">G Uswatte</name></author><author><name sortKey="Taub, E" uniqKey="Taub E">E Taub</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Marchal Crespo, L" uniqKey="Marchal Crespo L">L Marchal-Crespo</name></author><author><name sortKey="Reinkensmeyer, Dj" uniqKey="Reinkensmeyer D">DJ Reinkensmeyer</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Wolbrecht, E" uniqKey="Wolbrecht E">E Wolbrecht</name></author><author><name sortKey="Reinkensmeyer, Dj" uniqKey="Reinkensmeyer D">DJ Reinkensmeyer</name></author><author><name sortKey="Bobrow, Je" uniqKey="Bobrow J">JE Bobrow</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Barnes, M" uniqKey="Barnes M">M Barnes</name></author><author><name sortKey="Dobkin, B" uniqKey="Dobkin B">B Dobkin</name></author><author><name sortKey="Bogousslavsky, J" uniqKey="Bogousslavsky J">J Bogousslavsky</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Gillen, G" uniqKey="Gillen G">G Gillen</name></author><author><name sortKey="Burkhardt, A" uniqKey="Burkhardt A">A Burkhardt</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Lam, P" uniqKey="Lam P">P Lam</name></author><author><name sortKey="Hebert, D" uniqKey="Hebert D">D Hébert</name></author><author><name sortKey="Boger, J" uniqKey="Boger J">J Boger</name></author><author><name sortKey="Lacheray, H" uniqKey="Lacheray H">H Lacheray</name></author><author><name sortKey="Gardner, D" uniqKey="Gardner D">D Gardner</name></author><author><name sortKey="Apkarian, J" uniqKey="Apkarian J">J Apkarian</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Lovejoy, Ws" uniqKey="Lovejoy W">WS Lovejoy</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Kaelbling, Lp" uniqKey="Kaelbling L">LP Kaelbling</name></author><author><name sortKey="Littman, Ml" uniqKey="Littman M">ML Littman</name></author><author><name sortKey="Cassandra, Ar" uniqKey="Cassandra A">AR Cassandra</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Pineau, J" uniqKey="Pineau J">J Pineau</name></author><author><name sortKey="Montemerlo, M" uniqKey="Montemerlo M">M Montemerlo</name></author><author><name sortKey="Pollack, M" uniqKey="Pollack M">M Pollack</name></author><author><name sortKey="Roy, N" uniqKey="Roy N">N Roy</name></author><author><name sortKey="Thrun, S" uniqKey="Thrun S">S Thrun</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Hoey, J" uniqKey="Hoey J">J Hoey</name></author><author><name sortKey="Von Bertoldi, A" uniqKey="Von Bertoldi A">A von Bertoldi</name></author><author><name sortKey="Poupart, P" uniqKey="Poupart P">P Poupart</name></author><author><name sortKey="Mihailidis, A" uniqKey="Mihailidis A">A Mihailidis</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Pineau, J" uniqKey="Pineau J">J Pineau</name></author><author><name sortKey="Gordon, G" uniqKey="Gordon G">G Gordon</name></author><author><name sortKey="Thrun, S" uniqKey="Thrun S">S Thrun</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Dobkin, Bh" uniqKey="Dobkin B">BH Dobkin</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Poupart, P" uniqKey="Poupart P">P Poupart</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Spaan, Mtj" uniqKey="Spaan M">MTJ Spaan</name></author><author><name sortKey="Vlassis, N" uniqKey="Vlassis N">N Vlassis</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Gowland, C" uniqKey="Gowland C">C Gowland</name></author><author><name sortKey="Stratford, P" uniqKey="Stratford P">P Stratford</name></author><author><name sortKey="Ward, M" uniqKey="Ward M">M Ward</name></author><author><name sortKey="Moreland, J" uniqKey="Moreland J">J Moreland</name></author><author><name sortKey="Torresin, W" uniqKey="Torresin W">W Torresin</name></author><author><name sortKey="Van Hullenaar, S" uniqKey="Van Hullenaar S">S Van Hullenaar</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Lazo, Acgv" uniqKey="Lazo A">ACGV Lazo</name></author><author><name sortKey="Rathie, Pn" uniqKey="Rathie P">PN Rathie</name></author></analytic></biblStruct></listBibl></div1></back></TEI><pmc article-type="research-article"><pmc-dir>properties open_access</pmc-dir>
<front><journal-meta><journal-id journal-id-type="nlm-ta">J Neuroeng Rehabil</journal-id><journal-title-group><journal-title>Journal of NeuroEngineering and Rehabilitation</journal-title></journal-title-group><issn pub-type="epub">1743-0003</issn><publisher><publisher-name>BioMed Central</publisher-name></publisher></journal-meta><article-meta><article-id pub-id-type="pmid">21679457</article-id><article-id pub-id-type="pmc">3152889</article-id><article-id pub-id-type="publisher-id">1743-0003-8-33</article-id><article-id pub-id-type="doi">10.1186/1743-0003-8-33</article-id><article-categories><subj-group subj-group-type="heading"><subject>Research</subject></subj-group></article-categories><title-group><article-title>The development of an adaptive upper-limb stroke rehabilitation robotic system</article-title></title-group><contrib-group><contrib contrib-type="author" id="A1"><name><surname>Kan</surname><given-names>Patricia</given-names></name><xref ref-type="aff" rid="I1">1</xref><email>patricia.kan@utoronto.ca</email></contrib><contrib contrib-type="author" id="A2"><name><surname>Huq</surname><given-names>Rajibul</given-names></name><xref ref-type="aff" rid="I1">1</xref><email>rajibul.huq@utoronto.ca</email></contrib><contrib contrib-type="author" id="A3"><name><surname>Hoey</surname><given-names>Jesse</given-names></name><xref ref-type="aff" rid="I2">2</xref><email>jessehoey@computing.dundee.ac.uk</email></contrib><contrib contrib-type="author" id="A4"><name><surname>Goetschalckx</surname><given-names>Robby</given-names></name><xref ref-type="aff" rid="I2">2</xref><email>robby@computing.dundee.ac.uk</email></contrib><contrib contrib-type="author" corresp="yes" id="A5"><name><surname>Mihailidis</surname><given-names>Alex</given-names></name><xref ref-type="aff" rid="I1">1</xref><xref ref-type="aff" rid="I3">3</xref><xref ref-type="aff" rid="I4">4</xref><email>alex.mihailidis@utoronto.ca</email></contrib></contrib-group><aff id="I1"><label>1</label>Institute of Biomaterials and Biomedical Engineering, Rosebrugh Building, 164 College Street, Room 407, University of Toronto, Toronto, M5T 1P7, Canada</aff><aff id="I2"><label>2</label>School of Computing, University of Dundee, Dundee, DD1 4HN, UK</aff><aff id="I3"><label>3</label>Department of Occupational Science and Occupational Therapy, University of Toronto, 160-500 University Avenue, Toronto, M5G 1V7, Canada</aff><aff id="I4"><label>4</label>Toronto Rehabilitation Institute, 550 University Avenue, M5G 2A2, Toronto, Canada</aff><pub-date pub-type="collection"><year>2011</year></pub-date><pub-date pub-type="epub"><day>16</day><month>6</month><year>2011</year></pub-date><volume>8</volume><fpage>33</fpage><lpage>33</lpage><history><date date-type="received"><day>12</day><month>5</month><year>2010</year></date><date date-type="accepted"><day>16</day><month>6</month><year>2011</year></date></history><permissions><copyright-statement>Copyright ©2011 Kan et al; licensee BioMed Central Ltd.</copyright-statement><copyright-year>2011</copyright-year><copyright-holder>Kan et al; licensee BioMed Central Ltd.</copyright-holder><license license-type="open-access" xlink:href="http://creativecommons.org/licenses/by/2.0"><license-p>This is an Open Access article distributed under the terms of the Creative Commons Attribution License (<ext-link ext-link-type="uri" xlink:href="http://creativecommons.org/licenses/by/2.0">http://creativecommons.org/licenses/by/2.0</ext-link>), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.</license-p></license></permissions><self-uri xlink:href="http://www.jneuroengrehab.com/content/8/1/33"></self-uri><abstract><sec><title>Background</title><p>Stroke is the primary cause of adult disability. To support this large population in recovery, robotic technologies are being developed to assist in the delivery of rehabilitation. This paper presents an automated system for a rehabilitation robotic device that guides stroke patients through an upper-limb reaching task. The system uses a decision theoretic model (a partially observable Markov decision process, or POMDP) as its primary engine for decision making. The POMDP allows the system to automatically modify exercise parameters to account for the specific needs and abilities of different individuals, and to use these parameters to take appropriate decisions about stroke rehabilitation exercises.</p></sec><sec><title>Methods</title><p>The performance of the system was evaluated by comparing the decisions made by the system with those of a human therapist. A single patient participant was paired up with a therapist participant for the duration of the study, for a total of six sessions. Each session was an hour long and occurred three times a week for two weeks. During each session, three steps were followed: (A) after the system made a decision, the therapist either agreed or disagreed with the decision made; (B) the researcher had the device execute the decision made by the therapist; (C) the patient then performed the reaching exercise. These parts were repeated in the order of A-B-C until the end of the session. Qualitative and quantitative question were asked at the end of each session and at the completion of the study for both participants.</p></sec><sec><title>Results</title><p>Overall, the therapist agreed with the system decisions approximately 65% of the time. In general, the therapist thought the system decisions were believable and could envision this system being used in both a clinical and home setting. The patient was satisfied with the system and would use this system as his/her primary method of rehabilitation.</p></sec><sec><title>Conclusions</title><p>The data collected in this study can only be used to provide insight into the performance of the system since the sample size was limited. The next stage for this project is to test the system with a larger sample size to obtain significant results.</p></sec></abstract></article-meta></front><body><sec><title>Background</title><p>Stroke is the leading cause of physical disability and third leading cause of death in most countries around the world, including Canada [<xref ref-type="bibr" rid="B1">1</xref>] and the United States [<xref ref-type="bibr" rid="B2">2</xref>]. The consequences of stroke are devastating with approximately 75% of stroke sufferers being left with a permanent disability [<xref ref-type="bibr" rid="B3">3</xref>].</p><p>Research has shown that stroke rehabilitation can reduce the impairments and disabilities that are caused by stroke, and improve motor function, allowing stroke patients to regain much of their independence and quality of life. It is generally agreed that intensive, repetitive, and goal-directed rehabilitation improves motor function and cortical reorganization in stroke patients with both acute and long-term (chronic) impairments [<xref ref-type="bibr" rid="B4">4</xref>]. However, this recovery process is typically slow and labor-intensive, usually involving extensive interaction between one or more therapists and one patient. One of the main motivations for developing rehabilitation robotic devices is to automate interventions that are normally repetitive and physically demanding. These robots can provide stroke patients with intensive and reproducible movement training in time-unlimited durations, which can alleviate strain on therapists. In addition, these devices can provide therapists with accurate measures on patient performance and function (e.g. range of motion, speed, smoothness) during a therapeutic intervention, and also provide quantitative diagnosis and assessments of motor impairments such as spasticity, tone, and strength [<xref ref-type="bibr" rid="B5">5</xref>]. This technology makes it possible for a single therapist to supervise multiple patients simultaneously, which can contribute in the reduction of health care costs.</p><sec><title>Current upper-limb rehabilitation robotic devices</title><p>The upper extremities are typically affected more than the lower extremities after stroke [<xref ref-type="bibr" rid="B6">6</xref>]. Stroke patients with an affected upper-limb have difficulties performing many activities of daily living, such as reaching to grasp objects.</p><p>There have been several types of robotic devices designed to deliver upper-limb rehabilitation for people with paralyzed upper extremities. The Assisted Rehabilitation and Measurement (ARM) Guide [<xref ref-type="bibr" rid="B7">7</xref>] was designed to mimic the reaching motion. It consists of a single motor and chain drive that is used to move the user's hand along a linear constraint, which can be manually oriented in different angles to allow reaching in various directions. The ARM Guide implements a technique called "active assist therapy", in which its essential principle is to complete a desired movement for the user if they are unable to do so. The Mirror Image Movement Enabler (MIME) therapy system [<xref ref-type="bibr" rid="B8">8</xref>] consists of a six-degree of freedom (DOF) robot manipulator, which is attached to the orthosis supporting the user's affected arm. It applies forces to the limb during both unimanual and bimanual goal-directed movements in 3-dimensional (3D) space. Unilateral movements involve the robot moving or assisting the paretic limb towards a target in pre-programmed trajectories. The bimanual mode works in a slave configuration where the robot-assisted affected limb mirrors the unimpaired arm movements. The GENTLE/s system [<xref ref-type="bibr" rid="B9">9</xref>] is comprised of a commercially available 3-DOF robot, the HapticMASTER (FCS Robotics Inc.), which is attached to a wrist splint via a passive gimbal mechanism with 3-DOF. The gimbal allows for pronation/supination of the elbow as well as flexion and extension of the wrist. The seated user, whose arm is suspended from a sling to eliminate gravity effects, can perform reaching movements through interaction with the virtual environment on the computer screen. The rehabilitation robotic device that has received the most clinical testing is the Massachusetts Institute of Technology (MIT)-MANUS [<xref ref-type="bibr" rid="B10">10</xref>]. The MIT- MANUS consists of a 2-DOF robot manipulator that assists shoulder and elbow movements by moving the user's hand in the horizontal plane. Studies evaluating the effect of robotic therapy with the MIT-Manus in reducing chronic motor impairments show that there were statistically significant improvements in motor function [<xref ref-type="bibr" rid="B11">11</xref>-<xref ref-type="bibr" rid="B13">13</xref>]. The most recent study concluded that after nine months of robotic therapy, stroke patients with long-term impairments of the upper-limb improved in motor function compared with conventional therapy, but not with intensive therapy [<xref ref-type="bibr" rid="B14">14</xref>].</p><p>Recent work has attempted to make stroke rehabilitation exercises more relevant to real-life situations, by programming virtual reality games that mimic such situations (e.g. cooking, ironing, painting). The T-WREX system is one such attempt, an online Java-based set of exercises that can be combined with a stroke rehabilitation device such as the one described here [<xref ref-type="bibr" rid="B15">15</xref>]. Recent work has attempted to combine T-WREX with a non-invasive gesture exercise program based on computer vision. A user is observed with a camera, and his/her gestures are modeled and mapped into the T-WREX games. The user's progress can be monitored and reported to a therapist [<xref ref-type="bibr" rid="B16">16</xref>]. The work presented in [<xref ref-type="bibr" rid="B17">17</xref>] integrates virtual reality with robot assisted 3D haptic system for rehabilitation of children with hemiparetic cerebral palsy.</p><p>Researchers in the artificial intelligence community have started to design robot-assisted rehabilitation devices that implement artificial intelligence methods to improve upon the active assistance techniques found in the previous systems mentioned above. However, very few have been developed. An elbow and shoulder rehabilitation robot [<xref ref-type="bibr" rid="B18">18</xref>] was developed using a hybrid position/force fuzzy logic controller to assist the user's arm along predetermined linear or circular trajectories with specified loads. The robot helps to constrain the movements in the desired direction, if the user deviates from the predetermined path. Fuzzy logic was incorporated in the position and force control algorithms to cope with the nonlinear dynamics (i.e. uncertainty of the dynamics model of the user) of the robotic system to ensure operation for different users. An artificial neural network (ANN) based proportional-integral (PI) gain scheduling direct force controller [<xref ref-type="bibr" rid="B19">19</xref>] was developed to provide robotic assistance for upper extremity rehabilitation. The controller has the ability to automatically select appropriate PI gains to accommodate a wide range of users with varying physical conditions by training the ANN with estimated human arm parameters. The idea is to automatically tune the gains of the force controller based on the condition of each patient's arm parameters in order for it to apply the desired assistive force in an efficient and precise manner.</p><p>There exist several control approaches for robot assisted rehabilitation [<xref ref-type="bibr" rid="B20">20</xref>], however, most of them are devoted to modeling and prediction of the patients' motion trajectory and assisting them to complete the desired task. The work presented in [<xref ref-type="bibr" rid="B21">21</xref>] also proposes an adaptive system that provides minimum assistance to complete the desired task of the patients. While these robotic systems have shown promising results, none of them is able to provide an autonomous rehabilitation regime that accounts for the specific needs and abilities of each individual. Each user progresses in different ways and thus, exercises must be tailored to each individual differently. For example, the difficulty of an exercise should increase faster for those who are progressing well compared to those who are having trouble performing the exercise. The GENTLE/s system requires the user or therapist to constantly press a button in order for the system to be in operational mode [<xref ref-type="bibr" rid="B9">9</xref>]. It is imperative that a rehabilitation system operates with no or very little feedback as any direct input from the therapist (or user), such as setting a particular resistance level, prevents the user from performing the exercise uninterrupted. The system should be able to autonomously adjust different exercise parameters in accordance to each individual's needs. The rehabilitation systems discussed above also do not account for physiological factors, such as fatigue, which can have a significant impact on rehabilitation progress [<xref ref-type="bibr" rid="B22">22</xref>]. A system that can incorporate and estimate user fatigue can provide information as to when the user should take a break and rest, which may benefit rehabilitation progress.</p><p>The research described in this paper aims to fill these existing gaps by using stochastic modelling and decision theoretic reasoning to autonomously facilitate upper-limb reaching rehabilitation for moderate level stroke patients, tailor the exercise parameters for each individual, and estimate user fatigue. This paper will present a new controller that was developed based on a POMDP (partially observable Markov decision process), as well as early pilot data collected to show the efficacy of the new system.</p></sec></sec><sec><title>Rehabilitation system overview</title><p>The automated upper-limb stroke rehabilitation system consists of three main components: the exercise (Figure <xref ref-type="fig" rid="F1">1</xref>), the robotic system (Figure <xref ref-type="fig" rid="F2">2a</xref>), and the POMDP agent (Figure <xref ref-type="fig" rid="F2">2b</xref>). As the user performs the reaching exercise on the robot, data from the robotic system are used as input to the POMDP, which decides on the next action for the system to take.</p><fig id="F1" position="float"><label>Figure 1</label><caption><p><bold>The reaching exercise</bold>. Starting from an initial position (a), the reaching exercise consists of a forward extension of the arm until it reaches the final position (b), then the return path brings the arm back to the initial position.</p></caption><graphic xlink:href="1743-0003-8-33-1"></graphic></fig><fig id="F2" position="float"><label>Figure 2</label><caption><p><bold>Diagram of the reaching rehabilitation system</bold>. The reaching rehabilitation system consists of the robotic system (a) and POMDP agent (b). The robotic system automates the reaching exercise and captures compensatory events. The POMDP system is the decision-maker of the system.</p></caption><graphic xlink:href="1743-0003-8-33-2"></graphic></fig><sec><title>The exercise</title><p>A targeted, load-bearing, forward reaching exercise was chosen for this project. Discussions with experienced occupational and physical therapists (n = 7) in a large rehabilitation hospital (Toronto, Canada) identified that this is an area of rehabilitation that is in need of more efficient tools. Moreover, reaching is one of the most important abilities to possess, as it is the basic motion involved in many activities of daily living. Figure <xref ref-type="fig" rid="F1">1</xref> provides an overview of the reaching exercise. The reaching exercise is performed in the sagittal plane (aligned with the shoulder) and begins with a slight forward flexion of the shoulder, and extension of the elbow and wrist (Figure <xref ref-type="fig" rid="F1">1a</xref>). Weight is translated through the heel of the hand as it is pushed forward in the direction indicated by the arrow, until it reaches the final position (Figure <xref ref-type="fig" rid="F1">1b</xref>). The return path brings the arm back to the initial position. Therapists usually apply resistive forces (to emulate load- or weight-bearing) during the reaching exercise to strengthen the triceps and scapula musculature, which will help to provide postural support and anchoring for other body movements [<xref ref-type="bibr" rid="B23">23</xref>]. It is important to note that a proper reaching exercise is performed with control (e.g. no deviation from the straight path) and without compensation (e.g. trunk rotation, shoulder abduction/internal rotation).</p><p>The general progression during conventional reaching rehabilitation is to gradually increase target distance, and then to increase the resistance level, as indicated by one of the consulting therapists on this project. If patients are showing signs of fatigue during the exercise, therapists will typically let patients rest for a few minutes and then continue with the therapy session. The goal is to have patients successfully reach the furthest target at maximum resistance, while performing the exercise with control and proper posture.</p></sec><sec><title>Robotic system</title><p>A novel robotic system (Figure <xref ref-type="fig" rid="F2">2a</xref>) was designed to automate the reaching exercise as well as to capture any compensatory events. The system is comprised of three main components: the robotic device, which emulates the load-bearing reaching exercise with haptic feedback, the postural sensors, which identify abnormalities in the upper extremities during the exercise, and the virtual environment, which provides the user with visual feedback of the exercise on a computer monitor.</p><p>The robotic device, as detailed in [<xref ref-type="bibr" rid="B24">24</xref>] and shown in Figure <xref ref-type="fig" rid="F3">3</xref>, was built by Quanser Inc., a robotics company in Toronto. It features a non-restraining platform for better usability and freedom of movement, and has two degrees of freedom, which allow the reaching exercise to be performed in 2D space. The robotic device also incorporates haptic technology, which provides feedback through sense of touch. For the purpose of this research, the haptic device provided resistance and boundary guidance for the user during the exercise, which was performed only in 2D space (in the horizontal plane parallel to the floor). Encoders in the end-effector of the robotic device provide data to indicate hand position and shoulder abduction/internal rotation (i.e. compensation) during the exercise.</p><fig id="F3" position="float"><label>Figure 3</label><caption><p><bold>Actual robotic rehabilitation device</bold>. The robotic rehabilitation device features a non-restraining platform and allows the reaching exercise to be performed in 3D space.</p></caption><graphic xlink:href="1743-0003-8-33-3"></graphic></fig><p>The unobtrusive trunk sensors (Figure <xref ref-type="fig" rid="F4">4</xref>) provide data to indicate trunk rotation compensation. The trunk sensors are comprised of three photoresistors taped to the back of a chair, each in one of three locations: the lower back, lower left scapula, and lower right scapula. The detection of light during the exercise indicates trunk rotation, as it means a gap is present between the chair and user. Finally, the virtual environment provides the user with visual feedback on hand position and target location during the exercise. The reaching exercise is represented in the form of a 2D bull's eye game. The goal of the game is for the user to move the robot end-effector, which corresponds to the cross-tracker in the virtual environment, to the bull's eye target. The rectangular box is the virtual (haptic) boundary, which keeps the cross-tracker within those walls during the exercise.</p><fig id="F4" position="float"><label>Figure 4</label><caption><p><bold>Trunk photoresistor sensors</bold>. The trunk photoresistor sensors are placed in three locations: lower back, lower left scapula, and lower right scapula (a). The detection of light indicates trunk rotation compensation (b).</p></caption><graphic xlink:href="1743-0003-8-33-4"></graphic></fig></sec><sec><title>POMDP agent</title><p>The POMDP agent (Figure <xref ref-type="fig" rid="F2">2b</xref>) is the decision-maker of the system. Observation data from the robotic device is passed to a state estimator that estimates the progress of the user as a probability distribution over the possible states, known as a belief state. A policy then maps the belief state to an action for the system to execute, which can be either setting a new target position and resistance level or stopping the exercise. The goal of the POMDP agent is to help patients regain his/her maximum reaching distance at the most difficult level of resistance, while performing the exercises with control and proper posture.</p></sec></sec><sec><title>Partially observable Markov decision process</title><p>A POMDP is a decision-theoretic model that provides a natural framework for modeling complex planning problems with partial observability, uncertain action effects, incomplete knowledge of the state of the environment, and multiple interacting objectives. POMDPs are defined by: a finite set of world states <italic>S</italic>; a finite set of actions <italic>A</italic>; a finite set of observations <italic>O</italic>; a transition function <italic>T </italic>: <italic>S × A </italic>→ ∏(<italic>S</italic>), where ∏(<italic>S</italic>) denotes a probability distribution over states <italic>S</italic>, and P(<italic>s'</italic>|<italic>s</italic>,<italic>a</italic>) denotes the probability of transition from state <italic>s </italic>to <italic>s' </italic>when action <italic>a </italic>is performed; an observation function <italic>Z </italic>: <italic>S × A </italic>→ ∏(<italic>O</italic>), with P(<italic>o</italic>|<italic>a</italic>,<italic>s'</italic>) denoting the probability of observing <italic>o </italic>after performing action <italic>a </italic>and transiting to state <italic>s'</italic>; and a reward function <italic>R </italic>: <italic>S × A × 0 </italic>→ ℝ, with R(<italic>s</italic>,<italic>o,a</italic>) denoting the expected reward or cost (i.e. negative reward) incurred after performing action <italic>a </italic>and observing <italic>o </italic>in state <italic>s</italic>.</p><p>The POMDP agent is used to find a policy (i.e. course of action) that maximizes the expected discounted sum of rewards attained by the system over an infinite horizon, to monitor beliefs about the system state in real time, and to use the computed policy to decide which actions to take based on the belief states. For an overview of POMDPs, refer to [<xref ref-type="bibr" rid="B25">25</xref>,<xref ref-type="bibr" rid="B26">26</xref>].</p><sec><title>Examples of POMDPs in real-world applications</title><p>An increasing number of researchers in various fields are becoming interested in the application of POMDPs because they have shown promise in solving real-world problems.</p><p>Researchers at Carnegie Mellon University used a POMDP to model the high-level controller for an intelligent robot, Nursebot, designed to assist elderly individuals with mild cognitive and physical impairments in their daily activities such as taking medications, attending appointments, eating, drinking, bathing, and toileting [<xref ref-type="bibr" rid="B27">27</xref>]. Using variables such as the robot location, the user's location, and the user's status, the robot would decide whether to take an action, to provide the user a reminder or to guide the user where to move. By maintaining an accurate model of the user's daily plans and tracking his/her execution of the plans by observation, the robot could adapt to the user's behavior and take decisions about whether and when it was most appropriate to issue reminders.</p><p>A POMDP model was also used in a guidance system to assist people with dementia during the handwashing task [<xref ref-type="bibr" rid="B28">28</xref>]. By tracking the positions of the user's hands and towel with a camera mounted above the sink, the system could estimate the progress of the user during the handwashing task and provide assistance with the next step, if needed. Assistance was given in the form of verbal and/or visual prompts, or through the enlistment of a human caregiver's help. An important feature of this system is the ability to estimate and adapt to user states such as awareness, responsiveness, and overall dementia level which affect the amount of assistance given to the user during the handwashing activity.</p></sec><sec><title>Justification for using a POMDP to model reaching rehabilitation</title><p>Classical planning generally consists of agents which operate in environments that are fully observable, deterministic, static, and discrete. Although these techniques can solve increasingly large state-space problems, they are not suitable for most robotic applications, such as the reaching task in upper-limb rehabilitation, as they usually have partial observability, stochastic actions, and dynamic environments [<xref ref-type="bibr" rid="B29">29</xref>]. Planning under uncertainty aims to improve robustness by factoring in the types of uncertainties that can occur. A POMDP is perhaps the most general representation for (single-agent) planning under uncertainty. It surpasses other techniques in terms of representational power because it can combine many important aspects for planning under uncertainty as described below.</p><p>In reality, the state of the world cannot be known with certainty due to inaccurate measurements from noisy and imperfect sensors, or instances where observations may be impossible and inferences must be made, such as the fatigue state of the patient. POMDPs can handle this uncertainty in state observability by expressing the state of the world as a belief state - the probability distribution over all possible states of the world - rather than actual world states. By capturing this uncertainty in the model, the POMDP has the ability to make better decisions than fully observable techniques. For example, the reaching rehabilitation system does not consist of physical sensors that can detect user fatigue. By capturing observations in user compensation and control, POMDPs can use this information to infer or estimate how fatigued the user is. Fully observable methods cannot capture user fatigue in this way since it is impossible to observe fatigue, unless it is physically captured such as using electrical stimulation to measure muscle contractions [<xref ref-type="bibr" rid="B30">30</xref>]. However, these techniques are invasive and may not even guarantee full observability of the world state since sensor measurements may be inaccurate.</p><p>The reaching exercise is a stochastic (dynamic) decision problem where there is uncertainty in the outcome of actions and the environment is always changing. Thus, choosing a particular action at a particular state does not always produce the same results. Instead, the action has a random chance of producing a specific result with a known probability. POMDPs can account for the realistic uncertainty of action effects in the decision process through its transition probabilities and reward function. By knowing the probabilities and rewards of the outcomes of taking an action in a specific state, the POMDP agent can estimate the likelihood of future outcomes to determine the optimal course of action to take in the present. This ability to consider the future effects of current actions allows the POMDP to trade off between alternative ways to satisfy a goal and plan for multiple interacting goals. It also allows the agent to build a policy that is capable of handling unexpected outcomes more robustly than many classical planners.</p><p>Different stroke patients progress in different ways during rehabilitation depending on their ability and state of health. It is imperative for the rehabilitation system to be able to tailor and adapt to each individual's needs and abilities over time. POMDPs have the capability of incorporating user abilities autonomously in real-time by keeping track of which actions have been observed to be the most effective in the past. For example, the POMDP may decide to keep the target closer for a longer period of time for patients who are progressing slowly, but may increase the target location further at a quicker rate for those who are progressing faster.</p><p>Since one of the objectives of a rehabilitation robotic system is to reduce health care costs by having one therapist supervise multiple stroke patients simultaneously, it is imperative to design the system in which no or very little explicit feedback from the therapist is required during the therapy session. The system must be able to effectively guide the patient during the reaching exercise without the need for explicit input (e.g. a button press to set a particular resistance level), as any direct input from the therapist would be time consuming and prevent the user from intensive repetition. POMDPs have this ability to operate autonomously through the estimation of states and then automatically making decisions. For eventually practising therapy in the home setting, it is especially important that the system does not require any explicit feedback since no therapist will be present.</p></sec></sec><sec><title>POMDP model</title><p>The specific POMDP model for the reaching exercise is described as follows.</p><sec><title>Actions, variables, and observations</title><p>Figure <xref ref-type="fig" rid="F5">5</xref> shows the POMDP model as a dynamic Bayesian network (DBN). There are 10 possible actions the system can take. These are comprised of nine actions of which each is a different combination of setting a target distance <italic>d</italic>∈{<italic>d1</italic>,<italic>d2</italic>,<italic>d3</italic>}, and resistance level <italic>r</italic>∈{<italic>none</italic>,<italic>min</italic>,<italic>max</italic>}, and one action to stop the exercise when the user is fatigued.</p><fig id="F5" position="float"><label>Figure 5</label><caption><p><bold>POMDP model as a DBN</bold>. The POMDP model consists of 7 state variables, 10 actions, and 3 observation variables. The arrows indicate how the variables at time <italic>t-1 </italic>influence those at time <italic>t</italic>. The variable <italic>fatigue </italic>is abbreviated as <italic>fat</italic>.</p></caption><graphic xlink:href="1743-0003-8-33-5"></graphic></fig><p>Variables were chosen to meaningfully capture the aspects of the reaching task that the system would require in order to effectively guide a stroke patient during the exercise. Unique combinations of instantiations of these variables represent all the different possible states of the rehabilitation exercise that the system could be in. The following variables were chosen to represent the exercise:</p><p>• <italic>fatigue </italic>= {<italic>yes</italic>,<italic>no</italic>} describes the user's level of fatigue</p><p>• <italic>n</italic>(<italic>r</italic>) = {<italic>none</italic>,<italic>d1</italic>,<italic>d2</italic>,<italic>d3</italic>} describes the range (or ability) of the user at a particular resistance level, <italic>r</italic>∈{<italic>none</italic>,<italic>min</italic>,<italic>max</italic>}. The range is defined as the furthest target distance, <italic>d</italic>∈{<italic>d1</italic>,<italic>d2</italic>,<italic>d3</italic>}, the user is able to reach at a particular resistance. For example, if <italic>r </italic>= <italic>min </italic>and the furthest target the user can reach is <italic>d </italic>= <italic>d2</italic>, then the user's range is <italic>n</italic>(<italic>min</italic>)=<italic>d2</italic>.</p><p>• <italic>stretch </italic>= {<italic>+9</italic>,<italic>+8</italic>,<italic>+7</italic>,<italic>+6</italic>,<italic>+5</italic>,<italic>+4</italic>,<italic>+3</italic>,<italic>+2</italic>,<italic>+1</italic>,<italic>0</italic>,<italic>-1</italic>,<italic>-2</italic>} describes the amount the system is asking the user to go beyond their current range. It is a deterministic function of the system's choice of resistance level (<italic>a</italic><sub><italic>r</italic></sub>) and distance (<italic>a</italic><sub><italic>d</italic></sub>), which measures how much this choice is going to push a user beyond their range, and is computed as follows:<disp-formula id="bmcM1"><label>(1)</label><graphic xlink:href="1743-0003-8-33-i1.gif"></graphic></disp-formula></p><p>where <italic>r </italic>indexes the resistance level (with <italic>1 </italic>= <italic>none</italic>, <italic>2 </italic>= <italic>min</italic>, <italic>3 </italic>= <italic>max</italic>), <italic>a</italic><sub><italic>r</italic></sub><italic>,a</italic><sub><italic>d</italic></sub>∈{<italic>1</italic>,<italic>2</italic>,<italic>3</italic>} index the resistance level and distance set by the system, and <italic>n</italic><sub><italic>r</italic></sub>∈{<italic>0</italic>,<italic>1</italic>,<italic>2</italic>,<italic>3</italic>} indexes the range at <italic>r</italic>.</p><p>• <italic>learnrate </italic>= {<italic>lo</italic>,<italic>med</italic>,<italic>hi</italic>} describes how quickly the user is progressing during the exercise</p><p>The observations were chosen as follows:</p><p>• <italic>ttt </italic>= {<italic>none</italic>,<italic>slow</italic>,<italic>norm</italic>} describes the time it takes the user to reach the target</p><p>• <italic>ctrl </italic>= {<italic>none</italic>,<italic>min</italic>,<italic>max</italic>} describes the user's control level by their ability to stay on the straight path</p><p>• <italic>comp </italic>= {<italic>yes</italic>,<italic>no</italic>} describes any compensatory actions (i.e. improper posture) performed</p><p>Note that, although the observations are fully observable, the states are still not known with certainty since the fatigue, user range, stretch, and learning rate variables are unobservable and must be estimated.</p></sec><sec><title>Dynamics</title><p>The dynamics of all variables were specified manually using simple parametric functions of <italic>stretch </italic>and the user's <italic>fatigue. </italic>The functions relating <italic>stretch </italic>and <italic>fatigue </italic>levels to user performance are called <italic>pace functions</italic>. The pace function, <italic>φ</italic>, is a function of the stretch, <italic>s</italic>, and fatigue, <italic>f</italic>, and is a sigmoid function defined as follows:<disp-formula id="bmcM2"><label>(2)</label><graphic xlink:href="1743-0003-8-33-i2.gif"></graphic></disp-formula></p><p>where <italic>m </italic>is the mean stretch (the value of stretch for which the function <italic>φ </italic>is 0.5 when the user is not fatigued), <italic>m</italic>(<italic>f</italic>) is a shift function that is dependent on the user's fatigue level (e.g. 0 if the user is not fatigued), and <italic>σ</italic><sub><italic>s </italic></sub>is the slope of the pace function. There is one such pace function for each variable, and the value of the pace function at a particular stretch and fatigue level gives the probability of the variable in question being true in the following time step. Figure <xref ref-type="fig" rid="F6">6</xref> shows an example of pace function for <italic>comp = yes</italic>. It shows that when the user is not fatigued and the system sets a target with a stretch of 3 (upper pace limit), the user might have a 90% chance to compensate. However, if the stretch is -1 (lower pace limit), then this chance might decrease to 10%. The pace limits decrease when the user is fatigued (at the same probability). In other words, the user is more likely to compensate when fatigued.</p><fig id="F6" position="float"><label>Figure 6</label><caption><p><bold>Example pace function</bold>. This is an example pace function for <italic>comp </italic>= <italic>yes</italic>. It shows the upper and lower pace limits, and the pace function for each condition of <italic>fatigue </italic>(abbreviated as <italic>fat</italic>).</p></caption><graphic xlink:href="1743-0003-8-33-6"></graphic></fig><p>The detailed procedure of specifying <italic>m</italic>, <italic>σ</italic><sub><italic>s</italic></sub>, and <italic>m</italic>(<italic>f</italic>) has been described in <italic>Additional file </italic><xref ref-type="supplementary-material" rid="S1">1</xref> - <italic>Pace function parameters</italic>.</p><p>In the current model, the ranges <italic>n</italic>(<italic>r</italic>) were modeled separately, although they could also use the concept of pace functions. The dynamics for the ranges basically state that setting targets at or just above a user's range will cause their range to increase slowly, but less so if the user is fatigued. If a user's range is at <italic>d3 </italic>for a particular resistance, then practicing at that distance and resistance will increase their range at the next higher resistance from <italic>none </italic>to <italic>d1</italic>. The dynamics also includes constraints to ensure that ranges at higher resistances are always less than or equal to those at lower resistances. Finally, the dynamics of range include a dependency on the learning rate (<italic>learnrate</italic>): higher learning rates cause the ranges to increase more quickly.</p></sec><sec><title>Rewards and computation</title><p>The reward function was constructed to motivate the system to guide the user to exercise at maximum target distance and resistance level, while performing the task with maximum control and without compensation. Thus, the system was given a large reward for getting the user to reach the furthest target distance (<italic>d </italic>= <italic>d3</italic>) at maximum resistance (<italic>r </italic>= <italic>max</italic>). Smaller rewards were given when targets were set at or above the user's current range (i.e. when <italic>stretch </italic>> = 0), and when the user was performing well (i.e. <italic>ttt </italic>= <italic>norm</italic>, <italic>ctrl </italic>= <italic>max</italic>, <italic>comp </italic>= <italic>no</italic>, and <italic>fatigue </italic>= <italic>no</italic>). However, no reward was given when the user was fatigued, failed to reach the target, had no control, or showed signs of compensation during the exercise. Please see Additional file <xref ref-type="supplementary-material" rid="S2">2</xref> for the complete reward function of the model.</p><p>The POMDP model had 82,944 possible states. The size of this reaching rehabilitation model renders optimal solutions intractable, thus, an approximation method was used. This approximation technique exploits the structure of the large POMDP by first representing the model using algebraic decision diagrams (ADDs) and then employing a randomized point-based value iteration algorithm [<xref ref-type="bibr" rid="B31">31</xref>], which is based on the Perseus algorithm [<xref ref-type="bibr" rid="B32">32</xref>] with a bound on the size of the value function. The model was sampled with a set of 3,000 belief points that were generated through random simulation starting from 20 different initial belief states: one for every range possibility. The POMDP was solved on a dual AMD Opteron™ (2.4 GHz) CPU using a bound of 150 linear value functions and 150 iterations in approximately 13.96 hours.</p></sec><sec><title>Simulation</title><p>A simulation program was developed in MATLAB<sup>® </sup>(before user trials) to determine how well the model was performing in real-time. The performance of the POMDP model was subjectively rated by the researcher and focused on whether the system was making decisions in accordance to conventional reaching rehabilitation, which was: (i) gradually increasing target distance first, then resistance level as the user performed well (i.e. reached target in normal time, had maximum control, and did not compensate), and (ii) increasing the rate of fatigue if the user was not performing well (i.e. failed to reach the target, had no control, or compensated).</p><p>The simulation began with an initial belief state. The POMDP then decided on an action for the system to take, which was predetermined by the policy. Observation data was manually entered and a new belief state was computed. This cycle continued until the system stopped the exercise because the user was determined to be fatigued. Before the next cycle occurred, the simulation program reset the fatigue variable (i.e. user is un-fatigued after resting) and the user ranges were carried over.</p><p>Simulations performed on this model seemed to follow that of conventional reaching rehabilitation. During simulation, the POMDP slowly increased the target distance and resistance level when the user successfully reached the target in normal time, had maximum control, and did not compensate. However, once the user started to lose control, compensated, or had trouble reaching the target, the POMDP increased its belief that the user was fatigued and stopped the exercise to allow the user to rest. The following two examples illustrate the performance of the POMDP model.</p><p>Example 1 assumes that the user is able to reach the maximum target (<italic>d </italic>= <italic>d3</italic>) at the maximum resistance level (<italic>r </italic>= <italic>max</italic>), but then slowly starts to compensate after several repetitions. The initial belief state (Figure <xref ref-type="fig" rid="F7">7</xref>) assumes that the user's range at both zero and minimum resistance (i.e. <italic>n</italic>(<italic>none</italic>) and <italic>n</italic>(<italic>min</italic>)) is likely to be <italic>d3</italic>, and the user's range at maximum resistance (<italic>n</italic>(<italic>max</italic>)) is likely to be <italic>d1</italic>. In addition, the initial belief state assumes that the user is not fatigued with a 95% probability. From this belief state, the POMDP sets the first action to be <italic>d </italic>= <italic>d1 </italic>and <italic>r </italic>= <italic>max</italic>. According to the assumption, the user successfully reaches this target in normal time, with maximum control, and with no compensation. In the next five time steps, the POMDP sets the target at <italic>d </italic>= <italic>d2 </italic>and then increases it to <italic>d </italic>= <italic>d3</italic>, assuming the user successfully reaches each target with maximum control and no compensation. Here, the user's fatigue level has increased slowly from approximately 5% to 20% due to repetition of the exercise. Now, during the next time step when the POMDP decides to set the target at <italic>d </italic>= <italic>d3 </italic>again, the user compensates but is still able to reach the target with maximum control. Figure <xref ref-type="fig" rid="F8">8</xref> shows the updated belief state. The fatigue level has jumped to about 40% due to user compensation. The POMDP sets the same target during the next time step and the user compensates once more. This time, the POMDP decides to stop the exercise because it believes the user is fatigued due to performing compensatory movements for two consecutive times. For the complete simulation, please see <italic>Additional file </italic><xref ref-type="supplementary-material" rid="S3">3</xref> - <italic>POMDP Simulation Example 1</italic>.</p><fig id="F7" position="float"><label>Figure 7</label><caption><p><bold>Initial POMDP belief state of example 1</bold>. This figure shows the initial belief state of <italic>n</italic>(<italic>r</italic>), <italic>stretch</italic>, <italic>fatigue </italic>(abbreviated as <italic>fat</italic>), and <italic>learnrate</italic>. The POMDP sets the target at <italic>d </italic>= <italic>d1 </italic>and resistance at <italic>r </italic>= <italic>max</italic>. The user reaches the target with <italic>ttt </italic>= <italic>norm</italic>, <italic>ctrl </italic>= <italic>max</italic>, and <italic>comp </italic>= <italic>no</italic>.</p></caption><graphic xlink:href="1743-0003-8-33-7"></graphic></fig><fig id="F8" position="float"><label>Figure 8</label><caption><p><bold>Updated POMDP belief state of example 1</bold>. This figure shows the updated belief state of <italic>n</italic>(<italic>r</italic>), <italic>stretch</italic>, <italic>fatigue </italic>(abbreviated as <italic>fat</italic>), and <italic>learnrate </italic>after the user compensates for the first time. The POMDP sets the target at <italic>d </italic>= <italic>d3 </italic>and resistance at <italic>r </italic>= <italic>max</italic>. The user reaches the target with <italic>ttt </italic>= <italic>norm</italic>, <italic>ctrl </italic>= <italic>max</italic>, and <italic>comp </italic>= <italic>yes</italic>.</p></caption><graphic xlink:href="1743-0003-8-33-8"></graphic></fig><p>In the second simulation example, the user is assumed to have trouble reaching the maximum target, <italic>d </italic>= <italic>d3</italic>, at zero resistance, <italic>r </italic>= <italic>none</italic>. The simulation starts with the initial belief state (shown in Figure <xref ref-type="fig" rid="F9">9</xref>), which assumes that the user's range at each resistance (i.e. <italic>n</italic>(<italic>none</italic>), <italic>n</italic>(<italic>min</italic>), and <italic>n</italic>(<italic>max</italic>)) is likely to be <italic>none</italic>, and that the user is not fatigued with a 95% probability. The POMDP slowly increases the target distance from <italic>d1</italic>, to <italic>d2</italic>, and then to <italic>d3 </italic>while keeping at the same resistance level (<italic>r </italic>= <italic>none</italic>) when the user successfully reaches the target in normal time, with maximum control, and with no compensation. However, at <italic>d </italic>= <italic>d3 </italic>the user fails to reach the target (i.e. <italic>ttt </italic>= <italic>none</italic>), has minimum control (<italic>ctrl </italic>= <italic>min</italic>), and does not compensate (<italic>comp </italic>= <italic>no</italic>). The updated belief state is shown in Figure <xref ref-type="fig" rid="F10">10</xref>, where the fatigue level jumped from about 10% to 25% due to the failure in reaching target. After the user failed to reach <italic>d3</italic>, the POMDP decides to keep the same target at <italic>d3 </italic>since <italic>stretch </italic>is about 75% likely to be 0 (i.e. at the user's range). Again, the user fails to reach the target with minimum control and no compensation and the level of fatigue increased to about 40%. The POMDP decides to stop the exercise when the user again failed to reach d3 and performed a compensatory movement. Hence, the fatigue level changed to about 60%. For the complete simulation, please see <italic>Additional file </italic><xref ref-type="supplementary-material" rid="S4">4</xref> - <italic>POMDP Simulation Example 2</italic>.</p><fig id="F9" position="float"><label>Figure 9</label><caption><p><bold>Initial POMDP belief state of example 2</bold>. This figure shows the initial belief state of <italic>n</italic>(<italic>r</italic>), <italic>stretch</italic>, <italic>fatigue </italic>(abbreviated as <italic>fat</italic>), and <italic>learnrate</italic>. The POMDP sets the target at <italic>d </italic>= <italic>d1 </italic>and resistance at <italic>r </italic>= none. The user reaches the target with <italic>ttt </italic>= <italic>norm</italic>, <italic>ctrl </italic>= <italic>max</italic>, and <italic>comp </italic>= <italic>no</italic>.</p></caption><graphic xlink:href="1743-0003-8-33-9"></graphic></fig><fig id="F10" position="float"><label>Figure 10</label><caption><p><bold>Updated POMDP belief state of example 2</bold>. This figure shows the updated belief state of <italic>n</italic>(<italic>r</italic>), <italic>stretch</italic>, <italic>fatigue </italic>(abbreviated as <italic>fat</italic>), and <italic>learnrate </italic>when the user failed to reach the target at <italic>d </italic>= <italic>d3</italic>. The POMDP resets the target at <italic>d </italic>= <italic>d3 </italic>and resistance at <italic>r </italic>= <italic>none</italic>. The user reaches the target with <italic>ttt </italic>= <italic>norm</italic>, <italic>ctrl </italic>=<italic>max</italic>, and <italic>comp </italic>= <italic>no</italic>.</p></caption><graphic xlink:href="1743-0003-8-33-10"></graphic></fig></sec></sec><sec><title>Pilot Study - Efficacy of POMDP</title><p>A pilot study was conduced with therapists and stroke patients to evaluate the efficacy of the POMDP agent - i.e. the correctness of the decisions being made by the system.</p><sec><title>Participants</title><p>Due to a delay in receiving ethics approval, only one therapist and one patient were recruited for the study. As such, several simulations were also run (as previously described and presented later in this section) to help draw conclusions regarding the efficacy of the POMDP. The therapist was a physical therapist with more than nine years of experience in post-acute upper-limb stroke rehabilitation, and was fluent in English. The patient was right-side hemiparetic, had a stroke onset of 227 days (7 months and 14 days) before enrolment, scored 4 on the arm section of the Chedoke-McMaster Stroke Assessment (CMSA) Scale [<xref ref-type="bibr" rid="B33">33</xref>], was able to move to some degree but still had impaired movements as determined by their therapist, and could understand and respond to simple instructions.</p></sec><sec><title>Method</title><p>The patient participant was paired up with the therapist participant for the duration of the study. Each session lasted for approximately one hour and was completed three times a week for two weeks.</p><p>For each session, the therapist brought the patient to the testing room. The patient participant was seated on a regular, straight-back chair positioned to the left of the robotic device. The therapist was responsible for adjusting the position of the chair, placing the trunk sensors at the appropriate spots (lower back, lower left scapula, and lower right scapula), and adjusting the height of the robot to ensure that the end-effector was correctly positioned in the sagittal plane of the patient's right shoulder. When both participants were ready to begin, the researcher powered on the robotic device and started the computer programs that controlled the POMDP agent, robotic device, and virtual environment. The patient was asked to place his/her hand on the end-effector, which was secured with a comfortable strap, and when ready, the researcher set the initial belief state of the POMDP and started the exercise.</p><p>The exercise was performed in three parts: (A) after the POMDP made a decision (i.e. to set the target position and resistance level, or to stop the exercise) the therapist either agreed or disagreed with the decision made; (B) the researcher had the device either execute the decision made by the POMDP if the therapist agreed or execute the decision made by the therapist if the therapist disagreed; and (C) the patient then performed the reaching exercise by trying to reach the target on the computer screen. These parts were repeated in the order of A-B-C until the end of the session.</p><p>Questions were asked at the end of each session and at the completion of the study for both participants. The questionnaire for the therapist participant was designed to focus on rating the decision-making strategy of the POMDP agent. For the patient participant, the questionnaire focused on gathering feedback with respect to their satisfaction in using such a robotic system. Both questionnaires consisted of quantitative and qualitative questions for statistical analysis and to provide insight into future design improvements, respectively. A four-point Likert scale was used for each quantitative question, with 1 representing complete disagreement and 4 representing complete agreement.</p></sec></sec><sec><title>Results and discussion</title><p>The small sample size of the study limited the use of hypothesis testing to interpret the data. Thus, the data collected in the study from one therapist and one patient can only provide insight into the performance of the system. A more detailed study will be completed in the spring of 2010.</p><sec><title>Agreement of POMDP decisions</title><p>Every decision made by both the POMDP and therapist was decomposed into three separate decisions: 1) the distance to set the target, 2) the level to set the resistance, and 3) whether or not to stop the exercise. The level of agreement by the therapist to the decisions made by the POMDP was calculated based on the three separate decisions as described above. A point of agreement would be given if the therapist set the same target distance as the POMDP, set the same resistance level as the POMDP, or agreed with the POMDP to stop the exercise or not. Figure <xref ref-type="fig" rid="F11">11</xref> shows the percentage of agreement over all sessions. Note that there were 636 state transitions (i.e. total number of trials) and 1,154 decisions made during the study.</p><fig id="F11" position="float"><label>Figure 11</label><caption><p><bold>Percentage of therapist agreement with POMDP</bold>. This figure shows the percentage of therapist agreement with the decisions made by the POMDP on target distance, resistance level, and stopping the exercise, as well as the overall performance of the system, over all sessions.</p></caption><graphic xlink:href="1743-0003-8-33-11"></graphic></fig><p>The therapist agreed with both the target distance and resistance level decisions made by the POMDP approximately 94% and 90% of the time, respectively, during the study (shown in Figure <xref ref-type="fig" rid="F9">9</xref>). Most of this agreement was with the POMDP repeatedly setting the target distance at <italic>d3 </italic>and the resistance at <italic>max</italic>. Since the patient was able to reach this setting within the first session with proper posture and control, the POMDP continued to make this decision as it was given large rewards for getting the user to reach the furthest target at maximum resistance. The therapist generally agreed with these decisions, as she wanted the patient to work on strengthening. It is important to note that this was a problem in the experimental design, where the mapping from the system resistance levels to the actual physical resistance in the device was not tuned properly for the user in the study. Before the trials began, therapists tested the system and concluded that the resistance levels were sufficient for moderate-level stroke patients. However, for future trials, initializing the resistance levels for different users should be properly developed based on some initial trials.</p><p>The therapists only agreed with the POMDP approximately 43% of the time for the stop decision. The POMDP wanted to stop the exercise to let the user take a break far more often than the therapist wanted. If the therapist did not see any signs of fatigue from the user, she would have the patient continue practising the exercise for a longer period of time and not stop. The dynamics of the fatigue variable in the POMDP model caused its progression to <italic>fatigue </italic>= <italic>yes </italic>too quickly. Decreasing this progression to match that of the therapist's decision of stopping the exercise can be fixed by adjusting the fatigue effects in the model. Since the percentage of agreement for the stop decision was low, the overall therapist agreement with the POMDP decisions dropped to approximately 65%.</p><p>During each session, as soon as the POMDP estimated that the patient was fatigued, it continually made the decision to stop the exercise no matter the decision the therapist entered into the system. That is, the POMDP would continue to call for a stop from the time it first did so until the therapist finally agreed. If the repeated stop decisions were discarded, the percentage of agreement would have been approximately 94%.</p><p>The therapist's decisions alternated between having the patient work on muscle strengthening (by repeatedly setting the distance and resistance at the highest level) and on control (by randomizing the target distance and resistance levels). However, randomization was not part of the POMDP's initial objective and thus, the POMDP would never make the decision to randomize the target distance and resistance levels.</p></sec><sec><title>Questionnaire Data</title><p>Figure <xref ref-type="fig" rid="F12">12</xref> summarizes the therapist's session responses, in terms of mean and standard deviation (SD), regarding the appropriateness of the decisions made during the exercise and whether the patient was given enough time to complete each exercise before the next decision was made. The therapist's rating on the appropriateness of the amount of time given to complete each exercise before the next decision was made was generally favorable with a mean score of 3.2 out of 4.0 on the Likert scale. However, the appropriateness of the decisions made by the POMDP during the sessions was less favorable with a mean score of 2.8 out of 4.0. Comments from the therapist suggested that randomizing the target distance and resistance level would be beneficial for the patient to work on control in addition to strengthening, which the POMDP did by repeatedly setting the target distance at <italic>d3 </italic>and the resistance level at <italic>max </italic>(once the patient was able to perform the exercise at these settings). The initial specification of the POMDP model was based on the repeated exercise for strengthening only, and did not include any utility function promoting the practice of control through randomization. This could be included in future versions by explicitly modeling the fact that a sequence of differing resistance and distance levels can improve a client's control.</p><fig id="F12" position="float"><label>Figure 12</label><caption><p><bold>Therapist evaluation on POMDP decisions</bold>. This figure summarizes the evaluation of POMDP decisions made by the therapist on a Likert scale with a mean and SD of 2.833 and 0.408, respectively, for question (a) and a mean and SD of 3.167 and 0.408, respectively, for question (b).</p></caption><graphic xlink:href="1743-0003-8-33-12"></graphic></fig><p>In addition to the quantitative ratings in the session questionnaire, a qualitative question was asked to encourage the therapist to elaborate on any aspects related to the decisions made by the POMDP agent. The general qualitative results from the therapist for the final questionnaire can be summarized as follows: 1) the POMDP decisions were believable, except that the POMDP wanted to stop the exercise too early, and 2) the therapist could envision the rehabilitation system being used in both the clinic and home setting, as long as the system could vary the locations of the target and not restrict it to a straight path for more patient motivation, and was easy to set up for therapists.</p><p>With the help of a translator, the patient was able to answer the final questionnaire at the end of the study, which consisted of eight quantitative four-point Likert scale questions and four qualitative questions. From the patient's quantitative results, the patient found the quality of motion of the robotic device to be very smooth with a score of 4.0 out of 4.0. The patient also felt that the resistance applied by the robotic device was too little, scoring 1.0 out of 4.0. Throughout the study, the patient repeatedly commented that the exercise was "too easy", again a reflection of the device's resistance levels not being properly tuned to this particular user before the start of the trial. The patient was not able to feel the trunk sensors at all during the exercise, which suggests that trunk compensatory movements can be captured unobtrusively. The patient also felt that the bull's eye game was somewhat interesting, scoring 3.0 out of 4.0. The patient felt that the exercise closely resembled the reaching motion and conventional upper-limb therapy, scoring 3.0 out of 4.0 for both. In addition, the patient believed he would use this robotic system as his primary therapy, scoring 4.0 out of 4.0. The patient did not elaborate on the qualitative questions, thus, feedback from this section of the questionnaire was discarded.</p></sec></sec><sec><title>Future work</title><p>The immediate future work of this project is to test the POMDP model with more participants in order to obtain significant results. Besides this, the results from the pilot study provide the following insight into the future development of the POMDP model and overall system.</p><p>• The effect of randomization of different target distances and resistance levels on control needs to be studied.</p><p>The dynamics of the fatigue variable and the cost of stop action may need to be changed in order to stop the exercise less often. This problem can be solved in various ways. To show this, two simulations were run - 1) with varying costs of the <italic>stop </italic>decision, and 2) with varying horizontal shift of fatigue pace function. The result of the first experiment is shown in Figure <xref ref-type="fig" rid="F13">13</xref>, which shows that increasing the cost of the stop action generates, on average, longer runs. The result of the second experiment is shown in Figure <xref ref-type="fig" rid="F14">14</xref>, which shows that having a lower probability of fatigue generates longer runs. A lower probability of fatigue is achieved by shifting the fatigue pace function horizontally. In this case, the system thinks that the user will be less likely to get fatigued for exercises with the same stretch. These simulated results overall demonstrate that the therapist can adjust the policy of interaction substantially, to suit their and their client's needs.</p><fig id="F13" position="float"><label>Figure 13</label><caption><p><bold><italic>Exercise run lengths for different costs of </italic>stop</bold>. This figure shows the average run length for different costs of the stop action. Increasing the cost of the <italic>stop </italic>generates, on average, longer runs.</p></caption><graphic xlink:href="1743-0003-8-33-13"></graphic></fig><fig id="F14" position="float"><label>Figure 14</label><caption><p><bold>Exercise run lengths for different shifts in <italic>fatigue</italic></bold>. This figure shows the average run length for different horizontal shifts of the <italic>fatigue </italic>pace function. Lowering the probability of fatigue generates longer runs.</p></caption><graphic xlink:href="1743-0003-8-33-14"></graphic></fig><p>• The POMDP model needs to be expanded in order to include targets in 2D space. As a first step of this expansion, currently we are developing 2D virtual games that include target positions in 2D space. Figure <xref ref-type="fig" rid="F15">15</xref> shows an example where the target positions are set in a rectangular trajectory and the reaching task is to position the ball, which represents the end-effector of the robot, in the designated target position.</p><fig id="F15" position="float"><label>Figure 15</label><caption><p><bold>2D reaching exercise</bold>. This figure shows the virtual environment for 2D reaching exercise.</p></caption><graphic xlink:href="1743-0003-8-33-15"></graphic></fig><p>• The current robotic system only applies three discrete levels of resistance, which can be either increased or maintained at the same level during the exercise. The system will be more realistic if it is able to select varying levels of resistance that can be both increased and decreased to cope up with the need of an individual patient. Decreasing the resistance level may also result in lower fatigue probability and less frequent compensatory motion, which in turn may lead to longer duration of the exercise. To include these features into the current system, we are currently formulating a new probabilistic framework that models the user ability using Beta distributions [<xref ref-type="bibr" rid="B34">34</xref>] as a function of continuous resistance levels. A Beta distribution is initially chosen since it is suitable for modeling success or failure in continuous space. Figure <xref ref-type="fig" rid="F16">16</xref> shows a simulated example with a range of continuous resistance levels from 0-20, where the probability of successfully finishing an exercise at a given resistance level is modeled with the following Beta distributions: <italic>β</italic><sub><italic>n </italic></sub>in case the person is not fatigued and <italic>β</italic><sub><italic>y </italic></sub>in case the person is fatigued.</p><fig id="F16" position="float"><label>Figure 16</label><caption><p><bold>Beta distribution</bold>. This figure shows continuous action space using Beta distribution.</p></caption><graphic xlink:href="1743-0003-8-33-16"></graphic></fig><p>The total model is a weighted mixture of these two distributions, weighted according to the current belief that <italic>fatigue = yes</italic>. In this example, the posterior belief state assumes that probability of <italic>fatigue </italic>= <italic>no </italic>is 0.9 and probability of <italic>fatigue </italic>= <italic>yes </italic>is 0.1. The mixture model can be used to select the next resistance level for the exercise. In this example, the next resistance level 9.3 (shown in green circle in Figure <xref ref-type="fig" rid="F11">11</xref>) is selected as the maximum resistance that produces <italic>β<sub>sum </sub></italic>≥ 0.5. Figure <xref ref-type="fig" rid="F10">10(a)</xref> shows the next sequence where the distributions and the belief state are updated using the simulated observation that the person successfully completed the exercise (shown in red circle in Figure <xref ref-type="fig" rid="F17">17</xref>) at the resistance level 9.3.</p><fig id="F17" position="float"><label>Figure 17</label><caption><p><bold>Updated Beta distribution after the first observation</bold>. This figure shows the updated Beta distribution after the first observation.</p></caption><graphic xlink:href="1743-0003-8-33-17"></graphic></fig><p>The updated model is the posterior according to Bayes' rule. The next resistance level is set to 10.3 according to the updated . Figure <xref ref-type="fig" rid="F10">10(b)</xref> shows an instance where the distributions and belief state is updated after five observations. The first four observations are successful exercises (shown in red circle in Figure <xref ref-type="fig" rid="F10">10(b)</xref>) and the last one is an unsuccessful exercise (the person did not reach the goal within acceptable time and control or had to compensate too much - shown in blue circle in Figure <xref ref-type="fig" rid="F10">10(b)</xref>). As a result, the next resistance level is set smaller compared to the current resistance. The exercise can be continued until the probability of <italic>fatigue </italic>= <italic>yes </italic>reaches a predefined threshold. Hence, this formulation - 1) is able to increase and decrease resistance levels in continuous space, and 2) is more adaptive to each individual patient's need since the distributions - the model of the person's abilities - are updated with the new observations. The initial shapes of the distributions can also be varied according to the condition of individual patient so that it produces appropriate resistance level while starting the exercise.</p><p>As shown in Figure <xref ref-type="fig" rid="F18">18</xref>, the same formulation can be applied to other state variables of the system. The preceding simulations are meant to demonstrate the feasibility of such a representation, and we are currently in the process of applying them to our rehabilitation device.</p><fig id="F18" position="float"><label>Figure 18</label><caption><p><bold>Updated Beta distribution after five observations</bold>. This figure shows the updated Beta distribution after five observations.</p></caption><graphic xlink:href="1743-0003-8-33-18"></graphic></fig></sec><sec><title>Conclusions</title><p>This paper presents a POMDP system that is designed for an upper-limb rehabilitation robotic device. A POMDP was chosen for this system because it has the ability to handle partial observability (e.g. user fatigue), adapt to users' needs, and operate autonomously. The goal of the POMDP agent is to help patients regain their maximum reaching distance at the most difficult level of resistance, while performing the exercises with control and proper posture. Computer simulations of the POMDP model showed that the POMDP was making decisions in alignment to those of conventional reaching rehabilitation, which was to gradually increase target distance first, then resistance level as the user performed well, and increase the rate of fatigue if the user was not performing well.</p><p>The performance of the system was also evaluated by comparing the decisions made by the system with those of a human therapist. A single patient participant was paired up with a therapist participant for the duration of the study. Overall, the therapist agreed with the system decisions approximately 65% of the time. In general, the therapist thought the system decisions were believable and could envision this system being used in both a clinical and home setting. The patient was satisfied with the system and would use this system as her primary method of rehabilitation. The data collected in this study can only be used to provide insight into the performance of the system since the sample size was limited. As a result, the immediate future work of this project would be to test this POMDP model with more participants in order to obtain significant results.</p><p>The feedback from the therapist also suggests that the present system needs to include 2D target locations and varying levels of resistance. To include these features into the current system, we are currently developing virtual games with 2D target locations and a new probabilistic framework that expresses the probability of successfully completing an exercise using Beta distributions as a function of continuous resistance levels. The distributions are continuously updated with the new observations to reflect the performance of each individual patient. The system is also able to increase or decrease resistance levels according to the performance of a patient. The flexibility of decreasing resistance levels may also result in lower fatigue probability and thus may prevent early stopping of the exercise. The following suggestions of the therapist will also be considered in the future development:</p><p>• set mapping from resistance levels in the POMDP agent to actual resistance in the device for each user based on some initial trials,</p><p>• enhance the user interface to provide feedback for the user such as a scoring system or sounds to indicate that the user has reached the target, and</p><p>• develop an easier way to initialize the exercise such that all programs start automatically</p><p>Overall, this research demonstrates that POMDPs have promising potential to provide autonomous upper-limb rehabilitation for stroke patients, which may allow clients to perform guided rehabilitation when and where they prefer and enable them to progress at the best possible pace.</p></sec><sec><title>Competing interests</title><p>The authors declare that they have no competing interests.</p></sec><sec><title>Authors' contributions</title><p>PK and JH designed and developed the POMDP system. PK integrated the POMDP system with all aspects of the robotic system, developed and conducted the evaluation study of the overall integrated system, analyzed the study data, and drafted the manuscript. JH and RG conducted simulations post-trial to demonstrate how to solve the POMDP's early stopping problem. RH and RG added the simulation results of 2D virtual environment for reaching exercise and a new probabilistic framework that expresses the probability of successfully completing an exercise using Beta distributions as a function of continuous resistance levels. AM supervised the project. PK, JH, and AM participated in the conception and design of the POMDP system, and all authors read and approved the final manuscript.</p></sec><sec sec-type="supplementary-material"><title>Supplementary Material</title><supplementary-material content-type="local-data" id="S1"><caption><title>Additional file 1</title><p><bold>Pace functions parameters</bold>. This file describes the procedure of specifying the parameters of a pace function.</p></caption><media xlink:href="1743-0003-8-33-S1.DOC" mimetype="application" mime-subtype="msword"><caption><p>Click here for file</p></caption></media></supplementary-material><supplementary-material content-type="local-data" id="S2"><caption><title>Additional file 2</title><p><bold>Reward function</bold>. This file summarizes the reward function of the POMDP model.</p></caption><media xlink:href="1743-0003-8-33-S2.DOC" mimetype="application" mime-subtype="msword"><caption><p>Click here for file</p></caption></media></supplementary-material><supplementary-material content-type="local-data" id="S3"><caption><title>Additional file 3</title><p><bold>POMDP simulation example 1</bold>. This file shows the simulation steps of example 1.</p></caption><media xlink:href="1743-0003-8-33-S3.DOC" mimetype="application" mime-subtype="msword"><caption><p>Click here for file</p></caption></media></supplementary-material><supplementary-material content-type="local-data" id="S4"><caption><title>Additional file 4</title><p><bold>POMDP simulation example 2</bold>. This file shows the simulation steps of example 2.</p></caption><media xlink:href="1743-0003-8-33-S4.DOC" mimetype="application" mime-subtype="msword"><caption><p>Click here for file</p></caption></media></supplementary-material></sec></body><back><sec><title>Acknowledgements</title><p>The authors gratefully acknowledge the following people: Debbie Hebért for sharing her expertise in the field of occupational therapy, especially in the area of upper-limb stroke rehabilitation; and Quanser Inc. for all their technical support on the robotic device and virtual environment. This work was supported by CITO-Precarn Alliance Program, a grant from the NSERC-CIHR CHRP Program, Quanser Inc, and by FONCICYT contract number 000000000095185. The content of this document reflects only the author<sup>1</sup>s views. FONCICYT is not liable for any use that may be made of the contained information.</p></sec><ref-list><ref id="B1"><mixed-citation publication-type="other"><article-title>Canadian Stroke Network: Stroke 101</article-title><ext-link ext-link-type="uri" xlink:href="http://www.canadianstrokenetwork.ca/eng/about/stroke101.php">http://www.canadianstrokenetwork.ca/eng/about/stroke101.php</ext-link></mixed-citation></ref><ref id="B2"><mixed-citation publication-type="other"><article-title>American Heart Association: Stroke Statistics</article-title><ext-link ext-link-type="uri" xlink:href="http://www.americanheart.org/presenter.jhtml?identifier=4725">http://www.americanheart.org/presenter.jhtml?identifier=4725</ext-link></mixed-citation></ref><ref id="B3"><mixed-citation publication-type="other"><article-title>Heart and Stroke Foundation of Canada: Stroke Statistics</article-title><ext-link ext-link-type="uri" xlink:href="http://www.heartandstroke.com/site/c.ikIQLcMWJtE/b.3483991/k.34A8/Statistics.htm#stroke">http://www.heartandstroke.com/site/c.ikIQLcMWJtE/b.3483991/k.34A8/Statistics.htm#stroke</ext-link></mixed-citation></ref><ref id="B4"><mixed-citation publication-type="journal"><name><surname>Fasoli</surname><given-names>SE</given-names></name><name><surname>Krebs</surname><given-names>HI</given-names></name><name><surname>Hogan</surname><given-names>N</given-names></name><article-title>Robotic technology and stroke rehabilitation: Translating research into practice</article-title><source>Topics in Stroke Rehabilitation</source><year>2004</year><volume>11</volume><issue>4</issue><fpage>11</fpage><lpage>19</lpage><pub-id pub-id-type="doi">10.1310/G8XB-VM23-1TK7-PWQU</pub-id><pub-id pub-id-type="pmid">15592986</pub-id></mixed-citation></ref><ref id="B5"><mixed-citation publication-type="journal"><name><surname>Hidler</surname><given-names>J</given-names></name><name><surname>Nichols</surname><given-names>D</given-names></name><name><surname>Pelliccio</surname><given-names>M</given-names></name><name><surname>Brady</surname><given-names>K</given-names></name><article-title>Advances in the understanding and treatment of stroke impairment using robotic devices</article-title><source>Topics in Stroke Rehabilitation</source><year>2005</year><volume>12</volume><issue>2</issue><fpage>22</fpage><lpage>35</lpage><pub-id pub-id-type="doi">10.1310/RYT5-62N4-CTVX-8JTE</pub-id><pub-id pub-id-type="pmid">15940582</pub-id></mixed-citation></ref><ref id="B6"><mixed-citation publication-type="book"><name><surname>Caplan</surname><given-names>LR</given-names></name><source>Stroke</source><year>2006</year><publisher-name>New York: Demos Medical Publishing</publisher-name></mixed-citation></ref><ref id="B7"><mixed-citation publication-type="journal"><name><surname>Reinkensmeyer</surname><given-names>DJ</given-names></name><name><surname>Kahn</surname><given-names>LE</given-names></name><name><surname>Averbuch</surname><given-names>M</given-names></name><name><surname>McKenna-Cole</surname><given-names>A</given-names></name><name><surname>Schmit</surname><given-names>BD</given-names></name><name><surname>Rymer</surname><given-names>WZ</given-names></name><article-title>Understanding and treating arm movement impairment after chronic brain injury: Progress with the ARM guide</article-title><source>Journal of Rehabilitation Research and Development</source><year>2000</year><volume>37</volume><issue>6</issue><fpage>653</fpage><lpage>662</lpage><pub-id pub-id-type="pmid">11321001</pub-id></mixed-citation></ref><ref id="B8"><mixed-citation publication-type="journal"><name><surname>Lum</surname><given-names>PS</given-names></name><name><surname>Burgar</surname><given-names>CG</given-names></name><name><surname>Shor</surname><given-names>PC</given-names></name><name><surname>Majmundar</surname><given-names>M</given-names></name><name><surname>Van der Loos</surname><given-names>M</given-names></name><article-title>Robot-assisted movement training compared with conventional therapy techniques for the rehabilitation of upper-limb motor function after stroke</article-title><source>Archives of Physical Medicine and Rehabilitation</source><year>2002</year><volume>83</volume><issue>7</issue><fpage>952</fpage><lpage>959</lpage><pub-id pub-id-type="doi">10.1053/apmr.2001.33101</pub-id><pub-id pub-id-type="pmid">12098155</pub-id></mixed-citation></ref><ref id="B9"><mixed-citation publication-type="journal"><name><surname>Amirabdollahian</surname><given-names>F</given-names></name><name><surname>Loureiro</surname><given-names>R</given-names></name><name><surname>Gradwell</surname><given-names>E</given-names></name><name><surname>Collin</surname><given-names>C</given-names></name><name><surname>Harwin</surname><given-names>W</given-names></name><name><surname>Johnson</surname><given-names>G</given-names></name><article-title>Multivariate analysis of the Fugl-Meyer outcome measures assessing the effectiveness of GENTLE/S robot-mediated stroke therapy</article-title><source>Journal of NeuroEngineering and Rehabilitation</source><year>2007</year><volume>4</volume><issue>4</issue><fpage>1</fpage><lpage>16</lpage><pub-id pub-id-type="pmid">17244363</pub-id></mixed-citation></ref><ref id="B10"><mixed-citation publication-type="journal"><name><surname>Krebs</surname><given-names>HI</given-names></name><name><surname>Hogan</surname><given-names>N</given-names></name><name><surname>Aisen</surname><given-names>ML</given-names></name><name><surname>Volpe</surname><given-names>BT</given-names></name><article-title>Robot-aided neurorehabilitation</article-title><source>IEEE Transactions on Rehabilitation Engineering</source><year>1998</year><volume>6</volume><issue>1</issue><fpage>75</fpage><lpage>87</lpage><pub-id pub-id-type="doi">10.1109/86.662623</pub-id><pub-id pub-id-type="pmid">9535526</pub-id></mixed-citation></ref><ref id="B11"><mixed-citation publication-type="journal"><name><surname>Ferraro</surname><given-names>M</given-names></name><name><surname>Palazzolo</surname><given-names>JJ</given-names></name><name><surname>Krol</surname><given-names>J</given-names></name><name><surname>Krebs</surname><given-names>HI</given-names></name><name><surname>Hogan</surname><given-names>N</given-names></name><name><surname>Volpe</surname><given-names>BT</given-names></name><article-title>Robot-aided sensorimotor arm training improves outcome in patients with chronic stroke</article-title><source>Neurology</source><year>2003</year><volume>61</volume><issue>11</issue><fpage>1604</fpage><lpage>1607</lpage><pub-id pub-id-type="pmid">14663051</pub-id></mixed-citation></ref><ref id="B12"><mixed-citation publication-type="journal"><name><surname>Fasoli</surname><given-names>SE</given-names></name><name><surname>Krebs</surname><given-names>HI</given-names></name><name><surname>Stein</surname><given-names>J</given-names></name><name><surname>Frontera</surname><given-names>WR</given-names></name><name><surname>Hughes</surname><given-names>R</given-names></name><name><surname>Hogan</surname><given-names>N</given-names></name><article-title>Robotic therapy for chronic motor impairments after stroke: Follow-up results</article-title><source>Archives of Physical Medicine and Rehabilitation</source><year>2004</year><volume>85</volume><issue>7</issue><fpage>1106</fpage><lpage>1111</lpage><pub-id pub-id-type="doi">10.1016/j.apmr.2003.11.028</pub-id><pub-id pub-id-type="pmid">15241758</pub-id></mixed-citation></ref><ref id="B13"><mixed-citation publication-type="journal"><name><surname>MacClellan</surname><given-names>LR</given-names></name><name><surname>Bradham</surname><given-names>DD</given-names></name><name><surname>Whitall</surname><given-names>J</given-names></name><name><surname>Volpe</surname><given-names>B</given-names></name><name><surname>Wilson</surname><given-names>PD</given-names></name><name><surname>Ohlhoff</surname><given-names>J</given-names></name><etal></etal><article-title>Robotic upper-limb neurorehabilitation in chronic stroke patients</article-title><source>Journal of Rehabilitation Research and Development</source><year>2005</year><volume>42</volume><issue>6</issue><fpage>717</fpage><lpage>722</lpage><pub-id pub-id-type="doi">10.1682/JRRD.2004.606.0068</pub-id><pub-id pub-id-type="pmid">16680609</pub-id></mixed-citation></ref><ref id="B14"><mixed-citation publication-type="journal"><name><surname>Lo</surname><given-names>AL</given-names></name><name><surname>Guarino</surname><given-names>PD</given-names></name><name><surname>Richards</surname><given-names>LG</given-names></name><name><surname>Haselkorn</surname><given-names>JK</given-names></name><name><surname>Wittenberg</surname><given-names>GF</given-names></name><name><surname>Federman</surname><given-names>DG</given-names></name><etal></etal><article-title>Robot-assisted therapy for long-term upper-limb impairment after stroke</article-title><source>The New England Journal of Medicine</source><year>2010</year><volume>362</volume><fpage>1</fpage><lpage>13</lpage><pub-id pub-id-type="doi">10.1056/NEJMp0910194</pub-id><pub-id pub-id-type="pmid">20007553</pub-id></mixed-citation></ref><ref id="B15"><mixed-citation publication-type="journal"><name><surname>Reinkensmeyer</surname><given-names>D</given-names></name><name><surname>Pang</surname><given-names>C</given-names></name><name><surname>Nessler</surname><given-names>J</given-names></name><name><surname>Painter</surname><given-names>C</given-names></name><article-title>Web-based telerehabilitation for the upper-extremity after stroke</article-title><source>IEEE Transactions on Neural Science and Rehabilitation Engineering</source><year>2002</year><volume>10</volume><fpage>1</fpage><lpage>7</lpage><pub-id pub-id-type="doi">10.1109/TNSRE.2002.1021581</pub-id></mixed-citation></ref><ref id="B16"><mixed-citation publication-type="book"><name><surname>Sucar</surname><given-names>LE</given-names></name><name><surname>Leder</surname><given-names>R</given-names></name><name><surname>Reinkensmeyer</surname><given-names>D</given-names></name><name><surname>Hernández</surname><given-names>J</given-names></name><name><surname>Azcárate</surname><given-names>G</given-names></name><name><surname>Castañeda</surname><given-names>N</given-names></name><etal></etal><article-title>Gesture Therapy: A low-cost vision-based system for rehabilitation after stroke</article-title><source>Proceedings of the First International Conference on Health Informatics</source><year>2008</year><volume>2</volume><publisher-name>Funchal, Madeira, Portugal</publisher-name><fpage>107</fpage><lpage>111</lpage></mixed-citation></ref><ref id="B17"><mixed-citation publication-type="journal"><name><surname>Qiu</surname><given-names>Q</given-names></name><name><surname>Ramirez</surname><given-names>DA</given-names></name><name><surname>Saleh</surname><given-names>S</given-names></name><name><surname>Fluet</surname><given-names>GG</given-names></name><name><surname>Parikh</surname><given-names>HD</given-names></name><name><surname>Kelly</surname><given-names>D</given-names></name><name><surname>Adamovich</surname><given-names>S</given-names></name><article-title>The New Jersey Institute of Technology - Robot-Assisted Virtual Rehabilitation(NJIT-RAVR) system for children with cerebral palsy: A feasibility study</article-title><source>Journal of Nueroengineering and Rehabilitation</source><year>2009</year><volume>6</volume><issue>40</issue></mixed-citation></ref><ref id="B18"><mixed-citation publication-type="journal"><name><surname>Ju</surname><given-names>MS</given-names></name><name><surname>Lin</surname><given-names>CCK</given-names></name><name><surname>Lin</surname><given-names>DH</given-names></name><name><surname>Hwang</surname><given-names>IS</given-names></name><name><surname>Chen</surname><given-names>SM</given-names></name><article-title>A rehabilitation robot with force-position hybrid fuzzy controller: Hybrid fuzzy control of rehabilitation robot</article-title><source>IEEE Transactions on Neural Systems and Rehabilitation Engineering</source><year>2005</year><volume>13</volume><issue>3</issue><fpage>349</fpage><lpage>358</lpage><pub-id pub-id-type="doi">10.1109/TNSRE.2005.847354</pub-id><pub-id pub-id-type="pmid">16200758</pub-id></mixed-citation></ref><ref id="B19"><mixed-citation publication-type="book"><name><surname>Erol</surname><given-names>D</given-names></name><name><surname>Mallapragada</surname><given-names>V</given-names></name><name><surname>Sarkar</surname><given-names>N</given-names></name><name><surname>Uswatte</surname><given-names>G</given-names></name><name><surname>Taub</surname><given-names>E</given-names></name><article-title>Autonomously adapting robotic assistance for rehabilitation therapy</article-title><source>Paper presented at the First IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics</source><year>2006</year><publisher-name>Pisa, Italy</publisher-name><fpage>567</fpage><lpage>572</lpage></mixed-citation></ref><ref id="B20"><mixed-citation publication-type="journal"><name><surname>Marchal-Crespo</surname><given-names>L</given-names></name><name><surname>Reinkensmeyer</surname><given-names>DJ</given-names></name><article-title>Review of Control Strategies for Robotic Movement Training after Neurologic Injury</article-title><source>Journal of Neuroengineering and Neurorehabilitation</source><year>2009</year><volume>6</volume><issue>20</issue><ext-link ext-link-type="uri" xlink:href="http://www.jneuroengrehab.com/content/6/1/20">http://www.jneuroengrehab.com/content/6/1/20</ext-link></mixed-citation></ref><ref id="B21"><mixed-citation publication-type="journal"><name><surname>Wolbrecht</surname><given-names>E</given-names></name><name><surname>Reinkensmeyer</surname><given-names>DJ</given-names></name><name><surname>Bobrow</surname><given-names>JE</given-names></name><article-title>Optimizing Compliant, Model-Based Robotic Assistance to Promote Neurorehabilitation</article-title><source>IEEE Transactions on Neural Systems and Rehabilitation Engineering</source><year>2008</year><volume>16</volume><issue>3</issue><fpage>286</fpage><lpage>297</lpage><pub-id pub-id-type="pmid">18586608</pub-id></mixed-citation></ref><ref id="B22"><mixed-citation publication-type="book"><name><surname>Barnes</surname><given-names>M</given-names></name><name><surname>Dobkin</surname><given-names>B</given-names></name><name><surname>Bogousslavsky</surname><given-names>J</given-names></name><source>Recovery after stroke</source><year>2005</year><publisher-name>United Kingdom: Cambridge University Press</publisher-name></mixed-citation></ref><ref id="B23"><mixed-citation publication-type="book"><name><surname>Gillen</surname><given-names>G</given-names></name><name><surname>Burkhardt</surname><given-names>A</given-names></name><source>Stroke rehabilitation: A function-based approach</source><year>2004</year><edition>2</edition><publisher-name>Missouri: Mosby</publisher-name></mixed-citation></ref><ref id="B24"><mixed-citation publication-type="journal"><name><surname>Lam</surname><given-names>P</given-names></name><name><surname>Hébert</surname><given-names>D</given-names></name><name><surname>Boger</surname><given-names>J</given-names></name><name><surname>Lacheray</surname><given-names>H</given-names></name><name><surname>Gardner</surname><given-names>D</given-names></name><name><surname>Apkarian</surname><given-names>J</given-names></name><etal></etal><article-title>A haptic-robotic platform for upper-limb reaching stroke therapy: Preliminary design and evaluation results</article-title><source>Journal of NeuroEngineering and Rehabilitation</source><year>2008</year><volume>5</volume><issue>15</issue><fpage>1</fpage><lpage>13</lpage><pub-id pub-id-type="pmid">18171465</pub-id></mixed-citation></ref><ref id="B25"><mixed-citation publication-type="journal"><name><surname>Lovejoy</surname><given-names>WS</given-names></name><article-title>A survey of algorithmic methods for partially observable Markov decision processes</article-title><source>Annals of Operations Research</source><year>1991</year><volume>28</volume><fpage>47</fpage><lpage>66</lpage><pub-id pub-id-type="doi">10.1007/BF02055574</pub-id></mixed-citation></ref><ref id="B26"><mixed-citation publication-type="journal"><name><surname>Kaelbling</surname><given-names>LP</given-names></name><name><surname>Littman</surname><given-names>ML</given-names></name><name><surname>Cassandra</surname><given-names>AR</given-names></name><article-title>Planning and acting in partially observable stochastic domains</article-title><source>Artificial Intelligence</source><year>1998</year><volume>101</volume><fpage>99</fpage><lpage>134</lpage><pub-id pub-id-type="doi">10.1016/S0004-3702(98)00023-X</pub-id></mixed-citation></ref><ref id="B27"><mixed-citation publication-type="journal"><name><surname>Pineau</surname><given-names>J</given-names></name><name><surname>Montemerlo</surname><given-names>M</given-names></name><name><surname>Pollack</surname><given-names>M</given-names></name><name><surname>Roy</surname><given-names>N</given-names></name><name><surname>Thrun</surname><given-names>S</given-names></name><article-title>Towards robotic assistants in nursing homes: Challenges and results</article-title><source>Robotics and Autonomous Systems</source><year>2003</year><volume>42</volume><issue>3-4</issue><fpage>271</fpage><lpage>281</lpage><pub-id pub-id-type="doi">10.1016/S0921-8890(02)00381-0</pub-id></mixed-citation></ref><ref id="B28"><mixed-citation publication-type="book"><name><surname>Hoey</surname><given-names>J</given-names></name><name><surname>von Bertoldi</surname><given-names>A</given-names></name><name><surname>Poupart</surname><given-names>P</given-names></name><name><surname>Mihailidis</surname><given-names>A</given-names></name><article-title>Assisting persons with dementia during handwashing using a partially observable Markov decision process</article-title><source>Proceedings of the Fifth International Conference on Computer Vision Systems</source><year>2007</year><publisher-name>Bielefeld, Germany</publisher-name><ext-link ext-link-type="uri" xlink:href="http://biecoll.ub.uni-bielefeld.de/volltexte/2007/12/">http://biecoll.ub.uni-bielefeld.de/volltexte/2007/12/</ext-link></mixed-citation></ref><ref id="B29"><mixed-citation publication-type="journal"><name><surname>Pineau</surname><given-names>J</given-names></name><name><surname>Gordon</surname><given-names>G</given-names></name><name><surname>Thrun</surname><given-names>S</given-names></name><article-title>Anytime point-based approximations for large POMDPs</article-title><source>Journal of Artificial Intelligence Research</source><year>2006</year><volume>27</volume><fpage>335</fpage><lpage>380</lpage></mixed-citation></ref><ref id="B30"><mixed-citation publication-type="journal"><name><surname>Dobkin</surname><given-names>BH</given-names></name><article-title>Fatigue versus activity-dependent fatigability in patients with central or peripheral motor impairments</article-title><source>Neurorehabilitation and Neural Repair</source><year>2008</year><volume>22</volume><issue>2</issue><fpage>105</fpage><lpage>110</lpage><pub-id pub-id-type="pmid">18285599</pub-id></mixed-citation></ref><ref id="B31"><mixed-citation publication-type="book"><name><surname>Poupart</surname><given-names>P</given-names></name><article-title>Exploiting structure to efficiently solve large scale partially observable Markov decision processes</article-title><source>PhD thesis</source><year>2005</year><publisher-name>University of Toronto, Department of Computer Science</publisher-name></mixed-citation></ref><ref id="B32"><mixed-citation publication-type="journal"><name><surname>Spaan</surname><given-names>MTJ</given-names></name><name><surname>Vlassis</surname><given-names>N</given-names></name><article-title>Perseus: Randomized point-based value iteration for POMDPs</article-title><source>Journal of Artificial Intelligence Research</source><year>2005</year><volume>24</volume><fpage>195</fpage><lpage>220</lpage></mixed-citation></ref><ref id="B33"><mixed-citation publication-type="journal"><name><surname>Gowland</surname><given-names>C</given-names></name><name><surname>Stratford</surname><given-names>P</given-names></name><name><surname>Ward</surname><given-names>M</given-names></name><name><surname>Moreland</surname><given-names>J</given-names></name><name><surname>Torresin</surname><given-names>W</given-names></name><name><surname>Van Hullenaar</surname><given-names>S</given-names></name><etal></etal><article-title>Measuring physical impairment and disability with the Chedoke-McMaster Stroke Assessment</article-title><source>Stroke</source><year>1993</year><volume>24</volume><issue>1</issue><fpage>58</fpage><lpage>63</lpage><pub-id pub-id-type="doi">10.1161/01.STR.24.1.58</pub-id><pub-id pub-id-type="pmid">8418551</pub-id></mixed-citation></ref><ref id="B34"><mixed-citation publication-type="journal"><name><surname>Lazo</surname><given-names>ACGV</given-names></name><name><surname>Rathie</surname><given-names>PN</given-names></name><article-title>On the entropy of continuous probability distributions</article-title><source>IEEE Transactions on Information Theory</source><year>1978</year><volume>IT-24</volume><issue>24</issue><fpage>120</fpage><lpage>122</lpage></mixed-citation></ref></ref-list></back></pmc><affiliations><list><country><li>Canada</li><li>Royaume-Uni</li></country><region><li>Ontario</li></region><settlement><li>Toronto</li></settlement><orgName><li>Université de Toronto</li></orgName></list><tree><country name="Canada"><region name="Ontario"><name sortKey="Kan, Patricia" sort="Kan, Patricia" uniqKey="Kan P" first="Patricia" last="Kan">Patricia Kan</name></region><name sortKey="Huq, Rajibul" sort="Huq, Rajibul" uniqKey="Huq R" first="Rajibul" last="Huq">Rajibul Huq</name><name sortKey="Mihailidis, Alex" sort="Mihailidis, Alex" uniqKey="Mihailidis A" first="Alex" last="Mihailidis">Alex Mihailidis</name><name sortKey="Mihailidis, Alex" sort="Mihailidis, Alex" uniqKey="Mihailidis A" first="Alex" last="Mihailidis">Alex Mihailidis</name><name sortKey="Mihailidis, Alex" sort="Mihailidis, Alex" uniqKey="Mihailidis A" first="Alex" last="Mihailidis">Alex Mihailidis</name></country><country name="Royaume-Uni"><noRegion><name sortKey="Hoey, Jesse" sort="Hoey, Jesse" uniqKey="Hoey J" first="Jesse" last="Hoey">Jesse Hoey</name></noRegion><name sortKey="Goetschalckx, Robby" sort="Goetschalckx, Robby" uniqKey="Goetschalckx R" first="Robby" last="Goetschalckx">Robby Goetschalckx</name></country></tree></affiliations></record>Pour manipuler ce document sous Unix (Dilib)
EXPLOR_STEP=$WICRI_ROOT/Ticri/CIDE/explor/HapticV1/Data/Ncbi/Merge
HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 001A92 | SxmlIndent | more
Ou
HfdSelect -h $EXPLOR_AREA/Data/Ncbi/Merge/biblio.hfd -nk 001A92 | SxmlIndent | more
Pour mettre un lien sur cette page dans le réseau Wicri
{{Explor lien
|wiki= Ticri/CIDE
|area= HapticV1
|flux= Ncbi
|étape= Merge
|type= RBID
|clé= PMC:3152889
|texte= The development of an adaptive upper-limb stroke rehabilitation robotic system
}}
Pour générer des pages wiki
HfdIndexSelect -h $EXPLOR_AREA/Data/Ncbi/Merge/RBID.i -Sk "pubmed:21679457" \
| HfdSelect -Kh $EXPLOR_AREA/Data/Ncbi/Merge/biblio.hfd \
| NlmPubMed2Wicri -a HapticV1
| This area was generated with Dilib version V0.6.23. |  |