A Framework for Different Levels of Integration of Computational Models Into Web-Based Virtual Patients
Identifieur interne : 002021 ( Pmc/Curation ); précédent : 002020; suivant : 002022A Framework for Different Levels of Integration of Computational Models Into Web-Based Virtual Patients
Auteurs :Source :
- Journal of Medical Internet Research [ 1439-4456 ] ; 2014.
Abstract
Virtual patients are increasingly common tools used in health care education to foster learning of clinical reasoning skills. One potential way to expand their functionality is to augment virtual patients’ interactivity by enriching them with computational models of physiological and pathological processes.
The primary goal of this paper was to propose a conceptual framework for the integration of computational models within virtual patients, with particular focus on (1) characteristics to be addressed while preparing the integration, (2) the extent of the integration, (3) strategies to achieve integration, and (4) methods for evaluating the feasibility of integration. An additional goal was to pilot the first investigation of changing framework variables on altering perceptions of integration.
The framework was constructed using an iterative process informed by Soft System Methodology. The Virtual Physiological Human (VPH) initiative has been used as a source of new computational models. The technical challenges associated with development of virtual patients enhanced by computational models are discussed from the perspectives of a number of different stakeholders. Concrete design and evaluation steps are discussed in the context of an exemplar virtual patient employing the results of the VPH ARCH project, as well as improvements for future iterations.
The proposed framework consists of four main elements. The first element is a list of feasibility features characterizing the integration process from three perspectives: the computational modelling researcher, the health care educationalist, and the virtual patient system developer. The second element included three integration levels: basic, where a single set of simulation outcomes is generated for specific nodes in the activity graph; intermediate, involving pre-generation of simulation datasets over a range of input parameters; advanced, including dynamic solution of the model. The third element is the description of four integration strategies, and the last element consisted of evaluation profiles specifying the relevant feasibility features and acceptance thresholds for specific purposes. The group of experts who evaluated the virtual patient exemplar found higher integration more interesting, but at the same time they were more concerned with the validity of the result. The observed differences were not statistically significant.
This paper outlines a framework for the integration of computational models into virtual patients. The opportunities and challenges of model exploitation are discussed from a number of user perspectives, considering different levels of model integration. The long-term aim for future research is to isolate the most crucial factors in the framework and to determine their influence on the integration outcome.
Url:
DOI: 10.2196/jmir.2593
PubMed: 24463466
PubMed Central: 3906686
Links toward previous steps (curation, corpus...)
- to stream Pmc, to step Corpus: Pour aller vers cette notice dans l'étape Curation :002021
Links to Exploration step
PMC:3906686Le document en format XML
<record><TEI><teiHeader><fileDesc><titleStmt><title xml:lang="en">A Framework for Different Levels of Integration of Computational Models Into Web-Based Virtual Patients</title></titleStmt><publicationStmt><idno type="wicri:source">PMC</idno><idno type="pmid">24463466</idno><idno type="pmc">3906686</idno><idno type="url">http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3906686</idno><idno type="RBID">PMC:3906686</idno><idno type="doi">10.2196/jmir.2593</idno><date when="2014">2014</date><idno type="wicri:Area/Pmc/Corpus">002021</idno><idno type="wicri:Area/Pmc/Curation">002021</idno></publicationStmt><sourceDesc><biblStruct><analytic><title xml:lang="en" level="a" type="main">A Framework for Different Levels of Integration of Computational Models Into Web-Based Virtual Patients</title></analytic><series><title level="j">Journal of Medical Internet Research</title><idno type="ISSN">1439-4456</idno><idno type="eISSN">1438-8871</idno><imprint><date when="2014">2014</date></imprint></series></biblStruct></sourceDesc></fileDesc><profileDesc><textClass></textClass></profileDesc></teiHeader><front><div type="abstract" xml:lang="en"><sec sec-type="background"><title>Background</title><p>Virtual patients are increasingly common tools used in health care education to foster learning of clinical reasoning skills. One potential way to expand their functionality is to augment virtual patients’ interactivity by enriching them with computational models of physiological and pathological processes.</p></sec><sec sec-type="objective"><title>Objective</title><p>The primary goal of this paper was to propose a conceptual framework for the integration of computational models within virtual patients, with particular focus on (1) characteristics to be addressed while preparing the integration, (2) the extent of the integration, (3) strategies to achieve integration, and (4) methods for evaluating the feasibility of integration. An additional goal was to pilot the first investigation of changing framework variables on altering perceptions of integration.</p></sec><sec sec-type="methods"><title>Methods</title><p>The framework was constructed using an iterative process informed by Soft System Methodology. The Virtual Physiological Human (VPH) initiative has been used as a source of new computational models. The technical challenges associated with development of virtual patients enhanced by computational models are discussed from the perspectives of a number of different stakeholders. Concrete design and evaluation steps are discussed in the context of an exemplar virtual patient employing the results of the VPH ARCH project, as well as improvements for future iterations.</p></sec><sec sec-type="results"><title>Results</title><p>The proposed framework consists of four main elements. The first element is a list of feasibility features characterizing the integration process from three perspectives: the computational modelling researcher, the health care educationalist, and the virtual patient system developer. The second element included three integration levels: basic, where a single set of simulation outcomes is generated for specific nodes in the activity graph; intermediate, involving pre-generation of simulation datasets over a range of input parameters; advanced, including dynamic solution of the model. The third element is the description of four integration strategies, and the last element consisted of evaluation profiles specifying the relevant feasibility features and acceptance thresholds for specific purposes. The group of experts who evaluated the virtual patient exemplar found higher integration more interesting, but at the same time they were more concerned with the validity of the result. The observed differences were not statistically significant.</p></sec><sec sec-type="conclusions"><title>Conclusions</title><p>This paper outlines a framework for the integration of computational models into virtual patients. The opportunities and challenges of model exploitation are discussed from a number of user perspectives, considering different levels of model integration. The long-term aim for future research is to isolate the most crucial factors in the framework and to determine their influence on the integration outcome.</p></sec></div></front><back><div1 type="bibliography"><listBibl><biblStruct><analytic><author><name sortKey="Ellaway, R" uniqKey="Ellaway R">R Ellaway</name></author><author><name sortKey="Masters, K" uniqKey="Masters K">K Masters</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Ellaway, Rh" uniqKey="Ellaway R">RH Ellaway</name></author><author><name sortKey="Poulton, T" uniqKey="Poulton T">T Poulton</name></author><author><name sortKey="Smothers, V" uniqKey="Smothers V">V Smothers</name></author><author><name sortKey="Greene, P" uniqKey="Greene P">P Greene</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Poulton, T" uniqKey="Poulton T">T Poulton</name></author><author><name sortKey="Balasubramaniam, C" uniqKey="Balasubramaniam C">C Balasubramaniam</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Maloney, S" uniqKey="Maloney S">S Maloney</name></author><author><name sortKey="Haas, R" uniqKey="Haas R">R Haas</name></author><author><name sortKey="Keating, Jl" uniqKey="Keating J">JL Keating</name></author><author><name sortKey="Molloy, E" uniqKey="Molloy E">E Molloy</name></author><author><name sortKey="Jolly, B" uniqKey="Jolly B">B Jolly</name></author><author><name sortKey="Sims, J" uniqKey="Sims J">J Sims</name></author><author><name sortKey="Morgan, P" uniqKey="Morgan P">P Morgan</name></author><author><name sortKey="Haines, T" uniqKey="Haines T">T Haines</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Kononowicz, Aa" uniqKey="Kononowicz A">AA Kononowicz</name></author><author><name sortKey="Hege, I" uniqKey="Hege I">I Hege</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Ellaway, R" uniqKey="Ellaway R">R Ellaway</name></author><author><name sortKey="Poulton, T" uniqKey="Poulton T">T Poulton</name></author><author><name sortKey="Fors, U" uniqKey="Fors U">U Fors</name></author><author><name sortKey="Mcgee, Jb" uniqKey="Mcgee J">JB McGee</name></author><author><name sortKey="Albright, S" uniqKey="Albright S">S Albright</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Cook, Da" uniqKey="Cook D">DA Cook</name></author><author><name sortKey="Triola, Mm" uniqKey="Triola M">MM Triola</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Dev, P" uniqKey="Dev P">P Dev</name></author><author><name sortKey="Heinrichs, Wl" uniqKey="Heinrichs W">WL Heinrichs</name></author><author><name sortKey="Youngblood, P" uniqKey="Youngblood P">P Youngblood</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Huwendiek, S" uniqKey="Huwendiek S">S Huwendiek</name></author><author><name sortKey="De Leng, Ba" uniqKey="De Leng B">BA de Leng</name></author><author><name sortKey="Zary, N" uniqKey="Zary N">N Zary</name></author><author><name sortKey="Fischer, Mr" uniqKey="Fischer M">MR Fischer</name></author><author><name sortKey="Ruiz, Jg" uniqKey="Ruiz J">JG Ruiz</name></author><author><name sortKey="Ellaway, R" uniqKey="Ellaway R">R Ellaway</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Smothers, V" uniqKey="Smothers V">V Smothers</name></author><author><name sortKey="Azan, B" uniqKey="Azan B">B Azan</name></author><author><name sortKey="Ellaway, R" uniqKey="Ellaway R">R Ellaway</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Zary, N" uniqKey="Zary N">N Zary</name></author><author><name sortKey="Hege, I" uniqKey="Hege I">I Hege</name></author><author><name sortKey="Heid, J" uniqKey="Heid J">J Heid</name></author><author><name sortKey="Woodham, L" uniqKey="Woodham L">L Woodham</name></author><author><name sortKey="Donkers, J" uniqKey="Donkers J">J Donkers</name></author><author><name sortKey="Kononowicz, Aa" uniqKey="Kononowicz A">AA Kononowicz</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Garrido, J" uniqKey="Garrido J">J Garrido</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Van Meurs, W" uniqKey="Van Meurs W">W van Meurs</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Schwarz, D" uniqKey="Schwarz D">D Schwarz</name></author><author><name sortKey="Stoura, P" uniqKey="Stoura P">P Štourač</name></author><author><name sortKey="Komenda, M" uniqKey="Komenda M">M Komenda</name></author><author><name sortKey="Harazim, H" uniqKey="Harazim H">H Harazim</name></author><author><name sortKey="Kosinova, M" uniqKey="Kosinova M">M Kosinová</name></author><author><name sortKey="Gregor, J" uniqKey="Gregor J">J Gregor</name></author><author><name sortKey="Hulek, R" uniqKey="Hulek R">R Hůlek</name></author><author><name sortKey="Smekalova, O" uniqKey="Smekalova O">O Smékalová</name></author><author><name sortKey="K Ikava, I" uniqKey="K Ikava I">I Křikava</name></author><author><name sortKey="Stoudek, R" uniqKey="Stoudek R">R Štoudek</name></author><author><name sortKey="Dusek, L" uniqKey="Dusek L">L Dušek</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Ellaway, Rh" uniqKey="Ellaway R">RH Ellaway</name></author><author><name sortKey="Kneebone, R" uniqKey="Kneebone R">R Kneebone</name></author><author><name sortKey="Lachapelle, K" uniqKey="Lachapelle K">K Lachapelle</name></author><author><name sortKey="Topps, D" uniqKey="Topps D">D Topps</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="De Bono, B" uniqKey="De Bono B">B de Bono</name></author><author><name sortKey="Hoehndorf, R" uniqKey="Hoehndorf R">R Hoehndorf</name></author><author><name sortKey="Wimalaratne, S" uniqKey="Wimalaratne S">S Wimalaratne</name></author><author><name sortKey="Gkoutos, G" uniqKey="Gkoutos G">G Gkoutos</name></author><author><name sortKey="Grenon, P" uniqKey="Grenon P">P Grenon</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Halic, T" uniqKey="Halic T">T Halic</name></author><author><name sortKey="Venkata, Sa" uniqKey="Venkata S">SA Venkata</name></author><author><name sortKey="Sankaranarayanan, G" uniqKey="Sankaranarayanan G">G Sankaranarayanan</name></author><author><name sortKey="Lu, Z" uniqKey="Lu Z">Z Lu</name></author><author><name sortKey="Ahn, W" uniqKey="Ahn W">W Ahn</name></author><author><name sortKey="De, S" uniqKey="De S">S De</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Coveney, Pv" uniqKey="Coveney P">PV Coveney</name></author><author><name sortKey="Diaz, V" uniqKey="Diaz V">V Diaz</name></author><author><name sortKey="Hunter, P" uniqKey="Hunter P">P Hunter</name></author><author><name sortKey="Kohl, P" uniqKey="Kohl P">P Kohl</name></author><author><name sortKey="Viceconti, M" uniqKey="Viceconti M">M Viceconti</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Bamidis, Pd" uniqKey="Bamidis P">PD Bamidis</name></author><author><name sortKey="Kaldoudi, E" uniqKey="Kaldoudi E">E Kaldoudi</name></author><author><name sortKey="Pattichis, C" uniqKey="Pattichis C">C Pattichis</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Halic, T" uniqKey="Halic T">T Halic</name></author><author><name sortKey="Ahn, W" uniqKey="Ahn W">W Ahn</name></author><author><name sortKey="De, S" uniqKey="De S">S De</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Checkland, P" uniqKey="Checkland P">P Checkland</name></author><author><name sortKey="Poulter, J" uniqKey="Poulter J">J Poulter</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Ruotsalainen, Ps" uniqKey="Ruotsalainen P">PS Ruotsalainen</name></author><author><name sortKey="Blobel, Bg" uniqKey="Blobel B">BG Blobel</name></author><author><name sortKey="Sepp L, Av" uniqKey="Sepp L A">AV Seppälä</name></author><author><name sortKey="Sorvari, Ho" uniqKey="Sorvari H">HO Sorvari</name></author><author><name sortKey="Nyk Nen, Pa" uniqKey="Nyk Nen P">PA Nykänen</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Viceconti, M" uniqKey="Viceconti M">M Viceconti</name></author><author><name sortKey="Clapworthy, G" uniqKey="Clapworthy G">G Clapworthy</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Hunter, P" uniqKey="Hunter P">P Hunter</name></author><author><name sortKey="Coveney, Pv" uniqKey="Coveney P">PV Coveney</name></author><author><name sortKey="De Bono, B" uniqKey="De Bono B">B de Bono</name></author><author><name sortKey="Diaz, V" uniqKey="Diaz V">V Diaz</name></author><author><name sortKey="Fenner, J" uniqKey="Fenner J">J Fenner</name></author><author><name sortKey="Frangi, Af" uniqKey="Frangi A">AF Frangi</name></author><author><name sortKey="Harris, P" uniqKey="Harris P">P Harris</name></author><author><name sortKey="Hose, R" uniqKey="Hose R">R Hose</name></author><author><name sortKey="Kohl, P" uniqKey="Kohl P">P Kohl</name></author><author><name sortKey="Lawford, P" uniqKey="Lawford P">P Lawford</name></author><author><name sortKey="Mccormack, K" uniqKey="Mccormack K">K McCormack</name></author><author><name sortKey="Mendes, M" uniqKey="Mendes M">M Mendes</name></author><author><name sortKey="Omholt, S" uniqKey="Omholt S">S Omholt</name></author><author><name sortKey="Quarteroni, A" uniqKey="Quarteroni A">A Quarteroni</name></author><author><name sortKey="Sk R, J" uniqKey="Sk R J">J Skår</name></author><author><name sortKey="Tegner, J" uniqKey="Tegner J">J Tegner</name></author><author><name sortKey="Randall Thomas, S" uniqKey="Randall Thomas S">S Randall Thomas</name></author><author><name sortKey="Tollis, I" uniqKey="Tollis I">I Tollis</name></author><author><name sortKey="Tsamardinos, I" uniqKey="Tsamardinos I">I Tsamardinos</name></author><author><name sortKey="Van Beek, Jh" uniqKey="Van Beek J">JH van Beek</name></author><author><name sortKey="Viceconti, M" uniqKey="Viceconti M">M Viceconti</name></author></analytic></biblStruct><biblStruct></biblStruct><biblStruct></biblStruct><biblStruct><analytic><author><name sortKey="Bode, As" uniqKey="Bode A">AS Bode</name></author><author><name sortKey="Huberts, W" uniqKey="Huberts W">W Huberts</name></author><author><name sortKey="Bosboom, Em" uniqKey="Bosboom E">EM Bosboom</name></author><author><name sortKey="Kroon, W" uniqKey="Kroon W">W Kroon</name></author><author><name sortKey="Van Der Linden, Wp" uniqKey="Van Der Linden W">WP van der Linden</name></author><author><name sortKey="Planken, Rn" uniqKey="Planken R">RN Planken</name></author><author><name sortKey="Van De Vosse, Fn" uniqKey="Van De Vosse F">FN van de Vosse</name></author><author><name sortKey="Tordoir, Jh" uniqKey="Tordoir J">JH Tordoir</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Huberts, W" uniqKey="Huberts W">W Huberts</name></author><author><name sortKey="Bode, As" uniqKey="Bode A">AS Bode</name></author><author><name sortKey="Kroon, W" uniqKey="Kroon W">W Kroon</name></author><author><name sortKey="Planken, Rn" uniqKey="Planken R">RN Planken</name></author><author><name sortKey="Tordoir, Jh" uniqKey="Tordoir J">JH Tordoir</name></author><author><name sortKey="Van De Vosse, Fn" uniqKey="Van De Vosse F">FN van de Vosse</name></author><author><name sortKey="Bosboom, Em" uniqKey="Bosboom E">EM Bosboom</name></author></analytic></biblStruct><biblStruct></biblStruct><biblStruct></biblStruct><biblStruct><analytic><author><name sortKey="Kononowicz, Aa" uniqKey="Kononowicz A">AA Kononowicz</name></author><author><name sortKey="Holler, T" uniqKey="Holler T">T Holler</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Kononowicz, Aa" uniqKey="Kononowicz A">AA Kononowicz</name></author><author><name sortKey="Hege, I" uniqKey="Hege I">I Hege</name></author><author><name sortKey="Krawczyk, P" uniqKey="Krawczyk P">P Krawczyk</name></author><author><name sortKey="Zary, N" uniqKey="Zary N">N Zary</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Kolb, Da" uniqKey="Kolb D">DA Kolb</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Huwendiek, S" uniqKey="Huwendiek S">S Huwendiek</name></author><author><name sortKey="Reichert, F" uniqKey="Reichert F">F Reichert</name></author><author><name sortKey="Bosse, Hm" uniqKey="Bosse H">HM Bosse</name></author><author><name sortKey="De Leng, Ba" uniqKey="De Leng B">BA de Leng</name></author><author><name sortKey="Van Der Vleuten, Cp" uniqKey="Van Der Vleuten C">CP van der Vleuten</name></author><author><name sortKey="Haag, M" uniqKey="Haag M">M Haag</name></author><author><name sortKey="Hoffmann, Gf" uniqKey="Hoffmann G">GF Hoffmann</name></author><author><name sortKey="Tonshoff, B" uniqKey="Tonshoff B">B Tönshoff</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Kalet, Al" uniqKey="Kalet A">AL Kalet</name></author><author><name sortKey="Song, Hs" uniqKey="Song H">HS Song</name></author><author><name sortKey="Sarpel, U" uniqKey="Sarpel U">U Sarpel</name></author><author><name sortKey="Schwartz, R" uniqKey="Schwartz R">R Schwartz</name></author><author><name sortKey="Brenner, J" uniqKey="Brenner J">J Brenner</name></author><author><name sortKey="Ark, Tk" uniqKey="Ark T">TK Ark</name></author><author><name sortKey="Plass, J" uniqKey="Plass J">J Plass</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Hege, I" uniqKey="Hege I">I Hege</name></author><author><name sortKey="Kopp, V" uniqKey="Kopp V">V Kopp</name></author><author><name sortKey="Adler, M" uniqKey="Adler M">M Adler</name></author><author><name sortKey="Radon, K" uniqKey="Radon K">K Radon</name></author><author><name sortKey="M Sch, G" uniqKey="M Sch G">G Mäsch</name></author><author><name sortKey="Lyon, H" uniqKey="Lyon H">H Lyon</name></author><author><name sortKey="Fischer, Mr" uniqKey="Fischer M">MR Fischer</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Kim, S" uniqKey="Kim S">S Kim</name></author><author><name sortKey="Phillips, Wr" uniqKey="Phillips W">WR Phillips</name></author><author><name sortKey="Pinsky, L" uniqKey="Pinsky L">L Pinsky</name></author><author><name sortKey="Brock, D" uniqKey="Brock D">D Brock</name></author><author><name sortKey="Phillips, K" uniqKey="Phillips K">K Phillips</name></author><author><name sortKey="Keary, J" uniqKey="Keary J">J Keary</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Le Novere, N" uniqKey="Le Novere N">N Le Novère</name></author><author><name sortKey="Finney, A" uniqKey="Finney A">A Finney</name></author><author><name sortKey="Hucka, M" uniqKey="Hucka M">M Hucka</name></author><author><name sortKey="Bhalla, Us" uniqKey="Bhalla U">US Bhalla</name></author><author><name sortKey="Campagne, F" uniqKey="Campagne F">F Campagne</name></author><author><name sortKey="Collado Vides, J" uniqKey="Collado Vides J">J Collado-Vides</name></author><author><name sortKey="Crampin, Ej" uniqKey="Crampin E">EJ Crampin</name></author><author><name sortKey="Halstead, M" uniqKey="Halstead M">M Halstead</name></author><author><name sortKey="Klipp, E" uniqKey="Klipp E">E Klipp</name></author><author><name sortKey="Mendes, P" uniqKey="Mendes P">P Mendes</name></author><author><name sortKey="Nielsen, P" uniqKey="Nielsen P">P Nielsen</name></author><author><name sortKey="Sauro, H" uniqKey="Sauro H">H Sauro</name></author><author><name sortKey="Shapiro, B" uniqKey="Shapiro B">B Shapiro</name></author><author><name sortKey="Snoep, Jl" uniqKey="Snoep J">JL Snoep</name></author><author><name sortKey="Spence, Hd" uniqKey="Spence H">HD Spence</name></author><author><name sortKey="Wanner, Bl" uniqKey="Wanner B">BL Wanner</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Waltemath, D" uniqKey="Waltemath D">D Waltemath</name></author><author><name sortKey="Adams, R" uniqKey="Adams R">R Adams</name></author><author><name sortKey="Beard, Da" uniqKey="Beard D">DA Beard</name></author><author><name sortKey="Bergmann, Ft" uniqKey="Bergmann F">FT Bergmann</name></author><author><name sortKey="Bhalla, Us" uniqKey="Bhalla U">US Bhalla</name></author><author><name sortKey="Britten, R" uniqKey="Britten R">R Britten</name></author><author><name sortKey="Chelliah, V" uniqKey="Chelliah V">V Chelliah</name></author><author><name sortKey="Cooling, Mt" uniqKey="Cooling M">MT Cooling</name></author><author><name sortKey="Cooper, J" uniqKey="Cooper J">J Cooper</name></author><author><name sortKey="Crampin, Ej" uniqKey="Crampin E">EJ Crampin</name></author><author><name sortKey="Garny, A" uniqKey="Garny A">A Garny</name></author><author><name sortKey="Hoops, S" uniqKey="Hoops S">S Hoops</name></author><author><name sortKey="Hucka, M" uniqKey="Hucka M">M Hucka</name></author><author><name sortKey="Hunter, P" uniqKey="Hunter P">P Hunter</name></author><author><name sortKey="Klipp, E" uniqKey="Klipp E">E Klipp</name></author><author><name sortKey="Laibe, C" uniqKey="Laibe C">C Laibe</name></author><author><name sortKey="Miller, Ak" uniqKey="Miller A">AK Miller</name></author><author><name sortKey="Moraru, I" uniqKey="Moraru I">I Moraru</name></author><author><name sortKey="Nickerson, D" uniqKey="Nickerson D">D Nickerson</name></author><author><name sortKey="Nielsen, P" uniqKey="Nielsen P">P Nielsen</name></author><author><name sortKey="Nikolski, M" uniqKey="Nikolski M">M Nikolski</name></author><author><name sortKey="Sahle, S" uniqKey="Sahle S">S Sahle</name></author><author><name sortKey="Sauro, Hm" uniqKey="Sauro H">HM Sauro</name></author><author><name sortKey="Schmidt, H" uniqKey="Schmidt H">H Schmidt</name></author><author><name sortKey="Snoep, Jl" uniqKey="Snoep J">JL Snoep</name></author><author><name sortKey="Tolle, D" uniqKey="Tolle D">D Tolle</name></author><author><name sortKey="Wolkenhauer, O" uniqKey="Wolkenhauer O">O Wolkenhauer</name></author><author><name sortKey="Le Novere, N" uniqKey="Le Novere N">N Le Novère</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Posel, N" uniqKey="Posel N">N Posel</name></author><author><name sortKey="Fleiszer, D" uniqKey="Fleiszer D">D Fleiszer</name></author><author><name sortKey="Shore, Bm" uniqKey="Shore B">BM Shore</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Kononowicz, Aa" uniqKey="Kononowicz A">AA Kononowicz</name></author><author><name sortKey="Heid, J" uniqKey="Heid J">J Heid</name></author><author><name sortKey="Donkers, J" uniqKey="Donkers J">J Donkers</name></author><author><name sortKey="Hege, I" uniqKey="Hege I">I Hege</name></author><author><name sortKey="Woodham, L" uniqKey="Woodham L">L Woodham</name></author><author><name sortKey="Zary, N" uniqKey="Zary N">N Zary</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Tworek, J" uniqKey="Tworek J">J Tworek</name></author><author><name sortKey="Paget, M" uniqKey="Paget M">M Paget</name></author><author><name sortKey="Mclauglin, K" uniqKey="Mclauglin K">K McLauglin</name></author><author><name sortKey="Wright, B" uniqKey="Wright B">B Wright</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Kononowicz, Aa" uniqKey="Kononowicz A">AA Kononowicz</name></author><author><name sortKey="Zary, N" uniqKey="Zary N">N Zary</name></author><author><name sortKey="Davies, D" uniqKey="Davies D">D Davies</name></author><author><name sortKey="Heid, J" uniqKey="Heid J">J Heid</name></author><author><name sortKey="Woodham, L" uniqKey="Woodham L">L Woodham</name></author><author><name sortKey="Hege, I" uniqKey="Hege I">I Hege</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Bode, As" uniqKey="Bode A">AS Bode</name></author><author><name sortKey="Huberts, W" uniqKey="Huberts W">W Huberts</name></author><author><name sortKey="Bosboom, Em" uniqKey="Bosboom E">EM Bosboom</name></author><author><name sortKey="Kroon, W" uniqKey="Kroon W">W Kroon</name></author><author><name sortKey="Van Der Linden, Wp" uniqKey="Van Der Linden W">WP van der Linden</name></author><author><name sortKey="Planken, Rn" uniqKey="Planken R">RN Planken</name></author><author><name sortKey="Van De Vosse, Fn" uniqKey="Van De Vosse F">FN van de Vosse</name></author><author><name sortKey="Tordoir, Jh" uniqKey="Tordoir J">JH Tordoir</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Passera, K" uniqKey="Passera K">K Passera</name></author><author><name sortKey="Manini, S" uniqKey="Manini S">S Manini</name></author><author><name sortKey="Antiga, L" uniqKey="Antiga L">L Antiga</name></author><author><name sortKey="Remuzzi, A" uniqKey="Remuzzi A">A Remuzzi</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Ellaway, R" uniqKey="Ellaway R">R Ellaway</name></author><author><name sortKey="Topps, D" uniqKey="Topps D">D Topps</name></author><author><name sortKey="Macdonald, J" uniqKey="Macdonald J">J MacDonald</name></author><author><name sortKey="Copeland, B" uniqKey="Copeland B">B Copeland</name></author><author><name sortKey="Olmos, A" uniqKey="Olmos A">A Olmos</name></author><author><name sortKey="Spencer, B" uniqKey="Spencer B">B Spencer</name></author></analytic></biblStruct></listBibl></div1></back></TEI><pmc article-type="research-article"><pmc-dir>properties open_access</pmc-dir>
<front><journal-meta><journal-id journal-id-type="nlm-ta">J Med Internet Res</journal-id><journal-id journal-id-type="publisher-id">JMIR</journal-id><journal-title-group><journal-title>Journal of Medical Internet Research</journal-title></journal-title-group><issn pub-type="ppub">1439-4456</issn><issn pub-type="epub">1438-8871</issn><publisher><publisher-name>JMIR Publications Inc.</publisher-name><publisher-loc>Toronto, Canada</publisher-loc></publisher></journal-meta><article-meta><article-id pub-id-type="pmid">24463466</article-id><article-id pub-id-type="pmc">3906686</article-id><article-id pub-id-type="publisher-id">v16i1e23</article-id><article-id pub-id-type="doi">10.2196/jmir.2593</article-id><article-categories><subj-group subj-group-type="heading"><subject>Original Paper</subject></subj-group></article-categories><title-group><article-title>A Framework for Different Levels of Integration of Computational Models Into Web-Based Virtual Patients</article-title></title-group><contrib-group><contrib contrib-type="editor"><name><surname>Bamidis</surname><given-names>Panagiotis</given-names></name></contrib><contrib contrib-type="editor"><name><surname>Giordano</surname><given-names>Daniela</given-names></name></contrib></contrib-group><contrib-group><contrib contrib-type="reviewer"><name><surname>Lok</surname><given-names>Benjamin</given-names></name></contrib><contrib contrib-type="reviewer"><name><surname>Woodham</surname><given-names>Luke</given-names></name></contrib><contrib contrib-type="reviewer"><name><surname>Halic</surname><given-names>Tansel</given-names></name></contrib></contrib-group><contrib-group><contrib id="contrib1" contrib-type="author" corresp="yes"><name><surname>Kononowicz</surname><given-names>Andrzej A</given-names></name><degrees>PhD</degrees><xref ref-type="aff" rid="aff1">1</xref><address><institution>Digital Patient Lab</institution><institution>Department of Learning, Informatics, Management and Ethics</institution><institution>Karolinska Institutet</institution><addr-line>Tomtebodavägen 18A</addr-line><addr-line>Stockholm, 171 77</addr-line><country>Sweden</country><phone>46 8 524 83626</phone><fax>46 8 524 800 00</fax><email>andrzej.kononowicz@ki.se</email></address><xref ref-type="aff" rid="aff2">2</xref></contrib><contrib id="contrib2" contrib-type="author"><name><surname>Narracott</surname><given-names>Andrew J</given-names></name><degrees>PhD</degrees><xref ref-type="aff" rid="aff3">3</xref><xref ref-type="aff" rid="aff4">4</xref></contrib><contrib id="contrib3" contrib-type="author"><name><surname>Manini</surname><given-names>Simone</given-names></name><degrees>BEng</degrees><xref ref-type="aff" rid="aff5">5</xref><xref ref-type="aff" rid="aff6">6</xref></contrib><contrib id="contrib4" contrib-type="author"><name><surname>Bayley</surname><given-names>Martin J</given-names></name><degrees>PhD</degrees><xref ref-type="aff" rid="aff3">3</xref><xref ref-type="aff" rid="aff4">4</xref></contrib><contrib id="contrib5" contrib-type="author"><name><surname>Lawford</surname><given-names>Patricia V</given-names></name><degrees>PhD</degrees><xref ref-type="aff" rid="aff3">3</xref><xref ref-type="aff" rid="aff4">4</xref></contrib><contrib id="contrib6" contrib-type="author"><name><surname>McCormack</surname><given-names>Keith</given-names></name><degrees>PhD</degrees><xref ref-type="aff" rid="aff3">3</xref><xref ref-type="aff" rid="aff4">4</xref></contrib><contrib id="contrib7" contrib-type="author"><name><surname>Zary</surname><given-names>Nabil</given-names></name><degrees>MD, PhD</degrees><xref ref-type="aff" rid="aff1">1</xref></contrib></contrib-group><aff id="aff1" rid="aff1"><sup>1</sup><institution>Digital Patient Lab</institution><institution>Department of Learning, Informatics, Management and Ethics</institution><institution>Karolinska Institutet</institution><addr-line>Stockholm</addr-line><country>Sweden</country></aff><aff id="aff2" rid="aff2"><sup>2</sup><institution>Department of Bioinformatics and Telemedicine</institution><institution>Faculty of Medicine</institution><institution>Jagiellonian University</institution><addr-line>Kraków</addr-line><country>Poland</country></aff><aff id="aff3" rid="aff3"><sup>3</sup><institution>Medical Physics Group</institution><institution>Department of Cardiovascular Science, Faculty of Medicine, Dentistry and Health</institution><institution>University of Sheffield</institution><addr-line>Sheffield</addr-line><country>United Kingdom</country></aff><aff id="aff4" rid="aff4"><sup>4</sup><institution>INSIGNEO Institute for in silico Medicine</institution><institution>University of Sheffield</institution><addr-line>Sheffield</addr-line><country>United Kingdom</country></aff><aff id="aff5" rid="aff5"><sup>5</sup><institution>Department of Biomedical Engineering</institution><institution>IRCSS Mario Negri Institute</institution><addr-line>Bergamo</addr-line><country>Italy</country></aff><aff id="aff6" rid="aff6"><sup>6</sup><institution>Orobix Srl</institution><addr-line>Bergamo</addr-line><country>Italy</country></aff><author-notes><corresp>Corresponding Author: Andrzej A Kononowicz <email>andrzej.kononowicz@ki.se</email></corresp></author-notes><pub-date pub-type="collection"><month>1</month><year>2014</year></pub-date><pub-date pub-type="epub"><day>23</day><month>1</month><year>2014</year></pub-date><volume>16</volume><issue>1</issue><elocation-id>e23</elocation-id><history><date date-type="received"><day>01</day><month>3</month><year>2013</year></date><date date-type="rev-request"><day>12</day><month>5</month><year>2013</year></date><date date-type="rev-recd"><day>28</day><month>5</month><year>2013</year></date><date date-type="accepted"><day>27</day><month>12</month><year>2013</year></date></history><permissions><copyright-statement>©Andrzej A Kononowicz, Andrew J Narracott, Simone Manini, Martin J Bayley, Patricia V Lawford, Keith McCormack, Nabil Zary. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 23.01.2014. </copyright-statement><copyright-year>2014</copyright-year><license license-type="open-access" xlink:href="http://creativecommons.org/licenses/by/2.0/"><license-p><pmc-comment>CREATIVE COMMONS</pmc-comment>This is an open-access article distributed under the terms of the Creative Commons Attribution License (<ext-link ext-link-type="uri" xlink:href="http://creativecommons.org/licenses/by/2.0/">http://creativecommons.org/licenses/by/2.0/</ext-link>), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on <ext-link ext-link-type="uri" xlink:href="http://www.jmir.org/,">http://www.jmir.org/,</ext-link> as well as this copyright and license information must be included.</license-p></license></permissions><self-uri xlink:type="simple" xlink:href="http://www.jmir.org/2014/1/e23/"></self-uri><abstract><sec sec-type="background"><title>Background</title><p>Virtual patients are increasingly common tools used in health care education to foster learning of clinical reasoning skills. One potential way to expand their functionality is to augment virtual patients’ interactivity by enriching them with computational models of physiological and pathological processes.</p></sec><sec sec-type="objective"><title>Objective</title><p>The primary goal of this paper was to propose a conceptual framework for the integration of computational models within virtual patients, with particular focus on (1) characteristics to be addressed while preparing the integration, (2) the extent of the integration, (3) strategies to achieve integration, and (4) methods for evaluating the feasibility of integration. An additional goal was to pilot the first investigation of changing framework variables on altering perceptions of integration.</p></sec><sec sec-type="methods"><title>Methods</title><p>The framework was constructed using an iterative process informed by Soft System Methodology. The Virtual Physiological Human (VPH) initiative has been used as a source of new computational models. The technical challenges associated with development of virtual patients enhanced by computational models are discussed from the perspectives of a number of different stakeholders. Concrete design and evaluation steps are discussed in the context of an exemplar virtual patient employing the results of the VPH ARCH project, as well as improvements for future iterations.</p></sec><sec sec-type="results"><title>Results</title><p>The proposed framework consists of four main elements. The first element is a list of feasibility features characterizing the integration process from three perspectives: the computational modelling researcher, the health care educationalist, and the virtual patient system developer. The second element included three integration levels: basic, where a single set of simulation outcomes is generated for specific nodes in the activity graph; intermediate, involving pre-generation of simulation datasets over a range of input parameters; advanced, including dynamic solution of the model. The third element is the description of four integration strategies, and the last element consisted of evaluation profiles specifying the relevant feasibility features and acceptance thresholds for specific purposes. The group of experts who evaluated the virtual patient exemplar found higher integration more interesting, but at the same time they were more concerned with the validity of the result. The observed differences were not statistically significant.</p></sec><sec sec-type="conclusions"><title>Conclusions</title><p>This paper outlines a framework for the integration of computational models into virtual patients. The opportunities and challenges of model exploitation are discussed from a number of user perspectives, considering different levels of model integration. The long-term aim for future research is to isolate the most crucial factors in the framework and to determine their influence on the integration outcome.</p></sec></abstract><kwd-group><kwd>computer simulation</kwd><kwd>computer-assisted instruction</kwd><kwd>education, medical</kwd><kwd>medical informatics applications</kwd></kwd-group></article-meta></front><body><sec><title>Introduction</title><sec><title>Background</title><p>Computers and Internet technologies have already entered the mainstream of health care education [<xref rid="ref1" ref-type="bibr">1</xref>]. Although a gap still exists between what is technically possible in pilot studies and the realities of educational practice, we have reached a stage where Web-based training is regarded as routine [<xref rid="ref2" ref-type="bibr">2</xref>,<xref rid="ref3" ref-type="bibr">3</xref>]. The use of such technology is driven not only by the urge for innovation, a willingness to improve teaching quality, and limited personnel, but also by cost-effectiveness analysis [<xref rid="ref4" ref-type="bibr">4</xref>].</p><p>The use of virtual patients is undeniably one of the techniques most often associated with application of computer-aided training in health care [<xref rid="ref2" ref-type="bibr">2</xref>,<xref rid="ref5" ref-type="bibr">5</xref>] but, as is often the case with new concepts, the understanding of the term “virtual patient” varies depending on the research community. We define a virtual patient as “interactive computer simulation of real-life clinical scenarios for the purpose of health care and medical training, education or assessment” [<xref rid="ref6" ref-type="bibr">6</xref>]. This excludes other methods used in medical education such as human role-playing, computerized mannequins, part-task trainers, and systems requiring specialized equipment [<xref rid="ref7" ref-type="bibr">7</xref>], as well as all non-educational virtual patients.</p><p>While complex, immersive virtual reality scenarios are technologically possible (eg, [<xref rid="ref8" ref-type="bibr">8</xref>]), the routine use of virtual patients often focuses on technically simple solutions. Huwendiek et al attempted to classify this type of virtual patient solution with four example systems: CAMPUS, CASUS, Open Labyrinth, and Web-SP [<xref rid="ref9" ref-type="bibr">9</xref>]. While these systems differ in many respects, they have in common the presentation of a clinical, case-based scenario divided into discrete steps displayed either (1) linearly, with a single final outcome, or (2) branched, enabling different narration paths depending on user choice. All these systems are Web-based and simply require an Internet connection through a standard browser. By common agreement, the ANSI-accredited MedBiquitous Virtual Patient (MVP) standard [<xref rid="ref10" ref-type="bibr">10</xref>] has been adopted by the medical education community for the representation of virtual patient data. Implementation of support for the standard in four systems was achieved as part of the eViP project [<xref rid="ref11" ref-type="bibr">11</xref>].</p><p>There are at least two significant advantages of this type of virtual patient. The first is the possibility of significant teacher involvement in the development process. This has been enabled by investment in the user-friendliness of authoring tools and the simplification of technical workflows to enable medical experts to focus on the content of virtual patients. As a result, virtual patients of this class are generally tailored to the needs of a particular teacher and institution, to the type of educational activity, and to the specific learning objectives. This might not be the case with technologically complex virtual patients. The second advantage is the high level of accessibility of this type of virtual patient to learners over the Internet with personal computers, or even “just-in-time” access with mobile devices. Accessibility is also enhanced by the low cost of licensing of these virtual patient systems.</p><p>There are several possible “next steps” in the development of virtual patients from the class described above. Our chosen step is to increase the interactivity of virtual patients by enriching them with computational models of physiological and pathological processes. Here we use the term “computational model” following the definition given by Garrido [<xref rid="ref12" ref-type="bibr">12</xref>] as a “mathematical model implemented in a computer system that requires high performance computational resources to execute”. This usually requires the numerical solution of non-linear models. Being cognizant of the long research tradition of modeling and simulation in biomedical engineering (eg, [<xref rid="ref13" ref-type="bibr">13</xref>]), it is not our intention to create new models but rather to seek opportunities to integrate existing models into present virtual patient systems.</p><p>Here we propose a framework to support integration of existing computational models of physiological function within virtual patients. This framework is expected to define concepts that help in formalizing the integration and evaluating the feasibility of the integration. Extending virtual patients with computational models is recognized as a promising trend in the current virtual patient literature (eg, [<xref rid="ref3" ref-type="bibr">3</xref>,<xref rid="ref14" ref-type="bibr">14</xref>]).</p><p>While integration frameworks for simulation in health care education have been published previously (eg, [<xref rid="ref15" ref-type="bibr">15</xref>-<xref rid="ref17" ref-type="bibr">17</xref>]), the focus of these earlier publications is different from that envisaged here. The “Practica continua” framework, developed by Ellaway et al [<xref rid="ref15" ref-type="bibr">15</xref>], discusses integration of a broad range of simulation modalities, covering methods such as human role-playing (eg, standardized patients), physical equipment (eg, computerized mannequins, part-body task trainers, and haptic devices), teleconferencing, and 3D visualization tools. The framework we present is focused solely on integration of physiological computational models derived from existing research projects, such as the Virtual Physiological Human (VPH) initiative [<xref rid="ref18" ref-type="bibr">18</xref>], within screen-based, narrative virtual patient systems (as used in the eViP project [<xref rid="ref9" ref-type="bibr">9</xref>]).</p><p>The RICORDO framework has a more technical scope, with the aim of improving the accessibility of simulation data and model resources for physiology and pharmacology research, as well as health care education [<xref rid="ref16" ref-type="bibr">16</xref>]. This involves selection of technical standards for the use of controlled vocabularies, ontologies, and metadata encoding representation. In addition, an open toolkit for creation of ontology composites and annotation of resources was developed. A critical appraisal of technical standards for sharing educational resources in medical education was also undertaken by the mEducator Best Practice Network [<xref rid="ref19" ref-type="bibr">19</xref>]. These contributions can complement the methods described in the current paper by facilitating the discovery of new models to be integrated in existing virtual patients and by promoting dissemination of these based on clinical classification of the content.</p><p>Finally, frameworks are described within the literature, which facilitate the integration of various software libraries for high-fidelity medical simulation. For instance, Halic et al proposed a framework called SoFMIS for rendering 3D scenes in surgical simulation involving the use of haptic devices [<xref rid="ref17" ref-type="bibr">17</xref>]. A related framework adopts the SoFMIS architecture for simulation in Web environments [<xref rid="ref20" ref-type="bibr">20</xref>]. In contrast to this approach, our framework deals more with virtual patient authoring strategies than with concrete software solutions. The SoFMIS framework demonstrates how different software libraries may be grouped in modules supporting reusability. We intend to extend the existing, well-established systems with new elements. Furthermore, our virtual patients are directed towards improving clinical reasoning skills (as diagnostic and treatment selection processes) and do not focus on the psychomotor and procedural skills that would be required in high-fidelity surgical simulations.</p></sec><sec><title>Objective</title><p>The aim of this paper is to propose a conceptual framework to organize research and development towards the integration of computational models within virtual patients. In particular, we focus on (1) characteristics to be addressed while preparing the integration, (2) the extent of the integration, (3) strategies to achieve integration, and (4) methods for evaluating the feasibility of integration.</p><p>The framework will be considered successful if its application leads to practical recommendations and predictions enabling the medical education and simulation communities to collaborate in the integration of computational models within virtual patients. Our long-term aim is to isolate the crucial factors and determine their influences on the success of the integration process in supporting the practical use of the virtual patient for teaching. One way of achieving this is to evaluate the reaction of a suitable target audience to virtual patients implemented with different levels of model integration proposed by the framework. For this reason, an additional goal is to pilot an investigation of altered perceptions of virtual patients at two levels of integration.</p></sec></sec><sec sec-type="methods"><title>Methods</title><sec><title>Process for Developing the Framework</title><p>The framework will be initiated using an iterative process inspired by the Soft System Methodology (SSM) [<xref rid="ref21" ref-type="bibr">21</xref>]. The SSM method was designed to aid analysis of situations involving groups with interacting perceptions where human-related aspects play an important role. The methodology looks for solutions that are both desirable and feasible considering the components of the system.</p><p>While developing virtual patients, we should address human-related aspects. For instance, their construction should be influenced by actual learning needs and preferences of learners and not by the accuracy of simulation results alone. This makes SSM methodology particularly suitable for the purpose of developing the framework. This methodology has been previously applied to design conceptual frameworks to introduce information technologies into health care, for instance by Ruotsalainen et al to define a framework for Trusted Pervasive Health [<xref rid="ref22" ref-type="bibr">22</xref>].</p><p>The SSM method consists of four steps, constituting a learning cycle, which are repeated to improve the system design [<xref rid="ref21" ref-type="bibr">21</xref>]. This involves (SSM Step 1) characterization of the system in question including description of its features, problems of interest, risks, and challenges, (SSM Step 2) expression of the ideas and planned actions in the form of models encapsulating different perceptions, (SSM Step 3) testing of the model on a real-world example, and (SSM Step 4) synthesis of the system description with verification outcomes to propose improvements for the next development cycle.</p></sec><sec><title>Source of Computational Models</title><p>The initial framework design is informed by experience acquired by the authors in previous projects dealing with virtual patients, health care education and biomedical modeling simulation, including eViP [<xref rid="ref6" ref-type="bibr">6</xref>] and the Virtual Physiological Human (VPH) initiative [<xref rid="ref18" ref-type="bibr">18</xref>]. The recently established collaboration between the VPH Network of Excellence (NoE) and representatives of the Association for Medical Education in Europe (AMEE) health care educational community provided an appropriate environment to develop the framework.</p><p>The overarching aim of the VPH is to establish methods and tools for computational analysis of the human body, integrating the diverse nature of physiology and pathophysiology of the different organ systems [<xref rid="ref18" ref-type="bibr">18</xref>]. The VPH combines expertise in computer modeling and clinical research to deliver a spectrum of advanced simulations of physiological function, disease development and progression, and response to intervention. The ambitious goal is to develop a technological infrastructure based on patient-specific data to deliver prediction of clinical outcomes by means of quantitative models that integrate biophysical processes across diverse scales from the molecular level, to organ systems, and even populations [<xref rid="ref23" ref-type="bibr">23</xref>]. The clinical targets of VPH projects are diverse and address the challenges of computational modeling in cardiology (eg, euHeart), vascular pathology (eg, ARCH), or various types of cancer (eg, PASSPORT) [<xref rid="ref24" ref-type="bibr">24</xref>]. Effort has also been allocated to address the technical aspects of the computational infrastructure required by VPH projects (eg, VPH Share).</p><p>The VPH NoE [<xref rid="ref25" ref-type="bibr">25</xref>] has provided a central focus for VPH project output, developing best practice and support for the exposure and sustainability of VPH-related tools, training, and standards. An important task of the VPH NoE has been to ensure that the academic, medical, and industrial domains have access to a workforce that is well prepared to meet the possibilities offered by the VPH. This has been addressed through delivery of training activities (workshops, summer schools) and the development of educational materials to raise awareness of these new technologies [<xref rid="ref24" ref-type="bibr">24</xref>]. The educational goals of VPH NoE align well with the aims of the virtual patient community for more interactive virtual patients.</p></sec><sec><title>Demonstration of the Framework</title><sec><title>Selected Model for Integration</title><p>The proposed framework is showcased using a real-world example: the VPH ARCH project [<xref rid="ref26" ref-type="bibr">26</xref>]. This project developed vascular access modeling for surgical planning for hemodialysis in end-stage renal disease (ESRD) patients. While there are several methods to create vascular access for ESRD patients, the arteriovenous fistula (AVF) is preferred due to reduced potential complications. An AVF may be created either in the lower arm (radiocephalic fistula) or upper arm (brachiocephalic or brachiobasilic fistula) [<xref rid="ref27" ref-type="bibr">27</xref>]. Deciding where to create the fistula is informed by preoperative physical examination and duplex ultrasound evaluation of the vasculature of the arm. Within the VPH ARCH project a computational model was developed to support clinical decision making. The simulation is based on a distributed lumped-parameter implementation of a wave propagation model [<xref rid="ref28" ref-type="bibr">28</xref>], which has been evaluated with clinical data. The computational model is available under an open-source license as part of a software toolkit called archTk [<xref rid="ref29" ref-type="bibr">29</xref>] and as part of a clinical Web application [<xref rid="ref30" ref-type="bibr">30</xref>].</p></sec><sec><title>The Integration Process</title><p>Using the archTk model and a set of patient input parameters, we simulated blood flow and pressure in the upper extremity after implementing different types of AVF, incorporating simulation outcomes into a virtual patient. The case presents the story of John Jones: a 68-year-old male who was diagnosed with chronic kidney disease following an infection. The virtual patient was developed under the supervision of clinical experts by authors of this paper. The basic version contains 28 screen cards connected in a graph structure to provide branching possibilities (<xref ref-type="fig" rid="figure1">Figure 1</xref>). The screen cards comprise narratives, medical examination results (including simulation outcomes), and images. The integration was performed using Bit Pathways [<xref rid="ref31" ref-type="bibr">31</xref>,<xref rid="ref32" ref-type="bibr">32</xref>]. This authoring tool was developed by one of the paper’s authors and allows export of the graph structure, including the case data, as an MVP package [<xref rid="ref10" ref-type="bibr">10</xref>]. Such reusable learning objects are readable by all MVP-supported virtual patient systems. Alternatively, the content may be wrapped by the Bit Pathways authoring tool into an HTML/Java Script player to form a stand-alone or Web-based application (<xref ref-type="fig" rid="figure1">Figure 1</xref>).</p><fig id="figure1" position="float"><label>Figure 1</label><caption><p>The virtual patient case containing archTk simulation outcome wrapped in a stand-alone, Web-enabled virtual patient player.</p></caption><graphic xlink:href="jmir_v16i1e23_fig1"></graphic></fig></sec></sec><sec><title>Evaluation Event</title><sec><title>Context Description</title><p>To test the potential of the proposed framework as a tool for discriminating between different levels of simulation integration, a virtual patient exposing the archTk simulation results was prepared in two variants, reflecting the different integration levels defined by the framework. Both variants of the case were demonstrated during a dissemination and evaluation workshop organized by the VPH NoE at the European Vascular Course (EVC) conference in Maastricht in March 2013. Access to the virtual patients was offered to the participants by 10 Internet-connected Apple iPad 2 tablets. The virtual patient variants were stored on a remote Web server. The user interface of the Web application was tailored to the size and navigation possibilities of the tablet’s touch screen. Participants were allocated to one of the two virtual patient variants at random (Study Groups 1 and 2) based on the tablet the user selected to sit at in the workshop. Participants were blinded to this selection to prevent bias (Hawthorne effect).</p><p>Participation in the workshop was voluntary. Participants were recruited by email announcements broadcast by the conference organizers, posters at the conference venue, and direct invitations with pamphlets handed out during coffee breaks. Participation required registration for a specific workshop session due to a limit of 10 individuals per session. Use of the virtual patient was preceded by a 15-minute introduction to the VPH initiative and virtual patient tools. A Web-based evaluation questionnaire, authored with Google Forms, was completed immediately after the session. User identification was based only on the tablet ID and time of questionnaire completion. The questions asked pertained to the level of interest in the virtual patient and agreement with the content. In addition, the study group dealing with the higher integration level were given the possibility of commenting on the added interactivity (as explained in the results section of this paper).</p></sec><sec><title>Study Participants</title><p>Thirty-eight participants filled in the questionnaire. The distribution of the participants in study groups was well balanced with 20 (53%) in Study Group 1 and 18 (47%) in Study Group 2. Basic demographic and background data are presented in <xref ref-type="table" rid="table1">Table 1</xref>. Most of the participants were clinicians specializing in vascular surgery or nephrology (28/38, 74%). The remaining participants were nurses, biomedical engineers, and producers of medical equipment (9/38, 24%; 1 missing response).</p><table-wrap id="table1" position="float"><label>Table 1</label><caption><p>Summary of the study participants’ demographic data.</p></caption><table frame="hsides" rules="groups" width="619" border="0" cellpadding="7" cellspacing="0"><col width="356" span="1"></col><col width="96" span="1"></col><col width="125" span="1"></col><thead><tr valign="top"><td rowspan="1" colspan="1"><break></break></td><td rowspan="1" colspan="1">Group 1</td><td rowspan="1" colspan="1">Group 2</td></tr></thead><tbody><tr valign="top"><td rowspan="1" colspan="1">Number of participants</td><td rowspan="1" colspan="1">20</td><td rowspan="1" colspan="1">18</td></tr><tr valign="top"><td rowspan="1" colspan="1">Gender: female/male/no answer, %</td><td rowspan="1" colspan="1">75/25/–</td><td rowspan="1" colspan="1">61/22/17</td></tr><tr valign="top"><td rowspan="1" colspan="1">Age, yrs, mean (SD)</td><td rowspan="1" colspan="1">41.6 (12.6)</td><td rowspan="1" colspan="1">36.0 (11.8)</td></tr><tr valign="top"><td rowspan="1" colspan="1">Physician, %</td><td rowspan="1" colspan="1">85</td><td rowspan="1" colspan="1">61</td></tr><tr valign="top"><td rowspan="1" colspan="1">Work experience, yrs, mean (SD)</td><td rowspan="1" colspan="1">11.8 (12.5)</td><td rowspan="1" colspan="1">9.5 (9.0)</td></tr><tr valign="top"><td rowspan="1" colspan="1">Involved in teaching, %</td><td rowspan="1" colspan="1">80</td><td rowspan="1" colspan="1">72</td></tr></tbody></table></table-wrap></sec><sec><title>Statistical Analysis</title><p>Differences between the two groups were assessed using a non-parametric double-sided Mann-Whitney U-test. The significance level (alpha) was set to .05. The statistical analysis was performed using the R statistical package version 3.0.1 (the R Foundation for Statistical Computing, 2013).</p></sec></sec></sec><sec sec-type="results"><title>Results</title><sec><title>The Integration Framework</title><sec><title>System Definition</title><p>The central element of SSM methodology is a real-world “problematic situation”. In our case, we define it as low level of interactivity of virtual patients to be increased through the integration of existing simulation technologies. We envisage a community where computational models produced by research projects are publicly available on the Internet and selected by the health care education community to extend the interactivity of existing or newly developed virtual patients.</p><p>The challenges of this approach are defined by different “worldviews” of the stakeholders (SSM Step 1) that include the biomedical modeling community, health care educators and learners, and virtual patient system developers.</p><p>The current perspective of the biomedical modeling community is well represented by the diversity of the VPH initiative. It focuses on the development and validation of patient-specific simulations to address clinical problems relevant to a particular patient population. Model construction is intellectually demanding with, as yet, little formalization. This focus is driven by researcher motivation, clinical drivers, and the availability of research funding, involving state-of-the-art technologies (eg, grid systems, cloud computing, supercomputers). One priority of the VPH initiative is to develop models and simulations that are able to provide a holistic view of the human body and able to deliver a technical infrastructure capable of providing access to different models and tools within a single computational workflow. Authoring of models and simulations is typically carried out in large international teams and requires formal consideration of ethical and intellectual property issues, particularly when industrial partners are involved. VPH researchers are largely unaware of the learning objectives of medical curricula, and the research focus of projects means that resources to adapt or maintain the tools for educational purposes are limited.</p><p>The worldviews of health care educators and learners, though clearly distinct, are closely interconnected and can be treated as one subsystem at the current stage of development of this framework. Driven by sound theoretical foundations, such as Kolb’s experiential learning theory [<xref rid="ref33" ref-type="bibr">33</xref>], the community is building up an educational system where the learners experience the subject of interest by active experimentation, observations, and reflection. This group is willing to use simulations that clearly address learning objectives within the contemporary medical curriculum. It is important to consider factors such as the appropriate level of difficulty of the task and to promote high levels of interaction with the content or content-specific feedback [<xref rid="ref34" ref-type="bibr">34</xref>]. However, caution is advised because too much interactivity may be detrimental to the learning process [<xref rid="ref35" ref-type="bibr">35</xref>]. Most educators and students are not interested in, or do not have time, to study the technology that underpins the simulation. Their major expectation is to obtain a tool that is intuitive, works without delays, is well aligned with previous learning experience, and returns valid results. This group usually has access to standard IT equipment and Internet access, and neither the faculty nor the learners are willing to cover substantial additional expenses for access to simulation.</p><p>The worldview of the virtual patient system developer is, perhaps, not as clear to an outside observer. It is important to consider this perspective as the technical infrastructure in many medical universities has been in development for several years and is well established for educational workflows, particularly in the case of virtual patient systems [<xref rid="ref36" ref-type="bibr">36</xref>]. Replacing well-functioning systems or adding new systems to an already complex e-learning infrastructure is likely to encounter resistance. These objections will be supported by health care educators who have already invested substantial resources in creating content for existing virtual patient systems. The virtual patient format, while existing in several variants, follows similar design rules [<xref rid="ref34" ref-type="bibr">34</xref>,<xref rid="ref37" ref-type="bibr">37</xref>]. In practice, this involves the presentation of a consistent story for a single patient, with new information unraveling over time. Virtual patient systems, which have been implemented in different technologies, have already achieved a common technological denominator in the form of the MVP standard [<xref rid="ref10" ref-type="bibr">10</xref>].</p></sec><sec><title>Feasibility Features</title><p>The different perspectives characterized within the system definition were transformed into system features (<xref ref-type="table" rid="table2">Table 2</xref>). This list contains factors that should be considered when assessing the feasibility of simulation integration. To ensure that the most significant and timely factors have been included, its completeness has been validated against features identified in previous studies presenting the views of the three stakeholder groups (sources [<xref rid="ref9" ref-type="bibr">9</xref>,<xref rid="ref11" ref-type="bibr">11</xref>,<xref rid="ref16" ref-type="bibr">16</xref>,<xref rid="ref34" ref-type="bibr">34</xref>,<xref rid="ref37" ref-type="bibr">37</xref>-<xref rid="ref41" ref-type="bibr">41</xref>] are shown in the third column in <xref ref-type="table" rid="table2">Table 2</xref>). The list is not assumed to be exhaustive and has the potential to be extended in future.</p><table-wrap id="table2" position="float"><label>Table 2</label><caption><p>Features relevant for integrating computational models with virtual patients.</p></caption><table frame="hsides" rules="groups" width="619" border="0" cellpadding="7" cellspacing="0"><col width="105" span="1"></col><col width="411" span="1"></col><col width="61" span="1"></col><thead><tr valign="top"><td rowspan="1" colspan="1">Stakeholder group</td><td rowspan="1" colspan="1">Identified features relevant for integrating computational models with virtual patients</td><td rowspan="1" colspan="1">Previous study</td></tr></thead><tbody><tr valign="top"><td rowspan="8" colspan="1">Computational modeling researchers</td><td rowspan="1" colspan="1">Availability of high-quality documentation (including a clear description of modeled parameters: their permitted input ranges, simulation steps, and post-processing steps)</td><td rowspan="8" colspan="1">[<xref rid="ref16" ref-type="bibr">16</xref>,<xref rid="ref38" ref-type="bibr">38</xref>,<xref rid="ref39" ref-type="bibr">39</xref>]</td></tr><tr valign="top"><td rowspan="1" colspan="1">Validity of simulation results generated (compatibility with experimental data or expected observations)</td></tr><tr valign="top"><td rowspan="1" colspan="1">Availability of model in machine-readable (preferably popular) format</td></tr><tr valign="top"><td rowspan="1" colspan="1">Availability (and preferably mobility) of the simulation software for the model</td></tr><tr valign="top"><td rowspan="1" colspan="1">Information on the magnitude of computational time required for simulation</td></tr><tr valign="top"><td rowspan="1" colspan="1">Information on mobility and required storage space demands for input and output data, model, solver</td></tr><tr valign="top"><td rowspan="1" colspan="1">Clearance of copyright issues (information about the authors of the model and terms of use and distribution)</td></tr><tr valign="top"><td rowspan="1" colspan="1">Description of confidentiality constraints</td></tr><tr valign="top"><td rowspan="14" colspan="1">Health care education (educators and students)</td><td rowspan="1" colspan="1">Suitable learning objectives</td><td rowspan="14" colspan="1">[<xref rid="ref34" ref-type="bibr">34</xref>,<xref rid="ref37" ref-type="bibr">37</xref>,<xref rid="ref40" ref-type="bibr">40</xref>]</td></tr><tr valign="top"><td rowspan="1" colspan="1">Relevance for study</td></tr><tr valign="top"><td rowspan="1" colspan="1">Suitable target group</td></tr><tr valign="top"><td rowspan="1" colspan="1">Appropriate level of difficulty</td></tr><tr valign="top"><td rowspan="1" colspan="1">High interactivity</td></tr><tr valign="top"><td rowspan="1" colspan="1">Availability of specific feedback</td></tr><tr valign="top"><td rowspan="1" colspan="1">Optimal use of media</td></tr><tr valign="top"><td rowspan="1" colspan="1">Focus of attention on relevant learning points</td></tr><tr valign="top"><td rowspan="1" colspan="1">Recapitulation of key learning points</td></tr><tr valign="top"><td rowspan="1" colspan="1">Authentic Web-based interface</td></tr><tr valign="top"><td rowspan="1" colspan="1">Content tailored to the clinical reasoning process</td></tr><tr valign="top"><td rowspan="1" colspan="1">Realistic narration to include the simulation in the case</td></tr><tr valign="top"><td rowspan="1" colspan="1">Support for individualized approach to learning</td></tr><tr valign="top"><td rowspan="1" colspan="1">Support for collaborative learning</td></tr><tr valign="top"><td rowspan="2" colspan="1">Virtual patient system developers</td><td rowspan="1" colspan="1">Simulation elements supported by the virtual patient system</td><td rowspan="2" colspan="1">[<xref rid="ref9" ref-type="bibr">9</xref>,<xref rid="ref11" ref-type="bibr">11</xref>,<xref rid="ref41" ref-type="bibr">41</xref>]</td></tr><tr valign="top"><td rowspan="1" colspan="1">Simulation elements supported by the MVP standard</td></tr></tbody></table></table-wrap></sec><sec><title>Integration Levels</title><p>Analysis of the feasibility features presented above depends largely on the extent to which the VPH-related data, models, or simulations are integrated within the virtual patient. This may be addressed by introducing different levels of integration. We propose three fundamental integration levels: (1) basic, (2) intermediate, and (3) advanced (<xref ref-type="fig" rid="figure2">Figure 2</xref>). In the discussion that follows, it is assumed that the virtual patient has a branching navigation model where the learner can select from a number of alternative options, solving the virtual patient with different possible end points [<xref rid="ref9" ref-type="bibr">9</xref>]. The alternative linear model, where there is just one narration thread and one final end point, can be regarded as a special case of branching.</p><p>At the basic integration level, a single set of <italic>n</italic> input parameters is taken to generate a single outcome (or a single time-dependent outcome series) of biomedical variables for a single (virtual) patient. The results are reported within the narration of the virtual patient. For the intermediate level, pre-generation of simulation data over a range of input parameters is performed and included within the virtual patient package, for instance in the form of a look-up table. The execution environment enables the student to explore <italic>i</italic> different variants of data combination within the predefined constraints. The advanced level proposes a dynamic solution of the model while allowing the learner to work directly with the solver and freely specify the input within the domain of variables.</p><p><xref ref-type="fig" rid="figure2">Figure 2</xref> shows that, as the level of model integration is increased, the run-time control of the virtual patient simulation shifts from simulation infrastructure (green background in <xref ref-type="fig" rid="figure2">Figure 2</xref>) to the virtual patient player (red background in <xref ref-type="fig" rid="figure2">Figure 2</xref>). In the case of advanced integration, the model solver becomes an integral part of the virtual patient environment, enabling more self-reliance for the learner and, consequentially, supporting an explorative learning approach to a larger extent.</p><fig id="figure2" position="float"><label>Figure 2</label><caption><p>Three levels of integration of computational models into virtual patients.</p></caption><graphic xlink:href="jmir_v16i1e23_fig2"></graphic></fig></sec></sec><sec><title>Integration Strategies</title><sec><title>Overview</title><p>Integration strategies provide guidance on how to implement the integration in practice. The basic level integration strategy is to manually copy the output of the solver to the relevant places in the virtual patient narrative (s<sub>0</sub>). In this paper, we propose four higher level strategies that apply to the intermediate and advanced levels of integration: (1) narrative integration, (2) integration with branching nodes, (3) characteristics of data, and (4) model location. The strategies are not mutually exclusive and can be applied in parallel. The list of integration strategies has the potential to be extended to encompass additional considerations in the future as both simulation and virtual patient technologies continue to develop.</p></sec><sec><title>Narrative Integration (s1)</title><p>This strategy determines how simulation outcomes fit within the narrative of the virtual patient. This is highlighted by the situation where a single patient cannot have different values of various anatomical or physiological parameters. Two modes of integration are proposed: “what-if-nodes” and “multiple-case-packages”. The what-if-node interrupts the narration of a single activity node in the navigation graph with a discovery learning task. Students are presented with a simulation output following the main narrative of the virtual patient but are encouraged to reflect how the results would differ if the input parameters change. By manipulating input parameter values (eg, by sliders or combo boxes), different simulation outcomes are loaded from a set of pre-generated values (intermediate integration level) or generated by dynamic solution of the model (advanced integration level). Learners can alter the input parameters an unlimited number of times. Unless combined with an integration mechanism that influences the branching nodes, after leaving the what-if-node the case resumes the original narrative thread of the virtual patient. One single case could encompass several what-if-nodes. The multiple-case-package entails dynamic creation of a whole population of virtual patients from one virtual patient package. The narrative of the activity nodes are formulated as templates with empty locations to hold values from either a set of pre-generated values (intermediate integration level) or dynamically generated by an integrated solver (advanced level). The selection of values for input parameters could be influenced by student interaction, follow a predefined range, or be selected at random and would not be changed after virtual patient navigation has started. The generation of these multiple cases would be carried out by the virtual patient player. A similar method has been used by Tworek et al in the Open Labyrinth virtual patient system to produce 97 virtual patients [<xref rid="ref42" ref-type="bibr">42</xref>]. However, in this latter study the input values were taken from controlled vocabularies, statistical distributions of normal values, and manual correction of pathologies by subject experts and not dynamically created by solving computational models. Kononowicz et al proposed a variant of the multiple-case-package strategy for generating virtual patients from templates in computer-interpretable guidelines [<xref rid="ref32" ref-type="bibr">32</xref>].</p></sec><sec><title>Integration With Branching Nodes (s2)</title><p>This strategy specifies whether simulated results have dynamic influence on branching nodes. In the “no influence” mode, the what-if-nodes and multiple-case-packages retain the same static branched navigation structure in all cases. No matter how the input parameter is manipulated in a what-if-node, after the student leaves this node the virtual patient player resumes the original narration thread. Similarly, a multiple-case-package would always have the same solution path. Alternatively, the definition of branching nodes could contain simulation variables encapsulated in a formal logical expression (influence on branching mode). These expressions would be evaluated dynamically by the virtual patient player during the run-time to automatically perform branching or to alter the scoring of decisions made by the student while solving the case. In this mode, the what-if-node would change the succeeding activity sub-trees. A potential scenario might involve a student trying out different levels of drug dosage to discover immediate reactions of the virtual patient before making the final decision. For the multiple-case-package, the same route taken by the student through the activity graph could result in different scoring based on the simulated output of a randomized or pre-selected input set.</p></sec><sec><title>Characteristics of Data (s3)</title><p>This strategy describes the selection of model input parameters. The “simple data” mode allows selection of any combination of values from permitted input ranges. At the intermediate level of model integration, these ranges are discrete whereas, for the advanced level, they might be close to continuous. The designer of the virtual patient may wish to include input parameters that do not influence simulation outcomes to improve student motivation, include distractors as part of the learning design or increase the realism of the case. Examples of distractor parameters include the name of the virtual patient, description of the hospital setting, or physiological parameters that do not influence the simulation (eg, eye color) [<xref rid="ref42" ref-type="bibr">42</xref>]. The “simple data with exclusion” level enables specification of a set of excluded input values (to define a subspace of the permitted input space) for which no simulation results will be generated (eg, because they are physiologically impossible). Finally, for “interdependent data” a functional connection (eg, gender specific values, values with a specific non-negligible biological feedback loop, dependencies preserving anatomical continuity, etc) would be defined between some input variables to either guide or restrict the learner’s choice of input parameters.</p></sec><sec><title>Model Location (s4)</title><p>This strategy defines where simulated data or simulation software is located. For the intermediate level of integration operating in a “local” mode, the generated data are located within the virtual patient package. A “distant” mode would involve data dynamically loaded from a central repository of pre-generated data (because of substantial storage space requirements or for confidentiality due to sensitive, patient-specific information, for instance [<xref rid="ref43" ref-type="bibr">43</xref>]). This could be managed either by the virtual patient system itself or through a service independently accessible on the Internet. For the advanced level of integration, where model solution is integrated with virtual patient navigation, the local mode requires either direct incorporation of the solver within the virtual patient package or inclusion of the solver as part of virtual patient execution environment. The advanced level of integration operating in a distant mode requires communication with an external solver service, for example, by Web Services or some other form of Web interface.</p></sec><sec><title>Evaluation Profiles</title><p>Decisions about the feasibility and/or desirability of performing a particular level of integration for a given computational model should be made after considering the perspectives of the relevant stakeholders. This is represented visually in <xref ref-type="fig" rid="figure3">Figure 3</xref> and forms a tool for evaluation of the feasibility of integration (SSM Step 2). The columns represent the features identified in <xref ref-type="table" rid="table2">Table 2</xref> for particular stakeholder groups, identified by the notation f<sub>[{A,B,C}]</sub>, where the subscripts define the stakeholder group and the feature reference number. The integration levels are also parameterized by the integration strategies (s<sub>0</sub>,s<sub>1</sub>,s<sub>2</sub>,...,s<sub>p</sub>).</p><p>The evaluation table can be populated through the use of evaluation profiles to guide consultation with the appropriate stakeholder communities. An evaluation profile is defined by the selection of a subset of feasibility features and specification of acceptance thresholds for critical features to reflect the priorities and requirements of the integration. These evaluation profiles can be used to determine the status of cells using colors (green—threshold passed; red—failed; yellow—borderline or unknown; grey—not relevant for decision). If a threshold is not specified for a given feasibility feature, the feature may be discussed during consultation to provide qualitative feedback, but not quantitatively evaluated. Potential evaluation profiles classes might include “integration for optimal exposure of the VPH tool” (ie, project dissemination purpose), “integration for high interactivity”, “integration for high relevance to formative assessment within a particular curriculum”, and “integration for optimal use in a particular virtual patient environment” (eg, in “Open Labyrinth” virtual patient player).</p><fig id="figure3" position="float"><label>Figure 3</label><caption><p>Method for systematic evaluation of different levels of simulation outcome integration.</p></caption><graphic xlink:href="jmir_v16i1e23_fig3"></graphic></fig></sec></sec><sec><title>Demonstration of the Framework on a Real-World Example</title><sec><title>Profile</title><p>An evaluation profile of the type “integration for optimal exposure of the VPH tool” was defined for the archTk simulation (SSM Step 3). <xref ref-type="table" rid="table3">Tables 3</xref> and <xref ref-type="table" rid="table4">4</xref> present details of the profile including feasibility features, their acceptable thresholds (<xref ref-type="table" rid="table3">Table 3</xref>), and selected integration strategy modes (<xref ref-type="table" rid="table4">Table 4</xref>). The designated thresholds reflect the requirements for presenting the project outcomes in workshops and postgraduate courses using standard PCs or tablets without significant computational delays (<5 seconds).</p><table-wrap id="table3" position="float"><label>Table 3</label><caption><p>VPH ARCH evaluation profile—selected feasibility factors and thresholds.</p></caption><table frame="hsides" rules="groups" width="619" border="0" cellpadding="7" cellspacing="0"><col width="145" span="1"></col><col width="45" span="1"></col><col width="144" span="1"></col><col width="35" span="1"></col><col width="144" span="1"></col><col width="21" span="1"></col><thead><tr valign="bottom"><td rowspan="1" colspan="1">A. VPH researchers</td><td rowspan="1" colspan="1">AT<sup>1</sup></td><td rowspan="1" colspan="1">B. Medical education</td><td rowspan="1" colspan="1">AT</td><td rowspan="1" colspan="1">C. VP system developers</td><td rowspan="1" colspan="1">AT</td></tr></thead><tbody><tr valign="bottom"><td rowspan="1" colspan="1">f<sub>A1</sub>: computational time</td><td rowspan="1" colspan="1"><5s<sup>2</sup></td><td rowspan="1" colspan="1">f<sub>B1</sub>: suitable target group</td><td rowspan="1" colspan="1">Any<sup>3</sup></td><td rowspan="1" colspan="1">f<sub>C1</sub>: VP player support</td><td rowspan="1" colspan="1">–</td></tr><tr valign="top"><td rowspan="1" colspan="1">f<sub>A2</sub>: storage requirements</td><td rowspan="1" colspan="1">(<1MB)</td><td rowspan="1" colspan="1">f<sub>B2</sub>: appropriate level of difficulty</td><td rowspan="1" colspan="1">–</td><td rowspan="1" colspan="1">f<sub>C2</sub>: MVP standard support</td><td rowspan="1" colspan="1">–</td></tr><tr valign="bottom"><td rowspan="1" colspan="1">f<sub>A3</sub>: results validity</td><td rowspan="1" colspan="1"><sup>4</sup></td><td rowspan="1" colspan="1"><break></break></td><td rowspan="1" colspan="1"><break></break></td><td rowspan="1" colspan="1"><break></break></td><td rowspan="1" colspan="1"><break></break></td></tr><tr valign="top"><td rowspan="1" colspan="1">f<sub>A4</sub>: intellectual property</td><td rowspan="1" colspan="1">Yes</td><td rowspan="1" colspan="1"><break></break></td><td rowspan="1" colspan="1"><break></break></td><td rowspan="1" colspan="1"><break></break></td><td rowspan="1" colspan="1"><break></break></td></tr></tbody></table><table-wrap-foot><fn id="table3fn1"><p><sup>1</sup>AT: acceptance threshold.</p></fn><fn id="table3fn2"><p><sup>2</sup>Time measured by the student.</p></fn><fn id="table3fn3"><p><sup>3</sup>Within the context of health professionals.</p></fn><fn id="table3fn4"><p><sup>4</sup>Face validity (system performs as a subject matter expert would intuitively expect).</p></fn></table-wrap-foot></table-wrap><table-wrap id="table4" position="float"><label>Table 4</label><caption><p>VPH ARCH evaluation profile—selected integration strategies.</p></caption><table frame="hsides" rules="groups" width="619" border="0" cellpadding="7" cellspacing="0"><col width="211" span="1"></col><col width="380" span="1"></col><thead><tr valign="top"><td rowspan="1" colspan="1">Strategy</td><td rowspan="1" colspan="1"><break></break></td></tr></thead><tbody><tr valign="top"><td rowspan="1" colspan="1">s<sub>1</sub>: narration integration</td><td rowspan="1" colspan="1">(one) what-if-node</td></tr><tr valign="top"><td rowspan="1" colspan="1">s<sub>2</sub>: integration with branching nodes</td><td rowspan="1" colspan="1">No influence</td></tr><tr valign="top"><td rowspan="3" colspan="1">s<sub>3</sub>: characteristics of data</td><td rowspan="1" colspan="1">Changeable input parameters: gender, age, height, weight</td></tr><tr valign="top"><td rowspan="1" colspan="1">Output parameters: Predicted brachial artery flow (mL/min) and pressure (mmHg)</td></tr><tr valign="top"><td rowspan="1" colspan="1">Dependent variables: weight and height values specific to gender, typical distributions of vascular anatomy.</td></tr><tr valign="top"><td rowspan="1" colspan="1">s<sub>4</sub>: model location</td><td rowspan="1" colspan="1">Local (within virtual patient player)</td></tr></tbody></table></table-wrap></sec><sec><title>Computational Modeling Researchers Feasibility Features</title><p>The pulse wave propagation model applied in the VPH ARCH project as part of the archTk toolkit was initially developed by Huberts et al [<xref rid="ref28" ref-type="bibr">28</xref>]. It depends on over 70 parameters, many of which are patient-specific and difficult to measure [<xref rid="ref44" ref-type="bibr">44</xref>]. To decrease model complexity, the number of parameters may be reduced by generating typical distributions of vascular anatomy and material properties for a male and female of specific age and BMI. In the virtual patient context, the input parameters are expressed as weight, height, gender, and age. In general, the patient-specific nature of such computational models complicates the use of generic parameter sets. However, the use of this approach for the archTk model is supported by published data [<xref rid="ref45" ref-type="bibr">45</xref>]. Assessment of the influence of the choice of input parameters on the effectiveness of model integration has not been undertaken in this study and is an important topic for further refinement of the framework.</p><p>Generation of a single set of output values on a standard PC workstation (Intel Core 2; 1.66GHz; 4GB RAM) for the archTk model takes 5 minutes. At the intermediate level of integration, we have evaluated a test case with a range of input parameters that include two values for the gender parameter and two values of age, weight, and height, which depend on gender, giving a total of 16 result sets. Calculation of this level of simulation data requires 80 minutes of computation prior to virtual patient navigation. This is feasible. The display of pre-generated values takes less than one second, which is sufficiently short to ensure acceptable results at the basic and intermediate level of integration. Dynamic generation of a dataset is not efficient enough on a standard PC with the current solver implementation (pyNS 0.4.2) to support a swift (<5s) response of the graphical user interface. Storage requirements for all levels of integration are negligible as the model output consists of small numbers of ASCII files.</p><p>We were able to test the validity of the outcomes for the basic and intermediate level of integration with the experts with whom we are collaborating. The validity of the advanced level of integration cannot be assessed with the current level of development of the simulation software.</p><p>Intellectual property issues are not a barrier with this application. The software is available in an open-source form allowing unrestricted use for educational purposes.</p></sec><sec><title>Health Care Education Feasibility Features</title><p>An evaluation was performed of the integration potential of the VPH ARCH project virtual patient before initiating the study through collaboration with AMEE representatives. A challenge was identified during this consultation relating to the exploitation of such simulations for undergraduate level training. The high level of clinical specialization of the content exceeds the scope of the undergraduate curriculum; vascular access surgeon training was recommended as a more suitable target. A second target group identified was medical science or medical informatics students with an interest in the application of information and communications technology methods in medicine.</p></sec><sec><title>Virtual Patient System Developers Feasibility Features</title><p>Presentation of simulation data at the basic level of integration of the archTk is possible in any virtual patient system compatible with the MVP standard. The what-if-node is not currently supported by many virtual patient systems or by the MVP standard, and while it is possible to generate 18 activity nodes to represent each what-if node, this process is cumbersome for the end user and could result in practical limitations for large input parameter ranges. We do not regard the current status of the f<sub>C1</sub>feature (virtual patient player support) as critical since we intend to extend existing open-source virtual patient systems to accommodate such features on which the intermediate level of integration relies.</p><p>The feasibility analysis of the VPH ARCH simulation integration for “integration for optimal exposure of the VPH tool” profile (<xref ref-type="fig" rid="figure4">Figure 4</xref>) concluded that integration is possible for the basic and intermediate levels.</p><fig id="figure4" position="float"><label>Figure 4</label><caption><p>Evaluation profile for the VPH ARCH integration.</p></caption><graphic xlink:href="jmir_v16i1e23_fig4"></graphic></fig></sec></sec><sec><title>Implementation of Two Levels of Integration</title><p>Following the positive evaluation of the integration potential of the VPH ARCH project at the basic and intermediate levels, two variants of the same virtual patient were implemented reflecting these levels. The first case (Study Group 1, Basic Integration Level) presented static simulation results through manual pasting into the case (s<sub>0</sub> integration strategy), whereas the second case (Study Group 2, Intermediate Integration Level) provided the user with the option to manipulate different parameters of the simulation. The narrative integration (s<sub>1</sub>) was implemented by adding a what-if-node at the end of the case (<xref ref-type="fig" rid="figure5">Figure 5</xref>).</p><p>Users of the second group could not circumvent the opportunity to use this feature as it was placed in the central navigation path. There was no influence of the parameter manipulation on the virtual patient branching (s<sub>2</sub>). The input parameters of gender, age, height, and weight are interdependent and represent a set of input parameters with typical distributions of vascular anatomy for a male and female of specific age and Body Mass Index (s<sub>3</sub>). The simulation data (16 result sets) were generated prior to the experiment on a standard PC workstation (Intel Core 2; 1.66GHz; 4GB RAM), at run-time loaded from a pre-generated look-up table located in the virtual patient player (deployed at a WWW server) and available locally for manipulation in the Web browser through a JavaScript method (s<sub>4</sub>).</p><fig id="figure5" position="float"><label>Figure 5</label><caption><p>Second level computational model integration ("what-if-node") for the VPH ARCH project virtual patient.</p></caption><graphic xlink:href="jmir_v16i1e23_fig5"></graphic></fig></sec><sec><title>Perception of the Framework Output by Health Care Workers</title><p><xref ref-type="table" rid="table5">Table 5</xref> summarizes the responses of the participants at the EVC conference event. The virtual patient was interesting or very interesting for 84% of participants (32/38), 11% (4/38) were neutral, and 5% (2/38) found the virtual patient “not that interesting” or “not at all interesting”. The average evaluation score (Likert scale, 5=very interesting, 1=not at all interesting) was better for the intermediate integration level (4.22) than the basic level (4.10) in this sample, but the difference was not statistically significant (<italic>P</italic>=.84). None of the evaluators disagreed or strongly disagreed with the presented content, 76% (29/38) agreed or strongly agreed with it, and 24% (9/38) neither agreed nor disagreed. The average evaluation score (Likert scale, 5=strongly agree, 1=strongly disagree) was better in this case for the basic integration level (4.05) than the intermediate level (3.83). Again this difference was not statistically significant (<italic>P</italic>=.32).</p><p>In the questions directed to Study Group 2 only, “The possibility of manipulating simulation parameters (gender, age, weight, height) for the hypothetical patient was …”, 28% (5/18) responded with “very interesting”, 39% (7/18) “interesting”, 33% (6/18) “neutral”, and no one answered “not that”, “not at all interesting”. It should be stressed that no participants responded “I have not noticed this possibility” to this question, which was provided as an alternative option. The participants in Study Group 2 were also largely satisfied with the results of manipulating the simulation parameters. To the question “Did the results of manipulating the simulation parameters meet your expectations?”, 11% (2/18) answered “strongly agree”, 72% (13/18) “agree”, and 17% (3/18) “neither agree nor disagree”. No one disagreed with this statement or had not used this function.</p><table-wrap id="table5" position="float"><label>Table 5</label><caption><p>Summary of answers in the evaluation questionnaire.</p></caption><table frame="hsides" rules="groups" width="619" border="0" cellpadding="7" cellspacing="0"><col width="239" span="1"></col><col width="177" span="1"></col><col width="52" span="1"></col><col width="54" span="1"></col><col width="27" span="1"></col><thead><tr valign="top"><td rowspan="1" colspan="1">Question</td><td rowspan="1" colspan="1">Likert scale</td><td rowspan="1" colspan="1">Group 1 n=20</td><td rowspan="1" colspan="1">Group 2 n=18</td><td rowspan="1" colspan="1"><italic>P</italic></td></tr></thead><tbody><tr valign="top"><td rowspan="1" colspan="1">Was the presented virtual patient case interesting for you?</td><td rowspan="1" colspan="1">5=very interesting; 1=not at all interesting</td><td rowspan="1" colspan="1">4.10</td><td rowspan="1" colspan="1">4.22</td><td rowspan="1" colspan="1">.84</td></tr><tr valign="top"><td rowspan="1" colspan="1">Do you agree with the presented content of the virtual patient case?</td><td rowspan="1" colspan="1">5=strongly agree; 1=strongly disagree</td><td rowspan="1" colspan="1">4.05</td><td rowspan="1" colspan="1">3.83</td><td rowspan="1" colspan="1">.32</td></tr></tbody></table></table-wrap></sec></sec><sec sec-type="discussion"><title>Discussion</title><sec><title>Summary</title><p>Integration of simulation is gaining importance in health care education informatics. After a period where isolated systems were developed, the advantages of combining systems into “continua of simulation” have become apparent [<xref rid="ref15" ref-type="bibr">15</xref>]. This connection not only augments the value of learning activities by facilitating wider opportunities to synthesize knowledge and skills, but it also fosters reuse of existing models, thereby increasing the return on investment in their development.</p><p>We may regard the proposed framework as a specialization of a section of Ellaway’s et al “Practica continua” framework [<xref rid="ref15" ref-type="bibr">15</xref>]. Some integration strategies we have proposed are direct responses to the desiderata of this framework. For instance, our narrative integration strategy (s<sub>1</sub>) has a clear correspondence to the “narrative integration” dimension and requirements (eg, timeline and causality continua) in Ellaway et al. The added value is that we have exemplified their implementation in the context of virtual patients and computational models of physiology by concrete implementation proposals in the form of the “what-if-node” and “multi-case-package” strategies. On the other hand, our framework is not just a specialization of the “Practica continua” framework, since it discusses the integration of computational models that are products of research projects initially outside the educational sector. This consideration is given in our framework by inclusion of tools for systematic evaluation of the feasibility of integration. The framework described here is currently restricted to the integration activity phases, that is, the authoring process. This leaves the run-time and analysis phases for further iterations of framework development. The HSVO project, related to the “Practica continua” framework, demonstrated how the run-time phase may be implemented by a common execution interface and middleware layer [<xref rid="ref46" ref-type="bibr">46</xref>]. These outcomes will be useful in the further development of our framework.</p><p>It is important to stress that current virtual patient systems and the MVP standard are not yet, in general, ready for the higher levels of integration described in this paper. However, the effort involved in extending virtual patient systems to facilitate the intermediate level of model integration does not seem to be obstructive. A graphical user interface component to handle the what-if-node manipulation of model parameters and display the simulation results is of primary importance along with a standard mechanism to manage pre-generated data in virtual patient packages. The advanced level of integration poses a more significant challenge. Many models available today have been implemented using scientific tools such as MatLab (Mathworks) or specific numerical libraries. This limits their portability in virtual patient packages. A viable strategy might be to host the solver on a remote server, but this conflicts with the self-containment rule of content of a virtual patient package, raises security concerns, and requires the maintenance of an additional service. These considerations will fuel further research as suggested by this paper (SSM Step 4).</p><p>The proposal of this framework was helpful for the authors in developing, discussing, and reporting the integration of the archTk simulation within the virtual patient. It is challenging to demonstrate, in the short term, the long-term benefits of the framework for the users of virtual patients. The evaluation study carried out at the EVC conference provided an unambiguously positive response to the approach of integrating advanced computational models of physiological processes within virtual patients. Based on the obtained feedback sample, we hypothesize an altered perception of integration depending on changes in the framework’s variables. The queried group of experts found higher integration more interesting while, at the same time, being more concerned with the validity of the results. It has to be stressed, however, that this conclusion cannot be generalized as yet because the difference is not statistically significant and the influences of other factors (eg, age or experience) could not be excluded.</p></sec><sec><title>Limitations</title><p>The application of the integration framework has been demonstrated by a case study involving a single computational model. Improvements for the next SSM learning cycle should aim to extend the proposed lists of feasibility features, integration strategies and scope of exemplar projects. This cycle may be repeated several times depending on the research outcomes, prioritizing different simulation aspects.</p><p>Some proposed integration modes are currently theoretical constructs that require testing on concrete examples. In particular, this applies to the advanced level of integration that was not attempted in this case because the computational load was unsuitable for target application. Future SSM iterations should focus on concrete data representations and software solutions enabling further development of the integration strategies and evaluation process.</p><p>The educational impact of the proposed integration levels is still uncertain and needs to be tested in rigorously conducted comparative studies. The integration of the archTk simulation into a virtual patient was tested with clinical experts and health care education specialists, but not on a wider scale with different virtual patient systems. The completeness and priority of the proposed framework elements could be addressed by a Delphi study to formally collect feedback from computational modeling researchers and virtual patient system developers.</p></sec><sec><title>Conclusions</title><p>The paper has outlined a conceptual framework for the integration of computational models into virtual patients. This includes consideration of feasibility features, levels of integration, integration strategies with various modes, and evaluation profiles. The opportunities and challenges of model exploitation have been discussed in the context of a virtual patient developed from the VPH ARCH project, incorporating archTk simulation results at different integration levels. The empirical evaluation of two variants of the virtual patient provided positive feedback on the value of this type of integration. The responses suggest further investigation of increased user satisfaction, but decreased trust, at higher levels of simulation integration. The long-term research aim is to isolate the most crucial factors in the integration framework and their influence on the integration outcome.</p></sec></sec></body><back><ack><p>The authors would like to thank the ARCH consortium. The authors wish to acknowledge their gratitude to Dr Inga Hege for her comments on the paper and to Anneliese Lilienthal for graphical support. This work is partially funded by the European Community’s 7th Framework Programme (VPH NoE, Grant Agreement No 223920).</p></ack><fn-group><fn fn-type="conflict"><p>Conflicts of Interest: None declared.</p></fn></fn-group><glossary><title>Abbreviations</title><def-list><def-item><term id="abb1">AVF</term><def><p>arteriovenous fistula</p></def></def-item><def-item><term id="abb2">ESRD</term><def><p>end-stage renal disease</p></def></def-item><def-item><term id="abb3">EVC</term><def><p>European Vascular Course</p></def></def-item><def-item><term id="abb4">MVP</term><def><p>MedBiquitous Virtual Patient Standard</p></def></def-item><def-item><term id="abb5">NoE</term><def><p>Network of Excellence</p></def></def-item><def-item><term id="abb6">SSM</term><def><p>Soft System Methodology</p></def></def-item><def-item><term id="abb7">VP</term><def><p>virtual patient</p></def></def-item><def-item><term id="abb8">VPH</term><def><p>Virtual Physiological Human</p></def></def-item></def-list></glossary><ref-list><ref id="ref1"><label>1</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Ellaway</surname><given-names>R</given-names></name><name><surname>Masters</surname><given-names>K</given-names></name></person-group><article-title>AMEE Guide 32: e-Learning in medical education Part 1: Learning, teaching and assessment</article-title><source>Med Teach</source><year>2008</year><month>6</month><volume>30</volume><issue>5</issue><fpage>455</fpage><lpage>73</lpage><pub-id pub-id-type="doi">10.1080/01421590802108331</pub-id><pub-id pub-id-type="medline">18576185</pub-id><pmc-comment>793697811 </pmc-comment>
<pub-id pub-id-type="pmid">18576185</pub-id></element-citation></ref><ref id="ref2"><label>2</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Ellaway</surname><given-names>RH</given-names></name><name><surname>Poulton</surname><given-names>T</given-names></name><name><surname>Smothers</surname><given-names>V</given-names></name><name><surname>Greene</surname><given-names>P</given-names></name></person-group><article-title>Virtual patients come of age</article-title><source>Med Teach</source><year>2009</year><month>8</month><volume>31</volume><issue>8</issue><fpage>683</fpage><lpage>4</lpage><pub-id pub-id-type="medline">19811203</pub-id><pmc-comment>10.1080/01421590903124765 </pmc-comment>
<pub-id pub-id-type="pmid">19811203</pub-id></element-citation></ref><ref id="ref3"><label>3</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Poulton</surname><given-names>T</given-names></name><name><surname>Balasubramaniam</surname><given-names>C</given-names></name></person-group><article-title>Virtual patients: a year of change</article-title><source>Med Teach</source><year>2011</year><volume>33</volume><issue>11</issue><fpage>933</fpage><lpage>7</lpage><pub-id pub-id-type="doi">10.3109/0142159X.2011.613501</pub-id><pub-id pub-id-type="medline">22022903</pub-id><pub-id pub-id-type="pmid">22022903</pub-id></element-citation></ref><ref id="ref4"><label>4</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Maloney</surname><given-names>S</given-names></name><name><surname>Haas</surname><given-names>R</given-names></name><name><surname>Keating</surname><given-names>JL</given-names></name><name><surname>Molloy</surname><given-names>E</given-names></name><name><surname>Jolly</surname><given-names>B</given-names></name><name><surname>Sims</surname><given-names>J</given-names></name><name><surname>Morgan</surname><given-names>P</given-names></name><name><surname>Haines</surname><given-names>T</given-names></name></person-group><article-title>Breakeven, cost benefit, cost effectiveness, and willingness to pay for web-based versus face-to-face education delivery for health professionals</article-title><source>J Med Internet Res</source><year>2012</year><volume>14</volume><issue>2</issue><fpage>e47</fpage><comment><ext-link ext-link-type="uri" xlink:href="http://www.jmir.org/2012/2/e47/"></ext-link></comment><pub-id pub-id-type="doi">10.2196/jmir.2040</pub-id><pub-id pub-id-type="medline">22469659</pub-id><pmc-comment>v14i2e47 </pmc-comment>
<pmc-comment>PMC3376523 </pmc-comment>
<pub-id pub-id-type="pmid">22469659</pub-id></element-citation></ref><ref id="ref5"><label>5</label><element-citation publication-type="book"><person-group person-group-type="author"><name><surname>Kononowicz</surname><given-names>AA</given-names></name><name><surname>Hege</surname><given-names>I</given-names></name></person-group><person-group person-group-type="editor"><name><surname>Soomro</surname><given-names>S</given-names></name></person-group><article-title>Virtual patients as a practical realisation of the e-learning idea in medicine</article-title><source>E-learning, experience and future</source><year>2010</year><month>4</month><day>01</day><publisher-loc>Vukovar, Croatia</publisher-loc><publisher-name>In-Teh</publisher-name><fpage>345</fpage><lpage>70</lpage></element-citation></ref><ref id="ref6"><label>6</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Ellaway</surname><given-names>R</given-names></name><name><surname>Poulton</surname><given-names>T</given-names></name><name><surname>Fors</surname><given-names>U</given-names></name><name><surname>McGee</surname><given-names>JB</given-names></name><name><surname>Albright</surname><given-names>S</given-names></name></person-group><article-title>Building a virtual patient commons</article-title><source>Med Teach</source><year>2008</year><volume>30</volume><issue>2</issue><fpage>170</fpage><lpage>4</lpage><pub-id pub-id-type="doi">10.1080/01421590701874074</pub-id><pub-id pub-id-type="medline">18464142</pub-id><pmc-comment>792859283 </pmc-comment>
<pub-id pub-id-type="pmid">18464142</pub-id></element-citation></ref><ref id="ref7"><label>7</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Cook</surname><given-names>DA</given-names></name><name><surname>Triola</surname><given-names>MM</given-names></name></person-group><article-title>Virtual patients: a critical literature review and proposed next steps</article-title><source>Med Educ</source><year>2009</year><month>4</month><volume>43</volume><issue>4</issue><fpage>303</fpage><lpage>11</lpage><pub-id pub-id-type="doi">10.1111/j.1365-2923.2008.03286.x</pub-id><pub-id pub-id-type="medline">19335571</pub-id><pmc-comment>MED3286 </pmc-comment>
<pub-id pub-id-type="pmid">19335571</pub-id></element-citation></ref><ref id="ref8"><label>8</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Dev</surname><given-names>P</given-names></name><name><surname>Heinrichs</surname><given-names>WL</given-names></name><name><surname>Youngblood</surname><given-names>P</given-names></name></person-group><article-title>CliniSpace: a multiperson 3D online immersive training environment accessible through a browser</article-title><source>Stud Health Technol Inform</source><year>2011</year><volume>163</volume><fpage>173</fpage><lpage>9</lpage><pub-id pub-id-type="medline">21335784</pub-id><pub-id pub-id-type="pmid">21335784</pub-id></element-citation></ref><ref id="ref9"><label>9</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Huwendiek</surname><given-names>S</given-names></name><name><surname>de Leng</surname><given-names>BA</given-names></name><name><surname>Zary</surname><given-names>N</given-names></name><name><surname>Fischer</surname><given-names>MR</given-names></name><name><surname>Ruiz</surname><given-names>JG</given-names></name><name><surname>Ellaway</surname><given-names>R</given-names></name></person-group><article-title>Towards a typology of virtual patients</article-title><source>Med Teach</source><year>2009</year><month>8</month><volume>31</volume><issue>8</issue><fpage>743</fpage><lpage>8</lpage><pub-id pub-id-type="medline">19811212</pub-id><pmc-comment>10.1080/01421590903124708 </pmc-comment>
<pub-id pub-id-type="pmid">19811212</pub-id></element-citation></ref><ref id="ref10"><label>10</label><element-citation publication-type="book"><person-group person-group-type="author"><name><surname>Smothers</surname><given-names>V</given-names></name><name><surname>Azan</surname><given-names>B</given-names></name><name><surname>Ellaway</surname><given-names>R</given-names></name></person-group><source>ANSI/MEDBIQ VP.10.1 – 2010</source><year>2010</year><month>4</month><day>06</day><date-in-citation>2013-02-25</date-in-citation><publisher-loc>Baltimore, MD</publisher-loc><publisher-name>MedBiquitous Consortium</publisher-name><comment>MedBiquitous Virtual Patient Specifications and Description Documen<ext-link ext-link-type="uri" xlink:href="http://www.medbiq.org/working_groups/virtual_patient/VirtualPatientDataSpecification.pdf">http://www.medbiq.org/working_groups/virtual_patient/VirtualPatientDataSpecification.pdf</ext-link></comment><ext-link ext-link-type="webcite" xlink:href="6EgCCgsho"></ext-link></element-citation></ref><ref id="ref11"><label>11</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Zary</surname><given-names>N</given-names></name><name><surname>Hege</surname><given-names>I</given-names></name><name><surname>Heid</surname><given-names>J</given-names></name><name><surname>Woodham</surname><given-names>L</given-names></name><name><surname>Donkers</surname><given-names>J</given-names></name><name><surname>Kononowicz</surname><given-names>AA</given-names></name></person-group><article-title>Enabling interoperability, accessibility and reusability of virtual patients across Europe - design and implementation</article-title><source>Stud Health Technol Inform</source><year>2009</year><volume>150</volume><fpage>826</fpage><lpage>30</lpage><pub-id pub-id-type="medline">19745428</pub-id><pub-id pub-id-type="pmid">19745428</pub-id></element-citation></ref><ref id="ref12"><label>12</label><element-citation publication-type="book"><person-group person-group-type="author"><name><surname>Garrido</surname><given-names>J</given-names></name></person-group><source>Introduction to Elementary Computational Modeling: Essential Concepts, Principles, and Problem Solving</source><year>2011</year><publisher-loc>Boca Raton, FL</publisher-loc><publisher-name>CRC Press</publisher-name></element-citation></ref><ref id="ref13"><label>13</label><element-citation publication-type="book"><person-group person-group-type="author"><name><surname>van Meurs</surname><given-names>W</given-names></name></person-group><source>Modeling and Simulation in Biomedical Engineering: Applications in Cardiorespiratory Physiology</source><year>2011</year><publisher-loc>New York</publisher-loc><publisher-name>McGraw-Hill Professional</publisher-name></element-citation></ref><ref id="ref14"><label>14</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Schwarz</surname><given-names>D</given-names></name><name><surname>Štourač</surname><given-names>P</given-names></name><name><surname>Komenda</surname><given-names>M</given-names></name><name><surname>Harazim</surname><given-names>H</given-names></name><name><surname>Kosinová</surname><given-names>M</given-names></name><name><surname>Gregor</surname><given-names>J</given-names></name><name><surname>Hůlek</surname><given-names>R</given-names></name><name><surname>Smékalová</surname><given-names>O</given-names></name><name><surname>Křikava</surname><given-names>I</given-names></name><name><surname>Štoudek</surname><given-names>R</given-names></name><name><surname>Dušek</surname><given-names>L</given-names></name></person-group><article-title>Interactive algorithms for teaching and learning acute medicine in the network of medical faculties MEFANET</article-title><source>J Med Internet Res</source><year>2013</year><volume>15</volume><issue>7</issue><fpage>e135</fpage><comment><ext-link ext-link-type="uri" xlink:href="http://www.jmir.org/2013/7/e135/"></ext-link></comment><pub-id pub-id-type="doi">10.2196/jmir.2590</pub-id><pub-id pub-id-type="medline">23835586</pub-id><pmc-comment>v15i7e135 </pmc-comment>
<pmc-comment>PMC3714007 </pmc-comment>
<pub-id pub-id-type="pmid">23835586</pub-id></element-citation></ref><ref id="ref15"><label>15</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Ellaway</surname><given-names>RH</given-names></name><name><surname>Kneebone</surname><given-names>R</given-names></name><name><surname>Lachapelle</surname><given-names>K</given-names></name><name><surname>Topps</surname><given-names>D</given-names></name></person-group><article-title>Practica continua: Connecting and combining simulation modalities for integrated teaching, learning and assessment</article-title><source>Med Teach</source><year>2009</year><month>8</month><volume>31</volume><issue>8</issue><fpage>725</fpage><lpage>31</lpage><pub-id pub-id-type="medline">19811209</pub-id><pmc-comment>10.1080/01421590903124716 </pmc-comment>
<pub-id pub-id-type="pmid">19811209</pub-id></element-citation></ref><ref id="ref16"><label>16</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>de Bono</surname><given-names>B</given-names></name><name><surname>Hoehndorf</surname><given-names>R</given-names></name><name><surname>Wimalaratne</surname><given-names>S</given-names></name><name><surname>Gkoutos</surname><given-names>G</given-names></name><name><surname>Grenon</surname><given-names>P</given-names></name></person-group><article-title>The RICORDO approach to semantic interoperability for biomedical data and models: strategy, standards and solutions</article-title><source>BMC Res Notes</source><year>2011</year><volume>4</volume><fpage>313</fpage><comment><ext-link ext-link-type="uri" xlink:href="http://www.biomedcentral.com/1756-0500/4/313"></ext-link></comment><pub-id pub-id-type="doi">10.1186/1756-0500-4-313</pub-id><pub-id pub-id-type="medline">21878109</pub-id><pmc-comment>1756-0500-4-313 </pmc-comment>
<pmc-comment>PMC3192696 </pmc-comment>
<pub-id pub-id-type="pmid">21878109</pub-id></element-citation></ref><ref id="ref17"><label>17</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Halic</surname><given-names>T</given-names></name><name><surname>Venkata</surname><given-names>SA</given-names></name><name><surname>Sankaranarayanan</surname><given-names>G</given-names></name><name><surname>Lu</surname><given-names>Z</given-names></name><name><surname>Ahn</surname><given-names>W</given-names></name><name><surname>De</surname><given-names>S</given-names></name></person-group><article-title>A software framework for multimodal interactive simulations (SoFMIS)</article-title><source>Stud Health Technol Inform</source><year>2011</year><volume>163</volume><fpage>213</fpage><lpage>7</lpage><pub-id pub-id-type="medline">21335791</pub-id><pub-id pub-id-type="pmid">21335791</pub-id></element-citation></ref><ref id="ref18"><label>18</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Coveney</surname><given-names>PV</given-names></name><name><surname>Diaz</surname><given-names>V</given-names></name><name><surname>Hunter</surname><given-names>P</given-names></name><name><surname>Kohl</surname><given-names>P</given-names></name><name><surname>Viceconti</surname><given-names>M</given-names></name></person-group><article-title>The Virtual Physiological Human</article-title><source>Interface Focus</source><year>2011</year><month>3</month><day>31</day><volume>1</volume><issue>3</issue><fpage>281</fpage><lpage>285</lpage><pub-id pub-id-type="doi">10.1098/rsfs.2011.0020</pub-id></element-citation></ref><ref id="ref19"><label>19</label><element-citation publication-type="confproc"><person-group person-group-type="author"><name><surname>Bamidis</surname><given-names>PD</given-names></name><name><surname>Kaldoudi</surname><given-names>E</given-names></name><name><surname>Pattichis</surname><given-names>C</given-names></name></person-group><article-title>mEducator: a best practice network for repurposing and sharing medical educational multi-type content</article-title><source>Leveraging Knowledge for Innovation in Collaborative Networks: 10th IFIP WG 5.5 Working Conference on Virtual Enterprises</source><year>2009</year><conf-name>PRO-VE 2009</conf-name><conf-date>Oct 7-9, 2009</conf-date><conf-loc>Thessaloniki, Greece</conf-loc><publisher-loc>Berlin Heidelberg</publisher-loc><publisher-name>Springer</publisher-name><fpage>769</fpage><lpage>76</lpage></element-citation></ref><ref id="ref20"><label>20</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Halic</surname><given-names>T</given-names></name><name><surname>Ahn</surname><given-names>W</given-names></name><name><surname>De</surname><given-names>S</given-names></name></person-group><article-title>A framework for web browser-based medical simulation using WebGL</article-title><source>Stud Health Technol Inform</source><year>2012</year><volume>173</volume><fpage>149</fpage><lpage>55</lpage><pub-id pub-id-type="medline">22356977</pub-id><pub-id pub-id-type="pmid">22356977</pub-id></element-citation></ref><ref id="ref21"><label>21</label><element-citation publication-type="book"><person-group person-group-type="author"><name><surname>Checkland</surname><given-names>P</given-names></name><name><surname>Poulter</surname><given-names>J</given-names></name></person-group><person-group person-group-type="editor"><name><surname>Reynolds</surname><given-names>M</given-names></name><name><surname>Holwell</surname><given-names>S</given-names></name></person-group><article-title>Soft Systems Methodology</article-title><source>Systems Approaches to Managing Change: A Practical Guide</source><year>2010</year><publisher-loc>London</publisher-loc><publisher-name>Springer</publisher-name><fpage>191</fpage><lpage>242</lpage></element-citation></ref><ref id="ref22"><label>22</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Ruotsalainen</surname><given-names>PS</given-names></name><name><surname>Blobel</surname><given-names>BG</given-names></name><name><surname>Seppälä</surname><given-names>AV</given-names></name><name><surname>Sorvari</surname><given-names>HO</given-names></name><name><surname>Nykänen</surname><given-names>PA</given-names></name></person-group><article-title>A conceptual framework and principles for trusted pervasive health</article-title><source>J Med Internet Res</source><year>2012</year><volume>14</volume><issue>2</issue><fpage>e52</fpage><comment><ext-link ext-link-type="uri" xlink:href="http://www.jmir.org/2012/2/e52/"></ext-link></comment><pub-id pub-id-type="doi">10.2196/jmir.1972</pub-id><pub-id pub-id-type="medline">22481297</pub-id><pmc-comment>v14i2e52 </pmc-comment>
<pmc-comment>PMC3376512 </pmc-comment>
<pub-id pub-id-type="pmid">22481297</pub-id></element-citation></ref><ref id="ref23"><label>23</label><element-citation publication-type="book"><person-group person-group-type="author"><name><surname>Viceconti</surname><given-names>M</given-names></name><name><surname>Clapworthy</surname><given-names>G</given-names></name></person-group><source>VPH-FET Research Roadmap - Advanced Technologies for the Future of the Virtual Physiological Human</source><year>2011</year><month>9</month><date-in-citation>2013-02-24</date-in-citation><publisher-name>VPH-FET consortium</publisher-name><comment><ext-link ext-link-type="uri" xlink:href="http://www.vph-institute.org/upload/vph-fet-final-roadmap-1_519244713c477.pdf">http://www.vph-institute.org/upload/vph-fet-final-roadmap-1_519244713c477.pdf</ext-link></comment><ext-link ext-link-type="webcite" xlink:href="6Eg6MCAl5"></ext-link></element-citation></ref><ref id="ref24"><label>24</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Hunter</surname><given-names>P</given-names></name><name><surname>Coveney</surname><given-names>PV</given-names></name><name><surname>de Bono</surname><given-names>B</given-names></name><name><surname>Diaz</surname><given-names>V</given-names></name><name><surname>Fenner</surname><given-names>J</given-names></name><name><surname>Frangi</surname><given-names>AF</given-names></name><name><surname>Harris</surname><given-names>P</given-names></name><name><surname>Hose</surname><given-names>R</given-names></name><name><surname>Kohl</surname><given-names>P</given-names></name><name><surname>Lawford</surname><given-names>P</given-names></name><name><surname>McCormack</surname><given-names>K</given-names></name><name><surname>Mendes</surname><given-names>M</given-names></name><name><surname>Omholt</surname><given-names>S</given-names></name><name><surname>Quarteroni</surname><given-names>A</given-names></name><name><surname>Skår</surname><given-names>J</given-names></name><name><surname>Tegner</surname><given-names>J</given-names></name><name><surname>Randall Thomas</surname><given-names>S</given-names></name><name><surname>Tollis</surname><given-names>I</given-names></name><name><surname>Tsamardinos</surname><given-names>I</given-names></name><name><surname>van Beek</surname><given-names>JH</given-names></name><name><surname>Viceconti</surname><given-names>M</given-names></name></person-group><article-title>A vision and strategy for the virtual physiological human in 2010 and beyond</article-title><source>Philos Trans A Math Phys Eng Sci</source><year>2010</year><month>6</month><day>13</day><volume>368</volume><issue>1920</issue><fpage>2595</fpage><lpage>614</lpage><comment><ext-link ext-link-type="uri" xlink:href="http://europepmc.org/abstract/MED/20439264"></ext-link></comment><pub-id pub-id-type="doi">10.1098/rsta.2010.0048</pub-id><pub-id pub-id-type="medline">20439264</pub-id><pmc-comment>368/1920/2595 </pmc-comment>
<pmc-comment>PMC2944384 </pmc-comment>
<pub-id pub-id-type="pmid">20439264</pub-id></element-citation></ref><ref id="ref25"><label>25</label><element-citation publication-type="webpage"><source>Virtual Physiological – Human Network of Excellence</source><date-in-citation>2013-02-24</date-in-citation><comment><ext-link ext-link-type="uri" xlink:href="http://www.vph-noe.eu/">http://www.vph-noe.eu/</ext-link></comment><ext-link ext-link-type="webcite" xlink:href="6Eg6ifKEp"></ext-link></element-citation></ref><ref id="ref26"><label>26</label><element-citation publication-type="webpage"><source>VPH ARCH Project</source><year>2010</year><date-in-citation>2013-02-24</date-in-citation><comment><ext-link ext-link-type="uri" xlink:href="http://www.vph-arch.eu/">http://www.vph-arch.eu/</ext-link></comment><ext-link ext-link-type="webcite" xlink:href="6EgDz6xLN"></ext-link></element-citation></ref><ref id="ref27"><label>27</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Bode</surname><given-names>AS</given-names></name><name><surname>Huberts</surname><given-names>W</given-names></name><name><surname>Bosboom</surname><given-names>EM</given-names></name><name><surname>Kroon</surname><given-names>W</given-names></name><name><surname>van der Linden</surname><given-names>WP</given-names></name><name><surname>Planken</surname><given-names>RN</given-names></name><name><surname>van de Vosse</surname><given-names>FN</given-names></name><name><surname>Tordoir</surname><given-names>JH</given-names></name></person-group><article-title>Patient-specific computational modeling of upper extremity arteriovenous fistula creation: its feasibility to support clinical decision-making</article-title><source>PLoS One</source><year>2012</year><volume>7</volume><issue>4</issue><fpage>e34491</fpage><comment><ext-link ext-link-type="uri" xlink:href="http://dx.plos.org/10.1371/journal.pone.0034491"></ext-link></comment><pub-id pub-id-type="doi">10.1371/journal.pone.0034491</pub-id><pub-id pub-id-type="medline">22496816</pub-id><pmc-comment>PONE-D-11-20822 </pmc-comment>
<pmc-comment>PMC3319586 </pmc-comment>
<pub-id pub-id-type="pmid">22496816</pub-id></element-citation></ref><ref id="ref28"><label>28</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Huberts</surname><given-names>W</given-names></name><name><surname>Bode</surname><given-names>AS</given-names></name><name><surname>Kroon</surname><given-names>W</given-names></name><name><surname>Planken</surname><given-names>RN</given-names></name><name><surname>Tordoir</surname><given-names>JH</given-names></name><name><surname>van de Vosse</surname><given-names>FN</given-names></name><name><surname>Bosboom</surname><given-names>EM</given-names></name></person-group><article-title>A pulse wave propagation model to support decision-making in vascular access planning in the clinic</article-title><source>Med Eng Phys</source><year>2012</year><month>3</month><volume>34</volume><issue>2</issue><fpage>233</fpage><lpage>48</lpage><pub-id pub-id-type="doi">10.1016/j.medengphy.2011.07.015</pub-id><pub-id pub-id-type="medline">21840239</pub-id><pmc-comment>S1350-4533(11)00180-9 </pmc-comment>
<pub-id pub-id-type="pmid">21840239</pub-id></element-citation></ref><ref id="ref29"><label>29</label><element-citation publication-type="webpage"><source>ArchTk toolkit</source><year>2012</year><date-in-citation>2013-02-24</date-in-citation><comment>ARCH Consortium<ext-link ext-link-type="uri" xlink:href="http://archtk.github.com/">http://archtk.github.com/</ext-link></comment><ext-link ext-link-type="webcite" xlink:href="6EgEXzo70"></ext-link></element-citation></ref><ref id="ref30"><label>30</label><element-citation publication-type="webpage"><source>AVF.SIM - Patient-specific computer modelling for computer-based planning of vascular access surgery in haemodialysis patients</source><year>2013</year><date-in-citation>2013-09-26</date-in-citation><comment><ext-link ext-link-type="uri" xlink:href="http://avfsim.com/">http://avfsim.com/</ext-link></comment><ext-link ext-link-type="webcite" xlink:href="6Jv7GD0Kb"></ext-link></element-citation></ref><ref id="ref31"><label>31</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Kononowicz</surname><given-names>AA</given-names></name><name><surname>Holler</surname><given-names>T</given-names></name></person-group><article-title>Bit pathways–a tool for teaching and learning clinical pathways</article-title><source>Bio-Algorithms and Med-Systems</source><year>2008</year><volume>4</volume><issue>8</issue><fpage>33</fpage><lpage>40</lpage></element-citation></ref><ref id="ref32"><label>32</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Kononowicz</surname><given-names>AA</given-names></name><name><surname>Hege</surname><given-names>I</given-names></name><name><surname>Krawczyk</surname><given-names>P</given-names></name><name><surname>Zary</surname><given-names>N</given-names></name></person-group><article-title>New approaches to linking clinical guidelines to virtual patients</article-title><source>Stud Health Technol Inform</source><year>2012</year><volume>180</volume><fpage>958</fpage><lpage>62</lpage><pub-id pub-id-type="medline">22874335</pub-id><pub-id pub-id-type="pmid">22874335</pub-id></element-citation></ref><ref id="ref33"><label>33</label><element-citation publication-type="book"><person-group person-group-type="author"><name><surname>Kolb</surname><given-names>DA</given-names></name></person-group><source>Experiential learning: experience as the source of learning and development</source><year>1984</year><publisher-loc>Englewood Cliffs, NJ</publisher-loc><publisher-name>Prentice-Hall</publisher-name></element-citation></ref><ref id="ref34"><label>34</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Huwendiek</surname><given-names>S</given-names></name><name><surname>Reichert</surname><given-names>F</given-names></name><name><surname>Bosse</surname><given-names>HM</given-names></name><name><surname>de Leng</surname><given-names>BA</given-names></name><name><surname>van der Vleuten</surname><given-names>CP</given-names></name><name><surname>Haag</surname><given-names>M</given-names></name><name><surname>Hoffmann</surname><given-names>GF</given-names></name><name><surname>Tönshoff</surname><given-names>B</given-names></name></person-group><article-title>Design principles for virtual patients: a focus group study among students</article-title><source>Med Educ</source><year>2009</year><month>6</month><volume>43</volume><issue>6</issue><fpage>580</fpage><lpage>8</lpage><pub-id pub-id-type="doi">10.1111/j.1365-2923.2009.03369.x</pub-id><pub-id pub-id-type="medline">19493183</pub-id><pmc-comment>MED3369 </pmc-comment>
<pub-id pub-id-type="pmid">19493183</pub-id></element-citation></ref><ref id="ref35"><label>35</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Kalet</surname><given-names>AL</given-names></name><name><surname>Song</surname><given-names>HS</given-names></name><name><surname>Sarpel</surname><given-names>U</given-names></name><name><surname>Schwartz</surname><given-names>R</given-names></name><name><surname>Brenner</surname><given-names>J</given-names></name><name><surname>Ark</surname><given-names>TK</given-names></name><name><surname>Plass</surname><given-names>J</given-names></name></person-group><article-title>Just enough, but not too much interactivity leads to better clinical skills performance after a computer assisted learning module</article-title><source>Med Teach</source><year>2012</year><volume>34</volume><issue>10</issue><fpage>833</fpage><lpage>9</lpage><comment><ext-link ext-link-type="uri" xlink:href="http://europepmc.org/abstract/MED/22917265"></ext-link></comment><pub-id pub-id-type="doi">10.3109/0142159X.2012.706727</pub-id><pub-id pub-id-type="medline">22917265</pub-id><pmc-comment>PMC3826788 </pmc-comment>
<pub-id pub-id-type="pmid">22917265</pub-id></element-citation></ref><ref id="ref36"><label>36</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Hege</surname><given-names>I</given-names></name><name><surname>Kopp</surname><given-names>V</given-names></name><name><surname>Adler</surname><given-names>M</given-names></name><name><surname>Radon</surname><given-names>K</given-names></name><name><surname>Mäsch</surname><given-names>G</given-names></name><name><surname>Lyon</surname><given-names>H</given-names></name><name><surname>Fischer</surname><given-names>MR</given-names></name></person-group><article-title>Experiences with different integration strategies of case-based e-learning</article-title><source>Med Teach</source><year>2007</year><volume>29</volume><issue>8</issue><fpage>791</fpage><lpage>7</lpage><pub-id pub-id-type="doi">10.1080/01421590701589193</pub-id><pub-id pub-id-type="medline">18236274</pub-id><pmc-comment>789469965 </pmc-comment>
<pub-id pub-id-type="pmid">18236274</pub-id></element-citation></ref><ref id="ref37"><label>37</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Kim</surname><given-names>S</given-names></name><name><surname>Phillips</surname><given-names>WR</given-names></name><name><surname>Pinsky</surname><given-names>L</given-names></name><name><surname>Brock</surname><given-names>D</given-names></name><name><surname>Phillips</surname><given-names>K</given-names></name><name><surname>Keary</surname><given-names>J</given-names></name></person-group><article-title>A conceptual framework for developing teaching cases: a review and synthesis of the literature across disciplines</article-title><source>Med Educ</source><year>2006</year><month>9</month><volume>40</volume><issue>9</issue><fpage>867</fpage><lpage>76</lpage><pub-id pub-id-type="doi">10.1111/j.1365-2929.2006.02544.x</pub-id><pub-id pub-id-type="medline">16925637</pub-id><pmc-comment>MED2544 </pmc-comment>
<pub-id pub-id-type="pmid">16925637</pub-id></element-citation></ref><ref id="ref38"><label>38</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Le Novère</surname><given-names>N</given-names></name><name><surname>Finney</surname><given-names>A</given-names></name><name><surname>Hucka</surname><given-names>M</given-names></name><name><surname>Bhalla</surname><given-names>US</given-names></name><name><surname>Campagne</surname><given-names>F</given-names></name><name><surname>Collado-Vides</surname><given-names>J</given-names></name><name><surname>Crampin</surname><given-names>EJ</given-names></name><name><surname>Halstead</surname><given-names>M</given-names></name><name><surname>Klipp</surname><given-names>E</given-names></name><name><surname>Mendes</surname><given-names>P</given-names></name><name><surname>Nielsen</surname><given-names>P</given-names></name><name><surname>Sauro</surname><given-names>H</given-names></name><name><surname>Shapiro</surname><given-names>B</given-names></name><name><surname>Snoep</surname><given-names>JL</given-names></name><name><surname>Spence</surname><given-names>HD</given-names></name><name><surname>Wanner</surname><given-names>BL</given-names></name></person-group><article-title>Minimum information requested in the annotation of biochemical models (MIRIAM)</article-title><source>Nat Biotechnol</source><year>2005</year><month>12</month><volume>23</volume><issue>12</issue><fpage>1509</fpage><lpage>15</lpage><pub-id pub-id-type="doi">10.1038/nbt1156</pub-id><pub-id pub-id-type="medline">16333295</pub-id><pmc-comment>nbt1156 </pmc-comment>
<pub-id pub-id-type="pmid">16333295</pub-id></element-citation></ref><ref id="ref39"><label>39</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Waltemath</surname><given-names>D</given-names></name><name><surname>Adams</surname><given-names>R</given-names></name><name><surname>Beard</surname><given-names>DA</given-names></name><name><surname>Bergmann</surname><given-names>FT</given-names></name><name><surname>Bhalla</surname><given-names>US</given-names></name><name><surname>Britten</surname><given-names>R</given-names></name><name><surname>Chelliah</surname><given-names>V</given-names></name><name><surname>Cooling</surname><given-names>MT</given-names></name><name><surname>Cooper</surname><given-names>J</given-names></name><name><surname>Crampin</surname><given-names>EJ</given-names></name><name><surname>Garny</surname><given-names>A</given-names></name><name><surname>Hoops</surname><given-names>S</given-names></name><name><surname>Hucka</surname><given-names>M</given-names></name><name><surname>Hunter</surname><given-names>P</given-names></name><name><surname>Klipp</surname><given-names>E</given-names></name><name><surname>Laibe</surname><given-names>C</given-names></name><name><surname>Miller</surname><given-names>AK</given-names></name><name><surname>Moraru</surname><given-names>I</given-names></name><name><surname>Nickerson</surname><given-names>D</given-names></name><name><surname>Nielsen</surname><given-names>P</given-names></name><name><surname>Nikolski</surname><given-names>M</given-names></name><name><surname>Sahle</surname><given-names>S</given-names></name><name><surname>Sauro</surname><given-names>HM</given-names></name><name><surname>Schmidt</surname><given-names>H</given-names></name><name><surname>Snoep</surname><given-names>JL</given-names></name><name><surname>Tolle</surname><given-names>D</given-names></name><name><surname>Wolkenhauer</surname><given-names>O</given-names></name><name><surname>Le Novère</surname><given-names>N</given-names></name></person-group><article-title>Minimum Information About a Simulation Experiment (MIASE)</article-title><source>PLoS Comput Biol</source><year>2011</year><month>4</month><volume>7</volume><issue>4</issue><fpage>e1001122</fpage><comment><ext-link ext-link-type="uri" xlink:href="http://dx.plos.org/10.1371/journal.pcbi.1001122"></ext-link></comment><pub-id pub-id-type="doi">10.1371/journal.pcbi.1001122</pub-id><pub-id pub-id-type="medline">21552546</pub-id><pmc-comment>PMC3084216 </pmc-comment>
<pub-id pub-id-type="pmid">21552546</pub-id></element-citation></ref><ref id="ref40"><label>40</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Posel</surname><given-names>N</given-names></name><name><surname>Fleiszer</surname><given-names>D</given-names></name><name><surname>Shore</surname><given-names>BM</given-names></name></person-group><article-title>12 Tips: Guidelines for authoring virtual patient cases</article-title><source>Med Teach</source><year>2009</year><month>8</month><volume>31</volume><issue>8</issue><fpage>701</fpage><lpage>8</lpage><pub-id pub-id-type="doi">10.1080/01421590902793867</pub-id><pub-id pub-id-type="medline">19513927</pub-id><pmc-comment>912308904 </pmc-comment>
<pub-id pub-id-type="pmid">19513927</pub-id></element-citation></ref><ref id="ref41"><label>41</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Kononowicz</surname><given-names>AA</given-names></name><name><surname>Heid</surname><given-names>J</given-names></name><name><surname>Donkers</surname><given-names>J</given-names></name><name><surname>Hege</surname><given-names>I</given-names></name><name><surname>Woodham</surname><given-names>L</given-names></name><name><surname>Zary</surname><given-names>N</given-names></name></person-group><article-title>Development and validation of strategies to test for interoperability of virtual patients</article-title><source>Stud Health Technol Inform</source><year>2009</year><volume>150</volume><fpage>185</fpage><lpage>9</lpage><pub-id pub-id-type="medline">19745294</pub-id><pub-id pub-id-type="pmid">19745294</pub-id></element-citation></ref><ref id="ref42"><label>42</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Tworek</surname><given-names>J</given-names></name><name><surname>Paget</surname><given-names>M</given-names></name><name><surname>McLauglin</surname><given-names>K</given-names></name><name><surname>Wright</surname><given-names>B</given-names></name></person-group><article-title>How Dungeons & Dragons Made Us Better VPs: Randomizing Physiological Data to Rapidly Produce 97 Clinically Realistic VPs</article-title><source>Bio-Algorithms and Med-Systems</source><year>2010</year><volume>6</volume><issue>11</issue><fpage>41</fpage><lpage>5</lpage></element-citation></ref><ref id="ref43"><label>43</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Kononowicz</surname><given-names>AA</given-names></name><name><surname>Zary</surname><given-names>N</given-names></name><name><surname>Davies</surname><given-names>D</given-names></name><name><surname>Heid</surname><given-names>J</given-names></name><name><surname>Woodham</surname><given-names>L</given-names></name><name><surname>Hege</surname><given-names>I</given-names></name></person-group><article-title>Push and pull models to manage patient consent and licensing of multimedia resources in digital repositories for case-based reasoning</article-title><source>Stud Health Technol Inform</source><year>2011</year><volume>169</volume><fpage>203</fpage><lpage>7</lpage><pub-id pub-id-type="medline">21893742</pub-id><pub-id pub-id-type="pmid">21893742</pub-id></element-citation></ref><ref id="ref44"><label>44</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Bode</surname><given-names>AS</given-names></name><name><surname>Huberts</surname><given-names>W</given-names></name><name><surname>Bosboom</surname><given-names>EM</given-names></name><name><surname>Kroon</surname><given-names>W</given-names></name><name><surname>van der Linden</surname><given-names>WP</given-names></name><name><surname>Planken</surname><given-names>RN</given-names></name><name><surname>van de Vosse</surname><given-names>FN</given-names></name><name><surname>Tordoir</surname><given-names>JH</given-names></name></person-group><article-title>Patient-specific computational modeling of upper extremity arteriovenous fistula creation: its feasibility to support clinical decision-making</article-title><source>PLoS One</source><year>2012</year><volume>7</volume><issue>4</issue><fpage>e34491</fpage><comment><ext-link ext-link-type="uri" xlink:href="http://dx.plos.org/10.1371/journal.pone.0034491"></ext-link></comment><pub-id pub-id-type="doi">10.1371/journal.pone.0034491</pub-id><pub-id pub-id-type="medline">22496816</pub-id><pmc-comment>PONE-D-11-20822 </pmc-comment>
<pmc-comment>PMC3319586 </pmc-comment>
<pub-id pub-id-type="pmid">22496816</pub-id></element-citation></ref><ref id="ref45"><label>45</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Passera</surname><given-names>K</given-names></name><name><surname>Manini</surname><given-names>S</given-names></name><name><surname>Antiga</surname><given-names>L</given-names></name><name><surname>Remuzzi</surname><given-names>A</given-names></name></person-group><article-title>Patient-specific model of arterial circulation for surgical planning of vascular access</article-title><source>J Vasc Access</source><year>2013</year><volume>14</volume><issue>2</issue><fpage>180</fpage><lpage>92</lpage><pub-id pub-id-type="doi">10.5301/jva.5000099</pub-id><pub-id pub-id-type="medline">23032951</pub-id><pmc-comment>FC202852-483E-49C8-9A1B-23D2F146EDD1 </pmc-comment>
<pub-id pub-id-type="pmid">23032951</pub-id></element-citation></ref><ref id="ref46"><label>46</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Ellaway</surname><given-names>R</given-names></name><name><surname>Topps</surname><given-names>D</given-names></name><name><surname>MacDonald</surname><given-names>J</given-names></name><name><surname>Copeland</surname><given-names>B</given-names></name><name><surname>Olmos</surname><given-names>A</given-names></name><name><surname>Spencer</surname><given-names>B</given-names></name></person-group><article-title>HSVO: A Functional XML Specification for Integrating Simulation Devices</article-title><source>Bio-Algorithms and Med-Systems</source><year>2010</year><volume>6</volume><issue>11</issue><fpage>53</fpage><lpage>7</lpage></element-citation></ref></ref-list></back></pmc></record>Pour manipuler ce document sous Unix (Dilib)
EXPLOR_STEP=$WICRI_ROOT/Ticri/CIDE/explor/HapticV1/Data/Pmc/Curation
HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 002021 | SxmlIndent | more
Ou
HfdSelect -h $EXPLOR_AREA/Data/Pmc/Curation/biblio.hfd -nk 002021 | SxmlIndent | more
Pour mettre un lien sur cette page dans le réseau Wicri
{{Explor lien
|wiki= Ticri/CIDE
|area= HapticV1
|flux= Pmc
|étape= Curation
|type= RBID
|clé= PMC:3906686
|texte= A Framework for Different Levels of Integration of Computational Models Into Web-Based Virtual Patients
}}
Pour générer des pages wiki
HfdIndexSelect -h $EXPLOR_AREA/Data/Pmc/Curation/RBID.i -Sk "pubmed:24463466" \
| HfdSelect -Kh $EXPLOR_AREA/Data/Pmc/Curation/biblio.hfd \
| NlmPubMed2Wicri -a HapticV1
| This area was generated with Dilib version V0.6.23. |  |