Sensitivity analysis of infectious disease models: methods, advances and their application
Identifieur interne : 001F95 ( Istex/Corpus ); précédent : 001F94; suivant : 001F96Sensitivity analysis of infectious disease models: methods, advances and their application
Auteurs : Jianyong Wu ; Radhika Dhingra ; Manoj Gambhir ; Justin V. RemaisSource :
- Journal of the Royal Society Interface [ 1742-5689 ] ; 2013.
Abstract
Sensitivity analysis (SA) can aid in identifying influential model parameters and optimizing model structure, yet infectious disease modelling has yet to adopt advanced SA techniques that are capable of providing considerable insights over traditional methods. We investigate five global SA methods—scatter plots, the Morris and Sobol’ methods, Latin hypercube sampling-partial rank correlation coefficient and the sensitivity heat map method—and detail their relative merits and pitfalls when applied to a microparasite (cholera) and macroparasite (schistosomaisis) transmission model. The methods investigated yielded similar results with respect to identifying influential parameters, but offered specific insights that vary by method. The classical methods differed in their ability to provide information on the quantitative relationship between parameters and model output, particularly over time. The heat map approach provides information about the group sensitivity of all model state variables, and the parameter sensitivity spectrum obtained using this method reveals the sensitivity of all state variables to each parameter over the course of the simulation period, especially valuable for expressing the dynamic sensitivity of a microparasite epidemic model to its parameters. A summary comparison is presented to aid infectious disease modellers in selecting appropriate methods, with the goal of improving model performance and design.
Url:
DOI: 10.1098/rsif.2012.1018
Links to Exploration step
ISTEX:03CD6E3087F62163FAA9FBF4A67A8901D755C741Le document en format XML
<record><TEI wicri:istexFullTextTei="biblStruct"><teiHeader><fileDesc><titleStmt><title>Sensitivity analysis of infectious disease models: methods, advances and their application</title><author><name sortKey="Wu, Jianyong" sort="Wu, Jianyong" uniqKey="Wu J" first="Jianyong" last="Wu">Jianyong Wu</name><affiliation><mods:affiliation>Department of Environmental Health, Rollins School of Public Health, Emory University, 1518 Clifton Road NE, Atlanta, GA 30322 USA</mods:affiliation></affiliation></author><author><name sortKey="Dhingra, Radhika" sort="Dhingra, Radhika" uniqKey="Dhingra R" first="Radhika" last="Dhingra">Radhika Dhingra</name><affiliation><mods:affiliation>Department of Environmental Health, Rollins School of Public Health, Emory University, 1518 Clifton Road NE, Atlanta, GA 30322 USA</mods:affiliation></affiliation></author><author><name sortKey="Gambhir, Manoj" sort="Gambhir, Manoj" uniqKey="Gambhir M" first="Manoj" last="Gambhir">Manoj Gambhir</name><affiliation><mods:affiliation>MRC Centre for Outbreak Analysis and Modeling, Department of Infectious Disease Epidemiology, Imperial College London, London, UK</mods:affiliation></affiliation></author><author><name sortKey="Remais, Justin V" sort="Remais, Justin V" uniqKey="Remais J" first="Justin V." last="Remais">Justin V. Remais</name><affiliation><mods:affiliation>Department of Environmental Health, Rollins School of Public Health, Emory University, 1518 Clifton Road NE, Atlanta, GA 30322 USA</mods:affiliation></affiliation><affiliation><mods:affiliation>Program in Population Biology, Ecology and Evolution, Graduate Division of Biological and Biomedical Sciences, Emory University, 1510 Clifton Road, Atlanta, GA 30322, USA</mods:affiliation></affiliation><affiliation><mods:affiliation>E-mail: justin.remais@emory.edu</mods:affiliation></affiliation></author></titleStmt><publicationStmt><idno type="wicri:source">ISTEX</idno><idno type="RBID">ISTEX:03CD6E3087F62163FAA9FBF4A67A8901D755C741</idno><date when="2013" year="2013">2013</date><idno type="doi">10.1098/rsif.2012.1018</idno><idno type="url">https://api.istex.fr/ark:/67375/V84-PCJ0MG5K-1/fulltext.pdf</idno><idno type="wicri:Area/Istex/Corpus">001F95</idno><idno type="wicri:explorRef" wicri:stream="Istex" wicri:step="Corpus" wicri:corpus="ISTEX">001F95</idno></publicationStmt><sourceDesc><biblStruct><analytic><title level="a" type="main">Sensitivity analysis of infectious disease models: methods, advances and their application</title><author><name sortKey="Wu, Jianyong" sort="Wu, Jianyong" uniqKey="Wu J" first="Jianyong" last="Wu">Jianyong Wu</name><affiliation><mods:affiliation>Department of Environmental Health, Rollins School of Public Health, Emory University, 1518 Clifton Road NE, Atlanta, GA 30322 USA</mods:affiliation></affiliation></author><author><name sortKey="Dhingra, Radhika" sort="Dhingra, Radhika" uniqKey="Dhingra R" first="Radhika" last="Dhingra">Radhika Dhingra</name><affiliation><mods:affiliation>Department of Environmental Health, Rollins School of Public Health, Emory University, 1518 Clifton Road NE, Atlanta, GA 30322 USA</mods:affiliation></affiliation></author><author><name sortKey="Gambhir, Manoj" sort="Gambhir, Manoj" uniqKey="Gambhir M" first="Manoj" last="Gambhir">Manoj Gambhir</name><affiliation><mods:affiliation>MRC Centre for Outbreak Analysis and Modeling, Department of Infectious Disease Epidemiology, Imperial College London, London, UK</mods:affiliation></affiliation></author><author><name sortKey="Remais, Justin V" sort="Remais, Justin V" uniqKey="Remais J" first="Justin V." last="Remais">Justin V. Remais</name><affiliation><mods:affiliation>Department of Environmental Health, Rollins School of Public Health, Emory University, 1518 Clifton Road NE, Atlanta, GA 30322 USA</mods:affiliation></affiliation><affiliation><mods:affiliation>Program in Population Biology, Ecology and Evolution, Graduate Division of Biological and Biomedical Sciences, Emory University, 1510 Clifton Road, Atlanta, GA 30322, USA</mods:affiliation></affiliation><affiliation><mods:affiliation>E-mail: justin.remais@emory.edu</mods:affiliation></affiliation></author></analytic><monogr></monogr><series><title level="j" type="main">Journal of the Royal Society Interface</title><title level="j" type="abbrev">J. R. Soc. Interface</title><idno type="ISSN">1742-5689</idno><idno type="eISSN">1742-5662</idno><imprint><publisher>The Royal Society</publisher><date type="published">2013</date><biblScope unit="vol">10</biblScope><biblScope unit="issue">86</biblScope></imprint><idno type="ISSN">1742-5689</idno></series></biblStruct></sourceDesc><seriesStmt><idno type="ISSN">1742-5689</idno></seriesStmt></fileDesc><profileDesc><textClass></textClass></profileDesc></teiHeader><front><div type="abstract">Sensitivity analysis (SA) can aid in identifying influential model parameters and optimizing model structure, yet infectious disease modelling has yet to adopt advanced SA techniques that are capable of providing considerable insights over traditional methods. We investigate five global SA methods—scatter plots, the Morris and Sobol’ methods, Latin hypercube sampling-partial rank correlation coefficient and the sensitivity heat map method—and detail their relative merits and pitfalls when applied to a microparasite (cholera) and macroparasite (schistosomaisis) transmission model. The methods investigated yielded similar results with respect to identifying influential parameters, but offered specific insights that vary by method. The classical methods differed in their ability to provide information on the quantitative relationship between parameters and model output, particularly over time. The heat map approach provides information about the group sensitivity of all model state variables, and the parameter sensitivity spectrum obtained using this method reveals the sensitivity of all state variables to each parameter over the course of the simulation period, especially valuable for expressing the dynamic sensitivity of a microparasite epidemic model to its parameters. A summary comparison is presented to aid infectious disease modellers in selecting appropriate methods, with the goal of improving model performance and design.</div></front></TEI><istex><corpusName>rsl</corpusName><author><json:item><name>Jianyong Wu</name><affiliations><json:string>Department of Environmental Health, Rollins School of Public Health, Emory University, 1518 Clifton Road NE, Atlanta, GA 30322 USA</json:string></affiliations></json:item><json:item><name>Radhika Dhingra</name><affiliations><json:string>Department of Environmental Health, Rollins School of Public Health, Emory University, 1518 Clifton Road NE, Atlanta, GA 30322 USA</json:string></affiliations></json:item><json:item><name>Manoj Gambhir</name><affiliations><json:string>MRC Centre for Outbreak Analysis and Modeling, Department of Infectious Disease Epidemiology, Imperial College London, London, UK</json:string></affiliations></json:item><json:item><name>Justin V. Remais</name><affiliations><json:string>Department of Environmental Health, Rollins School of Public Health, Emory University, 1518 Clifton Road NE, Atlanta, GA 30322 USA</json:string><json:string>Program in Population Biology, Ecology and Evolution, Graduate Division of Biological and Biomedical Sciences, Emory University, 1510 Clifton Road, Atlanta, GA 30322, USA</json:string><json:string>E-mail: justin.remais@emory.edu</json:string></affiliations></json:item></author><subject><json:item><lang><json:string>eng</json:string></lang><value>sensitivity analysis</value></json:item><json:item><lang><json:string>eng</json:string></lang><value>infectious disease modelling</value></json:item><json:item><lang><json:string>eng</json:string></lang><value>Sobol’ method</value></json:item><json:item><lang><json:string>eng</json:string></lang><value>Morris method</value></json:item><json:item><lang><json:string>eng</json:string></lang><value>partial rank correlation coefficient</value></json:item><json:item><lang><json:string>eng</json:string></lang><value>sensitivity heat map</value></json:item></subject><articleId><json:string>rsif20121018</json:string></articleId><arkIstex>ark:/67375/V84-PCJ0MG5K-1</arkIstex><language><json:string>eng</json:string></language><originalGenre><json:string>review-article</json:string></originalGenre><abstract>Sensitivity analysis (SA) can aid in identifying influential model parameters and optimizing model structure, yet infectious disease modelling has yet to adopt advanced SA techniques that are capable of providing considerable insights over traditional methods. We investigate five global SA methods—scatter plots, the Morris and Sobol’ methods, Latin hypercube sampling-partial rank correlation coefficient and the sensitivity heat map method—and detail their relative merits and pitfalls when applied to a microparasite (cholera) and macroparasite (schistosomaisis) transmission model. The methods investigated yielded similar results with respect to identifying influential parameters, but offered specific insights that vary by method. The classical methods differed in their ability to provide information on the quantitative relationship between parameters and model output, particularly over time. The heat map approach provides information about the group sensitivity of all model state variables, and the parameter sensitivity spectrum obtained using this method reveals the sensitivity of all state variables to each parameter over the course of the simulation period, especially valuable for expressing the dynamic sensitivity of a microparasite epidemic model to its parameters. A summary comparison is presented to aid infectious disease modellers in selecting appropriate methods, with the goal of improving model performance and design.</abstract><qualityIndicators><score>8.031</score><pdfWordCount>8343</pdfWordCount><pdfCharCount>53425</pdfCharCount><pdfVersion>1.4</pdfVersion><pdfPageCount>14</pdfPageCount><pdfPageSize>595.276 x 841.89 pts (A4)</pdfPageSize><pdfWordsPerPage>596</pdfWordsPerPage><pdfText>true</pdfText><refBibsNative>true</refBibsNative><abstractWordCount>197</abstractWordCount><abstractCharCount>1450</abstractCharCount><keywordCount>6</keywordCount></qualityIndicators><title>Sensitivity analysis of infectious disease models: methods, advances and their application</title><genre><json:string>article</json:string></genre><host><title>Journal of the Royal Society Interface</title><language><json:string>unknown</json:string></language><issn><json:string>1742-5689</json:string></issn><eissn><json:string>1742-5662</json:string></eissn><publisherId><json:string>RSIF</json:string></publisherId><volume>10</volume><issue>86</issue><genre><json:string>journal</json:string></genre><subject><json:item><value>1004</value></json:item><json:item><value>181</value></json:item></subject></host><ark><json:string>ark:/67375/V84-PCJ0MG5K-1</json:string></ark><categories><wos><json:string>1 - science</json:string><json:string>2 - multidisciplinary sciences</json:string></wos><scienceMetrix><json:string>1 - general</json:string><json:string>2 - general science & technology</json:string><json:string>3 - general science & technology</json:string></scienceMetrix><scopus><json:string>1 - Physical Sciences</json:string><json:string>2 - Engineering</json:string><json:string>3 - Biomedical Engineering</json:string><json:string>1 - Life Sciences</json:string><json:string>2 - Biochemistry, Genetics and Molecular Biology</json:string><json:string>3 - Biochemistry</json:string><json:string>1 - Physical Sciences</json:string><json:string>2 - Materials Science</json:string><json:string>3 - Biomaterials</json:string><json:string>1 - Physical Sciences</json:string><json:string>2 - Chemical Engineering</json:string><json:string>3 - Bioengineering</json:string><json:string>1 - Life Sciences</json:string><json:string>2 - Biochemistry, Genetics and Molecular Biology</json:string><json:string>3 - Biophysics</json:string><json:string>1 - Life Sciences</json:string><json:string>2 - Biochemistry, Genetics and Molecular Biology</json:string><json:string>3 - Biotechnology</json:string></scopus><inist><json:string>1 - sciences appliquees, technologies et medecines</json:string><json:string>2 - sciences biologiques et medicales</json:string><json:string>3 - sciences biologiques fondamentales et appliquees. psychologie</json:string></inist></categories><publicationDate>2013</publicationDate><copyrightDate>2013</copyrightDate><doi><json:string>10.1098/rsif.2012.1018</json:string></doi><id>03CD6E3087F62163FAA9FBF4A67A8901D755C741</id><score>1</score><fulltext><json:item><extension>pdf</extension><original>true</original><mimetype>application/pdf</mimetype><uri>https://api.istex.fr/ark:/67375/V84-PCJ0MG5K-1/fulltext.pdf</uri></json:item><json:item><extension>zip</extension><original>false</original><mimetype>application/zip</mimetype><uri>https://api.istex.fr/ark:/67375/V84-PCJ0MG5K-1/bundle.zip</uri></json:item><json:item><extension>txt</extension><original>false</original><mimetype>text/plain</mimetype><uri>https://api.istex.fr/ark:/67375/V84-PCJ0MG5K-1/fulltext.txt</uri></json:item><istex:fulltextTEI uri="https://api.istex.fr/ark:/67375/V84-PCJ0MG5K-1/fulltext.tei"><teiHeader><fileDesc><titleStmt><title level="a" type="main">Sensitivity analysis of infectious disease models: methods, advances and their application</title></titleStmt><publicationStmt><authority>ISTEX</authority><publisher>The Royal Society</publisher><availability><licence>© 2013 The Author(s) Published by the Royal Society. All rights reserved.</licence></availability><date type="published">2013</date><date type="Copyright" when="2013">2013</date></publicationStmt><notesStmt><note type="content-type" source="review-article" scheme="https://content-type.data.istex.fr/ark:/67375/XTP-L5L7X3NF-P">review-article</note><note type="publication-type" scheme="https://publication-type.data.istex.fr/ark:/67375/JMC-0GLKJH51-B">journal</note></notesStmt><sourceDesc><biblStruct type="article"><analytic><title level="a" type="main">Sensitivity analysis of infectious disease models: methods, advances and their application</title><title level="a" type="short">Sensitivity analysis of infectious disease models: methods, advances and their application</title><author xml:id="author-0000"><persName><surname>Wu</surname><forename type="first">Jianyong</forename></persName><affiliation><orgName type="institution">Emory University</orgName><address><addrLine>Department of Environmental Health, Rollins School of Public Health</addrLine><addrLine>1518 Clifton Road NE, Atlanta, GA 30322</addrLine><country key="US" xml:lang="en">UNITED STATES</country></address></affiliation></author><author xml:id="author-0001"><persName><surname>Dhingra</surname><forename type="first">Radhika</forename></persName><affiliation><orgName type="institution">Emory University</orgName><address><addrLine>Department of Environmental Health, Rollins School of Public Health</addrLine><addrLine>1518 Clifton Road NE, Atlanta, GA 30322</addrLine><country key="US" xml:lang="en">UNITED STATES</country></address></affiliation></author><author xml:id="author-0002"><persName><surname>Gambhir</surname><forename type="first">Manoj</forename></persName><affiliation><orgName type="institution">Imperial College London</orgName><address><addrLine>MRC Centre for Outbreak Analysis and Modeling, Department of Infectious Disease Epidemiology</addrLine><addrLine>London</addrLine><country key="GB" xml:lang="en">UNITED KINGDOM</country></address></affiliation></author><author xml:id="author-0003"><persName><surname>Remais</surname><forename type="first">Justin V.</forename></persName><affiliation><orgName type="institution">Emory University</orgName><address><addrLine>Department of Environmental Health, Rollins School of Public Health</addrLine><addrLine>1518 Clifton Road NE, Atlanta, GA 30322</addrLine><country key="US" xml:lang="en">UNITED STATES</country></address></affiliation><affiliation><orgName type="institution">Emory University</orgName><address><addrLine>Program in Population Biology, Ecology and Evolution, Graduate Division of Biological and Biomedical Sciences</addrLine><addrLine>1510 Clifton Road, Atlanta, GA 30322</addrLine><country key="US" xml:lang="en">UNITED STATES</country></address></affiliation></author><idno type="istex">03CD6E3087F62163FAA9FBF4A67A8901D755C741</idno><idno type="ark">ark:/67375/V84-PCJ0MG5K-1</idno><idno type="DOI">10.1098/rsif.2012.1018</idno><idno type="publisher-id">rsif20121018</idno></analytic><monogr><title level="j" type="main">Journal of the Royal Society Interface</title><title level="j" type="abbrev">J. R. Soc. Interface</title><idno type="publisher-id">RSIF</idno><idno type="hwp">royinterface</idno><idno type="pISSN">1742-5689</idno><idno type="eISSN">1742-5662</idno><imprint><publisher>The Royal Society</publisher><date type="published">2013</date><biblScope unit="vol">10</biblScope><biblScope unit="issue">86</biblScope></imprint></monogr></biblStruct></sourceDesc></fileDesc><encodingDesc><schemaRef type="ODD" url="https://xml-schema.delivery.istex.fr/tei-istex.odd"></schemaRef><appInfo><application ident="pub2tei" version="1.0.40" when="2020-03-31"><label>pub2TEI-ISTEX</label><desc>A set of style sheets for converting XML documents encoded in various scientific publisher formats into a common TEI format.
<ref target="http://www.tei-c.org/">We use TEI</ref></desc></application></appInfo></encodingDesc><profileDesc><abstract><p>Sensitivity analysis (SA) can aid in identifying influential model parameters and optimizing model structure, yet infectious disease modelling has yet to adopt advanced SA techniques that are capable of providing considerable insights over traditional methods. We investigate five global SA methods—scatter plots, the Morris and Sobol’ methods, Latin hypercube sampling-partial rank correlation coefficient and the sensitivity heat map method—and detail their relative merits and pitfalls when applied to a microparasite (cholera) and macroparasite (schistosomaisis) transmission model. The methods investigated yielded similar results with respect to identifying influential parameters, but offered specific insights that vary by method. The classical methods differed in their ability to provide information on the quantitative relationship between parameters and model output, particularly over time. The heat map approach provides information about the group sensitivity of all model state variables, and the parameter sensitivity spectrum obtained using this method reveals the sensitivity of all state variables to each parameter over the course of the simulation period, especially valuable for expressing the dynamic sensitivity of a microparasite epidemic model to its parameters. A summary comparison is presented to aid infectious disease modellers in selecting appropriate methods, with the goal of improving model performance and design.</p></abstract><textClass ana="subject"><keywords scheme="hwp-journal-coll"><term>1004</term><term>181</term></keywords><keywords scheme="heading"><term>Review articles</term></keywords></textClass><textClass ana="keyword"><keywords><term>sensitivity analysis</term><term>infectious disease modelling</term><term>Sobol’ method</term><term>Morris method</term><term>partial rank correlation coefficient</term><term>sensitivity heat map</term></keywords></textClass><langUsage><language ident="EN"></language></langUsage></profileDesc><revisionDesc><change when="2020-03-31" who="#istex" xml:id="pub2tei">formatting</change></revisionDesc></teiHeader></istex:fulltextTEI></fulltext><metadata><istex:metadataXml wicri:clean="corpus rsl not found" wicri:toSee="no header"><istex:xmlDeclaration>version="1.0" encoding="utf-8"</istex:xmlDeclaration><istex:docType PUBLIC="-//NLM//DTD Journal Publishing DTD v2.3 20070202//EN" URI="journalpublishing.dtd" name="istex:docType"></istex:docType><istex:document><article article-type="review-article"><front><journal-meta><journal-id journal-id-type="publisher-id">RSIF</journal-id><journal-id journal-id-type="hwp">royinterface</journal-id><journal-title>Journal of the Royal Society Interface</journal-title><abbrev-journal-title>J. R. Soc. Interface</abbrev-journal-title><issn pub-type="ppub">1742-5689</issn><issn pub-type="epub">1742-5662</issn><publisher><publisher-name>The Royal Society</publisher-name></publisher></journal-meta><article-meta><article-id pub-id-type="doi">10.1098/rsif.2012.1018</article-id><article-id pub-id-type="publisher-id">rsif20121018</article-id><article-categories><subj-group subj-group-type="hwp-journal-coll"><subject>1004</subject><subject>181</subject></subj-group><subj-group subj-group-type="heading"><subject>Review articles</subject></subj-group></article-categories><title-group><article-title>Sensitivity analysis of infectious disease models: methods, advances and their application</article-title><alt-title alt-title-type="short">Sensitivity analysis of infectious disease models: methods, advances and their application</alt-title></title-group><contrib-group><contrib contrib-type="author"><name><surname>Wu</surname><given-names>Jianyong</given-names></name><xref ref-type="aff" rid="af1">1</xref></contrib><contrib contrib-type="author"><name><surname>Dhingra</surname><given-names>Radhika</given-names></name><xref ref-type="aff" rid="af1">1</xref></contrib><contrib contrib-type="author"><name><surname>Gambhir</surname><given-names>Manoj</given-names></name><xref ref-type="aff" rid="af2">2</xref></contrib><contrib contrib-type="author"><name><surname>Remais</surname><given-names>Justin V.</given-names></name><xref ref-type="aff" rid="af1">1</xref><xref ref-type="aff" rid="af3">3</xref><xref ref-type="corresp" rid="cor1"></xref></contrib></contrib-group><aff id="af1"><label>1</label><addr-line>Department of Environmental Health, Rollins School of Public Health</addr-line>, <institution>Emory University</institution>, <addr-line>1518 Clifton Road NE, Atlanta, GA 30322</addr-line> <country>USA</country></aff><aff id="af2"><label>2</label><addr-line>MRC Centre for Outbreak Analysis and Modeling, Department of Infectious Disease Epidemiology</addr-line>, <institution>Imperial College London</institution>, <addr-line>London</addr-line>, <country>UK</country></aff><aff id="af3"><label>3</label><addr-line>Program in Population Biology, Ecology and Evolution, Graduate Division of Biological and Biomedical Sciences</addr-line>, <institution>Emory University</institution>, <addr-line>1510 Clifton Road, Atlanta, GA 30322</addr-line>, <country>USA</country></aff><author-notes><corresp id="cor1">e-mail: <email>justin.remais@emory.edu</email></corresp></author-notes><pub-date pub-type="ppub"><day>6</day><month>9</month><year>2013</year></pub-date><volume>10</volume><issue>86</issue><elocation-id>20121018</elocation-id><history><date date-type="received"><day>11</day><month>12</month><year>2012</year></date><date date-type="accepted"><day>21</day><month>6</month><year>2013</year></date></history><permissions><copyright-statement>© 2013 The Author(s) Published by the Royal Society. All rights reserved.</copyright-statement><copyright-year>2013</copyright-year></permissions><self-uri content-type="pdf" xlink:href="rsif20121018.pdf"></self-uri><abstract><p>Sensitivity analysis (SA) can aid in identifying influential model parameters and optimizing model structure, yet infectious disease modelling has yet to adopt advanced SA techniques that are capable of providing considerable insights over traditional methods. We investigate five global SA methods—scatter plots, the Morris and Sobol’ methods, Latin hypercube sampling-partial rank correlation coefficient and the sensitivity heat map method—and detail their relative merits and pitfalls when applied to a microparasite (cholera) and macroparasite (schistosomaisis) transmission model. The methods investigated yielded similar results with respect to identifying influential parameters, but offered specific insights that vary by method. The classical methods differed in their ability to provide information on the quantitative relationship between parameters and model output, particularly over time. The heat map approach provides information about the group sensitivity of all model state variables, and the parameter sensitivity spectrum obtained using this method reveals the sensitivity of all state variables to each parameter over the course of the simulation period, especially valuable for expressing the dynamic sensitivity of a microparasite epidemic model to its parameters. A summary comparison is presented to aid infectious disease modellers in selecting appropriate methods, with the goal of improving model performance and design.</p></abstract><kwd-group><kwd>sensitivity analysis</kwd><kwd>infectious disease modelling</kwd><kwd>Sobol’ method</kwd><kwd>Morris method</kwd><kwd>partial rank correlation coefficient</kwd><kwd>sensitivity heat map</kwd></kwd-group><custom-meta-wrap><custom-meta><meta-name>cover-date</meta-name><meta-value>September 6, 2013</meta-value></custom-meta></custom-meta-wrap></article-meta></front><body><sec id="s1" sec-type="intro"><label>1.</label><title>Introduction</title><p>Mathematical models are widely used to examine, explain and predict the dynamics of infectious disease transmission, and models of specific diseases of global import have played a vital role in developing public health strategies for control and prevention [<xref ref-type="bibr" rid="RSIF20121018C1">1</xref>–<xref ref-type="bibr" rid="RSIF20121018C3">3</xref>]. Infectious disease models provide a mathematical representation of the dynamic transmission cycle, involving interactions between infected and susceptible hosts that are generally expressed as a set of coupled ordinary differential equations (ODEs) [<xref ref-type="bibr" rid="RSIF20121018C3">3</xref>]. Model outputs, the ODE solutions over a simulation interval, provide a dynamic representation of the transmission process. Where possible, model parameters are estimated based on experimental or observational data, and where parameter values have gone unestimated, they are often set to plausible values or ranges based on analogous systems, statistical inference or expert opinion [<xref ref-type="bibr" rid="RSIF20121018C3">3</xref>,<xref ref-type="bibr" rid="RSIF20121018C4">4</xref>]. Recent advances in computational science and information technology have made widespread applications of mathematical modelling possible [<xref ref-type="bibr" rid="RSIF20121018C2">2</xref>], yet model outputs often have complex, nonlinear relationships with model parameters, and inappropriate choices of parameter values can lead to bias in model outputs [<xref ref-type="bibr" rid="RSIF20121018C5">5</xref>,<xref ref-type="bibr" rid="RSIF20121018C6">6</xref>].</p><p>Sensitivity analysis (SA) characterizes the response of model outputs to parameter variation [<xref ref-type="bibr" rid="RSIF20121018C7">7</xref>], helping to allocate resources to follow-up experimentation and field study; to isolate major sources of parametric uncertainty; to identify parameters that can be shed to yield a simpler model; to elucidate the plausible range of system outcomes, for forecasting purposes, when data are not available; and to determine the robustness of a modelling study's qualitative conclusions [<xref ref-type="bibr" rid="RSIF20121018C7">7</xref>–<xref ref-type="bibr" rid="RSIF20121018C10">10</xref>]. Renewed attention to SA is warranted, particularly as advanced techniques have emerged that are capable of providing considerable insights over traditional SA methods. Modern parameter estimation techniques for complex models, often based upon Bayesian methods using, for example, Markov Chain Monte Carlo algorithms, cover complementary territory to SA; following data collection, parameter estimation ideally results in a joint posterior distribution over all of the model parameters given the available data. This posterior information could then be used to determine the sensitivity of the model fit—generally a log likelihood—to each of the parameters, by marginalizing the joint distribution. Much would remain to be known, however, about the parameter sensitivity of other model outputs of interest, leaving ample room for useful SA. Performing SA prior to data collection can also benefit our comprehension of a modelled complex system; indeed, SA can help to determine what data need be collected in order to most informatively narrow parameter and output uncertainty [<xref ref-type="bibr" rid="RSIF20121018C11">11</xref>]. Similar approaches exist in many disciplines and guises, e.g. elasticity theory in economics and ecology [<xref ref-type="bibr" rid="RSIF20121018C12">12</xref>,<xref ref-type="bibr" rid="RSIF20121018C13">13</xref>], response surface methodology [<xref ref-type="bibr" rid="RSIF20121018C14">14</xref>] and design of experiments [<xref ref-type="bibr" rid="RSIF20121018C15">15</xref>] each used in multiple fields.</p><p>It should also be noted that SA techniques such as these may be used to explore the parameter sensitivity of the log-likelihood surface itself, with the Fisher Information Matrix describing gradients on this surface and identifying the most profitable parameters to measure experimentally [<xref ref-type="bibr" rid="RSIF20121018C11">11</xref>]. There is, therefore, both a strong direct and indirect link between SA and parameter estimation. Here, we focus primarily on global sensitivity analysis (GSA) methods, which examine the response of model output variables to parameter variation within a selected parameter space, at a time when those working in epidemiological modelling may not be aware of recent advances in SA. GSA methods include variance-based methods (e.g. the Sobol’ method and Fourier Amplitude Sensitivity Test) [<xref ref-type="bibr" rid="RSIF20121018C16">16</xref>,<xref ref-type="bibr" rid="RSIF20121018C17">17</xref>], global screening methods (e.g. the Morris method, also called the elementary effect method) [<xref ref-type="bibr" rid="RSIF20121018C18">18</xref>,<xref ref-type="bibr" rid="RSIF20121018C19">19</xref>], sampling-based methods (e.g. Monte Carlo filtering; Latin hypercube sampling with partial rank correlation coefficient index, LHS-PRCC) [<xref ref-type="bibr" rid="RSIF20121018C20">20</xref>] and others. The recently developed sensitivity heat map (SHM) method, while not strictly a global method by the above definition, is capable of exploring the sensitivity of complex models to many parameters simultaneously and over time [<xref ref-type="bibr" rid="RSIF20121018C11">11</xref>] and provides comparable results to those above.</p><p>GSA techniques have been broadly applied in systems biology [<xref ref-type="bibr" rid="RSIF20121018C11">11</xref>,<xref ref-type="bibr" rid="RSIF20121018C21">21</xref>], environmental modelling [<xref ref-type="bibr" rid="RSIF20121018C22">22</xref>,<xref ref-type="bibr" rid="RSIF20121018C23">23</xref>] and infectious disease modelling. Examples of recent published infectious disease models that describe and carry out an SA are shown in <xref ref-type="table" rid="RSIF20121018TB1">table 1</xref>, and, among these, LHS-PRCC [<xref ref-type="bibr" rid="RSIF20121018C26">26</xref>–<xref ref-type="bibr" rid="RSIF20121018C28">28</xref>,<xref ref-type="bibr" rid="RSIF20121018C32">32</xref>] and other simple sensitivity methods [<xref ref-type="bibr" rid="RSIF20121018C24">24</xref>,<xref ref-type="bibr" rid="RSIF20121018C25">25</xref>,<xref ref-type="bibr" rid="RSIF20121018C31">31</xref>] are frequently used. GSA methods popular in other mathematical modelling applications, such as the Sobol’ and Morris methods, have sparsely been applied to infectious disease models. Observing the shortage of well-structured SA in such modelling studies, Bailey & Duppenthaler [<xref ref-type="bibr" rid="RSIF20121018C30">30</xref>] summarized the range of SA tools available in the early 1980s. More recently, Okais <italic>et al</italic>. [<xref ref-type="bibr" rid="RSIF20121018C5">5</xref>] emphasized the importance of SA in infectious disease modelling and used a varicella-zoster model to demonstrate how to conduct univariate and multivariate SA. As new GSA methods emerge that are capable of providing considerable insight over traditional methods, adoption by the infectious disease modelling community will be crucial to the advancement of the field.
<table-wrap id="RSIF20121018TB1" position="float"><label>Table 1.</label><caption><p>Select infectious disease modelling literature providing quantitative SA results. SA, sensitivity analysis; n.a., not applicable; PRCC, partial rank correlation coefficient; LHS, Latin hypercube sampling.</p></caption><table frame="hsides" rules="groups"><colgroup><col align="left"></col><col align="left"></col><col align="left"></col><col align="left"></col><col align="left"></col><col align="left"></col></colgroup><thead valign="bottom"><tr><th align="left">pathogen/disease</th><th align="left">no. state variables</th><th align="left">no. parameters</th><th align="left">SA method</th><th align="left">objectives and findings of the SA</th><th align="left">reference</th></tr></thead><tbody><tr><td>cholera</td><td>7</td><td>19</td><td>one-way SA</td><td><italic>objective:</italic> to determine which parameters had the greatest influence on model projections. <italic>Finding</italic>: reservoir size did not substantially affect projected cases</td><td>[<xref ref-type="bibr" rid="RSIF20121018C24">24</xref>]</td></tr><tr><td>rabies virus</td><td>8</td><td>18</td><td>simple graph comparison</td><td><italic>objectives:</italic> to examine whether initial conditions affected the number of rabies cases; to find which control strategies were most effective. <italic>Finding:</italic> initial conditions of dogs not only affected case numbers, but also affected the timing of peak cases; reducing the annual crop of newborn puppies and increasing the dog immunization rate reduced the rabies incidence rate</td><td>[<xref ref-type="bibr" rid="RSIF20121018C25">25</xref>]</td></tr><tr><td>bluetongue virus</td><td>8</td><td>18</td><td>LHS-PRCC</td><td><italic>objective:</italic> to find which parameters impact the basic reproduction number, <italic>R</italic><sub>0.</sub> <italic>Finding</italic>: temperature, the probability of transmission from host to vector, and the vector to host ratio were the most important factors</td><td>[<xref ref-type="bibr" rid="RSIF20121018C26">26</xref>]</td></tr><tr><td>malaria</td><td>7</td><td>17</td><td>normalized forward sensitivity index</td><td><italic>objective:</italic> to determine the sensitivity of the basic reproduction number, <italic>R</italic><sub>0,</sub> to model parameters. <italic>Finding: R</italic><sub>0</sub> was most sensitive to the mosquito biting rate, suggesting that strategies targeting the mosquito biting rate may be effective in malaria control and improved estimates of the mosquito biting rate may improve model precision</td><td>[<xref ref-type="bibr" rid="RSIF20121018C9">9</xref>]</td></tr><tr><td>HIV</td><td>34</td><td>20</td><td>LHS-PRCC</td><td><italic>objective:</italic> to determine which parameters were important in contributing uncertainty to predictions of an HIV model. <italic>Finding:</italic> three key parameters were identified as critical for precisely predicting the number of HIV cases</td><td>[<xref ref-type="bibr" rid="RSIF20121018C27">27</xref>]</td></tr><tr><td>tuberculosis</td><td>5</td><td>11</td><td>PRCC</td><td><italic>objective:</italic> to understand the role of key parameters in a model of tuberculosis transmission in the absence of treatment. <italic>Finding:</italic> uncertainty in the effective contact rate was an important contributor to uncertainty in prevalence of the disease</td><td>[<xref ref-type="bibr" rid="RSIF20121018C28">28</xref>]</td></tr><tr><td>dengue</td><td>n.a.</td><td>n.a.</td><td>regression analysis</td><td><italic>objective:</italic> to explore the response of dengue model outputs to changes in parameters. <italic>Finding:</italic> survival in each life stage of the mosquito, mosquito biting behaviour and the duration of the infectious period in humans were the most important parameters</td><td>[<xref ref-type="bibr" rid="RSIF20121018C29">29</xref>]</td></tr><tr><td>typhoid fever</td><td>9</td><td>24</td><td>partial differentiation</td><td><italic>objective:</italic> to demonstrate how to conduct SA on infectious disease models. <italic>Finding:</italic> partial differentiation was a useful method when applied to infectious disease models</td><td>[<xref ref-type="bibr" rid="RSIF20121018C30">30</xref>]</td></tr><tr><td>AIDS/HIV</td><td>n.a.</td><td>15</td><td>one-way SA</td><td><italic>objective:</italic> to illustrate the effect of small changes in parameter values on model outputs. <italic>Finding:</italic> the model was most sensitive to the migration rate, the number of stages in the progression to AIDS and the average number of new sexual partners</td><td>[<xref ref-type="bibr" rid="RSIF20121018C31">31</xref>]</td></tr><tr><td>tuberculosis</td><td>10</td><td>17</td><td>LHS-PRCC</td><td><italic>objective:</italic> to quantify the impact of variation in parameters on TB prevalence. <italic>Finding:</italic> transmission rate, proportion of fast progressors, initial immunity afforded by previous infection and death rate of untreated TB were the top influential factors</td><td>[<xref ref-type="bibr" rid="RSIF20121018C32">32</xref>]</td></tr></tbody></table></table-wrap></p><p>Here, we assess and compare the performance of five GSA techniques—scatter plots, LHS-PRCC, the Morris method, the Sobol’ method and the SHM method—by applying them to the previously published microparasite and macroparasite models and discuss their relative merits and pitfalls for application to infectious disease models generally.</p></sec><sec id="s2"><label>2.</label><title>Overview of sensitivity analysis methods for infectious disease models</title><sec id="s2a"><label>2.1.</label><title>Scatter plots</title><p>Scatter plots are occasionally used to visually examine the correlation between a model output variable and parameters. An output variable that is sensitive to the selected parameter will yield an obvious correlation pattern in the scatter plot. Generally, a Monte Carlo algorithm is used to sample the parameter space, and multiple scatter plots are drawn illustrating the relationship between each parameter and each output variable of interest [<xref ref-type="bibr" rid="RSIF20121018C7">7</xref>]. Visual recognition of the correlation between parameter and model output values can be contingent on the choice of axis scales.</p></sec><sec id="s2b"><label>2.2.</label><title>The Morris method</title><p>The Morris method, also called the elementary effects method, is based on the ratio of the change in an output variable to the change in an input parameter [<xref ref-type="bibr" rid="RSIF20121018C18">18</xref>]. Given the general relationship between a model's output <bold><italic>Y</italic></bold> and input parameters <bold><italic>X</italic></bold>, <bold><italic>Y</italic></bold> = <italic>ƒ</italic>(<bold><italic>X</italic></bold>), the elementary effect of <italic>x<sub>i</sub></italic> can be expressed as<disp-formula id="RSIF20121018M21"><label>2.1</label><graphic xlink:href="rsif20121018eq21"></graphic></disp-formula>where <inline-formula><inline-graphic xlink:href="rsif20121018ileq1"></inline-graphic></inline-formula> is a scaled vector of <italic>k</italic> input parameters, <italic>y<sub>j</sub></italic> is the state variable of interest, <italic>Δ</italic> is a value in the set {1<italic>/</italic>(<italic>p−</italic>1), … ,1 − 1<italic>/</italic>(<italic>p−</italic>1)} and <italic>p</italic> is the number of levels into which each dimension of the parameter space is divided. The distribution of <italic>EE<sub>i</sub></italic>(<bold><italic>X</italic></bold>), denoted <italic>F<sub>i</sub></italic>, is obtained by repeated random sampling of <bold><italic>X</italic></bold> from its <italic>k</italic>-dimensional, <italic>p</italic>-level parameter space.</p><p>The mean of <italic>F<sub>i</sub></italic> (denoted <italic>μ</italic>), the mean of |<italic>F<sub>i</sub></italic>| (denoted <italic>μ</italic>*; [<xref ref-type="bibr" rid="RSIF20121018C19">19</xref>]), and the standard deviation of <italic>F<sub>i</sub></italic> (denoted <italic>σ</italic>) are the resulting sensitivity measures of an output variable to a parameter. A large <italic>μ</italic> suggests that the parameter has strong influence on the output, while a large <italic>σ</italic> suggests either that the relationship between the parameter and output is nonlinear or the parameter interacts with other parameters [<xref ref-type="bibr" rid="RSIF20121018C33">33</xref>]. Importantly, when the <italic>F<sub>i</sub></italic> distribution contains both positive and negative values, elementary effects may cancel each other out to produce a small <italic>μ</italic>, but the parameter may still be influential; thus, the use of the absolute elementary effect, <italic>μ*</italic>, as a remedy, has been recommended. As the elementary effect of <italic>x<sub>i</sub></italic> as estimated by the Morris method bears more resemblance to <italic>Δ</italic><italic>y<sub>j</sub></italic>/<italic>Δ</italic><italic>x<sub>i</sub>|</italic><bold><sub>X</sub></bold> rather than ∂<italic>y<sub>j</sub></italic>/∂<italic>x<sub>i</sub>|</italic><bold><sub>X</sub></bold> [<xref ref-type="bibr" rid="RSIF20121018C18">18</xref>], the accuracy of the Morris index depends on the smoothness of <italic>y</italic> over the parameter domain.</p><p>The Morris method takes <italic>r</italic>(<italic>k</italic> + 1) model runs, where <italic>r</italic> is the number of trajectories (a sampling path which begins at a random point and each subsequent point varies only one randomly selected parameter, holding all others constant at the values from the last point) through the input parameter space [<xref ref-type="bibr" rid="RSIF20121018C18">18</xref>] and <italic>EE</italic>(<italic>x<sub>i</sub></italic>) is calculated at <italic>k</italic> + 1 points along each trajectory. Although the Morris method is easy to understand, does not depend on assumptions about the model (e.g. monotonicity) and is computationally inexpensive, it cannot quantify the contribution of a parameter to the variability of the output.</p></sec><sec id="s2c"><label>2.3.</label><title>Latin hypercube sampling-partial rank correlation coefficient</title><p>A measure of the nonlinear, but monotonic, relationship between two variables, the PRCC is an efficient and reliable sampling-based SA method that provides a measure of monotonicity between parameters and model output after removing the linear effects of all parameters except the parameter of interest [<xref ref-type="bibr" rid="RSIF20121018C21">21</xref>]. A standard correlation coefficient, <italic>ρ</italic>, for two variables, <italic>x</italic> and <italic>y</italic>, is calculated as follows:<disp-formula id="RSIF20121018M22"><label>2.2</label><graphic xlink:href="rsif20121018eq22"></graphic></disp-formula>where <inline-formula><inline-graphic xlink:href="rsif20121018ileq2"></inline-graphic></inline-formula> are the set of paired, sampled data, <inline-formula><inline-graphic xlink:href="rsif20121018ileq3"></inline-graphic></inline-formula> is the sample mean of <italic>x</italic>, and <inline-formula><inline-graphic xlink:href="rsif20121018ileq4"></inline-graphic></inline-formula> is the sample mean of <italic>y</italic>. The PRCC determines the sensitivity of an output state variable to an input parameter as the linear correlation, <italic>ρ</italic>, between the residuals, <inline-formula><inline-graphic xlink:href="rsif20121018ileq5"></inline-graphic></inline-formula> and <inline-formula><inline-graphic xlink:href="rsif20121018ileq6"></inline-graphic></inline-formula> where <bold><italic>X</italic></bold><italic><sub>j</sub></italic> is the rank transformed, sampled <italic>j</italic>th input parameter, and <bold><italic>Y</italic></bold> is the rank transformed output state variable, while keeping all other parameter values fixed [<xref ref-type="bibr" rid="RSIF20121018C34">34</xref>]; <inline-formula><inline-graphic xlink:href="rsif20121018ileq7"></inline-graphic></inline-formula> and <inline-formula><inline-graphic xlink:href="rsif20121018ileq8"></inline-graphic></inline-formula> are determined for <italic>k</italic> samples by the linear regression models<disp-formula id="RSIF20121018M23"><label>2.3</label><graphic xlink:href="rsif20121018eq23"></graphic></disp-formula>and<disp-formula id="RSIF20121018M24"><label>2.4</label><graphic xlink:href="rsif20121018eq24"></graphic></disp-formula>PRCC is often combined with LHS in the conduct of an SA [<xref ref-type="bibr" rid="RSIF20121018C21">21</xref>,<xref ref-type="bibr" rid="RSIF20121018C27">27</xref>]. LHS is a stratified Monte Carlo sampling method [<xref ref-type="bibr" rid="RSIF20121018C21">21</xref>,<xref ref-type="bibr" rid="RSIF20121018C35">35</xref>] that divides each parameter's range into <italic>N</italic> equal intervals and randomly draws one sample from each interval. The method explores the entire range of each parameter and each interval for each parameter is sampled only once [<xref ref-type="bibr" rid="RSIF20121018C21">21</xref>,<xref ref-type="bibr" rid="RSIF20121018C27">27</xref>]. Because of its dense stratification over the input parameter space and the rapid convergence of the sample mean to the true population mean as the number of samples increases, LHS is more efficient—requiring fewer simulations—than a general Monte Carlo sampling approach [<xref ref-type="bibr" rid="RSIF20121018C36">36</xref>]. The combined LHS-PRCC procedure is described fully elsewhere [<xref ref-type="bibr" rid="RSIF20121018C21">21</xref>,<xref ref-type="bibr" rid="RSIF20121018C27">27</xref>], but generally involves: (i) LHS of the parameter space, (ii) obtaining model output for each set of sampled parameters, (iii) ranking parameter and output values and replacing their original values with their ranks, and (iv) calculating the PRCC for each input parameter.</p></sec><sec id="s2d"><label>2.4.</label><title>The Sobol’ method</title><p>The Sobol’ method is a variance-based GSA technique capable of estimating the influence of individual parameters, or a group of parameters, on the output variables of a nonlinear model [<xref ref-type="bibr" rid="RSIF20121018C16">16</xref>,<xref ref-type="bibr" rid="RSIF20121018C37">37</xref>]. Given a model of the relationship between output variables and parameters, <bold><italic>Y</italic></bold> = <italic>f</italic>(<bold><italic>X</italic></bold>) = <italic>f</italic>(<italic>x</italic><sub>1</sub>, <italic>x</italic><sub>2</sub>, … , <italic>x<sub>k</sub></italic>) that is square integrable over its unit hypercube parameter space, the model function a single state variable, <italic>y</italic> = <italic>ƒ</italic>(<italic><bold>X</bold></italic>), can be decomposed into summands of increasing dimensionality, known as the high-dimensional model representation (HDMR; [<xref ref-type="bibr" rid="RSIF20121018C16">16</xref>])<disp-formula id="RSIF20121018M25"><label>2.5</label><graphic xlink:href="rsif20121018eq25"></graphic></disp-formula></p><p>Sobol’ demonstrated that if each term in this expansion has a zero mean, then the total variance of an output variable can be decomposed into the HDMR ANOVA, represented as<disp-formula id="RSIF20121018M26"><label>2.6</label><graphic xlink:href="rsif20121018eq26"></graphic></disp-formula>where <italic>V</italic>(<italic>y</italic>) is the variance of the model output <italic>y, k</italic> is the number of parameters and <inline-formula><inline-graphic xlink:href="rsif20121018ileq9a"></inline-graphic></inline-formula> <inline-formula><inline-graphic xlink:href="rsif20121018ileq9"></inline-graphic></inline-formula> for a given set of indices, <italic>i</italic><sub>1</sub>, … , <italic>i<sub>s</sub></italic>. The Sobol’ sensitivity indices are the ratios of the partial variance given an individual parameter or the interactions of a parameter subset to the total variance. Two Sobol’ indices are often calculated; the main effects—also called the ‘first-order index’—and the total effects of a parameter, are expressed, respectively, as [<xref ref-type="bibr" rid="RSIF20121018C16">16</xref>]<disp-formula id="RSIF20121018M27"><label>2.7</label><graphic xlink:href="rsif20121018eq27"></graphic></disp-formula>and<disp-formula id="RSIF20121018M28"><label>2.8</label><graphic xlink:href="rsif20121018eq28"></graphic></disp-formula>where <bold><italic>X</italic></bold><sub>∼<italic>i</italic></sub> denotes all elements of <bold><italic>X</italic></bold> except <italic>x<sub>i</sub></italic>. These indices have the property that <inline-formula><inline-graphic xlink:href="rsif20121018ileq10"></inline-graphic></inline-formula> and when <inline-formula><inline-graphic xlink:href="rsif20121018ileq11"></inline-graphic></inline-formula> it can be concluded that <italic>f</italic>(<italic><bold>X</bold></italic>) does not depend on <italic>x<sub>i</sub></italic>, while <inline-formula><inline-graphic xlink:href="rsif20121018ileq12"></inline-graphic></inline-formula> indicates that <italic>f</italic>(<italic><bold>X</bold></italic>) depends solely on <italic>x<sub>i</sub></italic> [<xref ref-type="bibr" rid="RSIF20121018C16">16</xref>].</p><p>Considered versatile and effective [<xref ref-type="bibr" rid="RSIF20121018C38">38</xref>], the Sobol’ method has several key advantages. The method is independent from model structure (e.g. linearity and monotonicity); it captures the effects of individual parameters as well as their interactions and it provides quantitative information on the contribution of each parameter to model sensitivity. A major disadvantage is the high computational cost: given a model with <italic>k</italic> parameters and <italic>n</italic> samples from each parameter, the model must run at least <italic>k*</italic>(<italic>n</italic> + 2) times [<xref ref-type="bibr" rid="RSIF20121018C7">7</xref>] to generate sufficient output in order to calculate partial variances for individual parameters.</p></sec><sec id="s2e"><label>2.5.</label><title>Sensitivity heat map method</title><p>Two new graphical techniques have been developed to analyse the sensitivity of complex models: the SHM and the parameter sensitivity spectrum (PSS) [<xref ref-type="bibr" rid="RSIF20121018C11">11</xref>]. An SHM depicts the sensitivity of each output variable to all (or a subset of) model parameters, while the PSS shows the sensitivity of all (or a subset of) output variables to each individual parameter. The methods were developed to take advantage of the fundamental property of dynamic systems, where a change in high-dimensional parameter spaces will lead to a low-dimensional change in output variables [<xref ref-type="bibr" rid="RSIF20121018C11">11</xref>]. Given a set of <italic>n</italic> ODEs:<disp-formula id="RSIF20121018M29"><label>2.9</label><graphic xlink:href="rsif20121018eq29"></graphic></disp-formula>where <bold><italic>X</italic></bold> = (<italic>x</italic><sub>1</sub>, … , <italic>x<sub>n</sub></italic>) is the vector of state variables, <italic>t</italic> is time and <bold><italic>K</italic></bold> = (<italic>k</italic><sub>1</sub>, … , <italic>k<sub>s</sub></italic>) is a vector of parameters, there is a solution or set of solutions <bold><italic>X</italic></bold> = <bold><italic>g</italic></bold>(<italic>t</italic>, <bold><italic>K</italic></bold>) for 0 ≤ <italic>t</italic> ≤ <italic>T</italic>. A change in parameter, <italic>δ</italic><bold><italic>K</italic></bold> results in a change in the solution, <italic>δ</italic><bold><italic>g</italic></bold> such that <inline-formula><inline-graphic xlink:href="rsif20121018ileq13"></inline-graphic></inline-formula> where <italic>M</italic> is the linear map from parameter space <inline-formula><inline-graphic xlink:href="rsif20121018ileq14"></inline-graphic></inline-formula> to a Hilbert space <inline-formula><inline-graphic xlink:href="rsif20121018ileq15"></inline-graphic></inline-formula> (as defined in [<xref ref-type="bibr" rid="RSIF20121018C39">39</xref>]). Using the notation of Rand [<xref ref-type="bibr" rid="RSIF20121018C11">11</xref>], a time-interval normalized (i.e. by <inline-formula><inline-graphic xlink:href="rsif20121018ileq16"></inline-graphic></inline-formula>) matrix, <italic>M<sub>1</sub></italic>, composed of the partial derivatives <inline-formula><inline-graphic xlink:href="rsif20121018ileq17"></inline-graphic></inline-formula> at all simulation time points <italic>t</italic> = {<italic>t</italic><sub>1</sub>, … ,<italic>t<sub>N</sub></italic>} is subjected to singular value decomposition to arrive at values for <bold><italic>U</italic></bold>, <bold><italic>W</italic></bold> and {<italic>σ</italic><sub><italic>i</italic></sub>}, defined below, which express the sensitivity of output variables to parameters [<xref ref-type="bibr" rid="RSIF20121018C11">11</xref>]. Rand shows that, at a minimum of average error, there exists a unique sequence of positive numbers <italic>σ</italic><sub>1</sub> ≥ … ≥ <italic>σ</italic><sub><italic>s</italic></sub>, a unique set of orthonormal vectors {<bold><italic>V</italic></bold><italic><sub>i</sub></italic>} in <inline-formula><inline-graphic xlink:href="rsif20121018ileq18"></inline-graphic></inline-formula> and a unique <italic>n</italic>-dimensional orthogonal time-series basis set <inline-formula><inline-graphic xlink:href="rsif20121018ileq19"></inline-graphic></inline-formula> in <inline-formula><inline-graphic xlink:href="rsif20121018ileq20"></inline-graphic></inline-formula>, such that a small change in <italic>δ</italic><bold><italic>g</italic></bold>(<italic>t</italic>, <bold><italic>K</italic></bold>) is given by <inline-formula><inline-graphic xlink:href="rsif20121018ileq21"></inline-graphic></inline-formula> where <italic>W<sub>ij</sub></italic> are the elements of <bold><italic>W</italic></bold> = <bold><italic>V</italic></bold><sup>−1</sup>, and <bold><italic>V</italic></bold> = [<bold><italic>V</italic></bold><sub>1</sub>, … ,<bold><italic>V<sub>n</sub></italic></bold>] [<xref ref-type="bibr" rid="RSIF20121018C11">11</xref>]. Thus, the effect of a single parameter on the system, the PSS, is expressed by <inline-formula><inline-graphic xlink:href="rsif20121018ileq22"></inline-graphic></inline-formula>, while the SHM for state variable <italic>g<sub>i</sub></italic>(<italic>t</italic>) is depicted as <inline-formula><inline-graphic xlink:href="rsif20121018ileq23"></inline-graphic></inline-formula>. Compared with other SA methods, this highly computationally efficient approach allows the modeller to observe the sensitivity characteristics of the set of all output variables, instead of a single output variable.</p></sec></sec><sec id="s3" sec-type="methods"><label>3.</label><title>Material and methods</title><sec id="s3a"><label>3.1.</label><title>Infectious disease models</title><p>A previously described a microparasite (cholera) model [<xref ref-type="bibr" rid="RSIF20121018C24">24</xref>] and a macroparasite (schistosomiasis) model [<xref ref-type="bibr" rid="RSIF20121018C40">40</xref>] were used to examine the merits of the five SA methods. Microparasites (e.g. bacteria and viruses) generally replicate within hosts at a rapid rate and are able to generate an epidemic in a short period of time. Characteristics of the microparasite epidemic curve (the time course of the infected population) are often studied using ODE models. Macroparasites are generally larger and longer lived than microparasites; they include parasitic helminths and certain vector-borne diseases. Macroparasites generally cannot carry out their full life cycle within a single host, and their transmission is often sustained over time, achieving steady-state, endemic disease dynamics capable of rising from low levels rapidly, akin to a microparasite epidemic curve, but then rising or declining from the endemic equilibrium very slowly [<xref ref-type="bibr" rid="RSIF20121018C1">1</xref>].</p><sec id="s3a1"><label>3.1.1.</label><title>Cholera model</title><p>A microparasite model describing the transmission dynamics of cholera in Haiti in 2010 was previously used to examine the potential effect of interventions on cases of cholera and mortality [<xref ref-type="bibr" rid="RSIF20121018C24">24</xref>]. This model was slightly modified for the present analysis by excluding the effects of vaccination, leading to a final model with six ODEs and 16 parameters<disp-formula id="RSIF20121018M31"><label>3.1</label><graphic xlink:href="rsif20121018eq31"></graphic></disp-formula><disp-formula id="RSIF20121018M32"><label>3.2</label><graphic xlink:href="rsif20121018eq32"></graphic></disp-formula><disp-formula id="RSIF20121018M33"><label>3.3</label><graphic xlink:href="rsif20121018eq33"></graphic></disp-formula><disp-formula id="RSIF20121018M34"><label>3.4</label><graphic xlink:href="rsif20121018eq34"></graphic></disp-formula><disp-formula id="RSIF20121018M35"><label>3.5</label><graphic xlink:href="rsif20121018eq35"></graphic></disp-formula><disp-formula id="RSIF20121018M36"><label>3.6</label><graphic xlink:href="rsif20121018eq36"></graphic></disp-formula>where <italic>N</italic>(<italic>t</italic>) is the total population, <italic>S</italic>(<italic>t</italic>) the susceptible population, <italic>A</italic>(<italic>t</italic>) the asymptomatic population, <italic>I</italic>(<italic>t</italic>) the infected population, <italic>R</italic>(<italic>t</italic>) the recovered population and <italic>B</italic><sub>H</sub>(<italic>t</italic>) and <italic>B</italic><sub>L</sub>(<italic>t</italic>) are the concentration of high-infectious cholera and low-infectious cholera, respectively, in environmental reservoirs. Model parameters are summarized in <xref ref-type="table" rid="RSIF20121018TB2">table 2</xref> and are fully described elsewhere [<xref ref-type="bibr" rid="RSIF20121018C24">24</xref>]. The model was run in Matlab (R 2011b) for 60 time steps with each parameter set, as described later.
<table-wrap id="RSIF20121018TB2" position="float"><label>Table 2.</label><caption><p>Parameter definitions and ranges for the cholera and schistosomiasis models.</p></caption><table frame="hsides" rules="groups"><colgroup><col align="left"></col><col align="left"></col><col align="left"></col><col align="left"></col><col align="left"></col><col align="left"></col></colgroup><thead valign="top"><tr><th align="left">model</th><th align="left">parameter (units)</th><th align="left">symbol</th><th align="left">value</th><th align="left">range</th><th align="left">code</th></tr></thead><tbody><tr><td rowspan="16">cholera model</td><td>rate of contaminated water consumption</td><td><italic>α</italic></td><td>0.1</td><td>0.05–0.2</td><td><italic>p1</italic></td></tr><tr><td>Low-infectious (LI) <italic>V. cholera</italic> infectious dose (cells)</td><td><italic>κ</italic><sub>L</sub></td><td>10<sup>5</sup></td><td>5 × 10<sup>4</sup>–2 × 10<sup>5</sup></td><td><italic>p2</italic></td></tr><tr><td>Hyperinfectious (HI) <italic>V. cholera</italic> infectious dose (cells)</td><td><italic>κ</italic><sub>H</sub></td><td>2000</td><td>1000–4000</td><td><italic>p3</italic></td></tr><tr><td>rate of decay of HI to LI <italic>V. cholera</italic> (per day)</td><td><italic>χ</italic></td><td>1</td><td>0.5–2</td><td><italic>p4</italic></td></tr><tr><td>death rate of <italic>V. cholera</italic> in the environment (per day)</td><td><italic>δ</italic></td><td>0.033</td><td>0.0167–0.0667</td><td><italic>p5</italic></td></tr><tr><td>natural birth/death rate (per day)</td><td><italic>μ</italic></td><td>4.49 × 10<sup>−5</sup></td><td>2.25 × 10<sup>−5</sup>–8.98 × 10<sup>−5</sup></td><td><italic>p6</italic></td></tr><tr><td>mortality rate, symptomatic cholera</td><td><italic>μ</italic><sub>c</sub></td><td>0.0194</td><td>0.0097–0.0388</td><td><italic>p7</italic></td></tr><tr><td>recovery rate, cholera (per day)</td><td><italic>γ</italic></td><td>0.2</td><td>0.1–0.4</td><td><italic>p8</italic></td></tr><tr><td>proportion of cases asymptomatic patient</td><td><italic>p</italic></td><td>0.79</td><td>0.6–0.9</td><td><italic>p9</italic></td></tr><tr><td>rate of excretion of <italic>V. cholera</italic>, symptomatic patient (cell/day)</td><td><italic>ξ</italic><sub>S</sub></td><td>1.3 × 10<sup>11</sup></td><td>6.5 × 10<sup>10</sup>–2.6×10<sup>11</sup></td><td><italic>p10</italic></td></tr><tr><td>rate of excretion of <italic>V. cholera</italic>, asymptomatic patient (cell/day)</td><td><italic>ξ</italic><sub>A</sub></td><td>1.3 × 10<sup>8</sup></td><td>6.5 × 10<sup>7</sup>–2.6 × 10<sup>8</sup></td><td><italic>p11</italic></td></tr><tr><td>rate of waning of natural immunity (per day)</td><td><italic>ω</italic></td><td>0.00137</td><td>0.00055–0.0055</td><td><italic>p12</italic></td></tr><tr><td>proportion symptomatic individuals receiving antibiotics</td><td><italic>Θ</italic></td><td>0.08</td><td>0.04–0.16</td><td><italic>p13</italic></td></tr><tr><td>relative rate of shedding, receiving antibiotics</td><td><italic>ψ</italic></td><td>0.52</td><td>0.26–1.04</td><td><italic>p14</italic></td></tr><tr><td>relative rate of recovery, receiving antibiotics</td><td><italic>λ</italic></td><td>2.3</td><td>1.05–4.6</td><td><italic>p15</italic></td></tr><tr><td>size of water reservoir (<italic>L</italic>*day)</td><td><italic>W</italic></td><td>5.3 × 10<sup>10</sup></td><td>5.3 × 10<sup>9</sup>–5.3 × 10<sup>11</sup></td><td><italic>p16</italic></td></tr><tr><td rowspan="14">schistosomiasis model</td><td>snail density in a village (snails per square metre)</td><td><italic>x</italic></td><td>20</td><td>10–40</td><td><italic>q1</italic></td></tr><tr><td>human population in a village</td><td><italic>n</italic></td><td>200</td><td>100–400</td><td><italic>q2</italic></td></tr><tr><td>snail habitat area in a village (m<sup>2</sup>)</td><td><italic>a</italic></td><td>20 000</td><td>10 000–40 000</td><td><italic>q3</italic></td></tr><tr><td>surface water area in a village (m<sup>2</sup>)</td><td><italic>b</italic></td><td>20 000</td><td>10 000–40 000</td><td><italic>q4</italic></td></tr><tr><td>snail infection</td><td><italic>ρ</italic></td><td>0.000005</td><td>0.0000025–0.00001</td><td><italic>q5</italic></td></tr><tr><td><italic>per capita</italic> snail mortality (per day)</td><td><italic>ɛ</italic></td><td>0.003</td><td>0.0015-0.006</td><td><italic>q6</italic></td></tr><tr><td>cercarial production (per infected snail)</td><td><italic>σ</italic></td><td>50</td><td>25–100</td><td><italic>q7</italic></td></tr><tr><td>human infection in a village (per cercaria per day)</td><td><italic>β</italic></td><td>0.01</td><td>0.0005-0.002</td><td><italic>q8</italic></td></tr><tr><td><italic>per capita</italic> worm mortality (per day)</td><td><italic>μ</italic></td><td>0.00091</td><td>0.00046–0.00182</td><td><italic>q9</italic></td></tr><tr><td>egg production (per worm pair per gram stool)</td><td><italic>h</italic></td><td>5</td><td>2.5–10</td><td><italic>q10</italic></td></tr><tr><td>stool production (g)</td><td><italic>g</italic></td><td>100</td><td>50–200</td><td><italic>q11</italic></td></tr><tr><td>worm mating probability</td><td><italic>ϕ</italic></td><td>0.4</td><td>0.2–0.8</td><td><italic>q12</italic></td></tr><tr><td>miracidial production (per egg)</td><td><italic>α</italic></td><td>0.01</td><td>0.005–0.02</td><td><italic>q13</italic></td></tr><tr><td>chemotherapy intervention in a village</td><td><italic>π</italic></td><td>0.05</td><td>0.025–0.1</td><td><italic>q14</italic></td></tr></tbody></table></table-wrap></p></sec><sec id="s3a2"><label>3.1.2.</label><title>Schistosomiasis model</title><p>A macroparasite model describing the transmission of schistosomiasis was previously used to simulate the worm burden in human hosts and the density of infected snails in China [<xref ref-type="bibr" rid="RSIF20121018C40">40</xref>,<xref ref-type="bibr" rid="RSIF20121018C41">41</xref>]. For the present analysis, the model was simplified to two state variables and 14 parameters as follows:<disp-formula id="RSIF20121018M37"><label>3.7</label><graphic xlink:href="rsif20121018eq37"></graphic></disp-formula><disp-formula id="RSIF20121018M38"><label>3.8</label><graphic xlink:href="rsif20121018eq38"></graphic></disp-formula><disp-formula id="RSIF20121018M39"><label>3.9</label><graphic xlink:href="rsif20121018eq39"></graphic></disp-formula><disp-formula id="RSIF20121018M310"><label>3.10</label><graphic xlink:href="rsif20121018eq310"></graphic></disp-formula><disp-formula id="RSIF20121018M311"><label>3.11</label><graphic xlink:href="rsif20121018eq311"></graphic></disp-formula>where <italic>W</italic>(<italic>t</italic>) is the average population worm burden, <italic>Z</italic>(<italic>t</italic>) is the density of infected snails and <italic>e</italic>, <italic>m</italic> and <italic>c</italic> are parasite eggs released into environment, miracidia density and cercaria density, respectively. Model parameters are summarized in <xref ref-type="table" rid="RSIF20121018TB2">table 2</xref> and fully described elsewhere [<xref ref-type="bibr" rid="RSIF20121018C40">40</xref>,<xref ref-type="bibr" rid="RSIF20121018C41">41</xref>]. The model was run in Matlab (R 2011b) for 300 time steps with each parameter set, as described later.</p></sec></sec><sec id="s3b"><label>3.2.</label><title>Set-up of parameter space and distribution</title><p>Parameter values and ranges were adopted from previous work [<xref ref-type="bibr" rid="RSIF20121018C24">24</xref>,<xref ref-type="bibr" rid="RSIF20121018C40">40</xref>,<xref ref-type="bibr" rid="RSIF20121018C41">41</xref>], and unspecified parameter ranges were assigned by dividing (lower bound) and multiplying (upper bound) the parameter point estimate by 2. A uniform distribution was assigned to parameters where the distribution was unspecified [<xref ref-type="bibr" rid="RSIF20121018C21">21</xref>]. All sensitivity analyses were run in Matlab (R 2011b) unless otherwise indicated, and model outputs were sampled at every time step (when examining sensitivity over time) and near the peak level of infection at the 15th simulation step (when examining sensitivity at a single point in time).</p></sec><sec id="s3c"><label>3.3.</label><title>Sensitivity analysis</title><p>The primary model outputs of interest for the sensitivity analyses were the infected population, <italic>I</italic>(<italic>t</italic>), in the microparasite model and the worm burden, <italic>W</italic>(<italic>t</italic>), in the macroparasite model. Recall, however, that the SHM method evaluates all output variables simultaneously.</p><sec id="s3c1"><label>3.3.1.</label><title>Scatter plots</title><p>For both models, 1000 samples from a uniform distribution of each parameter range were generated and used as simulation inputs to obtain the output values of interest. Scatter plot matrices were drawn showing the relationship between each parameter and the output variable of interest at a single time point (the 15th time step).</p></sec><sec id="s3c2"><label>3.3.2.</label><title>The Morris methods</title><p>For both models, each parameter range was divided into six levels (<italic>p</italic> = 6, <italic>Δ</italic> = 0.6). A total of 100 trajectories of 17 points and 15 points, respectively, were built for the cholera model and schistosomiasis model, as described elsewhere [<xref ref-type="bibr" rid="RSIF20121018C18">18</xref>]. For each parameter, 1700 and 1500 samples were generated for the cholera and schistosomiasis models, respectively. Model outputs were then obtained for each parameter set, and Morris indices, including <italic>μ</italic>, <italic>μ</italic>* and <italic>σ</italic>, were calculated as described earlier.</p></sec><sec id="s3c3"><label>3.3.3.</label><title>Latin hypercube sampling with partial rank correlation coefficient index</title><p>Using Latin hypercube sampling, 100 samples from a uniform distribution of the parameter ranges of both models were taken. Model outputs were obtained and the parameter and output values were transformed into their ranks. PRCCs were calculated following the procedures described previously [<xref ref-type="bibr" rid="RSIF20121018C21">21</xref>]. PRCC falls between −1 and +1, with an absolute value of PRCC close to 1 indicating the parameter has a strong impact on the model output.</p></sec><sec id="s3c4"><label>3.3.4.</label><title>The Sobol’ method</title><p>Using Latin hypercube sampling, 100 samples were drawn from a uniform distribution of each parameter range for both models. Samples were permuted randomly 100 times yielding 10 000 parameter sets that were used in models runs. The first order of the Sobol’ index was calculated as described earlier.</p></sec><sec id="s3c5"><label>3.3.5.</label><title>The sensitivity heat map method</title><p>The software package SASSY [<xref ref-type="bibr" rid="RSIF20121018C42">42</xref>] was used to analyse the ODE models and to generate SHMs and PSS within Matlab (R2011b). After carrying out a limit cycle simulation using the Matlab ODE45 solver, model output was obtained for 60 time steps for the cholera model and 300 time steps for the schistosomiasis model, and the derivatives of model variables with respect to model parameters for all simulation time points were entered into a partial derivative matrix. Following the previous work [<xref ref-type="bibr" rid="RSIF20121018C11">11</xref>], singular value decomposition of the partial derivative matrix was used to generate SHM and PSS, respectively. For SHMs, the sensitivity strip of each output variable was sorted from high to low by its maximum value and scaled by a ratio of its maximum value to the maximum value of the colour bar to produce the sensitivity strip scale.</p></sec></sec></sec><sec id="s4" sec-type="results"><label>4.</label><title>Results</title><p>A typical epidemic curve of the infected population size in the cholera model and the endemic curve of the worm burden in the human host in the schistosomiasis model are shown in the electronic supplementary material, figure S1, generated using the parameter values listed in <xref ref-type="table" rid="RSIF20121018TB2">table 2</xref>. The epidemic (cholera) curve rises sharply, quickly peaking at day 6 and dropping back to baseline after day 30. By contrast, the endemic (schistosomiasis) curve has a blunt peak at day 50 followed by a slow decline, illustrating the sustained burden of long-lived macroparasites in human hosts.</p><p>Cholera model scatter plots (<xref ref-type="fig" rid="RSIF20121018F1">figure 1</xref>) show the variation in the infected population size with changes in parameters when examined at the peak of the epidemic (time step 15). Model sensitivity to <italic>p8</italic> and <italic>p9</italic> is evident, but sensitivity to other parameters is unclear. The Morris index (<xref ref-type="fig" rid="RSIF20121018F2">figure 2</xref>) concurs, finding that <italic>p8</italic> and <italic>p9</italic> exert the strongest influence on the infected population size. However, the Morris index also identifies parameters <italic>p1</italic>, <italic>p15</italic>, <italic>p7</italic>, <italic>p13</italic> and <italic>p16</italic> as influential. The PRCC index provides a measure of the relative influence of these parameters (as well as the direction of the relationship), agreeing in the ranking of important parameters with the Morris index. Finally, the Sobol’ sensitivity index indicates that parameters <italic>p8</italic>, <italic>p9</italic> and <italic>p1</italic> most strongly influence infected population size when evaluated near the peak population.
<fig id="RSIF20121018F1"><label>Figure 1.</label><caption><p>Scatter plots illustrating the relationship between the cholera model infected population size, <italic>I</italic>, examined at the peak of the epidemic (<italic>t</italic> = 15) and parameters.</p></caption><graphic xlink:href="rsif20121018f01"></graphic></fig><fig id="RSIF20121018F2"><label>Figure 2.</label><caption><p>Sensitivity of the infected population size to changes in parameters in the cholera model as indicated by (<italic>a</italic>) the Morris index, <italic>μ</italic>*, (<italic>b</italic>) the PRCC index and (<italic>c</italic>) the Sobol’ index, <italic>S<sub>i</sub></italic>.</p></caption><graphic xlink:href="rsif20121018f02"></graphic></fig></p><p>When the sensitivity index for an output variable is examined over the time course of the simulation, the relative importance of parameters is shown to vary widely depending on the time point of inspection (<xref ref-type="fig" rid="RSIF20121018F3">figure 3</xref>). For the cholera model, the Morris index shows <italic>p9</italic>, <italic>p8</italic> and <italic>p1</italic> to be the most influential parameters on the infected population size, but their relative importance changes during the simulation, where for instance <italic>p8</italic> overtakes <italic>p1</italic> and <italic>p9</italic> in importance after day 10. However, the index values of these parameters all decline after day 10 and are close to 0 at the end of simulation. The PRCC and the Sobol’ indices also show variation in parameter sensitivity; however, these indices cannot meaningfully be interpreted over time. Both PRCC and Sobol’, for instance, suggest that <italic>p12</italic> has no influence on the infected population size state variable at the beginning of the simulation, but then has influence later in the simulation period (<xref ref-type="fig" rid="RSIF20121018F3">figure 3</xref>). Yet, neither measure is indexed to some unchanging value (e.g. a perturbation of the parameter of interest as in the Morris method or unit change in infected population). Thus, without accounting for the overall variation of the infected population size over the simulation period (see the electronic supplementary material, figure S1<italic>a</italic>), these indices cannot be interpreted as indicating that <italic>p12</italic> is more important later in the simulation than at early time points. If this variation is taken into account—i.e. that the state variable is small near the end of the outbreak—the influence of <italic>p12</italic> might not in fact be high.
<fig id="RSIF20121018F3"><label>Figure 3.</label><caption><p>The temporal variation of the sensitivity of the infected population size to key parameters in the cholera model indicated by (<italic>a</italic>) the Morris index <italic>μ</italic>*, (<italic>b</italic>) the PRCC index and (<italic>c</italic>) the Sobol’ index <italic>S<sub>i</sub></italic>. Sensitivity to all model parameters is shown in the electronic supplementary material, figure S2.</p></caption><graphic xlink:href="rsif20121018f03"></graphic></fig></p><p>The sensitivity of each cholera model output variable to all parameters as a whole is shown by the SHM in <xref ref-type="fig" rid="RSIF20121018F4">figure 4</xref><italic>a</italic>. The colour strip indicates the product of the first singular value (<italic>σ</italic>1) and the first principle component (<italic>pc1</italic>) of the derivative of the solution (the state variable) to the parameter (<italic>U1j</italic>). Thus, by comparison of the scale ratio for each sensitivity strip, it is clear that state variables <italic>B</italic><sub>H</sub>, <italic>B</italic><sub>L</sub> and <italic>I</italic> are much more sensitive to parameters as a whole than are <italic>A</italic>, <italic>S</italic> and <italic>R</italic>. Looking at individual state variables, sensitivity differs widely across the simulation time period. For <italic>B</italic><sub>H</sub>, <italic>A</italic>, <italic>I</italic> and <italic>S</italic>, sensitivity is highest early in the simulation interval, during the epidemic peak. High sensitivity of <italic>B</italic><sub>L</sub> is achieved in the middle of the simulation, versus late in the period for <italic>R</italic>. This is because <italic>B</italic><sub>L</sub> gradually peaks in the middle period and then decreases, while <italic>R</italic> reaches and remains at its highest value in the late period. <xref ref-type="fig" rid="RSIF20121018F4">Figure 4</xref><italic>b</italic> shows the sensitivity of output variables as a whole to each individual parameter, confirming parameter <italic>p9</italic> as most influential when evaluated simultaneously across all state variables. Interestingly, parameter <italic>p1</italic>, individually important for determining the infected population size (<xref ref-type="fig" rid="RSIF20121018F2">figure 2</xref>) is of limited importance when sensitivity is examined together for all state variables (<xref ref-type="fig" rid="RSIF20121018F4">figure 4</xref><italic>b</italic>).
<fig id="RSIF20121018F4"><label>Figure 4.</label><caption><p>(<italic>a</italic>) SHM and (<italic>b</italic>) PSS for the cholera model, following previous work [<xref ref-type="bibr" rid="RSIF20121018C11">11</xref>].</p></caption><graphic xlink:href="rsif20121018f04"></graphic></fig></p><p>Applying the five methods to the schistosomiasis model yields several interesting contrasts. The Morris, PRCC and Sobol’ indices all indicate the importance of parameters <italic>q3</italic>, <italic>q4</italic>, <italic>q6</italic>, <italic>q7</italic> and <italic>q14</italic>. Though both models show temporally varying sensitivity indices, the sensitivity indices of the most important parameters in the schistosomiasis model are, unlike the cholera model (<xref ref-type="fig" rid="RSIF20121018F3">figure 3</xref>), approximately constant later in the simulation (<xref ref-type="fig" rid="RSIF20121018F5">figure 5</xref>). The SHM for the schistosomiasis model (<xref ref-type="fig" rid="RSIF20121018F6">figure 6</xref><italic>a</italic>) shows that the worm burden state variable is considerably more sensitive to model parameters, when taken as a whole, than the infected snail density variable. That the worm burden is highly sensitive to model parameters from days 40 to 120, and is subsequently slightly less sensitive afterward, is consistent with the results of traditional sensitivity methods as shown in <xref ref-type="fig" rid="RSIF20121018F5">figure 5</xref> and corresponds to the endemic curve of the worm burden which reaches its peak at day 40 and is stable through the remainder of the simulation (see the electronic supplementary material, figure S1<italic>b</italic>). Finally, the PSS (<xref ref-type="fig" rid="RSIF20121018F6">figure 6</xref><italic>b</italic>) reveals that both state variables, considered together, are sensitive to <italic>q3</italic>, <italic>q4</italic>, <italic>q6</italic>, <italic>q7</italic>, <italic>q8</italic> and <italic>q14</italic>. These PSS-determined sensitive parameters correspond, with the exception of <italic>q8</italic>, with the results of traditional methods applied only to worm burden (<xref ref-type="fig" rid="RSIF20121018F5">figure 5</xref>).
<fig id="RSIF20121018F5"><label>Figure 5.</label><caption><p>Temporal variation of the sensitivity of the worm burden (<italic>W</italic>) to key parameters in the schistosomiasis model as indicated by (<italic>a</italic>) the Morris index <italic>μ</italic>*, (<italic>b</italic>) the PRCC index and (c) the Sobol’ index, <italic>S<sub>i</sub></italic>. Sensitivity to all model parameters is shown in the electronic supplementary material, figure S3.</p></caption><graphic xlink:href="rsif20121018f05"></graphic></fig><fig id="RSIF20121018F6"><label>Figure 6.</label><caption><p>(<italic>a</italic>) SHM and (<italic>b</italic>) PSS of worm burden (<italic>W</italic>) and snail density (<italic>Z</italic>) for the schistosomiasis model, following previous work [<xref ref-type="bibr" rid="RSIF20121018C11">11</xref>].</p></caption><graphic xlink:href="rsif20121018f06"></graphic></fig></p></sec><sec id="s5" sec-type="discussion"><label>5.</label><title>Discussion</title><p>We described five GSA methods and examined their relative merit in identifying important parameters and characterizing their influence in two infectious disease mathematical models. The methods yielded similar results with respect to identifying influential parameters, but offered different insights as to the importance of parameters over the simulation period, as well as different information when applied to microparasite versus macroparasite models.</p><p>Scatter plots offer a straightforward way to visualize the relationship between parameters and output variables, providing a natural starting point for SA [<xref ref-type="bibr" rid="RSIF20121018C7">7</xref>,<xref ref-type="bibr" rid="RSIF20121018C21">21</xref>]. The Morris, LHS-PRCC and Sobol’ methods can provide more definitive information, effectively distinguishing between parameters that are of high versus negligible importance. The Morris method is mainly used as a screening technique for identifying important parameters [<xref ref-type="bibr" rid="RSIF20121018C7">7</xref>,<xref ref-type="bibr" rid="RSIF20121018C35">35</xref>], while the PRCC index, ranging from −1 to +1, provides specific qualitative information on the relationship between an output variable and a parameter—as well as indicates whether the relationship is concordant or discordant—after controlling for the influence of other parameters [<xref ref-type="bibr" rid="RSIF20121018C21">21</xref>]. Based on the Sobol’ first-order sensitivity index, <italic>S<sub>i</sub></italic>, four parameters were found to be the primary contributors to the variance of the infected population size in the cholera model, and five were found to greatly influence worm burden in the schistosomiasis model. The Sobol’ index, a variance decomposition approach, also captures the influence of the interaction among parameters on a model output [<xref ref-type="bibr" rid="RSIF20121018C7">7</xref>]. The first-order sensitivity index, <italic>S<sub>i</sub></italic>, reflects the contribution of each parameter to the variance of the output variable, while the total effect index, <italic>S</italic><sub><italic>T<sub>i</sub></italic></sub>, takes both individual contributions and interaction effects into account [<xref ref-type="bibr" rid="RSIF20121018C7">7</xref>,<xref ref-type="bibr" rid="RSIF20121018C38">38</xref>]. An exploration of the interaction effects was outside the scope of our application of the Sobol’ index, and thus the total effect index was not calculated.</p><p>The temporal variation of state variable sensitivity to parameters is rarely addressed in infectious disease modelling studies. Modellers should explore this temporal effect, as their conclusions about the importance of particular parameters, or the robustness of their findings, may be mistaken if sensitivity is examined only at one time point. If, for instance, sensitivity is evaluated at a point following the epidemic peak, parameters that were instrumental to the rising epidemic curve may appear irrelevant. The Morris method can provide some information about temporal variation in a single parameter's sensitivity over the simulation period, being as the index is based on the ratio of the change of an output to the change of an input parameter, the latter of which can be kept constant over the period calculated. However, the index is sensitive to the magnitude of the state variables over the course of the simulation, and thus must be interpreted with care when examined over time.</p><p>By contrast, the Sobol’ method cannot provide a meaningful comparison of the change in sensitivity of an output variable to a parameter over the course of simulation. The Sobol’ sensitivity index for a particular parameter represents its contribution to the total variance of the system at the time point evaluated. If examined over time, a change in the index may be related to a change in the importance of the parameter or a change in the output variance, and these two possibilities cannot be distinguished. Likewise, the PRCC method does not allow for quantitative comparison of a single parameter's influence at two time points, because the calculation ranks parameters and state variables at a point in time; the rank, residual of a rank and the resulting PRCC of a particular parameter at one time point has no direct relationship to those at another time point. Modellers should thus be cautious about applying variance decomposition-based methods to quantitatively compare model sensitivity at multiple time points.</p><p>The SHM method, and the associated graphical objects SHM and PSS, offer considerable benefits over traditional techniques. This method efficiently handles models with high-dimensional parameter spaces and many state variables. SHM offers a powerful view of state variable sensitivity over time to many parameters taken together [<xref ref-type="bibr" rid="RSIF20121018C11">11</xref>]. The sensitivity strips clearly show the time period when a state variable is sensitive to model parameters as a whole, information that traditional methods do not provide. Such an examination of sensitivity over time is of particular value to the modeller interested in, say, the change in timing of particular events occurring over the course of the epidemic. PSS, on the other hand, summarizes the importance of individual parameters when considering all, or a set of, model state variables. This view may be of particular use in agent-based or network transmission models, where modellers are most interested in how parameters influence the model system as a whole, rather than the sensitivity of a specific state variable as defined for a particular network node or agent [<xref ref-type="bibr" rid="RSIF20121018C21">21</xref>,<xref ref-type="bibr" rid="RSIF20121018C43">43</xref>].</p><p>Alternatively, the simultaneous sensitivity of several state variables (e.g. infectious state variables of the cholera model, <italic>I</italic>(<italic>t</italic>), <italic>B</italic><sub>H</sub>(<italic>t</italic>) and <italic>B</italic><sub>L</sub>(<italic>t</italic>)), to a single parameter may also be of interest. Since the index for each parameter is constant, PSS avoids the complicated application of traditional SA methods at a single time point. When a model has only a few state variables and parameters, the strengths of SHM methods still hold as shown by SA of the schistosomiasis model with only two state variables. By comparison of figures <xref ref-type="fig" rid="RSIF20121018F5">5</xref> and <xref ref-type="fig" rid="RSIF20121018F6">6</xref>, the same parameters were generally identified as important in the schistosomiasis model by both SHM methods and the traditional methods. However, the former more flexibly examines the sensitivity associated with different parameter or output subsets and provides more information about sensitivity than the latter. To our knowledge, the SHM method has not yet been applied to infectious disease models, and future work should consider the insights these powerful techniques can provide on both autonomous and non-autonomous systems [<xref ref-type="bibr" rid="RSIF20121018C44">44</xref>].</p><p>In the development and evaluation of infectious disease models, our results can inform the selection of SA methods for aims such as model refinement, understanding model sensitivity over time, apportioning sensitivity across complex parameter spaces and intervention evaluation. If model refinement through improved parameter estimation is the research objective, PSS can be used to show the relative importance of each parameter over the period of simulation. The approach is particularly valuable if such model refinement is aimed at a time-specific goal (e.g. improved estimation of the peak population time). After the most critical parameters have been identified through these means, the feasibility of their refinement through additional data collection or experimentation can subsequently be determined. A modeller interested in examining model sensitivity at one point in time to parameters and their interactions may choose the Sobol’ method, whose chief advantage is the ability to tease out the model's response to a single parameter when all interactions are accounted for [<xref ref-type="bibr" rid="RSIF20121018C45">45</xref>]. This apportioning of output sensitivity to a specific parameter may be useful when, for instance, multiple interventions are compared by examining the output's response to each altered parameter as separate first-order effects, and as total effects including the parameter's interactions [<xref ref-type="bibr" rid="RSIF20121018C46">46</xref>]. On the other hand, a modeller interested in identifying when a parameter achieves maximum influence may benefit from the Morris method, which can provide information about the variation of sensitivity over the simulation interval. To our knowledge, the Morris method has not been applied in this fashion, and we hope to prompt its application in such infectious disease modelling contexts.</p><p>If a modeller aims to achieve specific goals defined as criteria on output variables, such as limiting the size of the infected population; the targeting of specific parameters that impact goal achievement are of interest, and PRCC can be particularly valuable [<xref ref-type="bibr" rid="RSIF20121018C21">21</xref>] and has been applied to several infectious disease models in this fashion [<xref ref-type="bibr" rid="RSIF20121018C26">26</xref>–<xref ref-type="bibr" rid="RSIF20121018C28">28</xref>,<xref ref-type="bibr" rid="RSIF20121018C32">32</xref>]. Other analyses have highlighted, through SA methods such as PRCC, the most effective times or parameters for efficient intervention [<xref ref-type="bibr" rid="RSIF20121018C9">9</xref>,<xref ref-type="bibr" rid="RSIF20121018C10">10</xref>], or the need for additional experimental data to aid research (e.g. estimates for host mixing rates [<xref ref-type="bibr" rid="RSIF20121018C8">8</xref>]). Since PRCC indicates the degree of monotonicity between an output variable and a specific parameter in a model, preliminary examination the behaviour of the model is necessary [<xref ref-type="bibr" rid="RSIF20121018C27">27</xref>]. Traditional scatter plots roughly screen the relationships between model outputs and parameters and may be used for a preliminary assessment. Otherwise, more powerful techniques, such as flow-based scatter plots, may be more suitable for the complexity typical of infectious disease models [<xref ref-type="bibr" rid="RSIF20121018C47">47</xref>].</p><p>In resource poor settings, models, coupled with SA, may be used as a first step towards understanding control options. Whether by means of modelling behavioural interventions, biomedical interventions or through evaluation of social conditions and their impact on the disease system, infectious disease models have been used by decision-making bodies to explore and evaluate various policy options [<xref ref-type="bibr" rid="RSIF20121018C48">48</xref>]. Where data are meager, SA can be used to identify key parameters upon which to focus limited data collection resources. Alternatively, an identified sensitive parameter may highlight a mechanism in the modelled system most suitable for cost-effective intervention. Prioritized parameters can also inform more efficient uncertainty analysis; assumed variation in only the most important parameters could be investigated to simulate the range of probable outcomes. GSA has also been used formally to aid in experimental design, to determine which parameters to focus upon when resources are scarce, and this has been achieved by analysing gradients of the model log-likelihood surface with respect to previously collected data [<xref ref-type="bibr" rid="RSIF20121018C11">11</xref>]. This approach may lead to a fertile partnering of parameter estimation and SA methods, for model/data pairs with complex likelihoods.</p><p>Computation time is a final important point of comparison between SA methods. The SHM and PSS methods exhibit highly optimized and efficient computation [<xref ref-type="bibr" rid="RSIF20121018C11">11</xref>], while the computational requirements of traditional methods depend on the number of model runs, which itself is contingent on the number of parameters and how exhaustive the parameter space is sampled. The computational cost of the Sobol’ method is the highest among the four traditional SA methods, and the more model runs undertaken, the more accurate the Sobol’ index is [<xref ref-type="bibr" rid="RSIF20121018C7">7</xref>,<xref ref-type="bibr" rid="RSIF20121018C38">38</xref>].</p><p>We conclude by pointing out interesting insights that emerge from a comparison of the terms that appear in <italic>R</italic><sub>0</sub> for the cholera model, and the relative importance of these and other parameters based on the SA findings. Examination of <italic>R</italic><sub>0</sub> (equation (5.1)) produced by the next generation method [<xref ref-type="bibr" rid="RSIF20121018C49">49</xref>] indicates that <italic>K</italic><sub>L</sub>, <italic>δ</italic>, <italic>γ</italic>, <italic>p</italic>, <italic>ξ</italic><sub>S</sub>, <italic>ξ</italic><sub>A</sub>, <italic>w</italic> and <italic>λ</italic>, (<italic>p3</italic>, <italic>p5</italic>, <italic>p8</italic>, <italic>p10</italic>, <italic>p11</italic>, <italic>p12</italic>, <italic>p15</italic>) are potentially influential terms.<disp-formula id="RSIF20121018M51"><label>5.1</label><graphic xlink:href="rsif20121018eq51"></graphic></disp-formula></p><p>The SA results indicate that <italic>p1</italic>, <italic>p8</italic>, <italic>p9</italic> and <italic>p12</italic> are important parameters and the remaining parameters have limited influence. Thus, SA has effectively narrowed the set of parameters worthy of additional scrutiny when compared with the qualitative conclusions a modeller might have reached based on a simple examination of <italic>R</italic><sub>0</sub>. Of course, at times, <italic>R</italic><sub>0</sub> is the model output of interest and is itself subjected to SA [<xref ref-type="bibr" rid="RSIF20121018C9">9</xref>].</p><p>By comparing the application of five GSA methods to two infectious disease models, we have explored the strengths, weaknesses and appropriate uses of GSA methods in the context of infectious disease models, and our assessments are summarized in <xref ref-type="table" rid="RSIF20121018TB3">table 3</xref>. While we found the LHS-PRCC and the SHM method to be the most widely applicable to infectious disease models, all five SA methods explored here can support improvements in model performance and design, if the method selected is consonant with the modeller's goals. As the acquisition of data on dynamic disease transmission phenomena expands, and the demand for models to analyse and interpret these data increases, modellers need to quickly understand what the main ‘levers’ of influence are in models that potentially contain dozens or even hundreds of parameters. Through comparison of the research questions being addressed, the model's structure and the characteristics of the SA methods investigated here, modellers can make informed SA choices that support their research aims and advance the productive application of mathematical models to infectious disease systems.
<table-wrap id="RSIF20121018TB3" position="float"><label>Table 3.</label><caption><p>Comparison of five GSA methods.</p></caption><table frame="hsides" rules="groups"><colgroup><col align="left"></col><col align="left"></col><col align="left"></col><col align="left"></col><col align="left"></col><col align="left"></col></colgroup><thead valign="top"><tr><th></th><th align="left">scatter plots</th><th align="left">the Morris method</th><th align="left">LHS-PRCC</th><th align="left">the Sobol’ method</th><th align="left">the SHM method</th></tr></thead><tbody><tr><td>How easily understood is the concept behind the method?</td><td>easily</td><td>easily</td><td>medium</td><td>medium</td><td>difficult</td></tr><tr><td>Can the method rank the importance of parameters?</td><td>no</td><td>yes</td><td>yes</td><td>yes</td><td>yes</td></tr><tr><td>Can the method measure the variation of sensitivity over time?</td><td>no</td><td>yes</td><td>no</td><td>no</td><td>yes</td></tr><tr><td>Can the method show the sensitivity of the full model system?</td><td>no</td><td>no</td><td>no</td><td>no</td><td>yes</td></tr><tr><td>Can the method show the sensitivity to all parameters simultaneously?</td><td>no</td><td>no</td><td>no</td><td>no</td><td>yes</td></tr><tr><td>Can the method attribute the variance of a model output to each parameter?</td><td>no</td><td>no</td><td>no</td><td>yes</td><td>no</td></tr><tr><td>Can the method show the interaction effects on the sensitivity between parameters?</td><td>no</td><td>no</td><td>no</td><td>yes</td><td>no</td></tr><tr><td>Can the method show the direction of sensitivity?<sup>a</sup></td><td>yes</td><td>no</td><td>yes</td><td>no</td><td>no</td></tr><tr><td>Is the method easy to carry out? (e.g. difficulty to calculate and present results)</td><td>easy</td><td>medium</td><td>slightly difficult</td><td>slightly difficult</td><td>easy<sup>b</sup></td></tr><tr><td>What is the relative computation cost?</td><td>low</td><td>medium</td><td>medium</td><td>high</td><td>low</td></tr></tbody></table><table-wrap-foot><fn><p><sup>a</sup>The direction of sensitivity refers to whether the change in parameters and the change in model outputs are concordant or discordant.</p></fn><fn><p><sup>b</sup>Execution is greatly facilitated by the SASSY Matlab package.</p></fn></table-wrap-foot></table-wrap></p></sec></body><back><ack><title>Acknowledgements</title><p>This manuscript was considerably strengthened by the critical remarks of four anonymous referees. Thanks, also, to Chloe Robbins and the rest of the Remais Lab. This work was supported in part by the Ecology of Infectious Disease program of the National Science Foundation under grant no. 0622743, by the Chemical, Bioengineering, Environmental, and Transport Systems Division of the National Science Foundation under grant no. 1249250, by the National Institute for Allergy and Infectious Disease (K01AI091864) and by the Global Health Institute at Emory University. The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.</p></ack><ref-list><title>References</title><ref id="RSIF20121018C1"><label>1</label><citation citation-type="book"><person-group person-group-type="author"><name><surname>Anderson</surname><given-names>RM</given-names></name><name><surname>May</surname><given-names>RM</given-names></name></person-group>. <year>1991</year> <source>Infectious diseases of humans: dynamics and control</source>. <publisher-loc>Oxford, UK</publisher-loc>: <publisher-name>Oxford University Press</publisher-name>.</citation></ref><ref id="RSIF20121018C2"><label>2</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Grassly</surname><given-names>NC</given-names></name><name><surname>Fraser</surname><given-names>C</given-names></name></person-group>. <year>2008</year> <article-title>Mathematical models of infectious disease transmission</article-title>. <source>Nat. Rev. Microbiol.</source> <volume>6</volume>, <fpage>477</fpage>–<lpage>487</lpage>.</citation></ref><ref id="RSIF20121018C3"><label>3</label><citation citation-type="book"><person-group person-group-type="author"><name><surname>Keeling</surname><given-names>M</given-names></name><name><surname>Rohani</surname><given-names>P</given-names></name></person-group>. <year>2007</year> <source>Modeling infectious diseases in human and animals</source>. <publisher-loc>Princeton, NJ</publisher-loc>: <publisher-name>Princeton University Press</publisher-name>.</citation></ref><ref id="RSIF20121018C4"><label>4</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Hooker</surname><given-names>G</given-names></name><name><surname>Ellner</surname><given-names>SP</given-names></name><name><surname>Roditi</surname><given-names>LD</given-names></name><name><surname>Earn</surname><given-names>DJD</given-names></name></person-group>. <year>2011</year> <article-title>Parameterizing state-space models for infectious disease dynamics by generalized profiling: measles in Ontario</article-title>. <source>J. R. Soc. Interface</source> <volume>8</volume>, <fpage>961</fpage>–<lpage>974</lpage>. (<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1098/rsif.2010.0412">doi:10.1098/rsif.2010.0412</ext-link>)</citation></ref><ref id="RSIF20121018C5"><label>5</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Okais</surname><given-names>C</given-names></name><name><surname>Roche</surname><given-names>S</given-names></name><name><surname>Kurzinger</surname><given-names>ML</given-names></name><name><surname>Riche</surname><given-names>B</given-names></name><name><surname>Bricout</surname><given-names>H</given-names></name><name><surname>Derrough</surname><given-names>T</given-names></name><name><surname>Simondon</surname><given-names>F</given-names></name><name><surname>Ecochard</surname><given-names>R.</given-names></name></person-group> <year>2010</year> <article-title>Methodology of the sensitivity analysis used for modeling an infectious disease</article-title>. <source>Vaccine</source> <volume>28</volume>, <fpage>8132</fpage>–<lpage>8140</lpage>. (<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1016/j.vaccine.2010.09.099">doi:10.1016/j.vaccine.2010.09.099</ext-link>)</citation></ref><ref id="RSIF20121018C6"><label>6</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Moore</surname><given-names>JL</given-names></name><name><surname>Liang</surname><given-names>S</given-names></name><name><surname>Akullian</surname><given-names>A</given-names></name><name><surname>Remais</surname><given-names>JV</given-names></name></person-group>. <year>2012</year> <article-title>Cautioning the use of degree-day models for climate change projections in the presence of parametric uncertainty</article-title>. <source>Ecol. Appl.</source> <volume>22</volume>, <fpage>2237</fpage>–<lpage>2247</lpage>. (<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1890/12-0127.1">doi:10.1890/12-0127.1</ext-link>)</citation></ref><ref id="RSIF20121018C7"><label>7</label><citation citation-type="book"><person-group person-group-type="author"><name><surname>Saltelli</surname><given-names>A</given-names></name><name><surname>Ratto</surname><given-names>M</given-names></name><name><surname>Andres</surname><given-names>T</given-names></name><name><surname>Campolongo</surname><given-names>F</given-names></name><name><surname>Cariboni</surname><given-names>J</given-names></name><name><surname>Gatelli</surname><given-names>D</given-names></name><name><surname>Saisana</surname><given-names>M</given-names></name><name><surname>Tarantola</surname><given-names>S</given-names></name></person-group>. <year>2008</year> <source>Global sensitivity analysis. The primer</source>. <publisher-loc>Chichester, UK</publisher-loc>: <publisher-name>John Wiley & Sons</publisher-name>.</citation></ref><ref id="RSIF20121018C8"><label>8</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Buhnerkempe</surname><given-names>MG</given-names></name><name><surname>Eisen</surname><given-names>RJ</given-names></name><name><surname>Goodell</surname><given-names>B</given-names></name><name><surname>Gage</surname><given-names>KL</given-names></name><name><surname>Antolin</surname><given-names>MF</given-names></name><name><surname>Webb</surname><given-names>CT</given-names></name></person-group>. <year>2011</year> <article-title>Transmission shifts underlie variability in population responses to Yersinia pestis infection</article-title>. <source>PLoS ONE</source> <volume>6</volume>, <fpage>e22498</fpage>. (<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1371/journal.pone.0022498">doi:10.1371/journal.pone.0022498</ext-link>)</citation></ref><ref id="RSIF20121018C9"><label>9</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Chitnis</surname><given-names>N</given-names></name><name><surname>Hyman</surname><given-names>JM</given-names></name><name><surname>Cushing</surname><given-names>JM</given-names></name></person-group>. <year>2008</year> <article-title>Determining important parameters in the spread of malaria through the sensitivity analysis of a mathematical model</article-title>. <source>Bull. Math. Biol.</source> <volume>70</volume>, <fpage>1272</fpage>–<lpage>1296</lpage>. (<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1007/s11538-008-9299-0">doi:10.1007/s11538-008-9299-0</ext-link>)</citation></ref><ref id="RSIF20121018C10"><label>10</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>McLeod</surname><given-names>RG</given-names></name><name><surname>Brewster</surname><given-names>JF</given-names></name><name><surname>Gumel</surname><given-names>AB</given-names></name><name><surname>Slonowsky</surname><given-names>DA</given-names></name></person-group>. <year>2006</year> <article-title>Sensitivity and uncertainty analyses for a SARS model with time-varying inputs and outputs</article-title>. <source>Math. Biosci. Eng.</source> <volume>3</volume>, <fpage>527</fpage>–<lpage>544</lpage>. (<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.3934/mbe.2006.3.527">doi:10.3934/mbe.2006.3.527</ext-link>)</citation></ref><ref id="RSIF20121018C11"><label>11</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Rand</surname><given-names>DA</given-names></name></person-group>. <year>2009</year> <article-title>Mapping global sensitivity of cellular network dynamics: sensitivity heat maps and a global summation law</article-title>. <source>J. R. Soc. Interface</source> <volume>5</volume>, <fpage>S59</fpage>–<lpage>S69</lpage>. (<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1098/rsif.2008.0084.focus">doi:10.1098/rsif.2008.0084.focus</ext-link>)</citation></ref><ref id="RSIF20121018C12"><label>12</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Benton</surname><given-names>TG</given-names></name><name><surname>Grant</surname><given-names>A</given-names></name></person-group>. <year>1999</year> <article-title>Elasticity analysis as an important tool in evolutionary and population ecology</article-title>. <source>Trends Ecol. Evol.</source> <volume>14</volume>, <fpage>467</fpage>–<lpage>471</lpage>. (<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1016/S0169-5347(99)01724-3">doi:10.1016/S0169-5347(99)01724-3</ext-link>)</citation></ref><ref id="RSIF20121018C13"><label>13</label><citation citation-type="book"><person-group person-group-type="author"><name><surname>Samuelson</surname><given-names>PA</given-names></name><name><surname>Nordhaus</surname><given-names>WD</given-names></name></person-group>. <year>2010</year> <source>Microeconomics</source>, <edition>19th edn</edition>. <publisher-loc>Boston, MA</publisher-loc>: <publisher-name>McGraw-Hill Irwin</publisher-name>.</citation></ref><ref id="RSIF20121018C14"><label>14</label><citation citation-type="book"><person-group person-group-type="author"><name><surname>Myers</surname><given-names>RH</given-names></name><name><surname>Montgomery</surname><given-names>DC</given-names></name><name><surname>Anderson-Cook</surname><given-names>CM</given-names></name></person-group>. <year>2009</year> <source>Response surface methodology: process and product optimization using designed experiments</source>, <edition>3rd edn</edition>. <publisher-loc>Hoboken, NJ</publisher-loc>: <publisher-name>Wiley</publisher-name>.</citation></ref><ref id="RSIF20121018C15"><label>15</label><citation citation-type="book"><person-group person-group-type="author"><name><surname>Montgomery</surname><given-names>DC</given-names></name></person-group>. <year>2013</year> <source>Design and analysis of experiments</source>, <edition>8th edn</edition>. <publisher-loc>Hoboken, NJ</publisher-loc>: <publisher-name>John Wiley & Sons, Inc</publisher-name>.</citation></ref><ref id="RSIF20121018C16"><label>16</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Sobol</surname><given-names>IM</given-names></name></person-group>. <year>1990</year> <article-title>On sensitivity estimates for nonlinear mathematical models</article-title>. <source>Matematicheskoe Modelirovanie</source> <volume>2</volume>, <fpage>112</fpage>–<lpage>118</lpage>. [<comment>In Russian</comment>.]</citation></ref><ref id="RSIF20121018C17"><label>17</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Saltelli</surname><given-names>A</given-names></name><name><surname>Tarantola</surname><given-names>S</given-names></name><name><surname>Chan</surname><given-names>KPS</given-names></name></person-group>. <year>1999</year> <article-title>A quantitative model-independent method for global sensitivity analysis of model output</article-title>. <source>Technometrics</source> <volume>41</volume>, <fpage>39</fpage>–<lpage>56</lpage>. (<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1080/00401706.1999.10485594">doi:10.1080/00401706.1999.10485594</ext-link>)</citation></ref><ref id="RSIF20121018C18"><label>18</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Morris</surname><given-names>MD</given-names></name></person-group>. <year>1991</year> <article-title>Factorial sampling plans for preliminary computational experiments</article-title>. <source>Technometrics</source> <volume>33</volume>, <fpage>161</fpage>–<lpage>174</lpage>. (<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1080/00401706.1991.10484804">doi:10.1080/00401706.1991.10484804</ext-link>)</citation></ref><ref id="RSIF20121018C19"><label>19</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Campolongo</surname><given-names>F</given-names></name><name><surname>Cariboni</surname><given-names>J</given-names></name><name><surname>Saltelli</surname><given-names>A</given-names></name></person-group>. <year>2007</year> <article-title>An effective screening design for sensitivity analysis of large models</article-title>. <source>Environ. Model. Softw.</source> <volume>22</volume>, <fpage>1509</fpage>–<lpage>1518</lpage>. (<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1016/j.envsoft.2006.10.004">doi:10.1016/j.envsoft.2006.10.004</ext-link>)</citation></ref><ref id="RSIF20121018C20"><label>20</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Helton</surname><given-names>JC</given-names></name><name><surname>Davis</surname><given-names>FJ</given-names></name></person-group>. <year>2002</year> <article-title>Illustration of sampling-based methods for uncertainty and sensitivity analysis</article-title>. <source>Risk Anal.</source> <volume>22</volume>, <fpage>591</fpage>–<lpage>622</lpage>. (<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1111/0272-4332.00041">doi:10.1111/0272-4332.00041</ext-link>)</citation></ref><ref id="RSIF20121018C21"><label>21</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Marino</surname><given-names>S</given-names></name><name><surname>Hogue</surname><given-names>IB</given-names></name><name><surname>Ray</surname><given-names>CJ</given-names></name><name><surname>Kirschner</surname><given-names>DE</given-names></name></person-group>. <year>2008</year> <article-title>A methodology for performing global uncertainty and sensitivity analysis in systems biology</article-title>. <source>J. Theor. Biol.</source> <volume>254</volume>, <fpage>178</fpage>–<lpage>196</lpage>. (<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1016/j.jtbi.2008.04.011">doi:10.1016/j.jtbi.2008.04.011</ext-link>)</citation></ref><ref id="RSIF20121018C22"><label>22</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Hamby</surname><given-names>DM</given-names></name></person-group>. <year>1994</year> <article-title>A review of techniques for parameter sensitivity analysis of environmental-models</article-title>. <source>Environ. Monit. Assess.</source> <volume>32</volume>, <fpage>135</fpage>–<lpage>154</lpage>. (<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1007/BF00547132">doi:10.1007/BF00547132</ext-link>)</citation></ref><ref id="RSIF20121018C23"><label>23</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Yang</surname><given-names>J</given-names></name></person-group>. <year>2011</year> <article-title>Convergence and uncertainty analyses in Monte-Carlo based sensitivity analysis</article-title>. <source>Environ. Model. Softw.</source> <volume>26</volume>, <fpage>444</fpage>–<lpage>457</lpage>. (<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1016/j.envsoft.2010.10.007">doi:10.1016/j.envsoft.2010.10.007</ext-link>)</citation></ref><ref id="RSIF20121018C24"><label>24</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Andrews</surname><given-names>JR</given-names></name><name><surname>Basu</surname><given-names>S</given-names></name></person-group>. <year>2011</year> <article-title>Transmission dynamics and control of cholera in Haiti: an epidemic model</article-title>. <source>Lancet</source> <volume>377</volume>, <fpage>1248</fpage>–<lpage>1255</lpage>. (<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1016/S0140-6736(11)60273-0">doi:10.1016/S0140-6736(11)60273-0</ext-link>)</citation></ref><ref id="RSIF20121018C25"><label>25</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Zhang</surname><given-names>J</given-names></name><name><surname>Jin</surname><given-names>Z</given-names></name><name><surname>Sun</surname><given-names>G</given-names></name><name><surname>Zhou</surname><given-names>T</given-names></name><name><surname>Ruan</surname><given-names>S</given-names></name></person-group>. <year>2011</year> <article-title>Analysis of Rabies in China: transmission dynamics and control</article-title>. <source>PLoS ONE</source> <volume>6</volume>, e20891. (<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1371/journal.pone.0020891">doi:10.1371/journal.pone.0020891</ext-link>)</citation></ref><ref id="RSIF20121018C26"><label>26</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Gubbins</surname><given-names>S</given-names></name><name><surname>Carpenter</surname><given-names>S</given-names></name><name><surname>Baylis</surname><given-names>M</given-names></name><name><surname>Wood</surname><given-names>JLN</given-names></name><name><surname>Mellor</surname><given-names>PS</given-names></name></person-group>. <year>2008</year> <article-title>Assessing the risk of bluetongue to UK livestock: uncertainty and sensitivity analyses of a temperature-dependent model for the basic reproduction number</article-title>. <source>J. R. Soc. Interface</source> <volume>5</volume>, <fpage>363</fpage>–<lpage>371</lpage>. (<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1098/rsif.2007.1110">doi:10.1098/rsif.2007.1110</ext-link>)</citation></ref><ref id="RSIF20121018C27"><label>27</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Blower</surname><given-names>SM</given-names></name><name><surname>Dowlatabadi</surname><given-names>H</given-names></name></person-group>. <year>1994</year> <article-title>Sensitivity and uncertainty analysis of complex-models of disease transmission—an HIV model, as an example</article-title>. <source>Int. Stat. Rev.</source> <volume>62</volume>, <fpage>229</fpage>–<lpage>243</lpage>. (<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.2307/1403510">doi:10.2307/1403510</ext-link>)</citation></ref><ref id="RSIF20121018C28"><label>28</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Porco</surname><given-names>TC</given-names></name><name><surname>Blower</surname><given-names>SM</given-names></name></person-group>. <year>1998</year> <article-title>Quantifying the intrinsic transmission dynamics of tuberculosis</article-title>. <source>Theor. Popul. Biol.</source> <volume>54</volume>, <fpage>117</fpage>–<lpage>132</lpage>. (<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1006/tpbi.1998.1366">doi:10.1006/tpbi.1998.1366</ext-link>)</citation></ref><ref id="RSIF20121018C29"><label>29</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Ellis</surname><given-names>AM</given-names></name><name><surname>Garcia</surname><given-names>AJ</given-names></name><name><surname>Focks</surname><given-names>DA</given-names></name><name><surname>Morrison</surname><given-names>AC</given-names></name><name><surname>Scott</surname><given-names>TW</given-names></name></person-group>. <year>2011</year> <article-title>Parameterization and sensitivity analysis of a complex simulation model for mosquito population dynamics, dengue transmission, and their control</article-title>. <source>Am. J. Trop. Med. Hyg.</source> <volume>85</volume>, <fpage>257</fpage>–<lpage>264</lpage>. (<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.4269/ajtmh.2011.10-0516">doi:10.4269/ajtmh.2011.10-0516</ext-link>)</citation></ref><ref id="RSIF20121018C30"><label>30</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Bailey</surname><given-names>NTJ</given-names></name><name><surname>Duppenthaler</surname><given-names>J</given-names></name></person-group>. <year>1980</year> <article-title>Sensitivity analysis in the modeling of infectious-disease dynamics</article-title>. <source>J. Math. Biol.</source> <volume>10</volume>, <fpage>113</fpage>–<lpage>131</lpage>. (<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1007/BF00275837">doi:10.1007/BF00275837</ext-link>)</citation></ref><ref id="RSIF20121018C31"><label>31</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Hethcote</surname><given-names>HW</given-names></name><name><surname>Vanark</surname><given-names>JW</given-names></name><name><surname>Karon</surname><given-names>JM</given-names></name></person-group>. <year>1991</year> <article-title>A simulation-model of AIDS in San Francisco. 2. Simulations, therapy, and sensitivity analysis</article-title>. <source>Math. Biosci.</source> <volume>106</volume>, <fpage>223</fpage>–<lpage>247</lpage>. (<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1016/0025-5564(91)90078-W">doi:10.1016/0025-5564(91)90078-W</ext-link>)</citation></ref><ref id="RSIF20121018C32"><label>32</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Legrand</surname><given-names>J</given-names></name><name><surname>Sanchez</surname><given-names>A</given-names></name><name><surname>Le Pont</surname><given-names>F</given-names></name><name><surname>Camacho</surname><given-names>L</given-names></name><name><surname>Larouze</surname><given-names>B</given-names></name></person-group>. <year>2008</year> <article-title>Modeling the impact of tuberculosis control strategies in highly endemic overcrowded prisons</article-title>. <source>PLoS ONE</source> <volume>3</volume>, <fpage>e2100</fpage>. (<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1371/journal.pone.0002100">doi:10.1371/journal.pone.0002100</ext-link>)</citation></ref><ref id="RSIF20121018C33"><label>33</label><citation citation-type="book"><person-group person-group-type="author"><name><surname>Saltelli</surname><given-names>A</given-names></name><name><surname>Tarantola</surname><given-names>S</given-names></name><name><surname>Campolongo</surname><given-names>F</given-names></name><name><surname>Ratto</surname><given-names>M</given-names></name></person-group>. <year>2004</year> <source>Sensitivity analysis in practice: a guide to assessing scientific models</source>. <publisher-name>Chichester, UK: John Wiley & Sons</publisher-name>.</citation></ref><ref id="RSIF20121018C34"><label>34</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Hamby</surname><given-names>DM</given-names></name></person-group>. <year>1995</year> <article-title>A comparison of sensitivity analysis techniques</article-title>. <source>Health Phys.</source> <volume>68</volume>, <fpage>195</fpage>–<lpage>204</lpage>. (<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1097/00004032-199502000-00005">doi:10.1097/00004032-199502000-00005</ext-link>)</citation></ref><ref id="RSIF20121018C35"><label>35</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>McKay</surname><given-names>MD</given-names></name><name><surname>Beckman</surname><given-names>RJ</given-names></name><name><surname>Conover</surname><given-names>WJ</given-names></name></person-group>. <year>1979</year> <article-title>A comparison of three methods for selecting values of input variables in the analysis of output from a computer code</article-title>. <source>Technometrics</source> <volume>21</volume>, <fpage>239</fpage>–<lpage>245</lpage>.</citation></ref><ref id="RSIF20121018C36"><label>36</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Helton</surname><given-names>JC</given-names></name><name><surname>Davis</surname><given-names>FJ</given-names></name></person-group>. <year>2003</year> <article-title>Latin hypercube sampling and the propagation of uncertainty in analyses of complex systems</article-title>. <source>Reliab. Eng. Syst. Safety</source> <volume>81</volume>, <fpage>23</fpage>–<lpage>69</lpage>. (<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1016/S0951-8320(03)00058-9">doi:10.1016/S0951-8320(03)00058-9</ext-link>)</citation></ref><ref id="RSIF20121018C37"><label>37</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Sobol</surname><given-names>IM</given-names></name></person-group>. <year>2001</year> <article-title>Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates</article-title>. <source>Math. Comp. Simul.</source> <volume>55</volume>, <fpage>271</fpage>–<lpage>280</lpage>. (<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1016/S0378-4754(00)00270-6">doi:10.1016/S0378-4754(00)00270-6</ext-link>)</citation></ref><ref id="RSIF20121018C38"><label>38</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Saltelli</surname><given-names>A</given-names></name><name><surname>Annoni</surname><given-names>P</given-names></name><name><surname>Azzini</surname><given-names>I</given-names></name><name><surname>Campolongo</surname><given-names>F</given-names></name><name><surname>Ratto</surname><given-names>M</given-names></name><name><surname>Tarantola</surname><given-names>S</given-names></name></person-group>. <year>2010</year> <article-title>Variance based sensitivity analysis of model output. Design and estimator for the total sensitivity index</article-title>. <source>Comp. Phys. Commun.</source> <volume>181</volume>, <fpage>259</fpage>–<lpage>270</lpage>. (<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1016/j.cpc.2009.09.018">doi:10.1016/j.cpc.2009.09.018</ext-link>)</citation></ref><ref id="RSIF20121018C39"><label>39</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Domijan</surname><given-names>M</given-names></name><name><surname>Rand</surname><given-names>DA</given-names></name></person-group>. <year>2011</year> <article-title>Balance equations can buffer noisy and sustained environmental perturbations of circadian clocks</article-title>. <source>Interface Focus</source> <volume>1</volume>, <fpage>177</fpage>–<lpage>186</lpage>. (<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1098/rsfs.2010.0007">doi:10.1098/rsfs.2010.0007</ext-link>)</citation></ref><ref id="RSIF20121018C40"><label>40</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Liang</surname><given-names>S</given-names></name><name><surname>Maszle</surname><given-names>D</given-names></name><name><surname>Spear</surname><given-names>RC</given-names></name></person-group>. <year>2002</year> <article-title>A quantitative framework for a multi-group model of <italic>Schistosomiasis japonicum</italic> transmission dynamics and control in Sichuan, China</article-title>. <source>Acta Trop.</source> <volume>82</volume>, <fpage>263</fpage>–<lpage>277</lpage>. (<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1016/S0001-706X(02)00018-9">doi:10.1016/S0001-706X(02)00018-9</ext-link>)</citation></ref><ref id="RSIF20121018C41"><label>41</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Liang</surname><given-names>S</given-names></name><etal></etal></person-group> <year>2007</year> <article-title>Environmental effects on parasitic disease transmission exemplified by schistosomiasis in western China</article-title>. <source>Proc. Natl Acad. Sci. USA</source> <volume>104</volume>, <fpage>7110</fpage>–<lpage>7115</lpage>. (<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1073/pnas.0701878104">doi:10.1073/pnas.0701878104</ext-link>)</citation></ref><ref id="RSIF20121018C42"><label>42</label><citation citation-type="other"><person-group person-group-type="author"><name><surname>Brown</surname><given-names>P</given-names></name><name><surname>Rand</surname><given-names>DA</given-names></name></person-group>. <year>2012</year> <comment>Using the Global Sensitivity GUIs</comment>. See <comment><uri xlink:href="http://wsbcwarwickacuk/software/indexhtml">http://wsbcwarwickacuk/software/indexhtml</uri></comment>.</citation></ref><ref id="RSIF20121018C43"><label>43</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Segovia-Juarez</surname><given-names>JL</given-names></name><name><surname>Ganguli</surname><given-names>S</given-names></name><name><surname>Kirschner</surname><given-names>D</given-names></name></person-group>. <year>2004</year> <article-title>Identifying control mechanisms of granuloma formation during M-tuberculosis infection using an agent-based model</article-title>. <source>J. Theor. Biol.</source> <volume>231</volume>, <fpage>357</fpage>–<lpage>376</lpage>. (<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1016/j.jtbi.2004.06.031">doi:10.1016/j.jtbi.2004.06.031</ext-link>)</citation></ref><ref id="RSIF20121018C44"><label>44</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Remais</surname><given-names>J</given-names></name><name><surname>Zhong</surname><given-names>B</given-names></name><name><surname>Carlton</surname><given-names>EJ</given-names></name><name><surname>Spear</surname><given-names>RC</given-names></name></person-group>. <year>2009</year> <article-title>Model approaches for estimating the influence of time-varying socio-environmental factors on macroparasite transmission in two endemic regions</article-title>. <source>Epidemics</source> <volume>1</volume>, <fpage>213</fpage>–<lpage>220</lpage>. (<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1016/j.epidem.2009.10.001">doi:10.1016/j.epidem.2009.10.001</ext-link>)</citation></ref><ref id="RSIF20121018C45"><label>45</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Mokhtari</surname><given-names>A</given-names></name><name><surname>Frey</surname><given-names>HC</given-names></name></person-group>. <year>2005</year> <article-title>Sensitivity analysis of a two-dimensional probabilistic risk assessment model using analysis of variance</article-title>. <source>Risk Anal.</source> <volume>25</volume>, <fpage>1511</fpage>–<lpage>1529</lpage>. (<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1111/j.1539-6924.2005.00679.x">doi:10.1111/j.1539-6924.2005.00679.x</ext-link>)</citation></ref><ref id="RSIF20121018C46"><label>46</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Busschaert</surname><given-names>P</given-names></name><name><surname>Geeraerd</surname><given-names>AH</given-names></name><name><surname>Uyttendaele</surname><given-names>M</given-names></name><name><surname>Van Impe</surname><given-names>JF</given-names></name></person-group>. <year>2011</year> <article-title>Sensitivity analysis of a two-dimensional quantitative microbiological risk assessment: keeping variability and uncertainty separated</article-title>. <source>Risk Anal.</source> <volume>31</volume>, <fpage>1295</fpage>–<lpage>1307</lpage>. (<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1111/j.1539-6924.2011.01592.x">doi:10.1111/j.1539-6924.2011.01592.x</ext-link>)</citation></ref><ref id="RSIF20121018C47"><label>47</label><citation citation-type="confproc"><person-group person-group-type="author"><name><surname>Chan</surname><given-names>YH</given-names></name><name><surname>Correa</surname><given-names>CD</given-names></name><name><surname>Ma</surname><given-names>KL</given-names></name></person-group> (eds) <year>2010</year> <article-title>Flow-based Scatterplots for Sensitivity Analysis</article-title>. In <conf-name>IEEE Symp. on Visual Analytics Science and Technology</conf-name> <publisher-loc>Salt Lake City, UT, 25–26 October 2010.</publisher-loc> <publisher-name>New York, NY: IEEE</publisher-name>.</citation></ref><ref id="RSIF20121018C48"><label>48</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Griffin</surname><given-names>JT</given-names></name><etal></etal></person-group> <year>2010</year> <article-title>Reducing <italic>Plasmodium falciparum</italic> malaria transmission in Africa: a model-based evaluation of intervention strategies</article-title>. <source>PLoS Med.</source> <volume>7</volume>, <fpage>e1000324</fpage>. (<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1371/journal.pmed.1000324">doi:10.1371/journal.pmed.1000324</ext-link>)</citation></ref><ref id="RSIF20121018C49"><label>49</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>van den Driessche</surname><given-names>P</given-names></name><name><surname>Watmough</surname><given-names>J</given-names></name></person-group>. <year>2002</year> <article-title>Reproduction numbers and sub-threshold endemic equilibria for compartmental models of disease transmission</article-title>. <source>Math. Biosci.</source> <volume>180</volume>, <fpage>29</fpage>–<lpage>48</lpage>. (<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1016/S0025-5564(02)00108-6">doi:10.1016/S0025-5564(02)00108-6</ext-link>)</citation></ref></ref-list></back></article></istex:document></istex:metadataXml><mods version="3.6"><titleInfo><title>Sensitivity analysis of infectious disease models: methods, advances and their application</title></titleInfo><titleInfo type="alternative" contentType="CDATA"><title>Sensitivity analysis of infectious disease models: methods, advances and their application</title></titleInfo><name type="personal"><namePart type="given">Jianyong</namePart><namePart type="family">Wu</namePart><affiliation>Department of Environmental Health, Rollins School of Public Health, Emory University, 1518 Clifton Road NE, Atlanta, GA 30322 USA</affiliation><role><roleTerm type="text">author</roleTerm></role></name><name type="personal"><namePart type="given">Radhika</namePart><namePart type="family">Dhingra</namePart><affiliation>Department of Environmental Health, Rollins School of Public Health, Emory University, 1518 Clifton Road NE, Atlanta, GA 30322 USA</affiliation><role><roleTerm type="text">author</roleTerm></role></name><name type="personal"><namePart type="given">Manoj</namePart><namePart type="family">Gambhir</namePart><affiliation>MRC Centre for Outbreak Analysis and Modeling, Department of Infectious Disease Epidemiology, Imperial College London, London, UK</affiliation><role><roleTerm type="text">author</roleTerm></role></name><name type="personal"><namePart type="given">Justin V.</namePart><namePart type="family">Remais</namePart><affiliation>Department of Environmental Health, Rollins School of Public Health, Emory University, 1518 Clifton Road NE, Atlanta, GA 30322 USA</affiliation><affiliation>Program in Population Biology, Ecology and Evolution, Graduate Division of Biological and Biomedical Sciences, Emory University, 1510 Clifton Road, Atlanta, GA 30322, USA</affiliation><affiliation>E-mail: justin.remais@emory.edu</affiliation><role><roleTerm type="text">author</roleTerm></role></name><typeOfResource>text</typeOfResource><genre type="article" displayLabel="review-article" authority="ISTEX" authorityURI="https://content-type.data.istex.fr" valueURI="https://content-type.data.istex.fr/ark:/67375/XTP-6N5SZHKN-D">article</genre><originInfo><publisher>The Royal Society</publisher><dateIssued encoding="w3cdtf">2013-09-06</dateIssued><copyrightDate encoding="w3cdtf">2013</copyrightDate></originInfo><language><languageTerm type="code" authority="iso639-2b">eng</languageTerm><languageTerm type="code" authority="rfc3066">en</languageTerm></language><abstract>Sensitivity analysis (SA) can aid in identifying influential model parameters and optimizing model structure, yet infectious disease modelling has yet to adopt advanced SA techniques that are capable of providing considerable insights over traditional methods. We investigate five global SA methods—scatter plots, the Morris and Sobol’ methods, Latin hypercube sampling-partial rank correlation coefficient and the sensitivity heat map method—and detail their relative merits and pitfalls when applied to a microparasite (cholera) and macroparasite (schistosomaisis) transmission model. The methods investigated yielded similar results with respect to identifying influential parameters, but offered specific insights that vary by method. The classical methods differed in their ability to provide information on the quantitative relationship between parameters and model output, particularly over time. The heat map approach provides information about the group sensitivity of all model state variables, and the parameter sensitivity spectrum obtained using this method reveals the sensitivity of all state variables to each parameter over the course of the simulation period, especially valuable for expressing the dynamic sensitivity of a microparasite epidemic model to its parameters. A summary comparison is presented to aid infectious disease modellers in selecting appropriate methods, with the goal of improving model performance and design.</abstract><subject><topic>sensitivity analysis</topic><topic>infectious disease modelling</topic><topic>Sobol’ method</topic><topic>Morris method</topic><topic>partial rank correlation coefficient</topic><topic>sensitivity heat map</topic></subject><relatedItem type="host"><titleInfo><title>Journal of the Royal Society Interface</title></titleInfo><titleInfo type="abbreviated"><title>J. R. Soc. Interface</title></titleInfo><genre type="journal" authority="ISTEX" authorityURI="https://publication-type.data.istex.fr" valueURI="https://publication-type.data.istex.fr/ark:/67375/JMC-0GLKJH51-B">journal</genre><subject><genre>hwp-journal-coll</genre><topic>1004</topic><topic>181</topic></subject><identifier type="ISSN">1742-5689</identifier><identifier type="eISSN">1742-5662</identifier><identifier type="PublisherID">RSIF</identifier><identifier type="PublisherID-hwp">royinterface</identifier><part><date>2013</date><detail type="volume"><caption>vol.</caption><number>10</number></detail><detail type="issue"><caption>no.</caption><number>86</number></detail></part></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C1"><titleInfo><title>Infectious diseases of humans: dynamics and control</title></titleInfo><name type="personal"><namePart type="given">RM</namePart><namePart type="family">Anderson</namePart></name><name type="personal"><namePart type="given">RM</namePart><namePart type="family">May</namePart></name><originInfo><publisher>Royal Society of London</publisher><publisher>Oxford University Press. </publisher><place><placeTerm type="text">Oxford, UK</placeTerm></place></originInfo><genre>book</genre><part><date>1991</date></part></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C2"><titleInfo><title>Mathematical models of infectious disease transmission</title></titleInfo><name type="personal"><namePart type="given">NC</namePart><namePart type="family">Grassly</namePart></name><name type="personal"><namePart type="given">C</namePart><namePart type="family">Fraser</namePart></name><genre>journal</genre><relatedItem type="host"><titleInfo><title>Nat. Rev. Microbiol.</title></titleInfo><part><date>2008</date><detail type="volume"><caption>vol.</caption><number>6</number></detail><extent unit="pages"><start>477</start><end>487</end></extent></part></relatedItem></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C3"><titleInfo><title>Modeling infectious diseases in human and animals</title></titleInfo><name type="personal"><namePart type="given">M</namePart><namePart type="family">Keeling</namePart></name><name type="personal"><namePart type="given">P</namePart><namePart type="family">Rohani</namePart></name><originInfo><publisher>Royal Society of London</publisher><publisher>Princeton University Press. </publisher><place><placeTerm type="text">Princeton, NJ</placeTerm></place></originInfo><genre>book</genre><part><date>2007</date></part></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C4"><titleInfo><title>Parameterizing state-space models for infectious disease dynamics by generalized profiling: measles in Ontario</title></titleInfo><name type="personal"><namePart type="given">G</namePart><namePart type="family">Hooker</namePart></name><name type="personal"><namePart type="given">SP</namePart><namePart type="family">Ellner</namePart></name><name type="personal"><namePart type="given">LD</namePart><namePart type="family">Roditi</namePart></name><name type="personal"><namePart type="given">DJD</namePart><namePart type="family">Earn</namePart></name><genre>journal</genre><relatedItem type="host"><titleInfo><title>J. R. Soc. Interface</title></titleInfo><part><date>2011</date><detail type="volume"><caption>vol.</caption><number>8</number></detail><extent unit="pages"><start>961</start><end>974</end></extent></part></relatedItem><identifier type="uri">http://dx.doi.org/10.1098/rsif.2010.0412</identifier><identifier type="doi">10.1098/rsif.2010.0412</identifier></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C5"><titleInfo><title>Methodology of the sensitivity analysis used for modeling an infectious disease</title></titleInfo><name type="personal"><namePart type="given">C</namePart><namePart type="family">Okais</namePart></name><name type="personal"><namePart type="given">S</namePart><namePart type="family">Roche</namePart></name><name type="personal"><namePart type="given">ML</namePart><namePart type="family">Kurzinger</namePart></name><name type="personal"><namePart type="given">B</namePart><namePart type="family">Riche</namePart></name><name type="personal"><namePart type="given">H</namePart><namePart type="family">Bricout</namePart></name><name type="personal"><namePart type="given">T</namePart><namePart type="family">Derrough</namePart></name><name type="personal"><namePart type="given">F</namePart><namePart type="family">Simondon</namePart></name><name type="personal"><namePart type="given">R.</namePart><namePart type="family">Ecochard</namePart></name><genre>journal</genre><relatedItem type="host"><titleInfo><title>Vaccine</title></titleInfo><part><date>2010</date><detail type="volume"><caption>vol.</caption><number>28</number></detail><extent unit="pages"><start>8132</start><end>8140</end></extent></part></relatedItem><identifier type="uri">http://dx.doi.org/10.1016/j.vaccine.2010.09.099</identifier><identifier type="doi">10.1016/j.vaccine.2010.09.099</identifier></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C6"><titleInfo><title>Cautioning the use of degree-day models for climate change projections in the presence of parametric uncertainty</title></titleInfo><name type="personal"><namePart type="given">JL</namePart><namePart type="family">Moore</namePart></name><name type="personal"><namePart type="given">S</namePart><namePart type="family">Liang</namePart></name><name type="personal"><namePart type="given">A</namePart><namePart type="family">Akullian</namePart></name><name type="personal"><namePart type="given">JV</namePart><namePart type="family">Remais</namePart></name><genre>journal</genre><relatedItem type="host"><titleInfo><title>Ecol. Appl.</title></titleInfo><part><date>2012</date><detail type="volume"><caption>vol.</caption><number>22</number></detail><extent unit="pages"><start>2237</start><end>2247</end></extent></part></relatedItem><identifier type="uri">http://dx.doi.org/10.1890/12-0127.1</identifier><identifier type="doi">10.1890/12-0127.1</identifier></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C7"><titleInfo><title>Global sensitivity analysis. The primer</title></titleInfo><name type="personal"><namePart type="given">A</namePart><namePart type="family">Saltelli</namePart></name><name type="personal"><namePart type="given">M</namePart><namePart type="family">Ratto</namePart></name><name type="personal"><namePart type="given">T</namePart><namePart type="family">Andres</namePart></name><name type="personal"><namePart type="given">F</namePart><namePart type="family">Campolongo</namePart></name><name type="personal"><namePart type="given">J</namePart><namePart type="family">Cariboni</namePart></name><name type="personal"><namePart type="given">D</namePart><namePart type="family">Gatelli</namePart></name><name type="personal"><namePart type="given">M</namePart><namePart type="family">Saisana</namePart></name><name type="personal"><namePart type="given">S</namePart><namePart type="family">Tarantola</namePart></name><originInfo><publisher>Royal Society of London</publisher><publisher>John Wiley & Sons. </publisher><place><placeTerm type="text">Chichester, UK</placeTerm></place></originInfo><genre>book</genre><part><date>2008</date></part></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C8"><titleInfo><title>Transmission shifts underlie variability in population responses to Yersinia pestis infection</title></titleInfo><name type="personal"><namePart type="given">MG</namePart><namePart type="family">Buhnerkempe</namePart></name><name type="personal"><namePart type="given">RJ</namePart><namePart type="family">Eisen</namePart></name><name type="personal"><namePart type="given">B</namePart><namePart type="family">Goodell</namePart></name><name type="personal"><namePart type="given">KL</namePart><namePart type="family">Gage</namePart></name><name type="personal"><namePart type="given">MF</namePart><namePart type="family">Antolin</namePart></name><name type="personal"><namePart type="given">CT</namePart><namePart type="family">Webb</namePart></name><genre>journal</genre><relatedItem type="host"><titleInfo><title>PLoS ONE</title></titleInfo><part><date>2011</date><detail type="volume"><caption>vol.</caption><number>6</number></detail><extent unit="pages"><start>e22498</start></extent></part></relatedItem><identifier type="uri">http://dx.doi.org/10.1371/journal.pone.0022498</identifier><identifier type="doi">10.1371/journal.pone.0022498</identifier></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C9"><titleInfo><title>Determining important parameters in the spread of malaria through the sensitivity analysis of a mathematical model</title></titleInfo><name type="personal"><namePart type="given">N</namePart><namePart type="family">Chitnis</namePart></name><name type="personal"><namePart type="given">JM</namePart><namePart type="family">Hyman</namePart></name><name type="personal"><namePart type="given">JM</namePart><namePart type="family">Cushing</namePart></name><genre>journal</genre><relatedItem type="host"><titleInfo><title>Bull. Math. Biol.</title></titleInfo><part><date>2008</date><detail type="volume"><caption>vol.</caption><number>70</number></detail><extent unit="pages"><start>1272</start><end>1296</end></extent></part></relatedItem><identifier type="uri">http://dx.doi.org/10.1007/s11538-008-9299-0</identifier><identifier type="doi">10.1007/s11538-008-9299-0</identifier></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C10"><titleInfo><title>Sensitivity and uncertainty analyses for a SARS model with time-varying inputs and outputs</title></titleInfo><name type="personal"><namePart type="given">RG</namePart><namePart type="family">McLeod</namePart></name><name type="personal"><namePart type="given">JF</namePart><namePart type="family">Brewster</namePart></name><name type="personal"><namePart type="given">AB</namePart><namePart type="family">Gumel</namePart></name><name type="personal"><namePart type="given">DA</namePart><namePart type="family">Slonowsky</namePart></name><genre>journal</genre><relatedItem type="host"><titleInfo><title>Math. Biosci. Eng.</title></titleInfo><part><date>2006</date><detail type="volume"><caption>vol.</caption><number>3</number></detail><extent unit="pages"><start>527</start><end>544</end></extent></part></relatedItem><identifier type="uri">http://dx.doi.org/10.3934/mbe.2006.3.527</identifier><identifier type="doi">10.3934/mbe.2006.3.527</identifier></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C11"><titleInfo><title>Mapping global sensitivity of cellular network dynamics: sensitivity heat maps and a global summation law</title></titleInfo><name type="personal"><namePart type="given">DA</namePart><namePart type="family">Rand</namePart></name><genre>journal</genre><relatedItem type="host"><titleInfo><title>J. R. Soc. Interface</title></titleInfo><part><date>2009</date><detail type="volume"><caption>vol.</caption><number>5</number></detail><extent unit="pages"><start>S59</start><end>S69</end></extent></part></relatedItem><identifier type="uri">http://dx.doi.org/10.1098/rsif.2008.0084.focus</identifier><identifier type="doi">10.1098/rsif.2008.0084.focus</identifier></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C12"><titleInfo><title>Elasticity analysis as an important tool in evolutionary and population ecology</title></titleInfo><name type="personal"><namePart type="given">TG</namePart><namePart type="family">Benton</namePart></name><name type="personal"><namePart type="given">A</namePart><namePart type="family">Grant</namePart></name><genre>journal</genre><relatedItem type="host"><titleInfo><title>Trends Ecol. Evol.</title></titleInfo><part><date>1999</date><detail type="volume"><caption>vol.</caption><number>14</number></detail><extent unit="pages"><start>467</start><end>471</end></extent></part></relatedItem><identifier type="uri">http://dx.doi.org/10.1016/S0169-5347(99)01724-3</identifier><identifier type="doi">10.1016/S0169-5347(99)01724-3</identifier></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C13"><titleInfo><title>Microeconomics</title></titleInfo><name type="personal"><namePart type="given">PA</namePart><namePart type="family">Samuelson</namePart></name><name type="personal"><namePart type="given">WD</namePart><namePart type="family">Nordhaus</namePart></name><originInfo><publisher>Royal Society of London</publisher><publisher>McGraw-Hill Irwin. </publisher><place><placeTerm type="text">Boston, MA</placeTerm></place></originInfo><genre>book</genre><part><date>2010</date></part></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C14"><titleInfo><title>Response surface methodology: process and product optimization using designed experiments</title></titleInfo><name type="personal"><namePart type="given">RH</namePart><namePart type="family">Myers</namePart></name><name type="personal"><namePart type="given">DC</namePart><namePart type="family">Montgomery</namePart></name><name type="personal"><namePart type="given">CM</namePart><namePart type="family">Anderson-Cook</namePart></name><originInfo><publisher>Royal Society of London</publisher><publisher>Wiley. </publisher><place><placeTerm type="text">Hoboken, NJ</placeTerm></place></originInfo><genre>book</genre><part><date>2009</date></part></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C15"><titleInfo><title>Design and analysis of experiments</title></titleInfo><name type="personal"><namePart type="given">DC</namePart><namePart type="family">Montgomery</namePart></name><originInfo><publisher>Royal Society of London</publisher><publisher>John Wiley & Sons, Inc. </publisher><place><placeTerm type="text">Hoboken, NJ</placeTerm></place></originInfo><genre>book</genre><part><date>2013</date></part></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C16"><titleInfo><title>On sensitivity estimates for nonlinear mathematical models</title></titleInfo><name type="personal"><namePart type="given">IM</namePart><namePart type="family">Sobol</namePart></name><genre>journal</genre><note>In Russian</note><relatedItem type="host"><titleInfo><title>Matematicheskoe Modelirovanie</title></titleInfo><part><date>1990</date><detail type="volume"><caption>vol.</caption><number>2</number></detail><extent unit="pages"><start>112</start><end>118</end></extent></part></relatedItem></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C17"><titleInfo><title>A quantitative model-independent method for global sensitivity analysis of model output</title></titleInfo><name type="personal"><namePart type="given">A</namePart><namePart type="family">Saltelli</namePart></name><name type="personal"><namePart type="given">S</namePart><namePart type="family">Tarantola</namePart></name><name type="personal"><namePart type="given">KPS</namePart><namePart type="family">Chan</namePart></name><genre>journal</genre><relatedItem type="host"><titleInfo><title>Technometrics</title></titleInfo><part><date>1999</date><detail type="volume"><caption>vol.</caption><number>41</number></detail><extent unit="pages"><start>39</start><end>56</end></extent></part></relatedItem><identifier type="uri">http://dx.doi.org/10.1080/00401706.1999.10485594</identifier><identifier type="doi">10.1080/00401706.1999.10485594</identifier></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C18"><titleInfo><title>Factorial sampling plans for preliminary computational experiments</title></titleInfo><name type="personal"><namePart type="given">MD</namePart><namePart type="family">Morris</namePart></name><genre>journal</genre><relatedItem type="host"><titleInfo><title>Technometrics</title></titleInfo><part><date>1991</date><detail type="volume"><caption>vol.</caption><number>33</number></detail><extent unit="pages"><start>161</start><end>174</end></extent></part></relatedItem><identifier type="uri">http://dx.doi.org/10.1080/00401706.1991.10484804</identifier><identifier type="doi">10.1080/00401706.1991.10484804</identifier></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C19"><titleInfo><title>An effective screening design for sensitivity analysis of large models</title></titleInfo><name type="personal"><namePart type="given">F</namePart><namePart type="family">Campolongo</namePart></name><name type="personal"><namePart type="given">J</namePart><namePart type="family">Cariboni</namePart></name><name type="personal"><namePart type="given">A</namePart><namePart type="family">Saltelli</namePart></name><genre>journal</genre><relatedItem type="host"><titleInfo><title>Environ. Model. Softw.</title></titleInfo><part><date>2007</date><detail type="volume"><caption>vol.</caption><number>22</number></detail><extent unit="pages"><start>1509</start><end>1518</end></extent></part></relatedItem><identifier type="uri">http://dx.doi.org/10.1016/j.envsoft.2006.10.004</identifier><identifier type="doi">10.1016/j.envsoft.2006.10.004</identifier></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C20"><titleInfo><title>Illustration of sampling-based methods for uncertainty and sensitivity analysis</title></titleInfo><name type="personal"><namePart type="given">JC</namePart><namePart type="family">Helton</namePart></name><name type="personal"><namePart type="given">FJ</namePart><namePart type="family">Davis</namePart></name><genre>journal</genre><relatedItem type="host"><titleInfo><title>Risk Anal.</title></titleInfo><part><date>2002</date><detail type="volume"><caption>vol.</caption><number>22</number></detail><extent unit="pages"><start>591</start><end>622</end></extent></part></relatedItem><identifier type="uri">http://dx.doi.org/10.1111/0272-4332.00041</identifier><identifier type="doi">10.1111/0272-4332.00041</identifier></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C21"><titleInfo><title>A methodology for performing global uncertainty and sensitivity analysis in systems biology</title></titleInfo><name type="personal"><namePart type="given">S</namePart><namePart type="family">Marino</namePart></name><name type="personal"><namePart type="given">IB</namePart><namePart type="family">Hogue</namePart></name><name type="personal"><namePart type="given">CJ</namePart><namePart type="family">Ray</namePart></name><name type="personal"><namePart type="given">DE</namePart><namePart type="family">Kirschner</namePart></name><genre>journal</genre><relatedItem type="host"><titleInfo><title>J. Theor. Biol.</title></titleInfo><part><date>2008</date><detail type="volume"><caption>vol.</caption><number>254</number></detail><extent unit="pages"><start>178</start><end>196</end></extent></part></relatedItem><identifier type="uri">http://dx.doi.org/10.1016/j.jtbi.2008.04.011</identifier><identifier type="doi">10.1016/j.jtbi.2008.04.011</identifier></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C22"><titleInfo><title>A review of techniques for parameter sensitivity analysis of environmental-models</title></titleInfo><name type="personal"><namePart type="given">DM</namePart><namePart type="family">Hamby</namePart></name><genre>journal</genre><relatedItem type="host"><titleInfo><title>Environ. Monit. Assess.</title></titleInfo><part><date>1994</date><detail type="volume"><caption>vol.</caption><number>32</number></detail><extent unit="pages"><start>135</start><end>154</end></extent></part></relatedItem><identifier type="uri">http://dx.doi.org/10.1007/BF00547132</identifier><identifier type="doi">10.1007/BF00547132</identifier></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C23"><titleInfo><title>Convergence and uncertainty analyses in Monte-Carlo based sensitivity analysis</title></titleInfo><name type="personal"><namePart type="given">J</namePart><namePart type="family">Yang</namePart></name><genre>journal</genre><relatedItem type="host"><titleInfo><title>Environ. Model. Softw.</title></titleInfo><part><date>2011</date><detail type="volume"><caption>vol.</caption><number>26</number></detail><extent unit="pages"><start>444</start><end>457</end></extent></part></relatedItem><identifier type="uri">http://dx.doi.org/10.1016/j.envsoft.2010.10.007</identifier><identifier type="doi">10.1016/j.envsoft.2010.10.007</identifier></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C24"><titleInfo><title>Transmission dynamics and control of cholera in Haiti: an epidemic model</title></titleInfo><name type="personal"><namePart type="given">JR</namePart><namePart type="family">Andrews</namePart></name><name type="personal"><namePart type="given">S</namePart><namePart type="family">Basu</namePart></name><genre>journal</genre><relatedItem type="host"><titleInfo><title>Lancet</title></titleInfo><part><date>2011</date><detail type="volume"><caption>vol.</caption><number>377</number></detail><extent unit="pages"><start>1248</start><end>1255</end></extent></part></relatedItem><identifier type="uri">http://dx.doi.org/10.1016/S0140-6736(11)60273-0</identifier><identifier type="doi">10.1016/S0140-6736(11)60273-0</identifier></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C25"><titleInfo><title>Analysis of Rabies in China: transmission dynamics and control</title></titleInfo><name type="personal"><namePart type="given">J</namePart><namePart type="family">Zhang</namePart></name><name type="personal"><namePart type="given">Z</namePart><namePart type="family">Jin</namePart></name><name type="personal"><namePart type="given">G</namePart><namePart type="family">Sun</namePart></name><name type="personal"><namePart type="given">T</namePart><namePart type="family">Zhou</namePart></name><name type="personal"><namePart type="given">S</namePart><namePart type="family">Ruan</namePart></name><genre>journal</genre><relatedItem type="host"><titleInfo><title>PLoS ONE</title></titleInfo><part><date>2011</date><detail type="volume"><caption>vol.</caption><number>6</number></detail></part></relatedItem><identifier type="uri">http://dx.doi.org/10.1371/journal.pone.0020891</identifier><identifier type="doi">10.1371/journal.pone.0020891</identifier></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C26"><titleInfo><title>Assessing the risk of bluetongue to UK livestock: uncertainty and sensitivity analyses of a temperature-dependent model for the basic reproduction number</title></titleInfo><name type="personal"><namePart type="given">S</namePart><namePart type="family">Gubbins</namePart></name><name type="personal"><namePart type="given">S</namePart><namePart type="family">Carpenter</namePart></name><name type="personal"><namePart type="given">M</namePart><namePart type="family">Baylis</namePart></name><name type="personal"><namePart type="given">JLN</namePart><namePart type="family">Wood</namePart></name><name type="personal"><namePart type="given">PS</namePart><namePart type="family">Mellor</namePart></name><genre>journal</genre><relatedItem type="host"><titleInfo><title>J. R. Soc. Interface</title></titleInfo><part><date>2008</date><detail type="volume"><caption>vol.</caption><number>5</number></detail><extent unit="pages"><start>363</start><end>371</end></extent></part></relatedItem><identifier type="uri">http://dx.doi.org/10.1098/rsif.2007.1110</identifier><identifier type="doi">10.1098/rsif.2007.1110</identifier></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C27"><titleInfo><title>Sensitivity and uncertainty analysis of complex-models of disease transmission—an HIV model, as an example</title></titleInfo><name type="personal"><namePart type="given">SM</namePart><namePart type="family">Blower</namePart></name><name type="personal"><namePart type="given">H</namePart><namePart type="family">Dowlatabadi</namePart></name><genre>journal</genre><relatedItem type="host"><titleInfo><title>Int. Stat. Rev.</title></titleInfo><part><date>1994</date><detail type="volume"><caption>vol.</caption><number>62</number></detail><extent unit="pages"><start>229</start><end>243</end></extent></part></relatedItem><identifier type="uri">http://dx.doi.org/10.2307/1403510</identifier><identifier type="doi">10.2307/1403510</identifier></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C28"><titleInfo><title>Quantifying the intrinsic transmission dynamics of tuberculosis</title></titleInfo><name type="personal"><namePart type="given">TC</namePart><namePart type="family">Porco</namePart></name><name type="personal"><namePart type="given">SM</namePart><namePart type="family">Blower</namePart></name><genre>journal</genre><relatedItem type="host"><titleInfo><title>Theor. Popul. Biol.</title></titleInfo><part><date>1998</date><detail type="volume"><caption>vol.</caption><number>54</number></detail><extent unit="pages"><start>117</start><end>132</end></extent></part></relatedItem><identifier type="uri">http://dx.doi.org/10.1006/tpbi.1998.1366</identifier><identifier type="doi">10.1006/tpbi.1998.1366</identifier></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C29"><titleInfo><title>Parameterization and sensitivity analysis of a complex simulation model for mosquito population dynamics, dengue transmission, and their control</title></titleInfo><name type="personal"><namePart type="given">AM</namePart><namePart type="family">Ellis</namePart></name><name type="personal"><namePart type="given">AJ</namePart><namePart type="family">Garcia</namePart></name><name type="personal"><namePart type="given">DA</namePart><namePart type="family">Focks</namePart></name><name type="personal"><namePart type="given">AC</namePart><namePart type="family">Morrison</namePart></name><name type="personal"><namePart type="given">TW</namePart><namePart type="family">Scott</namePart></name><genre>journal</genre><relatedItem type="host"><titleInfo><title>Am. J. Trop. Med. Hyg.</title></titleInfo><part><date>2011</date><detail type="volume"><caption>vol.</caption><number>85</number></detail><extent unit="pages"><start>257</start><end>264</end></extent></part></relatedItem><identifier type="uri">http://dx.doi.org/10.4269/ajtmh.2011.10-0516</identifier><identifier type="doi">10.4269/ajtmh.2011.10-0516</identifier></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C30"><titleInfo><title>Sensitivity analysis in the modeling of infectious-disease dynamics</title></titleInfo><name type="personal"><namePart type="given">NTJ</namePart><namePart type="family">Bailey</namePart></name><name type="personal"><namePart type="given">J</namePart><namePart type="family">Duppenthaler</namePart></name><genre>journal</genre><relatedItem type="host"><titleInfo><title>J. Math. Biol.</title></titleInfo><part><date>1980</date><detail type="volume"><caption>vol.</caption><number>10</number></detail><extent unit="pages"><start>113</start><end>131</end></extent></part></relatedItem><identifier type="uri">http://dx.doi.org/10.1007/BF00275837</identifier><identifier type="doi">10.1007/BF00275837</identifier></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C31"><titleInfo><title>A simulation-model of AIDS in San Francisco. 2. Simulations, therapy, and sensitivity analysis</title></titleInfo><name type="personal"><namePart type="given">HW</namePart><namePart type="family">Hethcote</namePart></name><name type="personal"><namePart type="given">JW</namePart><namePart type="family">Vanark</namePart></name><name type="personal"><namePart type="given">JM</namePart><namePart type="family">Karon</namePart></name><genre>journal</genre><relatedItem type="host"><titleInfo><title>Math. Biosci.</title></titleInfo><part><date>1991</date><detail type="volume"><caption>vol.</caption><number>106</number></detail><extent unit="pages"><start>223</start><end>247</end></extent></part></relatedItem><identifier type="uri">http://dx.doi.org/10.1016/0025-5564(91)90078-W</identifier><identifier type="doi">10.1016/0025-5564(91)90078-W</identifier></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C32"><titleInfo><title>Modeling the impact of tuberculosis control strategies in highly endemic overcrowded prisons</title></titleInfo><name type="personal"><namePart type="given">J</namePart><namePart type="family">Legrand</namePart></name><name type="personal"><namePart type="given">A</namePart><namePart type="family">Sanchez</namePart></name><name type="personal"><namePart type="given">F</namePart><namePart type="family">Le Pont</namePart></name><name type="personal"><namePart type="given">L</namePart><namePart type="family">Camacho</namePart></name><name type="personal"><namePart type="given">B</namePart><namePart type="family">Larouze</namePart></name><genre>journal</genre><relatedItem type="host"><titleInfo><title>PLoS ONE</title></titleInfo><part><date>2008</date><detail type="volume"><caption>vol.</caption><number>3</number></detail><extent unit="pages"><start>e2100</start></extent></part></relatedItem><identifier type="uri">http://dx.doi.org/10.1371/journal.pone.0002100</identifier><identifier type="doi">10.1371/journal.pone.0002100</identifier></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C33"><titleInfo><title>Sensitivity analysis in practice: a guide to assessing scientific models</title></titleInfo><name type="personal"><namePart type="given">A</namePart><namePart type="family">Saltelli</namePart></name><name type="personal"><namePart type="given">S</namePart><namePart type="family">Tarantola</namePart></name><name type="personal"><namePart type="given">F</namePart><namePart type="family">Campolongo</namePart></name><name type="personal"><namePart type="given">M</namePart><namePart type="family">Ratto</namePart></name><originInfo><publisher>Royal Society of London</publisher><publisher>Chichester, UK: John Wiley & Sons. </publisher></originInfo><genre>book</genre><part><date>2004</date></part></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C34"><titleInfo><title>A comparison of sensitivity analysis techniques</title></titleInfo><name type="personal"><namePart type="given">DM</namePart><namePart type="family">Hamby</namePart></name><genre>journal</genre><relatedItem type="host"><titleInfo><title>Health Phys.</title></titleInfo><part><date>1995</date><detail type="volume"><caption>vol.</caption><number>68</number></detail><extent unit="pages"><start>195</start><end>204</end></extent></part></relatedItem><identifier type="uri">http://dx.doi.org/10.1097/00004032-199502000-00005</identifier><identifier type="doi">10.1097/00004032-199502000-00005</identifier></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C35"><titleInfo><title>A comparison of three methods for selecting values of input variables in the analysis of output from a computer code</title></titleInfo><name type="personal"><namePart type="given">MD</namePart><namePart type="family">McKay</namePart></name><name type="personal"><namePart type="given">RJ</namePart><namePart type="family">Beckman</namePart></name><name type="personal"><namePart type="given">WJ</namePart><namePart type="family">Conover</namePart></name><genre>journal</genre><relatedItem type="host"><titleInfo><title>Technometrics</title></titleInfo><part><date>1979</date><detail type="volume"><caption>vol.</caption><number>21</number></detail><extent unit="pages"><start>239</start><end>245</end></extent></part></relatedItem></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C36"><titleInfo><title>Latin hypercube sampling and the propagation of uncertainty in analyses of complex systems</title></titleInfo><name type="personal"><namePart type="given">JC</namePart><namePart type="family">Helton</namePart></name><name type="personal"><namePart type="given">FJ</namePart><namePart type="family">Davis</namePart></name><genre>journal</genre><relatedItem type="host"><titleInfo><title>Reliab. Eng. Syst. Safety</title></titleInfo><part><date>2003</date><detail type="volume"><caption>vol.</caption><number>81</number></detail><extent unit="pages"><start>23</start><end>69</end></extent></part></relatedItem><identifier type="uri">http://dx.doi.org/10.1016/S0951-8320(03)00058-9</identifier><identifier type="doi">10.1016/S0951-8320(03)00058-9</identifier></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C37"><titleInfo><title>Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates</title></titleInfo><name type="personal"><namePart type="given">IM</namePart><namePart type="family">Sobol</namePart></name><genre>journal</genre><relatedItem type="host"><titleInfo><title>Math. Comp. Simul.</title></titleInfo><part><date>2001</date><detail type="volume"><caption>vol.</caption><number>55</number></detail><extent unit="pages"><start>271</start><end>280</end></extent></part></relatedItem><identifier type="uri">http://dx.doi.org/10.1016/S0378-4754(00)00270-6</identifier><identifier type="doi">10.1016/S0378-4754(00)00270-6</identifier></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C38"><titleInfo><title>Variance based sensitivity analysis of model output. Design and estimator for the total sensitivity index</title></titleInfo><name type="personal"><namePart type="given">A</namePart><namePart type="family">Saltelli</namePart></name><name type="personal"><namePart type="given">P</namePart><namePart type="family">Annoni</namePart></name><name type="personal"><namePart type="given">I</namePart><namePart type="family">Azzini</namePart></name><name type="personal"><namePart type="given">F</namePart><namePart type="family">Campolongo</namePart></name><name type="personal"><namePart type="given">M</namePart><namePart type="family">Ratto</namePart></name><name type="personal"><namePart type="given">S</namePart><namePart type="family">Tarantola</namePart></name><genre>journal</genre><relatedItem type="host"><titleInfo><title>Comp. Phys. Commun.</title></titleInfo><part><date>2010</date><detail type="volume"><caption>vol.</caption><number>181</number></detail><extent unit="pages"><start>259</start><end>270</end></extent></part></relatedItem><identifier type="uri">http://dx.doi.org/10.1016/j.cpc.2009.09.018</identifier><identifier type="doi">10.1016/j.cpc.2009.09.018</identifier></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C39"><titleInfo><title>Balance equations can buffer noisy and sustained environmental perturbations of circadian clocks</title></titleInfo><name type="personal"><namePart type="given">M</namePart><namePart type="family">Domijan</namePart></name><name type="personal"><namePart type="given">DA</namePart><namePart type="family">Rand</namePart></name><genre>journal</genre><relatedItem type="host"><titleInfo><title>Interface Focus</title></titleInfo><part><date>2011</date><detail type="volume"><caption>vol.</caption><number>1</number></detail><extent unit="pages"><start>177</start><end>186</end></extent></part></relatedItem><identifier type="uri">http://dx.doi.org/10.1098/rsfs.2010.0007</identifier><identifier type="doi">10.1098/rsfs.2010.0007</identifier></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C40"><titleInfo><title>A quantitative framework for a multi-group model of Schistosomiasis japonicum transmission dynamics and control in Sichuan, China</title></titleInfo><name type="personal"><namePart type="given">S</namePart><namePart type="family">Liang</namePart></name><name type="personal"><namePart type="given">D</namePart><namePart type="family">Maszle</namePart></name><name type="personal"><namePart type="given">RC</namePart><namePart type="family">Spear</namePart></name><genre>journal</genre><relatedItem type="host"><titleInfo><title>Acta Trop.</title></titleInfo><part><date>2002</date><detail type="volume"><caption>vol.</caption><number>82</number></detail><extent unit="pages"><start>263</start><end>277</end></extent></part></relatedItem><identifier type="uri">http://dx.doi.org/10.1016/S0001-706X(02)00018-9</identifier><identifier type="doi">10.1016/S0001-706X(02)00018-9</identifier></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C41"><titleInfo><title>Environmental effects on parasitic disease transmission exemplified by schistosomiasis in western China</title></titleInfo><name type="personal"><namePart type="given">S</namePart><namePart type="family">Liang</namePart></name><genre>journal</genre><relatedItem type="host"><titleInfo><title>Proc. Natl Acad. Sci. USA</title></titleInfo><part><date>2007</date><detail type="volume"><caption>vol.</caption><number>104</number></detail><extent unit="pages"><start>7110</start><end>7115</end></extent></part></relatedItem><identifier type="uri">http://dx.doi.org/10.1073/pnas.0701878104</identifier><identifier type="doi">10.1073/pnas.0701878104</identifier></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C42"><titleInfo><title>BrownPRandDA. 2012 Using the Global Sensitivity GUIs. See http://wsbcwarwickacuk/software/indexhtml.</title></titleInfo><name type="personal"><namePart type="given">P</namePart><namePart type="family">Brown</namePart></name><name type="personal"><namePart type="given">DA</namePart><namePart type="family">Rand</namePart></name><genre>other</genre><note>Using the Global Sensitivity GUIs</note><part><date>2012</date></part></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C43"><titleInfo><title>Identifying control mechanisms of granuloma formation during M-tuberculosis infection using an agent-based model</title></titleInfo><name type="personal"><namePart type="given">JL</namePart><namePart type="family">Segovia-Juarez</namePart></name><name type="personal"><namePart type="given">S</namePart><namePart type="family">Ganguli</namePart></name><name type="personal"><namePart type="given">D</namePart><namePart type="family">Kirschner</namePart></name><genre>journal</genre><relatedItem type="host"><titleInfo><title>J. Theor. Biol.</title></titleInfo><part><date>2004</date><detail type="volume"><caption>vol.</caption><number>231</number></detail><extent unit="pages"><start>357</start><end>376</end></extent></part></relatedItem><identifier type="uri">http://dx.doi.org/10.1016/j.jtbi.2004.06.031</identifier><identifier type="doi">10.1016/j.jtbi.2004.06.031</identifier></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C44"><titleInfo><title>Model approaches for estimating the influence of time-varying socio-environmental factors on macroparasite transmission in two endemic regions</title></titleInfo><name type="personal"><namePart type="given">J</namePart><namePart type="family">Remais</namePart></name><name type="personal"><namePart type="given">B</namePart><namePart type="family">Zhong</namePart></name><name type="personal"><namePart type="given">EJ</namePart><namePart type="family">Carlton</namePart></name><name type="personal"><namePart type="given">RC</namePart><namePart type="family">Spear</namePart></name><genre>journal</genre><relatedItem type="host"><titleInfo><title>Epidemics</title></titleInfo><part><date>2009</date><detail type="volume"><caption>vol.</caption><number>1</number></detail><extent unit="pages"><start>213</start><end>220</end></extent></part></relatedItem><identifier type="uri">http://dx.doi.org/10.1016/j.epidem.2009.10.001</identifier><identifier type="doi">10.1016/j.epidem.2009.10.001</identifier></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C45"><titleInfo><title>Sensitivity analysis of a two-dimensional probabilistic risk assessment model using analysis of variance</title></titleInfo><name type="personal"><namePart type="given">A</namePart><namePart type="family">Mokhtari</namePart></name><name type="personal"><namePart type="given">HC</namePart><namePart type="family">Frey</namePart></name><genre>journal</genre><relatedItem type="host"><titleInfo><title>Risk Anal.</title></titleInfo><part><date>2005</date><detail type="volume"><caption>vol.</caption><number>25</number></detail><extent unit="pages"><start>1511</start><end>1529</end></extent></part></relatedItem><identifier type="uri">http://dx.doi.org/10.1111/j.1539-6924.2005.00679.x</identifier><identifier type="doi">10.1111/j.1539-6924.2005.00679.x</identifier></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C46"><titleInfo><title>Sensitivity analysis of a two-dimensional quantitative microbiological risk assessment: keeping variability and uncertainty separated</title></titleInfo><name type="personal"><namePart type="given">P</namePart><namePart type="family">Busschaert</namePart></name><name type="personal"><namePart type="given">AH</namePart><namePart type="family">Geeraerd</namePart></name><name type="personal"><namePart type="given">M</namePart><namePart type="family">Uyttendaele</namePart></name><name type="personal"><namePart type="given">JF</namePart><namePart type="family">Van Impe</namePart></name><genre>journal</genre><relatedItem type="host"><titleInfo><title>Risk Anal.</title></titleInfo><part><date>2011</date><detail type="volume"><caption>vol.</caption><number>31</number></detail><extent unit="pages"><start>1295</start><end>1307</end></extent></part></relatedItem><identifier type="uri">http://dx.doi.org/10.1111/j.1539-6924.2011.01592.x</identifier><identifier type="doi">10.1111/j.1539-6924.2011.01592.x</identifier></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C47"><titleInfo><title>Flow-based Scatterplots for Sensitivity Analysis</title></titleInfo><name type="conference"><namePart>IEEE Symp. on Visual Analytics Science and Technology</namePart></name><name type="personal"><namePart type="given">YH</namePart><namePart type="family">Chan</namePart></name><name type="personal"><namePart type="given">CD</namePart><namePart type="family">Correa</namePart></name><name type="personal"><namePart type="given">KL</namePart><namePart type="family">Ma</namePart></name><genre>confproc</genre></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C48"><titleInfo><title>Reducing Plasmodium falciparum malaria transmission in Africa: a model-based evaluation of intervention strategies</title></titleInfo><name type="personal"><namePart type="given">JT</namePart><namePart type="family">Griffin</namePart></name><genre>journal</genre><relatedItem type="host"><titleInfo><title>PLoS Med.</title></titleInfo><part><date>2010</date><detail type="volume"><caption>vol.</caption><number>7</number></detail><extent unit="pages"><start>e1000324</start></extent></part></relatedItem><identifier type="uri">http://dx.doi.org/10.1371/journal.pmed.1000324</identifier><identifier type="doi">10.1371/journal.pmed.1000324</identifier></relatedItem><relatedItem type="references" displayLabel="RSIF20121018C49"><titleInfo><title>Reproduction numbers and sub-threshold endemic equilibria for compartmental models of disease transmission</title></titleInfo><name type="personal"><namePart type="given">P</namePart><namePart type="family">van den Driessche</namePart></name><name type="personal"><namePart type="given">J</namePart><namePart type="family">Watmough</namePart></name><genre>journal</genre><relatedItem type="host"><titleInfo><title>Math. Biosci.</title></titleInfo><part><date>2002</date><detail type="volume"><caption>vol.</caption><number>180</number></detail><extent unit="pages"><start>29</start><end>48</end></extent></part></relatedItem><identifier type="uri">http://dx.doi.org/10.1016/S0025-5564(02)00108-6</identifier><identifier type="doi">10.1016/S0025-5564(02)00108-6</identifier></relatedItem><identifier type="istex">03CD6E3087F62163FAA9FBF4A67A8901D755C741</identifier><identifier type="ark">ark:/67375/V84-PCJ0MG5K-1</identifier><identifier type="DOI">10.1098/rsif.2012.1018</identifier><identifier type="href">rsif20121018.pdf</identifier><identifier type="ArticleID">rsif20121018</identifier><accessCondition type="use and reproduction" contentType="copyright">© 2013 The Author(s) Published by the Royal Society. All rights reserved.</accessCondition><recordInfo><recordContentSource authority="ISTEX" authorityURI="https://loaded-corpus.data.istex.fr" valueURI="https://loaded-corpus.data.istex.fr/ark:/67375/XBH-W19DTZ70-2">RSL</recordContentSource><recordOrigin>Converted from (version 1.2.10) to MODS version 3.6.</recordOrigin><recordCreationDate encoding="w3cdtf">2020-04-03</recordCreationDate></recordInfo></mods><json:item><extension>json</extension><original>false</original><mimetype>application/json</mimetype><uri>https://api.istex.fr/ark:/67375/V84-PCJ0MG5K-1/record.json</uri></json:item></metadata><annexes><json:item><extension>tiff</extension><original>true</original><mimetype>image/tiff</mimetype><uri>https://api.istex.fr/document/03CD6E3087F62163FAA9FBF4A67A8901D755C741/annexes/tiff</uri></json:item><json:item><extension>jpeg</extension><original>true</original><mimetype>image/jpeg</mimetype><uri>https://api.istex.fr/ark:/67375/V84-PCJ0MG5K-1/annexes.jpeg</uri></json:item><json:item><extension>gif</extension><original>true</original><mimetype>image/gif</mimetype><uri>https://api.istex.fr/ark:/67375/V84-PCJ0MG5K-1/annexes.gif</uri></json:item></annexes><serie></serie></istex></record>Pour manipuler ce document sous Unix (Dilib)
EXPLOR_STEP=$WICRI_ROOT/Sante/explor/SrasV1/Data/Istex/Corpus
HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 001F95 | SxmlIndent | more
Ou
HfdSelect -h $EXPLOR_AREA/Data/Istex/Corpus/biblio.hfd -nk 001F95 | SxmlIndent | more
Pour mettre un lien sur cette page dans le réseau Wicri
{{Explor lien
|wiki= Sante
|area= SrasV1
|flux= Istex
|étape= Corpus
|type= RBID
|clé= ISTEX:03CD6E3087F62163FAA9FBF4A67A8901D755C741
|texte= Sensitivity analysis of infectious disease models: methods, advances and their application
}}
| This area was generated with Dilib version V0.6.33. |  |