Serveur d'exploration Cyberinfrastructure

Attention, ce site est en cours de développement !
Attention, site généré par des moyens informatiques à partir de corpus bruts.
Les informations ne sont donc pas validées.

Sensitivity Analysis of an ENteric Immunity SImulator (ENISI)-Based Model of Immune Responses to Helicobacter pylori Infection

Identifieur interne : 000078 ( Pmc/Corpus ); précédent : 000077; suivant : 000079

Sensitivity Analysis of an ENteric Immunity SImulator (ENISI)-Based Model of Immune Responses to Helicobacter pylori Infection

Auteurs : Maksudul Alam ; Xinwei Deng ; Casandra Philipson ; Josep Bassaganya-Riera ; Keith Bisset ; Adria Carbo ; Stephen Eubank ; Raquel Hontecillas ; Stefan Hoops ; Yongguo Mei ; Vida Abedi ; Madhav Marathe

Source :

RBID : PMC:4556515

Abstract

Agent-based models (ABM) are widely used to study immune systems, providing a procedural and interactive view of the underlying system. The interaction of components and the behavior of individual objects is described procedurally as a function of the internal states and the local interactions, which are often stochastic in nature. Such models typically have complex structures and consist of a large number of modeling parameters. Determining the key modeling parameters which govern the outcomes of the system is very challenging. Sensitivity analysis plays a vital role in quantifying the impact of modeling parameters in massively interacting systems, including large complex ABM. The high computational cost of executing simulations impedes running experiments with exhaustive parameter settings. Existing techniques of analyzing such a complex system typically focus on local sensitivity analysis, i.e. one parameter at a time, or a close “neighborhood” of particular parameter settings. However, such methods are not adequate to measure the uncertainty and sensitivity of parameters accurately because they overlook the global impacts of parameters on the system. In this article, we develop novel experimental design and analysis techniques to perform both global and local sensitivity analysis of large-scale ABMs. The proposed method can efficiently identify the most significant parameters and quantify their contributions to outcomes of the system. We demonstrate the proposed methodology for ENteric Immune SImulator (ENISI), a large-scale ABM environment, using a computational model of immune responses to Helicobacter pylori colonization of the gastric mucosa.


Url:
DOI: 10.1371/journal.pone.0136139
PubMed: 26327290
PubMed Central: 4556515

Links to Exploration step

PMC:4556515

Le document en format XML

<record>
<TEI>
<teiHeader>
<fileDesc>
<titleStmt>
<title xml:lang="en">Sensitivity Analysis of an ENteric Immunity SImulator (ENISI)-Based Model of Immune Responses to
<italic>Helicobacter pylori</italic>
Infection</title>
<author>
<name sortKey="Alam, Maksudul" sort="Alam, Maksudul" uniqKey="Alam M" first="Maksudul" last="Alam">Maksudul Alam</name>
<affiliation>
<nlm:aff id="aff001">
<addr-line>Network Dynamics and Simulation Science Laboratory, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
<affiliation>
<nlm:aff id="aff003">
<addr-line>Center for Modeling Immunity to Enteric Pathogens, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
<affiliation>
<nlm:aff id="aff005">
<addr-line>Department of Computer Science, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
</author>
<author>
<name sortKey="Deng, Xinwei" sort="Deng, Xinwei" uniqKey="Deng X" first="Xinwei" last="Deng">Xinwei Deng</name>
<affiliation>
<nlm:aff id="aff003">
<addr-line>Center for Modeling Immunity to Enteric Pathogens, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
<affiliation>
<nlm:aff id="aff006">
<addr-line>Department of Statistics, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
</author>
<author>
<name sortKey="Philipson, Casandra" sort="Philipson, Casandra" uniqKey="Philipson C" first="Casandra" last="Philipson">Casandra Philipson</name>
<affiliation>
<nlm:aff id="aff002">
<addr-line>Nutritional Immunology and Molecular Medicine Laboratory, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
<affiliation>
<nlm:aff id="aff003">
<addr-line>Center for Modeling Immunity to Enteric Pathogens, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
</author>
<author>
<name sortKey="Bassaganya Riera, Josep" sort="Bassaganya Riera, Josep" uniqKey="Bassaganya Riera J" first="Josep" last="Bassaganya-Riera">Josep Bassaganya-Riera</name>
<affiliation>
<nlm:aff id="aff002">
<addr-line>Nutritional Immunology and Molecular Medicine Laboratory, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
<affiliation>
<nlm:aff id="aff003">
<addr-line>Center for Modeling Immunity to Enteric Pathogens, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
<affiliation>
<nlm:aff id="aff004">
<addr-line>Department of Biomedical Sciences and Pathobiology, Virginia-Maryland Regional College of Veterinary Medicine, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
</author>
<author>
<name sortKey="Bisset, Keith" sort="Bisset, Keith" uniqKey="Bisset K" first="Keith" last="Bisset">Keith Bisset</name>
<affiliation>
<nlm:aff id="aff001">
<addr-line>Network Dynamics and Simulation Science Laboratory, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
<affiliation>
<nlm:aff id="aff003">
<addr-line>Center for Modeling Immunity to Enteric Pathogens, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
</author>
<author>
<name sortKey="Carbo, Adria" sort="Carbo, Adria" uniqKey="Carbo A" first="Adria" last="Carbo">Adria Carbo</name>
<affiliation>
<nlm:aff id="aff002">
<addr-line>Nutritional Immunology and Molecular Medicine Laboratory, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
<affiliation>
<nlm:aff id="aff003">
<addr-line>Center for Modeling Immunity to Enteric Pathogens, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
</author>
<author>
<name sortKey="Eubank, Stephen" sort="Eubank, Stephen" uniqKey="Eubank S" first="Stephen" last="Eubank">Stephen Eubank</name>
<affiliation>
<nlm:aff id="aff001">
<addr-line>Network Dynamics and Simulation Science Laboratory, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
<affiliation>
<nlm:aff id="aff003">
<addr-line>Center for Modeling Immunity to Enteric Pathogens, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
</author>
<author>
<name sortKey="Hontecillas, Raquel" sort="Hontecillas, Raquel" uniqKey="Hontecillas R" first="Raquel" last="Hontecillas">Raquel Hontecillas</name>
<affiliation>
<nlm:aff id="aff002">
<addr-line>Nutritional Immunology and Molecular Medicine Laboratory, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
<affiliation>
<nlm:aff id="aff003">
<addr-line>Center for Modeling Immunity to Enteric Pathogens, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
</author>
<author>
<name sortKey="Hoops, Stefan" sort="Hoops, Stefan" uniqKey="Hoops S" first="Stefan" last="Hoops">Stefan Hoops</name>
<affiliation>
<nlm:aff id="aff002">
<addr-line>Nutritional Immunology and Molecular Medicine Laboratory, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
<affiliation>
<nlm:aff id="aff003">
<addr-line>Center for Modeling Immunity to Enteric Pathogens, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
</author>
<author>
<name sortKey="Mei, Yongguo" sort="Mei, Yongguo" uniqKey="Mei Y" first="Yongguo" last="Mei">Yongguo Mei</name>
<affiliation>
<nlm:aff id="aff002">
<addr-line>Nutritional Immunology and Molecular Medicine Laboratory, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
<affiliation>
<nlm:aff id="aff003">
<addr-line>Center for Modeling Immunity to Enteric Pathogens, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
</author>
<author>
<name sortKey="Abedi, Vida" sort="Abedi, Vida" uniqKey="Abedi V" first="Vida" last="Abedi">Vida Abedi</name>
<affiliation>
<nlm:aff id="aff002">
<addr-line>Nutritional Immunology and Molecular Medicine Laboratory, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
<affiliation>
<nlm:aff id="aff003">
<addr-line>Center for Modeling Immunity to Enteric Pathogens, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
</author>
<author>
<name sortKey="Marathe, Madhav" sort="Marathe, Madhav" uniqKey="Marathe M" first="Madhav" last="Marathe">Madhav Marathe</name>
<affiliation>
<nlm:aff id="aff001">
<addr-line>Network Dynamics and Simulation Science Laboratory, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
<affiliation>
<nlm:aff id="aff003">
<addr-line>Center for Modeling Immunity to Enteric Pathogens, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
<affiliation>
<nlm:aff id="aff005">
<addr-line>Department of Computer Science, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
</author>
</titleStmt>
<publicationStmt>
<idno type="wicri:source">PMC</idno>
<idno type="pmid">26327290</idno>
<idno type="pmc">4556515</idno>
<idno type="url">http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4556515</idno>
<idno type="RBID">PMC:4556515</idno>
<idno type="doi">10.1371/journal.pone.0136139</idno>
<date when="2015">2015</date>
<idno type="wicri:Area/Pmc/Corpus">000078</idno>
</publicationStmt>
<sourceDesc>
<biblStruct>
<analytic>
<title xml:lang="en" level="a" type="main">Sensitivity Analysis of an ENteric Immunity SImulator (ENISI)-Based Model of Immune Responses to
<italic>Helicobacter pylori</italic>
Infection</title>
<author>
<name sortKey="Alam, Maksudul" sort="Alam, Maksudul" uniqKey="Alam M" first="Maksudul" last="Alam">Maksudul Alam</name>
<affiliation>
<nlm:aff id="aff001">
<addr-line>Network Dynamics and Simulation Science Laboratory, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
<affiliation>
<nlm:aff id="aff003">
<addr-line>Center for Modeling Immunity to Enteric Pathogens, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
<affiliation>
<nlm:aff id="aff005">
<addr-line>Department of Computer Science, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
</author>
<author>
<name sortKey="Deng, Xinwei" sort="Deng, Xinwei" uniqKey="Deng X" first="Xinwei" last="Deng">Xinwei Deng</name>
<affiliation>
<nlm:aff id="aff003">
<addr-line>Center for Modeling Immunity to Enteric Pathogens, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
<affiliation>
<nlm:aff id="aff006">
<addr-line>Department of Statistics, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
</author>
<author>
<name sortKey="Philipson, Casandra" sort="Philipson, Casandra" uniqKey="Philipson C" first="Casandra" last="Philipson">Casandra Philipson</name>
<affiliation>
<nlm:aff id="aff002">
<addr-line>Nutritional Immunology and Molecular Medicine Laboratory, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
<affiliation>
<nlm:aff id="aff003">
<addr-line>Center for Modeling Immunity to Enteric Pathogens, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
</author>
<author>
<name sortKey="Bassaganya Riera, Josep" sort="Bassaganya Riera, Josep" uniqKey="Bassaganya Riera J" first="Josep" last="Bassaganya-Riera">Josep Bassaganya-Riera</name>
<affiliation>
<nlm:aff id="aff002">
<addr-line>Nutritional Immunology and Molecular Medicine Laboratory, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
<affiliation>
<nlm:aff id="aff003">
<addr-line>Center for Modeling Immunity to Enteric Pathogens, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
<affiliation>
<nlm:aff id="aff004">
<addr-line>Department of Biomedical Sciences and Pathobiology, Virginia-Maryland Regional College of Veterinary Medicine, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
</author>
<author>
<name sortKey="Bisset, Keith" sort="Bisset, Keith" uniqKey="Bisset K" first="Keith" last="Bisset">Keith Bisset</name>
<affiliation>
<nlm:aff id="aff001">
<addr-line>Network Dynamics and Simulation Science Laboratory, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
<affiliation>
<nlm:aff id="aff003">
<addr-line>Center for Modeling Immunity to Enteric Pathogens, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
</author>
<author>
<name sortKey="Carbo, Adria" sort="Carbo, Adria" uniqKey="Carbo A" first="Adria" last="Carbo">Adria Carbo</name>
<affiliation>
<nlm:aff id="aff002">
<addr-line>Nutritional Immunology and Molecular Medicine Laboratory, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
<affiliation>
<nlm:aff id="aff003">
<addr-line>Center for Modeling Immunity to Enteric Pathogens, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
</author>
<author>
<name sortKey="Eubank, Stephen" sort="Eubank, Stephen" uniqKey="Eubank S" first="Stephen" last="Eubank">Stephen Eubank</name>
<affiliation>
<nlm:aff id="aff001">
<addr-line>Network Dynamics and Simulation Science Laboratory, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
<affiliation>
<nlm:aff id="aff003">
<addr-line>Center for Modeling Immunity to Enteric Pathogens, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
</author>
<author>
<name sortKey="Hontecillas, Raquel" sort="Hontecillas, Raquel" uniqKey="Hontecillas R" first="Raquel" last="Hontecillas">Raquel Hontecillas</name>
<affiliation>
<nlm:aff id="aff002">
<addr-line>Nutritional Immunology and Molecular Medicine Laboratory, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
<affiliation>
<nlm:aff id="aff003">
<addr-line>Center for Modeling Immunity to Enteric Pathogens, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
</author>
<author>
<name sortKey="Hoops, Stefan" sort="Hoops, Stefan" uniqKey="Hoops S" first="Stefan" last="Hoops">Stefan Hoops</name>
<affiliation>
<nlm:aff id="aff002">
<addr-line>Nutritional Immunology and Molecular Medicine Laboratory, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
<affiliation>
<nlm:aff id="aff003">
<addr-line>Center for Modeling Immunity to Enteric Pathogens, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
</author>
<author>
<name sortKey="Mei, Yongguo" sort="Mei, Yongguo" uniqKey="Mei Y" first="Yongguo" last="Mei">Yongguo Mei</name>
<affiliation>
<nlm:aff id="aff002">
<addr-line>Nutritional Immunology and Molecular Medicine Laboratory, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
<affiliation>
<nlm:aff id="aff003">
<addr-line>Center for Modeling Immunity to Enteric Pathogens, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
</author>
<author>
<name sortKey="Abedi, Vida" sort="Abedi, Vida" uniqKey="Abedi V" first="Vida" last="Abedi">Vida Abedi</name>
<affiliation>
<nlm:aff id="aff002">
<addr-line>Nutritional Immunology and Molecular Medicine Laboratory, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
<affiliation>
<nlm:aff id="aff003">
<addr-line>Center for Modeling Immunity to Enteric Pathogens, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
</author>
<author>
<name sortKey="Marathe, Madhav" sort="Marathe, Madhav" uniqKey="Marathe M" first="Madhav" last="Marathe">Madhav Marathe</name>
<affiliation>
<nlm:aff id="aff001">
<addr-line>Network Dynamics and Simulation Science Laboratory, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
<affiliation>
<nlm:aff id="aff003">
<addr-line>Center for Modeling Immunity to Enteric Pathogens, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
<affiliation>
<nlm:aff id="aff005">
<addr-line>Department of Computer Science, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</nlm:aff>
</affiliation>
</author>
</analytic>
<series>
<title level="j">PLoS ONE</title>
<idno type="eISSN">1932-6203</idno>
<imprint>
<date when="2015">2015</date>
</imprint>
</series>
</biblStruct>
</sourceDesc>
</fileDesc>
<profileDesc>
<textClass></textClass>
</profileDesc>
</teiHeader>
<front>
<div type="abstract" xml:lang="en">
<p>Agent-based models (ABM) are widely used to study immune systems, providing a procedural and interactive view of the underlying system. The interaction of components and the behavior of individual objects is described procedurally as a function of the internal states and the local interactions, which are often stochastic in nature. Such models typically have complex structures and consist of a large number of modeling parameters. Determining the key modeling parameters which govern the outcomes of the system is very challenging. Sensitivity analysis plays a vital role in quantifying the impact of modeling parameters in massively interacting systems, including large complex ABM. The high computational cost of executing simulations impedes running experiments with exhaustive parameter settings. Existing techniques of analyzing such a complex system typically focus on local sensitivity analysis, i.e. one parameter at a time, or a close “neighborhood” of particular parameter settings. However, such methods are not adequate to measure the uncertainty and sensitivity of parameters accurately because they overlook the global impacts of parameters on the system. In this article, we develop novel experimental design and analysis techniques to perform both global and local sensitivity analysis of large-scale ABMs. The proposed method can efficiently identify the most significant parameters and quantify their contributions to outcomes of the system. We demonstrate the proposed methodology for ENteric Immune SImulator (ENISI), a large-scale ABM environment, using a computational model of immune responses to
<italic>Helicobacter pylori</italic>
colonization of the gastric mucosa.</p>
</div>
</front>
<back>
<div1 type="bibliography">
<listBibl>
<biblStruct>
<analytic>
<author>
<name sortKey="Saltelli, A" uniqKey="Saltelli A">A Saltelli</name>
</author>
<author>
<name sortKey="Annoni, P" uniqKey="Annoni P">P Annoni</name>
</author>
<author>
<name sortKey="Azzini, I" uniqKey="Azzini I">I Azzini</name>
</author>
<author>
<name sortKey="Campolongo, F" uniqKey="Campolongo F">F Campolongo</name>
</author>
<author>
<name sortKey="Ratto, M" uniqKey="Ratto M">M Ratto</name>
</author>
<author>
<name sortKey="Tarantola, S" uniqKey="Tarantola S">S Tarantola</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Helton, Jc" uniqKey="Helton J">JC Helton</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Cacuci, Dg" uniqKey="Cacuci D">DG Cacuci</name>
</author>
<author>
<name sortKey="Ionescu Bujor, M" uniqKey="Ionescu Bujor M">M Ionescu-Bujor</name>
</author>
<author>
<name sortKey="Navon, Im" uniqKey="Navon I">IM Navon</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Brooks, Rj" uniqKey="Brooks R">RJ Brooks</name>
</author>
<author>
<name sortKey="Semenov, Ma" uniqKey="Semenov M">MA Semenov</name>
</author>
<author>
<name sortKey="Jamieson, Pd" uniqKey="Jamieson P">PD Jamieson</name>
</author>
</analytic>
</biblStruct>
<biblStruct></biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Sher, Aa" uniqKey="Sher A">AA Sher</name>
</author>
<author>
<name sortKey="Wang, K" uniqKey="Wang K">K Wang</name>
</author>
<author>
<name sortKey="Wathen, A" uniqKey="Wathen A">A Wathen</name>
</author>
<author>
<name sortKey="Maybank, Pj" uniqKey="Maybank P">PJ Maybank</name>
</author>
<author>
<name sortKey="Mirams, Gr" uniqKey="Mirams G">GR Mirams</name>
</author>
<author>
<name sortKey="Abramson, D" uniqKey="Abramson D">D Abramson</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Frey, Hc" uniqKey="Frey H">HC Frey</name>
</author>
<author>
<name sortKey="Patil, Sr" uniqKey="Patil S">SR Patil</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Hamby, Dm" uniqKey="Hamby D">DM Hamby</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Saltelli, A" uniqKey="Saltelli A">A Saltelli</name>
</author>
<author>
<name sortKey="Tarantola, S" uniqKey="Tarantola S">S Tarantola</name>
</author>
<author>
<name sortKey="Campolongo, F" uniqKey="Campolongo F">F Campolongo</name>
</author>
<author>
<name sortKey="Ratto, M" uniqKey="Ratto M">M Ratto</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Arwade, Sr" uniqKey="Arwade S">SR Arwade</name>
</author>
<author>
<name sortKey="Moradi, M" uniqKey="Moradi M">M Moradi</name>
</author>
<author>
<name sortKey="Louhghalam, A" uniqKey="Louhghalam A">A Louhghalam</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Bauer, Kw" uniqKey="Bauer K">KW Bauer</name>
</author>
<author>
<name sortKey="Parnell, Gs" uniqKey="Parnell G">GS Parnell</name>
</author>
<author>
<name sortKey="Meyers, Da" uniqKey="Meyers D">DA Meyers</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Lim, Jt" uniqKey="Lim J">JT Lim</name>
</author>
<author>
<name sortKey="Gold, Hj" uniqKey="Gold H">HJ Gold</name>
</author>
<author>
<name sortKey="Wilkerson, Gg" uniqKey="Wilkerson G">GG Wilkerson</name>
</author>
<author>
<name sortKey="Raper, Jcd" uniqKey="Raper J">JCD Raper</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Helton, Jc" uniqKey="Helton J">JC Helton</name>
</author>
<author>
<name sortKey="Johnson, Jd" uniqKey="Johnson J">JD Johnson</name>
</author>
<author>
<name sortKey="Sallaberry, Cj" uniqKey="Sallaberry C">CJ Sallaberry</name>
</author>
<author>
<name sortKey="Storlie, Cb" uniqKey="Storlie C">CB Storlie</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Bonabeau, E" uniqKey="Bonabeau E">E Bonabeau</name>
</author>
<author>
<name sortKey="Dorigo, M" uniqKey="Dorigo M">M Dorigo</name>
</author>
<author>
<name sortKey="Theraulaz, G" uniqKey="Theraulaz G">G Theraulaz</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Dong, X" uniqKey="Dong X">X Dong</name>
</author>
<author>
<name sortKey="Foteinou, Pt" uniqKey="Foteinou P">PT Foteinou</name>
</author>
<author>
<name sortKey="Calvano, Se" uniqKey="Calvano S">SE Calvano</name>
</author>
<author>
<name sortKey="Lowry, Sf" uniqKey="Lowry S">SF Lowry</name>
</author>
<author>
<name sortKey="Androulakis, Ip" uniqKey="Androulakis I">IP Androulakis</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Forrest, S" uniqKey="Forrest S">S Forrest</name>
</author>
<author>
<name sortKey="Beauchemin, C" uniqKey="Beauchemin C">C Beauchemin</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Neugebauer, Ea" uniqKey="Neugebauer E">EA Neugebauer</name>
</author>
<author>
<name sortKey="Willy, C" uniqKey="Willy C">C Willy</name>
</author>
<author>
<name sortKey="Sauerland, S" uniqKey="Sauerland S">S Sauerland</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="An, G" uniqKey="An G">G An</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Lowry, Sf" uniqKey="Lowry S">SF Lowry</name>
</author>
<author>
<name sortKey="Calvano, Se" uniqKey="Calvano S">SE Calvano</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Seely, Aj" uniqKey="Seely A">AJ Seely</name>
</author>
<author>
<name sortKey="Christou, Nv" uniqKey="Christou N">NV Christou</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="An, G" uniqKey="An G">G An</name>
</author>
<author>
<name sortKey="Faeder, J" uniqKey="Faeder J">J Faeder</name>
</author>
<author>
<name sortKey="Vodovotz, Y" uniqKey="Vodovotz Y">Y Vodovotz</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Vodovotz, Y" uniqKey="Vodovotz Y">Y Vodovotz</name>
</author>
<author>
<name sortKey="Chow, Cc" uniqKey="Chow C">CC Chow</name>
</author>
<author>
<name sortKey="Bartels, J" uniqKey="Bartels J">J Bartels</name>
</author>
<author>
<name sortKey="Lagoa, C" uniqKey="Lagoa C">C Lagoa</name>
</author>
<author>
<name sortKey="Prince, Jm" uniqKey="Prince J">JM Prince</name>
</author>
<author>
<name sortKey="Levy, Rm" uniqKey="Levy R">RM Levy</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Baldazzi, V" uniqKey="Baldazzi V">V Baldazzi</name>
</author>
<author>
<name sortKey="Castiglione, F" uniqKey="Castiglione F">F Castiglione</name>
</author>
<author>
<name sortKey="Bernaschi, M" uniqKey="Bernaschi M">M Bernaschi</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Mi, Q" uniqKey="Mi Q">Q Mi</name>
</author>
<author>
<name sortKey="Riviere, B" uniqKey="Riviere B">B Rivière</name>
</author>
<author>
<name sortKey="Clermont, G" uniqKey="Clermont G">G Clermont</name>
</author>
<author>
<name sortKey="Steed, Dl" uniqKey="Steed D">DL Steed</name>
</author>
<author>
<name sortKey="Vodovotz, Y" uniqKey="Vodovotz Y">Y Vodovotz</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="An, G" uniqKey="An G">G An</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="An, G" uniqKey="An G">G An</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Shapiro, M" uniqKey="Shapiro M">M Shapiro</name>
</author>
<author>
<name sortKey="Duca, Ka" uniqKey="Duca K">KA Duca</name>
</author>
<author>
<name sortKey="Lee, K" uniqKey="Lee K">K Lee</name>
</author>
<author>
<name sortKey="Delgado Eckert, E" uniqKey="Delgado Eckert E">E Delgado-Eckert</name>
</author>
<author>
<name sortKey="Hawkins, J" uniqKey="Hawkins J">J Hawkins</name>
</author>
<author>
<name sortKey="Jarrah, As" uniqKey="Jarrah A">AS Jarrah</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Duca, Ka" uniqKey="Duca K">KA Duca</name>
</author>
<author>
<name sortKey="Shapiro, M" uniqKey="Shapiro M">M Shapiro</name>
</author>
<author>
<name sortKey="Delgado Eckert, E" uniqKey="Delgado Eckert E">E Delgado-Eckert</name>
</author>
<author>
<name sortKey="Hadinoto, V" uniqKey="Hadinoto V">V Hadinoto</name>
</author>
<author>
<name sortKey="Jarrah, As" uniqKey="Jarrah A">AS Jarrah</name>
</author>
<author>
<name sortKey="Laubenbacher, R" uniqKey="Laubenbacher R">R Laubenbacher</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Bauer, Al" uniqKey="Bauer A">AL Bauer</name>
</author>
<author>
<name sortKey="Beauchemin, Caa" uniqKey="Beauchemin C">CAA Beauchemin</name>
</author>
<author>
<name sortKey="Perelson, As" uniqKey="Perelson A">AS Perelson</name>
</author>
</analytic>
</biblStruct>
<biblStruct></biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Barrett, Cl" uniqKey="Barrett C">CL Barrett</name>
</author>
<author>
<name sortKey="Bisset, K" uniqKey="Bisset K">K Bisset</name>
</author>
<author>
<name sortKey="Chen, J" uniqKey="Chen J">J Chen</name>
</author>
<author>
<name sortKey="Eubank, S" uniqKey="Eubank S">S Eubank</name>
</author>
<author>
<name sortKey="Lewis, B" uniqKey="Lewis B">B Lewis</name>
</author>
<author>
<name sortKey="Kumar, Vsa" uniqKey="Kumar V">VSA Kumar</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Eubank, S" uniqKey="Eubank S">S Eubank</name>
</author>
<author>
<name sortKey="Guclu, H" uniqKey="Guclu H">H Guclu</name>
</author>
<author>
<name sortKey="Kumar, Vsa" uniqKey="Kumar V">VSA Kumar</name>
</author>
<author>
<name sortKey="Marathe, M" uniqKey="Marathe M">M Marathe</name>
</author>
<author>
<name sortKey="Srinivasan, A" uniqKey="Srinivasan A">A Srinivasan</name>
</author>
<author>
<name sortKey="Toroczkai, Z" uniqKey="Toroczkai Z">Z Toroczkai</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Barrett, C" uniqKey="Barrett C">C Barrett</name>
</author>
<author>
<name sortKey="Beckman, R" uniqKey="Beckman R">R Beckman</name>
</author>
<author>
<name sortKey="Berkbigler, K" uniqKey="Berkbigler K">K Berkbigler</name>
</author>
<author>
<name sortKey="Bisset, K" uniqKey="Bisset K">K Bisset</name>
</author>
<author>
<name sortKey="Bush, B" uniqKey="Bush B">B Bush</name>
</author>
<author>
<name sortKey="Campbell, K" uniqKey="Campbell K">K Campbell</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Barrett, E" uniqKey="Barrett E">E Barrett</name>
</author>
<author>
<name sortKey="Ross, Rp" uniqKey="Ross R">RP Ross</name>
</author>
<author>
<name sortKey="Fitzgerald, Gf" uniqKey="Fitzgerald G">GF Fitzgerald</name>
</author>
<author>
<name sortKey="Stanton, C" uniqKey="Stanton C">C Stanton</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Karaoz, U" uniqKey="Karaoz U">U Karaoz</name>
</author>
<author>
<name sortKey="Murali, Tm" uniqKey="Murali T">TM Murali</name>
</author>
<author>
<name sortKey="Letovsky, S" uniqKey="Letovsky S">S Letovsky</name>
</author>
<author>
<name sortKey="Zheng, Y" uniqKey="Zheng Y">Y Zheng</name>
</author>
<author>
<name sortKey="Ding, C" uniqKey="Ding C">C Ding</name>
</author>
<author>
<name sortKey="Cantor, Cr" uniqKey="Cantor C">CR Cantor</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Wendelsdorf, Kv" uniqKey="Wendelsdorf K">KV Wendelsdorf</name>
</author>
<author>
<name sortKey="Bassaganya Riera, J" uniqKey="Bassaganya Riera J">J Bassaganya-Riera</name>
</author>
<author>
<name sortKey="Bisset, K" uniqKey="Bisset K">K Bisset</name>
</author>
<author>
<name sortKey="Eubank, S" uniqKey="Eubank S">S Eubank</name>
</author>
<author>
<name sortKey="Hontecillas, R" uniqKey="Hontecillas R">R Hontecillas</name>
</author>
<author>
<name sortKey="Marathe, M" uniqKey="Marathe M">M Marathe</name>
</author>
</analytic>
</biblStruct>
<biblStruct></biblStruct>
<biblStruct></biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Wendelsdorf, Kv" uniqKey="Wendelsdorf K">KV Wendelsdorf</name>
</author>
<author>
<name sortKey="Alam, M" uniqKey="Alam M">M Alam</name>
</author>
<author>
<name sortKey="Bassaganya Riera, J" uniqKey="Bassaganya Riera J">J Bassaganya-Riera</name>
</author>
<author>
<name sortKey="Bisset, K" uniqKey="Bisset K">K Bisset</name>
</author>
<author>
<name sortKey="Eubank, S" uniqKey="Eubank S">S Eubank</name>
</author>
<author>
<name sortKey="Hontecillas, R" uniqKey="Hontecillas R">R Hontecillas</name>
</author>
</analytic>
</biblStruct>
<biblStruct></biblStruct>
<biblStruct></biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Mei, Y" uniqKey="Mei Y">Y Mei</name>
</author>
<author>
<name sortKey="Abedi, V" uniqKey="Abedi V">V Abedi</name>
</author>
<author>
<name sortKey="Carbo, A" uniqKey="Carbo A">A Carbo</name>
</author>
<author>
<name sortKey="Zhang, X" uniqKey="Zhang X">X Zhang</name>
</author>
<author>
<name sortKey="Lu, P" uniqKey="Lu P">P Lu</name>
</author>
<author>
<name sortKey="Philipson, C" uniqKey="Philipson C">C Philipson</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Kelly, D" uniqKey="Kelly D">D Kelly</name>
</author>
<author>
<name sortKey="Campbell, Ji" uniqKey="Campbell J">JI Campbell</name>
</author>
<author>
<name sortKey="King, Tp" uniqKey="King T">TP King</name>
</author>
<author>
<name sortKey="Grant, G" uniqKey="Grant G">G Grant</name>
</author>
<author>
<name sortKey="Jansson, Ea" uniqKey="Jansson E">EA Jansson</name>
</author>
<author>
<name sortKey="Coutts, Ag" uniqKey="Coutts A">AG Coutts</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Marzo, Al" uniqKey="Marzo A">AL Marzo</name>
</author>
<author>
<name sortKey="Klonowski, Kd" uniqKey="Klonowski K">KD Klonowski</name>
</author>
<author>
<name sortKey="Bon, Al" uniqKey="Bon A">AL Bon</name>
</author>
<author>
<name sortKey="Borrow, P" uniqKey="Borrow P">P Borrow</name>
</author>
<author>
<name sortKey="Tough, Df" uniqKey="Tough D">DF Tough</name>
</author>
<author>
<name sortKey="Lefrancois, L" uniqKey="Lefrancois L">L Lefrancois</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Lanzavecchia, A" uniqKey="Lanzavecchia A">A Lanzavecchia</name>
</author>
<author>
<name sortKey="Sallusto, F" uniqKey="Sallusto F">F Sallusto</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Sallusto, F" uniqKey="Sallusto F">F Sallusto</name>
</author>
<author>
<name sortKey="Geginat, J" uniqKey="Geginat J">J Geginat</name>
</author>
<author>
<name sortKey="Lanzavecchia, A" uniqKey="Lanzavecchia A">A Lanzavecchia</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Kaech, Sm" uniqKey="Kaech S">SM Kaech</name>
</author>
<author>
<name sortKey="Ahmed, R" uniqKey="Ahmed R">R Ahmed</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Naresh, R" uniqKey="Naresh R">R Naresh</name>
</author>
<author>
<name sortKey="Song, Y" uniqKey="Song Y">Y Song</name>
</author>
<author>
<name sortKey="Hampson, Dj" uniqKey="Hampson D">DJ Hampson</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Carbo, A" uniqKey="Carbo A">A Carbo</name>
</author>
<author>
<name sortKey="Hontecillas, R" uniqKey="Hontecillas R">R Hontecillas</name>
</author>
<author>
<name sortKey="Kronsteiner, B" uniqKey="Kronsteiner B">B Kronsteiner</name>
</author>
<author>
<name sortKey="Viladomiu, M" uniqKey="Viladomiu M">M Viladomiu</name>
</author>
<author>
<name sortKey="Pedragosa, M" uniqKey="Pedragosa M">M Pedragosa</name>
</author>
<author>
<name sortKey="Lu, P" uniqKey="Lu P">P Lu</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Carbo, A" uniqKey="Carbo A">A Carbo</name>
</author>
<author>
<name sortKey="Hontecillas, R" uniqKey="Hontecillas R">R Hontecillas</name>
</author>
<author>
<name sortKey="Andrew, T" uniqKey="Andrew T">T Andrew</name>
</author>
<author>
<name sortKey="Eden, K" uniqKey="Eden K">K Eden</name>
</author>
<author>
<name sortKey="Mei, Y" uniqKey="Mei Y">Y Mei</name>
</author>
<author>
<name sortKey="Hoops, S" uniqKey="Hoops S">S Hoops</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Carbo, A" uniqKey="Carbo A">A Carbo</name>
</author>
<author>
<name sortKey="Olivares Villagomez, D" uniqKey="Olivares Villagomez D">D Olivares-Villagomez</name>
</author>
<author>
<name sortKey="Hontecillas, R" uniqKey="Hontecillas R">R Hontecillas</name>
</author>
<author>
<name sortKey="Bassaganya Riera, J" uniqKey="Bassaganya Riera J">J Bassaganya-Riera</name>
</author>
<author>
<name sortKey="Chaturvedi, R" uniqKey="Chaturvedi R">R Chaturvedi</name>
</author>
<author>
<name sortKey="Piazuelo, Mb" uniqKey="Piazuelo M">MB Piazuelo</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Carbo, A" uniqKey="Carbo A">A Carbo</name>
</author>
<author>
<name sortKey="Hontecillas, R" uniqKey="Hontecillas R">R Hontecillas</name>
</author>
<author>
<name sortKey="Viladomiu, M" uniqKey="Viladomiu M">M Viladomiu</name>
</author>
<author>
<name sortKey="Leber, A" uniqKey="Leber A">A Leber</name>
</author>
<author>
<name sortKey="Hoops, S" uniqKey="Hoops S">S Hoops</name>
</author>
<author>
<name sortKey="Bassaganya Riera, J" uniqKey="Bassaganya Riera J">J Bassaganya-Riera</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Leber, A" uniqKey="Leber A">A Leber</name>
</author>
<author>
<name sortKey="Viladomiu, M" uniqKey="Viladomiu M">M Viladomiu</name>
</author>
<author>
<name sortKey="Hontecillas, R" uniqKey="Hontecillas R">R Hontecillas</name>
</author>
<author>
<name sortKey="Abedi, V" uniqKey="Abedi V">V Abedi</name>
</author>
<author>
<name sortKey="Philipson, C" uniqKey="Philipson C">C Philipson</name>
</author>
<author>
<name sortKey="Hoops, S" uniqKey="Hoops S">S Hoops</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Sumner, T" uniqKey="Sumner T">T Sumner</name>
</author>
<author>
<name sortKey="Hetherington, J" uniqKey="Hetherington J">J Hetherington</name>
</author>
<author>
<name sortKey="Seymour, Rm" uniqKey="Seymour R">RM Seymour</name>
</author>
<author>
<name sortKey="Li, L" uniqKey="Li L">L Li</name>
</author>
<author>
<name sortKey="Rey, Mv" uniqKey="Rey M">MV Rey</name>
</author>
<author>
<name sortKey="Yamaji, S" uniqKey="Yamaji S">S Yamaji</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Sumner, T" uniqKey="Sumner T">T Sumner</name>
</author>
<author>
<name sortKey="Shephard, E" uniqKey="Shephard E">E Shephard</name>
</author>
<author>
<name sortKey="Bogle, Id" uniqKey="Bogle I">ID Bogle</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Sumner, T" uniqKey="Sumner T">T Sumner</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Broeke, Gat" uniqKey="Broeke G">GAT Broeke</name>
</author>
<author>
<name sortKey="Voorn, Gakv" uniqKey="Voorn G">GAKv Voorn</name>
</author>
<author>
<name sortKey="Ligtenberg, A" uniqKey="Ligtenberg A">A Ligtenberg</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Schouten, M" uniqKey="Schouten M">M Schouten</name>
</author>
<author>
<name sortKey="Verwaart, T" uniqKey="Verwaart T">T Verwaart</name>
</author>
<author>
<name sortKey="Heijman, W" uniqKey="Heijman W">W Heijman</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Ligmann Zielinska, A" uniqKey="Ligmann Zielinska A">A Ligmann-Zielinska</name>
</author>
<author>
<name sortKey="Kramer, Db" uniqKey="Kramer D">DB Kramer</name>
</author>
<author>
<name sortKey="Cheruvelil, Ks" uniqKey="Cheruvelil K">KS Cheruvelil</name>
</author>
<author>
<name sortKey="Soranno, Pa" uniqKey="Soranno P">PA Soranno</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Parry, Hr" uniqKey="Parry H">HR Parry</name>
</author>
<author>
<name sortKey="Topping, Cj" uniqKey="Topping C">CJ Topping</name>
</author>
<author>
<name sortKey="Kennedy, Mc" uniqKey="Kennedy M">MC Kennedy</name>
</author>
<author>
<name sortKey="Boatman, Nd" uniqKey="Boatman N">ND Boatman</name>
</author>
<author>
<name sortKey="Murray, Awa" uniqKey="Murray A">AWA Murray</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Fonoberova, M" uniqKey="Fonoberova M">M Fonoberova</name>
</author>
<author>
<name sortKey="Fonoberov, Va" uniqKey="Fonoberov V">VA Fonoberov</name>
</author>
<author>
<name sortKey="Mezi, I" uniqKey="Mezi I">I Mezić</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Kandulski, A" uniqKey="Kandulski A">A Kandulski</name>
</author>
<author>
<name sortKey="Malfertheiner, P" uniqKey="Malfertheiner P">P Malfertheiner</name>
</author>
<author>
<name sortKey="Wex, T" uniqKey="Wex T">T Wex</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Ramarao, N" uniqKey="Ramarao N">N Ramarao</name>
</author>
<author>
<name sortKey="Meyer, Tf" uniqKey="Meyer T">TF Meyer</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Carbo, A" uniqKey="Carbo A">A Carbo</name>
</author>
<author>
<name sortKey="Bassaganya Riera, J" uniqKey="Bassaganya Riera J">J Bassaganya-Riera</name>
</author>
<author>
<name sortKey="Pedragosa, M" uniqKey="Pedragosa M">M Pedragosa</name>
</author>
<author>
<name sortKey="Viladomiu, M" uniqKey="Viladomiu M">M Viladomiu</name>
</author>
<author>
<name sortKey="Marathe, M" uniqKey="Marathe M">M Marathe</name>
</author>
<author>
<name sortKey="Eubank, S" uniqKey="Eubank S">S Eubank</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Wu, Cj" uniqKey="Wu C">CJ Wu</name>
</author>
<author>
<name sortKey="Hamada, Ms" uniqKey="Hamada M">MS Hamada</name>
</author>
</analytic>
</biblStruct>
<biblStruct></biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Rodriguez Fernandez, M" uniqKey="Rodriguez Fernandez M">M Rodriguez-Fernandez</name>
</author>
<author>
<name sortKey="Banga, Jr" uniqKey="Banga J">JR Banga</name>
</author>
<author>
<name sortKey="Doyle, Fj" uniqKey="Doyle F">FJ Doyle</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Box, Ge" uniqKey="Box G">GE Box</name>
</author>
<author>
<name sortKey="Hunter, Js" uniqKey="Hunter J">JS Hunter</name>
</author>
<author>
<name sortKey="Hunter, Wg" uniqKey="Hunter W">WG Hunter</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Kleijnen, Jpc" uniqKey="Kleijnen J">JPC Kleijnen</name>
</author>
<author>
<name sortKey="Groenendaal, Wv" uniqKey="Groenendaal W">Wv Groenendaal</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Law, Am" uniqKey="Law A">AM Law</name>
</author>
<author>
<name sortKey="Kelton, Dm" uniqKey="Kelton D">DM Kelton</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Noordegraaf, Av" uniqKey="Noordegraaf A">AV Noordegraaf</name>
</author>
<author>
<name sortKey="Nielen, M" uniqKey="Nielen M">M Nielen</name>
</author>
<author>
<name sortKey="Kleijnen, Jpc" uniqKey="Kleijnen J">JPC Kleijnen</name>
</author>
</analytic>
</biblStruct>
<biblStruct></biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Montgomery, Dc" uniqKey="Montgomery D">DC Montgomery</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Hedayat, As" uniqKey="Hedayat A">AS Hedayat</name>
</author>
<author>
<name sortKey="Sloane, Nja" uniqKey="Sloane N">NJA Sloane</name>
</author>
<author>
<name sortKey="Stufken, J" uniqKey="Stufken J">J Stufken</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Morris, Md" uniqKey="Morris M">MD Morris</name>
</author>
<author>
<name sortKey="Moore, Lm" uniqKey="Moore L">LM Moore</name>
</author>
<author>
<name sortKey="Mckay, Md" uniqKey="Mckay M">MD McKay</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Scheffe, H" uniqKey="Scheffe H">H Scheffé</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Seber, Gaf" uniqKey="Seber G">GAF Seber</name>
</author>
<author>
<name sortKey="Lee, Aj" uniqKey="Lee A">AJ Lee</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Bassaganya Riera, J" uniqKey="Bassaganya Riera J">J Bassaganya-Riera</name>
</author>
<author>
<name sortKey="Dominguez Bello, Mg" uniqKey="Dominguez Bello M">MG Dominguez-Bello</name>
</author>
<author>
<name sortKey="Kronsteiner, B" uniqKey="Kronsteiner B">B Kronsteiner</name>
</author>
<author>
<name sortKey="Carbo, A" uniqKey="Carbo A">A Carbo</name>
</author>
<author>
<name sortKey="Lu, P" uniqKey="Lu P">P Lu</name>
</author>
<author>
<name sortKey="Viladomiu, M" uniqKey="Viladomiu M">M Viladomiu</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Schumacher, Ma" uniqKey="Schumacher M">MA Schumacher</name>
</author>
<author>
<name sortKey="Donnelly, Jm" uniqKey="Donnelly J">JM Donnelly</name>
</author>
<author>
<name sortKey="Engevik, Ac" uniqKey="Engevik A">AC Engevik</name>
</author>
<author>
<name sortKey="Xiao, C" uniqKey="Xiao C">C Xiao</name>
</author>
<author>
<name sortKey="Yang, L" uniqKey="Yang L">L Yang</name>
</author>
<author>
<name sortKey="Kenny, S" uniqKey="Kenny S">S Kenny</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Kronsteiner, B" uniqKey="Kronsteiner B">B Kronsteiner</name>
</author>
<author>
<name sortKey="Bassaganya Riera, J" uniqKey="Bassaganya Riera J">J Bassaganya-Riera</name>
</author>
<author>
<name sortKey="Philipson, N" uniqKey="Philipson N">N Philipson</name>
</author>
<author>
<name sortKey="Hontecillas, R" uniqKey="Hontecillas R">R Hontecillas</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Sigal, M" uniqKey="Sigal M">M Sigal</name>
</author>
<author>
<name sortKey="Rothenberg, Me" uniqKey="Rothenberg M">ME Rothenberg</name>
</author>
<author>
<name sortKey="Logan, Cy" uniqKey="Logan C">CY Logan</name>
</author>
<author>
<name sortKey="Lee, Jy" uniqKey="Lee J">JY Lee</name>
</author>
<author>
<name sortKey="Honaker, Rw" uniqKey="Honaker R">RW Honaker</name>
</author>
<author>
<name sortKey="Cooper, Rl" uniqKey="Cooper R">RL Cooper</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Sacks, J" uniqKey="Sacks J">J Sacks</name>
</author>
<author>
<name sortKey="Welch, Wj" uniqKey="Welch W">WJ Welch</name>
</author>
<author>
<name sortKey="Mitchell, Tj" uniqKey="Mitchell T">TJ Mitchell</name>
</author>
<author>
<name sortKey="Wynn, Hp" uniqKey="Wynn H">HP Wynn</name>
</author>
</analytic>
</biblStruct>
<biblStruct>
<analytic>
<author>
<name sortKey="Santner, Tj" uniqKey="Santner T">TJ Santner</name>
</author>
<author>
<name sortKey="Williams, Bj" uniqKey="Williams B">BJ Williams</name>
</author>
<author>
<name sortKey="Notz, W" uniqKey="Notz W">W Notz</name>
</author>
</analytic>
</biblStruct>
</listBibl>
</div1>
</back>
</TEI>
<pmc article-type="research-article">
<pmc-dir>properties open_access</pmc-dir>
<front>
<journal-meta>
<journal-id journal-id-type="nlm-ta">PLoS One</journal-id>
<journal-id journal-id-type="iso-abbrev">PLoS ONE</journal-id>
<journal-id journal-id-type="publisher-id">plos</journal-id>
<journal-id journal-id-type="pmc">plosone</journal-id>
<journal-title-group>
<journal-title>PLoS ONE</journal-title>
</journal-title-group>
<issn pub-type="epub">1932-6203</issn>
<publisher>
<publisher-name>Public Library of Science</publisher-name>
<publisher-loc>San Francisco, CA USA</publisher-loc>
</publisher>
</journal-meta>
<article-meta>
<article-id pub-id-type="pmid">26327290</article-id>
<article-id pub-id-type="pmc">4556515</article-id>
<article-id pub-id-type="publisher-id">PONE-D-15-10087</article-id>
<article-id pub-id-type="doi">10.1371/journal.pone.0136139</article-id>
<article-categories>
<subj-group subj-group-type="heading">
<subject>Research Article</subject>
</subj-group>
</article-categories>
<title-group>
<article-title>Sensitivity Analysis of an ENteric Immunity SImulator (ENISI)-Based Model of Immune Responses to
<italic>Helicobacter pylori</italic>
Infection</article-title>
<alt-title alt-title-type="running-head">Sensitivity Analysis of ENISI-Based Model</alt-title>
</title-group>
<contrib-group>
<contrib contrib-type="author">
<name>
<surname>Alam</surname>
<given-names>Maksudul</given-names>
</name>
<xref ref-type="aff" rid="aff001">
<sup>1</sup>
</xref>
<xref ref-type="aff" rid="aff003">
<sup>3</sup>
</xref>
<xref ref-type="aff" rid="aff005">
<sup>5</sup>
</xref>
</contrib>
<contrib contrib-type="author">
<name>
<surname>Deng</surname>
<given-names>Xinwei</given-names>
</name>
<xref ref-type="aff" rid="aff003">
<sup>3</sup>
</xref>
<xref ref-type="aff" rid="aff006">
<sup>6</sup>
</xref>
<xref ref-type="corresp" rid="cor001">*</xref>
</contrib>
<contrib contrib-type="author">
<name>
<surname>Philipson</surname>
<given-names>Casandra</given-names>
</name>
<xref ref-type="aff" rid="aff002">
<sup>2</sup>
</xref>
<xref ref-type="aff" rid="aff003">
<sup>3</sup>
</xref>
</contrib>
<contrib contrib-type="author">
<name>
<surname>Bassaganya-Riera</surname>
<given-names>Josep</given-names>
</name>
<xref ref-type="aff" rid="aff002">
<sup>2</sup>
</xref>
<xref ref-type="aff" rid="aff003">
<sup>3</sup>
</xref>
<xref ref-type="aff" rid="aff004">
<sup>4</sup>
</xref>
</contrib>
<contrib contrib-type="author">
<name>
<surname>Bisset</surname>
<given-names>Keith</given-names>
</name>
<xref ref-type="aff" rid="aff001">
<sup>1</sup>
</xref>
<xref ref-type="aff" rid="aff003">
<sup>3</sup>
</xref>
</contrib>
<contrib contrib-type="author">
<name>
<surname>Carbo</surname>
<given-names>Adria</given-names>
</name>
<xref ref-type="aff" rid="aff002">
<sup>2</sup>
</xref>
<xref ref-type="aff" rid="aff003">
<sup>3</sup>
</xref>
</contrib>
<contrib contrib-type="author">
<name>
<surname>Eubank</surname>
<given-names>Stephen</given-names>
</name>
<xref ref-type="aff" rid="aff001">
<sup>1</sup>
</xref>
<xref ref-type="aff" rid="aff003">
<sup>3</sup>
</xref>
</contrib>
<contrib contrib-type="author">
<name>
<surname>Hontecillas</surname>
<given-names>Raquel</given-names>
</name>
<xref ref-type="aff" rid="aff002">
<sup>2</sup>
</xref>
<xref ref-type="aff" rid="aff003">
<sup>3</sup>
</xref>
</contrib>
<contrib contrib-type="author">
<name>
<surname>Hoops</surname>
<given-names>Stefan</given-names>
</name>
<xref ref-type="aff" rid="aff002">
<sup>2</sup>
</xref>
<xref ref-type="aff" rid="aff003">
<sup>3</sup>
</xref>
</contrib>
<contrib contrib-type="author">
<name>
<surname>Mei</surname>
<given-names>Yongguo</given-names>
</name>
<xref ref-type="aff" rid="aff002">
<sup>2</sup>
</xref>
<xref ref-type="aff" rid="aff003">
<sup>3</sup>
</xref>
</contrib>
<contrib contrib-type="author">
<name>
<surname>Abedi</surname>
<given-names>Vida</given-names>
</name>
<xref ref-type="aff" rid="aff002">
<sup>2</sup>
</xref>
<xref ref-type="aff" rid="aff003">
<sup>3</sup>
</xref>
</contrib>
<contrib contrib-type="author">
<name>
<surname>Marathe</surname>
<given-names>Madhav</given-names>
</name>
<xref ref-type="aff" rid="aff001">
<sup>1</sup>
</xref>
<xref ref-type="aff" rid="aff003">
<sup>3</sup>
</xref>
<xref ref-type="aff" rid="aff005">
<sup>5</sup>
</xref>
</contrib>
</contrib-group>
<aff id="aff001">
<label>1</label>
<addr-line>Network Dynamics and Simulation Science Laboratory, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</aff>
<aff id="aff002">
<label>2</label>
<addr-line>Nutritional Immunology and Molecular Medicine Laboratory, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</aff>
<aff id="aff003">
<label>3</label>
<addr-line>Center for Modeling Immunity to Enteric Pathogens, Virginia Bioinformatics Institute, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</aff>
<aff id="aff004">
<label>4</label>
<addr-line>Department of Biomedical Sciences and Pathobiology, Virginia-Maryland Regional College of Veterinary Medicine, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</aff>
<aff id="aff005">
<label>5</label>
<addr-line>Department of Computer Science, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</aff>
<aff id="aff006">
<label>6</label>
<addr-line>Department of Statistics, Virginia Tech, Blacksburg, Virginia, United States of America</addr-line>
</aff>
<contrib-group>
<contrib contrib-type="editor">
<name>
<surname>Castiglione</surname>
<given-names>Filippo</given-names>
</name>
<role>Editor</role>
<xref ref-type="aff" rid="edit1"></xref>
</contrib>
</contrib-group>
<aff id="edit1">
<addr-line>National Research Council of Italy (CNR), ITALY</addr-line>
</aff>
<author-notes>
<fn fn-type="conflict" id="coi001">
<p>
<bold>Competing Interests: </bold>
The authors have declared that no competing interests exist.</p>
</fn>
<fn fn-type="con" id="contrib001">
<p>Conceived and designed the experiments: MA XD MM. Performed the experiments: MA XD. Analyzed the data: MA XD CP AC RH JBR MM. Contributed reagents/materials/analysis tools: SE KB SH YM VA. Wrote the paper: MA XD VA RH MM.</p>
</fn>
<corresp id="cor001">* E-mail:
<email>xdeng@vt.edu</email>
</corresp>
</author-notes>
<pub-date pub-type="collection">
<year>2015</year>
</pub-date>
<pub-date pub-type="epub">
<day>1</day>
<month>9</month>
<year>2015</year>
</pub-date>
<volume>10</volume>
<issue>9</issue>
<elocation-id>e0136139</elocation-id>
<history>
<date date-type="received">
<day>7</day>
<month>3</month>
<year>2015</year>
</date>
<date date-type="accepted">
<day>31</day>
<month>7</month>
<year>2015</year>
</date>
</history>
<permissions>
<copyright-year>2015</copyright-year>
<copyright-holder>Alam et al</copyright-holder>
<license xlink:href="http://creativecommons.org/licenses/by/4.0/">
<license-p>This is an open access article distributed under the terms of the
<ext-link ext-link-type="uri" xlink:href="http://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution License</ext-link>
, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited</license-p>
</license>
</permissions>
<self-uri content-type="pdf" xlink:type="simple" xlink:href="pone.0136139.pdf"></self-uri>
<abstract>
<p>Agent-based models (ABM) are widely used to study immune systems, providing a procedural and interactive view of the underlying system. The interaction of components and the behavior of individual objects is described procedurally as a function of the internal states and the local interactions, which are often stochastic in nature. Such models typically have complex structures and consist of a large number of modeling parameters. Determining the key modeling parameters which govern the outcomes of the system is very challenging. Sensitivity analysis plays a vital role in quantifying the impact of modeling parameters in massively interacting systems, including large complex ABM. The high computational cost of executing simulations impedes running experiments with exhaustive parameter settings. Existing techniques of analyzing such a complex system typically focus on local sensitivity analysis, i.e. one parameter at a time, or a close “neighborhood” of particular parameter settings. However, such methods are not adequate to measure the uncertainty and sensitivity of parameters accurately because they overlook the global impacts of parameters on the system. In this article, we develop novel experimental design and analysis techniques to perform both global and local sensitivity analysis of large-scale ABMs. The proposed method can efficiently identify the most significant parameters and quantify their contributions to outcomes of the system. We demonstrate the proposed methodology for ENteric Immune SImulator (ENISI), a large-scale ABM environment, using a computational model of immune responses to
<italic>Helicobacter pylori</italic>
colonization of the gastric mucosa.</p>
</abstract>
<funding-group>
<funding-statement>This work has been partially supported by Defense Threat Reduction Agency (DTRA)—R&D (HPC) Award No. HDTRA1-09-1-0017, DTRA—Validation Award No. HDTRA1-11-1-0016, DTRA—Comprehensive National Incident Management System (CNIMS) Award No. HDRTA1-11-D-0016-0001, National Science Foundation (NSF) PetaApps Grant OCI-0904844, NSF Network Science and Engineering (NetSE) Grant CNS-1011769, NSF Software Development for Cyberinfrastructure (SDCI) Grant OCI-1032677, National Institutes of Health (NIH) MIDAS project 2U01GM070694-7, and National Institute of Allergy and Infectious Diseases (NIAID) & NIH project HHSN272201000056C.</funding-statement>
</funding-group>
<counts>
<fig-count count="12"></fig-count>
<table-count count="4"></table-count>
<page-count count="25"></page-count>
</counts>
<custom-meta-group>
<custom-meta id="data-availability">
<meta-name>Data Availability</meta-name>
<meta-value>The data can be obtained without restriction after creating an account from:
<ext-link ext-link-type="uri" xlink:href="https://immport.niaid.nih.gov/immportWeb/clinical/study/displayStudyDetails.do?itemList=SDY230">https://immport.niaid.nih.gov/immportWeb/clinical/study/displayStudyDetails.do?itemList=SDY230</ext-link>
. The Study number is SDY230.</meta-value>
</custom-meta>
</custom-meta-group>
</article-meta>
<notes>
<title>Data Availability</title>
<p>The data can be obtained without restriction after creating an account from:
<ext-link ext-link-type="uri" xlink:href="https://immport.niaid.nih.gov/immportWeb/clinical/study/displayStudyDetails.do?itemList=SDY230">https://immport.niaid.nih.gov/immportWeb/clinical/study/displayStudyDetails.do?itemList=SDY230</ext-link>
. The Study number is SDY230.</p>
</notes>
</front>
<body>
<sec sec-type="intro" id="sec001">
<title>Introduction</title>
<p>Sensitivity analysis (SA) is the study of the impact of different parameters in the outcomes of a system [
<xref rid="pone.0136139.ref001" ref-type="bibr">1</xref>
,
<xref rid="pone.0136139.ref002" ref-type="bibr">2</xref>
]. SA has been used in many scientific fields to highlight important data, optimize the design of a system, simplify models, and rank the influence of various parameters on a given system [
<xref rid="pone.0136139.ref003" ref-type="bibr">3</xref>
<xref rid="pone.0136139.ref006" ref-type="bibr">6</xref>
]. SA can be performed in a local or global context. Local SA examines the effect of deviations of a parameter (within its range), on system outcomes around a base setting [
<xref rid="pone.0136139.ref007" ref-type="bibr">7</xref>
], i.e., only one parameter is changed while all others remain fixed. Global SA evaluates the entire parameter space to determine all of the system’s critical points [
<xref rid="pone.0136139.ref003" ref-type="bibr">3</xref>
,
<xref rid="pone.0136139.ref008" ref-type="bibr">8</xref>
]. Both statistical and deterministic methods can be used for SA [
<xref rid="pone.0136139.ref002" ref-type="bibr">2</xref>
,
<xref rid="pone.0136139.ref009" ref-type="bibr">9</xref>
]. Other approaches for SA include variance decomposition [
<xref rid="pone.0136139.ref010" ref-type="bibr">10</xref>
], response surface [
<xref rid="pone.0136139.ref011" ref-type="bibr">11</xref>
,
<xref rid="pone.0136139.ref012" ref-type="bibr">12</xref>
], and sampling methodologies [
<xref rid="pone.0136139.ref013" ref-type="bibr">13</xref>
]. The choice of method depends on the purpose and the system under study. Typically the systems under review are computationally expensive and hence SA becomes very challenging.</p>
<p>ABM is a computational technique where the behavior of individual agents or groups of agents is encoded by simple rules, and the outcomes are observed at the system scale. ABM is a widely used technique in different areas of study such as computer science [
<xref rid="pone.0136139.ref014" ref-type="bibr">14</xref>
], economics, biology, ecology, and social phenomena. Popularity of ABM is also due to its intrinsically spatial component [
<xref rid="pone.0136139.ref015" ref-type="bibr">15</xref>
] and the fact that the system allows integration of different experimental data into one
<italic>in silico</italic>
experimental system very easily [
<xref rid="pone.0136139.ref016" ref-type="bibr">16</xref>
]. In fact, this modeling framework can be used to intuitively map biological (or social) phenomena and at the same time capture some of the inherent stochastic nature of these systems without an added layer of complexity [
<xref rid="pone.0136139.ref015" ref-type="bibr">15</xref>
,
<xref rid="pone.0136139.ref017" ref-type="bibr">17</xref>
<xref rid="pone.0136139.ref020" ref-type="bibr">20</xref>
]. Furthermore, ABM offers modeling intuitive implementation of interactions (e.g. binding of molecules) where rules that describe the interactions are often taken from existing literature. A concrete example: a large body of studies have successfully used ABM to simulate dynamics of inflammation [
<xref rid="pone.0136139.ref021" ref-type="bibr">21</xref>
<xref rid="pone.0136139.ref026" ref-type="bibr">26</xref>
]. ABM has become even more popular during the past decade with the exponential growth of technological and computational power. For instance, in 2008, agent based modeling and simulation studies were used to explore the development and resolution of virtual infections in a unique way [
<xref rid="pone.0136139.ref027" ref-type="bibr">27</xref>
,
<xref rid="pone.0136139.ref028" ref-type="bibr">28</xref>
]; the study was fundamental in the identification of key parameters capable of inducing clearance, persistent infection, or even death. For a comprehensive review of ABM applied to host-pathogen systems, the reader is referred to a review by Bauer
<italic>et al</italic>
., [
<xref rid="pone.0136139.ref029" ref-type="bibr">29</xref>
], where authors describe various models but also discuss some of the limitations and challenges.</p>
<p>We use a formal mathematical framework called graphical dynamical systems (GDS) [
<xref rid="pone.0136139.ref030" ref-type="bibr">30</xref>
,
<xref rid="pone.0136139.ref031" ref-type="bibr">31</xref>
] (see
<xref ref-type="supplementary-material" rid="pone.0136139.s001">S1 Text</xref>
for definition) to represent the agent based models studied here. This representation elucidates the formal aspects of the model as a discrete and finite dynamical system. GDS is a widely used framework for capturing distributed, dynamical systems such as social contact graphs [
<xref rid="pone.0136139.ref032" ref-type="bibr">32</xref>
], urban traffic and transportation [
<xref rid="pone.0136139.ref033" ref-type="bibr">33</xref>
], epidemics [
<xref rid="pone.0136139.ref030" ref-type="bibr">30</xref>
], dynamics of biological systems [
<xref rid="pone.0136139.ref034" ref-type="bibr">34</xref>
], and gene annotation [
<xref rid="pone.0136139.ref035" ref-type="bibr">35</xref>
]. The configuration space of an ABM, represented as a GDS
<italic>G</italic>
, can be conveniently viewed as a discrete Markov chain
<italic>M</italic>
. Each node in
<italic>M</italic>
is the state vector of the node states in
<italic>G</italic>
. A directed edge from node
<italic>A</italic>
to
<italic>B</italic>
in
<italic>M</italic>
with label
<italic>p</italic>
denotes the probability of transition from
<italic>A</italic>
to
<italic>B</italic>
. The nodes of
<italic>M</italic>
are also referred to as the state space of
<italic>M</italic>
. For example, a complex ABM comprised of one million agents, each of these agents having six possible states, results in a Markov chain with a state space consisting of 6
<sup>10
<sup>6</sup>
</sup>
different configurations. Note that the Markov chain is represented succinctly by the ABM. The chain is exponentially larger than its representation. Straightforward methods for global sensitivity analysis on such large chains are unlikely to scale. It is worth pointing out that the proposed sensitivity analysis are not restricted to GDS. It is rather a general methodology applicable to a wide range of complex simulation models. However, we believe having a good understanding of the presented system can be beneficial as it can provide a direct example of a complex system where such a method can be used. The size of the state space makes the experimental design of interest. If the state space was small, one could instead use many other designs that can exhaustively cover the range of parameter values.</p>
<p>In this paper, we perform sensitivity analysis on a complex ABM named ENteric Immunity SImulator (ENISI) [
<xref rid="pone.0136139.ref036" ref-type="bibr">36</xref>
<xref rid="pone.0136139.ref038" ref-type="bibr">38</xref>
]. ENISI is a modeling environment for studying the inflammatory and regulatory immune responses in the gut. During the development of ENISI, three versions were released; ENISI HPC [
<xref rid="pone.0136139.ref039" ref-type="bibr">39</xref>
], ENISI Visual [
<xref rid="pone.0136139.ref040" ref-type="bibr">40</xref>
], and ENISI MSM [
<xref rid="pone.0136139.ref041" ref-type="bibr">41</xref>
,
<xref rid="pone.0136139.ref042" ref-type="bibr">42</xref>
]. ENISI HPC focuses on scalability by implementing a parallel simulation framework. ENISI Visual focuses on visualizations. ENISI MSM, a multiscale modeling framework, focuses on the integration of heterogeneous modeling technologies. The SA method presented here can therefore be utilized directly for the three different technologies in addition to being valuable to other ABM systems. The novelty lies in the choice of appropriate designs that enable statistical methods to explore parts of the Markov chain in a computationally efficient manner and still guarantee certain statistical properties. We use orthogonal arrays and variation-based sensitivity analysis to achieve this, which is discussed in detail in subsequent sections.</p>
<p>Developing novel approaches for sensitivity analysis of agent-based models enhances understanding of the influence of different input parameters and their variations on model outcomes and has the potential to improve the parameter estimation process by making it more systematic and streamlined. The objective of the sensitivity analysis is to identify the greatest and least significant parameters in the model, and to quantify how parameter uncertainty influences outcomes. Analytically exploring the behavior of such systems, even if model outcomes mirror reality, is challenging due to the large number of parameters [
<xref rid="pone.0136139.ref002" ref-type="bibr">2</xref>
]. Sensitivity analysis is also important for understanding the relationships between input and output variables, for testing the robustness of the system, and for identifying errors in the model.</p>
<sec id="sec002">
<title>ENISI: an agent-based modeling environment</title>
<p>ENteric Immunity SImulator (ENISI) is a modeling environment for studying inflammatory/effector and regulatory immune responses in the gut [
<xref rid="pone.0136139.ref036" ref-type="bibr">36</xref>
<xref rid="pone.0136139.ref038" ref-type="bibr">38</xref>
]. ENISI encodes immune pathways as an agent-based model representing individual cells that participate and interact with other cells in the gastrointestinal tract. The individual cells are distributed spatially to four
<italic>locations</italic>
: lumen, epithelial barrier (EB), lamina propria (LP), and gastric lymph node (GLN).
<italic>Locations</italic>
are further divided into many discrete patches called
<italic>sublocations</italic>
. A
<italic>sublocation</italic>
is defined as the maximum volume at which a cell can be assumed to be in contact with all other cells.
<italic>Sublocations</italic>
represent the discrete region where cells remain and interact. Each cell moves between
<italic>sublocations</italic>
according to its own
<italic>schedule</italic>
. A cell can have its own movement
<italic>schedule</italic>
assigned by phenotype or
<italic>state</italic>
and
<italic>location</italic>
. Within a
<italic>location</italic>
, cells move between the
<italic>scheduled</italic>
<italic>sublocations</italic>
at short time intervals, which represents random movement and produces a dynamic contact network. This random movement, which does not favor movement to adjacent
<italic>sublocations</italic>
, is added because of an assumption that cells are moving at a much faster rate than the time period for
<italic>sublocation</italic>
update (30 minutes). Therefore, the probability of an individual cell having traveled far from the current
<italic>sublocation</italic>
is the same as the probability of being in a nearby
<italic>sublocation</italic>
after the 30 minute interval.</p>
</sec>
<sec id="sec003">
<title>Parameters</title>
<p>The ENISI model of immune responses to
<italic>H. pylori</italic>
has a set of 25 independent modeling parameters. These modeling parameters include transition probabilities, constant factors of group-cell interactions, stimulation/inhibition factors, and migration rates. Each of the 25 parameters is a continuous input variable within corresponding parameter ranges. Parameter values are obtained from the literature whenever available or estimated during model development [
<xref rid="pone.0136139.ref043" ref-type="bibr">43</xref>
<xref rid="pone.0136139.ref048" ref-type="bibr">48</xref>
]. A complete list of parameters along with a short description, range, and default value is shown in
<xref ref-type="table" rid="pone.0136139.t001">Table 1</xref>
.</p>
<table-wrap id="pone.0136139.t001" orientation="portrait" position="float">
<object-id pub-id-type="doi">10.1371/journal.pone.0136139.t001</object-id>
<label>Table 1</label>
<caption>
<title>A complete list of parameter values for ENISI.</title>
</caption>
<alternatives>
<graphic id="pone.0136139.t001g" xlink:href="pone.0136139.t001"></graphic>
<table frame="box" rules="all" border="0">
<colgroup span="1">
<col align="center" valign="top" span="1"></col>
<col align="left" valign="top" span="1"></col>
<col align="left" valign="top" span="1"></col>
<col align="left" valign="top" span="1"></col>
</colgroup>
<thead>
<tr>
<th align="center" rowspan="1" colspan="1">Parameter</th>
<th align="left" rowspan="1" colspan="1">Description</th>
<th align="left" rowspan="1" colspan="1">Range</th>
<th align="left" rowspan="1" colspan="1">Default Value</th>
</tr>
</thead>
<tbody>
<tr>
<td align="center" rowspan="1" colspan="1">
<italic>α</italic>
<sub>
<italic>T</italic>
</sub>
</td>
<td align="left" rowspan="1" colspan="1">probability of resting T cell stimulation</td>
<td align="left" rowspan="1" colspan="1">[0, 1]</td>
<td align="left" rowspan="1" colspan="1">1</td>
</tr>
<tr>
<td align="center" rowspan="1" colspan="1">
<italic>p</italic>
<sub>17</sub>
</td>
<td align="left" rowspan="1" colspan="1">probability of resting T cell stimulation to
<italic>Th</italic>
1 vs.
<italic>Th</italic>
17 by
<italic>eDC</italic>
or
<italic>M</italic>
1</td>
<td align="left" rowspan="1" colspan="1">[0, 1]</td>
<td align="left" rowspan="1" colspan="1">0.5</td>
</tr>
<tr>
<td align="center" rowspan="1" colspan="1">
<italic>α</italic>
<sub>
<italic>nTreg</italic>
</sub>
</td>
<td align="left" rowspan="1" colspan="1">probability of resting
<italic>nTreg</italic>
stimulation</td>
<td align="left" rowspan="1" colspan="1">[0, 1]</td>
<td align="left" rowspan="1" colspan="1">1</td>
</tr>
<tr>
<td align="center" rowspan="1" colspan="1">
<italic>ν</italic>
<sub>
<italic>T</italic>
</sub>
</td>
<td align="left" rowspan="1" colspan="1">fraction of active T cells that become memory T cells</td>
<td align="left" rowspan="1" colspan="1">[0, 1]</td>
<td align="left" rowspan="1" colspan="1">0.1 [
<xref rid="pone.0136139.ref047" ref-type="bibr">47</xref>
]</td>
</tr>
<tr>
<td align="center" rowspan="1" colspan="1">
<italic>a</italic>
<sub>1</sub>
</td>
<td align="left" rowspan="1" colspan="1">co-efficient of
<inline-formula id="pone.0136139.e001">
<alternatives>
<graphic id="pone.0136139.e001g" xlink:href="pone.0136139.e001"></graphic>
<mml:math id="M1">
<mml:mrow>
<mml:msub>
<mml:mi>ν</mml:mi>
<mml:mn>12</mml:mn>
</mml:msub>
<mml:mo>=</mml:mo>
<mml:msup>
<mml:mrow>
<mml:mo stretchy="true">(</mml:mo>
<mml:mfrac>
<mml:mrow>
<mml:msub>
<mml:mi>a</mml:mi>
<mml:mn>1</mml:mn>
</mml:msub>
<mml:mi>R</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:msub>
<mml:mi>a</mml:mi>
<mml:mn>1</mml:mn>
</mml:msub>
<mml:mi>R</mml:mi>
<mml:mo>+</mml:mo>
<mml:msub>
<mml:mi>i</mml:mi>
<mml:mn>1</mml:mn>
</mml:msub>
<mml:mi>N</mml:mi>
</mml:mrow>
</mml:mfrac>
<mml:mo stretchy="true">)</mml:mo>
</mml:mrow>
<mml:msub>
<mml:mi>y</mml:mi>
<mml:mn>1</mml:mn>
</mml:msub>
</mml:msup>
</mml:mrow>
</mml:math>
</alternatives>
</inline-formula>
(Pr(
<italic>M</italic>
1 →
<italic>M</italic>
2)) for activators</td>
<td align="left" rowspan="1" colspan="1">[0, 10]</td>
<td align="left" rowspan="1" colspan="1">1</td>
</tr>
<tr>
<td align="center" rowspan="1" colspan="1">
<italic>i</italic>
<sub>1</sub>
</td>
<td align="left" rowspan="1" colspan="1">co-efficient of
<italic>ν</italic>
<sub>12</sub>
for inhibitors</td>
<td align="left" rowspan="1" colspan="1">[0, 10]</td>
<td align="left" rowspan="1" colspan="1">1</td>
</tr>
<tr>
<td align="center" rowspan="1" colspan="1">
<italic>y</italic>
<sub>1</sub>
</td>
<td align="left" rowspan="1" colspan="1">exponent of
<italic>ν</italic>
<sub>12</sub>
</td>
<td align="left" rowspan="1" colspan="1">[0, 4]</td>
<td align="left" rowspan="1" colspan="1">4</td>
</tr>
<tr>
<td align="center" rowspan="1" colspan="1">
<italic>a</italic>
<sub>2</sub>
</td>
<td align="left" rowspan="1" colspan="1">co-efficient of
<inline-formula id="pone.0136139.e002">
<alternatives>
<graphic id="pone.0136139.e002g" xlink:href="pone.0136139.e002"></graphic>
<mml:math id="M2">
<mml:mrow>
<mml:msub>
<mml:mi>ν</mml:mi>
<mml:mn>21</mml:mn>
</mml:msub>
<mml:mo>=</mml:mo>
<mml:msup>
<mml:mrow>
<mml:mo stretchy="true">(</mml:mo>
<mml:mfrac>
<mml:mrow>
<mml:msub>
<mml:mi>a</mml:mi>
<mml:mn>2</mml:mn>
</mml:msub>
<mml:mi>R</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:msub>
<mml:mi>a</mml:mi>
<mml:mn>2</mml:mn>
</mml:msub>
<mml:mi>R</mml:mi>
<mml:mo>+</mml:mo>
<mml:msub>
<mml:mi>i</mml:mi>
<mml:mn>2</mml:mn>
</mml:msub>
<mml:mi>N</mml:mi>
</mml:mrow>
</mml:mfrac>
<mml:mo stretchy="true">)</mml:mo>
</mml:mrow>
<mml:msub>
<mml:mi>y</mml:mi>
<mml:mn>2</mml:mn>
</mml:msub>
</mml:msup>
</mml:mrow>
</mml:math>
</alternatives>
</inline-formula>
(Pr(
<italic>M</italic>
2 →
<italic>M</italic>
1)) for activators</td>
<td align="left" rowspan="1" colspan="1">[0, 10]</td>
<td align="left" rowspan="1" colspan="1">1</td>
</tr>
<tr>
<td align="center" rowspan="1" colspan="1">
<italic>i</italic>
<sub>2</sub>
</td>
<td align="left" rowspan="1" colspan="1">co-efficient of
<italic>ν</italic>
<sub>21</sub>
for inhibitors</td>
<td align="left" rowspan="1" colspan="1">[0, 10]</td>
<td align="left" rowspan="1" colspan="1">1</td>
</tr>
<tr>
<td align="center" rowspan="1" colspan="1">
<italic>y</italic>
<sub>2</sub>
</td>
<td align="left" rowspan="1" colspan="1">exponent of
<italic>ν</italic>
<sub>21</sub>
</td>
<td align="left" rowspan="1" colspan="1">[0, 4]</td>
<td align="left" rowspan="1" colspan="1">4</td>
</tr>
<tr>
<td align="center" rowspan="1" colspan="1">
<italic>a</italic>
<sub>
<italic>r</italic>
</sub>
</td>
<td align="left" rowspan="1" colspan="1">co-efficient of
<inline-formula id="pone.0136139.e003">
<alternatives>
<graphic id="pone.0136139.e003g" xlink:href="pone.0136139.e003"></graphic>
<mml:math id="M3">
<mml:mrow>
<mml:msub>
<mml:mi>ν</mml:mi>
<mml:mrow>
<mml:mi>r</mml:mi>
<mml:mn>17</mml:mn>
</mml:mrow>
</mml:msub>
<mml:mo>=</mml:mo>
<mml:msup>
<mml:mrow>
<mml:mo stretchy="true">(</mml:mo>
<mml:mfrac>
<mml:mrow>
<mml:msub>
<mml:mi>a</mml:mi>
<mml:mi>r</mml:mi>
</mml:msub>
<mml:mi>N</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:msub>
<mml:mi>i</mml:mi>
<mml:mi>r</mml:mi>
</mml:msub>
<mml:mi>R</mml:mi>
<mml:mo>+</mml:mo>
<mml:msub>
<mml:mi>a</mml:mi>
<mml:mi>r</mml:mi>
</mml:msub>
<mml:mi>N</mml:mi>
</mml:mrow>
</mml:mfrac>
<mml:mo stretchy="true">)</mml:mo>
</mml:mrow>
<mml:msub>
<mml:mi>y</mml:mi>
<mml:mi>r</mml:mi>
</mml:msub>
</mml:msup>
</mml:mrow>
</mml:math>
</alternatives>
</inline-formula>
(Pr(
<italic>iTreg</italic>
<italic>Th</italic>
17)) for activators</td>
<td align="left" rowspan="1" colspan="1">[0, 10]</td>
<td align="left" rowspan="1" colspan="1">1</td>
</tr>
<tr>
<td align="center" rowspan="1" colspan="1">
<italic>i</italic>
<sub>
<italic>r</italic>
</sub>
</td>
<td align="left" rowspan="1" colspan="1">co-efficient of
<italic>ν</italic>
<sub>
<italic>r</italic>
17</sub>
for inhibitors</td>
<td align="left" rowspan="1" colspan="1">[0, 10]</td>
<td align="left" rowspan="1" colspan="1">1</td>
</tr>
<tr>
<td align="center" rowspan="1" colspan="1">
<italic>y</italic>
<sub>
<italic>r</italic>
</sub>
</td>
<td align="left" rowspan="1" colspan="1">exponent of
<italic>ν</italic>
<sub>
<italic>r</italic>
17</sub>
</td>
<td align="left" rowspan="1" colspan="1">[0, 4]</td>
<td align="left" rowspan="1" colspan="1">4</td>
</tr>
<tr>
<td align="center" rowspan="1" colspan="1">
<italic>a</italic>
<sub>17</sub>
</td>
<td align="left" rowspan="1" colspan="1">co-efficient of
<inline-formula id="pone.0136139.e004">
<alternatives>
<graphic id="pone.0136139.e004g" xlink:href="pone.0136139.e004"></graphic>
<mml:math id="M4">
<mml:mrow>
<mml:msub>
<mml:mi>ν</mml:mi>
<mml:mrow>
<mml:mn>17</mml:mn>
<mml:mi>r</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo>=</mml:mo>
<mml:msup>
<mml:mrow>
<mml:mo stretchy="true">(</mml:mo>
<mml:mfrac>
<mml:mrow>
<mml:msub>
<mml:mi>a</mml:mi>
<mml:mn>17</mml:mn>
</mml:msub>
<mml:mi>R</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:msub>
<mml:mi>a</mml:mi>
<mml:mn>17</mml:mn>
</mml:msub>
<mml:mi>R</mml:mi>
<mml:mo>+</mml:mo>
<mml:msub>
<mml:mi>i</mml:mi>
<mml:mn>17</mml:mn>
</mml:msub>
<mml:mi>N</mml:mi>
</mml:mrow>
</mml:mfrac>
<mml:mo stretchy="true">)</mml:mo>
</mml:mrow>
<mml:msub>
<mml:mi>y</mml:mi>
<mml:mn>17</mml:mn>
</mml:msub>
</mml:msup>
</mml:mrow>
</mml:math>
</alternatives>
</inline-formula>
(Pr(
<italic>Th</italic>
17 →
<italic>iTreg</italic>
)) for activators</td>
<td align="left" rowspan="1" colspan="1">[0, 10]</td>
<td align="left" rowspan="1" colspan="1">1</td>
</tr>
<tr>
<td align="center" rowspan="1" colspan="1">
<italic>i</italic>
<sub>17</sub>
</td>
<td align="left" rowspan="1" colspan="1">co-efficient of
<italic>ν</italic>
<sub>17
<italic>r</italic>
</sub>
for inhibitors</td>
<td align="left" rowspan="1" colspan="1">[0, 10]</td>
<td align="left" rowspan="1" colspan="1">1</td>
</tr>
<tr>
<td align="center" rowspan="1" colspan="1">
<italic>y</italic>
<sub>17</sub>
</td>
<td align="left" rowspan="1" colspan="1">exponent of
<italic>ν</italic>
<sub>17
<italic>r</italic>
</sub>
</td>
<td align="left" rowspan="1" colspan="1">[0, 4]</td>
<td align="left" rowspan="1" colspan="1">4</td>
</tr>
<tr>
<td align="center" rowspan="1" colspan="1">
<italic>ν</italic>
<sub>
<italic>BM</italic>
</sub>
</td>
<td align="left" rowspan="1" colspan="1">probability that commensal bacteria induces inflammatory phenotype in macrophages</td>
<td align="left" rowspan="1" colspan="1">[0, 1]</td>
<td align="left" rowspan="1" colspan="1">0</td>
</tr>
<tr>
<td align="center" rowspan="1" colspan="1">
<italic>ν</italic>
<sub>
<italic>BD</italic>
</sub>
</td>
<td align="left" rowspan="1" colspan="1">probability that commensal bacteria induces inflammatory phenotype in dendritic cells</td>
<td align="left" rowspan="1" colspan="1">[0, 1]</td>
<td align="left" rowspan="1" colspan="1">0</td>
</tr>
<tr>
<td align="center" rowspan="1" colspan="1">
<italic>ν</italic>
<sub>
<italic>Bs</italic>
</sub>
</td>
<td align="left" rowspan="1" colspan="1">probability that commensal bacteria induces inflammatory phenotype in ‘sampling’ dendritic cells</td>
<td align="left" rowspan="1" colspan="1">[0, 1]</td>
<td align="left" rowspan="1" colspan="1">0</td>
</tr>
<tr>
<td align="center" rowspan="1" colspan="1">
<italic>ν</italic>
<sub>
<italic>EC</italic>
</sub>
</td>
<td align="left" rowspan="1" colspan="1">probability that
<italic>EC</italic>
transitions to
<italic>pEcell</italic>
upon contact within flammatory factors</td>
<td align="left" rowspan="1" colspan="1">[0, 1]</td>
<td align="left" rowspan="1" colspan="1">0.05</td>
</tr>
<tr>
<td align="center" rowspan="1" colspan="1">
<italic>ν</italic>
<sub>
<italic>EB</italic>
</sub>
</td>
<td align="left" rowspan="1" colspan="1">probability that
<italic>EC</italic>
is damaged by microbial toxins</td>
<td align="left" rowspan="1" colspan="1">[0, 1]</td>
<td align="left" rowspan="1" colspan="1">1 [
<xref rid="pone.0136139.ref048" ref-type="bibr">48</xref>
]</td>
</tr>
<tr>
<td align="center" rowspan="1" colspan="1">
<italic>β</italic>
<sub>
<italic>r</italic>
</sub>
</td>
<td align="left" rowspan="1" colspan="1">ability of commensal or inflammatory bacteria to induce chemoattractant expression in epithelial cells</td>
<td align="left" rowspan="1" colspan="1">[0, 1]</td>
<td align="left" rowspan="1" colspan="1">1</td>
</tr>
<tr>
<td align="center" rowspan="1" colspan="1">
<italic>μ</italic>
<sub>
<italic>ce</italic>
</sub>
</td>
<td align="left" rowspan="1" colspan="1">probability that
<italic>pEcell</italic>
is killed by inflammatory factors</td>
<td align="left" rowspan="1" colspan="1">[0, 1]</td>
<td align="left" rowspan="1" colspan="1">0.5</td>
</tr>
<tr>
<td align="center" rowspan="1" colspan="1">
<italic>β</italic>
<sub>
<italic>d</italic>
</sub>
</td>
<td align="left" rowspan="1" colspan="1">relative amount of microbicide secreted by
<italic>pECell</italic>
,
<italic>pEC</italic>
_
<italic>noR</italic>
in response to commensal or inflammatory bacteria</td>
<td align="left" rowspan="1" colspan="1">[0, 1]</td>
<td align="left" rowspan="1" colspan="1">1</td>
</tr>
<tr>
<td align="center" rowspan="1" colspan="1">
<italic>μ</italic>
<sub>
<italic>M</italic>
1</sub>
</td>
<td align="left" rowspan="1" colspan="1">ability of
<italic>M</italic>
1 to eliminate bacteria</td>
<td align="left" rowspan="1" colspan="1">[0, 1]</td>
<td align="left" rowspan="1" colspan="1">1</td>
</tr>
</tbody>
</table>
</alternatives>
</table-wrap>
</sec>
<sec id="sec004">
<title>System output</title>
<p>The ENISI model under study has eight types of immune cells and bacteria (epithelial cells, tolerogenic bacteria, commensal bacteria, dendritic cells, sampling dendritic cells, macrophages, CD4+ T cells and natural T-regulatory cells) involved in the gastric immune responses to
<italic>H. pylori</italic>
. Each individual cell has many different functional states called phenotypes. There are a total of thirty different phenotypes in this model. Among those, activated/effector CD4+ T cells and macrophages are the most interesting cell types for the disease model under study. Furthermore, there are four different tissue sites modeled in ENISI: lumen, epithelial barrier (EB), lamina propria (LP), and gastric lymph node (GLN). ENISI provides the number of cells of each phenotype grouped by
<italic>location</italic>
as the system output. A complete list of cell types is available in [
<xref rid="pone.0136139.ref036" ref-type="bibr">36</xref>
].</p>
</sec>
<sec id="sec005">
<title>Aims and challenges</title>
<p>The aims of this study are to (1) evaluate the influence of parameters on model outcomes, (2) rank the parameters based on their global influence, and (3) refine the ABM based on the result of the SA of the parameters. In this study, we used ENISI to recreate an experimental infection of an individual mouse with
<italic>Helicobacter pylori</italic>
, a bacteria that colonizes the human stomach chronically causing disease in 15% of carriers. The particular strain in this experiment is the Cag Pathogenicity Island (CagPAI)+
<italic>H. pylori</italic>
strain 26695 [
<xref rid="pone.0136139.ref036" ref-type="bibr">36</xref>
<xref rid="pone.0136139.ref038" ref-type="bibr">38</xref>
]. However, it is important to indicate that this analysis can be applied to an array of complex biological networks including the model of CD4+ T cell differentiation [
<xref rid="pone.0136139.ref049" ref-type="bibr">49</xref>
<xref rid="pone.0136139.ref051" ref-type="bibr">51</xref>
], network dynamics of T helper 17 induction and differentiation [
<xref rid="pone.0136139.ref052" ref-type="bibr">52</xref>
], and network of interactions between mucosal immunity and the gut microbiome during
<italic>Clostridium difficile</italic>
infection [
<xref rid="pone.0136139.ref053" ref-type="bibr">53</xref>
].</p>
<p>The key challenges of the study lie in the size and number of parameters of the model and the computational cost associated with the stochastic nature of the process. A simulation in ENISI consists of a magnitude of 10
<sup>7</sup>
individual agents. There are thirty different types of agents and a few hundred individual rules. On a modern high performance computing cluster of 48 nodes, a single run of the simulation takes about ninety minutes on average. Because of this limitation, the runtime of multiple simulations must be taken into consideration. Each of the 25 parameters is considered to be continuous; however, if each of the 25 parameters takes only four different values, then a full factorial experimental design would require 4
<sup>25</sup>
runs, which is not possible in terms of the system’s runtime. To alleviate this problem we propose a novel experimental design with an economic number of simulations.</p>
</sec>
</sec>
<sec sec-type="materials|methods" id="sec006">
<title>Materials and Methods</title>
<sec id="sec007">
<title>Related Works</title>
<p>Because sensitivity analysis can be an effective tool for systematically identifying the most and the least significant parameters, it can provide valuable information on the robustness of biological systems. Summer
<italic>et al</italic>
., [
<xref rid="pone.0136139.ref054" ref-type="bibr">54</xref>
,
<xref rid="pone.0136139.ref055" ref-type="bibr">55</xref>
] implemented a SA method by exploring a group-based procedure, where SA is applied to groups of parameters. The result of this analysis can be used to identify the components of the model that are most important in determining the model behavior. This method was applied to a composite model of blood glucose homeostasis that combines models of processes at the sub-cellular, cellular and organ level to describe the physiological system [
<xref rid="pone.0136139.ref054" ref-type="bibr">54</xref>
<xref rid="pone.0136139.ref056" ref-type="bibr">56</xref>
]. Furthermore, using a simple ABM to assess different methodologies for SA, van Voorn
<italic>et al</italic>
. [
<xref rid="pone.0136139.ref057" ref-type="bibr">57</xref>
] highlighted the importance of using a local as well as global method; while the local sensitivity can capture tipping points, the global approach can provide a summary measure of parameters. Along the same line of research, Schouten
<italic>et al</italic>
., [
<xref rid="pone.0136139.ref058" ref-type="bibr">58</xref>
] compared two methods of SA–local and global and concluded that a mixed approach of SA can lead to a better understanding of model behavior. In the global SA, the authors used an approach based on Monte Carlo sampling of parameters. In a recent study, Ligmann-Zielinska
<italic>et al</italic>
. [
<xref rid="pone.0136139.ref059" ref-type="bibr">59</xref>
] argue that SA as well as uncertainty analysis (UA) are important elements in quantification of model output variability and its sensitivity to inputs. On the basis of this argument, authors utilized variance decomposition of the ABM output to demonstrate how an integrated quantitative UA in combination with SA can be used in ABM development. Related analyses include the work done by Parry
<italic>et al</italic>
., [
<xref rid="pone.0136139.ref060" ref-type="bibr">60</xref>
] using a Bayesian approach for SA for the identification of sensitive elements, and work done by Fonoberova
<italic>et al</italic>
., [
<xref rid="pone.0136139.ref061" ref-type="bibr">61</xref>
] using a variance-based as well as derivative-based global SA in the identification of key elements for model reduction. In this paper we consider both global and local sensitivity analysis of ABM using ANOVA and discuss the most and the least significant parameters which could be used for model reduction as in [
<xref rid="pone.0136139.ref061" ref-type="bibr">61</xref>
]. Though there are numerous instances of SA applied to ABMs, few attempt to classify the behavior of a parameter. We used main effect plots to classify the behavior of input parameters into four classes in a novel fashion that provides both visual representations of and statistical insights into a parameter. Our choice of four levels reduces the computational costs associated with large scale ABM while regaining valuable insights into the parameter. These statistical analyses along with sparse orthogonal array based experimental design provide a concrete framework to perform SA on any agent based models.</p>
</sec>
<sec id="sec008">
<title>Global sensitivity analysis</title>
<p>In global sensitivity analysis we study the behavior of input parameters on the variation of the model output. Not every modeling parameter has equal effects on system outcomes. Some parameters play more significant roles than others. The goal is to quantify the significance of the parameters based on some measurement.</p>
</sec>
<sec id="sec009">
<title>Global versus Local sensitivity analysis</title>
<p>A local sensitivity analysis addresses sensitivity relative to change of a single parameter value, while a global analysis examines sensitivity with regard to the entire parameter distribution.</p>
<p>Global sensitivity analysis focuses on the variance of model outputs and determines how input parameters influence the output parameters. It is a central tool in sensitivity analysis since it provides a quantitative and rigorous overview of how different inputs influence the output.</p>
<p>Local sensitivity analysis focuses more on a single input’s behavior while other parts remain the same. It is narrow in this aspect as the effect of input parameter is not measured for settings other than the base. Global sensitivity analysis is often preferred when possible, due to its greater detail, but for a large system it is time inefficient to run a model. Local sensitivity analysis method can be preferred because it requires less computation.</p>
<sec id="sec010">
<title>Choosing suitable model output</title>
<p>To measure the sensitivity of the model with respect to modeling parameters we need to select suitable output variables. One challenge regarding a complex system like ENISI is that it has many output variables. One of the outputs provided by ENISI is the number of cells of each cell type in every
<italic>location</italic>
. There are about 30 total possible phenotypes for the eight types of cells modeled in ENISI. Measuring sensitivity against all the model outputs is difficult and time consuming. In order to conduct the analysis more effectively, we chose nine output variables that are the primary concerns of immunologists [
<xref rid="pone.0136139.ref062" ref-type="bibr">62</xref>
,
<xref rid="pone.0136139.ref063" ref-type="bibr">63</xref>
]. In this study, three phenotypes of T cells are considered:
<italic>Th</italic>
1,
<italic>Th</italic>
17, and
<italic>iTreg</italic>
in the
<italic>locations</italic>
LP and GLN as the primary output of the system. Three types of macrophages:
<italic>M</italic>
1 (classically activated, prone to promote inflammation),
<italic>M</italic>
2 (alternatively activated with regulatory and pro-resolutory functions) and
<italic>M</italic>
0 (precursor of
<italic>M</italic>
1 and
<italic>M</italic>
2 macrophages) in the LP
<italic>location</italic>
are also considered. A list of these nine outputs is provided in
<xref ref-type="table" rid="pone.0136139.t002">Table 2</xref>
.</p>
<table-wrap id="pone.0136139.t002" orientation="portrait" position="float">
<object-id pub-id-type="doi">10.1371/journal.pone.0136139.t002</object-id>
<label>Table 2</label>
<caption>
<title>System Output Phenotypes.</title>
</caption>
<alternatives>
<graphic id="pone.0136139.t002g" xlink:href="pone.0136139.t002"></graphic>
<table frame="box" rules="all" border="0">
<colgroup span="1">
<col align="left" valign="top" span="1"></col>
<col align="left" valign="top" span="1"></col>
</colgroup>
<thead>
<tr>
<th align="left" rowspan="1" colspan="1">State</th>
<th align="left" rowspan="1" colspan="1">Description</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left" rowspan="1" colspan="1">
<italic>Th</italic>
1_
<italic>LP</italic>
</td>
<td align="left" rowspan="1" colspan="1">Active T helper-1 cell in lamina propria</td>
</tr>
<tr>
<td align="left" rowspan="1" colspan="1">
<italic>Th</italic>
17_
<italic>LP</italic>
</td>
<td align="left" rowspan="1" colspan="1">Active T helper-17 cell in lamina propria</td>
</tr>
<tr>
<td align="left" rowspan="1" colspan="1">
<italic>iTreg</italic>
_
<italic>LP</italic>
</td>
<td align="left" rowspan="1" colspan="1">Induced T regulatory cell in lamina propria</td>
</tr>
<tr>
<td align="left" rowspan="1" colspan="1">
<italic>Th</italic>
1_
<italic>GLN</italic>
</td>
<td align="left" rowspan="1" colspan="1">Active T helper-1 cell in gastric lymph node</td>
</tr>
<tr>
<td align="left" rowspan="1" colspan="1">
<italic>Th</italic>
17_
<italic>GLN</italic>
</td>
<td align="left" rowspan="1" colspan="1">Active T helper-17 cell in gastric lymph node</td>
</tr>
<tr>
<td align="left" rowspan="1" colspan="1">
<italic>iTreg</italic>
_
<italic>GLN</italic>
</td>
<td align="left" rowspan="1" colspan="1">Induced T regulatory cell in gastric lymph node</td>
</tr>
<tr>
<td align="left" rowspan="1" colspan="1">
<italic>M</italic>
0_
<italic>LP</italic>
</td>
<td align="left" rowspan="1" colspan="1">Undifferentiated macrophage in lamina propria</td>
</tr>
<tr>
<td align="left" rowspan="1" colspan="1">
<italic>M</italic>
1_
<italic>LP</italic>
</td>
<td align="left" rowspan="1" colspan="1">Activated inflammatory macrophage in lamina propria</td>
</tr>
<tr>
<td align="left" rowspan="1" colspan="1">
<italic>M</italic>
2_
<italic>LP</italic>
</td>
<td align="left" rowspan="1" colspan="1">Activated regulatory macrophage in lamina propria</td>
</tr>
</tbody>
</table>
</alternatives>
</table-wrap>
<p>ENISI models the immune response and pathology at the gastric mucosa during
<italic>H. pylori</italic>
infection. In the stomach
<italic>H. pylori</italic>
is mainly found in the mucus layer lining the epithelial cells, and a small fraction is present in the lamina propria (LP). Cells participating in the immune response to the bacteria, such as CD4+ T cells and macrophages, are found in the lamina propria. Cells in the gastric lymph nodes (GLN) region also play an important role in the induction of the immune response. Different phenotypes of T cells and macrophages play vital roles in the inflammation and immuno-regulation process [
<xref rid="pone.0136139.ref062" ref-type="bibr">62</xref>
<xref rid="pone.0136139.ref064" ref-type="bibr">64</xref>
]. For this reason we chose regulatory T cells, T helper 1, and T helper 17 cells in both LP and GLN sites and
<italic>M</italic>
0,
<italic>M</italic>
1, and
<italic>M</italic>
2 macrophages in the LP site as the output variables. We also have biological lab data available for these cell types for model calibration.</p>
</sec>
<sec id="sec011">
<title>Choosing input parameter values</title>
<p>In this study, the value of each parameter has continuous values with a fixed range as shown in
<xref ref-type="table" rid="pone.0136139.t001">Table 1</xref>
. In order to conduct variation-based sensitivity analysis [
<xref rid="pone.0136139.ref001" ref-type="bibr">1</xref>
], a common approach is to choose several levels for each parameter such that the analysis of variance can be performed in a rigorous fashion. In this work, we consider each parameter to have four discrete values (levels) in increasing order: Level 1 to Level 4. The exact value for each level is dependent on the corresponding parameter range from
<xref ref-type="table" rid="pone.0136139.t001">Table 1</xref>
. Exact values for the four levels of each parameter are given in
<xref ref-type="supplementary-material" rid="pone.0136139.s003">S1 File</xref>
. We observed that most of the parameter values in the default calibrated settings tend to take values lower in the range. This is reflected in our choice of values for the levels. With four levels in each parameter, each input parameter has three degrees of freedom when conducting variation-based analysis. In a simple word, one can fit a polynomial function for each parameter with degrees three. Thus, one can study the significance of the linear, quadratic, and cubic effects of the input parameter on the output. It provides flexibility on investigating how the input parameter affects the output for the agent-based simulation model. If one only chooses two levels for each parameter, then only the linear effect of each parameter could be studied [
<xref rid="pone.0136139.ref065" ref-type="bibr">65</xref>
]. It is worth noting that for a different agent based model, it may not be feasible to have the same number of levels for each parameter. In this situation, we would consider a mixed-level experimental design [
<xref rid="pone.0136139.ref066" ref-type="bibr">66</xref>
] to address this challenge. We also like to remark that the choice of four levels provides enough information to extract linear and non-linear effects in this study. However, when having fewer parameters and high computational power, modelers can further increase the number of levels to enrich the predictive power of the analysis.</p>
</sec>
</sec>
<sec id="sec012">
<title>Design of Experiments</title>
<p>The results of global sensitivity analysis can be dependent on the distribution of input variables [
<xref rid="pone.0136139.ref067" ref-type="bibr">67</xref>
]. In this work we have performed global sensitivity analysis on ENISI using 25 input factors. Each experiment has 250 time steps. Each simulation time step represents six hours in real world. For each factor, there are four different levels. Note that if one conducts a full factorial design for the 25 factors, it requires 4
<sup>25</sup>
= 1.126 × 10
<sup>15</sup>
runs of experiments. Running one experiment in ENISI can take ninety minutes on average. Clearly, it is not possible to consider all possible combinations of parameter settings. A sparse design with economic run size is needed to conduct the sensitivity analysis.</p>
<p>As an effective statistical technique for sensitivity analysis, design of experiments (DOE) provides a way to carry out simulation experiments systematically that takes into account parameter interactions [
<xref rid="pone.0136139.ref068" ref-type="bibr">68</xref>
,
<xref rid="pone.0136139.ref069" ref-type="bibr">69</xref>
]. By using experimental design, one can systematically bring statistical aspects into the analysis of results [
<xref rid="pone.0136139.ref070" ref-type="bibr">70</xref>
,
<xref rid="pone.0136139.ref071" ref-type="bibr">71</xref>
]. It helps understand the relationships between input parameters and outputs in the model. DOE originates from real world experimentation in agriculture and engineering, and the techniques can be transferred to experiments with computer simulation such as agent-based models [
<xref rid="pone.0136139.ref072" ref-type="bibr">72</xref>
]. In the context of agent-based models, sensitivity analysis using DOE can uncover details about model behavior, help to identify the relative importance of inputs, and provide a common basis for discussing simulation results.</p>
<p>In traditional DOE a fractional factorial design, which considers a fraction of all possible level combinations, is often used for the analysis [
<xref rid="pone.0136139.ref073" ref-type="bibr">73</xref>
]. However, for a problem with a large number of parameters, the number of runs of the fractional factorial design is still large. To overcome the drawback of using a fractional factorial design, we utilize an experimental design strategy using an orthogonal array [
<xref rid="pone.0136139.ref065" ref-type="bibr">65</xref>
,
<xref rid="pone.0136139.ref068" ref-type="bibr">68</xref>
,
<xref rid="pone.0136139.ref073" ref-type="bibr">73</xref>
] to obtain a sparse design with desirable properties. The designs of orthogonal arrays often have small run sizes, but can accommodate all possible level combinations for at least any two columns. Specifically, an orthogonal array (OA) with strength
<italic>t</italic>
in experimental design is a design matrix such that for every
<italic>t</italic>
column, the possible distinct rows all appear the same number of times. That is, when projecting the designs in OA onto any subspace with dimensionality
<italic>t</italic>
, the coverage of parameter space is the same as that using full factorial design in
<italic>t</italic>
-dimensional space. Such a property maintains a good balance of level assignment for each factor. A random design will not guarantee this property [
<xref rid="pone.0136139.ref065" ref-type="bibr">65</xref>
,
<xref rid="pone.0136139.ref068" ref-type="bibr">68</xref>
,
<xref rid="pone.0136139.ref073" ref-type="bibr">73</xref>
,
<xref rid="pone.0136139.ref074" ref-type="bibr">74</xref>
]. Thus, sensitivity analysis based on a design of orthogonal array enables the comprehensive investigation of parameters without dramatically increasing the number of simulation runs. Designs of orthogonal array have been widely used in many engineering applications [
<xref rid="pone.0136139.ref074" ref-type="bibr">74</xref>
] and sensitivity analysis of computer models [
<xref rid="pone.0136139.ref075" ref-type="bibr">75</xref>
]. For the sensitivity analysis of ENISI, we consider an orthogonal array of 128 runs on 25 factors with strength 2 (see
<xref ref-type="supplementary-material" rid="pone.0136139.s004">S2 File</xref>
for the complete design). The details of this design can be seen in [
<xref rid="pone.0136139.ref074" ref-type="bibr">74</xref>
]. This design has the following attractive properties: (i) projecting the design points onto any factor, there are exactly 16 replicates for each level; (ii) projecting the design points onto any two variables (i.e., any two columns of the design matrix), it is a full factorial (4 × 4) with 8 replications for each level combination.
<xref ref-type="fig" rid="pone.0136139.g001">Fig 1</xref>
illustrates an OA design with 128 runs in 25-dimensional space. When it project points to a three-dimension subspace (labeled as
<italic>x</italic>
<sub>1</sub>
,
<italic>x</italic>
<sub>2</sub>
, and
<italic>x</italic>
<sub>3</sub>
), the projected points consist of full factorial design in any two dimensional space of (
<italic>x</italic>
<sub>1</sub>
,
<italic>x</italic>
<sub>2</sub>
), (
<italic>x</italic>
<sub>1</sub>
,
<italic>x</italic>
<sub>3</sub>
), and (
<italic>x</italic>
<sub>2</sub>
,
<italic>x</italic>
<sub>3</sub>
). A random design will not be able to achieve this property. Hence, the proposed design is sparse in the sense that there are only 128 points in a 25-dimensional space, while the design points are well spread out with good properties. Therefore, the proposed design enables the effective study of the main effect of any factor.</p>
<fig id="pone.0136139.g001" orientation="portrait" position="float">
<object-id pub-id-type="doi">10.1371/journal.pone.0136139.g001</object-id>
<label>Fig 1</label>
<caption>
<title>Illustration of orthogonal array experimental design with 128 runs and 25 factors.</title>
<p>projecting the design points onto any two-dimensional subspace, the projected points always form a full factorial (4 × 4) with 8 replications for each level of combination.</p>
</caption>
<graphic xlink:href="pone.0136139.g001"></graphic>
</fig>
<sec id="sec013">
<title>Experiments and Analysis</title>
<p>We used a high performance computing cluster of 48 Intel Sandy Bridge nodes. Each node consists of two dual-socket Intel Sandy Bridge E5-2670 2.60GHz 8-core processors (16 cores per node) and 64GB of 1600MHz DDR3 RAM. The nodes are linked by QLogic QDR InfiniBand interconnects. For the MPI based implementation of our algorithms, we used the MPICH2 (version 1.7), which is optimized for QLogic InfiniBand cards. For each experiment we used 48 processors. On average it took ninety minutes to complete one simulation. Our cluster allowed us to run around five simulations in parallel.</p>
<p>According to the design of experiment we selected 128 sets of distinct parameter settings. As the model is stochastic in nature, there is inherent uncertainty in the output even though all the input parameters are kept same. Therefore we repeated the experiment for every parameter settings 15 times. In total 128 × 15 = 1920 simulation runs were conducted for global sensitivity analysis. A summary of the experiments is given in
<xref ref-type="table" rid="pone.0136139.t003">Table 3</xref>
.</p>
<table-wrap id="pone.0136139.t003" orientation="portrait" position="float">
<object-id pub-id-type="doi">10.1371/journal.pone.0136139.t003</object-id>
<label>Table 3</label>
<caption>
<title>Summary of experimental design.</title>
</caption>
<alternatives>
<graphic id="pone.0136139.t003g" xlink:href="pone.0136139.t003"></graphic>
<table frame="box" rules="all" border="0">
<colgroup span="1">
<col align="left" valign="top" span="1"></col>
<col align="left" valign="top" span="1"></col>
</colgroup>
<tbody>
<tr>
<td align="left" rowspan="1" colspan="1">Input size</td>
<td align="center" rowspan="1" colspan="1">10
<sup>7</sup>
individual cells</td>
</tr>
<tr>
<td align="left" rowspan="1" colspan="1">Number of parameters</td>
<td align="center" rowspan="1" colspan="1">25</td>
</tr>
<tr>
<td align="left" rowspan="1" colspan="1">Number of observed output variables</td>
<td align="center" rowspan="1" colspan="1">9</td>
</tr>
<tr>
<td align="left" rowspan="1" colspan="1">Number of experiments</td>
<td align="center" rowspan="1" colspan="1">128</td>
</tr>
<tr>
<td align="left" rowspan="1" colspan="1">Number of replicates</td>
<td align="center" rowspan="1" colspan="1">15</td>
</tr>
</tbody>
</table>
</alternatives>
</table-wrap>
</sec>
</sec>
</sec>
<sec sec-type="results" id="sec014">
<title>Results</title>
<p>Aim 1: Determine the influence of parameters by performing global sensitivity analysis using partial factorial experiments. An analysis of variance (ANOVA) is also performed to compute the level of significance for every parameter. Aim 2: Ranking the parameters (from greatest to least effect) from Aim 1, followed by studying the underlying causes with biological validation. Aim 3: Fine tuning of parameter values by applying local sensitivity analysis to the model.</p>
<sec id="sec015">
<title>Global sensitivity analysis determines the influence of a modeling parameter</title>
<p>The first attempt is to determine if there are any patterns of parameter influence on the system. To achieve this, we performed global sensitivity analysis of the system by following the design described in the “Materials and Methods” section. In this design each modeling parameter takes four monotonically increasing discrete values called “Levels.” Thus every parameter has four values: Level 1 to 4.</p>
<p>ENISI provides the total number of agents per phenotype in each time step as the system output. Typically these output are represented as time series plots, where the
<italic>x</italic>
and
<italic>y</italic>
axes represent time step and the number of cells respectively. To quantify the influence of any parameter on the model output, we generated time series plots for each input parameter. The time series plot for a parameter has four curves, each belonging to a distinct parameter Level. As shown in the design diagram, a total of 128 experiments with different parameter settings were performed, where each experiment was replicated 15 times. According to the experimental design, each parameter has 32 distinct experiments per level. Hence, each curve of the time series plot has 32 × 15 = 480 simulation runs. The average of the runs are drawn along with the associated error bars in each curve. For example,
<xref ref-type="fig" rid="pone.0136139.g002">Fig 2(a)</xref>
shows the time series curves (along with error-bars) of
<italic>iTreg</italic>
cells in the gastric lymph node (GLN) for modeling parameter
<italic>v</italic>
<sub>
<italic>T</italic>
</sub>
.</p>
<fig id="pone.0136139.g002" orientation="portrait" position="float">
<object-id pub-id-type="doi">10.1371/journal.pone.0136139.g002</object-id>
<label>Fig 2</label>
<caption>
<title>Timeseries and Main Effects of
<italic>iTreg</italic>
cells in gastric lymph node for Parameter
<italic>v</italic>
<sub>
<italic>T</italic>
</sub>
.</title>
</caption>
<graphic xlink:href="pone.0136139.g002"></graphic>
</fig>
<p>Though these curves provide intuitive visual representation, comparisons between curves are difficult. These curves also fail to show the changes of system output when the levels of the corresponding modeling parameter are changed. Therefore, we used a different representation called a “main effect” plot (see
<xref ref-type="fig" rid="pone.0136139.g002">Fig 2(b)</xref>
). In a main effect plot, the x-axis represents the levels of the modeling parameter. The y-axis represents the average number of cells per level over the entire time period. Note that each experiment has 250 time steps and is replicated 15 times for each level. The average number of cells per level is determined from the average of all these experiments over the 250 time steps for the main effect plot. In this way, a time series curve is converted into a scalar value.
<xref ref-type="fig" rid="pone.0136139.g002">Fig 2(b)</xref>
shows the main effect plot of
<italic>iTreg</italic>
phenotype in the gastric lymph node (GLN) for parameter
<italic>v</italic>
<sub>
<italic>T</italic>
</sub>
. From this, it is easy to see that with increasing levels of
<italic>v</italic>
<sub>
<italic>T</italic>
</sub>
, the average number of
<italic>iTreg</italic>
phenotypes in GLN increases in a monotonic fashion. Though the pattern of this particular parameter is clear, there exist other patterns of main effect plots that are more challenging to interpret. We will address these patterns herein.</p>
<sec id="sec016">
<title>Main effect plots can be classified into four distinct patterns</title>
<p>The analysis of the main effect plots for every output reveals four distinct patterns. They are: 1) monotonic, 2) bell shaped, 3) sigmoid, and 4) complex patterns. These are shown in
<xref ref-type="fig" rid="pone.0136139.g003">Fig 3</xref>
.</p>
<fig id="pone.0136139.g003" orientation="portrait" position="float">
<object-id pub-id-type="doi">10.1371/journal.pone.0136139.g003</object-id>
<label>Fig 3</label>
<caption>
<title>Four distinct patterns for
<italic>iTreg</italic>
cells in the LP.</title>
</caption>
<graphic xlink:href="pone.0136139.g003"></graphic>
</fig>
<p>The monotonic pattern in
<xref ref-type="fig" rid="pone.0136139.g003">Fig 3(a)</xref>
shows that increasing the values of the parameter
<italic>a</italic>
<sub>
<italic>T</italic>
</sub>
, results in the average cell count going up (or down) monotonically. The bell shaped pattern (
<xref ref-type="fig" rid="pone.0136139.g003">Fig 3(b)</xref>
) shows the average number of cells in the middle of the parameter range are significantly higher (or lower) than the extreme ends of the parameter range. The sigmoid pattern is shown in
<xref ref-type="fig" rid="pone.0136139.g003">Fig 3(c)</xref>
. Notice that between Level 2 and Level 3 the average number of cells are changed dynamically but Level 2 & Level 1 and Level 3 & Level 4 have almost the same average number of cells. Thus, sigmoid patterns can be considered a switch between two consecutive parameter levels. The complex pattern in
<xref ref-type="fig" rid="pone.0136139.g003">Fig 3(d)</xref>
shows the average number of cells vary significantly between any two consecutive parameter levels, thereby forming a zig-zag shaped complex pattern.</p>
</sec>
<sec id="sec017">
<title>Monotonic main effect patterns have significant influence over the system</title>
<p>One of the prominent effects of a parameter on a complex system is monotonic in nature. In this case, with the increase of a parameter we observe monotonic increase or decrease of the model output.
<xref ref-type="fig" rid="pone.0136139.g004">Fig 4</xref>
shows the time series and main effect plot of induced regulatory T cells in the gastric lamina propria for the parameter
<italic>a</italic>
<sub>
<italic>T</italic>
</sub>
. Parameter
<italic>a</italic>
<sub>
<italic>T</italic>
</sub>
represents the stimulation rate of naïve T cells into active cells. With the increase of
<italic>a</italic>
<sub>
<italic>T</italic>
</sub>
the population of Treg cells is increased. From the main effect plot, we can clearly observe that with the increase of parameter
<italic>a</italic>
<sub>
<italic>T</italic>
</sub>
= 0 (Level 1) to
<italic>a</italic>
<sub>
<italic>T</italic>
</sub>
= 1 (Level 4), the population of Treg cells are also increased monotonically. Monotonic relationships of parameters are very common in immune responses and play crucial role in biological systems. Indeed, experimental data published in [
<xref rid="pone.0136139.ref064" ref-type="bibr">64</xref>
] shows an increase in numbers of the regulatory CD4+ T cells (Treg) subset during
<italic>Helicobacter pylori</italic>
infection. Based on these results, we could partially attribute the upregulation of the regulatory T cell population, among other effects, to the upregulation of the rate of T cell activation due to cytokine signaling [
<xref rid="pone.0136139.ref064" ref-type="bibr">64</xref>
].</p>
<fig id="pone.0136139.g004" orientation="portrait" position="float">
<object-id pub-id-type="doi">10.1371/journal.pone.0136139.g004</object-id>
<label>Fig 4</label>
<caption>
<title>Timeseries and main effects of
<italic>iTreg</italic>
cells in lamina propria for parameter
<italic>a</italic>
<sub>
<italic>T</italic>
</sub>
exhibits monotonic pattern.</title>
</caption>
<graphic xlink:href="pone.0136139.g004"></graphic>
</fig>
</sec>
<sec id="sec018">
<title>Bell shaped patterns reveal interdependency among modeling parameters</title>
<p>The effects of parameters can vary dramatically based on the parameter levels.
<xref ref-type="fig" rid="pone.0136139.g005">Fig 5</xref>
shows the effect of parameter
<italic>p</italic>
<sub>17</sub>
, which is the probability that a resting T cell differentiates to
<italic>Th</italic>
17 with the presence of inflammatory phenotypes. In the figure, on the boundary conditions (Level 1 and Level 4) the average population of
<italic>Th</italic>
17 is lower than the population simulations for Levels 2 and 3. This can be explained by the fact that, at
<italic>p</italic>
<sub>17</sub>
= 0 (Level 1), according to the model, resting T cells do not produce
<italic>Th</italic>
17, instead only inflammatory
<italic>Th</italic>
1 cells are produced.
<italic>Th</italic>
17 is only produced from the transition of
<italic>iTreg</italic>
to
<italic>Th</italic>
17. For this reason, we observe a lower number of
<italic>Th</italic>
17 cells for Level 1. On the other hand, at
<italic>p</italic>
<sub>17</sub>
= 1 (Level 4) all resting T cells produce only
<italic>Th</italic>
17 cells but not
<italic>Th</italic>
1. But, this differentiation requires inflammatory phenotypes such as
<italic>Th</italic>
1. Since no
<italic>Th</italic>
1 is directly produced from resting T cells by the model, there is a lack of inflammatory phenotypes to make the transition from resting T to
<italic>Th</italic>
17. For this reason, a lower number of
<italic>Th</italic>
17 cells is observed. For the intermediate levels (Level 2 & 3) a higher number of
<italic>Th</italic>
17 cells is observed as there is a balance between the rate of resting T cells to
<italic>Th</italic>
17 cell transition and an adequate number of
<italic>Th</italic>
1 cells as activators. This depicts one of the complex behaviors of parameter levels, which are typically bell shaped or inverted bell shaped in the model output.</p>
<fig id="pone.0136139.g005" orientation="portrait" position="float">
<object-id pub-id-type="doi">10.1371/journal.pone.0136139.g005</object-id>
<label>Fig 5</label>
<caption>
<title>Timeseries and main effects of
<italic>Th</italic>
17 cells in GLN for parameter
<italic>p</italic>
<sub>17</sub>
exhibits bell shaped pattern.</title>
</caption>
<graphic xlink:href="pone.0136139.g005"></graphic>
</fig>
</sec>
<sec id="sec019">
<title>Sigmoid main effect patterns exhibit switching behavior</title>
<p>Minor differences in parameter levels play crucial roles in delineation of model output. For instance, in
<xref ref-type="fig" rid="pone.0136139.g006">Fig 6</xref>
, when the parameter
<italic>i</italic>
<sub>
<italic>r</italic>
</sub>
’s value is changed from Level 1 to Level 2 we observe a dramatic change. However, we do not observe changes when Levels 3 or 4 are incremented. It appears that the effect of parameter value of
<italic>i</italic>
<sub>
<italic>r</italic>
</sub>
is saturated after Level 2. Hence the significant value of
<italic>i</italic>
<sub>
<italic>r</italic>
</sub>
must be within Levels 1 and 2. Parameter
<italic>i</italic>
<sub>
<italic>r</italic>
</sub>
is the inhibitory factor of
<italic>iTreg</italic>
to
<italic>Th</italic>
17 transition. The Treg population is predominant over the
<italic>Th</italic>
17 cells during
<italic>H. pylori</italic>
infection. Hence, for lower values, a slight change of
<italic>i</italic>
<sub>
<italic>r</italic>
</sub>
parameter has a significant effect on the system. Once the regulatory T cell population is significantly increased, the difference observed in these values is lower for values higher than 1, as the probability of transition from
<italic>iTreg</italic>
to
<italic>Th</italic>
17 is already very low. This kind of behavior results in a sigmoid main effect plot. Thus, a dramatic transition between two parameter values are observed; however, other values beyond the two levels do not have significant effect on the system output.</p>
<fig id="pone.0136139.g006" orientation="portrait" position="float">
<object-id pub-id-type="doi">10.1371/journal.pone.0136139.g006</object-id>
<label>Fig 6</label>
<caption>
<title>Timeseries and main effects of
<italic>iTreg</italic>
cells in LP for parameter
<italic>i</italic>
<sub>
<italic>r</italic>
</sub>
exhibits sigmoid pattern.</title>
</caption>
<graphic xlink:href="pone.0136139.g006"></graphic>
</fig>
</sec>
<sec id="sec020">
<title>The behaviors of complex patterns are not well understood</title>
<p>A fourth pattern is also observed in this study. With the increase of parameter values from Level 1 to Level 4, the number of cells increases and decreases creating a zig-zag shaped pattern in the main effect plot (see
<xref ref-type="fig" rid="pone.0136139.g007">Fig 7</xref>
). For example, consider the effect of parameter
<italic>μ</italic>
<sub>
<italic>CE</italic>
</sub>
on
<italic>iTreg</italic>
cells in lamina propria. Parameter
<italic>μ</italic>
<sub>
<italic>CE</italic>
</sub>
is the probability that pro-inflammatory epithelial cells are damaged by the inflammatory factors. By moving from Level 1 to Level 2, it is observed that the population of
<italic>iTreg</italic>
cells falls drastically and rises again for Level 3. The cell population falls again for Level 4. This type of parameter behavior requires further investigation.</p>
<fig id="pone.0136139.g007" orientation="portrait" position="float">
<object-id pub-id-type="doi">10.1371/journal.pone.0136139.g007</object-id>
<label>Fig 7</label>
<caption>
<title>Timeseries and main effects of
<italic>iTreg</italic>
cells in the GLN for parameter
<italic>μ</italic>
<sub>
<italic>CE</italic>
</sub>
exhibits complex pattern.</title>
</caption>
<graphic xlink:href="pone.0136139.g007"></graphic>
</fig>
</sec>
</sec>
<sec id="sec021">
<title>ANOVA based analysis of global sensitivities</title>
<p>For the output immune responses and its 25 input parameters, it is important to investigate the underlying causes for how the parameters affect model outputs. To determine the impact of each parameter on the ENISI model, analysis of variance (ANOVA) was performed to find the most significant parameters. The key idea behind ANOVA use here is to link output and input through a statistical model with a normally distributed random error term. Based on the data, one can fit the model and quantify how the total variation of the output can be decomposed to variations contributed by estimated parameters, thus explaining the effects of sensitivity of input parameters.</p>
<p>Note that there are 128 different parameter setting for global sensitivity analysis, and each setting is replicated for 15 times in simulations. Therefore, there are 128 × 15 = 1920 data points for the ANOVA. For the output, the average count of each cell type over a time period (e.g., the entire time period or a time segmentation) as the response was used. In this study, by applying the ANOVA including the main effects, the observed variance in the output response is partitioned into components attributable to the 25 different parameters. Such an analysis provides a statistical quantification of whether or not each parameter plays a vital role to contribute the observed total variation of response output. Specifically, the significance of a parameter contributing to the total variation is examined by the statistical hypothesis testing using
<italic>F</italic>
statistic [
<xref rid="pone.0136139.ref076" ref-type="bibr">76</xref>
]. The
<italic>F</italic>
statistic is obtained by computing the ratio between the variation of output explained by the parameter and the total variation of response output. Thus, a large value of
<italic>F</italic>
statistic indicates a strong evidence of the significance of the parameter. The
<italic>p</italic>
-value based on the
<italic>F</italic>
test can be calculated accordingly [
<xref rid="pone.0136139.ref077" ref-type="bibr">77</xref>
], having small values with large
<italic>F</italic>
statistic. A statistically significant result with
<italic>p</italic>
-value lower than a user-defined threshold (i.e. significance level) justifies the rejection of the null hypothesis that the parameter is insignificant. Here the significant level is set to be 0.05. Recall that the design used for the experiment is an orthogonal array with 128 runs. Such a design makes each column of the design matrix mutually orthogonal to each other. As a result, the ANOVA based on such a design allows the estimated main effects to not correlate with each other. It provides an efficient technique to determine the sensitivity of each parameter with respect to the output of cell counts.</p>
<p>To ensure the validity of the ANOVA results, we also examine the model assumptions of the ANOVA approach. Note that the ANOVA method assumes that the independent and identically distributed error term exhibits normal distribution. For the global sensitivity analysis, we use the residual scatter plot to examine the independence of the data, and examine the normality assumption of the error terms through the histogram of residuals and the QQ-normal plot. The scatter plot, histogram, and QQ-normal plots of the residuals for the nine cells types as responses are shown in
<xref ref-type="supplementary-material" rid="pone.0136139.s002">S1 Fig</xref>
. As we can see from the figures, the independence and normality assumptions are generally valid when using the ANOVA for the global sensitivity analysis for our data. It is worth pointing out there are some potential outliers existing in the data, which could distort the model assumptions to some extent. Detailed investigation of those abnormal points with domain experts are often recommended before removing these points from further analysis.</p>
<sec id="sec022">
<title>Overall significance of modeling parameters</title>
<p>Using the technique described above, ANOVA analysis was performed to calculate the
<italic>p</italic>
-values of significance of a parameter for each of the nine output variables. This is shown in
<xref ref-type="fig" rid="pone.0136139.g008">Fig 8</xref>
. The x-axis represents the 25 parameters and the y-axis represents the nine output cell types.</p>
<fig id="pone.0136139.g008" orientation="portrait" position="float">
<object-id pub-id-type="doi">10.1371/journal.pone.0136139.g008</object-id>
<label>Fig 8</label>
<caption>
<title>Overall
<italic>p</italic>
-values of modeling parameters on cell subsets in the synthetic gastric mucosa.</title>
</caption>
<graphic xlink:href="pone.0136139.g008"></graphic>
</fig>
<p>From
<xref ref-type="fig" rid="pone.0136139.g008">Fig 8</xref>
it is clear that some parameters, such as
<italic>a</italic>
<sub>
<italic>T</italic>
</sub>
,
<italic>v</italic>
<sub>
<italic>T</italic>
</sub>
,
<italic>v</italic>
<sub>
<italic>EC</italic>
</sub>
, are significantly important for each cell type. There are certain parameters that are significant only for T cell populations (e.g.
<italic>p</italic>
<sub>17</sub>
,
<italic>y</italic>
<sub>17</sub>
) and some that are responsible for macrophages (e.g.
<italic>b</italic>
<sub>
<italic>r</italic>
</sub>
).</p>
<p>The global sensitivity analysis shows which parameters are most important for model tuning. Hence, it is also helpful to identify the parameters that have the smallest effects on the model. For example it is clear that parameter
<italic>a</italic>
<sub>17</sub>
and
<italic>i</italic>
<sub>17</sub>
have little effect on the system.</p>
</sec>
<sec id="sec023">
<title>Temporal significance of modeling parameters</title>
<p>In addition to measuring sensitivity analysis for a parameter over the entire time period, it is possible to analyze in which time period a parameter is most significant. This is particularly important because it allows investigation of the significance of a parameter over time.</p>
<p>For this analysis we divided the entire time period into nine segments, each representing one experimental week. The one experimental week time interval is chosen from a biological point of view, as we are interested to see how the inflammation progresses per week. Also, in the
<italic>in vivo</italic>
experiments, biological data is collected weekly, which allows us to compare and validate simulated and biological results. Next, the significance of a parameter within the nine week period was plotted. The significance of a parameter is represented by the
<italic>p</italic>
-value for the changes of the parameter determined using ANOVA. The results for parameters
<italic>v</italic>
<sub>
<italic>EC</italic>
</sub>
,
<italic>a</italic>
<sub>
<italic>T</italic>
</sub>
and
<italic>p</italic>
<sub>17</sub>
are given in
<xref ref-type="fig" rid="pone.0136139.g009">Fig 9</xref>
. Major changes in the dynamic significance of parameters were identified. In part the ANOVA findings are intuitive and serve as a measure to ensure that the model behaves in a biologically relevant manner. For instance, the probability of resting T cell stimulation,
<italic>a</italic>
<sub>
<italic>T</italic>
</sub>
, has a high impact on all T cell subsets and activated macrophages with increasing significance over time. This is explained primarily by T cell population density during the course of infection. At the beginning of infection,
<italic>Th</italic>
1 and
<italic>Th</italic>
17 populations are low in contrast to when infection progresses, which is when these cell types become more critical for proliferation and differentiation of additional T cells in the gastric lamina propria (
<xref ref-type="fig" rid="pone.0136139.g009">Fig 9</xref>
).</p>
<fig id="pone.0136139.g009" orientation="portrait" position="float">
<object-id pub-id-type="doi">10.1371/journal.pone.0136139.g009</object-id>
<label>Fig 9</label>
<caption>
<title>Temporal effects of parameters
<italic>a</italic>
<sub>
<italic>T</italic>
</sub>
,
<italic>p</italic>
<sub>17</sub>
and
<italic>v</italic>
<sub>
<italic>EC</italic>
</sub>
on output cells type per week.</title>
</caption>
<graphic xlink:href="pone.0136139.g009"></graphic>
</fig>
<p>One interesting finding is seen in the behavior of epithelial cells in response to inflammatory factors. Specifically, the probability that an epithelial cell transitions into a proinflammatory state as determined by parameter
<italic>v</italic>
<sub>
<italic>EC</italic>
</sub>
has a higher impact on overall immunity when compared to epithelial damage (
<italic>b</italic>
<sub>
<italic>r</italic>
</sub>
). In fact,
<italic>v</italic>
<sub>
<italic>EC</italic>
</sub>
is the single most significant parameter in the model as determined by ANOVA. These results suggest that epithelial cells that come in contact with
<italic>H. pylori</italic>
significantly influence the overall immune response. Moreover, the temporal effects of
<italic>v</italic>
<sub>
<italic>EC</italic>
</sub>
increase as infection progresses and don’t approach the significant
<italic>p</italic>
-value bracket of 0.01–0.1 until at least 4 weeks post-infection, implying that this response wouldn’t be detected until late after infection (
<xref ref-type="fig" rid="pone.0136139.g009">Fig 9c</xref>
). Although out of the scope of this paper, a longer time course simulation should be performed to predict at which time frame this epithelial cell property is most crucial to host immunity to
<italic>H. pylori</italic>
.</p>
</sec>
<sec id="sec024">
<title>Selection of most significant parameters</title>
<p>In order to refine parameter selection we need to determine the most significant parameters based on their impact on model dynamics. To rank the parameters we use two different systems using global sensitivity analysis. First we take average
<italic>p</italic>
-values for each of the output cell types and order them from smallest to largest. In this way, we find that the top ten significant parameters are:
<italic>ν</italic>
<sub>
<italic>EC</italic>
</sub>
,
<italic>a</italic>
<sub>
<italic>T</italic>
</sub>
,
<italic>ν</italic>
<sub>
<italic>T</italic>
</sub>
,
<italic>a</italic>
<sub>2</sub>
<italic>y</italic>
<sub>2</sub>
,
<italic>a</italic>
<sub>1</sub>
,
<italic>i</italic>
<sub>2</sub>
,
<italic>ν</italic>
<sub>
<italic>Bs</italic>
</sub>
,
<italic>p</italic>
<sub>17</sub>
, and
<italic>b</italic>
<sub>
<italic>r</italic>
</sub>
. We also ranked the parameters based on how many outputs are significant for each parameter. For this type of ranking we find that:
<italic>a</italic>
<sub>
<italic>T</italic>
</sub>
,
<italic>ν</italic>
<sub>
<italic>T</italic>
</sub>
,
<italic>ν</italic>
<sub>
<italic>EC</italic>
</sub>
,
<italic>y</italic>
<sub>2</sub>
,
<italic>p</italic>
<sub>17</sub>
,
<italic>i</italic>
<sub>2</sub>
,
<italic>y</italic>
<sub>17</sub>
,
<italic>a</italic>
<sub>2</sub>
,
<italic>a</italic>
<sub>1</sub>
, and
<italic>ν</italic>
<sub>
<italic>Bs</italic>
</sub>
are the top ten significant parameters. Both lists are almost identical other than
<italic>b</italic>
<sub>
<italic>r</italic>
</sub>
and
<italic>y</italic>
<sub>17</sub>
. Parameter
<italic>y</italic>
<sub>17</sub>
can be considered more important if
<italic>p</italic>
-values and the number of significant output parameters are compared.</p>
</sec>
</sec>
<sec id="sec025">
<title>Local sensitivity analysis</title>
<p>Global sensitivity analysis measures the significance of each parameter. We can rank the most significant parameters based on
<italic>p</italic>
-values as described in the previous section. We choose the top ten significant parameters to analyze their local sensitivity behavior. The ten most significant parameters along with the corresponding test values are shown in
<xref ref-type="table" rid="pone.0136139.t004">Table 4</xref>
.</p>
<table-wrap id="pone.0136139.t004" orientation="portrait" position="float">
<object-id pub-id-type="doi">10.1371/journal.pone.0136139.t004</object-id>
<label>Table 4</label>
<caption>
<title>The list of parameter values used for mono-factorial sensitivity analysis.</title>
</caption>
<alternatives>
<graphic id="pone.0136139.t004g" xlink:href="pone.0136139.t004"></graphic>
<table frame="box" rules="all" border="0">
<colgroup span="1">
<col align="left" valign="top" span="1"></col>
<col align="center" valign="top" span="1"></col>
<col align="center" valign="top" span="1"></col>
<col align="center" valign="top" span="1"></col>
<col align="center" valign="top" span="1"></col>
</colgroup>
<thead>
<tr>
<th align="left" rowspan="1" colspan="1">Parameter</th>
<th align="center" rowspan="1" colspan="1">Value 1</th>
<th align="center" rowspan="1" colspan="1">Value 2</th>
<th align="center" rowspan="1" colspan="1">Value 3</th>
<th align="center" rowspan="1" colspan="1">Value 4</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left" rowspan="1" colspan="1">
<italic>a</italic>
<sub>
<italic>T</italic>
</sub>
</td>
<td align="center" rowspan="1" colspan="1">0.5</td>
<td align="center" rowspan="1" colspan="1">0.7</td>
<td align="center" rowspan="1" colspan="1">0.8</td>
<td align="center" rowspan="1" colspan="1">1</td>
</tr>
<tr>
<td align="left" rowspan="1" colspan="1">
<italic>v</italic>
<sub>
<italic>T</italic>
</sub>
</td>
<td align="center" rowspan="1" colspan="1">0.5</td>
<td align="center" rowspan="1" colspan="1">0.7</td>
<td align="center" rowspan="1" colspan="1">0.8</td>
<td align="center" rowspan="1" colspan="1">1</td>
</tr>
<tr>
<td align="left" rowspan="1" colspan="1">
<italic>v</italic>
<sub>
<italic>EC</italic>
</sub>
</td>
<td align="center" rowspan="1" colspan="1">0.01</td>
<td align="center" rowspan="1" colspan="1">0.1</td>
<td align="center" rowspan="1" colspan="1">0.25</td>
<td align="center" rowspan="1" colspan="1">0.5</td>
</tr>
<tr>
<td align="left" rowspan="1" colspan="1">
<italic>y</italic>
<sub>2</sub>
</td>
<td align="center" rowspan="1" colspan="1">0.1</td>
<td align="center" rowspan="1" colspan="1">1</td>
<td align="center" rowspan="1" colspan="1">4</td>
<td align="center" rowspan="1" colspan="1">10</td>
</tr>
<tr>
<td align="left" rowspan="1" colspan="1">
<italic>p</italic>
<sub>17</sub>
</td>
<td align="center" rowspan="1" colspan="1">0.1</td>
<td align="center" rowspan="1" colspan="1">0.25</td>
<td align="center" rowspan="1" colspan="1">0.5</td>
<td align="center" rowspan="1" colspan="1">0.9</td>
</tr>
<tr>
<td align="left" rowspan="1" colspan="1">
<italic>i</italic>
<sub>2</sub>
</td>
<td align="center" rowspan="1" colspan="1">0.1</td>
<td align="center" rowspan="1" colspan="1">1</td>
<td align="center" rowspan="1" colspan="1">4</td>
<td align="center" rowspan="1" colspan="1">10</td>
</tr>
<tr>
<td align="left" rowspan="1" colspan="1">
<italic>y</italic>
<sub>17</sub>
</td>
<td align="center" rowspan="1" colspan="1">0.1</td>
<td align="center" rowspan="1" colspan="1">1</td>
<td align="center" rowspan="1" colspan="1">4</td>
<td align="center" rowspan="1" colspan="1">10</td>
</tr>
<tr>
<td align="left" rowspan="1" colspan="1">
<italic>a</italic>
<sub>2</sub>
</td>
<td align="center" rowspan="1" colspan="1">0.1</td>
<td align="center" rowspan="1" colspan="1">11</td>
<td align="center" rowspan="1" colspan="1">10</td>
<td align="center" rowspan="1" colspan="1">5</td>
</tr>
<tr>
<td align="left" rowspan="1" colspan="1">
<italic>a</italic>
<sub>1</sub>
</td>
<td align="center" rowspan="1" colspan="1">1</td>
<td align="center" rowspan="1" colspan="1">1</td>
<td align="center" rowspan="1" colspan="1">10</td>
<td align="center" rowspan="1" colspan="1">5</td>
</tr>
<tr>
<td align="left" rowspan="1" colspan="1">
<italic>v</italic>
<sub>
<italic>Bs</italic>
</sub>
</td>
<td align="center" rowspan="1" colspan="1">0.05</td>
<td align="center" rowspan="1" colspan="1">0</td>
<td align="center" rowspan="1" colspan="1">1</td>
<td align="center" rowspan="1" colspan="1">0.5</td>
</tr>
<tr>
<td align="left" rowspan="1" colspan="1">
<italic>μ</italic>
<sub>
<italic>M</italic>
1</sub>
</td>
<td align="center" rowspan="1" colspan="1">1</td>
<td align="center" rowspan="1" colspan="1">0.5</td>
<td align="center" rowspan="1" colspan="1">0.25</td>
<td align="center" rowspan="1" colspan="1">0.0</td>
</tr>
</tbody>
</table>
</alternatives>
</table-wrap>
<p>To study the effect of a single parameter over the population, we created a base set of parameter values called the base configuration from studies in literature and experimental calibration process. As the global sensitivity results show the region of interest, we selected three new different values from the region. Thus every parameter has four different values.
<xref ref-type="table" rid="pone.0136139.t004">Table 4</xref>
lists all the parameters and the tested values.</p>
<p>In the following sections, the local effects of the top two significant parameters
<italic>p</italic>
<sub>17</sub>
and
<italic>v</italic>
<sub>
<italic>EC</italic>
</sub>
along with a biologically interesting parameter
<italic>μ</italic>
<sub>
<italic>M</italic>
1</sub>
are described in detail.</p>
<sec id="sec026">
<title>Parameter
<italic>p</italic>
<sub>17</sub>
control the T cell dynamics significantly</title>
<p>Parameter
<italic>p</italic>
<sub>17</sub>
is the probability of resting T cell stimulation to
<italic>Th</italic>
17. The default value for the model is set at 0.1. We also observed the output for values of 0.25, 0.5, and 0.9. The experimental results are shown in
<xref ref-type="fig" rid="pone.0136139.g010">Fig 10</xref>
. When the value of
<italic>p</italic>
<sub>17</sub>
is increased, resting T cells are more likely to produce
<italic>Th</italic>
17
<italic>Prolif</italic>
than
<italic>Th</italic>
1
<italic>Prolif</italic>
. So, production of
<italic>Th</italic>
17 is increased. We can see that with increasing
<italic>p</italic>
<sub>17</sub>
we should create lower numbers of
<italic>Th</italic>
1 as compared to
<italic>Th</italic>
17. We also implemented transition of resting T to contact dependent transition. The transition depends on
<italic>eDC</italic>
and
<italic>M</italic>
1. As
<italic>p</italic>
<sub>17</sub>
does not directly affect
<italic>M</italic>
1 or
<italic>eDC</italic>
, we do not observe much change in the dynamics of
<italic>M</italic>
1 and
<italic>eDC</italic>
for
<italic>p</italic>
<sub>17</sub>
. So, the effect of parameter
<italic>p</italic>
<sub>17</sub>
for the
<italic>Th</italic>
1,
<italic>Th</italic>
17 and
<italic>iTreg</italic>
dynamics are not very sensitive.</p>
<fig id="pone.0136139.g010" orientation="portrait" position="float">
<object-id pub-id-type="doi">10.1371/journal.pone.0136139.g010</object-id>
<label>Fig 10</label>
<caption>
<title>Effect of parameter
<italic>p</italic>
<sub>17</sub>
on
<italic>Th</italic>
1,
<italic>Th</italic>
17,
<italic>iTreg</italic>
,
<italic>M</italic>
0,
<italic>M</italic>
1, and
<italic>M</italic>
2 phenotypes.</title>
</caption>
<graphic xlink:href="pone.0136139.g010"></graphic>
</fig>
</sec>
<sec id="sec027">
<title>Parameter
<italic>v</italic>
<sub>
<italic>EC</italic>
</sub>
has very high sensitivity</title>
<p>Parameter
<italic>v</italic>
<sub>
<italic>EC</italic>
</sub>
is the probability that EC cells transition into
<italic>pEcell</italic>
. The experimental results are shown in
<xref ref-type="fig" rid="pone.0136139.g011">Fig 11</xref>
. The default value of
<italic>v</italic>
<sub>
<italic>EC</italic>
</sub>
is 0.01. This is the lowest probability value of the parameter that was tested. When we increased the probability to a slightly higher value of 0.1, we observed significant change. Increasing
<italic>v</italic>
<sub>
<italic>EC</italic>
</sub>
from 0.01 to 0.1 increases
<italic>iTreg</italic>
cells at the GLN, and
<italic>M</italic>
0,
<italic>M</italic>
1, and
<italic>M</italic>
2 macrophages in the LP primarily, and exerts a lesser impact in
<italic>Th</italic>
1 and
<italic>Th</italic>
17 cell subsets. A recent paper implicates Sonic Hedgehog, a regulator of gastric epithelial cell differentiation and function, which is increased after
<italic>H. pylori</italic>
infection [
<xref rid="pone.0136139.ref078" ref-type="bibr">78</xref>
], as critical for macrophage recruitment to the stomach LP early after
<italic>H. pylori</italic>
infection [
<xref rid="pone.0136139.ref079" ref-type="bibr">79</xref>
].</p>
<fig id="pone.0136139.g011" orientation="portrait" position="float">
<object-id pub-id-type="doi">10.1371/journal.pone.0136139.g011</object-id>
<label>Fig 11</label>
<caption>
<title>Effect of parameter
<italic>v</italic>
<sub>
<italic>EC</italic>
</sub>
on
<italic>Th</italic>
1,
<italic>Th</italic>
17,
<italic>iTreg</italic>
,
<italic>M</italic>
0,
<italic>M</italic>
1, and
<italic>M</italic>
2 phenotypes.</title>
</caption>
<graphic xlink:href="pone.0136139.g011"></graphic>
</fig>
</sec>
<sec id="sec028">
<title>Parameter
<italic>μ</italic>
<sub>
<italic>M</italic>
1</sub>
controls macrophage dynamics</title>
<p>Parameter
<italic>μ</italic>
<sub>
<italic>M</italic>
1</sub>
represents the ability for pro-inflammatory
<italic>M</italic>
1 cells to eliminate bacteria. The default value of
<italic>μ</italic>
<sub>
<italic>M</italic>
1</sub>
is 1. To observe the effect of
<italic>μ</italic>
<sub>
<italic>M</italic>
1</sub>
on the system, the value was changed to 0.5, 0.25 and 0.0 in decreasing order such that fewer bacteria are killed. The ability for macrophages to rid the system of
<italic>H. pylori</italic>
caused the most noticeable changes in a peak of lamina propria myeloid cells (
<italic>M</italic>
0) between days 10 and 22 (
<xref ref-type="fig" rid="pone.0136139.g012">Fig 12</xref>
). More than a 10-day delay occurs in the
<italic>M</italic>
0 population when
<italic>μ</italic>
<sub>
<italic>M</italic>
1</sub>
from is shifted from 0.25 to 0.5. Additionally, at the higher killing capacity (
<italic>μ</italic>
<sub>
<italic>M</italic>
1</sub>
= 0.5) bacteria is cleared more rapidly but causes more significant inflammation when compared to
<italic>μ</italic>
<sub>
<italic>M</italic>
1</sub>
= 0.25. When
<italic>μ</italic>
<sub>
<italic>M</italic>
1</sub>
is set to its default value, the myeloid population peaks between days 15 and 16 and prevents an overzealous cellular response.</p>
<fig id="pone.0136139.g012" orientation="portrait" position="float">
<object-id pub-id-type="doi">10.1371/journal.pone.0136139.g012</object-id>
<label>Fig 12</label>
<caption>
<title>Effect of parameter
<italic>μ</italic>
<sub>
<italic>M</italic>
1</sub>
on
<italic>Th</italic>
1,
<italic>Th</italic>
17,
<italic>iTreg</italic>
,
<italic>M</italic>
0,
<italic>M</italic>
1, and
<italic>M</italic>
2 phenotypes.</title>
</caption>
<graphic xlink:href="pone.0136139.g012"></graphic>
</fig>
</sec>
</sec>
<sec id="sec029">
<title>Interpretation of sensitivity analysis results as new hypothesis</title>
<p>Based on the sensitivity analysis,
<italic>ν</italic>
<sub>
<italic>EC</italic>
</sub>
is the most critical factor in the system’s dynamics. This finding indicates that perturbations induced by contact of
<italic>H. pylori</italic>
with epithelial cells are the major drivers of the events and reactions that characterize the immune response to the bacteria. The great majority of
<italic>H. pylori</italic>
in the gastric mucosa is found free-floating in the mucus layer and associated with the apical side of epithelial cells [
<xref rid="pone.0136139.ref080" ref-type="bibr">80</xref>
,
<xref rid="pone.0136139.ref081" ref-type="bibr">81</xref>
]. Thus, the model recapitulates this principle and the sensitivity analysis indicates that the epithelium is the compartment which receives the major impact of the infection. An interesting and less intuitive finding of the model is the relationship between epithelium and macrophages. Based on analysis across
<italic>ν</italic>
<sub>
<italic>EC</italic>
</sub>
ranges, activation of epithelial cells by
<italic>H. pylori</italic>
leads to a sequential accumulation of macrophage precursors (
<italic>M</italic>
0) between days 0 and 14/21 post-infection. This first wave is followed by a subset of
<italic>M</italic>
2 with regulatory/suppressor role, which peaks between days 21 and 35 post-infection. Starting at day 45 there is a predominance of
<italic>M</italic>
1 pro-inflammatory macrophages. This sequence of changes is caused by increased probability of epithelial cells transitioning to a proinflammatory state, which should directly induce a proinflammatory macrophage. It will be interesting to screen for the presence of regulatory macrophages in the gastric mucosa following
<italic>H. pylori</italic>
infection and evaluate their role in the outcome of the infection.</p>
</sec>
</sec>
<sec sec-type="conclusions" id="sec030">
<title>Discussion</title>
<p>Sensitivity analysis of a large scale agent-based model is challenging due to complex model structure and expensive computation needed to run a full factorial experiment
<italic>in silico</italic>
. The running time of the models prohibits the consideration of a full factorial combination of parameters for global sensitivity analysis. In this work, we have developed a novel and comprehensive sensitivity analysis by using an effective experimental design strategy. In spite of a large number of parameters and a continuous space of parameter values, the proposed method can capture the global nature of the system and gain proper knowledge of the significance of parameters on the system.</p>
<p>The proposed experimental methods tackle the challenges faced in obtaining meaningful results and proper insight into the model. The design of experiments only needs an economic run size in order to obtain the proper sensitivity analysis on a global scale. The results also showed that the design, with four levels for each parameter, captures some interesting main-effect shapes. We observed the various shapes of main-effect plots and classified them into four general categories. Meaningful explanations are provided to illustrate the aptness of these shapes in the ENISI model with validation from biological reality.</p>
<p>The global sensitivity analysis guided us as to the range of parameter values. The model was then calibrated to a base parameter setting using biological literature and findings from
<italic>in vivo</italic>
experiments [
<xref rid="pone.0136139.ref064" ref-type="bibr">64</xref>
]. To further fine tune the model, we performed local sensitivity analysis of the ten most important parameters. We have varied the values of the parameters around the base value to observe changes in output. Whenever the outputs were deemed closer to experimental results, the parameter value was updated. With a few iterations, we achieved a calibrated model of the mucosal immune responses to
<italic>H. pylori</italic>
infection.</p>
<p>In this study, the global sensitivity analysis of ENISI revealed four distinct patterns for the main effects of the input parameters. For the monotonic pattern for a parameter, its contribution to the output is easy to understand and can be controlled by simply increasing or decreasing the parameter value. Bell shaped patterns reveal complex inter-dependency among more than one parameter. Also the complex zig-zag shaped patterns for a parameter are not yet well understood and require further investigation. Future work can investigate more on inter-dependencies and relationships among the parameters. Orthogonal arrays provided a natural way to capture pairwise interactions. The interaction effect plots can show the interaction of parameters with the system and their interdependencies. If the focus of the study is on three-way or higher order interactions, one can consider orthogonal arrays with a strength larger than two or some continuous-based design such as Latin hypercube [
<xref rid="pone.0136139.ref082" ref-type="bibr">82</xref>
,
<xref rid="pone.0136139.ref083" ref-type="bibr">83</xref>
].</p>
<p>In this work, the experiment and analysis are performed on the
<italic>H. pylori</italic>
model implemented using the ENISI platform, an agent-based model. However, the proposed methodology is general in nature, and can work for other complex simulation systems with a large number modeling parameters, such as models in computational epidemics, social network analysis, etc. The proposed method can also be a useful tool for validation and parameter estimation with a limited budget in terms of experiment time and effort. The temporal sensitivity analysis also reveals the time periods when the system is sensitive to other parameters. To calibrate the computational model with some existing systems, it can be useful to collect experimental data in the specific time period of interest.</p>
</sec>
<sec sec-type="supplementary-material" id="sec031">
<title>Supporting Information</title>
<supplementary-material content-type="local-data" id="pone.0136139.s001">
<label>S1 Text</label>
<caption>
<title>Formalization of ENISI MSM.</title>
<p>(PDF)</p>
</caption>
<media xlink:href="pone.0136139.s001.pdf">
<caption>
<p>Click here for additional data file.</p>
</caption>
</media>
</supplementary-material>
<supplementary-material content-type="local-data" id="pone.0136139.s002">
<label>S1 Fig</label>
<caption>
<title>Residual scatter, histogram, and QQ-normal plots for nine output parameters.</title>
<p>(TIF)</p>
</caption>
<media xlink:href="pone.0136139.s002.tif">
<caption>
<p>Click here for additional data file.</p>
</caption>
</media>
</supplementary-material>
<supplementary-material content-type="local-data" id="pone.0136139.s003">
<label>S1 File</label>
<caption>
<title>Values of parameters for each of the four levels.</title>
<p>(XLSX)</p>
</caption>
<media xlink:href="pone.0136139.s003.xlsx">
<caption>
<p>Click here for additional data file.</p>
</caption>
</media>
</supplementary-material>
<supplementary-material content-type="local-data" id="pone.0136139.s004">
<label>S2 File</label>
<caption>
<title>Orthogonal array of 128 experiment runs on 25 factors.</title>
<p>(XLSX)</p>
</caption>
<media xlink:href="pone.0136139.s004.xlsx">
<caption>
<p>Click here for additional data file.</p>
</caption>
</media>
</supplementary-material>
</sec>
</body>
<back>
<ack>
<p>We thank our external collaborators and members of the Network Dynamics and Simulation Science Laboratory (NDSSL) and the Nutritional Immunology and Molecular Medicine Laboratory (NIMML,
<ext-link ext-link-type="uri" xlink:href="http://www.nimml.org">http://www.nimml.org</ext-link>
) for their suggestions and comments. This work has been partially supported by Defense Threat Reduction Agency (DTRA)—R&D (HPC) Award No. HDTRA1-09-1-0017, DTRA—Validation Award No. HDTRA1-11-1-0016, DTRA—Comprehensive National Incident Management System (CNIMS) Award No. HDRTA1-11-D-0016-0001, National Science Foundation (NSF) PetaApps Grant OCI-0904844, NSF Network Science and Engineering (NetSE) Grant CNS-1011769, NSF Software Development for Cyberinfrastructure (SDCI) Grant OCI-1032677, National Institutes of Health (NIH) MIDAS project 2U01GM070694-7 and National Institute of Allergy and Infectious Diseases (NIAID) & NIH project HHSN272201000056C.</p>
</ack>
<ref-list>
<title>References</title>
<ref id="pone.0136139.ref001">
<label>1</label>
<mixed-citation publication-type="book">
<name>
<surname>Saltelli</surname>
<given-names>A</given-names>
</name>
,
<name>
<surname>Annoni</surname>
<given-names>P</given-names>
</name>
,
<name>
<surname>Azzini</surname>
<given-names>I</given-names>
</name>
,
<name>
<surname>Campolongo</surname>
<given-names>F</given-names>
</name>
,
<name>
<surname>Ratto</surname>
<given-names>M</given-names>
</name>
,
<name>
<surname>Tarantola</surname>
<given-names>S</given-names>
</name>
.
<chapter-title>Variance based sensitivity analysis of model output</chapter-title>
<source>Design and estimator for the total sensitivity index</source>
.
<publisher-name>Computer Physics Communications</publisher-name>
<year>2010</year>
;.</mixed-citation>
</ref>
<ref id="pone.0136139.ref002">
<label>2</label>
<mixed-citation publication-type="book">
<name>
<surname>Helton</surname>
<given-names>JC</given-names>
</name>
.
<chapter-title>Uncertainty and Sensitivity Analysis for Models of Complex Systems</chapter-title>
In:
<source>Computational Methods in Transport: Verification and Validation</source>
.
<publisher-name>Springer Berlin Heidelberg</publisher-name>
;
<year>2008</year>
.</mixed-citation>
</ref>
<ref id="pone.0136139.ref003">
<label>3</label>
<mixed-citation publication-type="book">
<name>
<surname>Cacuci</surname>
<given-names>DG</given-names>
</name>
,
<name>
<surname>Ionescu-Bujor</surname>
<given-names>M</given-names>
</name>
,
<name>
<surname>Navon</surname>
<given-names>IM</given-names>
</name>
.
<source>Sensitivity And Uncertainty Analysis: Applications to Large-Scale Systems</source>
.
<publisher-name>Chapman & Hall/CRC</publisher-name>
;
<year>2005</year>
.</mixed-citation>
</ref>
<ref id="pone.0136139.ref004">
<label>4</label>
<mixed-citation publication-type="journal">
<name>
<surname>Brooks</surname>
<given-names>RJ</given-names>
</name>
,
<name>
<surname>Semenov</surname>
<given-names>MA</given-names>
</name>
,
<name>
<surname>Jamieson</surname>
<given-names>PD</given-names>
</name>
.
<article-title>Simplifying Sirius: sensitivity analysis and development of a meta-model for wheat yield prediction</article-title>
.
<source>European Journal of Agronomy</source>
.
<year>2001</year>
;.</mixed-citation>
</ref>
<ref id="pone.0136139.ref005">
<label>5</label>
<mixed-citation publication-type="other">Do H, Rothermel G. Using Sensitivity Analysis to Create Simplified Economic Models for Regression Testing. In: Proceedings of the 2008 International Symposium on Software Testing and Analysis; 2008.</mixed-citation>
</ref>
<ref id="pone.0136139.ref006">
<label>6</label>
<mixed-citation publication-type="journal">
<name>
<surname>Sher</surname>
<given-names>AA</given-names>
</name>
,
<name>
<surname>Wang</surname>
<given-names>K</given-names>
</name>
,
<name>
<surname>Wathen</surname>
<given-names>A</given-names>
</name>
,
<name>
<surname>Maybank</surname>
<given-names>PJ</given-names>
</name>
,
<name>
<surname>Mirams</surname>
<given-names>GR</given-names>
</name>
,
<name>
<surname>Abramson</surname>
<given-names>D</given-names>
</name>
,
<etal>et al</etal>
<article-title>A local sensitivity analysis method for developing biological models with identifiable parameters: Application to cardiac ionic channel modelling</article-title>
.
<source>Future Generation Computer Systems</source>
.
<year>2013</year>
;.
<comment>doi:
<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1016/j.future.2011.09.006">10.1016/j.future.2011.09.006</ext-link>
</comment>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref007">
<label>7</label>
<mixed-citation publication-type="journal">
<name>
<surname>Frey</surname>
<given-names>HC</given-names>
</name>
,
<name>
<surname>Patil</surname>
<given-names>SR</given-names>
</name>
.
<article-title>Identification and Review of Sensitivity Analysis Methods</article-title>
.
<source>Risk Analysis</source>
.
<year>2002</year>
;.</mixed-citation>
</ref>
<ref id="pone.0136139.ref008">
<label>8</label>
<mixed-citation publication-type="journal">
<name>
<surname>Hamby</surname>
<given-names>DM</given-names>
</name>
.
<article-title>A comparison of sensitivity analysis techniques</article-title>
.
<source>Health Physics</source>
.
<year>1995</year>
;.
<pub-id pub-id-type="pmid">7814253</pub-id>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref009">
<label>9</label>
<mixed-citation publication-type="book">
<name>
<surname>Saltelli</surname>
<given-names>A</given-names>
</name>
,
<name>
<surname>Tarantola</surname>
<given-names>S</given-names>
</name>
,
<name>
<surname>Campolongo</surname>
<given-names>F</given-names>
</name>
,
<name>
<surname>Ratto</surname>
<given-names>M</given-names>
</name>
.
<source>Sensitivity Analysis in Practice: A Guide to Assessing Scientific Models</source>
.
<publisher-name>Halsted Press</publisher-name>
;
<year>2004</year>
.</mixed-citation>
</ref>
<ref id="pone.0136139.ref010">
<label>10</label>
<mixed-citation publication-type="journal">
<name>
<surname>Arwade</surname>
<given-names>SR</given-names>
</name>
,
<name>
<surname>Moradi</surname>
<given-names>M</given-names>
</name>
,
<name>
<surname>Louhghalam</surname>
<given-names>A</given-names>
</name>
.
<article-title>Variance decomposition and global sensitivity for structural systems</article-title>
.
<source>Engineering Structures</source>
.
<year>2010</year>
;.
<comment>doi:
<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1016/j.engstruct.2009.08.011">10.1016/j.engstruct.2009.08.011</ext-link>
</comment>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref011">
<label>11</label>
<mixed-citation publication-type="journal">
<name>
<surname>Bauer</surname>
<given-names>KW</given-names>
</name>
,
<name>
<surname>Parnell</surname>
<given-names>GS</given-names>
</name>
,
<name>
<surname>Meyers</surname>
<given-names>DA</given-names>
</name>
.
<article-title>Response surface methodology as a sensitivity analysis tool in decision analysis</article-title>
.
<source>Journal of Multi-Criteria Decision Analysis</source>
.
<year>1999</year>
;.
<comment>doi:
<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1002/(SICI)1099-1360(199905)8:3%3C162::AID-MCDA241%3E3.3.CO;2-O">10.1002/(SICI)1099-1360(199905)8:3%3C162::AID-MCDA241%3E3.3.CO;2-O</ext-link>
</comment>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref012">
<label>12</label>
<mixed-citation publication-type="journal">
<name>
<surname>Lim</surname>
<given-names>JT</given-names>
</name>
,
<name>
<surname>Gold</surname>
<given-names>HJ</given-names>
</name>
,
<name>
<surname>Wilkerson</surname>
<given-names>GG</given-names>
</name>
,
<name>
<surname>Raper</surname>
<given-names>JCD</given-names>
</name>
.
<article-title>A Monte Carlo/response surface strategy for sensitivity analysis: application to a dynamic model of vegetative plant growth</article-title>
.
<source>Appl Math Model</source>
.
<year>1989</year>
;.
<comment>doi:
<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1016/0307-904X(89)90096-6">10.1016/0307-904X(89)90096-6</ext-link>
</comment>
<pub-id pub-id-type="pmid">11538114</pub-id>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref013">
<label>13</label>
<mixed-citation publication-type="journal">
<name>
<surname>Helton</surname>
<given-names>JC</given-names>
</name>
,
<name>
<surname>Johnson</surname>
<given-names>JD</given-names>
</name>
,
<name>
<surname>Sallaberry</surname>
<given-names>CJ</given-names>
</name>
,
<name>
<surname>Storlie</surname>
<given-names>CB</given-names>
</name>
.
<article-title>Survey of sampling-based methods for uncertainty and sensitivity analysis</article-title>
.
<source>Reliability Engineering & System Safety</source>
.
<year>2006</year>
;.
<comment>doi:
<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1016/j.ress.2005.11.017">10.1016/j.ress.2005.11.017</ext-link>
</comment>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref014">
<label>14</label>
<mixed-citation publication-type="book">
<name>
<surname>Bonabeau</surname>
<given-names>E</given-names>
</name>
,
<name>
<surname>Dorigo</surname>
<given-names>M</given-names>
</name>
,
<name>
<surname>Theraulaz</surname>
<given-names>G</given-names>
</name>
.
<source>Swarm Intelligence: From Natural to Artificial Systems</source>
.
<publisher-name>Oxford University Press, Inc</publisher-name>
;
<year>1999</year>
.</mixed-citation>
</ref>
<ref id="pone.0136139.ref015">
<label>15</label>
<mixed-citation publication-type="journal">
<name>
<surname>Dong</surname>
<given-names>X</given-names>
</name>
,
<name>
<surname>Foteinou</surname>
<given-names>PT</given-names>
</name>
,
<name>
<surname>Calvano</surname>
<given-names>SE</given-names>
</name>
,
<name>
<surname>Lowry</surname>
<given-names>SF</given-names>
</name>
,
<name>
<surname>Androulakis</surname>
<given-names>IP</given-names>
</name>
.
<article-title>Agent-Based Modeling of Endotoxin-Induced Acute Inflammatory Response in Human Blood Leukocytes</article-title>
.
<source>PLoS ONE</source>
.
<year>2010</year>
;.
<comment>doi:
<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1371/journal.pone.0009249">10.1371/journal.pone.0009249</ext-link>
</comment>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref016">
<label>16</label>
<mixed-citation publication-type="journal">
<name>
<surname>Forrest</surname>
<given-names>S</given-names>
</name>
,
<name>
<surname>Beauchemin</surname>
<given-names>C</given-names>
</name>
.
<article-title>Computer immunology</article-title>
.
<source>Immunol Rev</source>
.
<year>2007</year>
;.
<pub-id pub-id-type="pmid">17367343</pub-id>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref017">
<label>17</label>
<mixed-citation publication-type="journal">
<name>
<surname>Neugebauer</surname>
<given-names>EA</given-names>
</name>
,
<name>
<surname>Willy</surname>
<given-names>C</given-names>
</name>
,
<name>
<surname>Sauerland</surname>
<given-names>S</given-names>
</name>
.
<article-title>Complexity and non-linearity in shock research: reductionism or synthesis?</article-title>
<source>Shock</source>
.
<year>2001</year>
;.</mixed-citation>
</ref>
<ref id="pone.0136139.ref018">
<label>18</label>
<mixed-citation publication-type="journal">
<name>
<surname>An</surname>
<given-names>G</given-names>
</name>
.
<article-title>Concepts for developing a collaborative in silico model of the acute inflammatory response using agent-based modeling</article-title>
.
<source>J Crit Care</source>
.
<year>2006</year>
;.
<comment>doi:
<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1016/j.jcrc.2005.11.012">10.1016/j.jcrc.2005.11.012</ext-link>
</comment>
<pub-id pub-id-type="pmid">16616634</pub-id>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref019">
<label>19</label>
<mixed-citation publication-type="journal">
<name>
<surname>Lowry</surname>
<given-names>SF</given-names>
</name>
,
<name>
<surname>Calvano</surname>
<given-names>SE</given-names>
</name>
.
<article-title>Challenges for modeling and interpreting the complex biology of severe injury and inflammation</article-title>
.
<source>J Leukoc Biol</source>
.
<year>2008</year>
;.
<pub-id pub-id-type="pmid">17984288</pub-id>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref020">
<label>20</label>
<mixed-citation publication-type="journal">
<name>
<surname>Seely</surname>
<given-names>AJ</given-names>
</name>
,
<name>
<surname>Christou</surname>
<given-names>NV</given-names>
</name>
.
<article-title>Multiple organ dysfunction syndrome: exploring the paradigm of complex nonlinear systems</article-title>
.
<source>Crit Care Med</source>
.
<year>2000</year>
;.
<pub-id pub-id-type="pmid">10921540</pub-id>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref021">
<label>21</label>
<mixed-citation publication-type="journal">
<name>
<surname>An</surname>
<given-names>G</given-names>
</name>
,
<name>
<surname>Faeder</surname>
<given-names>J</given-names>
</name>
,
<name>
<surname>Vodovotz</surname>
<given-names>Y</given-names>
</name>
.
<article-title>Translational systems biology: introduction of an engineering approach to the pathophysiology of the burn patient</article-title>
.
<source>J Burn Care Res</source>
.
<year>2008</year>
;. </mixed-citation>
</ref>
<ref id="pone.0136139.ref022">
<label>22</label>
<mixed-citation publication-type="journal">
<name>
<surname>Vodovotz</surname>
<given-names>Y</given-names>
</name>
,
<name>
<surname>Chow</surname>
<given-names>CC</given-names>
</name>
,
<name>
<surname>Bartels</surname>
<given-names>J</given-names>
</name>
,
<name>
<surname>Lagoa</surname>
<given-names>C</given-names>
</name>
,
<name>
<surname>Prince</surname>
<given-names>JM</given-names>
</name>
,
<name>
<surname>Levy</surname>
<given-names>RM</given-names>
</name>
,
<etal>et al</etal>
<article-title>In silico models of acute inflammation in animals</article-title>
.
<source>Shock</source>
.
<year>2006</year>
;. </mixed-citation>
</ref>
<ref id="pone.0136139.ref023">
<label>23</label>
<mixed-citation publication-type="journal">
<name>
<surname>Baldazzi</surname>
<given-names>V</given-names>
</name>
,
<name>
<surname>Castiglione</surname>
<given-names>F</given-names>
</name>
,
<name>
<surname>Bernaschi</surname>
<given-names>M</given-names>
</name>
.
<article-title>An enhanced agent based model of the immune system response</article-title>
.
<source>Cellular Immunology</source>
.
<year>2006</year>
;.
<comment>doi:
<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1016/j.cellimm.2006.12.006">10.1016/j.cellimm.2006.12.006</ext-link>
</comment>
<pub-id pub-id-type="pmid">17416357</pub-id>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref024">
<label>24</label>
<mixed-citation publication-type="journal">
<name>
<surname>Mi</surname>
<given-names>Q</given-names>
</name>
,
<name>
<surname>Rivière</surname>
<given-names>B</given-names>
</name>
,
<name>
<surname>Clermont</surname>
<given-names>G</given-names>
</name>
,
<name>
<surname>Steed</surname>
<given-names>DL</given-names>
</name>
,
<name>
<surname>Vodovotz</surname>
<given-names>Y</given-names>
</name>
.
<article-title>Agent-based model of inflammation and wound healing: insights into diabetic foot ulcer pathology and the role of transforming growth factor-
<italic>β</italic>
1</article-title>
.
<source>Wound Repair and Regeneration</source>
.
<year>2007</year>
;.
<comment>doi:
<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1111/j.1524-475X.2007.00271.x">10.1111/j.1524-475X.2007.00271.x</ext-link>
</comment>
<pub-id pub-id-type="pmid">17971013</pub-id>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref025">
<label>25</label>
<mixed-citation publication-type="journal">
<name>
<surname>An</surname>
<given-names>G</given-names>
</name>
.
<article-title>Agent-based computer simulation and sirs: building a bridge between basic science and clinical trials</article-title>
.
<source>Shock</source>
.
<year>2001</year>
;.
<pub-id pub-id-type="pmid">11580108</pub-id>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref026">
<label>26</label>
<mixed-citation publication-type="journal">
<name>
<surname>An</surname>
<given-names>G</given-names>
</name>
.
<article-title>Introduction of an agent-based multi-scale modular architecture for dynamic knowledge representation of acute inflammation</article-title>
.
<source>Theoretical Biology & Medical Modelling</source>
.
<year>2008</year>
;.
<comment>doi:
<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1186/1742-4682-5-11">10.1186/1742-4682-5-11</ext-link>
</comment>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref027">
<label>27</label>
<mixed-citation publication-type="journal">
<name>
<surname>Shapiro</surname>
<given-names>M</given-names>
</name>
,
<name>
<surname>Duca</surname>
<given-names>KA</given-names>
</name>
,
<name>
<surname>Lee</surname>
<given-names>K</given-names>
</name>
,
<name>
<surname>Delgado-Eckert</surname>
<given-names>E</given-names>
</name>
,
<name>
<surname>Hawkins</surname>
<given-names>J</given-names>
</name>
,
<name>
<surname>Jarrah</surname>
<given-names>AS</given-names>
</name>
,
<etal>et al</etal>
<article-title>A virtual look at Epstein-Barr virus infection: simulation mechanism</article-title>
.
<source>J Theor Biol</source>
.
<year>2008</year>
;.
<comment>doi:
<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1016/j.jtbi.2008.01.032">10.1016/j.jtbi.2008.01.032</ext-link>
</comment>
<pub-id pub-id-type="pmid">18371986</pub-id>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref028">
<label>28</label>
<mixed-citation publication-type="journal">
<name>
<surname>Duca</surname>
<given-names>KA</given-names>
</name>
,
<name>
<surname>Shapiro</surname>
<given-names>M</given-names>
</name>
,
<name>
<surname>Delgado-Eckert</surname>
<given-names>E</given-names>
</name>
,
<name>
<surname>Hadinoto</surname>
<given-names>V</given-names>
</name>
,
<name>
<surname>Jarrah</surname>
<given-names>AS</given-names>
</name>
,
<name>
<surname>Laubenbacher</surname>
<given-names>R</given-names>
</name>
,
<etal>et al</etal>
<article-title>A virtual look at Epstein-Barr virus infection: biological interpretations</article-title>
.
<source>PLoS Pathog</source>
.
<year>2007</year>
;.
<comment>doi:
<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1371/journal.ppat.0030137">10.1371/journal.ppat.0030137</ext-link>
</comment>
<pub-id pub-id-type="pmid">17953479</pub-id>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref029">
<label>29</label>
<mixed-citation publication-type="journal">
<name>
<surname>Bauer</surname>
<given-names>AL</given-names>
</name>
,
<name>
<surname>Beauchemin</surname>
<given-names>CAA</given-names>
</name>
,
<name>
<surname>Perelson</surname>
<given-names>AS</given-names>
</name>
.
<article-title>Agent-based modeling of host-pathogen systems: The successes and challenges</article-title>
.
<source>Information Sciences</source>
.
<year>2009</year>
;.
<comment>doi:
<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1016/j.ins.2008.11.012">10.1016/j.ins.2008.11.012</ext-link>
</comment>
<pub-id pub-id-type="pmid">20161146</pub-id>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref030">
<label>30</label>
<mixed-citation publication-type="other">Barrett, CL, Bisset, KR, Eubank, SG, Feng, X, Marathe, MV. EpiSimdemics: an Efficient Algorithm for Simulating the Spread of Infectious Disease over Large Realistic Social Networks. In: SC’08: ACM/IEEE Conference on Supercomputing; 2008.</mixed-citation>
</ref>
<ref id="pone.0136139.ref031">
<label>31</label>
<mixed-citation publication-type="book">
<name>
<surname>Barrett</surname>
<given-names>CL</given-names>
</name>
,
<name>
<surname>Bisset</surname>
<given-names>K</given-names>
</name>
,
<name>
<surname>Chen</surname>
<given-names>J</given-names>
</name>
,
<name>
<surname>Eubank</surname>
<given-names>S</given-names>
</name>
,
<name>
<surname>Lewis</surname>
<given-names>B</given-names>
</name>
,
<name>
<surname>Kumar</surname>
<given-names>VSA</given-names>
</name>
,
<etal>et al</etal>
<chapter-title>Interactions among human behavior, social networks, and societal infrastructures: A Case Study in Computational Epidemiology</chapter-title>
In:
<source>Fundamental Problems in Computing</source>
.
<publisher-name>Springer Netherlands</publisher-name>
;
<year>2009</year>
.</mixed-citation>
</ref>
<ref id="pone.0136139.ref032">
<label>32</label>
<mixed-citation publication-type="journal">
<name>
<surname>Eubank</surname>
<given-names>S</given-names>
</name>
,
<name>
<surname>Guclu</surname>
<given-names>H</given-names>
</name>
,
<name>
<surname>Kumar</surname>
<given-names>VSA</given-names>
</name>
,
<name>
<surname>Marathe</surname>
<given-names>M</given-names>
</name>
,
<name>
<surname>Srinivasan</surname>
<given-names>A</given-names>
</name>
,
<name>
<surname>Toroczkai</surname>
<given-names>Z</given-names>
</name>
,
<etal>et al</etal>
<article-title>Modelling disease outbreaks in realistic urban social networks</article-title>
.
<source>Nature</source>
.
<year>2004</year>
;.
<comment>doi:
<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1038/nature02541">10.1038/nature02541</ext-link>
</comment>
<pub-id pub-id-type="pmid">15141212</pub-id>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref033">
<label>33</label>
<mixed-citation publication-type="journal">
<name>
<surname>Barrett</surname>
<given-names>C</given-names>
</name>
,
<name>
<surname>Beckman</surname>
<given-names>R</given-names>
</name>
,
<name>
<surname>Berkbigler</surname>
<given-names>K</given-names>
</name>
,
<name>
<surname>Bisset</surname>
<given-names>K</given-names>
</name>
,
<name>
<surname>Bush</surname>
<given-names>B</given-names>
</name>
,
<name>
<surname>Campbell</surname>
<given-names>K</given-names>
</name>
,
<etal>et al</etal>
<article-title>TRANSIMS: Transportation Analysis Simulation System</article-title>
.
<source>LANL</source>
;
<year>2001</year>
.</mixed-citation>
</ref>
<ref id="pone.0136139.ref034">
<label>34</label>
<mixed-citation publication-type="journal">
<name>
<surname>Barrett</surname>
<given-names>E</given-names>
</name>
,
<name>
<surname>Ross</surname>
<given-names>RP</given-names>
</name>
,
<name>
<surname>Fitzgerald</surname>
<given-names>GF</given-names>
</name>
,
<name>
<surname>Stanton</surname>
<given-names>C</given-names>
</name>
.
<article-title>Rapid screening method for analyzing the conjugated linoleic acid production capabilities of bacterial cultures</article-title>
.
<source>Appl Environ Microbiol</source>
.
<year>2007</year>
;.
<comment>doi:
<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1128/AEM.01855-06">10.1128/AEM.01855-06</ext-link>
</comment>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref035">
<label>35</label>
<mixed-citation publication-type="journal">
<name>
<surname>Karaoz</surname>
<given-names>U</given-names>
</name>
,
<name>
<surname>Murali</surname>
<given-names>TM</given-names>
</name>
,
<name>
<surname>Letovsky</surname>
<given-names>S</given-names>
</name>
,
<name>
<surname>Zheng</surname>
<given-names>Y</given-names>
</name>
,
<name>
<surname>Ding</surname>
<given-names>C</given-names>
</name>
,
<name>
<surname>Cantor</surname>
<given-names>CR</given-names>
</name>
,
<etal>et al</etal>
<article-title>Whole-genome annotation by using evidence integration in functional-linkage networks</article-title>
.
<source>Proceedings of the National Academy of Sciences of the United States of America</source>
.
<year>2004</year>
;.
<comment>doi:
<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1073/pnas.0307326101">10.1073/pnas.0307326101</ext-link>
</comment>
<pub-id pub-id-type="pmid">14981259</pub-id>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref036">
<label>36</label>
<mixed-citation publication-type="journal">
<name>
<surname>Wendelsdorf</surname>
<given-names>KV</given-names>
</name>
,
<name>
<surname>Bassaganya-Riera</surname>
<given-names>J</given-names>
</name>
,
<name>
<surname>Bisset</surname>
<given-names>K</given-names>
</name>
,
<name>
<surname>Eubank</surname>
<given-names>S</given-names>
</name>
,
<name>
<surname>Hontecillas</surname>
<given-names>R</given-names>
</name>
,
<name>
<surname>Marathe</surname>
<given-names>M</given-names>
</name>
.
<article-title>ENteric Immunity Simulator Version 0.9: A tool for in silico study of gut immunopathologies</article-title>
.
<source>NDSSL</source>
;
<year>2011</year>
.</mixed-citation>
</ref>
<ref id="pone.0136139.ref037">
<label>37</label>
<mixed-citation publication-type="other">Wendelsdorf KV, Bassaganya-Riera J, Bisset K, Eubank S, Hontecillas R, Marathe M. ENteric Immunity Simulator: A tool for in silico study of gut immunopathologies. Proceedings of the IEEE International Conference Bioinformatics and Biomedicine. 2011;.</mixed-citation>
</ref>
<ref id="pone.0136139.ref038">
<label>38</label>
<mixed-citation publication-type="other">Bisset K, Alam M, Bassaganya-Riera J, Carbo A, Eubank S, Hontecillas R, et al. High-Performance Interaction-Based Simulation of Gut Immunopathologies with ENteric Immunity SImulator (ENISI). In: Parallel Distributed Processing Symposium (IPDPS), 2012 IEEE 26
<sup>th</sup>
International; 2012.</mixed-citation>
</ref>
<ref id="pone.0136139.ref039">
<label>39</label>
<mixed-citation publication-type="journal">
<name>
<surname>Wendelsdorf</surname>
<given-names>KV</given-names>
</name>
,
<name>
<surname>Alam</surname>
<given-names>M</given-names>
</name>
,
<name>
<surname>Bassaganya-Riera</surname>
<given-names>J</given-names>
</name>
,
<name>
<surname>Bisset</surname>
<given-names>K</given-names>
</name>
,
<name>
<surname>Eubank</surname>
<given-names>S</given-names>
</name>
,
<name>
<surname>Hontecillas</surname>
<given-names>R</given-names>
</name>
,
<etal>et al</etal>
<article-title>ENteric Immunity SImulator: a tool for in silico study of gastroenteric infections</article-title>
.
<source>IEEE Trans Nanobioscience</source>
.
<year>2012</year>
;.
<comment>doi:
<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1109/TNB.2012.2211891">10.1109/TNB.2012.2211891</ext-link>
</comment>
<pub-id pub-id-type="pmid">22987134</pub-id>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref040">
<label>40</label>
<mixed-citation publication-type="other">Mei Y, Hontecillas R, Zhang X, Bisset K, Eubank S, Hoops S, et al. ENISI Visual, an agent-based simulator for modeling gut immunity. In: Bioinformatics and Biomedicine (BIBM), 2012 IEEE International Conference on; 2012.</mixed-citation>
</ref>
<ref id="pone.0136139.ref041">
<label>41</label>
<mixed-citation publication-type="other">Mei Y, Carbo A, Hontecillas R, Hoops S, Liles N, Lu P, et al. ENISI MSM: A novel multi-scale modeling platform for computational immunology. In: Bioinformatics and Biomedicine (BIBM), 2014 IEEE International Conference on; 2014.</mixed-citation>
</ref>
<ref id="pone.0136139.ref042">
<label>42</label>
<mixed-citation publication-type="journal">
<name>
<surname>Mei</surname>
<given-names>Y</given-names>
</name>
,
<name>
<surname>Abedi</surname>
<given-names>V</given-names>
</name>
,
<name>
<surname>Carbo</surname>
<given-names>A</given-names>
</name>
,
<name>
<surname>Zhang</surname>
<given-names>X</given-names>
</name>
,
<name>
<surname>Lu</surname>
<given-names>P</given-names>
</name>
,
<name>
<surname>Philipson</surname>
<given-names>C</given-names>
</name>
,
<etal>et al</etal>
<article-title>Multiscale modeling of mucosal immune responses</article-title>
.
<source>BMC Bioinformatics</source>
.
<year>2015</year>
;.</mixed-citation>
</ref>
<ref id="pone.0136139.ref043">
<label>43</label>
<mixed-citation publication-type="journal">
<name>
<surname>Kelly</surname>
<given-names>D</given-names>
</name>
,
<name>
<surname>Campbell</surname>
<given-names>JI</given-names>
</name>
,
<name>
<surname>King</surname>
<given-names>TP</given-names>
</name>
,
<name>
<surname>Grant</surname>
<given-names>G</given-names>
</name>
,
<name>
<surname>Jansson</surname>
<given-names>EA</given-names>
</name>
,
<name>
<surname>Coutts</surname>
<given-names>AG</given-names>
</name>
,
<etal>et al</etal>
<article-title>Commensal anaerobic gut bacteria attenuate inflammation by regulating nuclear-cytoplasmic shuttling of PPAR-gamma and RelA</article-title>
.
<source>Nat Immunol</source>
.
<year>2004</year>
;.
<comment>doi:
<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1038/ni1018">10.1038/ni1018</ext-link>
</comment>
<pub-id pub-id-type="pmid">14691478</pub-id>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref044">
<label>44</label>
<mixed-citation publication-type="journal">
<name>
<surname>Marzo</surname>
<given-names>AL</given-names>
</name>
,
<name>
<surname>Klonowski</surname>
<given-names>KD</given-names>
</name>
,
<name>
<surname>Bon</surname>
<given-names>AL</given-names>
</name>
,
<name>
<surname>Borrow</surname>
<given-names>P</given-names>
</name>
,
<name>
<surname>Tough</surname>
<given-names>DF</given-names>
</name>
,
<name>
<surname>Lefrancois</surname>
<given-names>L</given-names>
</name>
.
<article-title>Initial T cell frequency dictates memory CD8+ T cell lineage commitment</article-title>
.
<source>Nat Immunol</source>
.
<year>2005</year>
;.
<comment>doi:
<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1038/ni1227">10.1038/ni1227</ext-link>
</comment>
<pub-id pub-id-type="pmid">16025119</pub-id>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref045">
<label>45</label>
<mixed-citation publication-type="journal">
<name>
<surname>Lanzavecchia</surname>
<given-names>A</given-names>
</name>
,
<name>
<surname>Sallusto</surname>
<given-names>F</given-names>
</name>
.
<article-title>Regulation of T cell immunity by dendritic cells</article-title>
.
<source>Cell</source>
.
<year>2001</year>
;.
<comment>doi:
<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1016/S0092-8674(01)00455-X">10.1016/S0092-8674(01)00455-X</ext-link>
</comment>
<pub-id pub-id-type="pmid">11509174</pub-id>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref046">
<label>46</label>
<mixed-citation publication-type="journal">
<name>
<surname>Sallusto</surname>
<given-names>F</given-names>
</name>
,
<name>
<surname>Geginat</surname>
<given-names>J</given-names>
</name>
,
<name>
<surname>Lanzavecchia</surname>
<given-names>A</given-names>
</name>
.
<article-title>Central memory and effector memory T cell subsets: function, generation, and maintenance</article-title>
.
<source>Annu Rev Immunol</source>
.
<year>2004</year>
;.
<comment>doi:
<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1146/annurev.immunol.22.012703.104702">10.1146/annurev.immunol.22.012703.104702</ext-link>
</comment>
<pub-id pub-id-type="pmid">15032595</pub-id>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref047">
<label>47</label>
<mixed-citation publication-type="journal">
<name>
<surname>Kaech</surname>
<given-names>SM</given-names>
</name>
,
<name>
<surname>Ahmed</surname>
<given-names>R</given-names>
</name>
.
<article-title>Immunology: CD8 T cells remember with a little help</article-title>
.
<source>Science</source>
.
<year>2003</year>
;.
<comment>doi:
<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1126/science.1084511">10.1126/science.1084511</ext-link>
</comment>
<pub-id pub-id-type="pmid">12690179</pub-id>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref048">
<label>48</label>
<mixed-citation publication-type="journal">
<name>
<surname>Naresh</surname>
<given-names>R</given-names>
</name>
,
<name>
<surname>Song</surname>
<given-names>Y</given-names>
</name>
,
<name>
<surname>Hampson</surname>
<given-names>DJ</given-names>
</name>
.
<article-title>The intestinal spirochete Brachyspira pilosicoli attaches to cultured Caco-2 cells and induces pathological changes</article-title>
.
<source>PLoS One</source>
.
<year>2009</year>
;.
<comment>doi:
<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1371/journal.pone.0008352">10.1371/journal.pone.0008352</ext-link>
</comment>
<pub-id pub-id-type="pmid">20020053</pub-id>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref049">
<label>49</label>
<mixed-citation publication-type="journal">
<name>
<surname>Carbo</surname>
<given-names>A</given-names>
</name>
,
<name>
<surname>Hontecillas</surname>
<given-names>R</given-names>
</name>
,
<name>
<surname>Kronsteiner</surname>
<given-names>B</given-names>
</name>
,
<name>
<surname>Viladomiu</surname>
<given-names>M</given-names>
</name>
,
<name>
<surname>Pedragosa</surname>
<given-names>M</given-names>
</name>
,
<name>
<surname>Lu</surname>
<given-names>P</given-names>
</name>
,
<etal>et al</etal>
<article-title>Systems modeling of molecular mechanisms controlling cytokine-driven CD4+ T cell differentiation and phenotype plasticity</article-title>
.
<source>PLoS Comput Biol</source>
.
<year>2013</year>
;.
<comment>doi:
<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1371/journal.pcbi.1003027">10.1371/journal.pcbi.1003027</ext-link>
</comment>
<pub-id pub-id-type="pmid">23592971</pub-id>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref050">
<label>50</label>
<mixed-citation publication-type="journal">
<name>
<surname>Carbo</surname>
<given-names>A</given-names>
</name>
,
<name>
<surname>Hontecillas</surname>
<given-names>R</given-names>
</name>
,
<name>
<surname>Andrew</surname>
<given-names>T</given-names>
</name>
,
<name>
<surname>Eden</surname>
<given-names>K</given-names>
</name>
,
<name>
<surname>Mei</surname>
<given-names>Y</given-names>
</name>
,
<name>
<surname>Hoops</surname>
<given-names>S</given-names>
</name>
,
<etal>et al</etal>
<article-title>Computational modeling of heterogeneity and function of CD4+ T cells</article-title>
.
<source>Front Cell Dev Biol</source>
.
<year>2014</year>
;.
<comment>doi:
<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.3389/fcell.2014.00031">10.3389/fcell.2014.00031</ext-link>
</comment>
<pub-id pub-id-type="pmid">25364738</pub-id>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref051">
<label>51</label>
<mixed-citation publication-type="journal">
<name>
<surname>Carbo</surname>
<given-names>A</given-names>
</name>
,
<name>
<surname>Olivares-Villagomez</surname>
<given-names>D</given-names>
</name>
,
<name>
<surname>Hontecillas</surname>
<given-names>R</given-names>
</name>
,
<name>
<surname>Bassaganya-Riera</surname>
<given-names>J</given-names>
</name>
,
<name>
<surname>Chaturvedi</surname>
<given-names>R</given-names>
</name>
,
<name>
<surname>Piazuelo</surname>
<given-names>MB</given-names>
</name>
,
<etal>et al</etal>
<article-title>Systems modeling of the role of interleukin-21 in the maintenance of effector CD4+ T cell responses during chronic Helicobacter pylori infection</article-title>
.
<source>MBio</source>
.
<year>2014</year>
;.
<comment>doi:
<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1128/mBio.01243-14">10.1128/mBio.01243-14</ext-link>
</comment>
<pub-id pub-id-type="pmid">25053783</pub-id>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref052">
<label>52</label>
<mixed-citation publication-type="journal">
<name>
<surname>Carbo</surname>
<given-names>A</given-names>
</name>
,
<name>
<surname>Hontecillas</surname>
<given-names>R</given-names>
</name>
,
<name>
<surname>Viladomiu</surname>
<given-names>M</given-names>
</name>
,
<name>
<surname>Leber</surname>
<given-names>A</given-names>
</name>
,
<name>
<surname>Hoops</surname>
<given-names>S</given-names>
</name>
,
<name>
<surname>Bassaganya-Riera</surname>
<given-names>J</given-names>
</name>
.
<article-title>Modeling the Dynamics of T Helper 17 Induction and Differentiation</article-title>
.
<source>MOJ Immunol</source>
.
<year>2015</year>
;.</mixed-citation>
</ref>
<ref id="pone.0136139.ref053">
<label>53</label>
<mixed-citation publication-type="journal">
<name>
<surname>Leber</surname>
<given-names>A</given-names>
</name>
,
<name>
<surname>Viladomiu</surname>
<given-names>M</given-names>
</name>
,
<name>
<surname>Hontecillas</surname>
<given-names>R</given-names>
</name>
,
<name>
<surname>Abedi</surname>
<given-names>V</given-names>
</name>
,
<name>
<surname>Philipson</surname>
<given-names>C</given-names>
</name>
,
<name>
<surname>Hoops</surname>
<given-names>S</given-names>
</name>
,
<etal>et al</etal>
<article-title>Systems modeling of interactions between mucosal immunity and the gut microbiome during Clostridium difficile infection</article-title>
.
<source>PLoS One</source>
.
<year>2015</year>
;.
<comment>doi:
<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1371/journal.pone.0134849">10.1371/journal.pone.0134849</ext-link>
</comment>
<pub-id pub-id-type="pmid">26230099</pub-id>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref054">
<label>54</label>
<mixed-citation publication-type="journal">
<name>
<surname>Sumner</surname>
<given-names>T</given-names>
</name>
,
<name>
<surname>Hetherington</surname>
<given-names>J</given-names>
</name>
,
<name>
<surname>Seymour</surname>
<given-names>RM</given-names>
</name>
,
<name>
<surname>Li</surname>
<given-names>L</given-names>
</name>
,
<name>
<surname>Rey</surname>
<given-names>MV</given-names>
</name>
,
<name>
<surname>Yamaji</surname>
<given-names>S</given-names>
</name>
,
<etal>et al</etal>
<article-title>A composite computational model of liver glucose homeostasis. II. Exploring system behaviour</article-title>
.
<source>J R Soc Interface</source>
.
<year>2012</year>
;.
<comment>doi:
<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1098/rsif.2011.0783">10.1098/rsif.2011.0783</ext-link>
</comment>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref055">
<label>55</label>
<mixed-citation publication-type="journal">
<name>
<surname>Sumner</surname>
<given-names>T</given-names>
</name>
,
<name>
<surname>Shephard</surname>
<given-names>E</given-names>
</name>
,
<name>
<surname>Bogle</surname>
<given-names>ID</given-names>
</name>
.
<article-title>A methodology for global-sensitivity analysis of time-dependent outputs in systems biology modelling</article-title>
.
<source>J R Soc Interface</source>
.
<year>2012</year>
;.
<comment>doi:
<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1098/rsif.2011.0891">10.1098/rsif.2011.0891</ext-link>
</comment>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref056">
<label>56</label>
<mixed-citation publication-type="book">
<name>
<surname>Sumner</surname>
<given-names>T</given-names>
</name>
.
<source>Sensitivity analysis in systems biology modelling and its application to a multi-scale model of blood glucose homeostasis</source>
.
<publisher-name>University College London</publisher-name>
;
<year>2010</year>
.</mixed-citation>
</ref>
<ref id="pone.0136139.ref057">
<label>57</label>
<mixed-citation publication-type="journal">
<name>
<surname>Broeke</surname>
<given-names>GAT</given-names>
</name>
,
<name>
<surname>Voorn</surname>
<given-names>GAKv</given-names>
</name>
,
<name>
<surname>Ligtenberg</surname>
<given-names>A</given-names>
</name>
.
<article-title>Sensitivity analysis for agent-based models: a low complexity test-case</article-title>
.
<source>Laboratory of Geo-information Science and Remote Sensing</source>
;
<year>2014</year>
.</mixed-citation>
</ref>
<ref id="pone.0136139.ref058">
<label>58</label>
<mixed-citation publication-type="journal">
<name>
<surname>Schouten</surname>
<given-names>M</given-names>
</name>
,
<name>
<surname>Verwaart</surname>
<given-names>T</given-names>
</name>
,
<name>
<surname>Heijman</surname>
<given-names>W</given-names>
</name>
.
<article-title>Comparing two sensitivity analysis approaches for two scenarios with a spatially explicit rural agent-based model</article-title>
.
<source>Environmental Modelling & Software</source>
.
<year>2014</year>
;.
<comment>doi:
<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1016/j.envsoft.2014.01.003">10.1016/j.envsoft.2014.01.003</ext-link>
</comment>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref059">
<label>59</label>
<mixed-citation publication-type="journal">
<name>
<surname>Ligmann-Zielinska</surname>
<given-names>A</given-names>
</name>
,
<name>
<surname>Kramer</surname>
<given-names>DB</given-names>
</name>
,
<name>
<surname>Cheruvelil</surname>
<given-names>KS</given-names>
</name>
,
<name>
<surname>Soranno</surname>
<given-names>PA</given-names>
</name>
.
<article-title>Using Uncertainty and Sensitivity Analyses in Socioecological Agent-Based Models to Improve Their Analytical Performance and Policy Relevance</article-title>
.
<source>PLoS ONE</source>
.
<year>2014</year>
;.
<comment>doi:
<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1371/journal.pone.0109779">10.1371/journal.pone.0109779</ext-link>
</comment>
<pub-id pub-id-type="pmid">25340764</pub-id>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref060">
<label>60</label>
<mixed-citation publication-type="journal">
<name>
<surname>Parry</surname>
<given-names>HR</given-names>
</name>
,
<name>
<surname>Topping</surname>
<given-names>CJ</given-names>
</name>
,
<name>
<surname>Kennedy</surname>
<given-names>MC</given-names>
</name>
,
<name>
<surname>Boatman</surname>
<given-names>ND</given-names>
</name>
,
<name>
<surname>Murray</surname>
<given-names>AWA</given-names>
</name>
.
<article-title>A Bayesian sensitivity analysis applied to an Agent-based model of bird population response to landscape change</article-title>
.
<source>Environmental Modelling & Software</source>
.
<year>2013</year>
;.
<comment>doi:
<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1016/j.envsoft.2012.08.006">10.1016/j.envsoft.2012.08.006</ext-link>
</comment>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref061">
<label>61</label>
<mixed-citation publication-type="journal">
<name>
<surname>Fonoberova</surname>
<given-names>M</given-names>
</name>
,
<name>
<surname>Fonoberov</surname>
<given-names>VA</given-names>
</name>
,
<name>
<surname>Mezić</surname>
<given-names>I</given-names>
</name>
.
<article-title>Global sensitivity/uncertainty analysis for agent-based models</article-title>
.
<source>Reliability Engineering & System Safety</source>
.
<year>2013</year>
;.
<comment>doi:
<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1016/j.ress.2013.04.004">10.1016/j.ress.2013.04.004</ext-link>
</comment>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref062">
<label>62</label>
<mixed-citation publication-type="journal">
<name>
<surname>Kandulski</surname>
<given-names>A</given-names>
</name>
,
<name>
<surname>Malfertheiner</surname>
<given-names>P</given-names>
</name>
,
<name>
<surname>Wex</surname>
<given-names>T</given-names>
</name>
.
<article-title>Role of regulatory T-cells in H. pylori-induced gastritis and gastric cancer</article-title>
.
<source>Anticancer Res</source>
.
<year>2010</year>
;.
<pub-id pub-id-type="pmid">20530414</pub-id>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref063">
<label>63</label>
<mixed-citation publication-type="journal">
<name>
<surname>Ramarao</surname>
<given-names>N</given-names>
</name>
,
<name>
<surname>Meyer</surname>
<given-names>TF</given-names>
</name>
.
<article-title>Helicobacter pylori resists phagocytosis by macrophages: Quantitative assessment by confocal microscopy and fluorescence-activated cell sorting</article-title>
.
<source>Infection and Immunity</source>
.
<year>2001</year>
;.
<comment>doi:
<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1128/IAI.69.4.2604-2611.2001">10.1128/IAI.69.4.2604-2611.2001</ext-link>
</comment>
<pub-id pub-id-type="pmid">11254625</pub-id>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref064">
<label>64</label>
<mixed-citation publication-type="journal">
<name>
<surname>Carbo</surname>
<given-names>A</given-names>
</name>
,
<name>
<surname>Bassaganya-Riera</surname>
<given-names>J</given-names>
</name>
,
<name>
<surname>Pedragosa</surname>
<given-names>M</given-names>
</name>
,
<name>
<surname>Viladomiu</surname>
<given-names>M</given-names>
</name>
,
<name>
<surname>Marathe</surname>
<given-names>M</given-names>
</name>
,
<name>
<surname>Eubank</surname>
<given-names>S</given-names>
</name>
,
<etal>et al</etal>
<article-title>Predictive Computational Modeling of the Mucosal Immune Responses during Helicobacter pylori Infection</article-title>
.
<source>PLoS ONE</source>
.
<year>2013</year>
;.
<comment>doi:
<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1371/journal.pone.0073365">10.1371/journal.pone.0073365</ext-link>
</comment>
<pub-id pub-id-type="pmid">24039925</pub-id>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref065">
<label>65</label>
<mixed-citation publication-type="book">
<name>
<surname>Wu</surname>
<given-names>CJ</given-names>
</name>
,
<name>
<surname>Hamada</surname>
<given-names>MS</given-names>
</name>
.
<source>Experiments: planning, analysis, and optimization</source>
.
<publisher-name>John Wiley & Sons</publisher-name>
;
<year>2011</year>
.</mixed-citation>
</ref>
<ref id="pone.0136139.ref066">
<label>66</label>
<mixed-citation publication-type="other">Happe, K. Agent-based modelling and sensitivity analysis by experimental design and metamodelling: an application to modelling regional structural change. In: XIth International Congress of the European Association of Agricultural Economists; 2005.</mixed-citation>
</ref>
<ref id="pone.0136139.ref067">
<label>67</label>
<mixed-citation publication-type="journal">
<name>
<surname>Rodriguez-Fernandez</surname>
<given-names>M</given-names>
</name>
,
<name>
<surname>Banga</surname>
<given-names>JR</given-names>
</name>
,
<name>
<surname>Doyle</surname>
<given-names>FJ</given-names>
</name>
.
<article-title>Novel global sensitivity analysis methodology accounting for the crucial role of the distribution of input parameters: application to systems biology models</article-title>
.
<source>International Journal of Robust and Nonlinear Control</source>
.
<year>2012</year>
;.
<comment>doi:
<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1002/rnc.2797">10.1002/rnc.2797</ext-link>
</comment>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref068">
<label>68</label>
<mixed-citation publication-type="book">
<name>
<surname>Box</surname>
<given-names>GE</given-names>
</name>
,
<name>
<surname>Hunter</surname>
<given-names>JS</given-names>
</name>
,
<name>
<surname>Hunter</surname>
<given-names>WG</given-names>
</name>
.
<source>Statistics for experimenters</source>
.
<publisher-name>Wiley New York</publisher-name>
;
<year>2005</year>
.</mixed-citation>
</ref>
<ref id="pone.0136139.ref069">
<label>69</label>
<mixed-citation publication-type="book">
<name>
<surname>Kleijnen</surname>
<given-names>JPC</given-names>
</name>
,
<name>
<surname>Groenendaal</surname>
<given-names>Wv</given-names>
</name>
.
<source>Simulation: A Statistical Perspective</source>
.
<publisher-name>John Wiley & Sons, Inc</publisher-name>
;
<year>1992</year>
.</mixed-citation>
</ref>
<ref id="pone.0136139.ref070">
<label>70</label>
<mixed-citation publication-type="book">
<name>
<surname>Law</surname>
<given-names>AM</given-names>
</name>
,
<name>
<surname>Kelton</surname>
<given-names>DM</given-names>
</name>
.
<source>Simulation Modeling and Analysis</source>
.
<publisher-name>McGraw-Hill Higher Education</publisher-name>
;
<year>1999</year>
.</mixed-citation>
</ref>
<ref id="pone.0136139.ref071">
<label>71</label>
<mixed-citation publication-type="journal">
<name>
<surname>Noordegraaf</surname>
<given-names>AV</given-names>
</name>
,
<name>
<surname>Nielen</surname>
<given-names>M</given-names>
</name>
,
<name>
<surname>Kleijnen</surname>
<given-names>JPC</given-names>
</name>
.
<article-title>Sensitivity analysis by experimental design and metamodelling: Case study on simulation in national animal disease control</article-title>
.
<source>European Journal of Operational Research</source>
.
<year>2003</year>
;.</mixed-citation>
</ref>
<ref id="pone.0136139.ref072">
<label>72</label>
<mixed-citation publication-type="other">Kleijnen JP, Sanchez SM, Lucas TW, Cioppa TM. A user’s guide to the brave new world of designing simulation experiments; 2003.</mixed-citation>
</ref>
<ref id="pone.0136139.ref073">
<label>73</label>
<mixed-citation publication-type="book">
<name>
<surname>Montgomery</surname>
<given-names>DC</given-names>
</name>
.
<source>Design and analysis of experiments</source>
.
<publisher-name>John Wiley & Sons</publisher-name>
;
<year>2012</year>
.</mixed-citation>
</ref>
<ref id="pone.0136139.ref074">
<label>74</label>
<mixed-citation publication-type="book">
<name>
<surname>Hedayat</surname>
<given-names>AS</given-names>
</name>
,
<name>
<surname>Sloane</surname>
<given-names>NJA</given-names>
</name>
,
<name>
<surname>Stufken</surname>
<given-names>J</given-names>
</name>
.
<source>Orthogonal arrays: theory and applications</source>
.
<publisher-name>Springer</publisher-name>
;
<year>1999</year>
.</mixed-citation>
</ref>
<ref id="pone.0136139.ref075">
<label>75</label>
<mixed-citation publication-type="journal">
<name>
<surname>Morris</surname>
<given-names>MD</given-names>
</name>
,
<name>
<surname>Moore</surname>
<given-names>LM</given-names>
</name>
,
<name>
<surname>McKay</surname>
<given-names>MD</given-names>
</name>
.
<article-title>Using orthogonal arrays in the sensitivity analysis of computer models</article-title>
.
<source>Technometrics</source>
.
<year>2008</year>
;.
<comment>doi:
<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1198/004017008000000208">10.1198/004017008000000208</ext-link>
</comment>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref076">
<label>76</label>
<mixed-citation publication-type="book">
<name>
<surname>Scheffé</surname>
<given-names>H</given-names>
</name>
.
<source>The Analysis of Variance</source>
.
<publisher-name>Wiley</publisher-name>
;
<year>1959</year>
.</mixed-citation>
</ref>
<ref id="pone.0136139.ref077">
<label>77</label>
<mixed-citation publication-type="book">
<name>
<surname>Seber</surname>
<given-names>GAF</given-names>
</name>
,
<name>
<surname>Lee</surname>
<given-names>AJ</given-names>
</name>
.
<source>Linear Regression Analysis</source>
.
<publisher-name>Wiley</publisher-name>
;
<year>2012</year>
.</mixed-citation>
</ref>
<ref id="pone.0136139.ref078">
<label>78</label>
<mixed-citation publication-type="journal">
<name>
<surname>Bassaganya-Riera</surname>
<given-names>J</given-names>
</name>
,
<name>
<surname>Dominguez-Bello</surname>
<given-names>MG</given-names>
</name>
,
<name>
<surname>Kronsteiner</surname>
<given-names>B</given-names>
</name>
,
<name>
<surname>Carbo</surname>
<given-names>A</given-names>
</name>
,
<name>
<surname>Lu</surname>
<given-names>P</given-names>
</name>
,
<name>
<surname>Viladomiu</surname>
<given-names>M</given-names>
</name>
,
<etal>et al</etal>
<article-title>Helicobacter pylori Colonization Ameliorates Glucose Homeostasis in Mice through a PPAR
<italic>γ</italic>
-Dependent Mechanism</article-title>
.
<source>PLoS ONE</source>
.
<year>2012</year>
;.
<comment>doi:
<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1371/journal.pone.0050069">10.1371/journal.pone.0050069</ext-link>
</comment>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref079">
<label>79</label>
<mixed-citation publication-type="journal">
<name>
<surname>Schumacher</surname>
<given-names>MA</given-names>
</name>
,
<name>
<surname>Donnelly</surname>
<given-names>JM</given-names>
</name>
,
<name>
<surname>Engevik</surname>
<given-names>AC</given-names>
</name>
,
<name>
<surname>Xiao</surname>
<given-names>C</given-names>
</name>
,
<name>
<surname>Yang</surname>
<given-names>L</given-names>
</name>
,
<name>
<surname>Kenny</surname>
<given-names>S</given-names>
</name>
,
<etal>et al</etal>
<article-title>Gastric Sonic Hedgehog Acts as a Macrophage Chemoattractant During the Immune Response to
<italic>Helicobacter pylori</italic>
</article-title>
.
<source>Gastroenterology</source>
.
<year>2012</year>
;.
<comment>doi:
<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1053/j.gastro.2012.01.029">10.1053/j.gastro.2012.01.029</ext-link>
</comment>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref080">
<label>80</label>
<mixed-citation publication-type="journal">
<name>
<surname>Kronsteiner</surname>
<given-names>B</given-names>
</name>
,
<name>
<surname>Bassaganya-Riera</surname>
<given-names>J</given-names>
</name>
,
<name>
<surname>Philipson</surname>
<given-names>N</given-names>
</name>
,
<name>
<surname>Hontecillas</surname>
<given-names>R</given-names>
</name>
.
<article-title>Novel insights on the role of CD8+ T cells and cytotoxic responses during Helicobacter pylori infection</article-title>
.
<source>Gut Microbes</source>
.
<year>2014</year>
;.
<comment>doi:
<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.4161/gmic.28899">10.4161/gmic.28899</ext-link>
</comment>
<pub-id pub-id-type="pmid">24755940</pub-id>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref081">
<label>81</label>
<mixed-citation publication-type="journal">
<name>
<surname>Sigal</surname>
<given-names>M</given-names>
</name>
,
<name>
<surname>Rothenberg</surname>
<given-names>ME</given-names>
</name>
,
<name>
<surname>Logan</surname>
<given-names>CY</given-names>
</name>
,
<name>
<surname>Lee</surname>
<given-names>JY</given-names>
</name>
,
<name>
<surname>Honaker</surname>
<given-names>RW</given-names>
</name>
,
<name>
<surname>Cooper</surname>
<given-names>RL</given-names>
</name>
,
<etal>et al</etal>
<article-title>Helicobacter pylori Activates and Expands Lgr5 Stem Cells Through Direct Colonization of the Gastric Glands</article-title>
.
<source>Gastroenterology</source>
.
<year>2015</year>
;.
<comment>doi:
<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1053/j.gastro.2015.02.049">10.1053/j.gastro.2015.02.049</ext-link>
</comment>
<pub-id pub-id-type="pmid">25725293</pub-id>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref082">
<label>82</label>
<mixed-citation publication-type="journal">
<name>
<surname>Sacks</surname>
<given-names>J</given-names>
</name>
,
<name>
<surname>Welch</surname>
<given-names>WJ</given-names>
</name>
,
<name>
<surname>Mitchell</surname>
<given-names>TJ</given-names>
</name>
,
<name>
<surname>Wynn</surname>
<given-names>HP</given-names>
</name>
.
<article-title>Design and analysis of computer experiments</article-title>
.
<source>Statistical science</source>
.
<year>1989</year>
;.
<comment>doi:
<ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1214/ss/1177012420">10.1214/ss/1177012420</ext-link>
</comment>
</mixed-citation>
</ref>
<ref id="pone.0136139.ref083">
<label>83</label>
<mixed-citation publication-type="book">
<name>
<surname>Santner</surname>
<given-names>TJ</given-names>
</name>
,
<name>
<surname>Williams</surname>
<given-names>BJ</given-names>
</name>
,
<name>
<surname>Notz</surname>
<given-names>W</given-names>
</name>
.
<source>The Design and analysis of computer experiments</source>
.
<publisher-name>Springer</publisher-name>
;
<year>2003</year>
.</mixed-citation>
</ref>
</ref-list>
</back>
</pmc>
</record>

Pour manipuler ce document sous Unix (Dilib)

EXPLOR_STEP=$WICRI_ROOT/Ticri/CIDE/explor/CyberinfraV1/Data/Pmc/Corpus
HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 000078 | SxmlIndent | more

Ou

HfdSelect -h $EXPLOR_AREA/Data/Pmc/Corpus/biblio.hfd -nk 000078 | SxmlIndent | more

Pour mettre un lien sur cette page dans le réseau Wicri

{{Explor lien
   |wiki=    Ticri/CIDE
   |area=    CyberinfraV1
   |flux=    Pmc
   |étape=   Corpus
   |type=    RBID
   |clé=     PMC:4556515
   |texte=   Sensitivity Analysis of an ENteric Immunity SImulator (ENISI)-Based Model of Immune Responses to Helicobacter pylori Infection
}}

Pour générer des pages wiki

HfdIndexSelect -h $EXPLOR_AREA/Data/Pmc/Corpus/RBID.i   -Sk "pubmed:26327290" \
       | HfdSelect -Kh $EXPLOR_AREA/Data/Pmc/Corpus/biblio.hfd   \
       | NlmPubMed2Wicri -a CyberinfraV1 

Wicri

This area was generated with Dilib version V0.6.25.
Data generation: Thu Oct 27 09:30:58 2016. Site generation: Sun Mar 10 23:08:40 2024