A Bayesian approach for detecting a disease that is not being modeled
Identifieur interne : 001055 ( Pmc/Corpus ); précédent : 001054; suivant : 001056A Bayesian approach for detecting a disease that is not being modeled
Auteurs : John M. Aronis ; Jeffrey P. Ferraro ; Per H. Gesteland ; Fuchiang Tsui ; Ye Ye ; Michael M. Wagner ; Gregory F. CooperSource :
- PLoS ONE [ 1932-6203 ] ; 2020.
Abstract
Over the past decade, outbreaks of new or reemergent viruses such as severe acute respiratory syndrome (SARS) virus, Middle East respiratory syndrome (MERS) virus, and Zika have claimed thousands of lives and cost governments and healthcare systems billions of dollars. Because the appearance of new or transformed diseases is likely to continue, the detection and characterization of emergent diseases is an important problem. We describe a Bayesian statistical model that can detect and characterize previously unknown and unmodeled diseases from patient-care reports and evaluate its performance on historical data.
Url:
DOI: 10.1371/journal.pone.0229658
PubMed: 32109254
PubMed Central: 7048291
Links to Exploration step
PMC:7048291Le document en format XML
<record><TEI><teiHeader><fileDesc><titleStmt><title xml:lang="en">A Bayesian approach for detecting a disease that is not being modeled</title><author><name sortKey="Aronis, John M" sort="Aronis, John M" uniqKey="Aronis J" first="John M." last="Aronis">John M. Aronis</name><affiliation><nlm:aff id="aff001"><addr-line>Real-time Outbreak and Disease Surveillance (RODS) Laboratory, Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, United States of America</addr-line></nlm:aff></affiliation></author><author><name sortKey="Ferraro, Jeffrey P" sort="Ferraro, Jeffrey P" uniqKey="Ferraro J" first="Jeffrey P." last="Ferraro">Jeffrey P. Ferraro</name><affiliation><nlm:aff id="aff002"><addr-line>Department of Biomedical Informatics, University of Utah, Salt Lake City, Utah, United States of America</addr-line></nlm:aff></affiliation></author><author><name sortKey="Gesteland, Per H" sort="Gesteland, Per H" uniqKey="Gesteland P" first="Per H." last="Gesteland">Per H. Gesteland</name><affiliation><nlm:aff id="aff002"><addr-line>Department of Biomedical Informatics, University of Utah, Salt Lake City, Utah, United States of America</addr-line></nlm:aff></affiliation></author><author><name sortKey="Tsui, Fuchiang" sort="Tsui, Fuchiang" uniqKey="Tsui F" first="Fuchiang" last="Tsui">Fuchiang Tsui</name><affiliation><nlm:aff id="aff001"><addr-line>Real-time Outbreak and Disease Surveillance (RODS) Laboratory, Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, United States of America</addr-line></nlm:aff></affiliation></author><author><name sortKey="Ye, Ye" sort="Ye, Ye" uniqKey="Ye Y" first="Ye" last="Ye">Ye Ye</name><affiliation><nlm:aff id="aff001"><addr-line>Real-time Outbreak and Disease Surveillance (RODS) Laboratory, Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, United States of America</addr-line></nlm:aff></affiliation></author><author><name sortKey="Wagner, Michael M" sort="Wagner, Michael M" uniqKey="Wagner M" first="Michael M." last="Wagner">Michael M. Wagner</name><affiliation><nlm:aff id="aff001"><addr-line>Real-time Outbreak and Disease Surveillance (RODS) Laboratory, Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, United States of America</addr-line></nlm:aff></affiliation></author><author><name sortKey="Cooper, Gregory F" sort="Cooper, Gregory F" uniqKey="Cooper G" first="Gregory F." last="Cooper">Gregory F. Cooper</name><affiliation><nlm:aff id="aff001"><addr-line>Real-time Outbreak and Disease Surveillance (RODS) Laboratory, Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, United States of America</addr-line></nlm:aff></affiliation></author></titleStmt><publicationStmt><idno type="wicri:source">PMC</idno><idno type="pmid">32109254</idno><idno type="pmc">7048291</idno><idno type="url">http://www.ncbi.nlm.nih.gov/pmc/articles/PMC7048291</idno><idno type="RBID">PMC:7048291</idno><idno type="doi">10.1371/journal.pone.0229658</idno><date when="2020">2020</date><idno type="wicri:Area/Pmc/Corpus">001055</idno><idno type="wicri:explorRef" wicri:stream="Pmc" wicri:step="Corpus" wicri:corpus="PMC">001055</idno></publicationStmt><sourceDesc><biblStruct><analytic><title xml:lang="en" level="a" type="main">A Bayesian approach for detecting a disease that is not being modeled</title><author><name sortKey="Aronis, John M" sort="Aronis, John M" uniqKey="Aronis J" first="John M." last="Aronis">John M. Aronis</name><affiliation><nlm:aff id="aff001"><addr-line>Real-time Outbreak and Disease Surveillance (RODS) Laboratory, Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, United States of America</addr-line></nlm:aff></affiliation></author><author><name sortKey="Ferraro, Jeffrey P" sort="Ferraro, Jeffrey P" uniqKey="Ferraro J" first="Jeffrey P." last="Ferraro">Jeffrey P. Ferraro</name><affiliation><nlm:aff id="aff002"><addr-line>Department of Biomedical Informatics, University of Utah, Salt Lake City, Utah, United States of America</addr-line></nlm:aff></affiliation></author><author><name sortKey="Gesteland, Per H" sort="Gesteland, Per H" uniqKey="Gesteland P" first="Per H." last="Gesteland">Per H. Gesteland</name><affiliation><nlm:aff id="aff002"><addr-line>Department of Biomedical Informatics, University of Utah, Salt Lake City, Utah, United States of America</addr-line></nlm:aff></affiliation></author><author><name sortKey="Tsui, Fuchiang" sort="Tsui, Fuchiang" uniqKey="Tsui F" first="Fuchiang" last="Tsui">Fuchiang Tsui</name><affiliation><nlm:aff id="aff001"><addr-line>Real-time Outbreak and Disease Surveillance (RODS) Laboratory, Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, United States of America</addr-line></nlm:aff></affiliation></author><author><name sortKey="Ye, Ye" sort="Ye, Ye" uniqKey="Ye Y" first="Ye" last="Ye">Ye Ye</name><affiliation><nlm:aff id="aff001"><addr-line>Real-time Outbreak and Disease Surveillance (RODS) Laboratory, Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, United States of America</addr-line></nlm:aff></affiliation></author><author><name sortKey="Wagner, Michael M" sort="Wagner, Michael M" uniqKey="Wagner M" first="Michael M." last="Wagner">Michael M. Wagner</name><affiliation><nlm:aff id="aff001"><addr-line>Real-time Outbreak and Disease Surveillance (RODS) Laboratory, Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, United States of America</addr-line></nlm:aff></affiliation></author><author><name sortKey="Cooper, Gregory F" sort="Cooper, Gregory F" uniqKey="Cooper G" first="Gregory F." last="Cooper">Gregory F. Cooper</name><affiliation><nlm:aff id="aff001"><addr-line>Real-time Outbreak and Disease Surveillance (RODS) Laboratory, Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, United States of America</addr-line></nlm:aff></affiliation></author></analytic><series><title level="j">PLoS ONE</title><idno type="eISSN">1932-6203</idno><imprint><date when="2020">2020</date></imprint></series></biblStruct></sourceDesc></fileDesc><profileDesc><textClass></textClass></profileDesc></teiHeader><front><div type="abstract" xml:lang="en"><p>Over the past decade, outbreaks of new or reemergent viruses such as severe acute respiratory syndrome (SARS) virus, Middle East respiratory syndrome (MERS) virus, and Zika have claimed thousands of lives and cost governments and healthcare systems billions of dollars. Because the appearance of new or transformed diseases is likely to continue, the detection and characterization of emergent diseases is an important problem. We describe a Bayesian statistical model that can detect and characterize previously unknown and unmodeled diseases from patient-care reports and evaluate its performance on historical data.</p></div></front><back><div1 type="bibliography"><listBibl><biblStruct><analytic><author><name sortKey="Metcalf, Cje" uniqKey="Metcalf C">CJE Metcalf</name></author><author><name sortKey="Lessler, J" uniqKey="Lessler J">J Lessler</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Holmes, Ec" uniqKey="Holmes E">EC Holmes</name></author><author><name sortKey="Rambaut, A" uniqKey="Rambaut A">A Rambaut</name></author><author><name sortKey="Andersen, Kg" uniqKey="Andersen K">KG Andersen</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Dato, V" uniqKey="Dato V">V Dato</name></author><author><name sortKey="Shephard, R" uniqKey="Shephard R">R Shephard</name></author><author><name sortKey="Wagner, Mm" uniqKey="Wagner M">MM Wagner</name></author><author><name sortKey="Wagner, Mm" uniqKey="Wagner M">MM Wagner</name></author><author><name sortKey="Moore, Aw" uniqKey="Moore A">AW Moore</name></author><author><name sortKey="Aryel, Rm" uniqKey="Aryel R">RM Aryel</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Wagner, Mm" uniqKey="Wagner M">MM Wagner</name></author><author><name sortKey="Gresham, Ls" uniqKey="Gresham L">LS Gresham</name></author><author><name sortKey="Dato, V" uniqKey="Dato V">V Dato</name></author><author><name sortKey="Wagner, Mm" uniqKey="Wagner M">MM Wagner</name></author><author><name sortKey="Moore, Aw" uniqKey="Moore A">AW Moore</name></author><author><name sortKey="Aryel, Rm" uniqKey="Aryel R">RM Aryel</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Velikina, R" uniqKey="Velikina R">R Velikina</name></author><author><name sortKey="Dato, V" uniqKey="Dato V">V Dato</name></author><author><name sortKey="Wagner, Mm" uniqKey="Wagner M">MM Wagner</name></author><author><name sortKey="Wagner, Mm" uniqKey="Wagner M">MM Wagner</name></author><author><name sortKey="Moore, Aw" uniqKey="Moore A">AW Moore</name></author><author><name sortKey="Aryel, Rm" uniqKey="Aryel R">RM Aryel</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Wagner, Mm" uniqKey="Wagner M">MM Wagner</name></author><author><name sortKey="Hogan, Wr" uniqKey="Hogan W">WR Hogan</name></author><author><name sortKey="Aryel, Rm" uniqKey="Aryel R">RM Aryel</name></author><author><name sortKey="Wagner, Mm" uniqKey="Wagner M">MM Wagner</name></author><author><name sortKey="Moore, Aw" uniqKey="Moore A">AW Moore</name></author><author><name sortKey="Aryel, Rm" uniqKey="Aryel R">RM Aryel</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Brokopp, C" uniqKey="Brokopp C">C Brokopp</name></author><author><name sortKey="Resultan, E" uniqKey="Resultan E">E Resultan</name></author><author><name sortKey="Holmes, H" uniqKey="Holmes H">H Holmes</name></author><author><name sortKey="Wagner, Mm" uniqKey="Wagner M">MM Wagner</name></author><author><name sortKey="Wagner, Mm" uniqKey="Wagner M">MM Wagner</name></author><author><name sortKey="Moore, Aw" uniqKey="Moore A">AW Moore</name></author><author><name sortKey="Aryel, Rm" uniqKey="Aryel R">RM Aryel</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Wong, Wk" uniqKey="Wong W">WK Wong</name></author><author><name sortKey="Moore, Aw" uniqKey="Moore A">AW Moore</name></author><author><name sortKey="Wagner, Mm" uniqKey="Wagner M">MM Wagner</name></author><author><name sortKey="Moore, Aw" uniqKey="Moore A">AW Moore</name></author><author><name sortKey="Aryel, Rm" uniqKey="Aryel R">RM Aryel</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Moore, Aw" uniqKey="Moore A">AW Moore</name></author><author><name sortKey="Anderson, B" uniqKey="Anderson B">B Anderson</name></author><author><name sortKey="Das, K" uniqKey="Das K">K Das</name></author><author><name sortKey="Wong, Wk" uniqKey="Wong W">WK Wong</name></author><author><name sortKey="Wagner, Mm" uniqKey="Wagner M">MM Wagner</name></author><author><name sortKey="Moore, Aw" uniqKey="Moore A">AW Moore</name></author><author><name sortKey="Aryel, Rm" uniqKey="Aryel R">RM Aryel</name></author></analytic></biblStruct><biblStruct></biblStruct><biblStruct></biblStruct><biblStruct></biblStruct><biblStruct></biblStruct><biblStruct></biblStruct><biblStruct><analytic><author><name sortKey="Cooper, Gf" uniqKey="Cooper G">GF Cooper</name></author><author><name sortKey="Villamarin, R" uniqKey="Villamarin R">R Villamarin</name></author><author><name sortKey="Tsui, Fcr" uniqKey="Tsui F">FCR Tsui</name></author><author><name sortKey="Millett, N" uniqKey="Millett N">N Millett</name></author><author><name sortKey="Espino, Ju" uniqKey="Espino J">JU Espino</name></author><author><name sortKey="Wagner, Mm" uniqKey="Wagner M">MM Wagner</name></author></analytic></biblStruct><biblStruct></biblStruct><biblStruct></biblStruct><biblStruct></biblStruct><biblStruct></biblStruct><biblStruct></biblStruct><biblStruct><analytic><author><name sortKey="Aliabadi, N" uniqKey="Aliabadi N">N Aliabadi</name></author><author><name sortKey="Messacar, K" uniqKey="Messacar K">K Messacar</name></author><author><name sortKey="Pastula, Dm" uniqKey="Pastula D">DM Pastula</name></author><author><name sortKey="Robinson, Cc" uniqKey="Robinson C">CC Robinson</name></author><author><name sortKey="Leshem, E" uniqKey="Leshem E">E Leshem</name></author><author><name sortKey="Sejvar, J" uniqKey="Sejvar J">J Sejvar</name></author></analytic></biblStruct><biblStruct></biblStruct></listBibl></div1></back></TEI><pmc article-type="research-article"><pmc-dir>properties open_access</pmc-dir>
<front><journal-meta><journal-id journal-id-type="nlm-ta">PLoS One</journal-id><journal-id journal-id-type="iso-abbrev">PLoS ONE</journal-id><journal-id journal-id-type="publisher-id">plos</journal-id><journal-id journal-id-type="pmc">plosone</journal-id><journal-title-group><journal-title>PLoS ONE</journal-title></journal-title-group><issn pub-type="epub">1932-6203</issn><publisher><publisher-name>Public Library of Science</publisher-name><publisher-loc>San Francisco, CA USA</publisher-loc></publisher></journal-meta><article-meta><article-id pub-id-type="pmid">32109254</article-id><article-id pub-id-type="pmc">7048291</article-id><article-id pub-id-type="publisher-id">PONE-D-19-24937</article-id><article-id pub-id-type="doi">10.1371/journal.pone.0229658</article-id><article-categories><subj-group subj-group-type="heading"><subject>Research Article</subject></subj-group><subj-group subj-group-type="Discipline-v3"><subject>Medicine and Health Sciences</subject><subj-group><subject>Epidemiology</subject></subj-group></subj-group><subj-group subj-group-type="Discipline-v3"><subject>Medicine and Health Sciences</subject><subj-group><subject>Infectious Diseases</subject><subj-group><subject>Viral Diseases</subject><subj-group><subject>Influenza</subject></subj-group></subj-group></subj-group></subj-group><subj-group subj-group-type="Discipline-v3"><subject>Medicine and Health Sciences</subject><subj-group><subject>Critical Care and Emergency Medicine</subject></subj-group></subj-group><subj-group subj-group-type="Discipline-v3"><subject>Biology and Life Sciences</subject><subj-group><subject>Physiology</subject><subj-group><subject>Physiological Processes</subject><subj-group><subject>Coughing</subject></subj-group></subj-group></subj-group></subj-group><subj-group subj-group-type="Discipline-v3"><subject>Medicine and Health Sciences</subject><subj-group><subject>Physiology</subject><subj-group><subject>Physiological Processes</subject><subj-group><subject>Coughing</subject></subj-group></subj-group></subj-group></subj-group><subj-group subj-group-type="Discipline-v3"><subject>Medicine and Health Sciences</subject><subj-group><subject>Diagnostic Medicine</subject><subj-group><subject>Signs and Symptoms</subject><subj-group><subject>Coughing</subject></subj-group></subj-group></subj-group></subj-group><subj-group subj-group-type="Discipline-v3"><subject>Medicine and Health Sciences</subject><subj-group><subject>Pathology and Laboratory Medicine</subject><subj-group><subject>Signs and Symptoms</subject><subj-group><subject>Coughing</subject></subj-group></subj-group></subj-group></subj-group><subj-group subj-group-type="Discipline-v3"><subject>Medicine and Health Sciences</subject><subj-group><subject>Infectious Diseases</subject><subj-group><subject>Viral Diseases</subject><subj-group><subject>SARS</subject></subj-group></subj-group></subj-group></subj-group><subj-group subj-group-type="Discipline-v3"><subject>Engineering and Technology</subject><subj-group><subject>Equipment</subject><subj-group><subject>Measurement Equipment</subject><subj-group><subject>Thermometers</subject></subj-group></subj-group></subj-group></subj-group><subj-group subj-group-type="Discipline-v3"><subject>Medicine and Health Sciences</subject><subj-group><subject>Diagnostic Medicine</subject><subj-group><subject>Signs and Symptoms</subject><subj-group><subject>Fevers</subject></subj-group></subj-group></subj-group></subj-group><subj-group subj-group-type="Discipline-v3"><subject>Medicine and Health Sciences</subject><subj-group><subject>Pathology and Laboratory Medicine</subject><subj-group><subject>Signs and Symptoms</subject><subj-group><subject>Fevers</subject></subj-group></subj-group></subj-group></subj-group><subj-group subj-group-type="Discipline-v3"><subject>Medicine and Health Sciences</subject><subj-group><subject>Health Care</subject></subj-group></subj-group></article-categories><title-group><article-title>A Bayesian approach for detecting a disease that is not being modeled</article-title><alt-title alt-title-type="running-head">A Bayesian approach for detecting a disease that is not being modeled</alt-title></title-group><contrib-group><contrib contrib-type="author" equal-contrib="yes"><contrib-id authenticated="true" contrib-id-type="orcid">http://orcid.org/0000-0001-5821-3672</contrib-id><name><surname>Aronis</surname><given-names>John M.</given-names></name><role content-type="http://credit.casrai.org/">Conceptualization</role><role content-type="http://credit.casrai.org/">Investigation</role><role content-type="http://credit.casrai.org/">Methodology</role><role content-type="http://credit.casrai.org/">Software</role><role content-type="http://credit.casrai.org/">Validation</role><role content-type="http://credit.casrai.org/">Writing – original draft</role><role content-type="http://credit.casrai.org/">Writing – review & editing</role><xref ref-type="aff" rid="aff001"><sup>1</sup></xref><xref ref-type="corresp" rid="cor001">*</xref></contrib><contrib contrib-type="author"><name><surname>Ferraro</surname><given-names>Jeffrey P.</given-names></name><role content-type="http://credit.casrai.org/">Data curation</role><role content-type="http://credit.casrai.org/">Investigation</role><role content-type="http://credit.casrai.org/">Writing – review & editing</role><xref ref-type="aff" rid="aff002"><sup>2</sup></xref><xref ref-type="author-notes" rid="econtrib001"><sup>‡</sup></xref></contrib><contrib contrib-type="author"><name><surname>Gesteland</surname><given-names>Per H.</given-names></name><role content-type="http://credit.casrai.org/">Data curation</role><role content-type="http://credit.casrai.org/">Investigation</role><role content-type="http://credit.casrai.org/">Writing – review & editing</role><xref ref-type="aff" rid="aff002"><sup>2</sup></xref><xref ref-type="author-notes" rid="econtrib001"><sup>‡</sup></xref></contrib><contrib contrib-type="author"><name><surname>Tsui</surname><given-names>Fuchiang</given-names></name><role content-type="http://credit.casrai.org/">Conceptualization</role><role content-type="http://credit.casrai.org/">Data curation</role><role content-type="http://credit.casrai.org/">Writing – review & editing</role><xref ref-type="aff" rid="aff001"><sup>1</sup></xref><xref ref-type="author-notes" rid="econtrib001"><sup>‡</sup></xref></contrib><contrib contrib-type="author"><name><surname>Ye</surname><given-names>Ye</given-names></name><role content-type="http://credit.casrai.org/">Data curation</role><role content-type="http://credit.casrai.org/">Writing – review & editing</role><xref ref-type="aff" rid="aff001"><sup>1</sup></xref><xref ref-type="author-notes" rid="econtrib001"><sup>‡</sup></xref></contrib><contrib contrib-type="author"><name><surname>Wagner</surname><given-names>Michael M.</given-names></name><role content-type="http://credit.casrai.org/">Conceptualization</role><role content-type="http://credit.casrai.org/">Data curation</role><role content-type="http://credit.casrai.org/">Funding acquisition</role><role content-type="http://credit.casrai.org/">Investigation</role><role content-type="http://credit.casrai.org/">Methodology</role><role content-type="http://credit.casrai.org/">Project administration</role><role content-type="http://credit.casrai.org/">Writing – review & editing</role><xref ref-type="aff" rid="aff001"><sup>1</sup></xref><xref ref-type="author-notes" rid="econtrib001"><sup>‡</sup></xref></contrib><contrib contrib-type="author" equal-contrib="yes"><name><surname>Cooper</surname><given-names>Gregory F.</given-names></name><role content-type="http://credit.casrai.org/">Conceptualization</role><role content-type="http://credit.casrai.org/">Formal analysis</role><role content-type="http://credit.casrai.org/">Investigation</role><role content-type="http://credit.casrai.org/">Methodology</role><role content-type="http://credit.casrai.org/">Supervision</role><role content-type="http://credit.casrai.org/">Writing – original draft</role><role content-type="http://credit.casrai.org/">Writing – review & editing</role><xref ref-type="aff" rid="aff001"><sup>1</sup></xref></contrib></contrib-group><aff id="aff001"><label>1</label><addr-line>Real-time Outbreak and Disease Surveillance (RODS) Laboratory, Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, United States of America</addr-line></aff><aff id="aff002"><label>2</label><addr-line>Department of Biomedical Informatics, University of Utah, Salt Lake City, Utah, United States of America</addr-line></aff><contrib-group><contrib contrib-type="editor"><name><surname>d’Onofrio</surname><given-names>Alberto</given-names></name><role>Editor</role><xref ref-type="aff" rid="edit1"></xref></contrib></contrib-group><aff id="edit1"><addr-line>International Prevention Research Institute, FRANCE</addr-line></aff><author-notes><fn fn-type="COI-statement" id="coi001"><p><bold>Competing Interests: </bold>The authors have declared that no competing interests exist.</p></fn><fn fn-type="other" id="econtrib001"><p>‡These authors also contributed equally to this work.</p></fn><corresp id="cor001">* E-mail: <email>jma18@pitt.edu</email></corresp></author-notes><pub-date pub-type="collection"><year>2020</year></pub-date><pub-date pub-type="epub"><day>28</day><month>2</month><year>2020</year></pub-date><volume>15</volume><issue>2</issue><elocation-id>e0229658</elocation-id><history><date date-type="received"><day>4</day><month>9</month><year>2019</year></date><date date-type="accepted"><day>12</day><month>2</month><year>2020</year></date></history><permissions><copyright-statement>© 2020 Aronis et al</copyright-statement><copyright-year>2020</copyright-year><copyright-holder>Aronis et al</copyright-holder><license xlink:href="http://creativecommons.org/licenses/by/4.0/"><license-p>This is an open access article distributed under the terms of the <ext-link ext-link-type="uri" xlink:href="http://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution License</ext-link>, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.</license-p></license></permissions><self-uri content-type="pdf" xlink:href="pone.0229658.pdf"></self-uri><abstract><p>Over the past decade, outbreaks of new or reemergent viruses such as severe acute respiratory syndrome (SARS) virus, Middle East respiratory syndrome (MERS) virus, and Zika have claimed thousands of lives and cost governments and healthcare systems billions of dollars. Because the appearance of new or transformed diseases is likely to continue, the detection and characterization of emergent diseases is an important problem. We describe a Bayesian statistical model that can detect and characterize previously unknown and unmodeled diseases from patient-care reports and evaluate its performance on historical data.</p></abstract><funding-group><award-group id="award001"><funding-source><institution-wrap><institution-id institution-id-type="funder-id">http://dx.doi.org/10.13039/100000002</institution-id><institution>National Institutes of Health</institution></institution-wrap></funding-source><award-id>R01 LM011370</award-id><principal-award-recipient><name><surname>Wagner</surname><given-names>Michael M.</given-names></name></principal-award-recipient></award-group><award-group id="award002"><funding-source><institution-wrap><institution-id institution-id-type="funder-id">http://dx.doi.org/10.13039/100000057</institution-id><institution>National Institute of General Medical Sciences</institution></institution-wrap></funding-source><award-id>1U24GM110707</award-id><principal-award-recipient><name><surname>Wagner</surname><given-names>Michael M.</given-names></name></principal-award-recipient></award-group><funding-statement>MMW was supported by R01 LM011370, National Institutes of Health, nih.gov, Probabilistic Disease Surveillance; and 1U24GM110707, National Institute of General Medical Sciences, nigms.nih.gov, Modeling Infectious Disease Agent Study (MIDAS). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.</funding-statement></funding-group><counts><fig-count count="6"></fig-count><table-count count="1"></table-count><page-count count="15"></page-count></counts><custom-meta-group><custom-meta id="data-availability"><meta-name>Data Availability</meta-name><meta-value>Patient records were used in this work and cannot be shared publicly to maintain patient confidentiality. The data for this study may be requested from the third-party data owner (Intermountain Healthcare) via the Institutional Review Board at the University of Pittsburgh (<email>IRB@imail.org</email>). Access may be granted on a case-by-case basis to researchers who meet the necessary criteria.</meta-value></custom-meta></custom-meta-group></article-meta><notes><title>Data Availability</title><p>Patient records were used in this work and cannot be shared publicly to maintain patient confidentiality. The data for this study may be requested from the third-party data owner (Intermountain Healthcare) via the Institutional Review Board at the University of Pittsburgh (<email>IRB@imail.org</email>). Access may be granted on a case-by-case basis to researchers who meet the necessary criteria.</p></notes></front><body><sec sec-type="intro" id="sec001"><title>Introduction</title><p>Over the past decade, outbreaks of new or reemergent viruses such as severe acute respiratory syndrome (SARS) virus, Middle East respiratory syndrome (MERS) virus, and Zika have claimed thousands of lives and cost governments and healthcare systems billions of dollars. Whether these outbreaks were caused by increased international travel, mutating viruses, or climate change, it is clear that our generation and future generations must find ways to recognize and contain outbreaks of new viruses quickly.</p><p>The ultimate goal should be the <italic>prediction</italic> of emergent diseases before they strike the human population [<xref rid="pone.0229658.ref001" ref-type="bibr">1</xref>]. This is a lofty goal that will require dramatic advances in pathology, genetics, and ecology, along with major advances in computational science and public health practice. A complementary and achievable near-term strategy is <italic>surveillance</italic> of human populations for early signs of infectious outbreaks [<xref rid="pone.0229658.ref002" ref-type="bibr">2</xref>]. While still a formidable task, we can build on comprehensive surveillance systems already in place in the United States and much of the world [<xref rid="pone.0229658.ref003" ref-type="bibr">3</xref>–<xref rid="pone.0229658.ref007" ref-type="bibr">7</xref>].</p><p>The simplest <italic>univariate</italic> detection algorithms [<xref rid="pone.0229658.ref008" ref-type="bibr">8</xref>] track a time-series of a single value, such as emergency department visits or thermometer sales, and look for significant deviations from a baseline level of expected activity. <italic>Multivariate</italic> systems [<xref rid="pone.0229658.ref009" ref-type="bibr">9</xref>] combine several indicators into a single compound indicator in seeking to increase performance. However, these systems suffer from two problems. First, if an outbreak of a new disease occurs during a larger outbreak of a known disease, it might not be noticed. For instance, imagine if a new disease causes fever. People with this disease might purchase thermometers. However, if an outbreak of this new disease occurs during a large outbreak of influenza the increased thermometer sales due to the new disease would be overshadowed by the number of thermometer sales due to influenza. Second, they assume that <italic>the future will be like the past</italic>, and that outbreaks of influenza and other known diseases occur at the same time each year. For instance, suppose that the system expects increased thermometer sales during the month of January because that is when outbreaks of influenza typically occur but, in fact, there is no January outbreak of influenza in the current year. Then, an increase in thermometer sales in January due to an outbreak of a new disease might well be attributed to the expected outbreak of influenza and dismissed.</p><p>The WSARE (<italic>What’s Strange About Recent Events</italic>) system [<xref rid="pone.0229658.ref010" ref-type="bibr">10</xref>] addresses these issues by representing the joint distribution of patient data with a Bayesian network that includes several <italic>environmental attributes</italic> to represent <italic>influenza activity</italic>, <italic>season</italic>, <italic>weather</italic>, etc. and several <italic>response attributes</italic> to represent patient attributes such as <italic>age</italic>, <italic>gender</italic>, <italic>location</italic>, and <italic>reported symptom</italic> (which is similar to a patient’s chief complaint and takes a value from <italic>none</italic>, <italic>respiratory problems</italic>, <italic>nausea</italic>, or <italic>rash</italic>). The network is conditioned on the current values of the enviromental attributes to create a conditional joint distribution of response variables for the current day. Thus, the conditional joint distribution represents what would be expected for the current day if there are no outbreaks of new diseases. Thus, WSARE does not take a strictly technical approach to predicting cyclic patterns, but rather incorporates current conditions. The WSARE system then searches for rules that describe significant differences between the actual current data and the conditional joint distribution. For instance, if there are many patients with fever in the current data, but the conditional joint distribution predicted few patients with fever, WSARE would report this. The WSARE system has two important shortcomings, however. First, as with time-series methods, if an outbreak of a new disease occurs during a large outbreak of influenza it might not be noticed. Second, it can be misled by outbreaks of other known diseases such as RSV, parainfluenza, hMPV, etc. For instance, if the environmental attribute <italic>influenza activity</italic> is low, but there is an outbreak of RSV, the resulting surge of patients with respiratory ailments would cause a false alarm. (We note that these shortcomings could be corrected by adding additional enviromental attributes for each known disease and additional response attributes for a set of clinical findings sufficient to distinguish between different respiratory illnesses).</p><p>A patient’s chief complaint is a concise statement of the symptom or problem that caused the patient to seek medical care. Often stated in the patient’s own words, a chief complaint typically does not mention predefined syndromes or disease categories. Keyword-based systems can operate in real-time to identify anomalies or clusters of unusual findings from text [<xref rid="pone.0229658.ref011" ref-type="bibr">11</xref>]. [<xref rid="pone.0229658.ref012" ref-type="bibr">12</xref>] describes a semantic scan sytem that infers a set of topics (probability distributions over words) from the free-text of chief complaints. The system learns one set of topics using past data and a second set of topics from the most recent 3-hour moving window, assigns each patient case to its most likely topic, then looks for anomalous counts and clusters of topics. [<xref rid="pone.0229658.ref013" ref-type="bibr">13</xref>] extends this approach to identify clusters also based on geography or social patterns.</p><p>A distinctive feature of an infectious outbreak is an initial period of exponential growth. [<xref rid="pone.0229658.ref014" ref-type="bibr">14</xref>] describes a Bayesian system, based on moving windows of ILI records, to detect a period of exponential growth that can signal the start of an outbreak. That system uses Bayes’ factors to determine when it is likely that an outbreak has started.</p><p>This paper describes a Bayesian modeling approach called DUDE (<italic>Detection of Unmodeled Diseases from Evidence</italic>) that can recognize outbreaks of new forms of <italic>influenza-like illness</italic> (ILI) and create clinical characterizations of them. We demonstrate its operation on data from real-world outbreaks including an outbreak of Enterovirus EV-D68. DUDE avoids the shortcomings of previous sytems by building probabilistic models of normal (baseline) ILI activity using a large set of patient findings extracted from patient-care reports using natural language processing. (This approach to ILI detection and characterization was developed in [<xref rid="pone.0229658.ref015" ref-type="bibr">15</xref>]). It then looks for statistically significant deviations from baseline normal activity. Thus, DUDE does not rely on just a small set of findings (as might be extracted from patients’ chief complaints). Also, it does not model temporal patterns and therefore does not assume that the present is like the past. Finally, by removing cases of known forms of ILI (such as influenza, RSV, etc.) from the input data, it can recognize new, emergent kinds of ILI.</p></sec><sec sec-type="materials|methods" id="sec002"><title>Materials and methods</title><sec id="sec003"><title>Software design</title><p>DUDE uses emergency department (ED) reports of patients with an ILI (defined as patients with fever and cough or sore throat) who have definitively tested negative for a set of common viruses. Each report has been processed using natural language processing (NLP) to extract binary features for a set of symptoms relevant to the diagnosis of ILI.</p><p>We create two <italic>windows</italic> on the data. The <italic>baseline window</italic> includes patient records from a controlled time period when no outbreaks of new viruses are suspected. DUDE collects statistics about rates of symptoms among patients in the baseline window to model <italic>background ILI</italic>. The <italic>monitor window</italic> includes patient records from a later time period. DUDE then computes the likelihood of the data in the monitor period assuming that only background ILI is present, and the likelihood of the data in the monitor period assuming that both background ILI and a different <italic>unmodeled</italic> ILI is present. We then compute the odds that an unmodeled ILI is present. Note that our data are collected in hospitals. The number of patients with a disease who go to a hospital is certainly related to the number of cases in the general population. Even if this is a simple linear relationship it is hidden since we have no way to directly count the number of people in the general population with a disease We avoid the difficulty of determining this relationship by working solely with ED data.</p><p>The dataset includes patient records from one outbreak year (from June 1 of one year through May 31 of the next year), and parameters including the length of a wait period <italic>w</italic>, and the length of a monitor window <italic>m</italic>. For each current day <italic>c</italic>, the baseline window includes data from days 1 through <italic>c</italic> − (<italic>w</italic> + <italic>m</italic>) and the monitor window includes data from days <italic>c</italic> − <italic>m</italic> through <italic>c</italic>. For the experiments reported in this paper we set <italic>w</italic> = 14, <italic>m</italic> = 28, and start looking for outbreaks on September 1 (day <italic>c</italic> = 93). Starting with <italic>c</italic> = 93 ensures that the baseline window includes data from at least 50 days which is sufficient to characterize baseline ILI. See <xref ref-type="fig" rid="pone.0229658.g001">Fig 1</xref>.</p><fig id="pone.0229658.g001" orientation="portrait" position="float"><object-id pub-id-type="doi">10.1371/journal.pone.0229658.g001</object-id><label>Fig 1</label><caption><title>Baseline and monitor windows.</title></caption><graphic xlink:href="pone.0229658.g001"></graphic></fig><p>DUDE was implemented in Java running under Ubuntu Linux. All experiments were run on a single 2.5 GHz processor.</p></sec><sec id="sec004"><title>Mathematical modeling</title><p>The baseline and monitor windows contain patients who have definitively tested negative for a set of known types of ILI. Therefore, both the baseline and monitor windows contain patients who each have either <italic>background</italic> ILI, denoted <inline-formula id="pone.0229658.e001"><alternatives><graphic xlink:href="pone.0229658.e001.jpg" id="pone.0229658.e001g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M1"><mml:mi mathvariant="script">B</mml:mi></mml:math></alternatives></inline-formula>, or an unmodeled ILI, denoted <inline-formula id="pone.0229658.e002"><alternatives><graphic xlink:href="pone.0229658.e002.jpg" id="pone.0229658.e002g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M2"><mml:mi mathvariant="script">U</mml:mi></mml:math></alternatives></inline-formula>. We assume that <inline-formula id="pone.0229658.e003"><alternatives><graphic xlink:href="pone.0229658.e003.jpg" id="pone.0229658.e003g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M3"><mml:mi mathvariant="script">B</mml:mi></mml:math></alternatives></inline-formula> and <inline-formula id="pone.0229658.e004"><alternatives><graphic xlink:href="pone.0229658.e004.jpg" id="pone.0229658.e004g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M4"><mml:mi mathvariant="script">U</mml:mi></mml:math></alternatives></inline-formula> are disjoint and that few (if any) patients with <inline-formula id="pone.0229658.e005"><alternatives><graphic xlink:href="pone.0229658.e005.jpg" id="pone.0229658.e005g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M5"><mml:mi mathvariant="script">U</mml:mi></mml:math></alternatives></inline-formula> appear in the baseline window.</p><p>Let <inline-formula id="pone.0229658.e006"><alternatives><graphic xlink:href="pone.0229658.e006.jpg" id="pone.0229658.e006g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M6"><mml:mrow><mml:mo>#</mml:mo><mml:mi mathvariant="script">B</mml:mi></mml:mrow></mml:math></alternatives></inline-formula> and <inline-formula id="pone.0229658.e007"><alternatives><graphic xlink:href="pone.0229658.e007.jpg" id="pone.0229658.e007g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M7"><mml:mrow><mml:mo>#</mml:mo><mml:mi mathvariant="script">U</mml:mi></mml:mrow></mml:math></alternatives></inline-formula> be the number of patients in the monitor window with <inline-formula id="pone.0229658.e008"><alternatives><graphic xlink:href="pone.0229658.e008.jpg" id="pone.0229658.e008g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M8"><mml:mi mathvariant="script">B</mml:mi></mml:math></alternatives></inline-formula> or <inline-formula id="pone.0229658.e009"><alternatives><graphic xlink:href="pone.0229658.e009.jpg" id="pone.0229658.e009g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M9"><mml:mi mathvariant="script">U</mml:mi></mml:math></alternatives></inline-formula> respectively. We do not know the diagnosis of any patient, the values of <inline-formula id="pone.0229658.e010"><alternatives><graphic xlink:href="pone.0229658.e010.jpg" id="pone.0229658.e010g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M10"><mml:mrow><mml:mo>#</mml:mo><mml:mi mathvariant="script">B</mml:mi></mml:mrow></mml:math></alternatives></inline-formula> or <inline-formula id="pone.0229658.e011"><alternatives><graphic xlink:href="pone.0229658.e011.jpg" id="pone.0229658.e011g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M11"><mml:mrow><mml:mo>#</mml:mo><mml:mi mathvariant="script">U</mml:mi></mml:mrow></mml:math></alternatives></inline-formula>, or even if <inline-formula id="pone.0229658.e012"><alternatives><graphic xlink:href="pone.0229658.e012.jpg" id="pone.0229658.e012g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M12"><mml:mrow><mml:mo>#</mml:mo><mml:mi mathvariant="script">B</mml:mi><mml:mo>=</mml:mo><mml:mn>0</mml:mn></mml:mrow></mml:math></alternatives></inline-formula> or <inline-formula id="pone.0229658.e013"><alternatives><graphic xlink:href="pone.0229658.e013.jpg" id="pone.0229658.e013g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M13"><mml:mrow><mml:mo>#</mml:mo><mml:mi mathvariant="script">U</mml:mi><mml:mo>=</mml:mo><mml:mn>0</mml:mn></mml:mrow></mml:math></alternatives></inline-formula>. In fact, our objective is to determine if <inline-formula id="pone.0229658.e014"><alternatives><graphic xlink:href="pone.0229658.e014.jpg" id="pone.0229658.e014g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M14"><mml:mrow><mml:mo>#</mml:mo><mml:mi mathvariant="script">U</mml:mi><mml:mo>></mml:mo><mml:mn>0</mml:mn></mml:mrow></mml:math></alternatives></inline-formula>.</p><p>Each patient <italic>p</italic> has a set of boolean-valued findings <inline-formula id="pone.0229658.e015"><alternatives><graphic xlink:href="pone.0229658.e015.jpg" id="pone.0229658.e015g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M15"><mml:mrow><mml:mover accent="true"><mml:mi>f</mml:mi><mml:mo>¯</mml:mo></mml:mover><mml:mrow><mml:mo>(</mml:mo><mml:mi>p</mml:mi><mml:mo>)</mml:mo></mml:mrow><mml:mo>=</mml:mo><mml:mrow><mml:mo>{</mml:mo><mml:msub><mml:mi>f</mml:mi><mml:mn>1</mml:mn></mml:msub><mml:mrow><mml:mo>(</mml:mo><mml:mi>p</mml:mi><mml:mo>)</mml:mo></mml:mrow><mml:mo>,</mml:mo><mml:mo>…</mml:mo><mml:mo>,</mml:mo><mml:msub><mml:mi>f</mml:mi><mml:mi>n</mml:mi></mml:msub><mml:mrow><mml:mo>(</mml:mo><mml:mi>p</mml:mi><mml:mo>)</mml:mo></mml:mrow><mml:mo>}</mml:mo></mml:mrow></mml:mrow></mml:math></alternatives></inline-formula> (where <italic>f</italic><sub><italic>i</italic></sub>(<italic>p</italic>) is true if patient <italic>p</italic> has finding <italic>i</italic>). We make two independence assumptions:
<disp-formula id="pone.0229658.e016"><alternatives><graphic xlink:href="pone.0229658.e016.jpg" id="pone.0229658.e016g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M16"><mml:mtable displaystyle="true"><mml:mtr><mml:mtd columnalign="right"><mml:mrow><mml:mi>P</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mover accent="true"><mml:mi>f</mml:mi><mml:mo>¯</mml:mo></mml:mover><mml:mrow><mml:mo>(</mml:mo><mml:msub><mml:mi>p</mml:mi><mml:mn>1</mml:mn></mml:msub><mml:mo>)</mml:mo></mml:mrow><mml:mo>,</mml:mo><mml:mo>…</mml:mo><mml:mo>,</mml:mo><mml:mover accent="true"><mml:mi>f</mml:mi><mml:mo>¯</mml:mo></mml:mover><mml:mrow><mml:mo>(</mml:mo><mml:msub><mml:mi>p</mml:mi><mml:mi>N</mml:mi></mml:msub><mml:mo>)</mml:mo></mml:mrow><mml:mo>|</mml:mo><mml:mi>d</mml:mi><mml:mi>i</mml:mi><mml:mi>s</mml:mi><mml:mi>e</mml:mi><mml:mi>a</mml:mi><mml:mi>s</mml:mi><mml:mi>e</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:msub><mml:mi>p</mml:mi><mml:mn>1</mml:mn></mml:msub><mml:mo>)</mml:mo></mml:mrow><mml:mo>,</mml:mo><mml:mo>…</mml:mo><mml:mo>,</mml:mo><mml:mi>d</mml:mi><mml:mi>i</mml:mi><mml:mi>s</mml:mi><mml:mi>e</mml:mi><mml:mi>a</mml:mi><mml:mi>s</mml:mi><mml:mi>e</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:msub><mml:mi>p</mml:mi><mml:mi>N</mml:mi></mml:msub><mml:mo>)</mml:mo></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:mo>=</mml:mo><mml:munderover><mml:mo>∏</mml:mo><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mi>N</mml:mi></mml:munderover><mml:mi>P</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mover accent="true"><mml:mi>f</mml:mi><mml:mo>¯</mml:mo></mml:mover><mml:mrow><mml:mo>(</mml:mo><mml:msub><mml:mi>p</mml:mi><mml:mi>i</mml:mi></mml:msub><mml:mo>)</mml:mo></mml:mrow><mml:mo>|</mml:mo><mml:mi>d</mml:mi><mml:mi>i</mml:mi><mml:mi>s</mml:mi><mml:mi>e</mml:mi><mml:mi>a</mml:mi><mml:mi>s</mml:mi><mml:mi>e</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:msub><mml:mi>p</mml:mi><mml:mi>i</mml:mi></mml:msub><mml:mo>)</mml:mo></mml:mrow><mml:mo>)</mml:mo></mml:mrow></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:math></alternatives><label>(1)</label></disp-formula><disp-formula id="pone.0229658.e017"><alternatives><graphic xlink:href="pone.0229658.e017.jpg" id="pone.0229658.e017g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M17"><mml:mtable displaystyle="true"><mml:mtr><mml:mtd columnalign="right"><mml:mrow><mml:mi>P</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mover accent="true"><mml:mi>f</mml:mi><mml:mo>¯</mml:mo></mml:mover><mml:mrow><mml:mo>(</mml:mo><mml:mi>p</mml:mi><mml:mo>)</mml:mo></mml:mrow><mml:mo>|</mml:mo><mml:mi>d</mml:mi><mml:mi>i</mml:mi><mml:mi>s</mml:mi><mml:mi>e</mml:mi><mml:mi>a</mml:mi><mml:mi>s</mml:mi><mml:mi>e</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mi>p</mml:mi><mml:mo>)</mml:mo></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:mo>=</mml:mo><mml:munderover><mml:mo>∏</mml:mo><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mi>n</mml:mi></mml:munderover><mml:mi>P</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:msub><mml:mi>f</mml:mi><mml:mi>i</mml:mi></mml:msub><mml:mrow><mml:mo>(</mml:mo><mml:mi>p</mml:mi><mml:mo>)</mml:mo></mml:mrow><mml:mo>|</mml:mo><mml:mi>d</mml:mi><mml:mi>i</mml:mi><mml:mi>s</mml:mi><mml:mi>e</mml:mi><mml:mi>a</mml:mi><mml:mi>s</mml:mi><mml:mi>e</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mi>p</mml:mi><mml:mo>)</mml:mo></mml:mrow><mml:mo>)</mml:mo></mml:mrow></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:math></alternatives><label>(2)</label></disp-formula>where <inline-formula id="pone.0229658.e018"><alternatives><graphic xlink:href="pone.0229658.e018.jpg" id="pone.0229658.e018g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M18"><mml:mrow><mml:mi>N</mml:mi><mml:mo>=</mml:mo><mml:mo>#</mml:mo><mml:mi mathvariant="script">B</mml:mi><mml:mo>+</mml:mo><mml:mo>#</mml:mo><mml:mi mathvariant="script">U</mml:mi></mml:mrow></mml:math></alternatives></inline-formula> is the total number of patients in the monitor window and <italic>disease</italic>(<italic>p</italic>) is the disease of patient <italic>p</italic> (either <inline-formula id="pone.0229658.e019"><alternatives><graphic xlink:href="pone.0229658.e019.jpg" id="pone.0229658.e019g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M19"><mml:mi mathvariant="script">B</mml:mi></mml:math></alternatives></inline-formula> or <inline-formula id="pone.0229658.e020"><alternatives><graphic xlink:href="pone.0229658.e020.jpg" id="pone.0229658.e020g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M20"><mml:mi mathvariant="script">U</mml:mi></mml:math></alternatives></inline-formula>). The first assumption says that each patient’s findings depend only on his or her disease. The second says that a patient’s findings are independent given their disease.</p><p>For each finding <italic>f</italic>, let:
<disp-formula id="pone.0229658.e021"><alternatives><graphic xlink:href="pone.0229658.e021.jpg" id="pone.0229658.e021g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M21"><mml:mtable displaystyle="true"><mml:mtr><mml:mtd columnalign="right"><mml:mrow><mml:msub><mml:mi>θ</mml:mi><mml:mrow><mml:mi mathvariant="script">B</mml:mi><mml:mo>,</mml:mo><mml:mi>f</mml:mi></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:mi>P</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mi>f</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mi>p</mml:mi><mml:mo>)</mml:mo></mml:mrow><mml:mo>=</mml:mo><mml:mi>T</mml:mi><mml:mo>|</mml:mo><mml:mi>d</mml:mi><mml:mi>i</mml:mi><mml:mi>s</mml:mi><mml:mi>e</mml:mi><mml:mi>a</mml:mi><mml:mi>s</mml:mi><mml:mi>e</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mi>p</mml:mi><mml:mo>)</mml:mo></mml:mrow><mml:mo>=</mml:mo><mml:mi mathvariant="script">B</mml:mi><mml:mo>)</mml:mo></mml:mrow></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:math></alternatives><label>(3)</label></disp-formula><disp-formula id="pone.0229658.e022"><alternatives><graphic xlink:href="pone.0229658.e022.jpg" id="pone.0229658.e022g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M22"><mml:mtable displaystyle="true"><mml:mtr><mml:mtd columnalign="right"><mml:mrow><mml:msub><mml:mi>θ</mml:mi><mml:mrow><mml:mi mathvariant="script">U</mml:mi><mml:mo>,</mml:mo><mml:mi>f</mml:mi></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:mi>P</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mi>f</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mi>p</mml:mi><mml:mo>)</mml:mo></mml:mrow><mml:mo>=</mml:mo><mml:mi>T</mml:mi><mml:mo>|</mml:mo><mml:mi>d</mml:mi><mml:mi>i</mml:mi><mml:mi>s</mml:mi><mml:mi>e</mml:mi><mml:mi>a</mml:mi><mml:mi>s</mml:mi><mml:mi>e</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mi>p</mml:mi><mml:mo>)</mml:mo></mml:mrow><mml:mo>=</mml:mo><mml:mi mathvariant="script">U</mml:mi><mml:mo>)</mml:mo></mml:mrow></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:math></alternatives><label>(4)</label></disp-formula></p><p>That is, <inline-formula id="pone.0229658.e023"><alternatives><graphic xlink:href="pone.0229658.e023.jpg" id="pone.0229658.e023g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M23"><mml:msub><mml:mi>θ</mml:mi><mml:mrow><mml:mi mathvariant="script">B</mml:mi><mml:mo>,</mml:mo><mml:mi>f</mml:mi></mml:mrow></mml:msub></mml:math></alternatives></inline-formula> is the probability that a patient with disease <inline-formula id="pone.0229658.e024"><alternatives><graphic xlink:href="pone.0229658.e024.jpg" id="pone.0229658.e024g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M24"><mml:mi mathvariant="script">B</mml:mi></mml:math></alternatives></inline-formula> has finding <italic>f</italic>, and <inline-formula id="pone.0229658.e025"><alternatives><graphic xlink:href="pone.0229658.e025.jpg" id="pone.0229658.e025g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M25"><mml:msub><mml:mi>θ</mml:mi><mml:mrow><mml:mi mathvariant="script">U</mml:mi><mml:mo>,</mml:mo><mml:mi>f</mml:mi></mml:mrow></mml:msub></mml:math></alternatives></inline-formula> is the probability that a patient with disease <inline-formula id="pone.0229658.e026"><alternatives><graphic xlink:href="pone.0229658.e026.jpg" id="pone.0229658.e026g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M26"><mml:mi mathvariant="script">U</mml:mi></mml:math></alternatives></inline-formula> has finding <italic>f</italic>. We estimate each <inline-formula id="pone.0229658.e027"><alternatives><graphic xlink:href="pone.0229658.e027.jpg" id="pone.0229658.e027g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M27"><mml:msub><mml:mi>θ</mml:mi><mml:mrow><mml:mi mathvariant="script">B</mml:mi><mml:mo>,</mml:mo><mml:mi>f</mml:mi></mml:mrow></mml:msub></mml:math></alternatives></inline-formula> with:
<disp-formula id="pone.0229658.e028"><alternatives><graphic xlink:href="pone.0229658.e028.jpg" id="pone.0229658.e028g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M28"><mml:mtable displaystyle="true"><mml:mtr><mml:mtd columnalign="right"><mml:mrow><mml:msub><mml:mi>θ</mml:mi><mml:mrow><mml:mi mathvariant="script">B</mml:mi><mml:mo>,</mml:mo><mml:mi>f</mml:mi></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:mrow><mml:mo>(</mml:mo><mml:mo>#</mml:mo><mml:msub><mml:mi>f</mml:mi><mml:mi>b</mml:mi></mml:msub><mml:mo>+</mml:mo><mml:mn>1</mml:mn><mml:mo>)</mml:mo></mml:mrow><mml:mo>/</mml:mo><mml:mrow><mml:mo>(</mml:mo><mml:mo>#</mml:mo><mml:mi>t</mml:mi><mml:mi>o</mml:mi><mml:mi>t</mml:mi><mml:mi>a</mml:mi><mml:msub><mml:mi>l</mml:mi><mml:mi>b</mml:mi></mml:msub><mml:mo>+</mml:mo><mml:mn>2</mml:mn><mml:mo>)</mml:mo></mml:mrow></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:math></alternatives><label>(5)</label></disp-formula>where #<italic>f</italic><sub><italic>b</italic></sub> is the number of patients in the baseline window with finding <italic>f</italic> and #<italic>total</italic><sub><italic>b</italic></sub> is the total number of patients in the baseline window. This estimate is based on our assumption that the baseline window includes only patients with background ILI. Also, let #<italic>f</italic><sub><italic>m</italic></sub> be the number of patients in the monitor window with finding <italic>f</italic>, and let <inline-formula id="pone.0229658.e029"><alternatives><graphic xlink:href="pone.0229658.e029.jpg" id="pone.0229658.e029g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M29"><mml:mrow><mml:mo>{</mml:mo><mml:msub><mml:mi>θ</mml:mi><mml:mrow><mml:mi mathvariant="script">B</mml:mi><mml:mo>,</mml:mo><mml:mi>f</mml:mi></mml:mrow></mml:msub><mml:mo>}</mml:mo></mml:mrow></mml:math></alternatives></inline-formula>, <inline-formula id="pone.0229658.e030"><alternatives><graphic xlink:href="pone.0229658.e030.jpg" id="pone.0229658.e030g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M30"><mml:mrow><mml:mo>{</mml:mo><mml:msub><mml:mi>θ</mml:mi><mml:mrow><mml:mi mathvariant="script">U</mml:mi><mml:mo>,</mml:mo><mml:mi>f</mml:mi></mml:mrow></mml:msub><mml:mo>}</mml:mo></mml:mrow></mml:math></alternatives></inline-formula>, and {#<italic>f</italic><sub><italic>m</italic></sub>} denote the sets of these values where <italic>f</italic> ranges over the set of findings.</p><p>Given that <inline-formula id="pone.0229658.e031"><alternatives><graphic xlink:href="pone.0229658.e031.jpg" id="pone.0229658.e031g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M31"><mml:mrow><mml:mo>#</mml:mo><mml:mi mathvariant="script">B</mml:mi></mml:mrow></mml:math></alternatives></inline-formula> and <inline-formula id="pone.0229658.e032"><alternatives><graphic xlink:href="pone.0229658.e032.jpg" id="pone.0229658.e032g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M32"><mml:mrow><mml:mo>#</mml:mo><mml:mi mathvariant="script">U</mml:mi></mml:mrow></mml:math></alternatives></inline-formula> patients in the monitor window have diseases <inline-formula id="pone.0229658.e033"><alternatives><graphic xlink:href="pone.0229658.e033.jpg" id="pone.0229658.e033g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M33"><mml:mi mathvariant="script">B</mml:mi></mml:math></alternatives></inline-formula> and <inline-formula id="pone.0229658.e034"><alternatives><graphic xlink:href="pone.0229658.e034.jpg" id="pone.0229658.e034g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M34"><mml:mi mathvariant="script">U</mml:mi></mml:math></alternatives></inline-formula>, respectively, the probabilities <inline-formula id="pone.0229658.e035"><alternatives><graphic xlink:href="pone.0229658.e035.jpg" id="pone.0229658.e035g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M35"><mml:msub><mml:mi>θ</mml:mi><mml:mrow><mml:mi mathvariant="script">B</mml:mi><mml:mo>,</mml:mo><mml:mi>f</mml:mi></mml:mrow></mml:msub></mml:math></alternatives></inline-formula> and <inline-formula id="pone.0229658.e036"><alternatives><graphic xlink:href="pone.0229658.e036.jpg" id="pone.0229658.e036g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M36"><mml:msub><mml:mi>θ</mml:mi><mml:mrow><mml:mi mathvariant="script">U</mml:mi><mml:mo>,</mml:mo><mml:mi>f</mml:mi></mml:mrow></mml:msub></mml:math></alternatives></inline-formula> from Eqs <xref ref-type="disp-formula" rid="pone.0229658.e021">3</xref> and <xref ref-type="disp-formula" rid="pone.0229658.e022">4</xref>, and Assumptions 1 and 2, the probability that exactly <italic>i</italic> of the patients with <inline-formula id="pone.0229658.e037"><alternatives><graphic xlink:href="pone.0229658.e037.jpg" id="pone.0229658.e037g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M37"><mml:mi mathvariant="script">B</mml:mi></mml:math></alternatives></inline-formula> and <italic>j</italic> of the patients with <inline-formula id="pone.0229658.e038"><alternatives><graphic xlink:href="pone.0229658.e038.jpg" id="pone.0229658.e038g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M38"><mml:mi mathvariant="script">U</mml:mi></mml:math></alternatives></inline-formula> have finding <italic>f</italic> is:
<disp-formula id="pone.0229658.e039"><alternatives><graphic xlink:href="pone.0229658.e039.jpg" id="pone.0229658.e039g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M39"><mml:mtable displaystyle="true"><mml:mtr><mml:mtd columnalign="right"><mml:mrow><mml:mi>b</mml:mi><mml:mi>i</mml:mi><mml:mi>n</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mo>#</mml:mo><mml:mi mathvariant="script">B</mml:mi><mml:mo>,</mml:mo><mml:mi>i</mml:mi><mml:mo>,</mml:mo><mml:msub><mml:mi>θ</mml:mi><mml:mrow><mml:mi mathvariant="script">B</mml:mi><mml:mo>,</mml:mo><mml:mi>f</mml:mi></mml:mrow></mml:msub><mml:mo>)</mml:mo></mml:mrow><mml:mspace width="4pt"></mml:mspace><mml:mi>b</mml:mi><mml:mi>i</mml:mi><mml:mi>n</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mo>#</mml:mo><mml:mi mathvariant="script">U</mml:mi><mml:mo>,</mml:mo><mml:mi>j</mml:mi><mml:mo>,</mml:mo><mml:msub><mml:mi>θ</mml:mi><mml:mrow><mml:mi mathvariant="script">U</mml:mi><mml:mo>,</mml:mo><mml:mi>f</mml:mi></mml:mrow></mml:msub><mml:mo>)</mml:mo></mml:mrow></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:math></alternatives><label>(6)</label></disp-formula>where <italic>bin</italic> is the binomial distribution. (That is, <italic>bin</italic>(<italic>n</italic>, <italic>r</italic>, <italic>p</italic>) is the probability of choosing <italic>r</italic> items from a set of <italic>n</italic> items if each is selected independently with probability <italic>p</italic> if 0 ≤ <italic>r</italic> ≤ <italic>n</italic>, and 0 otherwise).</p><p>Since #<italic>f</italic><sub><italic>m</italic></sub> patients in the monitor window have finding <italic>f</italic>, then between 0 and #<italic>f</italic><sub><italic>m</italic></sub> of those patients have disease <inline-formula id="pone.0229658.e040"><alternatives><graphic xlink:href="pone.0229658.e040.jpg" id="pone.0229658.e040g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M40"><mml:mi mathvariant="script">B</mml:mi></mml:math></alternatives></inline-formula> and the remainder must have disease <inline-formula id="pone.0229658.e041"><alternatives><graphic xlink:href="pone.0229658.e041.jpg" id="pone.0229658.e041g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M41"><mml:mi mathvariant="script">U</mml:mi></mml:math></alternatives></inline-formula>. Since these are disjoint we have:
<disp-formula id="pone.0229658.e042"><alternatives><graphic xlink:href="pone.0229658.e042.jpg" id="pone.0229658.e042g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M42"><mml:mtable displaystyle="true"><mml:mtr><mml:mtd columnalign="right"><mml:mrow><mml:mi>P</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mo>#</mml:mo><mml:msub><mml:mi>f</mml:mi><mml:mi>m</mml:mi></mml:msub><mml:mo>|</mml:mo><mml:mo>#</mml:mo><mml:mi mathvariant="script">B</mml:mi><mml:mo>,</mml:mo><mml:mo>#</mml:mo><mml:mi mathvariant="script">U</mml:mi><mml:mo>,</mml:mo><mml:msub><mml:mi>θ</mml:mi><mml:mrow><mml:mi mathvariant="script">B</mml:mi><mml:mo>,</mml:mo><mml:mi>f</mml:mi></mml:mrow></mml:msub><mml:mo>,</mml:mo><mml:msub><mml:mi>θ</mml:mi><mml:mrow><mml:mi mathvariant="script">U</mml:mi><mml:mo>,</mml:mo><mml:mi>f</mml:mi></mml:mrow></mml:msub><mml:mo>)</mml:mo></mml:mrow><mml:mo>=</mml:mo><mml:munderover><mml:mo>∑</mml:mo><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>0</mml:mn></mml:mrow><mml:mrow><mml:mo>#</mml:mo><mml:msub><mml:mi>f</mml:mi><mml:mi>m</mml:mi></mml:msub></mml:mrow></mml:munderover><mml:mrow><mml:mo>[</mml:mo><mml:mi>b</mml:mi><mml:mi>i</mml:mi><mml:mi>n</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mo>#</mml:mo><mml:mi mathvariant="script">B</mml:mi><mml:mo>,</mml:mo><mml:mi>i</mml:mi><mml:mo>,</mml:mo><mml:msub><mml:mi>θ</mml:mi><mml:mrow><mml:mi mathvariant="script">B</mml:mi><mml:mo>,</mml:mo><mml:mi>f</mml:mi></mml:mrow></mml:msub><mml:mo>)</mml:mo></mml:mrow><mml:mspace width="4pt"></mml:mspace><mml:mi>b</mml:mi><mml:mi>i</mml:mi><mml:mi>n</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mo>#</mml:mo><mml:mi mathvariant="script">U</mml:mi><mml:mo>,</mml:mo><mml:mo>#</mml:mo><mml:msub><mml:mi>f</mml:mi><mml:mi>m</mml:mi></mml:msub><mml:mo>-</mml:mo><mml:mi>i</mml:mi><mml:mo>,</mml:mo><mml:msub><mml:mi>θ</mml:mi><mml:mrow><mml:mi mathvariant="script">U</mml:mi><mml:mo>,</mml:mo><mml:mi>f</mml:mi></mml:mrow></mml:msub><mml:mo>)</mml:mo></mml:mrow><mml:mo>]</mml:mo></mml:mrow></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:math></alternatives><label>(7)</label></disp-formula></p><p>This is simply the sum of probabilities over all possible ways that #<italic>f</italic><sub><italic>m</italic></sub> patients with finding <italic>f</italic> can be divided between a set of <inline-formula id="pone.0229658.e043"><alternatives><graphic xlink:href="pone.0229658.e043.jpg" id="pone.0229658.e043g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M43"><mml:mrow><mml:mo>#</mml:mo><mml:mi mathvariant="script">B</mml:mi></mml:mrow></mml:math></alternatives></inline-formula> patients and a set of <inline-formula id="pone.0229658.e044"><alternatives><graphic xlink:href="pone.0229658.e044.jpg" id="pone.0229658.e044g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M44"><mml:mrow><mml:mo>#</mml:mo><mml:mi mathvariant="script">U</mml:mi></mml:mrow></mml:math></alternatives></inline-formula> patients. We are ignorant of the values of <inline-formula id="pone.0229658.e045"><alternatives><graphic xlink:href="pone.0229658.e045.jpg" id="pone.0229658.e045g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M45"><mml:msub><mml:mi>θ</mml:mi><mml:mrow><mml:mi mathvariant="script">U</mml:mi><mml:mo>,</mml:mo><mml:mi>f</mml:mi></mml:mrow></mml:msub></mml:math></alternatives></inline-formula> so we assume a uniform PDF for it and integrate over [0, 1]:
<disp-formula id="pone.0229658.e046"><alternatives><graphic xlink:href="pone.0229658.e046.jpg" id="pone.0229658.e046g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M46"><mml:mtable displaystyle="true"><mml:mtr><mml:mtd columnalign="right"><mml:mrow><mml:mi>P</mml:mi><mml:mo>(</mml:mo><mml:mo>#</mml:mo><mml:msub><mml:mi>f</mml:mi><mml:mi>m</mml:mi></mml:msub><mml:mo>|</mml:mo><mml:mo>#</mml:mo><mml:mi mathvariant="script">B</mml:mi><mml:mo>,</mml:mo><mml:mo>#</mml:mo><mml:mi mathvariant="script">U</mml:mi><mml:mo>,</mml:mo><mml:msub><mml:mi>θ</mml:mi><mml:mrow><mml:mi mathvariant="script">B</mml:mi><mml:mo>,</mml:mo><mml:mi>f</mml:mi></mml:mrow></mml:msub><mml:mo>)</mml:mo></mml:mrow></mml:mtd><mml:mtd><mml:mo>=</mml:mo></mml:mtd><mml:mtd columnalign="left"><mml:mrow><mml:msubsup><mml:mo>∫</mml:mo><mml:mrow><mml:mn>0</mml:mn></mml:mrow><mml:mn>1</mml:mn></mml:msubsup><mml:munderover><mml:mo>∑</mml:mo><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>0</mml:mn></mml:mrow><mml:mrow><mml:mo>#</mml:mo><mml:msub><mml:mi>f</mml:mi><mml:mi>m</mml:mi></mml:msub></mml:mrow></mml:munderover><mml:mrow><mml:mo>[</mml:mo><mml:mi>b</mml:mi><mml:mi>i</mml:mi><mml:mi>n</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mo>#</mml:mo><mml:mi mathvariant="script">B</mml:mi><mml:mo>,</mml:mo><mml:mi>i</mml:mi><mml:mo>,</mml:mo><mml:msub><mml:mi>θ</mml:mi><mml:mrow><mml:mi mathvariant="script">B</mml:mi><mml:mo>,</mml:mo><mml:mi>f</mml:mi></mml:mrow></mml:msub><mml:mo>)</mml:mo></mml:mrow><mml:mspace width="4pt"></mml:mspace><mml:mi>b</mml:mi><mml:mi>i</mml:mi><mml:mi>n</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mo>#</mml:mo><mml:mi mathvariant="script">U</mml:mi><mml:mo>,</mml:mo><mml:mo>#</mml:mo><mml:msub><mml:mi>f</mml:mi><mml:mi>m</mml:mi></mml:msub><mml:mo>-</mml:mo><mml:mi>i</mml:mi><mml:mo>,</mml:mo><mml:msub><mml:mi>θ</mml:mi><mml:mrow><mml:mi mathvariant="script">U</mml:mi><mml:mo>,</mml:mo><mml:mi>f</mml:mi></mml:mrow></mml:msub><mml:mo>)</mml:mo></mml:mrow><mml:mo>]</mml:mo></mml:mrow><mml:mspace width="4pt"></mml:mspace><mml:mi>d</mml:mi><mml:msub><mml:mi>θ</mml:mi><mml:mrow><mml:mi mathvariant="script">U</mml:mi><mml:mo>,</mml:mo><mml:mi>f</mml:mi></mml:mrow></mml:msub></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:math></alternatives><label>(8)</label></disp-formula><disp-formula id="pone.0229658.e047"><alternatives><graphic xlink:href="pone.0229658.e047.jpg" id="pone.0229658.e047g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M47"><mml:mtable displaystyle="true"><mml:mtr><mml:mtd></mml:mtd><mml:mtd><mml:mo>=</mml:mo></mml:mtd><mml:mtd columnalign="left"><mml:mrow><mml:mfrac><mml:mn>1</mml:mn><mml:mrow><mml:mo>#</mml:mo><mml:mi mathvariant="script">U</mml:mi><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:mfrac><mml:munderover><mml:mo>∑</mml:mo><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>0</mml:mn></mml:mrow><mml:mrow><mml:mo>#</mml:mo><mml:msub><mml:mi>f</mml:mi><mml:mi>m</mml:mi></mml:msub></mml:mrow></mml:munderover><mml:mi>b</mml:mi><mml:mi>i</mml:mi><mml:mi>n</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mo>#</mml:mo><mml:mi mathvariant="script">B</mml:mi><mml:mo>,</mml:mo><mml:mi>i</mml:mi><mml:mo>,</mml:mo><mml:msub><mml:mi>θ</mml:mi><mml:mrow><mml:mi mathvariant="script">B</mml:mi><mml:mo>,</mml:mo><mml:mi>f</mml:mi></mml:mrow></mml:msub><mml:mo>)</mml:mo></mml:mrow></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:math></alternatives><label>(9)</label></disp-formula></p><p>The details of this derivation are in <xref ref-type="supplementary-material" rid="pone.0229658.s001">S1 Appendix</xref>.</p><p>The evidence consists of {#<italic>f</italic><sub><italic>m</italic></sub>}, which is the set of counts of each finding in the monitor window. Then:
<disp-formula id="pone.0229658.e048"><alternatives><graphic xlink:href="pone.0229658.e048.jpg" id="pone.0229658.e048g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M48"><mml:mtable displaystyle="true"><mml:mtr><mml:mtd columnalign="right"><mml:mrow><mml:mi>P</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mo>{</mml:mo><mml:mo>#</mml:mo><mml:msub><mml:mi>f</mml:mi><mml:mi>m</mml:mi></mml:msub><mml:mo>}</mml:mo></mml:mrow><mml:mo>|</mml:mo><mml:mo>#</mml:mo><mml:mi mathvariant="script">B</mml:mi><mml:mo>,</mml:mo><mml:mo>#</mml:mo><mml:mi mathvariant="script">U</mml:mi><mml:mo>,</mml:mo><mml:mrow><mml:mo>{</mml:mo><mml:msub><mml:mi>θ</mml:mi><mml:mrow><mml:mi mathvariant="script">B</mml:mi><mml:mo>,</mml:mo><mml:mi>f</mml:mi></mml:mrow></mml:msub><mml:mo>}</mml:mo></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:mo>=</mml:mo><mml:munder><mml:mo>∏</mml:mo><mml:mrow><mml:mi>f</mml:mi><mml:mo>∈</mml:mo><mml:mi>f</mml:mi><mml:mi>i</mml:mi><mml:mi>n</mml:mi><mml:mi>d</mml:mi><mml:mi>i</mml:mi><mml:mi>n</mml:mi><mml:mi>g</mml:mi><mml:mi>s</mml:mi></mml:mrow></mml:munder><mml:mi>P</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mo>#</mml:mo><mml:msub><mml:mi>f</mml:mi><mml:mi>m</mml:mi></mml:msub><mml:mo>|</mml:mo><mml:mo>#</mml:mo><mml:mi mathvariant="script">B</mml:mi><mml:mo>,</mml:mo><mml:mo>#</mml:mo><mml:mi mathvariant="script">U</mml:mi><mml:mo>,</mml:mo><mml:msub><mml:mi>θ</mml:mi><mml:mrow><mml:mi mathvariant="script">B</mml:mi><mml:mo>,</mml:mo><mml:mi>f</mml:mi></mml:mrow></mml:msub><mml:mo>)</mml:mo></mml:mrow></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:math></alternatives><label>(10)</label></disp-formula></p><p>The details of this derivation are also in <xref ref-type="supplementary-material" rid="pone.0229658.s001">S1 Appendix</xref>. <xref ref-type="disp-formula" rid="pone.0229658.e048">Eq 10</xref> allows us to compute the probability of the evidence in the monitor window, {#<italic>f</italic><sub><italic>m</italic></sub>}, given values for <inline-formula id="pone.0229658.e049"><alternatives><graphic xlink:href="pone.0229658.e049.jpg" id="pone.0229658.e049g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M49"><mml:mrow><mml:mo>#</mml:mo><mml:mi mathvariant="script">B</mml:mi></mml:mrow></mml:math></alternatives></inline-formula>, <inline-formula id="pone.0229658.e050"><alternatives><graphic xlink:href="pone.0229658.e050.jpg" id="pone.0229658.e050g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M50"><mml:mrow><mml:mo>#</mml:mo><mml:mi mathvariant="script">U</mml:mi></mml:mrow></mml:math></alternatives></inline-formula> and each <inline-formula id="pone.0229658.e051"><alternatives><graphic xlink:href="pone.0229658.e051.jpg" id="pone.0229658.e051g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M51"><mml:msub><mml:mi>θ</mml:mi><mml:mrow><mml:mi mathvariant="script">B</mml:mi><mml:mo>,</mml:mo><mml:mi>f</mml:mi></mml:mrow></mml:msub></mml:math></alternatives></inline-formula>. We do not know <inline-formula id="pone.0229658.e052"><alternatives><graphic xlink:href="pone.0229658.e052.jpg" id="pone.0229658.e052g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M52"><mml:mrow><mml:mo>#</mml:mo><mml:mi mathvariant="script">B</mml:mi></mml:mrow></mml:math></alternatives></inline-formula> or <inline-formula id="pone.0229658.e053"><alternatives><graphic xlink:href="pone.0229658.e053.jpg" id="pone.0229658.e053g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M53"><mml:mrow><mml:mo>#</mml:mo><mml:mi mathvariant="script">U</mml:mi></mml:mrow></mml:math></alternatives></inline-formula> specifically, but we do know that <inline-formula id="pone.0229658.e054"><alternatives><graphic xlink:href="pone.0229658.e054.jpg" id="pone.0229658.e054g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M54"><mml:mrow><mml:mo>#</mml:mo><mml:mi mathvariant="script">B</mml:mi><mml:mo>+</mml:mo><mml:mo>#</mml:mo><mml:mi mathvariant="script">U</mml:mi><mml:mo>=</mml:mo><mml:mi>N</mml:mi></mml:mrow></mml:math></alternatives></inline-formula> where <italic>N</italic> is the total number of patients in the monitor window, so:
<disp-formula id="pone.0229658.e055"><alternatives><graphic xlink:href="pone.0229658.e055.jpg" id="pone.0229658.e055g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M55"><mml:mtable displaystyle="true"><mml:mtr><mml:mtd columnalign="right"><mml:mrow><mml:mi>P</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mo>{</mml:mo><mml:mo>#</mml:mo><mml:msub><mml:mi>f</mml:mi><mml:mi>m</mml:mi></mml:msub><mml:mo>}</mml:mo></mml:mrow><mml:mo>|</mml:mo><mml:mo>#</mml:mo><mml:mi mathvariant="script">U</mml:mi><mml:mo>,</mml:mo><mml:mrow><mml:mo>{</mml:mo><mml:msub><mml:mi>θ</mml:mi><mml:mrow><mml:mi mathvariant="script">B</mml:mi><mml:mo>,</mml:mo><mml:mi>f</mml:mi></mml:mrow></mml:msub><mml:mo>}</mml:mo></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:mo>=</mml:mo><mml:mi>P</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mo>{</mml:mo><mml:mo>#</mml:mo><mml:msub><mml:mi>f</mml:mi><mml:mi>m</mml:mi></mml:msub><mml:mo>}</mml:mo></mml:mrow><mml:mo>|</mml:mo><mml:mo>#</mml:mo><mml:mi mathvariant="script">B</mml:mi><mml:mo>=</mml:mo><mml:mi>N</mml:mi><mml:mo>-</mml:mo><mml:mo>#</mml:mo><mml:mi mathvariant="script">U</mml:mi><mml:mo>,</mml:mo><mml:mo>#</mml:mo><mml:mi mathvariant="script">U</mml:mi><mml:mo>,</mml:mo><mml:mrow><mml:mo>{</mml:mo><mml:msub><mml:mi>θ</mml:mi><mml:mrow><mml:mi mathvariant="script">B</mml:mi><mml:mo>,</mml:mo><mml:mi>f</mml:mi></mml:mrow></mml:msub><mml:mo>}</mml:mo></mml:mrow><mml:mo>)</mml:mo></mml:mrow></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:math></alternatives><label>(11)</label></disp-formula></p><p>The probability of the evidence {#<italic>f</italic><sub><italic>m</italic></sub>} given that at least one patient in the monitor window has the unmodeled disease <inline-formula id="pone.0229658.e056"><alternatives><graphic xlink:href="pone.0229658.e056.jpg" id="pone.0229658.e056g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M56"><mml:mi mathvariant="script">U</mml:mi></mml:math></alternatives></inline-formula> is:
<disp-formula id="pone.0229658.e057"><alternatives><graphic xlink:href="pone.0229658.e057.jpg" id="pone.0229658.e057g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M57"><mml:mtable displaystyle="true"><mml:mtr><mml:mtd columnalign="right"><mml:mrow><mml:mi>P</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mo>{</mml:mo><mml:mo>#</mml:mo><mml:msub><mml:mi>f</mml:mi><mml:mi>m</mml:mi></mml:msub><mml:mo>}</mml:mo></mml:mrow><mml:mo>|</mml:mo><mml:mo>#</mml:mo><mml:mi mathvariant="script">U</mml:mi><mml:mo>></mml:mo><mml:mn>0</mml:mn><mml:mo>,</mml:mo><mml:mrow><mml:mo>{</mml:mo><mml:msub><mml:mi>θ</mml:mi><mml:mrow><mml:mi mathvariant="script">B</mml:mi><mml:mo>,</mml:mo><mml:mi>f</mml:mi></mml:mrow></mml:msub><mml:mo>}</mml:mo></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:mo>=</mml:mo><mml:mfrac><mml:mn>1</mml:mn><mml:mi>N</mml:mi></mml:mfrac><mml:munderover><mml:mo>∑</mml:mo><mml:mrow><mml:mo>#</mml:mo><mml:mi mathvariant="script">U</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mi>N</mml:mi></mml:munderover><mml:mi>P</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mo>{</mml:mo><mml:mo>#</mml:mo><mml:msub><mml:mi>f</mml:mi><mml:mi>m</mml:mi></mml:msub><mml:mo>}</mml:mo></mml:mrow><mml:mo>|</mml:mo><mml:mo>#</mml:mo><mml:mi mathvariant="script">B</mml:mi><mml:mo>=</mml:mo><mml:mi>N</mml:mi><mml:mo>-</mml:mo><mml:mo>#</mml:mo><mml:mi mathvariant="script">U</mml:mi><mml:mo>,</mml:mo><mml:mo>#</mml:mo><mml:mi mathvariant="script">U</mml:mi><mml:mo>,</mml:mo><mml:mrow><mml:mo>{</mml:mo><mml:msub><mml:mi>θ</mml:mi><mml:mrow><mml:mi mathvariant="script">B</mml:mi><mml:mo>,</mml:mo><mml:mi>f</mml:mi></mml:mrow></mml:msub><mml:mo>}</mml:mo></mml:mrow><mml:mo>)</mml:mo></mml:mrow></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:math></alternatives><label>(12)</label></disp-formula>assuming that it is equally likely that 1 through <italic>N</italic> patients have <inline-formula id="pone.0229658.e058"><alternatives><graphic xlink:href="pone.0229658.e058.jpg" id="pone.0229658.e058g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M58"><mml:mi mathvariant="script">U</mml:mi></mml:math></alternatives></inline-formula>. Also, the probability of the evidence given that no patient in the monitor window has the unmodeled disease is:
<disp-formula id="pone.0229658.e059"><alternatives><graphic xlink:href="pone.0229658.e059.jpg" id="pone.0229658.e059g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M59"><mml:mtable displaystyle="true"><mml:mtr><mml:mtd columnalign="right"><mml:mrow><mml:mi>P</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mo>{</mml:mo><mml:mo>#</mml:mo><mml:msub><mml:mi>f</mml:mi><mml:mi>m</mml:mi></mml:msub><mml:mo>}</mml:mo></mml:mrow><mml:mo>|</mml:mo><mml:mo>#</mml:mo><mml:mi mathvariant="script">U</mml:mi><mml:mo>=</mml:mo><mml:mn>0</mml:mn><mml:mo>,</mml:mo><mml:mrow><mml:mo>{</mml:mo><mml:msub><mml:mi>θ</mml:mi><mml:mrow><mml:mi mathvariant="script">B</mml:mi><mml:mo>,</mml:mo><mml:mi>f</mml:mi></mml:mrow></mml:msub><mml:mo>}</mml:mo></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:mo>=</mml:mo><mml:munder><mml:mo>∏</mml:mo><mml:mrow><mml:mi>f</mml:mi><mml:mo>∈</mml:mo><mml:mi>f</mml:mi><mml:mi>i</mml:mi><mml:mi>n</mml:mi><mml:mi>d</mml:mi><mml:mi>i</mml:mi><mml:mi>n</mml:mi><mml:mi>g</mml:mi><mml:mi>s</mml:mi></mml:mrow></mml:munder><mml:mi>b</mml:mi><mml:mi>i</mml:mi><mml:mi>n</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mi>N</mml:mi><mml:mo>,</mml:mo><mml:mo>#</mml:mo><mml:msub><mml:mi>f</mml:mi><mml:mi>m</mml:mi></mml:msub><mml:mo>,</mml:mo><mml:msub><mml:mi>θ</mml:mi><mml:mrow><mml:mi mathvariant="script">B</mml:mi><mml:mo>,</mml:mo><mml:mi>f</mml:mi></mml:mrow></mml:msub><mml:mo>)</mml:mo></mml:mrow></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:math></alternatives><label>(13)</label></disp-formula></p><p>Putting together Eqs <xref ref-type="disp-formula" rid="pone.0229658.e057">12</xref> and <xref ref-type="disp-formula" rid="pone.0229658.e059">13</xref>, we find the odds that some patients in the monitor window have an unmodeled disease:
<disp-formula id="pone.0229658.e060"><alternatives><graphic xlink:href="pone.0229658.e060.jpg" id="pone.0229658.e060g" mimetype="image" position="anchor" orientation="portrait"></graphic><mml:math id="M60"><mml:mtable displaystyle="true"><mml:mtr><mml:mtd columnalign="right"><mml:mfrac><mml:mrow><mml:mi>P</mml:mi><mml:mo>(</mml:mo><mml:mrow><mml:mo>{</mml:mo><mml:mo>#</mml:mo><mml:msub><mml:mi>f</mml:mi><mml:mi>m</mml:mi></mml:msub><mml:mo>}</mml:mo></mml:mrow><mml:mo>|</mml:mo><mml:mo>#</mml:mo><mml:mi mathvariant="script">U</mml:mi><mml:mo>></mml:mo><mml:mn>0</mml:mn><mml:mo>,</mml:mo><mml:mrow><mml:mo>{</mml:mo><mml:msub><mml:mi>θ</mml:mi><mml:mrow><mml:mi mathvariant="script">B</mml:mi><mml:mo>,</mml:mo><mml:mi>f</mml:mi></mml:mrow></mml:msub><mml:mo>}</mml:mo></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:mrow><mml:mi>P</mml:mi><mml:mo>(</mml:mo><mml:mrow><mml:mo>{</mml:mo><mml:mo>#</mml:mo><mml:msub><mml:mi>f</mml:mi><mml:mi>m</mml:mi></mml:msub><mml:mo>}</mml:mo></mml:mrow><mml:mo>|</mml:mo><mml:mo>#</mml:mo><mml:mi mathvariant="script">U</mml:mi><mml:mo>=</mml:mo><mml:mn>0</mml:mn><mml:mo>,</mml:mo><mml:mrow><mml:mo>{</mml:mo><mml:msub><mml:mi>θ</mml:mi><mml:mrow><mml:mi mathvariant="script">B</mml:mi><mml:mo>,</mml:mo><mml:mi>f</mml:mi></mml:mrow></mml:msub><mml:mo>}</mml:mo></mml:mrow><mml:mo>)</mml:mo></mml:mrow></mml:mfrac></mml:mtd></mml:mtr></mml:mtable></mml:math></alternatives><label>(14)</label></disp-formula></p><p>Note that <xref ref-type="disp-formula" rid="pone.0229658.e060">Eq 14</xref> is a <italic>Bayes factor</italic> and can be used to evaluate the weight of the data in favor of the presence of an unmodeled disease.</p></sec><sec id="sec005"><title>Data</title><p>The data consists of patient-care reports from emergency departments in the Intermountain Healthcare system in Salt Lake County, Utah from June 1, 2010 through May 31, 2015. These emergency departments capture about 55% of emergency department visits in Salt Lake County. Each report is processed with natural language processing software [<xref rid="pone.0229658.ref016" ref-type="bibr">16</xref>] to extract a set of 65 medical findings that clinicians determined are relevant to the diagnosis of influenza-like illnesses (listed in <xref rid="pone.0229658.t001" ref-type="table">Table 1</xref>). The data were joined to a database of results of laboratory tests. There were a total of 944, 562 patient records. After a strict syntax check of records 3, 063 were removed due to inappropriate laboratory codes leaving 941, 499 reports. From these, we selected 32, 249 reports of patients who definitively tested positive for exactly one of influenza, respiratory syncytial virus (RSV), parainfluenza, or human metapneumovirus (hMPV), or were negative for all four tested diseases.</p><table-wrap id="pone.0229658.t001" orientation="portrait" position="float"><object-id pub-id-type="doi">10.1371/journal.pone.0229658.t001</object-id><label>Table 1</label><caption><title>Medical findings that clinicians determined are relevant to the diagnosis of influenza-like illnesses.</title></caption><alternatives><graphic id="pone.0229658.t001g" xlink:href="pone.0229658.t001"></graphic><table frame="box" rules="all" border="0"><colgroup span="1"><col align="left" valign="middle" span="1"></col><col align="left" valign="middle" span="1"></col><col align="left" valign="middle" span="1"></col></colgroup><tbody><tr><td align="left" rowspan="1" colspan="1">abdominal pain</td><td align="left" rowspan="1" colspan="1">dyspnea</td><td align="left" rowspan="1" colspan="1">productive cough</td></tr><tr><td align="left" rowspan="1" colspan="1">abdominal tenderness</td><td align="left" rowspan="1" colspan="1">grunting</td><td align="left" rowspan="1" colspan="1">rales</td></tr><tr><td align="left" rowspan="1" colspan="1">abnormal chest radiograph</td><td align="left" rowspan="1" colspan="1">headache</td><td align="left" rowspan="1" colspan="1">reported fever</td></tr><tr><td align="left" rowspan="1" colspan="1">acute onset</td><td align="left" rowspan="1" colspan="1">hemoptysis</td><td align="left" rowspan="1" colspan="1">respiratory distress</td></tr><tr><td align="left" rowspan="1" colspan="1">age under six years</td><td align="left" rowspan="1" colspan="1">hoarseness</td><td align="left" rowspan="1" colspan="1">rhonchi</td></tr><tr><td align="left" rowspan="1" colspan="1">anorexia</td><td align="left" rowspan="1" colspan="1">hypoxemia</td><td align="left" rowspan="1" colspan="1">rigor</td></tr><tr><td align="left" rowspan="1" colspan="1">apnea</td><td align="left" rowspan="1" colspan="1">ill appearing</td><td align="left" rowspan="1" colspan="1">runny nose</td></tr><tr><td align="left" rowspan="1" colspan="1">arthralgia</td><td align="left" rowspan="1" colspan="1">infiltrate</td><td align="left" rowspan="1" colspan="1">seizure</td></tr><tr><td align="left" rowspan="1" colspan="1">barking cough</td><td align="left" rowspan="1" colspan="1">influenza like illness</td><td align="left" rowspan="1" colspan="1">sore throat</td></tr><tr><td align="left" rowspan="1" colspan="1">bilateral acute conjunctivitis</td><td align="left" rowspan="1" colspan="1">malaise</td><td align="left" rowspan="1" colspan="1">staccato cough</td></tr><tr><td align="left" rowspan="1" colspan="1">bronchiolitis</td><td align="left" rowspan="1" colspan="1">myalgia</td><td align="left" rowspan="1" colspan="1">streptococcal pharyngitis</td></tr><tr><td align="left" rowspan="1" colspan="1">bronchitis</td><td align="left" rowspan="1" colspan="1">nasal flaring</td><td align="left" rowspan="1" colspan="1">stridor</td></tr><tr><td align="left" rowspan="1" colspan="1">cervical lymphadenopathy</td><td align="left" rowspan="1" colspan="1">nausea</td><td align="left" rowspan="1" colspan="1">stuffy nose</td></tr><tr><td align="left" rowspan="1" colspan="1">chest pain</td><td align="left" rowspan="1" colspan="1">nonproductive cough,</td><td align="left" rowspan="1" colspan="1">tachypnea</td></tr><tr><td align="left" rowspan="1" colspan="1">chest wall retractions</td><td align="left" rowspan="1" colspan="1">other abnormal breath sounds</td><td align="left" rowspan="1" colspan="1">toxic appearance</td></tr><tr><td align="left" rowspan="1" colspan="1">chills</td><td align="left" rowspan="1" colspan="1">other cough</td><td align="left" rowspan="1" colspan="1">uri</td></tr><tr><td align="left" rowspan="1" colspan="1">conjunctivitis</td><td align="left" rowspan="1" colspan="1">other pneumonia</td><td align="left" rowspan="1" colspan="1">viral pneumonia</td></tr><tr><td align="left" rowspan="1" colspan="1">crackles</td><td align="left" rowspan="1" colspan="1">paroxysmal cough</td><td align="left" rowspan="1" colspan="1">viral syndrome</td></tr><tr><td align="left" rowspan="1" colspan="1">croup</td><td align="left" rowspan="1" colspan="1">pharyngitis diagnosis</td><td align="left" rowspan="1" colspan="1">vomiting</td></tr><tr><td align="left" rowspan="1" colspan="1">cyanosis</td><td align="left" rowspan="1" colspan="1">pharyngitis on exam</td><td align="left" rowspan="1" colspan="1">weakness or fatigue</td></tr><tr><td align="left" rowspan="1" colspan="1">decreased activity</td><td align="left" rowspan="1" colspan="1">poor feeding</td><td align="left" rowspan="1" colspan="1">wheezing</td></tr><tr><td align="left" rowspan="1" colspan="1">diarrhea</td><td align="left" rowspan="1" colspan="1">poor response to antipyretics</td><td align="left" rowspan="1" colspan="1"></td></tr></tbody></table></alternatives></table-wrap><p>The data is divided into five years: <italic>outbreak year 2010-2011</italic> contains data from June 1, 2010 through May 31, 2011, <italic>outbreak year 2011-2012</italic> contains data from June 1, 2011 through May 31, 2012, etc. For each outbreak year, we designate June 1 to be <italic>day 1</italic>. <xref ref-type="fig" rid="pone.0229658.g002">Fig 2</xref> shows the daily counts (14-day moving averages) of confirmed cases of influenza, RSV, parainfluenza, hMPV, and other (<italic>e.g.</italic> negative for all four) for Intermountain Healthcare emergency departments in Salt Lake County for outbreak years 2010-2011 (top) through 2014-2015 (bottom).</p><fig id="pone.0229658.g002" orientation="portrait" position="float"><object-id pub-id-type="doi">10.1371/journal.pone.0229658.g002</object-id><label>Fig 2</label><caption><title>Daily counts of confirmed cases of influenza, RSV, parainfluenza, hMPV, and other for outbreak years 2010-2011 through 2014-2015.</title></caption><graphic xlink:href="pone.0229658.g002"></graphic></fig><p>To search for an unmodeled disease we selected records of patients who were negative for all four tested diseases. We used these records to search for outbreaks of ILI other than influenza, RSV, parainfluenza, or hMPV. <italic>That is, outbreaks of unknown or unmodeled diseases.</italic> There were 3, 487 such records in outbreak year 2010-2011, 3, 666 in outbreak year 2011-2012, 4, 501 in outbreak year 2012-2013, 4, 752 in outbreak year 2013-2014, and 5, 884 in outbreak year 2014-2015. (These were all the cases for which laboratory testing was done and all four of influenza, RSV, parainfluenza, and hMPV were negative.) Thus, we had five outbreak years of data in which to search for outbreaks of unmodeled diseases.</p><p>To further test DUDE, we <italic>pretended</italic> to be ignorant of each disease (influenza, RSV, parainfluenza, and hMPV) for each year. That is, for each disease and each year we change the positive tests for that disease to negative and create a new dataset that includes a disease that is ‘unmodeled’ to the sytem but known to us. Thus, we created twenty datasets (four diseases times five years) for experimental purposes that include a well-understood disease that is virtually unmodeled.</p></sec><sec id="sec006"><title>Ethics statement</title><p>The research protocol was approved by both institutional IRBs (University of Pittsburgh PRO08030129 and Intermountain Healthcare 1024664). All the research patient data were de-identified.</p><p>We cannot directly share the data used in this study because it contains potentially identifying information about individual patients. This is an ethical and legal restriction that has been imposed by Intermountain Healthcare. Interested parties may request access to the data from the Intermountain Healthcare IRB, 8th Avenue & C Street, Salt Lake City, UT 84143, phone (801) 408-1991, email <email>IRB@imail.org</email>.</p></sec></sec><sec sec-type="results" id="sec007"><title>Results</title><sec id="sec008"><title>Detecting an unmodeled ILI</title><p>We ran the algorithm on each outbreak year using only patients who tested negative for all four of influenza, RSV, parainfluenza, and hMPV. Because they were tested, we assume they have an ILI. However, since all of their tests were negative, their diagnoses were indeterminate. That is, they have some kind of ILI but do not have any of the modeled diseases.</p><p><xref ref-type="fig" rid="pone.0229658.g003">Fig 3</xref> shows the (logarithm of the) daily odds of the presence of an unmodeled disease in the monitor window for outbreak year 2014-2015. DUDE begins computing odds on day 93 (September 1). The odds of the presence of an unmodeled disease slowly increased and was greater than 1 on day 106 (September 14, 2014) indicating that it was more likely than not that an unmodeled disease was present. After day 106 the odds of the presence of an unmodeled disease increased dramatically. An examination of records in the monitor window at that time showed a prevalence of patients with wheezing, chest wall retractions, runny nose, respiratory distress, crackles, tachypnea, abnormal breath sounds, headache, stuffy nose, and dyspnea. (These are the findings that were at least 25% more likely to occur in a patient in the monitor window than one in the baseline window and were present in at least 10% of the patients in the monitor window).</p><fig id="pone.0229658.g003" orientation="portrait" position="float"><object-id pub-id-type="doi">10.1371/journal.pone.0229658.g003</object-id><label>Fig 3</label><caption><title>Number of unexplained ILI cases and log-odds that an unmodeled ILI is present for outbreak year 2014-2015.</title></caption><graphic xlink:href="pone.0229658.g003"></graphic></fig><p>During this time period, the CDC identified an outbreak of <italic>Enterovirus D68</italic> (EV-D68) [<xref rid="pone.0229658.ref017" ref-type="bibr">17</xref>]. In mid-August 2014 hospitals in Missouri and Illinois notified the CDC of an increase in admissions of children with severe respiratory illness. By September 8, 2014 officials at Primary Children’s hospital in Salt Lake City, Utah suspected the presence of EV-D68 [<xref rid="pone.0229658.ref018" ref-type="bibr">18</xref>], and by September 23, 2014 the CDC confirmed the existence of EV-D68 in Utah [<xref rid="pone.0229658.ref019" ref-type="bibr">19</xref>]. Since August 2014, the CDC and states began doing more testing for EV-D68, and have found that EV-D68 was causing severe respiratory illness in almost all states [<xref rid="pone.0229658.ref020" ref-type="bibr">20</xref>]. Symptoms of EV-D68 include wheezing, difficulty breathing, runny nose, sneezing, cough, body aches, and muscle aches. (Severe symptoms of EV-D68 may also include acute flaccid paralysis [<xref rid="pone.0229658.ref021" ref-type="bibr">21</xref>], but this is not among the symptoms used by DUDE).</p><p><xref ref-type="fig" rid="pone.0229658.g004">Fig 4</xref> shows the results of running the algorithm on patients who tested negative for all four of influenza, RSV, parainfluenza, and hMPV patients for outbreak years 2010-2011 (top) through 2014-2015 (bottom).</p><fig id="pone.0229658.g004" orientation="portrait" position="float"><object-id pub-id-type="doi">10.1371/journal.pone.0229658.g004</object-id><label>Fig 4</label><caption><title>Number of unexplained ILI cases and log-odds that an unmodeled ILI is present for outbreak years 2010-2011 through 2014-2015.</title></caption><graphic xlink:href="pone.0229658.g004"></graphic></fig></sec><sec id="sec009"><title>Accuracy</title><p>An outbreak detection system must detect outbreaks in a timely fashion with a minimum of false alarms. Clearly, there is a tradeoff between these two requirements. We can balance these requirements by setting an <italic>alarm threshold</italic>. Each day the likelihood of the presence of an unmodeled disease is greater than this threshold an <italic>alarm</italic> is generated. Ideally, the first alarm is generated on the first day of an outbreak (and each subsequent day while the outbreak is ongoing). Given data for an outbreak year that contains an outbreak of an unmodeled disease, and a specific value for the alarm threshold, the <italic>number of false alarms</italic> is the number of alarms before the start of the outbreak. The <italic>number of days to detection</italic> is the number of days from the start of the outbreak to the first alarm. If the first (false) alarm is before the start of the outbreak, we define the number of days to detection to be zero.</p><p>An <italic>activity monitoring operator characteristic</italic> (AMOC) curve is a graph that characterizes the timeliness of a detector [<xref rid="pone.0229658.ref004" ref-type="bibr">4</xref>]. It plots the expected time to detection as a function of the false-positive rate. AMOC curves can be used to compare the timeliness of different detectors. We can illustrate the performance of DUDE with an AMOC curve that plots the number of days to detection versus the number of false alarms for various threshold values. However, to generate an AMOC curve we need to define the start of an outbreak. It is also helpful to average over several outbreaks to measure performance under various circumstances. Since this is not practical to do with unmodeled outbreaks, we used data for outbreaks of known diseases and <italic>pretended</italic> to be ignorant of them, thus making them unmodeled.</p><p>There were outbreaks of influenza, RSV, and hMPV in each of the five outbreak years of data. Thus, we had fifteen outbreaks of known diseases. For each disease <italic>D</italic> and each year, we created a dataset that included the records of patients who tested negative for all of the tested diseases in that year, plus the records of patients who tested positive for disease <italic>D</italic> in that year. <italic>Again, we pretended to be ignorant of disease D and call it an assumed unmodeled disease.</italic> To calculate the start date of a modeled outbreak we measure the baseline frequency of positive laboratory tests from June 1 (the first day of data) through September 30 (the last day that an outbreak is unlikely to occur). Then for dates after September 30 we signal the start of an outbreak when the frequency increases significantly for several consequtive days. Specifically:</p><list list-type="order"><list-item><p>Set the baseline <italic>b</italic> = 122, window size <italic>w</italic> = 7, notify period <italic>n</italic> = 7, and threshold <italic>t</italic> = 0.1.</p></list-item><list-item><p>Find the average number of lab-confirmed cases for each window in the baseline period from day 1 (June 1) through day <italic>b</italic> (September 30 = 122). This defines a Poisson distribution.</p></list-item><list-item><p>For each window ending on some day after <italic>b</italic> count the number of lab-confirmed cases in that window and find its probability.</p></list-item><list-item><p>If the probability of the number of cases is below the threshold for <italic>n</italic> consequtive windows claim that an outbreak started on the first day of the first of the <italic>n</italic> windows.</p></list-item></list><p>We then ran DUDE on each of these fifteen datasets with various threshold values. For each threshold, we found the number of false alarms and the number of days to detection for each of the fifteen outbreaks. Then, for each threshold, we averaged the number of false alarms and the number of days to detection over the set of fifteen outbreaks. Thus, each threshold value produced a single point on the AMOC curve shown in <xref ref-type="fig" rid="pone.0229658.g005">Fig 5</xref>.</p><fig id="pone.0229658.g005" orientation="portrait" position="float"><object-id pub-id-type="doi">10.1371/journal.pone.0229658.g005</object-id><label>Fig 5</label><caption><title>Average number of days to detection versus average number of false alarms for assumed unmodeled diseases.</title></caption><graphic xlink:href="pone.0229658.g005"></graphic></fig></sec><sec id="sec010"><title>Heuristic case selection</title><p>The AMOC curve in <xref ref-type="fig" rid="pone.0229658.g005">Fig 5</xref> depends on assumptions we used to calculate the start dates of outbreaks. Nonetheless, such curves (when generated using identical assumptions) can be used to compare different techniques. In this section, we use AMOC curves to compare the basic algorithm described above to the basic algorithm enhanced with a heuristic.</p><p>An outbreak of an unmodeled disease is signaled by the presence of atypical patients in the monitor window. Because this is done by statistically comparing the features of the patients in the monitor window to the features of the patients in the baseline window, the presence of only a few atypical patients in the monitor window might not cause an alarm. This suggests that we can increase the effectiveness of DUDE by concentrating atypical patients in the monitor window and using them for detection. We have developed a heuristic method to do this. First, we compute the probability of each finding using all of the patients in the monitor window. Then we select patients in the monitor window whose findings are least likely given those probabilities. These selected patient cases comprise the new monitor window. Then, for comparison, we do the same with the patients in the baseline window to create a new baseline window comprised of the least likely patients.</p><p>We applied this approach to the assumed unmodeled outbreaks for the five years of data by selecting the 30% most atypical patients. <xref ref-type="fig" rid="pone.0229658.g006">Fig 6</xref> compares doing so to using all patients. Note that the AMOC graph corresponding to the 30% heuristically selected patients is lower at nearly every point than the AMOC curve without heuristic patient selection. This means that given thresholds for each approach that led to a particular number of false alarms, the heuristic approach will almost always have fewer days to detection. (We also applied this heuristic using the 10%, 50%, 70%, and 90% most atypical patients and obtained similar, but less, improvement). Although this heuristic worked well with the assumed unmodeled outbreaks (influenza, RSV, parainfluenza, and hMPV) for the five years of data we used, it failed to identify the unmodeled outbreak near day 106 of 2014-2015 that was identified by DUDE when it used all patient records. Although the likelihood of the presence of an unmodeled outbreak did increase slightly it never exceeded 1 indicating that the presence of an unmodeled disease was likely.</p><fig id="pone.0229658.g006" orientation="portrait" position="float"><object-id pub-id-type="doi">10.1371/journal.pone.0229658.g006</object-id><label>Fig 6</label><caption><title>Average number of days to detection versus average number of false alarms for assumed unmodeled diseases with heuristic case selection using only the 30% most atypical patient records.</title></caption><graphic xlink:href="pone.0229658.g006"></graphic></fig></sec></sec><sec sec-type="conclusions" id="sec011"><title>Discussion</title><p>We conclude by noting three limitations of our approach and discuss how DUDE can be extended to overcome them.</p><p>Cases of a new, unmodeled disease are different from other cases that came before. The new disease might differ in an obvious way, such as displaying a rare symptom, or more subtly, such as in new combinations or rates of common symptoms. By design, DUDE identifies sets of cases with rates of symptoms that are statistically different from a baseline of earlier cases. However, it does this with a limited set of medical findings. While this highlights DUDE’s ability to identify outbreaks of diseases such as EV-D68 merely from common symptoms, it potentially misses diseases that are characterized by the presence of unusual symptoms (such as paralysis which is associated with EV-D68 [<xref rid="pone.0229658.ref022" ref-type="bibr">22</xref>]). To be more effective, DUDE will need access to a much wider range of symptoms than it currently uses. Of course, by using a large set of findings we risk overfitting, so this must be done in a probabilistically sound way.</p><p>Outbreaks of new (or old) diseases typically display an initial exponential rate of growth [<xref rid="pone.0229658.ref014" ref-type="bibr">14</xref>]. DUDE compares the <italic>set</italic> of cases in the monitor window with a baseline set of cases and generates an alarm if the presence of an unmodeled disease better explains the difference than simple variation. However, this ignores the daily increase in the number of cases of the unmodeled disease. It would be better if DUDE generated an alarm when it detects an <italic>increasing number of cases of an unmodeled disease</italic>. Doing so would increase the computational complexity since it will need to search over (the parameters of) a set of increasing curves and their start dates, but this can be ameliorated by heuristics that favor increasing disease curves that match increasing numbers of findings.</p><p>Cases of unmodeled diseases are often uncommon, at least initially. Because DUDE is built to identify new, unmodeled diseases it only considers records of patients who have definitively tested negative for all modeled diseases. However, since only a fraction of patients are comprehensively tested, this has the effect of ignoring much of the incoming data. A better approach would be to probabilistically model each of the known diseases and compute the daily expected number of each disease in the entire dataset. We could then compute the expected number of each finding—assuming that no unmodeled disease is present—then find the likelihood of an unmodeled disease based on how the actual distributions of findings differ from what would be expected if no unmodeled disease is present.</p><p>Finally, we note that we do not expect DUDE—or any system designed to detect an outbreak of an unmodeled disease—to work entirely autonomously. The tradoff between timeliness and false alarms dictates that occasional false alarms are inevitable and in times of heightened alertness (for instance, in the presence of anecdotal evidence) it may be prudent to lower the alarm threshold and examine the cases that generate alarms.</p></sec><sec sec-type="conclusions" id="sec012"><title>Conclusions</title><p>We have demonstrated a Bayesian approach to modeling influenza-like illnesses, Detect Unmodeled Diseases from Evidence (DUDE), that is able to identify and characterize new, unmodeled diseases. We have measured its performance when detecting known diseases (while pretending to know nothing about them), and also shown that it is able (retroactively) to identify an outbreak of a new disease in a timely fashion. We have also identified future extensions that may improve its performance.</p></sec><sec sec-type="supplementary-material" id="sec013"><title>Supporting information</title><supplementary-material content-type="local-data" id="pone.0229658.s001"><label>S1 Appendix</label><caption><title>Derivation of Eqs <xref ref-type="disp-formula" rid="pone.0229658.e047">9</xref> and <xref ref-type="disp-formula" rid="pone.0229658.e048">10</xref>.</title><p>(PDF)</p></caption><media xlink:href="pone.0229658.s001.pdf"><caption><p>Click here for additional data file.</p></caption></media></supplementary-material></sec></body><back><ack><p>John Aronis thanks the Department of Biomedical Informatics at the University of Pittsburgh for their continued support.</p></ack><ref-list><title>References</title><ref id="pone.0229658.ref001"><label>1</label><mixed-citation publication-type="journal"><name><surname>Metcalf</surname><given-names>CJE</given-names></name>, <name><surname>Lessler</surname><given-names>J</given-names></name>. <article-title>Opportunities and challenges in modeling emerging infectious diseases</article-title>. <source>Science</source>. <year>2017</year>;<volume>357</volume>:<fpage>149</fpage>–<lpage>152</lpage>. <pub-id pub-id-type="doi">10.1126/science.aam8335</pub-id><pub-id pub-id-type="pmid">28706037</pub-id></mixed-citation></ref><ref id="pone.0229658.ref002"><label>2</label><mixed-citation publication-type="journal"><name><surname>Holmes</surname><given-names>EC</given-names></name>, <name><surname>Rambaut</surname><given-names>A</given-names></name>, <name><surname>Andersen</surname><given-names>KG</given-names></name>. <article-title>Pandemics: spend on surveillance, not prediction</article-title>. <source>Nature</source>. <year>2018</year>;<volume>558</volume>:<fpage>180</fpage>–<lpage>182</lpage>. <pub-id pub-id-type="doi">10.1038/d41586-018-05373-w</pub-id><pub-id pub-id-type="pmid">29880819</pub-id></mixed-citation></ref><ref id="pone.0229658.ref003"><label>3</label><mixed-citation publication-type="book"><name><surname>Dato</surname><given-names>V</given-names></name>, <name><surname>Shephard</surname><given-names>R</given-names></name>, <name><surname>Wagner</surname><given-names>MM</given-names></name>. <chapter-title>Outbreaks and investigations</chapter-title> In: <name><surname>Wagner</surname><given-names>MM</given-names></name>, <name><surname>Moore</surname><given-names>AW</given-names></name>, <name><surname>Aryel</surname><given-names>RM</given-names></name>, editors. <source>Handbook of Biosurveillance</source>. <publisher-name>Elsevier Academic Press</publisher-name>; <year>2006</year> p. <fpage>13</fpage>–<lpage>26</lpage>.</mixed-citation></ref><ref id="pone.0229658.ref004"><label>4</label><mixed-citation publication-type="book"><name><surname>Wagner</surname><given-names>MM</given-names></name>, <name><surname>Gresham</surname><given-names>LS</given-names></name>, <name><surname>Dato</surname><given-names>V</given-names></name>. <chapter-title>Case detection, outbreak detection, and outbreak characterization</chapter-title> In: <name><surname>Wagner</surname><given-names>MM</given-names></name>, <name><surname>Moore</surname><given-names>AW</given-names></name>, <name><surname>Aryel</surname><given-names>RM</given-names></name>, editors. <source>Handbook of Biosurveillance</source>. <publisher-name>Elsevier Academic Press</publisher-name>; <year>2006</year> p. <fpage>27</fpage>–<lpage>50</lpage>.</mixed-citation></ref><ref id="pone.0229658.ref005"><label>5</label><mixed-citation publication-type="book"><name><surname>Velikina</surname><given-names>R</given-names></name>, <name><surname>Dato</surname><given-names>V</given-names></name>, <name><surname>Wagner</surname><given-names>MM</given-names></name>. <chapter-title>Governmental Public Health</chapter-title> In: <name><surname>Wagner</surname><given-names>MM</given-names></name>, <name><surname>Moore</surname><given-names>AW</given-names></name>, <name><surname>Aryel</surname><given-names>RM</given-names></name>, editors. <source>Handbook of Biosurveillance</source>. <publisher-name>Elsevier Academic Press</publisher-name>; <year>2006</year> p. <fpage>67</fpage>–<lpage>88</lpage>.</mixed-citation></ref><ref id="pone.0229658.ref006"><label>6</label><mixed-citation publication-type="book"><name><surname>Wagner</surname><given-names>MM</given-names></name>, <name><surname>Hogan</surname><given-names>WR</given-names></name>, <name><surname>Aryel</surname><given-names>RM</given-names></name>. <chapter-title>The Healthcare System</chapter-title> In: <name><surname>Wagner</surname><given-names>MM</given-names></name>, <name><surname>Moore</surname><given-names>AW</given-names></name>, <name><surname>Aryel</surname><given-names>RM</given-names></name>, editors. <source>Handbook of Biosurveillance</source>. <publisher-name>Elsevier Academic Press</publisher-name>; <year>2006</year> p. <fpage>89</fpage>–<lpage>110</lpage>.</mixed-citation></ref><ref id="pone.0229658.ref007"><label>7</label><mixed-citation publication-type="book"><name><surname>Brokopp</surname><given-names>C</given-names></name>, <name><surname>Resultan</surname><given-names>E</given-names></name>, <name><surname>Holmes</surname><given-names>H</given-names></name>, <name><surname>Wagner</surname><given-names>MM</given-names></name>. <chapter-title>Laboratories</chapter-title> In: <name><surname>Wagner</surname><given-names>MM</given-names></name>, <name><surname>Moore</surname><given-names>AW</given-names></name>, <name><surname>Aryel</surname><given-names>RM</given-names></name>, editors. <source>Handbook of Biosurveillance</source>. <publisher-name>Elsevier Academic Press</publisher-name>; <year>2006</year> p. <fpage>129</fpage>–<lpage>142</lpage>.</mixed-citation></ref><ref id="pone.0229658.ref008"><label>8</label><mixed-citation publication-type="book"><name><surname>Wong</surname><given-names>WK</given-names></name>, <name><surname>Moore</surname><given-names>AW</given-names></name>. <chapter-title>Classical time-series methods for biosurveillance</chapter-title> In: <name><surname>Wagner</surname><given-names>MM</given-names></name>, <name><surname>Moore</surname><given-names>AW</given-names></name>, <name><surname>Aryel</surname><given-names>RM</given-names></name>, editors. <source>Handbook of Biosurveillance</source>. <publisher-name>Elsevier Academic Press</publisher-name>; <year>2006</year> p. <fpage>217</fpage>–<lpage>234</lpage>.</mixed-citation></ref><ref id="pone.0229658.ref009"><label>9</label><mixed-citation publication-type="book"><name><surname>Moore</surname><given-names>AW</given-names></name>, <name><surname>Anderson</surname><given-names>B</given-names></name>, <name><surname>Das</surname><given-names>K</given-names></name>, <name><surname>Wong</surname><given-names>WK</given-names></name>. <chapter-title>Combining multiple signals for biosurveillance</chapter-title> In: <name><surname>Wagner</surname><given-names>MM</given-names></name>, <name><surname>Moore</surname><given-names>AW</given-names></name>, <name><surname>Aryel</surname><given-names>RM</given-names></name>, editors. <source>Handbook of Biosurveillance</source>. <publisher-name>Elsevier Academic Press</publisher-name>; <year>2006</year> p. <fpage>235</fpage>–<lpage>242</lpage>.</mixed-citation></ref><ref id="pone.0229658.ref010"><label>10</label><mixed-citation publication-type="other">Wong WK, Moore AW, Cooper GF, Wagner MM. Bayesian network anomaly pattern detection for disease outbreaks. In: Proceedings of the Twentieth International Conference on Machine Learning. AAAI Press; 2003.</mixed-citation></ref><ref id="pone.0229658.ref011"><label>11</label><mixed-citation publication-type="other">Burkom H, Elbert Y, Piatko C, Fink C. A term-based approach to asyndromic determination of significant case clusters. Online Journal of Public Health Informatics;2015.</mixed-citation></ref><ref id="pone.0229658.ref012"><label>12</label><mixed-citation publication-type="other">Nobles M, Deyneka L, Ising A, Neill DB. Identifying emerging novel outbreaks in textual emergency department data. Online Journal of Public Health Informatics;2014.</mixed-citation></ref><ref id="pone.0229658.ref013"><label>13</label><mixed-citation publication-type="other">Nobles M, Lall R, Mathes R, Neill DB. Multidimensional semantic scan for pre-syndromic disease surveillance. Online Journal of Public Health Informatics;2019.</mixed-citation></ref><ref id="pone.0229658.ref014"><label>14</label><mixed-citation publication-type="other">García YE, Christen JA, Capistrán MA. A Bayesian outbreak detection method for influenza-like illness. BioMed Research International;2015.</mixed-citation></ref><ref id="pone.0229658.ref015"><label>15</label><mixed-citation publication-type="journal"><name><surname>Cooper</surname><given-names>GF</given-names></name>, <name><surname>Villamarin</surname><given-names>R</given-names></name>, <name><surname>Tsui</surname><given-names>FCR</given-names></name>, <name><surname>Millett</surname><given-names>N</given-names></name>, <name><surname>Espino</surname><given-names>JU</given-names></name>, <name><surname>Wagner</surname><given-names>MM</given-names></name>. <article-title>A Method for Detecting and Characterizing Outbreaks of Infectious Disease from Clinical Reports</article-title>. <source>Journal of Biomedical Informatics</source>. <year>2015</year>;<volume>53</volume>:<fpage>15</fpage>–<lpage>26</lpage>. <pub-id pub-id-type="doi">10.1016/j.jbi.2014.08.011</pub-id><pub-id pub-id-type="pmid">25181466</pub-id></mixed-citation></ref><ref id="pone.0229658.ref016"><label>16</label><mixed-citation publication-type="other">Tsui F, Ye Y, Ruiz V, Cooper GF, Wagner MM. Automated influenza case detection for public health surveillance and clinical diagnosis using dynamic influenza prevalence method. Journal of Public Health. 2017;.</mixed-citation></ref><ref id="pone.0229658.ref017"><label>17</label><mixed-citation publication-type="other">for Disease Control C, Prevention. Non-Polio Enterovirus. <ext-link ext-link-type="uri" xlink:href="http://www.cdc.gov/non-polio-enterovirus">www.cdc.gov/non-polio-enterovirus</ext-link>;2014.</mixed-citation></ref><ref id="pone.0229658.ref018"><label>18</label><mixed-citation publication-type="other">Kewish A. Uncommon respiratory illness may be in Utah. <ext-link ext-link-type="uri" xlink:href="http://www.ksl.com/article/31482832/uncommon-respiratory-illness-may-be-in-423utah">www.ksl.com/article/31482832/uncommon-respiratory-illness-may-be-in-423utah</ext-link>;2014.</mixed-citation></ref><ref id="pone.0229658.ref019"><label>19</label><mixed-citation publication-type="other">Grimmett B. CDC confirms existence of enterovirus D68 in Utah. <ext-link ext-link-type="uri" xlink:href="http://www.kuer.org/post/cdc-confirms-existence-enterovirus-d68-utah">www.kuer.org/post/cdc-confirms-existence-enterovirus-d68-utah</ext-link>;2014.</mixed-citation></ref><ref id="pone.0229658.ref020"><label>20</label><mixed-citation publication-type="other">for Disease Control C, Prevention. 2014: Identifying enterovirus D68 in children with respiratory illness. <ext-link ext-link-type="uri" xlink:href="http://www.cdc.gov/amd/whats-new/enteroviruseshtml">www.cdc.gov/amd/whats-new/enteroviruseshtml</ext-link>;2014.</mixed-citation></ref><ref id="pone.0229658.ref021"><label>21</label><mixed-citation publication-type="journal"><name><surname>Aliabadi</surname><given-names>N</given-names></name>, <name><surname>Messacar</surname><given-names>K</given-names></name>, <name><surname>Pastula</surname><given-names>DM</given-names></name>, <name><surname>Robinson</surname><given-names>CC</given-names></name>, <name><surname>Leshem</surname><given-names>E</given-names></name>, <name><surname>Sejvar</surname><given-names>J</given-names></name>, <etal>et al</etal><article-title>Enterovirus D68 infection in children with acute flaccid myelitis</article-title>. <source>Emerging Infectious Diseases</source>. <year>2016</year>;<volume>22</volume>(<issue>8</issue>):<fpage>1387</fpage>–<lpage>1394</lpage>. <pub-id pub-id-type="doi">10.3201/eid2208.151949</pub-id><pub-id pub-id-type="pmid">27434186</pub-id></mixed-citation></ref><ref id="pone.0229658.ref022"><label>22</label><mixed-citation publication-type="other">Uprety P, Curtis D, Elkan M, Fink J, Rajagopalan R, Zhao C, et al. Association of enterovirus D68 with acute flaccid myelitis. Emerging Infectious Diseases;2019.</mixed-citation></ref></ref-list></back></pmc></record>Pour manipuler ce document sous Unix (Dilib)
EXPLOR_STEP=$WICRI_ROOT/Sante/explor/MersV1/Data/Pmc/Corpus
HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 001055 | SxmlIndent | more
Ou
HfdSelect -h $EXPLOR_AREA/Data/Pmc/Corpus/biblio.hfd -nk 001055 | SxmlIndent | more
Pour mettre un lien sur cette page dans le réseau Wicri
{{Explor lien
|wiki= Sante
|area= MersV1
|flux= Pmc
|étape= Corpus
|type= RBID
|clé= PMC:7048291
|texte= A Bayesian approach for detecting a disease that is not being modeled
}}
Pour générer des pages wiki
HfdIndexSelect -h $EXPLOR_AREA/Data/Pmc/Corpus/RBID.i -Sk "pubmed:32109254" \
| HfdSelect -Kh $EXPLOR_AREA/Data/Pmc/Corpus/biblio.hfd \
| NlmPubMed2Wicri -a MersV1
| This area was generated with Dilib version V0.6.33. |  |