Links to Exploration step
Le document en format XML
<record><TEI><teiHeader><fileDesc><titleStmt><title xml:lang="en">An evaluation of emergency guidelines issued by the World Health Organization in response to four infectious disease outbreaks</title><author><name sortKey="Norris, Susan L" sort="Norris, Susan L" uniqKey="Norris S" first="Susan L." last="Norris">Susan L. Norris</name><affiliation><nlm:aff id="aff001"><addr-line>WHO Guidelines Review Committee, World Health Organization, Geneva, Switzerland</addr-line></nlm:aff></affiliation></author><author><name sortKey="Sawin, Veronica Ivey" sort="Sawin, Veronica Ivey" uniqKey="Sawin V" first="Veronica Ivey" last="Sawin">Veronica Ivey Sawin</name><affiliation><nlm:aff id="aff001"><addr-line>WHO Guidelines Review Committee, World Health Organization, Geneva, Switzerland</addr-line></nlm:aff></affiliation></author><author><name sortKey="Ferri, Mauricio" sort="Ferri, Mauricio" uniqKey="Ferri M" first="Mauricio" last="Ferri">Mauricio Ferri</name><affiliation><nlm:aff id="aff001"><addr-line>WHO Guidelines Review Committee, World Health Organization, Geneva, Switzerland</addr-line></nlm:aff></affiliation></author><author><name sortKey="Raques Sastre, Laura" sort="Raques Sastre, Laura" uniqKey="Raques Sastre L" first="Laura" last="Raques Sastre">Laura Raques Sastre</name><affiliation><nlm:aff id="aff002"><addr-line>National School of Public Health, Madrid, Spain</addr-line></nlm:aff></affiliation></author><author><name sortKey="Porgo, Teegwende V" sort="Porgo, Teegwende V" uniqKey="Porgo T" first="Teegwendé V." last="Porgo">Teegwendé V. Porgo</name><affiliation><nlm:aff id="aff003"><addr-line>Department of Social and Preventative Medicine, Laval University, Quebec City, Canada</addr-line></nlm:aff></affiliation></author></titleStmt><publicationStmt><idno type="wicri:source">PMC</idno><idno type="pmid">29847593</idno><idno type="pmc">5976182</idno><idno type="url">http://www.ncbi.nlm.nih.gov/pmc/articles/PMC5976182</idno><idno type="RBID">PMC:5976182</idno><idno type="doi">10.1371/journal.pone.0198125</idno><date when="2018">2018</date><idno type="wicri:Area/Pmc/Corpus">001044</idno><idno type="wicri:explorRef" wicri:stream="Pmc" wicri:step="Corpus" wicri:corpus="PMC">001044</idno></publicationStmt><sourceDesc><biblStruct><analytic><title xml:lang="en" level="a" type="main">An evaluation of emergency guidelines issued by the World Health Organization in response to four infectious disease outbreaks</title><author><name sortKey="Norris, Susan L" sort="Norris, Susan L" uniqKey="Norris S" first="Susan L." last="Norris">Susan L. Norris</name><affiliation><nlm:aff id="aff001"><addr-line>WHO Guidelines Review Committee, World Health Organization, Geneva, Switzerland</addr-line></nlm:aff></affiliation></author><author><name sortKey="Sawin, Veronica Ivey" sort="Sawin, Veronica Ivey" uniqKey="Sawin V" first="Veronica Ivey" last="Sawin">Veronica Ivey Sawin</name><affiliation><nlm:aff id="aff001"><addr-line>WHO Guidelines Review Committee, World Health Organization, Geneva, Switzerland</addr-line></nlm:aff></affiliation></author><author><name sortKey="Ferri, Mauricio" sort="Ferri, Mauricio" uniqKey="Ferri M" first="Mauricio" last="Ferri">Mauricio Ferri</name><affiliation><nlm:aff id="aff001"><addr-line>WHO Guidelines Review Committee, World Health Organization, Geneva, Switzerland</addr-line></nlm:aff></affiliation></author><author><name sortKey="Raques Sastre, Laura" sort="Raques Sastre, Laura" uniqKey="Raques Sastre L" first="Laura" last="Raques Sastre">Laura Raques Sastre</name><affiliation><nlm:aff id="aff002"><addr-line>National School of Public Health, Madrid, Spain</addr-line></nlm:aff></affiliation></author><author><name sortKey="Porgo, Teegwende V" sort="Porgo, Teegwende V" uniqKey="Porgo T" first="Teegwendé V." last="Porgo">Teegwendé V. Porgo</name><affiliation><nlm:aff id="aff003"><addr-line>Department of Social and Preventative Medicine, Laval University, Quebec City, Canada</addr-line></nlm:aff></affiliation></author></analytic><series><title level="j">PLoS ONE</title><idno type="eISSN">1932-6203</idno><imprint><date when="2018">2018</date></imprint></series></biblStruct></sourceDesc></fileDesc><profileDesc><textClass></textClass></profileDesc></teiHeader><front><div type="abstract" xml:lang="en"><sec id="sec001"><title>Background</title><p>The production of high-quality guidelines in response to public health emergencies poses challenges for the World Health Organization (WHO). The urgent need for guidance and the paucity of structured scientific data on emerging diseases hinder the formulation of evidence-informed recommendations using standard methods and procedures.</p></sec><sec id="sec002"><title>Objectives</title><p>In the context of the response to recent public health emergencies, this project aimed to describe the information products produced by WHO and assess the quality and trustworthiness of a subset of these products classified as guidelines.</p></sec><sec id="sec003"><title>Methods</title><p>We selected four recent infectious disease emergencies: outbreaks of avian influenza A—H1N1 virus (2009) and H7N9 virus (2013), Middle East respiratory syndrome coronavirus (MERS-CoV) (2013), and Ebola virus disease (EVD) (2014 to 2016). We analyzed the development and publication processes and evaluated the quality of emergency guidelines using AGREE-II.</p></sec><sec id="sec004"><title>Results</title><p>We included 175 information products of which 87 were guidelines. These products demonstrated variable adherence to WHO publication requirements including the listing of external contributors, management of declarations of interest, and entry into WHO’s public database of publications. For guidelines, the methods for development were incompletely reported; WHO’s quality assurance process was rarely used; systematic or other evidence reviews were infrequently referenced; external peer review was not performed; and they scored poorly with AGREE II, particularly for rigour of development and editorial independence.</p></sec><sec id="sec005"><title>Conclusions</title><p>Our study suggests that WHO guidelines produced in the context of a public health emergency can be improved upon, helping to assure the trustworthiness and utility of WHO information products in future emergencies.</p></sec></div></front><back><div1 type="bibliography"><listBibl><biblStruct></biblStruct><biblStruct></biblStruct><biblStruct></biblStruct><biblStruct></biblStruct><biblStruct></biblStruct><biblStruct></biblStruct><biblStruct></biblStruct><biblStruct></biblStruct><biblStruct></biblStruct><biblStruct><analytic><author><name sortKey="Brouwers, Mc" uniqKey="Brouwers M">MC Brouwers</name></author><author><name sortKey="Kho, Me" uniqKey="Kho M">ME Kho</name></author><author><name sortKey="Browman, Gp" uniqKey="Browman G">GP Browman</name></author><author><name sortKey="Burgers, Js" uniqKey="Burgers J">JS Burgers</name></author><author><name sortKey="Cluzear, F" uniqKey="Cluzear F">F Cluzear</name></author><author><name sortKey="Feder, G" uniqKey="Feder G">G Feder</name></author></analytic></biblStruct><biblStruct></biblStruct><biblStruct><analytic><author><name sortKey="Norris, Sl" uniqKey="Norris S">SL Norris</name></author><author><name sortKey="Holmer, Hk" uniqKey="Holmer H">HK Holmer</name></author><author><name sortKey="Ogden, La" uniqKey="Ogden L">LA Ogden</name></author><author><name sortKey="Burda, Bu" uniqKey="Burda B">BU Burda</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Eikermann, M" uniqKey="Eikermann M">M Eikermann</name></author><author><name sortKey="Holzmann, N" uniqKey="Holzmann N">N Holzmann</name></author><author><name sortKey="Siering, U" uniqKey="Siering U">U Siering</name></author><author><name sortKey="Ruther, A" uniqKey="Ruther A">A Rüther</name></author></analytic></biblStruct></listBibl></div1></back></TEI><pmc article-type="research-article"><pmc-dir>properties open_access</pmc-dir>
<front><journal-meta><journal-id journal-id-type="nlm-ta">PLoS One</journal-id><journal-id journal-id-type="iso-abbrev">PLoS ONE</journal-id><journal-id journal-id-type="publisher-id">plos</journal-id><journal-id journal-id-type="pmc">plosone</journal-id><journal-title-group><journal-title>PLoS ONE</journal-title></journal-title-group><issn pub-type="epub">1932-6203</issn><publisher><publisher-name>Public Library of Science</publisher-name><publisher-loc>San Francisco, CA USA</publisher-loc></publisher></journal-meta><article-meta><article-id pub-id-type="pmid">29847593</article-id><article-id pub-id-type="pmc">5976182</article-id><article-id pub-id-type="doi">10.1371/journal.pone.0198125</article-id><article-id pub-id-type="publisher-id">PONE-D-17-39172</article-id><article-categories><subj-group subj-group-type="heading"><subject>Research Article</subject></subj-group><subj-group subj-group-type="Discipline-v3"><subject>Medicine and Health Sciences</subject><subj-group><subject>Critical Care and Emergency Medicine</subject></subj-group></subj-group><subj-group subj-group-type="Discipline-v3"><subject>Medicine and Health Sciences</subject><subj-group><subject>Public and Occupational Health</subject></subj-group></subj-group><subj-group subj-group-type="Discipline-v3"><subject>Biology and life sciences</subject><subj-group><subject>Organisms</subject><subj-group><subject>Viruses</subject><subj-group><subject>RNA viruses</subject><subj-group><subject>Coronaviruses</subject></subj-group></subj-group></subj-group></subj-group></subj-group><subj-group subj-group-type="Discipline-v3"><subject>Biology and Life Sciences</subject><subj-group><subject>Microbiology</subject><subj-group><subject>Medical Microbiology</subject><subj-group><subject>Microbial Pathogens</subject><subj-group><subject>Viral Pathogens</subject><subj-group><subject>Coronaviruses</subject></subj-group></subj-group></subj-group></subj-group></subj-group></subj-group><subj-group subj-group-type="Discipline-v3"><subject>Medicine and Health Sciences</subject><subj-group><subject>Pathology and Laboratory Medicine</subject><subj-group><subject>Pathogens</subject><subj-group><subject>Microbial Pathogens</subject><subj-group><subject>Viral Pathogens</subject><subj-group><subject>Coronaviruses</subject></subj-group></subj-group></subj-group></subj-group></subj-group></subj-group><subj-group subj-group-type="Discipline-v3"><subject>Biology and Life Sciences</subject><subj-group><subject>Organisms</subject><subj-group><subject>Viruses</subject><subj-group><subject>Viral Pathogens</subject><subj-group><subject>Coronaviruses</subject></subj-group></subj-group></subj-group></subj-group></subj-group><subj-group subj-group-type="Discipline-v3"><subject>Medicine and Health Sciences</subject><subj-group><subject>Epidemiology</subject></subj-group></subj-group><subj-group subj-group-type="Discipline-v3"><subject>Biology and life sciences</subject><subj-group><subject>Organisms</subject><subj-group><subject>Viruses</subject><subj-group><subject>RNA viruses</subject><subj-group><subject>Orthomyxoviruses</subject><subj-group><subject>Influenza viruses</subject><subj-group><subject>Influenza A virus</subject><subj-group><subject>H7N9</subject></subj-group></subj-group></subj-group></subj-group></subj-group></subj-group></subj-group></subj-group><subj-group subj-group-type="Discipline-v3"><subject>Biology and life sciences</subject><subj-group><subject>Microbiology</subject><subj-group><subject>Medical microbiology</subject><subj-group><subject>Microbial pathogens</subject><subj-group><subject>Viral pathogens</subject><subj-group><subject>Orthomyxoviruses</subject><subj-group><subject>Influenza viruses</subject><subj-group><subject>Influenza A virus</subject><subj-group><subject>H7N9</subject></subj-group></subj-group></subj-group></subj-group></subj-group></subj-group></subj-group></subj-group></subj-group><subj-group subj-group-type="Discipline-v3"><subject>Medicine and health sciences</subject><subj-group><subject>Pathology and laboratory medicine</subject><subj-group><subject>Pathogens</subject><subj-group><subject>Microbial pathogens</subject><subj-group><subject>Viral pathogens</subject><subj-group><subject>Orthomyxoviruses</subject><subj-group><subject>Influenza viruses</subject><subj-group><subject>Influenza A virus</subject><subj-group><subject>H7N9</subject></subj-group></subj-group></subj-group></subj-group></subj-group></subj-group></subj-group></subj-group></subj-group><subj-group subj-group-type="Discipline-v3"><subject>Biology and life sciences</subject><subj-group><subject>Organisms</subject><subj-group><subject>Viruses</subject><subj-group><subject>Viral pathogens</subject><subj-group><subject>Orthomyxoviruses</subject><subj-group><subject>Influenza viruses</subject><subj-group><subject>Influenza A virus</subject><subj-group><subject>H7N9</subject></subj-group></subj-group></subj-group></subj-group></subj-group></subj-group></subj-group></subj-group><subj-group subj-group-type="Discipline-v3"><subject>Medicine and Health Sciences</subject><subj-group><subject>Tropical Diseases</subject><subj-group><subject>Neglected Tropical Diseases</subject><subj-group><subject>Viral Hemorrhagic Fevers</subject><subj-group><subject>Ebola Hemorrhagic Fever</subject></subj-group></subj-group></subj-group></subj-group></subj-group><subj-group subj-group-type="Discipline-v3"><subject>Medicine and Health Sciences</subject><subj-group><subject>Infectious Diseases</subject><subj-group><subject>Viral Diseases</subject><subj-group><subject>Viral Hemorrhagic Fevers</subject><subj-group><subject>Ebola Hemorrhagic Fever</subject></subj-group></subj-group></subj-group></subj-group></subj-group><subj-group subj-group-type="Discipline-v3"><subject>Research and Analysis Methods</subject><subj-group><subject>Research Assessment</subject><subj-group><subject>Systematic Reviews</subject></subj-group></subj-group></subj-group><subj-group subj-group-type="Discipline-v3"><subject>Research and Analysis Methods</subject><subj-group><subject>Research Assessment</subject><subj-group><subject>Peer Review</subject></subj-group></subj-group></subj-group></article-categories><title-group><article-title>An evaluation of emergency guidelines issued by the World Health Organization in response to four infectious disease outbreaks</article-title><alt-title alt-title-type="running-head">Evaluation of WHO emergency guidelines for four infectious disease outbreaks</alt-title></title-group><contrib-group><contrib contrib-type="author"><name><surname>Norris</surname><given-names>Susan L.</given-names></name><role content-type="http://credit.casrai.org/">Conceptualization</role><role content-type="http://credit.casrai.org/">Formal analysis</role><role content-type="http://credit.casrai.org/">Funding acquisition</role><role content-type="http://credit.casrai.org/">Methodology</role><role content-type="http://credit.casrai.org/">Resources</role><role content-type="http://credit.casrai.org/">Supervision</role><role content-type="http://credit.casrai.org/">Writing – review & editing</role><xref ref-type="aff" rid="aff001"><sup>1</sup></xref><xref ref-type="corresp" rid="cor001">*</xref></contrib><contrib contrib-type="author"><name><surname>Sawin</surname><given-names>Veronica Ivey</given-names></name><role content-type="http://credit.casrai.org/">Conceptualization</role><role content-type="http://credit.casrai.org/">Data curation</role><role content-type="http://credit.casrai.org/">Formal analysis</role><role content-type="http://credit.casrai.org/">Investigation</role><role content-type="http://credit.casrai.org/">Methodology</role><role content-type="http://credit.casrai.org/">Project administration</role><role content-type="http://credit.casrai.org/">Writing – original draft</role><role content-type="http://credit.casrai.org/">Writing – review & editing</role><xref ref-type="aff" rid="aff001"><sup>1</sup></xref></contrib><contrib contrib-type="author"><contrib-id authenticated="true" contrib-id-type="orcid">http://orcid.org/0000-0002-7246-9049</contrib-id><name><surname>Ferri</surname><given-names>Mauricio</given-names></name><role content-type="http://credit.casrai.org/">Formal analysis</role><role content-type="http://credit.casrai.org/">Writing – original draft</role><role content-type="http://credit.casrai.org/">Writing – review & editing</role><xref ref-type="aff" rid="aff001"><sup>1</sup></xref></contrib><contrib contrib-type="author"><name><surname>Raques Sastre</surname><given-names>Laura</given-names></name><role content-type="http://credit.casrai.org/">Data curation</role><role content-type="http://credit.casrai.org/">Investigation</role><xref ref-type="aff" rid="aff002"><sup>2</sup></xref></contrib><contrib contrib-type="author"><name><surname>Porgo</surname><given-names>Teegwendé V.</given-names></name><role content-type="http://credit.casrai.org/">Data curation</role><role content-type="http://credit.casrai.org/">Investigation</role><xref ref-type="aff" rid="aff003"><sup>3</sup></xref></contrib></contrib-group><aff id="aff001"><label>1</label><addr-line>WHO Guidelines Review Committee, World Health Organization, Geneva, Switzerland</addr-line></aff><aff id="aff002"><label>2</label><addr-line>National School of Public Health, Madrid, Spain</addr-line></aff><aff id="aff003"><label>3</label><addr-line>Department of Social and Preventative Medicine, Laval University, Quebec City, Canada</addr-line></aff><contrib-group><contrib contrib-type="editor"><name><surname>Andrei</surname><given-names>Graciela</given-names></name><role>Editor</role><xref ref-type="aff" rid="edit1"></xref></contrib></contrib-group><aff id="edit1"><addr-line>Katholieke Universiteit Leuven Rega Institute for Medical Research, BELGIUM</addr-line></aff><author-notes><fn fn-type="COI-statement" id="coi001"><p><bold>Competing Interests: </bold>SLN is an employee of the World Health Organization (WHO). When this study was executed, VIS was a contractor, MF a consultant, and TVP an intern at WHO. SLN and MF were involved in the development of several of the information products assessed in this study. MF's spouse is a current employee of Bristol Myers-Squibb and owns stock as part of her remuneration. The authors have no other interests to declare. This does not alter the authors' adherence to PLOS ONE policies on sharing data and materials.</p></fn><corresp id="cor001">* E-mail: <email>norriss@who.int</email></corresp></author-notes><pub-date pub-type="epub"><day>30</day><month>5</month><year>2018</year></pub-date><pub-date pub-type="collection"><year>2018</year></pub-date><volume>13</volume><issue>5</issue><elocation-id>e0198125</elocation-id><history><date date-type="received"><day>3</day><month>11</month><year>2017</year></date><date date-type="accepted"><day>14</day><month>5</month><year>2018</year></date></history><permissions><copyright-statement>© 2018 Norris et al</copyright-statement><copyright-year>2018</copyright-year><copyright-holder>Norris et al</copyright-holder><license xlink:href="http://creativecommons.org/licenses/by/4.0/"><license-p>This is an open access article distributed under the terms of the <ext-link ext-link-type="uri" xlink:href="http://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution License</ext-link>, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.</license-p></license></permissions><self-uri content-type="pdf" xlink:href="pone.0198125.pdf"></self-uri><abstract><sec id="sec001"><title>Background</title><p>The production of high-quality guidelines in response to public health emergencies poses challenges for the World Health Organization (WHO). The urgent need for guidance and the paucity of structured scientific data on emerging diseases hinder the formulation of evidence-informed recommendations using standard methods and procedures.</p></sec><sec id="sec002"><title>Objectives</title><p>In the context of the response to recent public health emergencies, this project aimed to describe the information products produced by WHO and assess the quality and trustworthiness of a subset of these products classified as guidelines.</p></sec><sec id="sec003"><title>Methods</title><p>We selected four recent infectious disease emergencies: outbreaks of avian influenza A—H1N1 virus (2009) and H7N9 virus (2013), Middle East respiratory syndrome coronavirus (MERS-CoV) (2013), and Ebola virus disease (EVD) (2014 to 2016). We analyzed the development and publication processes and evaluated the quality of emergency guidelines using AGREE-II.</p></sec><sec id="sec004"><title>Results</title><p>We included 175 information products of which 87 were guidelines. These products demonstrated variable adherence to WHO publication requirements including the listing of external contributors, management of declarations of interest, and entry into WHO’s public database of publications. For guidelines, the methods for development were incompletely reported; WHO’s quality assurance process was rarely used; systematic or other evidence reviews were infrequently referenced; external peer review was not performed; and they scored poorly with AGREE II, particularly for rigour of development and editorial independence.</p></sec><sec id="sec005"><title>Conclusions</title><p>Our study suggests that WHO guidelines produced in the context of a public health emergency can be improved upon, helping to assure the trustworthiness and utility of WHO information products in future emergencies.</p></sec></abstract><funding-group><award-group id="award001"><funding-source><institution>TDR</institution></funding-source><principal-award-recipient><name><surname>Norris</surname><given-names>Susan L.</given-names></name></principal-award-recipient></award-group><funding-statement>This work was funded by the UNICEF/UNDP/World Bank/WHO Special Programme for Research and Training in Tropical Diseases (TDR) (the Knowledge Management work plan). The funders had no role in the study conception or design, in the collection, analysis or interpretation of the data, in the writing of this report or in the decision to submit the paper for publication.</funding-statement></funding-group><counts><fig-count count="2"></fig-count><table-count count="2"></table-count><page-count count="10"></page-count></counts><custom-meta-group><custom-meta id="data-availability"><meta-name>Data Availability</meta-name><meta-value>All relevant data are within the paper and its Supporting Information files.</meta-value></custom-meta></custom-meta-group></article-meta><notes><title>Data Availability</title><p>All relevant data are within the paper and its Supporting Information files.</p></notes></front><body><sec sec-type="intro" id="sec006"><title>Introduction</title><p>The World Health Organization (WHO) shapes public health policy for United Nations Member States through the development of technical guidance. More than a decade ago, in response to harsh external criticism of its guidelines, WHO instituted the Guidelines Review Committee (GRC) with the mandate to develop rigorous methods and procedures ensuring that each WHO guideline is evidence-based and meets the highest international standards so that guidelines are credible, trustworthy and relevant to end-users [<xref rid="pone.0198125.ref001" ref-type="bibr">1</xref>]. WHO quality standards dictate that guidelines must address a critical public health problem, use transparent and explicit processes minimizing potential sources of bias such as conflicts of interest, include diverse perspectives in the guideline development group, reflect the current state of the evidence, and provide a clear link between the evidence and recommendations taking into consideration the balance of benefits and harms of interventions and other important considerations [<xref rid="pone.0198125.ref002" ref-type="bibr">2</xref>].</p><p>WHO’s scope of work includes producing guidelines and other information products in the context of public health emergency response such as natural and technological disasters, disease outbreaks, armed conflicts and other humanitarian crises. When relevant guidelines do not exist, either because the situation is new or because current guidelines are not applicable, it is extremely challenging to produce new guidelines. Significant uncertainty in the field leads to urgent need, imposing short development timelines; scientific data are scarce and the collection of new data is hindered; health systems may be poorly functioning or nonexistent; and resources (both money and expertise) for guideline development may be scant. Despite this challenging context, adherence to the principles and standards for high-quality, trustworthy and usable guidelines is essential such that end-users adopting and implementing recommendations in the field can optimally impact population health outcomes.</p><p>The best way to apply traditional quality standards for guidelines to the context of emergencies remains unclear. WHO does not currently have an established approach with defined processes and tools to develop emergency guidelines. Thus, guideline development groups at WHO and in other organizations must improvise and adapt standard methods on an <italic>ad hoc</italic> basis, potentially compromising the quality of the final product. To date there are no comprehensive evaluations of guidelines produced in the context of a public health emergency. The aim of this study is to describe the information products produced by WHO in recent emergencies and to assess the quality and trustworthiness of a subset of these products classified as guidelines. The overarching goal of this evaluation was to inform future processes and methods for producing these types of guidelines at WHO and by other organizations globally.</p></sec><sec sec-type="materials|methods" id="sec007"><title>Methods</title><sec id="sec008"><title>Inclusion criteria for information products</title><p>We examined information products issued by WHO for four recent public health emergencies: outbreaks of avian influenza A—H1N1 virus (2009), avian influenza A—H7N9 virus (2013), Middle East respiratory syndrome coronavirus (MERS-CoV) (2013) and Ebola virus disease (EVD) (2014 to 2016). There were other outbreaks, humanitarian emergencies, and natural disasters between 2009 and 2016 that were graded by the Global Emergency Management Team [<xref rid="pone.0198125.ref003" ref-type="bibr">3</xref>], however we selected these four emergencies as purposeful sample as they all focused on infectious diseases outbreaks, and involved a significant response by WHO including the production of a large number of information products. All occurred after the institution of the GRC so that a standard approach to develop guidelines in the non-emergency context was in place at WHO.</p><p>We included all information products published in any language that were specifically developed or adapted by WHO for the four selected emergencies. We excluded testimonials, media releases, situation reports and meeting reports. Two independent reviewers assessed each document against study inclusion criteria. As per WHO’s definition [<xref rid="pone.0198125.ref002" ref-type="bibr">2</xref>], we classified an information product as a guideline based on the content of that document (i.e., whether it contained a specific recommendation for the end-user), rather than by the process for development. Guidelines included products that contained recommendations and did not reference an underlying, source guideline. These covered a wide range of topics including clinical care [<xref rid="pone.0198125.ref004" ref-type="bibr">4</xref>], water and sanitation [<xref rid="pone.0198125.ref005" ref-type="bibr">5</xref>], and occupational health and safety [<xref rid="pone.0198125.ref006" ref-type="bibr">6</xref>], among many other topics. All other information products were classified as non-guidelines, which included, for example, publications derived from existing guidelines [<xref rid="pone.0198125.ref007" ref-type="bibr">7</xref>], assessments of preparedness in countries[<xref rid="pone.0198125.ref008" ref-type="bibr">8</xref>], and tools for assessment or response [<xref rid="pone.0198125.ref009" ref-type="bibr">9</xref>]. Non-guidelines are not expected to contain detailed methods and other types of information that is expected in a high-quality guideline.</p></sec><sec id="sec009"><title>Identification of included information products</title><p>We identified eligible information products available on 24 April 2016, from lists of technical documents related to each emergency assembled by the WHO Department of Communicable Disease Surveillance and Response. We had difficulty identifying relevant documents and therefore searched multiple potential sources such as WHO's global digital library (Institutional Repository for Information Sharing (IRIS)), hand-searched WHO websites, consulted key WHO staff in technical units and reviewed WHO resource toolkits.</p></sec><sec id="sec010"><title>Data extraction and analysis</title><p>Two independent reviewers extracted information product characteristics and publication format, and identified translations. For guidelines, we extracted data on standard WHO guideline development processes (e.g., based on a systematic review of the evidence, involved external experts in the formulation of recommendations) and had two independent reviewers appraise each guideline using the Appraisal of Guidelines for Research and Evaluation (AGREE) II instrument[<xref rid="pone.0198125.ref010" ref-type="bibr">10</xref>]. This instrument, the most widely-used guideline quality appraisal tool, is comprised of 23 items, six quality domains and a 7-point Likert-like response scale.</p><p>We report the prevalence of key characteristics for information products in each emergency. For continuous data, we report the mean and standard deviation (SD). We summarize AGREE II scores for each domain (scaled to a percentage of the maximum score) across guidelines using the median value and the interquartile range, as scores were not normally distributed and sample sizes were small for some emergencies. All statistical analyses were conducted using Stata 12.1 (StataCorp LP, College Station, Texas, USA).</p></sec></sec><sec sec-type="results" id="sec011"><title>Results</title><p>We identified 175 information products of which 87 were classified as guidelines. <xref ref-type="table" rid="pone.0198125.t001">Table 1</xref> summarizes the characteristics of all information products. <xref ref-type="table" rid="pone.0198125.t002">Table 2</xref> outlines the methods and processes used in the development of guidelines specifically.</p><table-wrap id="pone.0198125.t001" orientation="portrait" position="float"><object-id pub-id-type="doi">10.1371/journal.pone.0198125.t001</object-id><label>Table 1</label><caption><title>Characteristics of information products produced by WHO in four recent public health emergencies.</title></caption><alternatives><graphic id="pone.0198125.t001g" xlink:href="pone.0198125.t001"></graphic><table frame="hsides" rules="groups"><colgroup span="1"><col align="left" valign="middle" span="1"></col><col align="left" valign="middle" span="1"></col><col align="left" valign="middle" span="1"></col><col align="left" valign="middle" span="1"></col><col align="left" valign="middle" span="1"></col><col align="left" valign="middle" span="1"></col><col align="left" valign="middle" span="1"></col><col align="left" valign="middle" span="1"></col><col align="left" valign="middle" span="1"></col></colgroup><thead><tr><th align="left" rowspan="2" colspan="1"><break></break></th><th align="center" colspan="2" rowspan="1">H1N1 (2009)</th><th align="center" colspan="2" rowspan="1">H7N9 (2013)</th><th align="center" colspan="2" rowspan="1">MERS-CoV (2013)</th><th align="center" colspan="2" rowspan="1">EVD (2014–16)</th></tr><tr><th align="left" rowspan="1" colspan="1">Guideline</th><th align="left" rowspan="1" colspan="1">Non-guideline</th><th align="left" rowspan="1" colspan="1">Guideline</th><th align="left" rowspan="1" colspan="1">Non-guideline</th><th align="left" rowspan="1" colspan="1">Guideline</th><th align="left" rowspan="1" colspan="1">Non-guideline</th><th align="left" rowspan="1" colspan="1">Guideline</th><th align="left" rowspan="1" colspan="1">Non-guideline</th></tr></thead><tbody><tr><td align="left" rowspan="1" colspan="1">Number</td><td align="center" rowspan="1" colspan="1">15</td><td align="center" rowspan="1" colspan="1">39</td><td align="center" rowspan="1" colspan="1">5</td><td align="center" rowspan="1" colspan="1">9</td><td align="center" rowspan="1" colspan="1">16</td><td align="center" rowspan="1" colspan="1">7</td><td align="center" rowspan="1" colspan="1">52</td><td align="center" rowspan="1" colspan="1">32</td></tr><tr><td align="left" rowspan="1" colspan="1">Publication year</td><td align="center" rowspan="1" colspan="1">2009: 11<break></break>2010: 3<break></break>2011: 1</td><td align="center" rowspan="1" colspan="1">2009: 36<break></break>2010: 2<break></break>2011: 1</td><td align="center" rowspan="1" colspan="1">2013: 4<break></break>2014: 1<break></break>2015: 0</td><td align="center" rowspan="1" colspan="1">2013: 8<break></break>2014: 0<break></break>2015: 1</td><td align="center" rowspan="1" colspan="1">2013: 2<break></break>2014: 4<break></break>2015: 10</td><td align="center" rowspan="1" colspan="1">2013: 2<break></break>2014: 4<break></break>2015: 0</td><td align="center" rowspan="1" colspan="1">2014: 28<break></break>2015: 18<break></break>2016: 5</td><td align="center" rowspan="1" colspan="1">2014: 15<break></break>2015: 15<break></break>2016: 1</td></tr><tr><td align="left" rowspan="1" colspan="1">Translations available<xref ref-type="table-fn" rid="t001fn002"><sup>a</sup></xref></td><td align="center" rowspan="1" colspan="1">0 (0)</td><td align="center" rowspan="1" colspan="1">0 (0)</td><td align="center" rowspan="1" colspan="1">0 (0)</td><td align="center" rowspan="1" colspan="1">1 (11)</td><td align="center" rowspan="1" colspan="1">11 (73)</td><td align="center" rowspan="1" colspan="1">0 (0)</td><td align="center" rowspan="1" colspan="1">32 (65)</td><td align="center" rowspan="1" colspan="1">10 (32)</td></tr><tr><td align="left" rowspan="1" colspan="1">Downloadable format <xref ref-type="table-fn" rid="t001fn002"><sup>a</sup></xref></td><td align="center" rowspan="1" colspan="1">15 (100)</td><td align="center" rowspan="1" colspan="1">37 (95)</td><td align="center" rowspan="1" colspan="1">5 (100)</td><td align="center" rowspan="1" colspan="1">9 (100)</td><td align="center" rowspan="1" colspan="1">15 (94)</td><td align="center" rowspan="1" colspan="1">6 (86)</td><td align="center" rowspan="1" colspan="1">49 (94)</td><td align="center" rowspan="1" colspan="1">31 (97)</td></tr><tr><td align="left" rowspan="1" colspan="1">Title contains:<break></break>"guideline" or "guidance"<break></break>"interim"</td><td align="center" rowspan="1" colspan="1"><break></break>3 (20)<break></break>2 (13)</td><td align="center" rowspan="1" colspan="1"><break></break>1 (3)<break></break>0 (0)</td><td align="center" rowspan="1" colspan="1"><break></break>1 (20)<break></break>2 (40)</td><td align="center" rowspan="1" colspan="1"><break></break>0 (0)<break></break>0 (0)</td><td align="center" rowspan="1" colspan="1"><break></break>8 (50)<break></break>9 (56)</td><td align="center" rowspan="1" colspan="1"><break></break>0 (0)<break></break>0 (0)</td><td align="center" rowspan="1" colspan="1"><break></break>25 (48)<break></break>19 (37)</td><td align="center" rowspan="1" colspan="1"><break></break>6 (19)<break></break>4 (13)</td></tr><tr><td align="left" rowspan="1" colspan="1">Number of pages, mean (range)</td><td align="center" rowspan="1" colspan="1">13 (2–39)</td><td align="center" rowspan="1" colspan="1">6 (1–72)</td><td align="center" rowspan="1" colspan="1">7 (2–13)</td><td align="center" rowspan="1" colspan="1">11 (1–51)</td><td align="center" rowspan="1" colspan="1">6 (1–22)</td><td align="center" rowspan="1" colspan="1">31 (4–94)</td><td align="center" rowspan="1" colspan="1">24 (1–102)</td><td align="center" rowspan="1" colspan="1">19 (2–49)</td></tr><tr><td align="left" rowspan="1" colspan="1">Involved external individuals</td><td align="center" rowspan="1" colspan="1">8 (53)</td><td align="center" rowspan="1" colspan="1">31 (79)</td><td align="center" rowspan="1" colspan="1">3 (60)</td><td align="center" rowspan="1" colspan="1">7 (78)</td><td align="center" rowspan="1" colspan="1">5 (31)</td><td align="center" rowspan="1" colspan="1">4 (57)</td><td align="center" rowspan="1" colspan="1">27 (52)</td><td align="center" rowspan="1" colspan="1">16 (50)</td></tr><tr><td align="left" rowspan="1" colspan="1">Provided publication date</td><td align="center" rowspan="1" colspan="1">13 (87)</td><td align="center" rowspan="1" colspan="1">27 (69)</td><td align="center" rowspan="1" colspan="1">5 (100)</td><td align="center" rowspan="1" colspan="1">8 (89)</td><td align="center" rowspan="1" colspan="1">13 (81)</td><td align="center" rowspan="1" colspan="1">6 (86)</td><td align="center" rowspan="1" colspan="1">51 (98)</td><td align="center" rowspan="1" colspan="1">31 (97)</td></tr><tr><td align="left" rowspan="1" colspan="1">Included the WHO logo <xref ref-type="table-fn" rid="t001fn002"><sup>a</sup></xref></td><td align="center" rowspan="1" colspan="1">14 (93)</td><td align="center" rowspan="1" colspan="1">34 (92)</td><td align="center" rowspan="1" colspan="1">3 (60)</td><td align="center" rowspan="1" colspan="1">8 (89)</td><td align="center" rowspan="1" colspan="1">15 (100)</td><td align="center" rowspan="1" colspan="1">4 (67)</td><td align="center" rowspan="1" colspan="1">47 (96)</td><td align="center" rowspan="1" colspan="1">28 (90)</td></tr><tr><td align="left" rowspan="1" colspan="1">Provided contact details for the responsible officer or team</td><td align="center" rowspan="1" colspan="1">1 (7)</td><td align="center" rowspan="1" colspan="1">12 (31)</td><td align="center" rowspan="1" colspan="1">1 (20)</td><td align="center" rowspan="1" colspan="1">7 (78)</td><td align="center" rowspan="1" colspan="1">4 (25)</td><td align="center" rowspan="1" colspan="1">4 (57)</td><td align="center" rowspan="1" colspan="1">5 (10)</td><td align="center" rowspan="1" colspan="1">2 (6)</td></tr><tr><td align="left" rowspan="1" colspan="1">Located in IRIS <xref ref-type="table-fn" rid="t001fn002"><sup>a</sup></xref></td><td align="center" rowspan="1" colspan="1">1 (7)</td><td align="center" rowspan="1" colspan="1">0 (0)</td><td align="center" rowspan="1" colspan="1">0 (0)</td><td align="center" rowspan="1" colspan="1">0 (0)</td><td align="center" rowspan="1" colspan="1">8 (53)</td><td align="center" rowspan="1" colspan="1">0 (0)</td><td align="center" rowspan="1" colspan="1">45 (92)</td><td align="center" rowspan="1" colspan="1">25 (81)</td></tr></tbody></table></alternatives><table-wrap-foot><fn id="t001fn001"><p>EVD, Ebola virus disease; IRIS, Institutional Repository of Information Sharing; MERS-CoV, Middle East Respiratory Syndrome Coronavirus; SD, standard deviation; WHO, World Health Organization. Data represent number (%) unless otherwise indicated.</p></fn><fn id="t001fn002"><p><sup>a</sup> Of information products for which a downloadable file was available (H1N1: 15 guidelines, 37 non-guidelines; H7N9: 5 guidelines; 9 non-guidelines; MERS-CoV: 15 guidelines, 6 non-guidelines; EVD: 49 guidelines, 31 non-guidelines).</p></fn></table-wrap-foot></table-wrap><table-wrap id="pone.0198125.t002" orientation="portrait" position="float"><object-id pub-id-type="doi">10.1371/journal.pone.0198125.t002</object-id><label>Table 2</label><caption><title>Guideline development processes in recent public health emergencies.</title></caption><alternatives><graphic id="pone.0198125.t002g" xlink:href="pone.0198125.t002"></graphic><table frame="hsides" rules="groups"><colgroup span="1"><col align="left" valign="middle" span="1"></col><col align="left" valign="middle" span="1"></col><col align="left" valign="middle" span="1"></col><col align="left" valign="middle" span="1"></col><col align="left" valign="middle" span="1"></col></colgroup><thead><tr><th align="left" rowspan="1" colspan="1"></th><th align="center" rowspan="1" colspan="1">H1N1<break></break>n = 15</th><th align="center" rowspan="1" colspan="1">H7N9<break></break>n = 5</th><th align="center" rowspan="1" colspan="1">MERS-CoV<break></break>n = 16</th><th align="center" rowspan="1" colspan="1">EVD<break></break>n = 52</th></tr></thead><tbody><tr><td align="left" rowspan="1" colspan="1">References a systematic or rapid review</td><td align="center" rowspan="1" colspan="1">2 (13)</td><td align="center" rowspan="1" colspan="1">1 (20)</td><td align="center" rowspan="1" colspan="1">0 (0)</td><td align="center" rowspan="1" colspan="1">3 (6)</td></tr><tr><td align="left" rowspan="1" colspan="1">Contains citations to other documents</td><td align="center" rowspan="1" colspan="1">12 (80)</td><td align="center" rowspan="1" colspan="1">4 (80)</td><td align="center" rowspan="1" colspan="1">12 (75)</td><td align="center" rowspan="1" colspan="1">42 (81)</td></tr><tr><td align="left" rowspan="1" colspan="1">Lists external contributors</td><td align="center" rowspan="1" colspan="1">4 (27)</td><td align="center" rowspan="1" colspan="1">1 (20)</td><td align="center" rowspan="1" colspan="1">4 (25)</td><td align="center" rowspan="1" colspan="1">10 (19)</td></tr><tr><td align="left" rowspan="1" colspan="1">Reports collecting declarations of interest</td><td align="center" rowspan="1" colspan="1">3 (20)</td><td align="center" rowspan="1" colspan="1">1 (20)</td><td align="center" rowspan="1" colspan="1">1 (6)</td><td align="center" rowspan="1" colspan="1">4 (8)</td></tr><tr><td align="left" rowspan="1" colspan="1">Peer review conducted</td><td align="center" rowspan="1" colspan="1">2 (13)</td><td align="center" rowspan="1" colspan="1">0 (0)</td><td align="center" rowspan="1" colspan="1">6 (38)</td><td align="center" rowspan="1" colspan="1">5 (10)</td></tr><tr><td align="left" rowspan="1" colspan="1">States the date of expiration or planned update</td><td align="center" rowspan="1" colspan="1">8 (53)</td><td align="center" rowspan="1" colspan="1">0 (0)</td><td align="center" rowspan="1" colspan="1">3 (19)</td><td align="center" rowspan="1" colspan="1">4 (8)</td></tr><tr><td align="left" rowspan="1" colspan="1">States the source of funding</td><td align="center" rowspan="1" colspan="1">0 (0)</td><td align="center" rowspan="1" colspan="1">0 (0)</td><td align="center" rowspan="1" colspan="1">0 (0)</td><td align="center" rowspan="1" colspan="1">5 (10)</td></tr><tr><td align="left" rowspan="1" colspan="1">Approved by the WHO Guidelines Review Committee</td><td align="center" rowspan="1" colspan="1">1 (7)</td><td align="center" rowspan="1" colspan="1">0 (0)</td><td align="center" rowspan="1" colspan="1">0 (0)</td><td align="center" rowspan="1" colspan="1">2 (4)</td></tr></tbody></table></alternatives><table-wrap-foot><fn id="t002fn001"><p>EVD, Ebola virus disease; MERS-CoV, Middle East Respiratory Syndrome Coronavirus; n, number; SD, standard deviation</p></fn><fn id="t002fn002"><p>WHO, World Health Organization.</p></fn><fn id="t002fn003"><p>Data represent number (%) unless otherwise indicated.</p></fn></table-wrap-foot></table-wrap><sec id="sec012"><title>H1N1</title><p>For the H1N1 emergency response, 54 information products met our inclusion criteria: 15 guidelines and 39 other information products [<xref rid="pone.0198125.ref011" ref-type="bibr">11</xref>]. Systematic reviews were cited in only two guidelines (13%) and four other guidelines (27%) referred to a literature search. External experts were involved in 53% of guidelines, but only four named these contributors of which three reported that declaration of interests (DOI) had been obtained. Peer review rarely occurred. Dates of expiration or planned update for the guidelines were provided in eight documents (53%). H1N1-related guidelines received the highest scores in the AGREE II domains of clarity of presentation (median 78%; interquartile range (IQR) 74%-83%) and scope and purpose (median 69%; IQR 63%-85%) (<xref ref-type="fig" rid="pone.0198125.g001">Fig 1</xref>). The lowest scores were assessed in editorial independence (median 0%; IQR 0%-0%), rigour of development (median 13%; IQR 4%-20%), and applicability (median 15%; IQR 9%-19%).</p><fig id="pone.0198125.g001" orientation="portrait" position="float"><object-id pub-id-type="doi">10.1371/journal.pone.0198125.g001</object-id><label>Fig 1</label><caption><title>AGREE II scaled domain scores of guidelines developed for H1N1, H7N9, MERS-CoV and EVD.</title><p>EVD, Ebola virus disease; MERS-CoV, Middle East Respiratory Syndrome Coronavirus. The AGREE II domains are displayed along the horizontal axis. The data points are the median, interquartile range, and minimum and maximum standardized scores based on the scores of the two reviewers for each domain and outbreak.</p></caption><graphic xlink:href="pone.0198125.g001"></graphic></fig></sec><sec id="sec013"><title>H7N9</title><p>For H7N9, we identified 14 emergency information products, five emergency guidelines and nine other information products. All but one document was available only in English. One guideline referenced a systematic review; another cited evidence, but did not indicate whether the evidence review was systematic. Of the three guidelines that reported involving external experts, only one listed those individuals and provided their DOI. None of the guidelines provided a date of expiration or planned update, though one guideline indicated that recommendations would be updated as new information emerged. Based on the AGREE II instrument, H7N9 emergency guidelines scored well in clarity of presentation (median 89%; IQR 86%-89%) and scope and purpose (median 72%; IQR 69%-78%). The lowest-scoring AGREE II domains were rigor of development (median 7%; IQR 6%-25%), applicability (median 19%; IQR 15%-27%) and editorial independence (median 0%; IQR 0%-0%) (<xref ref-type="fig" rid="pone.0198125.g001">Fig 1</xref>).</p></sec><sec id="sec014"><title>MERS-CoV</title><p>Twenty-three information products, including 16 emergency guidelines and seven other information products, met our inclusion criteria for MERS-CoV documents. Eleven guidelines (73%) were available in at least two languages (English and Arabic); we identified only an English version of the remaining information products. None of the MERS-CoV emergency guidelines referenced a systematic review. Although five guidelines (31%) reported that external experts contributed to the recommendations, only four of these listed the names of individual experts and only one reported that DOI had been collected. Three guidelines (19%) provided a specific date for the expiration or update of the guideline. The AGREE II domains that scored the highest across these MERS-CoV-related emergency guidelines were clarity of presentation (median 83%; IQR 72%-92%) and scope and purpose (median 76%; IQR 53%-78%). The lowest scoring domains were editorial independence (median 0%; IQR 0%-0%) and rigour of development (median 5%; IQR 1%-9%) (<xref ref-type="fig" rid="pone.0198125.g001">Fig 1</xref>).</p></sec><sec id="sec015"><title>EVD</title><p>We included 84 EVD information products, including 52 emergency guidelines and 32 other information products. Overall, 42 (50%) information products were available in English and French, including 32 (62%) guidelines and 10 (31%) non-guidelines; the remainder were available only in English. Only two EVD guidelines reported or referenced a systematic review, although a third guideline reported that a rapid review of the evidence had been performed. An additional five guidelines (10%) referenced evidence or indicated that a literature search had been performed, but it was unclear if the review methods were systematic. Four guidelines (8%) reported that recommendations were based entirely on expert opinion. Most guidelines (81%) cited other publications, including other WHO information products. Four guidelines (8%) reported that DOI had been collected from external experts. Four guidelines provided a date for the expiration or update of the guideline; eight additional guidelines indicated that regular updates would be provided as new information became available. EVD guidelines scored the highest in the AGREE II domains of clarity of presentation (median 81%; IQR 69%-86%) and scope and purpose (median 69%; IQR 60%-81%). The lowest scores were assessed in editorial independence (median 0%; IQR 0%-0%) and rigour of development (median 7%; IQR 2%-18%) (<xref ref-type="fig" rid="pone.0198125.g001">Fig 1</xref>).</p><p>Approximately half (51%) were published in 2014 (<xref ref-type="fig" rid="pone.0198125.g002">Fig 2</xref>). Guidelines were published an average of 178 days (SD 200) following the initial emergency grading of EVD in Sierra Leone on 25 July 2014; two documents (4%) were published prior to the assignment of the emergency grade.</p><fig id="pone.0198125.g002" orientation="portrait" position="float"><object-id pub-id-type="doi">10.1371/journal.pone.0198125.g002</object-id><label>Fig 2</label><caption><title>Ebola virus disease information products and timing of the epidemic.</title><p>EVD, Ebola virus disease.</p></caption><graphic xlink:href="pone.0198125.g002"></graphic></fig></sec></sec><sec sec-type="conclusions" id="sec016"><title>Discussion</title><p>Information products issued by WHO during four recent infectious disease epidemics were difficult to find, were often published at long intervals after outbreaks were graded, and demonstrated variable adherence to WHO publication and reporting requirements. Few guidelines met international standards for the development and presentation of trustworthy and usable guidelines, as assessed with tools designed for standard (non-emergency) guidelines.</p><p>The principles for trustworthy guidelines were developed in the context of standard (non-emergency) clinical guidelines where there is a much longer time-period for development, end-users’ needs may be much clearer, there may be much more (high-quality) evidence, and the implementation setting and health systems may be much less fragile and chaotic [<xref rid="pone.0198125.ref001" ref-type="bibr">1</xref>]. Universally agreed-upon standards and methods for the optimal development of public health guidelines in the context of emergencies have not yet been established and thus our assessments using these standards must be interpreted with caution. Nonetheless, the principles of transparency, minimization of risk of bias in the formulation of recommendations, and the presentation of implementable, impactful recommendations inarguably apply to all types of guidelines and recommendations.</p><p>The difficulty we faced to identify and retrieve relevant documents suggests that end-users may have similar or greater difficulties during the emergency response. In addition, incomplete reporting of key information such as the responsible technical unit and publication date may prevent end-users from determining the relevance of a product to their specific context as the epidemic trajectory changes or new information emerges.</p><p>There was no consistent format of the information products that we examined, including within specific emergencies. Information products should vary in their content and appearance to meet the needs of the intended end-users, however WHO products should be readily identifiable with consistent appearance and branding, including the WHO logo. The labelling of information products varied across products of the same or similar types, even within emergencies. In particular, guidelines were referred to by a vast array of terms that did not appear to have a specific meaning. For example, “interim guidance” was relatively frequently used in EVD-related documents, appropriately suggesting a limited shelf-life for these documents, however specific plans for updating were rarely presented.</p><p>We noted incomplete implementation of WHO’s mandatory policy on conflicts of interest of external contributors and the reporting of funding sources for guidelines. This is an important finding as the interests of both author and funder are correlated with the conclusions of systematic reviews and the recommendations in clinical practice guidelines [<xref rid="pone.0198125.ref012" ref-type="bibr">12</xref>]. Although DOI does not in itself minimize the risk of bias associated with those interests, disclosure is widely regarded as the first and essential step in addressing competing interests and their potential effects on the conclusions of an evidence review or the recommendations in a guideline.</p><p>Despite the WHO GRC’s responsibilities, only three guidelines were approved by the quality assurance agency. The specific reasons were unclear, although the pressure to produce guidelines quickly could lead to <italic>ad hoc</italic> and variable processes and procedures. Nonetheless, quality assurance and control processes must still be used for all WHO information products.</p><p>Although incomplete reporting likely impeded our assessment, the rigour of development of the guidelines examined in our study appears to be low, as assessed by AGREE II and infrequent reference to systematic reviews. Compressed timelines in response to urgent needs necessitate abbreviated methods, however an explicit description of the approaches used to develop a guideline is always possible. The information and considerations used to inform each recommendation must be clearly described and when direct evidence is scarce, it is even more important to be transparent about the basis for recommendations. The applicability of indirect evidence must be carefully assessed and the rationale for its inclusion provided (for example, the use of data from related viruses or other infectious agents with similar vectors).</p><p>Draft guidelines rarely underwent peer review by individuals and organizations distinct from the primary contributors to the guideline. This is an important gap in the development process, as peer review is a standard step in guideline development [<xref rid="pone.0198125.ref001" ref-type="bibr">1</xref>,<xref rid="pone.0198125.ref002" ref-type="bibr">2</xref>], and has important benefits including stakeholder engagement, identification of errors and attention to key implementation issues. Peer review can be executed within days and comments can be quickly evaluated with careful planning and ready access to a cohort of experts and stakeholders.</p><p>A clear indication of plans for updating recommendations is a critical issue in the emergency context, as the evidence base and other considerations such as those related to implementation are continually evolving. All information products developed in response to an emergency, particularly those labelled “interim”, should have a “review by” date and sufficient resources must be available to evaluate and update information products as required.</p><p>High scores in the AGREE II domains of clarity of presentation and scope and purpose may be attributable to the operational focus of most emergency guidelines, which typically requires a concise, clear format. Other AGREE II domain scores suggest significant room for improvement. For example, guidelines scored inconsistently in the domain of applicability, which includes criteria such as descriptions of barriers and facilitators to implementation of the recommendations, and provision of or reference to other tools to support implementation.</p><p>Although this evaluation examines a number of important measures, there are many additional questions that were not addressed. We focused on emergency information products for infectious disease outbreaks and it is important to examine information products developed in response to other types of emergencies such as natural disasters and humanitarian crises caused by displacement of populations secondary to conflict or climate change. We did not evaluate dissemination, utilization, perceived usefulness by end-users, implementation and uptake of recommendations, or most importantly, the impact on health outcomes and well-being. Such impact evaluations are needed. Nor did we try to identify gaps in the guidance covered by the available information products.</p></sec><sec id="sec017"><title>Limitations</title><p>Despite our efforts to compile comprehensive lists of information products developed by WHO in response to H1N1, H7N9, MERS-CoV and EVD, we may not have captured all relevant documents. We examined only the latest version available: some information products may have been updates which may have different characteristics than previous versions that were developed in shorter timelines or earlier in the emergency response. For practical reasons, we evaluated only information reported in the documents. We may have misclassified derivative products as guidelines if they failed to reference the underlying guideline.</p><p>Although widely used to assess the quality of reporting and conduct of guidelines, the AGREE II instrument has limitations [<xref rid="pone.0198125.ref013" ref-type="bibr">13</xref>], and it was developed for clinical guidelines and not for public health or emergency guidelines. Thus, the domain scores that we report should be interpreted with caution. The AGREE II tool is also limited in its scope. For example, the domain of applicability examines the inclusion of tools for implementation and discussion of barriers, cost implications, and monitoring of the guideline’s recommendations, but it does not address actual implementation or impact on health. In addition, we had only two assessors for the AGREE II appraisals; although acceptable according to AGREE II guidance, four reviewers would have enhanced the reliability of these assessments [<xref rid="pone.0198125.ref010" ref-type="bibr">10</xref>].</p><p>This study did not intend to compare the quality of guidelines for the four responses with each other or with standard WHO guidelines developed in stable settings. It is challenging to compare guidelines across the four epidemics as each event differed with respect to incidence, mortality rate, rate of change in the burden of disease, geographic spread, prior experience with the disease, and the availability of resources.</p></sec><sec sec-type="conclusions" id="sec018"><title>Conclusion</title><p>WHO guidelines produced in the context of a public health emergency can be improved upon, helping to assure the trustworthiness and utility of WHO information products in future emergencies. The principles of transparency, minimization of the risk of bias, and reliance on research and other evidence over expert opinion, apply to guidelines developed in any setting or context. With careful self-evaluation, thoughtful reflection on lessons learned in these emergencies, and a firm commitment to continuous improvement, WHO will continue to provide timely and high-quality guidance in the context of public health emergencies.</p></sec><sec sec-type="supplementary-material" id="sec019"><title>Supporting information</title><supplementary-material content-type="local-data" id="pone.0198125.s001"><label>S1 Table</label><caption><title>Individual-level AGREE-II scores.</title><p>(XLSX)</p></caption><media xlink:href="pone.0198125.s001.xlsx"><caption><p>Click here for additional data file.</p></caption></media></supplementary-material></sec></body><back><ack><p>The authors gratefully acknowledge the administrative support of Ms Myriam Felber and Dr Martin Mendelson.</p></ack><ref-list><title>References</title><ref id="pone.0198125.ref001"><label>1</label><mixed-citation publication-type="book"><collab>US Insitute of Medicine (IOM)</collab>. <source>Clinical Practice Guidelines We Can Trust</source>. <publisher-loc>Washington (DC)</publisher-loc>: <publisher-name>National Academies Press (US)</publisher-name>; <year>2011</year><comment>doi: <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.17226/13058">10.17226/13058</ext-link></comment></mixed-citation></ref><ref id="pone.0198125.ref002"><label>2</label><mixed-citation publication-type="book"><collab>World Health Organizaton</collab>. <chapter-title>WHO handbook for guideline development</chapter-title>-<edition>2nd edition</edition><publisher-loc>Geneva</publisher-loc>: <publisher-name>World Health Organization</publisher-name>; <year>2014</year> Availble from: <ext-link ext-link-type="uri" xlink:href="http://apps.who.int/iris/bitstream/10665/145714/1/9789241548960_eng.pdf?ua=1">http://apps.who.int/iris/bitstream/10665/145714/1/9789241548960_eng.pdf?ua=1</ext-link> (Accessed 11 April 2018.)</mixed-citation></ref><ref id="pone.0198125.ref003"><label>3</label><mixed-citation publication-type="book"><collab>World Health Organization</collab>. <chapter-title>Emergency response framework (ERF)</chapter-title><publisher-loc>Geneva</publisher-loc>: <publisher-name>World Health Organization</publisher-name>; <year>2013</year>. ISBN 978 92 4 150497 3.</mixed-citation></ref><ref id="pone.0198125.ref004"><label>4</label><mixed-citation publication-type="other">World Health Organization. Facilitation Manual: Psychological First Aid during Ebola Virus Disease Outbreaks; 2014. Available from: <ext-link ext-link-type="uri" xlink:href="http://www.who.int/mental_health/emergencies/psychological_first_aid_ebola_manual/en/">http://www.who.int/mental_health/emergencies/psychological_first_aid_ebola_manual/en/</ext-link> (Accessed 11 April 2018.)</mixed-citation></ref><ref id="pone.0198125.ref005"><label>5</label><mixed-citation publication-type="other">World Health Organization. Ebola virus disease: Key questions and answers concerning water, sanitation and hygiene; 2014. Available from: <ext-link ext-link-type="uri" xlink:href="http://www.who.int/csr/resources/publications/ebola/water-sanitation-hygiene/en/">http://www.who.int/csr/resources/publications/ebola/water-sanitation-hygiene/en/</ext-link> (Accessed 11 April 2018.)</mixed-citation></ref><ref id="pone.0198125.ref006"><label>6</label><mixed-citation publication-type="other">World Health Organization. Ebola virus disease (EVD): occupational safety and health: Joint WHO/ILO briefing note for workers and employers; 2014. Available from: <ext-link ext-link-type="uri" xlink:href="http://apps.who.int/iris/handle/10665/146427">http://apps.who.int/iris/handle/10665/146427</ext-link>.</mixed-citation></ref><ref id="pone.0198125.ref007"><label>7</label><mixed-citation publication-type="other">World Health Organization. Infection prevention and control (IPC) guidance summary: Ebola guidance package; 2014. Available from: <ext-link ext-link-type="uri" xlink:href="http://www.who.int/csr/resources/publications/ebola/evd-guidance-summary/en/">http://www.who.int/csr/resources/publications/ebola/evd-guidance-summary/en/</ext-link>. (Accessed 11 April 2018.)</mixed-citation></ref><ref id="pone.0198125.ref008"><label>8</label><mixed-citation publication-type="other">World Health Organization. Ebola virus disease preparedness strengthening team: Mali country visit 20–24 October 2014. Avabailabe from: <ext-link ext-link-type="uri" xlink:href="http://apps.who.int/iris/handle/10665/144504">http://apps.who.int/iris/handle/10665/144504</ext-link>. (Accessed 11 April 2018.)</mixed-citation></ref><ref id="pone.0198125.ref009"><label>9</label><mixed-citation publication-type="other">World Health Organization. Facility questionnaire for Ebola virus disease/viral haemorrhagic fever diagnosis capacity 2015. Availale from: <ext-link ext-link-type="uri" xlink:href="http://www.who.int/csr/resources/publications/ebola/laboratory/en/">http://www.who.int/csr/resources/publications/ebola/laboratory/en/</ext-link>. (Accessed 11 April 2018.)</mixed-citation></ref><ref id="pone.0198125.ref010"><label>10</label><mixed-citation publication-type="journal"><name><surname>Brouwers</surname><given-names>MC</given-names></name>, <name><surname>Kho</surname><given-names>ME</given-names></name>, <name><surname>Browman</surname><given-names>GP</given-names></name>, <name><surname>Burgers</surname><given-names>JS</given-names></name>, <name><surname>Cluzear</surname><given-names>F</given-names></name>, <name><surname>Feder</surname><given-names>G</given-names></name>, <etal>et al</etal><article-title>AGREE II: advancing guideline development, reporting and evaluation in health care</article-title>. . <year>2010</year>;<volume>182</volume>(<issue>18</issue>):<fpage>E839</fpage>–<lpage>842</lpage>. <comment>doi: <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1503/cmaj.090449">10.1503/cmaj.090449</ext-link></comment><pub-id pub-id-type="pmid">20603348</pub-id></mixed-citation></ref><ref id="pone.0198125.ref011"><label>11</label><mixed-citation publication-type="other">World Health Organization. Pandemic (H1N1) 2009 guidance documents. In: Emergencies preparedness, response [website]. Geneva: 2016. Available from: <ext-link ext-link-type="uri" xlink:href="http://www.who.int/csr/resources/publications/swineflu/en/">http://www.who.int/csr/resources/publications/swineflu/en/</ext-link>. (Accessed 11 April 2018).</mixed-citation></ref><ref id="pone.0198125.ref012"><label>12</label><mixed-citation publication-type="journal"><name><surname>Norris</surname><given-names>SL</given-names></name>, <name><surname>Holmer</surname><given-names>HK</given-names></name>, <name><surname>Ogden</surname><given-names>LA</given-names></name>, <name><surname>Burda</surname><given-names>BU</given-names></name>. <article-title>Conflict of interest in clinical practice guideline development: a systematic review</article-title>. . <year>2011</year>;<volume>6</volume>(<issue>10</issue>):<fpage>e25153</fpage><comment>doi: <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1371/journal.pone.0025153">10.1371/journal.pone.0025153</ext-link></comment><pub-id pub-id-type="pmid">22039406</pub-id></mixed-citation></ref><ref id="pone.0198125.ref013"><label>13</label><mixed-citation publication-type="journal"><name><surname>Eikermann</surname><given-names>M</given-names></name>, <name><surname>Holzmann</surname><given-names>N</given-names></name>, <name><surname>Siering</surname><given-names>U</given-names></name>, <name><surname>Rüther</surname><given-names>A</given-names></name>. <article-title>Tools for assessing the content of guidelines are needed to enable their effective use–a systematic comparison</article-title>. <source>B<italic toggle="yes">MC Research Notes</italic></source>. <year>2014</year>;<volume>7</volume>:<fpage>853</fpage><comment>doi: <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1186/1756-0500-7-853">10.1186/1756-0500-7-853</ext-link></comment><pub-id pub-id-type="pmid">25427972</pub-id></mixed-citation></ref></ref-list></back></pmc></record>Pour manipuler ce document sous Unix (Dilib)
EXPLOR_STEP=$WICRI_ROOT/Sante/explor/MersV1/Data/Pmc/Corpus
HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 001044 | SxmlIndent | more
Ou
HfdSelect -h $EXPLOR_AREA/Data/Pmc/Corpus/biblio.hfd -nk 001044 | SxmlIndent | more
Pour mettre un lien sur cette page dans le réseau Wicri
{{Explor lien
|wiki= Sante
|area= MersV1
|flux= Pmc
|étape= Corpus
|type= RBID
|clé=
|texte=
}}
| This area was generated with Dilib version V0.6.33. |  |