Links to Exploration step
Le document en format XML
<record><TEI><teiHeader><fileDesc><titleStmt><title xml:lang="en">Beyond Open Big Data: Addressing Unreliable Research</title></titleStmt><publicationStmt><idno type="wicri:source">PMC</idno><idno type="pmid">25405277</idno><idno type="pmc">4260008</idno><idno type="url">http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4260008</idno><idno type="RBID">PMC:4260008</idno><idno type="doi">10.2196/jmir.3871</idno><date when="2014">2014</date><idno type="wicri:Area/Pmc/Corpus">000304</idno><idno type="wicri:explorRef" wicri:stream="Pmc" wicri:step="Corpus" wicri:corpus="PMC">000304</idno></publicationStmt><sourceDesc><biblStruct><analytic><title xml:lang="en" level="a" type="main">Beyond Open Big Data: Addressing Unreliable Research</title></analytic><series><title level="j">Journal of Medical Internet Research</title><idno type="ISSN">1439-4456</idno><idno type="eISSN">1438-8871</idno><imprint><date when="2014">2014</date></imprint></series></biblStruct></sourceDesc></fileDesc><profileDesc><textClass></textClass></profileDesc></teiHeader><front><div type="abstract" xml:lang="en"><p>The National Institute of Health invests US $30.9 billion annually in medical research. However, the subsequent impact of this research output on society and the economy is amplified dramatically as a result of the actual medical treatments, biomedical innovations, and various commercial enterprises that emanate from and depend on these findings. It is therefore a great concern to discover that much of published research is unreliable. We propose extending the open data concept to the culture of the scientific research community. By dialing down unproductive features of secrecy and competition, while ramping up cooperation and transparency, we make a case that what is published would then be less susceptible to the sometimes corrupting and confounding pressures to be first or journalistically attractive, which can compromise the more fundamental need to be robustly correct.</p></div></front><back><div1 type="bibliography"><listBibl><biblStruct><analytic><author><name sortKey="Macleod, Mr" uniqKey="Macleod M">MR Macleod</name></author><author><name sortKey="Michie, S" uniqKey="Michie S">S Michie</name></author><author><name sortKey="Roberts, I" uniqKey="Roberts I">I Roberts</name></author><author><name sortKey="Dirnagl, U" uniqKey="Dirnagl U">U Dirnagl</name></author><author><name sortKey="Chalmers, I" uniqKey="Chalmers I">I Chalmers</name></author><author><name sortKey="Ioannidis, Jp" uniqKey="Ioannidis J">JP Ioannidis</name></author><author><name sortKey="Al Shahi Salman, R" uniqKey="Al Shahi Salman R">R Al-Shahi Salman</name></author><author><name sortKey="Chan, Aw" uniqKey="Chan A">AW Chan</name></author><author><name sortKey="Glasziou, P" uniqKey="Glasziou P">P Glasziou</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Ioannidis, Jp" uniqKey="Ioannidis J">JP Ioannidis</name></author><author><name sortKey="Greenland, S" uniqKey="Greenland S">S Greenland</name></author><author><name sortKey="Hlatky, Ma" uniqKey="Hlatky M">MA Hlatky</name></author><author><name sortKey="Khoury, Mj" uniqKey="Khoury M">MJ Khoury</name></author><author><name sortKey="Macleod, Mr" uniqKey="Macleod M">MR Macleod</name></author><author><name sortKey="Moher, D" uniqKey="Moher D">D Moher</name></author><author><name sortKey="Schulz, Kf" uniqKey="Schulz K">KF Schulz</name></author><author><name sortKey="Tibshirani, R" uniqKey="Tibshirani R">R Tibshirani</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Chan, Aw" uniqKey="Chan A">AW Chan</name></author><author><name sortKey="Song, F" uniqKey="Song F">F Song</name></author><author><name sortKey="Vickers, A" uniqKey="Vickers A">A Vickers</name></author><author><name sortKey="Jefferson, T" uniqKey="Jefferson T">T Jefferson</name></author><author><name sortKey="Dickersin, K" uniqKey="Dickersin K">K Dickersin</name></author><author><name sortKey="G Tzsche, Pc" uniqKey="G Tzsche P">PC Gøtzsche</name></author><author><name sortKey="Krumholz, Hm" uniqKey="Krumholz H">HM Krumholz</name></author><author><name sortKey="Ghersi, D" uniqKey="Ghersi D">D Ghersi</name></author><author><name sortKey="Van Der Worp, Hb" uniqKey="Van Der Worp H">HB van der Worp</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Godlee, F" uniqKey="Godlee F">F Godlee</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Peierls, R" uniqKey="Peierls R">R Peierls</name></author></analytic></biblStruct><biblStruct></biblStruct><biblStruct><analytic><author><name sortKey="Doshi, P" uniqKey="Doshi P">P Doshi</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Jefferson, T" uniqKey="Jefferson T">T Jefferson</name></author><author><name sortKey="Jones, M" uniqKey="Jones M">M Jones</name></author><author><name sortKey="Doshi, P" uniqKey="Doshi P">P Doshi</name></author><author><name sortKey="Del Mar, C" uniqKey="Del Mar C">C Del Mar</name></author><author><name sortKey="Dooley, L" uniqKey="Dooley L">L Dooley</name></author><author><name sortKey="Foxlee, R" uniqKey="Foxlee R">R Foxlee</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="El Dib, Rp" uniqKey="El Dib R">RP El Dib</name></author><author><name sortKey="Atallah, An" uniqKey="Atallah A">AN Atallah</name></author><author><name sortKey="Andriolo, Rb" uniqKey="Andriolo R">RB Andriolo</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Celi, La" uniqKey="Celi L">LA Celi</name></author><author><name sortKey="Mark, Rg" uniqKey="Mark R">RG Mark</name></author><author><name sortKey="Stone, Dj" uniqKey="Stone D">DJ Stone</name></author><author><name sortKey="Montgomery, Ra" uniqKey="Montgomery R">RA Montgomery</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Feigelson, Ed" uniqKey="Feigelson E">ED Feigelson</name></author><author><name sortKey="Babu, Gj" uniqKey="Babu G">GJ Babu</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Gibb, Bc" uniqKey="Gibb B">BC Gibb</name></author></analytic></biblStruct><biblStruct></biblStruct><biblStruct><analytic><author><name sortKey="Marx, V" uniqKey="Marx V">V Marx</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Hesla, L" uniqKey="Hesla L">L Hesla</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Califf, Rm" uniqKey="Califf R">RM Califf</name></author><author><name sortKey="Mccall, J" uniqKey="Mccall J">J McCall</name></author><author><name sortKey="Harrington, Ra" uniqKey="Harrington R">RA Harrington</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Christakis, Da" uniqKey="Christakis D">DA Christakis</name></author><author><name sortKey="Zimmerman, Fj" uniqKey="Zimmerman F">FJ Zimmerman</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Nielsen, M" uniqKey="Nielsen M">M Nielsen</name></author></analytic></biblStruct><biblStruct></biblStruct><biblStruct><analytic><author><name sortKey="Wilbanks, J" uniqKey="Wilbanks J">J Wilbanks</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Watson, Jd" uniqKey="Watson J">JD Watson</name></author></analytic></biblStruct><biblStruct></biblStruct><biblStruct></biblStruct><biblStruct></biblStruct><biblStruct></biblStruct><biblStruct></biblStruct><biblStruct></biblStruct><biblStruct><analytic><author><name sortKey="Badawi, O" uniqKey="Badawi O">O Badawi</name></author><author><name sortKey="Brennan, T" uniqKey="Brennan T">T Brennan</name></author><author><name sortKey="Celi, La" uniqKey="Celi L">LA Celi</name></author><author><name sortKey="Feng, M" uniqKey="Feng M">M Feng</name></author><author><name sortKey="Ghassemi, M" uniqKey="Ghassemi M">M Ghassemi</name></author><author><name sortKey="Ippolito, A" uniqKey="Ippolito A">A Ippolito</name></author><author><name sortKey="Johnson, A" uniqKey="Johnson A">A Johnson</name></author><author><name sortKey="Mark, Rg" uniqKey="Mark R">RG Mark</name></author><author><name sortKey="Mayaud, L" uniqKey="Mayaud L">L Mayaud</name></author><author><name sortKey="Moody, G" uniqKey="Moody G">G Moody</name></author><author><name sortKey="Moses, C" uniqKey="Moses C">C Moses</name></author><author><name sortKey="Naumann, T" uniqKey="Naumann T">T Naumann</name></author><author><name sortKey="Pimentel, M" uniqKey="Pimentel M">M Pimentel</name></author><author><name sortKey="Pollard, Tj" uniqKey="Pollard T">TJ Pollard</name></author><author><name sortKey="Santos, M" uniqKey="Santos M">M Santos</name></author><author><name sortKey="Stone, Dj" uniqKey="Stone D">DJ Stone</name></author><author><name sortKey="Zimolzak, A" uniqKey="Zimolzak A">A Zimolzak</name></author></analytic></biblStruct><biblStruct></biblStruct><biblStruct></biblStruct><biblStruct><analytic><author><name sortKey="Celi, La" uniqKey="Celi L">LA Celi</name></author><author><name sortKey="Ippolito, A" uniqKey="Ippolito A">A Ippolito</name></author><author><name sortKey="Montgomery, Ra" uniqKey="Montgomery R">RA Montgomery</name></author><author><name sortKey="Moses, C" uniqKey="Moses C">C Moses</name></author><author><name sortKey="Stone, Dj" uniqKey="Stone D">DJ Stone</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Harrison, C" uniqKey="Harrison C">C Harrison</name></author></analytic></biblStruct><biblStruct></biblStruct><biblStruct></biblStruct><biblStruct><analytic><author><name sortKey="Moses, C" uniqKey="Moses C">C Moses</name></author><author><name sortKey="Celi, La" uniqKey="Celi L">LA Celi</name></author><author><name sortKey="Marshall, J" uniqKey="Marshall J">J Marshall</name></author></analytic></biblStruct><biblStruct></biblStruct><biblStruct></biblStruct><biblStruct></biblStruct><biblStruct></biblStruct><biblStruct></biblStruct><biblStruct></biblStruct></listBibl></div1></back></TEI><pmc article-type="letter"><pmc-dir>properties open_access</pmc-dir>
<front><journal-meta><journal-id journal-id-type="nlm-ta">J Med Internet Res</journal-id><journal-id journal-id-type="iso-abbrev">J. Med. Internet Res</journal-id><journal-id journal-id-type="publisher-id">JMIR</journal-id><journal-title-group><journal-title>Journal of Medical Internet Research</journal-title></journal-title-group><issn pub-type="ppub">1439-4456</issn><issn pub-type="epub">1438-8871</issn><publisher><publisher-name>JMIR Publications Inc.</publisher-name><publisher-loc>Toronto, Canada</publisher-loc></publisher></journal-meta><article-meta><article-id pub-id-type="pmid">25405277</article-id><article-id pub-id-type="pmc">4260008</article-id><article-id pub-id-type="publisher-id">v16i11e259</article-id><article-id pub-id-type="doi">10.2196/jmir.3871</article-id><article-categories><subj-group subj-group-type="heading"><subject>Viewpoint</subject></subj-group><subj-group subj-group-type="article-type"><subject>Original Paper</subject></subj-group></article-categories><title-group><article-title>Beyond Open Big Data: Addressing Unreliable Research</article-title></title-group><contrib-group><contrib contrib-type="editor"><name><surname>Eysenbach</surname><given-names>Gunther</given-names></name></contrib></contrib-group><contrib-group><contrib contrib-type="reviewer"><name><surname>Stroetmann</surname><given-names>Karl</given-names></name></contrib><contrib contrib-type="reviewer"><name><surname>Hesse</surname><given-names>Bradford</given-names></name></contrib><contrib contrib-type="reviewer"><name><surname>Domnich</surname><given-names>Alexander</given-names></name></contrib></contrib-group><contrib-group><contrib id="contrib1" contrib-type="author"><name><surname>Moseley</surname><given-names>Edward T</given-names></name><degrees>BS</degrees><xref ref-type="aff" rid="aff1">1</xref><contrib-id contrib-id-type="orcid">http://orcid.org/0000-0003-4334-5618</contrib-id></contrib><contrib id="contrib2" contrib-type="author"><name><surname>Hsu</surname><given-names>Douglas J</given-names></name><degrees>MD</degrees><xref ref-type="aff" rid="aff2">2</xref><contrib-id contrib-id-type="orcid">http://orcid.org/0000-0002-5341-3277</contrib-id></contrib><contrib id="contrib3" contrib-type="author"><name><surname>Stone</surname><given-names>David J</given-names></name><degrees>MD</degrees><xref ref-type="aff" rid="aff3">3</xref><xref ref-type="aff" rid="aff4">4</xref><contrib-id contrib-id-type="orcid">http://orcid.org/0000-0003-4855-4283</contrib-id></contrib><contrib id="contrib4" contrib-type="author" corresp="yes"><name><surname>Celi</surname><given-names>Leo Anthony</given-names></name><degrees>MPH, MS, MD</degrees><xref ref-type="aff" rid="aff2">2</xref><xref ref-type="aff" rid="aff5">5</xref><address><institution>Institute for Medical Engineering & Science</institution><institution>Massachusetts Institute of Technology</institution><addr-line>77 Massachusetts Avenue</addr-line><addr-line>E25-505</addr-line><addr-line>Cambridge, MA, 02139</addr-line><country>United States</country><phone>1 6172537937</phone><fax>1 6172587859</fax><email>lceli@mit.edu</email></address><contrib-id contrib-id-type="orcid">http://orcid.org/0000-0001-6712-6626</contrib-id></contrib></contrib-group><aff id="aff1"><sup>1</sup><institution>Division of Vaccine Research</institution><institution>Beth Israel Deaconess Medical Center</institution><addr-line>Boston, MA</addr-line><country>United States</country></aff><aff id="aff2"><sup>2</sup><institution>Division of Pulmonary, Critical Care and Sleep Medicine</institution><institution>Department of Medicine</institution><institution>Beth Israel Deaconess Medical Center</institution><addr-line>Boston, MA</addr-line><country>United States</country></aff><aff id="aff3"><sup>3</sup><institution>UVA Center for Wireless Health</institution><institution>Departments of Anesthesiology and Neurological Surgery</institution><institution>University of Virginia</institution><addr-line>Charlottesville, VA</addr-line><country>United States</country></aff><aff id="aff4"><sup>4</sup><institution>Center for Wireless Health</institution><institution>University of Virginia</institution><addr-line>Charlottesville, VA</addr-line><country>United States</country></aff><aff id="aff5"><sup>5</sup><institution>Institute for Medical Engineering & Science</institution><institution>Massachusetts Institute of Technology</institution><addr-line>Cambridge, MA</addr-line><country>United States</country></aff><author-notes><corresp>Corresponding Author: Leo Anthony Celi <email>lceli@mit.edu</email></corresp></author-notes><pub-date pub-type="collection"><month>11</month><year>2014</year></pub-date><pub-date pub-type="epub"><day>11</day><month>11</month><year>2014</year></pub-date><volume>16</volume><issue>11</issue><elocation-id>e259</elocation-id><history><date date-type="received"><day>17</day><month>9</month><year>2014</year></date><date date-type="rev-request"><day>07</day><month>10</month><year>2014</year></date><date date-type="rev-recd"><day>12</day><month>10</month><year>2014</year></date><date date-type="accepted"><day>20</day><month>10</month><year>2014</year></date></history><permissions><copyright-statement>©Edward T Moseley, Douglas J Hsu, David J Stone, Leo Anthony Celi. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 11.11.2014. </copyright-statement><copyright-year>2014</copyright-year><license license-type="open-access" xlink:href="http://creativecommons.org/licenses/by/2.0/"><license-p><pmc-comment>CREATIVE COMMONS</pmc-comment>This is an open-access article distributed under the terms of the Creative Commons Attribution License (<ext-link ext-link-type="uri" xlink:href="http://creativecommons.org/licenses/by/2.0/">http://creativecommons.org/licenses/by/2.0/</ext-link>), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on <ext-link ext-link-type="uri" xlink:href="http://www.jmir.org/,">http://www.jmir.org/,</ext-link> as well as this copyright and license information must be included.</license-p></license></permissions><self-uri xlink:type="simple" xlink:href="http://www.jmir.org/2014/11/e259/"></self-uri><abstract><p>The National Institute of Health invests US $30.9 billion annually in medical research. However, the subsequent impact of this research output on society and the economy is amplified dramatically as a result of the actual medical treatments, biomedical innovations, and various commercial enterprises that emanate from and depend on these findings. It is therefore a great concern to discover that much of published research is unreliable. We propose extending the open data concept to the culture of the scientific research community. By dialing down unproductive features of secrecy and competition, while ramping up cooperation and transparency, we make a case that what is published would then be less susceptible to the sometimes corrupting and confounding pressures to be first or journalistically attractive, which can compromise the more fundamental need to be robustly correct.</p></abstract><kwd-group><kwd>open data</kwd><kwd>unreliable research</kwd><kwd>collaborative learning</kwd><kwd>knowledge discovery</kwd><kwd>peer review</kwd><kwd>research culture</kwd></kwd-group></article-meta></front><body><sec><title>Under the Magnifying Lens: The Reliability of Medical Research</title><disp-quote><p>It ought to be remembered that there is nothing more difficult to take in hand, more perilous to conduct, or more uncertain in its success, than to take the lead in the introduction of a new order of things. Because the innovator has for enemies all those who have done well under the old conditions, and lukewarm defenders in those who may do well under the new.</p><attrib>Niccolo Machiavelli</attrib></disp-quote><p>The reliability of research is coming under increasing scrutiny. Over the past several years, editorials and entire issues of journals, including <italic>British Medical Journal, The Lancet,</italic> and even <italic>The Economist</italic> have highlighted the discordance between the amount of money invested in the biomedical research enterprise with the lack of reliability in published studies [<xref rid="ref1" ref-type="bibr">1</xref>-<xref rid="ref4" ref-type="bibr">4</xref>]. Despite the peer-review process, there is a fundamental problem with the reliability of a disturbing amount of scientific, particularly biomedical, research. The problem is basically two-fold and lies in the unreliability of what does get published and the inability of other researchers to know what is not getting published. False positive research results are successfully published with alarming frequency. Replication is intrinsically difficult for a variety of reasons including limited access to original data, intellectual property issues, the general perception that replication is not doing original (and rewarding) work, and also the possible appearance of being defiant to more senior authority. Of course, work that is never published cannot be examined or replicated. When re-examination does occur, irreproducibility is surprisingly rampant. Furthermore, the peer-review system is less robust than often assumed, and there are enormous career structure-related pressures to publish. It is remarkably easy for well-meaning but perhaps imperfectly objective researchers to be duped by false positive results that are generated by technical laboratory execution issues (including simple inconsistency) and faulty statistical analyses. The latter lies in inadequately powered studies, unlikely hypotheses being tested, lack of proper blinding, and the bias towards reporting and publishing something new. Wolfgang Pauli, the eminent physicist, might have classified this kind of work under his most brutal characterization of sloppy thinking, “It is not only not right, it is not even wrong,” or even better in the original German, “Das ist nicht nur nicht richtig, es ist nicht einmal falsch!” [<xref rid="ref5" ref-type="bibr">5</xref>].</p><p>In the current research reward system, sensational positive findings—some, perhaps even the majority, of dubious reliability—are overvalued, while potentially important negative findings are underreported. In the words of a University of Virginia psychologist Brian Nosek, “There is no cost to getting things wrong. The cost is not getting them published” [<xref rid="ref6" ref-type="bibr">6</xref>]. This underreporting may be due to several reasons including the reluctance of journals to publish negative results because they are perceived as intrinsically less interesting and important; the lower importance given to negative results in the sphere of promotions, awards, and grants; and unconscious or conscious self-censorship when negative results contradict a personal or industrial research agenda.</p><p>High-quality systematic reviews and meta-analyses are considered the strongest form of medical evidence. By including only published studies with sound methodology and robust analysis, the noise from irreproducible or improperly conducted research is diminished. However, systematic reviews do not adequately address publication bias. As exemplified by the drawn out Cochrane review of the neuraminidase inhibitors [<xref rid="ref7" ref-type="bibr">7</xref>], this approach to evidence creation will be reliable only if there is access to all clinical trial data, and not only the ones that were analyzed and presented in publications. After finally getting access to previously unreleased studies (20 from Roche and 24 from GlaxoSmithKline), Cochrane Library found that the benefits of oseltamivir and zanamavir in the prevention and treatment of influenza had been overstated in previous meta-analyses [<xref rid="ref8" ref-type="bibr">8</xref>].</p><p>More alarmingly, systematic reviews shed some light on the inefficiency of research in providing clinical guidelines. In a cross-sectional study of more than 1000 systematic reviews in the Cochrane Library across all 50 Collaborative Review Groups, 96% recommended further research to fully evaluate the intervention in question [<xref rid="ref9" ref-type="bibr">9</xref>].</p><p>We have previously commented on the use of the vast amounts of data generated in the critical care setting to provide population-based data-driven care of individual patients [<xref rid="ref10" ref-type="bibr">10</xref>]. Big data is an all-encompassing term for any collection of datasets so large and complex that it becomes difficult to process using traditional data processing applications. It has become an increasingly important element of research in many scientific areas such as astronomy [<xref rid="ref11" ref-type="bibr">11</xref>], chemistry [<xref rid="ref12" ref-type="bibr">12</xref>], microbiology [<xref rid="ref13" ref-type="bibr">13</xref>], molecular biology [<xref rid="ref14" ref-type="bibr">14</xref>], and physics [<xref rid="ref15" ref-type="bibr">15</xref>]. Given the availability and use of these data and the dependency of a variety of enterprises, including clinical practice, on data analytics, there is justifiable concern regarding the issue. In addition, the use of increasingly big(ger) data will only further augment the noise resulting from these biases and problems that currently plague the scientific literature. We share these concerns and propose measures to mitigate the risk and improve the reliability and efficiency of the research enterprise.</p></sec><sec><title>From Open Data to Collaborative Learning</title><p>We suggest building on a system, previously described to some extent, where data and methods are freely shared among different groups of investigators addressing the same or similar questions [<xref rid="ref16" ref-type="bibr">16</xref>,<xref rid="ref17" ref-type="bibr">17</xref>]. Clearly, the infrastructure for this sharing process would rest extensively on carefully engineered Internet-based processes. The Internet has become such a quietly ubiquitous factor in our lives that we may forget to explicitly acknowledge the huge fundamental impact that it has had and is most likely to continue having on information storage and exchange, and in the process dramatically expanding our problem-solving ability and increasing our combined brainpower [<xref rid="ref18" ref-type="bibr">18</xref>]. Such processes would be created and implemented to validate and build on each group’s findings. Complete data interoperability would be required; this has been a particular issue with clinical data that emanate from the silos of different vendors. An open culture of sharing data would require a paradigm shift whereby individual research groups no longer compete for publication and funding. Research would be conducted with laboratories cooperatively testing the hypotheses of others with the goal of joint publication rather than independently developing similar hypotheses, then working in silos towards separate publications. Web-based applications would be further developed for the specific purpose of supporting research cooperation. Telematik-Plattform für Medizinische Forschungsnet (TMF) in Germany, for example, offers an open-access platform for interdisciplinary exchange as well as cross-project and cross-location cooperation in order to identify and address organizational, legal, ethical, and technological problems of modern medical research [<xref rid="ref19" ref-type="bibr">19</xref>].</p><p>This system enhancement will allow investigators to avoid the politics, secrecy, and inefficiencies that characterize the pursuit of publicly funded research in academic institutions. The current system actually inhibits advances by discouraging investigations in areas that appear to be already locked-up with funded but patented research. Those researchers who sought to restrict the use of their materials and methods in the reproduction of their experiments before, during, or after peer review, would be excluded, retrospectively if necessary, from the scientific research publication community. Researchers should be sufficiently confident in the merit and integrity of their work to provide this kind of cooperative transparency for the sake of facilitating scientific progress. We seek to systematically disassemble and revise the perception of laboratories conducting similar research as “competing” laboratories.</p><p>The best way to take down the wall between potentially competitive laboratories is to open the data gates [<xref rid="ref20" ref-type="bibr">20</xref>]. By sharing data, competitors will transform into collaborators. Grants would generally be awarded to laboratories collaborating in the same area, rather than to individual laboratories. Since results would be reported by more than one group, groups would review each other’s data with great scrutiny because fabricated data would tarnish the reputations of all involved parties on publication. Furthermore, with the availability and use of more complete datasets from past studies, future studies could be more efficiently constructed. The “open access” data model would apply to both published as well as unpublished data. Data can remain unpublished for a variety of reasons that include lack of submission or rejection. Researchers may not want to share certain findings or they may not be able to publish important negative or confirmatory work. In either case, it is critical that these data also be accessible. While this access presents additional technical and administrative difficulties, in the current environment of continuously improving (and cheaper) data storage, it should not be insurmountable.</p></sec><sec><title>An Entrenched Culture of Competition</title><p>Some may argue that competition, as opposed to cooperation, is the engine that drives scientific discovery. For example, one might maintain that fierce competition accelerated the process of Watson and Crick’s solution of the structure of deoxyribonucleic acid (DNA). However, the competitive process can become counterproductive when secrecy transcends honest collegiality and an “end justifies any means” approach is adopted. Open collaboration with Pauling, Franklin, and Wilkins may well have <italic>shortened</italic> the discovery process. In addition, the credit would have been distributed differently or at least, more widely shared. Watson and Crick feared the possibility of Pauling’s latching onto the solution first far more than they welcomed the potential contribution of Pauling’s genius. Indeed, it was not until Watson and Crick obtained unauthorized and questionably ethical access to the crystallographic work of Rosalind Franklin that they were able to correctly deduce the double helical structure [<xref rid="ref21" ref-type="bibr">21</xref>].</p><p>As biomedical science becomes less of an individualistic pursuit, as substantiated by the large author numbers listed for many publications, the primary driver for scientific knowledge must shift from individual glory to group accomplishment. Outstanding individuals can and will still be recognized and rewarded, but their roles may lie primarily in leading and coordinating groups rather than carrying out entire complex projects on their own. For example, promotion and tenure committees have adjusted to the shift to multi-authorship whereas in a prior era, such publications may have been significantly underweighted due to the small fractional contribution of each author.</p></sec><sec><title>Towards Open Continuous Peer Review</title><p>All data from taxpayer-supported funding sources would truly be open and freely available to the public. This would include unpublished or privately published data as well as data from papers published in publicly available journals. This would create an environment where diverse groups could access the relevant database(s) to further investigate and validate (or invalidate) the published findings. Unpublished material may contain keys to interpreting the published results in both positive and negative, reliable and unreliable lights. No research finding would be immune from query and possible challenge. The possibility that an investigator’s published material might even be proven false by a very junior researcher or a layperson would spark the due diligence necessary for accurate data collection and a robust aversion to falsified or fabricated data—as well as to those individuals who engage in such practice. The academic promotion process would adjust to this new culture but, except for the recognized rare lone wolf genius, would actually require investigators to participate in research collaboratives. In addition, the reward system (eg, promotions, awards) would more strongly recognize the publication of important negative results and more negatively weigh the publication of research that proved to be unreliable, required retraction, etc. While it is more difficult and time consuming for evaluative bodies to do so, the content of articles should be carefully considered in detail by such committees. This may not be the case when the process rests on a more superficial count of publications along with the impact factors of the journals in which they are published.</p><p>Scholarly journals would need to work together in order to specifically prevent the publication of unreliable research. Post-publication vigilance for reliability on the part of journals (akin to the post-marketing life cycles of drugs) would become a fundamental element of the scholarly publication process. Those journals best at tracking research post publication might see their impact factors rise in a manner proportional to their engagement in the process. Even better, perhaps more valid and useful metrics than impact factors, such as a “collaboration index”, could be developed to better represent journal quality. Post-publication vigilance on the part of journals would become a fundamental element of the scholarly publication process.</p></sec><sec><title>Shouldering the Cost of Reliability</title><p>Without the unlikely <italic>deus ex machina</italic> intervention of a Gates or a Buffett to fund a new non-profit organization devoted to disinterested auditing for the sake of improving research reliability, the funding would likely have to derive from a consortium of the interested parties. The culture would need to develop the embedded philosophy that these costs are simply a part of “doing business” in the sense of producing reliably performed and reported research. The possible involved actors include the researchers themselves; the “payers”, or the source of funding, including non-profit, governmental, and industrial agencies; journals; universities for academic researchers; and hospital systems and professional societies for medical research. We propose for a lean and efficient organization, perhaps an independent non-profit entity, funded in an acceptably fair fashion by the involved parties, whose mission would be to provide what increasingly appears to be much required oversight to the research enterprise.</p><p>There might also be an unintended educational benefit from the auditing system. These datasets could be used as exercise for graduate or undergraduate students in their respective fields to review and validate, and credit could be awarded to students who participate as a means of displaying their scholarship in the field. Such measures may be necessary as statistical and machine learning methodologies continue to evolve, and a number of data and statistical expert types may be required to validate the findings. Current technology allows for the simple collection and transfer of vast amounts of data, and this portability should be a vehicle for broadening the base that view and validate research for publication, application, and dissemination.</p><p>There are examples of new journals that support and promote data sharing. <italic>BMJ Open</italic> was launched in 2011 and was the first medical journal to integrate its submission process with the Dryad digital repository, so that data deposition is part of authors’ submission/workflow [<xref rid="ref22" ref-type="bibr">22</xref>]. Data Dryad is a curated general-purpose repository that makes the data underlying scientific publications discoverable, freely reusable, and citable [<xref rid="ref23" ref-type="bibr">23</xref>]. It now has integrated data submission for a growing list of journals. <italic>Scientific Data</italic> is a new open-access, online-only publication for descriptions of scientifically valuable datasets [<xref rid="ref24" ref-type="bibr">24</xref>]. It is currently calling for submissions and was launched in May 2014. In fact, the traditional journal model may not be the only modality for the public transmission of scientific information: data may be provided freely online as in the Genomes Unzipped project [<xref rid="ref25" ref-type="bibr">25</xref>].</p><p>With open-access data, research will be democratized and no longer confined within traditional academic environments or industry-funded laboratories. This will allow interacting groups of investigators to conduct research with varying methods on the same and/or related topics using the same data. This will mitigate false positives based on biases related to researchers’ hypotheses, reduce unnecessary experimental reproduction, and open research fields to fuller participation.</p></sec><sec><title>Crowdsourcing the Validation of Research Findings</title><p>Replication will not remove either bias or residual confounding. For observational studies, hidden variables, including provider bias and local medical culture issues, that confound the true relationship between an exposure and an outcome, tend to differ across heterogeneous settings. If the association between an intervention and an outcome remains strong across countries with very different practices, for example, then the probability that this association is noise rather than signal is reduced (vs an observation in a single locale).</p><p>A major challenge with observational studies that is not addressed by the size of the data is the presence of residual confounding: there may be characteristics of the patients or the diseases not captured in clinical databases that may explain why patients who got treatment A got better while those that got treatment B did not. The social determinants of health—the conditions in which people are born, grow, live, work, and age, and most importantly, the individual behaviors in response to these variables—are seldom captured in electronic health records (EHR). The sociological factors are becoming increasingly available as large cities build open data platforms. For example, the NYC Open Data repository [<xref rid="ref26" ref-type="bibr">26</xref>] contains over 1100 datasets spanning Business, City Government, Education, Environment, Health, Housing and Development, Public Safety, Recreation, Social Services, and Transportation. Behavioral data are likewise increasingly captured digitally through mobile phones, tracking of Internet usage (including social media), global positioning system (GPS) devices and other wireless sensors, purchases, and other financial transactions, etc. The challenge, needless to say, is to map these disparate data sources without significant risk to privacy and security.</p><p>For translational and clinical research, the additional research scrutiny would set the stage for the development of useful standard thresholds at which a research finding could be considered valid and reliable, for example, after n number of replications or after a particular statistical requirement is achieved. Within a bigger data framework, the threshold standardization would become progressively more valid due to the statistical confidence afforded by larger datasets.</p><p>A key ingredient to making this vision successful is efficient crowdsourcing. Models that may be emulated already exist and are efficiently being used by researchers today. The Multi-parameter Intelligent Monitoring in Intensive Care (MIMIC) database is one such successful initiative [<xref rid="ref27" ref-type="bibr">27</xref>]. The MIMIC database, developed and maintained by the Laboratory of Computational Physiology at the Massachusetts Institute of Technology (MIT), contains health care metrics of over 60,000 de-identified intensive care unit patients admitted to the Beth Israel Deaconess Medical Center (BIDMC). Information is carefully de-identified to minimize identification risks without excessive deletion of information of clinical value. Natural language processing is used to correctly capture the correct and precise meaning of clinical entries that can be difficult to elucidate based on factors such as physician abbreviations and a plethora of contextual modifiers (eg, “rule out”, “suspect”, “history of”). The success of MIMIC at an admittedly relative small scale gives great hope for our ability to similarly use larger clinical databases for both dynamic and retrospective data mining purposes. This database has already been employed in several publications that have brought together frontline clinicians with data scientists and computer engineers, under the guidance of the authors. The Institutional Review Boards of both MIT and BIDMC have approved the use of MIMIC for research purposes. The authors also recently organized the Critical Data Marathon and Conference held at MIT in January 2014 [<xref rid="ref28" ref-type="bibr">28</xref>]. The conference’s theme was to address concerns that big data will only augment the problem of unreliable research. Professors Jeffrey Drazen, New England Journal of Medicine editor-in-chief, and John Ioannidis, director of the Meta-Research Innovation Center at Stanford, were the keynote speakers. The second MIMIC data marathon was held concurrently at MIT, in London and in Paris on September 2014 and attracted more than 200 participants.</p><p>The PCORnet is a new initiative by the Patient-Centered Outcomes Research Institute [<xref rid="ref29" ref-type="bibr">29</xref>]. It is a database that will consist of 11 clinical data research networks across the country cataloguing primarily EHR data, with some degree of mapping to 18 patient-powered research networks archiving all types of data collected by patients. If all goes well, by September 2015 PCORnet will be a giant repository of medical information from 26-30 million Americans. In the United Kingdom, the National Institute for Health Research Health Informatics Collaboration was launched in November 2013 [<xref rid="ref30" ref-type="bibr">30</xref>]. Five National Health Service (NHS) trusts are working together to make NHS clinical data more readily available and accessible to researchers, industry, and the NHS community. The main objectives are to develop, design, and provide common infrastructure, standards, and services that will allow users to perform secondary analysis of EHR data in the fields of viral hepatology, acute coronary syndrome, ovarian cancer, renal transplantation, and critical care. But as we have previously emphasized, the value of these large databases hinges on the researchers’ transparency in their methodology and the creation of a continuous and more effective peer review, leading to improvement in method quality with each iteration of analysis and resulting in more reliability.</p><p>Data marathons held around open data (including MIMIC) have attracted both students and postgraduate trainees [<xref rid="ref31" ref-type="bibr">31</xref>]. Closed research networks would expand and evolve, removing the barriers that have made research activity accessible only to a limited group. In effect, research networks would expand exponentially breaking down the walls that have made much research activity accessible to only a relatively small group of academics.</p></sec><sec><title>Open Big Data Hits and Misses</title><p>As impressive as it is, the NYC Open Data repository highlights some of the problems that we see in big data initiatives. For example, urban planning has to date not incorporated health care data to fully exploit the data’s potential to inform public health policies. As shown in <xref ref-type="fig" rid="figure1">Figure 1</xref>, health data are directly shared only between the Health and Hospitals Corporation, the Department of City Planning, and the Department of Health and Mental Hygiene. Noticeably missing are direct data-sharing connections between these organizations and the New York Department of Sanitation, Department of Homeless Services, Department for the Aging, and the Office of Emergency Management. The association map illustrates that organizations that would benefit from direct multidisciplinary collaboration appear to be operating in informational siloes, which is a recurring theme among big data projects.</p><p>But there have been success stories. In October 2012, GlaxoSmithKline announced that it would make detailed data from its clinical trials available to researchers outside its own walls [<xref rid="ref32" ref-type="bibr">32</xref>]. For a company that spends $6.5 billion a year on research and development, it was a surprising departure from the entrenched system of data secrecy. True to its word, the company began posting its patient-level clinical trial data online in May 2013 and then invited other pharmaceutical companies to do the same. Consequently the Clinical Trial Data Request project was launched [<xref rid="ref33" ref-type="bibr">33</xref>]. Pharmaceutical companies that have so far committed to contribute (apart from GlaxoSmithKline) include Bayer, Boehringer Ingelheim, Lilly, Novartis, Roche, Sanofi, Takeda, Union Chimique Belge (UCB), and ViiV Healthcare. To date, more than 1000 clinical trials have been uploaded. Trial transparency is appealing because of a growing sense that it could make drug development more efficient, saving the industry billions while also getting breakthrough therapies to patients more quickly.</p><p>Finally, the Global Alliance for Genomics and Health was established in 2013 and consists of genomics researchers, funders, businesses, and advocates [<xref rid="ref34" ref-type="bibr">34</xref>]. The coalition develops and implements technical, ethical, legal, and clinical guidelines to make it easier to share genomic data. This is an example of an international multidisciplinary collaboration around a big data initiative representing various sectors including academia, industry, and the government. The current focus of the group is the creation of a genomics application programming interface to enable the interoperable exchange of data in DNA sequence reads and a framework for data sharing to guide governance and research.</p><fig id="figure1" position="float"><label>Figure 1</label><caption><p>Visualization of NYC Open Data. Figure courtesy of Yuan Lai.</p></caption><graphic xlink:href="jmir_v16i11e259_fig1"></graphic></fig></sec><sec><title>Implementing Transparent Oversight Systems</title><p>Systems will need to be put in place to institute the replication and cross validation of experiments and analyses (<xref ref-type="fig" rid="figure2">Figure 2</xref>). Universities, professional societies, government agencies, and research-driven companies are examples of organizations that could develop and operate these systems. Regulatory boards such as the Food and Drug Administration (FDA) and European Medicines Association (EMA) will have to revise existing approval requirements; for example, trials backing a drug or device will require replication as well as validation by different groups. To renew approval, companies will be required to submit ongoing, regular reports that track the effectiveness of their products in the real world. A pharmacovigilance system has been described that proactively uses clinical database networks to accumulate safety and efficacy evidence when drugs are used in wider, more diverse patient populations than those typically examined during pre-market approval clinical studies [<xref rid="ref35" ref-type="bibr">35</xref>]. Both the FDA and EMA have already proposed the expansion of data access submitted in regulatory applications [<xref rid="ref36" ref-type="bibr">36</xref>,<xref rid="ref37" ref-type="bibr">37</xref>]. In 2012, The Royal Society published a report on science as an open enterprise that mapped out the changes required of scientists, their institutions, and those that fund and support science in order to optimize the potential of the huge deluge of data created by modern technologies [<xref rid="ref38" ref-type="bibr">38</xref>]. Last year, the AllTrials campaign was launched globally with a cross-sectoral support, calling for all clinical trials to be registered and all results reported [<xref rid="ref39" ref-type="bibr">39</xref>]. The initiative pushes for researchers, funding bodies, institutions, ethics committees, and regulators to work together to ensure that the value from the resources used to produce research is maximized. In the United States, the Center for Open Science was inaugurated [<xref rid="ref40" ref-type="bibr">40</xref>]. It is a non-profit technology organization whose mission is to increase the openness, integrity, and of scientific research. At the heart of the organization is the Open Science Framework, an open source software that facilitates collaboration in science research. Most recently, the Meta-Research Innovation Center at Stanford was launched. Headed by Professors John Ioannidis and Steven Goodman, the center will undertake rigorous evaluation of research practices with the aim of optimizing the reliability of scientific investigations and the efficiency of the biomedical research enterprise [<xref rid="ref41" ref-type="bibr">41</xref>].</p><p>With such systems in place, investors will be able to invest in a group of companies working on a related product or idea, rather than solely in individual companies. The ability to invest in a “fund” of related research initiatives should also reduce investor risk in a manner akin to the reduced risk associated with financial index investing versus individual stock picking. Perhaps in the future, new investment products would arise that focus on companies associated with a particular research subject. As noted, funding agencies like the National Institute of Health and the National Science Foundation would award grants to collaborative groups of laboratories. The extensive time and effort spent in preparing grants that may or may not be successfully funded would presumably be reduced, allowing scientists to focus their efforts on research.</p><p>The solution to the conundrum of unreliable research lies not only in complete transparency, but more importantly in cooperation among investigators, and with a more lateral distribution of investments, grant funding, and credit for scientific discoveries. We expect these proposals would bring about a culture of collaboration and shared data as well as more complete and accurate reporting of scientific findings. The added accuracy of the scientific findings is only one of the benefits of the systematization of data interrogation. Another will be the enhanced ability of individuals of every educational level and area of expertise to thrust themselves into the fray and contribute to science. We wish to echo the sentiments of Louis Pasteur when he stated, “Science knows no country, because knowledge belongs to humanity, and is the torch which illuminates the world.”</p><fig id="figure2" position="float"><label>Figure 2</label><caption><p>Graphic illustrating how to address unreliable research. Figure courtesy of Kai-ou Tang.</p></caption><graphic xlink:href="jmir_v16i11e259_fig2"></graphic></fig></sec></body><back><fn-group><fn fn-type="conflict"><p>Conflicts of Interest: None declared.</p></fn></fn-group><glossary><title>Abbreviations</title><def-list><def-item><term id="abb1">BIDMC</term><def><p>Beth Israel Deaconess Medical Center</p></def></def-item><def-item><term id="abb2">EHR</term><def><p>electronic health record</p></def></def-item><def-item><term id="abb3">EMA</term><def><p>European Medicines Association</p></def></def-item><def-item><term id="abb4">FDA</term><def><p>US Food and Drug Administration</p></def></def-item><def-item><term id="abb5">MIMIC</term><def><p>Multi-parameter Intelligent Monitoring in Intensive Care</p></def></def-item><def-item><term id="abb6">MIT</term><def><p>Massachusetts Institute of Technology</p></def></def-item><def-item><term id="abb7">NHS</term><def><p>National Health Service</p></def></def-item></def-list></glossary><ref-list><ref id="ref1"><label>1</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Macleod</surname><given-names>MR</given-names></name><name><surname>Michie</surname><given-names>S</given-names></name><name><surname>Roberts</surname><given-names>I</given-names></name><name><surname>Dirnagl</surname><given-names>U</given-names></name><name><surname>Chalmers</surname><given-names>I</given-names></name><name><surname>Ioannidis</surname><given-names>JP</given-names></name><name><surname>Al-Shahi Salman</surname><given-names>R</given-names></name><name><surname>Chan</surname><given-names>AW</given-names></name><name><surname>Glasziou</surname><given-names>P</given-names></name></person-group><article-title>Biomedical research: increasing value, reducing waste</article-title><source>Lancet</source><year>2014</year><month>1</month><day>11</day><volume>383</volume><issue>9912</issue><fpage>101</fpage><lpage>4</lpage><pub-id pub-id-type="doi">10.1016/S0140-6736(13)62329-6</pub-id><pub-id pub-id-type="medline">24411643</pub-id><pmc-comment>S0140-6736(13)62329-6 </pmc-comment>
<pub-id pub-id-type="pmid">24411643</pub-id></element-citation></ref><ref id="ref2"><label>2</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Ioannidis</surname><given-names>JP</given-names></name><name><surname>Greenland</surname><given-names>S</given-names></name><name><surname>Hlatky</surname><given-names>MA</given-names></name><name><surname>Khoury</surname><given-names>MJ</given-names></name><name><surname>Macleod</surname><given-names>MR</given-names></name><name><surname>Moher</surname><given-names>D</given-names></name><name><surname>Schulz</surname><given-names>KF</given-names></name><name><surname>Tibshirani</surname><given-names>R</given-names></name></person-group><article-title>Increasing value and reducing waste in research design, conduct, and analysis</article-title><source>Lancet</source><year>2014</year><month>1</month><day>11</day><volume>383</volume><issue>9912</issue><fpage>166</fpage><lpage>75</lpage><pub-id pub-id-type="doi">10.1016/S0140-6736(13)62227-8</pub-id><pub-id pub-id-type="medline">24411645</pub-id><pmc-comment>S0140-6736(13)62227-8 </pmc-comment>
<pub-id pub-id-type="pmid">24411645</pub-id></element-citation></ref><ref id="ref3"><label>3</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Chan</surname><given-names>AW</given-names></name><name><surname>Song</surname><given-names>F</given-names></name><name><surname>Vickers</surname><given-names>A</given-names></name><name><surname>Jefferson</surname><given-names>T</given-names></name><name><surname>Dickersin</surname><given-names>K</given-names></name><name><surname>Gøtzsche</surname><given-names>PC</given-names></name><name><surname>Krumholz</surname><given-names>HM</given-names></name><name><surname>Ghersi</surname><given-names>D</given-names></name><name><surname>van der Worp</surname><given-names>HB</given-names></name></person-group><article-title>Increasing value and reducing waste: addressing inaccessible research</article-title><source>Lancet</source><year>2014</year><month>1</month><day>18</day><volume>383</volume><issue>9913</issue><fpage>257</fpage><lpage>66</lpage><pub-id pub-id-type="doi">10.1016/S0140-6736(13)62296-5</pub-id><pub-id pub-id-type="medline">24411650</pub-id><pmc-comment>S0140-6736(13)62296-5 </pmc-comment>
<pub-id pub-id-type="pmid">24411650</pub-id></element-citation></ref><ref id="ref4"><label>4</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Godlee</surname><given-names>F</given-names></name></person-group><article-title>Research misconduct is widespread and harms patients</article-title><source>BMJ</source><year>2012</year><month>1</month><day>05</day><volume>344</volume><issue>jan05 1</issue><fpage>e14</fpage><lpage>e14</lpage><pub-id pub-id-type="doi">10.1136/bmj.e14</pub-id></element-citation></ref><ref id="ref5"><label>5</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Peierls</surname><given-names>R</given-names></name></person-group><article-title>Wolfgang Ernst Pauli, 1900-1958</article-title><source>Biogr Mem Fellows R Soc</source><year>1960</year><volume>5</volume><fpage>174</fpage><lpage>192</lpage></element-citation></ref><ref id="ref6"><label>6</label><element-citation publication-type="webpage"><source>The Economist</source><date-in-citation>2014-09-17</date-in-citation><comment>Unreliable Research: Trouble at the lab<ext-link ext-link-type="uri" xlink:href="http://www.economist.com/news/briefing/21588057-scientists-think-science-self-correcting-alarming-degree-it-not-trouble">http://www.economist.com/news/briefing/21588057-scientists-think-science-self-correcting-alarming-degree-it-not-trouble</ext-link></comment><ext-link ext-link-type="webcite" xlink:href="6Se5SDqRi"></ext-link></element-citation></ref><ref id="ref7"><label>7</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Doshi</surname><given-names>P</given-names></name></person-group><article-title>Neuraminidase inhibitors--the story behind the Cochrane review</article-title><source>BMJ</source><year>2009</year><volume>339</volume><fpage>b5164</fpage><pub-id pub-id-type="medline">19995813</pub-id><pub-id pub-id-type="pmid">19995813</pub-id></element-citation></ref><ref id="ref8"><label>8</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Jefferson</surname><given-names>T</given-names></name><name><surname>Jones</surname><given-names>M</given-names></name><name><surname>Doshi</surname><given-names>P</given-names></name><name><surname>Del Mar</surname><given-names>C</given-names></name><name><surname>Dooley</surname><given-names>L</given-names></name><name><surname>Foxlee</surname><given-names>R</given-names></name></person-group><article-title>Neuraminidase inhibitors for preventing and treating influenza in healthy adults</article-title><source>Cochrane Database Syst Rev</source><year>2010</year><issue>2</issue><fpage>CD001265</fpage><pub-id pub-id-type="doi">10.1002/14651858.CD001265.pub3</pub-id><pub-id pub-id-type="medline">20166059</pub-id><pub-id pub-id-type="pmid">20166059</pub-id></element-citation></ref><ref id="ref9"><label>9</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>El Dib</surname><given-names>RP</given-names></name><name><surname>Atallah</surname><given-names>AN</given-names></name><name><surname>Andriolo</surname><given-names>RB</given-names></name></person-group><article-title>Mapping the Cochrane evidence for decision making in health care</article-title><source>J Eval Clin Pract</source><year>2007</year><month>8</month><volume>13</volume><issue>4</issue><fpage>689</fpage><lpage>92</lpage><pub-id pub-id-type="doi">10.1111/j.1365-2753.2007.00886.x</pub-id><pub-id pub-id-type="medline">17683315</pub-id><pmc-comment>JEP886 </pmc-comment>
<pub-id pub-id-type="pmid">17683315</pub-id></element-citation></ref><ref id="ref10"><label>10</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Celi</surname><given-names>LA</given-names></name><name><surname>Mark</surname><given-names>RG</given-names></name><name><surname>Stone</surname><given-names>DJ</given-names></name><name><surname>Montgomery</surname><given-names>RA</given-names></name></person-group><article-title>"Big data" in the intensive care unit. Closing the data loop</article-title><source>Am J Respir Crit Care Med</source><year>2013</year><month>6</month><day>1</day><volume>187</volume><issue>11</issue><fpage>1157</fpage><lpage>60</lpage><comment><ext-link ext-link-type="uri" xlink:href="http://europepmc.org/abstract/MED/23725609"></ext-link></comment><pub-id pub-id-type="doi">10.1164/rccm.201212-2311ED</pub-id><pub-id pub-id-type="medline">23725609</pub-id><pmc-comment>PMC3734609 </pmc-comment>
<pub-id pub-id-type="pmid">23725609</pub-id></element-citation></ref><ref id="ref11"><label>11</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Feigelson</surname><given-names>ED</given-names></name><name><surname>Babu</surname><given-names>GJ</given-names></name></person-group><article-title>Big data in astronomy</article-title><source>Significance</source><year>2012</year><month>8</month><day>09</day><volume>9</volume><issue>4</issue><fpage>208</fpage><lpage>11</lpage><pub-id pub-id-type="doi">10.1111/j.1740-9713.2012.00587.x</pub-id></element-citation></ref><ref id="ref12"><label>12</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Gibb</surname><given-names>BC</given-names></name></person-group><article-title>Big (chemistry) data</article-title><source>Nat Chem</source><year>2013</year><month>4</month><volume>5</volume><issue>4</issue><fpage>248</fpage><lpage>9</lpage><pub-id pub-id-type="doi">10.1038/nchem.1604</pub-id><pub-id pub-id-type="medline">23511403</pub-id><pmc-comment>nchem.1604 </pmc-comment>
<pub-id pub-id-type="pmid">23511403</pub-id></element-citation></ref><ref id="ref13"><label>13</label><element-citation publication-type="journal"><person-group person-group-type="author"><collab>Editorial</collab></person-group><article-title>Microbiota meet big data</article-title><source>Nat Chem Biol</source><year>2014</year><month>8</month><volume>10</volume><issue>8</issue><fpage>605</fpage><pub-id pub-id-type="doi">10.1038/nchembio.1604</pub-id><pub-id pub-id-type="medline">25036299</pub-id><pmc-comment>nchembio.1604 </pmc-comment>
<pub-id pub-id-type="pmid">25036299</pub-id></element-citation></ref><ref id="ref14"><label>14</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Marx</surname><given-names>V</given-names></name></person-group><article-title>Biology: The big challenges of big data</article-title><source>Nature</source><year>2013</year><month>6</month><day>13</day><volume>498</volume><issue>7453</issue><fpage>255</fpage><lpage>60</lpage><pub-id pub-id-type="doi">10.1038/498255a</pub-id><pub-id pub-id-type="medline">23765498</pub-id><pmc-comment>498255a </pmc-comment>
<pub-id pub-id-type="pmid">23765498</pub-id></element-citation></ref><ref id="ref15"><label>15</label><element-citation publication-type="webpage"><person-group person-group-type="author"><name><surname>Hesla</surname><given-names>L</given-names></name></person-group><source>Symmetry</source><year>2012</year><month>8</month><day>1</day><date-in-citation>2014-10-12</date-in-citation><comment>Particle physics tames big data<ext-link ext-link-type="uri" xlink:href="http://www.symmetrymagazine.org/article/august-2012/particle-physics-tames-big-data">http://www.symmetrymagazine.org/article/august-2012/particle-physics-tames-big-data</ext-link></comment><ext-link ext-link-type="webcite" xlink:href="6TFxKStK2"></ext-link></element-citation></ref><ref id="ref16"><label>16</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Califf</surname><given-names>RM</given-names></name><name><surname>McCall</surname><given-names>J</given-names></name><name><surname>Harrington</surname><given-names>RA</given-names></name></person-group><article-title>Assessing research results in the medical literature: trust but verify</article-title><source>JAMA Intern Med</source><year>2013</year><month>6</month><day>24</day><volume>173</volume><issue>12</issue><fpage>1053</fpage><lpage>5</lpage><pub-id pub-id-type="doi">10.1001/jamainternmed.2013.829</pub-id><pub-id pub-id-type="medline">23609490</pub-id><pmc-comment>1680145 </pmc-comment>
<pub-id pub-id-type="pmid">23609490</pub-id></element-citation></ref><ref id="ref17"><label>17</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Christakis</surname><given-names>DA</given-names></name><name><surname>Zimmerman</surname><given-names>FJ</given-names></name></person-group><article-title>Rethinking reanalysis</article-title><source>JAMA</source><year>2013</year><month>12</month><day>18</day><volume>310</volume><issue>23</issue><fpage>2499</fpage><lpage>500</lpage><pub-id pub-id-type="doi">10.1001/jama.2013.281337</pub-id><pub-id pub-id-type="medline">24346985</pub-id><pmc-comment>1790902 </pmc-comment>
<pub-id pub-id-type="pmid">24346985</pub-id></element-citation></ref><ref id="ref18"><label>18</label><element-citation publication-type="book"><person-group person-group-type="author"><name><surname>Nielsen</surname><given-names>M</given-names></name></person-group><source>Reinventing Discovery: The New Era of Networked Science</source><year>2011</year><publisher-loc>Princeton, NJ</publisher-loc><publisher-name>Princeton University Press</publisher-name></element-citation></ref><ref id="ref19"><label>19</label><element-citation publication-type="webpage"><source>Telematik-Plattform für Medizinische Forschungsnet</source><date-in-citation>2014-10-12</date-in-citation><comment><ext-link ext-link-type="uri" xlink:href="http://www.tmf-ev.de/EnglishSite/AboutUs.aspx">http://www.tmf-ev.de/EnglishSite/AboutUs.aspx</ext-link></comment><ext-link ext-link-type="webcite" xlink:href="6TFxUMs21"></ext-link></element-citation></ref><ref id="ref20"><label>20</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Wilbanks</surname><given-names>J</given-names></name></person-group><article-title>Another reason for opening access to research</article-title><source>BMJ</source><year>2006</year><month>12</month><day>23</day><volume>333</volume><issue>7582</issue><fpage>1306</fpage><lpage>8</lpage><comment><ext-link ext-link-type="uri" xlink:href="http://europepmc.org/abstract/MED/17185718"></ext-link></comment><pub-id pub-id-type="doi">10.1136/sbmj.39063.730660.F7</pub-id><pub-id pub-id-type="medline">17185718</pub-id><pmc-comment>333/7582/1306 </pmc-comment>
<pmc-comment>PMC1761190 </pmc-comment>
<pub-id pub-id-type="pmid">17185718</pub-id></element-citation></ref><ref id="ref21"><label>21</label><element-citation publication-type="book"><person-group person-group-type="author"><name><surname>Watson</surname><given-names>JD</given-names></name></person-group><source>The Double Helix: A Personal Account of the Discovery of the Structure of DNA (1968)</source><year>1980</year><publisher-loc>New York, NY</publisher-loc><publisher-name>Atheneum Books</publisher-name></element-citation></ref><ref id="ref22"><label>22</label><element-citation publication-type="webpage"><source>BMJ Open</source><date-in-citation>2014-09-16</date-in-citation><comment><ext-link ext-link-type="uri" xlink:href="http://bmjopen.bmj.com/">http://bmjopen.bmj.com/</ext-link></comment><ext-link ext-link-type="webcite" xlink:href="6SdosWgCQ"></ext-link></element-citation></ref><ref id="ref23"><label>23</label><element-citation publication-type="webpage"><source>Data Dryad</source><date-in-citation>2014-09-16</date-in-citation><comment><ext-link ext-link-type="uri" xlink:href="http://datadryad.org/">http://datadryad.org/</ext-link></comment><ext-link ext-link-type="webcite" xlink:href="6Sdptvc1x"></ext-link></element-citation></ref><ref id="ref24"><label>24</label><element-citation publication-type="webpage"><source>Scientific Data</source><date-in-citation>2014-09-16</date-in-citation><comment><ext-link ext-link-type="uri" xlink:href="http://www.nature.com/sdata/">http://www.nature.com/sdata/</ext-link></comment><ext-link ext-link-type="webcite" xlink:href="6SdqgOoQC"></ext-link></element-citation></ref><ref id="ref25"><label>25</label><element-citation publication-type="webpage"><source>Genomes Unzipped</source><date-in-citation>2014-09-16</date-in-citation><comment><ext-link ext-link-type="uri" xlink:href="http://genomesunzipped.org/">http://genomesunzipped.org/</ext-link></comment><ext-link ext-link-type="webcite" xlink:href="6Sdr1IKnc"></ext-link></element-citation></ref><ref id="ref26"><label>26</label><element-citation publication-type="webpage"><person-group person-group-type="author"><collab>The City of New York</collab></person-group><source>NYC Open Data</source><date-in-citation>2014-10-12</date-in-citation><comment><ext-link ext-link-type="uri" xlink:href="https://nycopendata.socrata.com/dashboard">https://nycopendata.socrata.com/dashboard</ext-link></comment><ext-link ext-link-type="webcite" xlink:href="6TFyA8JFS"></ext-link></element-citation></ref><ref id="ref27"><label>27</label><element-citation publication-type="webpage"><source>Massachusetts Institute of Technology</source><date-in-citation>2014-09-16</date-in-citation><comment>MIMIC II<ext-link ext-link-type="uri" xlink:href="https://mimic.physionet.org/">https://mimic.physionet.org/</ext-link></comment><ext-link ext-link-type="webcite" xlink:href="6SdrIXV53"></ext-link></element-citation></ref><ref id="ref28"><label>28</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Badawi</surname><given-names>O</given-names></name><name><surname>Brennan</surname><given-names>T</given-names></name><name><surname>Celi</surname><given-names>LA</given-names></name><name><surname>Feng</surname><given-names>M</given-names></name><name><surname>Ghassemi</surname><given-names>M</given-names></name><name><surname>Ippolito</surname><given-names>A</given-names></name><name><surname>Johnson</surname><given-names>A</given-names></name><name><surname>Mark</surname><given-names>RG</given-names></name><name><surname>Mayaud</surname><given-names>L</given-names></name><name><surname>Moody</surname><given-names>G</given-names></name><name><surname>Moses</surname><given-names>C</given-names></name><name><surname>Naumann</surname><given-names>T</given-names></name><name><surname>Pimentel</surname><given-names>M</given-names></name><name><surname>Pollard</surname><given-names>TJ</given-names></name><name><surname>Santos</surname><given-names>M</given-names></name><name><surname>Stone</surname><given-names>DJ</given-names></name><name><surname>Zimolzak</surname><given-names>A</given-names></name></person-group><article-title>Making Big Data Useful for Health Care: A Summary of the Inaugural MIT Critical Data Conference</article-title><source>JMIR Med Inform</source><year>2014</year><month>8</month><day>22</day><volume>2</volume><issue>2</issue><fpage>e22</fpage><pub-id pub-id-type="doi">10.2196/medinform.3447</pub-id></element-citation></ref><ref id="ref29"><label>29</label><element-citation publication-type="webpage"><source>Patient-Centered Outcomes Research Institute</source><date-in-citation>2014-10-12</date-in-citation><comment>PCORnet: The National Patient-Centered Clinical Research Network<ext-link ext-link-type="uri" xlink:href="http://www.pcori.org/content/pcornet-national-patient-centered-clinical-research-network">http://www.pcori.org/content/pcornet-national-patient-centered-clinical-research-network</ext-link></comment><ext-link ext-link-type="webcite" xlink:href="6TFyNjJCD"></ext-link></element-citation></ref><ref id="ref30"><label>30</label><element-citation publication-type="webpage"><source>National Institute for Health Research</source><date-in-citation>2014-10-12</date-in-citation><comment>Major health informatics programme announced<ext-link ext-link-type="uri" xlink:href="http://www.uclhospitals.brc.nihr.ac.uk/news/major-health-informatics-programme-announced">http://www.uclhospitals.brc.nihr.ac.uk/news/major-health-informatics-programme-announced</ext-link></comment><ext-link ext-link-type="webcite" xlink:href="6TFyV35VI"></ext-link></element-citation></ref><ref id="ref31"><label>31</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Celi</surname><given-names>LA</given-names></name><name><surname>Ippolito</surname><given-names>A</given-names></name><name><surname>Montgomery</surname><given-names>RA</given-names></name><name><surname>Moses</surname><given-names>C</given-names></name><name><surname>Stone</surname><given-names>DJ</given-names></name></person-group><article-title>Crowdsourcing knowledge discovery and innovations in medicine</article-title><source>J Med Internet Res</source><year>2014</year><volume>16</volume><issue>9</issue><fpage>e216</fpage><comment><ext-link ext-link-type="uri" xlink:href="http://www.jmir.org/2014/9/e216/"></ext-link></comment><pub-id pub-id-type="doi">10.2196/jmir.3761</pub-id><pub-id pub-id-type="medline">25239002</pub-id><pmc-comment>v16i9e216 </pmc-comment>
<pmc-comment>PMC4180345 </pmc-comment>
<pub-id pub-id-type="pmid">25239002</pub-id></element-citation></ref><ref id="ref32"><label>32</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Harrison</surname><given-names>C</given-names></name></person-group><article-title>GlaxoSmithKline opens the door on clinical data sharing</article-title><source>Nat Rev Drug Discov</source><year>2012</year><month>12</month><volume>11</volume><issue>12</issue><fpage>891</fpage><lpage>2</lpage><pub-id pub-id-type="doi">10.1038/nrd3907</pub-id><pub-id pub-id-type="medline">23197021</pub-id><pmc-comment>nrd3907 </pmc-comment>
<pub-id pub-id-type="pmid">23197021</pub-id></element-citation></ref><ref id="ref33"><label>33</label><element-citation publication-type="webpage"><source>Clinical Trial Data Request</source><date-in-citation>2014-10-12</date-in-citation><comment><ext-link ext-link-type="uri" xlink:href="https://clinicalstudydatarequest.com/">https://clinicalstudydatarequest.com/</ext-link></comment><ext-link ext-link-type="webcite" xlink:href="6TFyjeT7t"></ext-link></element-citation></ref><ref id="ref34"><label>34</label><element-citation publication-type="webpage"><source>Global Alliance for Genomics and Health</source><date-in-citation>2014-11-01</date-in-citation><comment><ext-link ext-link-type="uri" xlink:href="http://genomicsandhealth.org/">http://genomicsandhealth.org/</ext-link></comment><ext-link ext-link-type="webcite" xlink:href="6TlGtW1e4"></ext-link></element-citation></ref><ref id="ref35"><label>35</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Moses</surname><given-names>C</given-names></name><name><surname>Celi</surname><given-names>LA</given-names></name><name><surname>Marshall</surname><given-names>J</given-names></name></person-group><article-title>Pharmacovigilance: an active surveillance system to proactively identify risks for adverse events</article-title><source>Popul Health Manag</source><year>2013</year><month>6</month><volume>16</volume><issue>3</issue><fpage>147</fpage><lpage>9</lpage><pub-id pub-id-type="doi">10.1089/pop.2012.0100</pub-id><pub-id pub-id-type="medline">23530466</pub-id><pub-id pub-id-type="pmid">23530466</pub-id></element-citation></ref><ref id="ref36"><label>36</label><element-citation publication-type="webpage"><person-group person-group-type="author"><collab>United States Food and Drug Administration</collab></person-group><source>Federal Register</source><date-in-citation>2014-09-16</date-in-citation><comment>Availability of Masked and De-identified Non-Summary Safety and Efficacy Data; Request for Comments<ext-link ext-link-type="uri" xlink:href="https://www.federalregister.gov/articles/2013/06/04/2013-13083/availability-of-masked-and-de-identified-non-summary-safety-and-efficacy-data-request-for-comments">https://www.federalregister.gov/articles/2013/06/04/2013-13083/availability-of-masked-and-de-identified-non-summary-safety-and-efficacy-data-request-for-comments</ext-link></comment><ext-link ext-link-type="webcite" xlink:href="6SdsUk8eG"></ext-link></element-citation></ref><ref id="ref37"><label>37</label><element-citation publication-type="webpage"><source>European Medicines Agency</source><date-in-citation>2014-09-16</date-in-citation><comment>Draft policy 70: Publication and access to clinical-trial data<ext-link ext-link-type="uri" xlink:href="http://www.ema.europa.eu/ema/doc_index.jsp?curl=pages/includes/document/document_detail.jsp?webContentId=WC500144730&murl=menus/document_library/document_library.jsp&mid=0b01ac058009a3dc">http://www.ema.europa.eu/ema/doc_index.jsp?curl=pages/includes/document/document_detail.jsp?webContentId=WC500144730&murl=menus/document_library/document_library.jsp&mid=0b01ac058009a3dc</ext-link></comment><ext-link ext-link-type="webcite" xlink:href="6SdtZTC7M"></ext-link></element-citation></ref><ref id="ref38"><label>38</label><element-citation publication-type="webpage"><source>The Royal Society</source><date-in-citation>2014-09-17</date-in-citation><comment>Science as an open enterprise final report<ext-link ext-link-type="uri" xlink:href="https://royalsociety.org/policy/projects/science-public-enterprise/report/">https://royalsociety.org/policy/projects/science-public-enterprise/report/</ext-link></comment><ext-link ext-link-type="webcite" xlink:href="6Sdws19wz"></ext-link></element-citation></ref><ref id="ref39"><label>39</label><element-citation publication-type="webpage"><source>AllTrials</source><date-in-citation>2014-09-17</date-in-citation><comment><ext-link ext-link-type="uri" xlink:href="http://www.alltrials.net/">http://www.alltrials.net/</ext-link></comment><ext-link ext-link-type="webcite" xlink:href="6Sdx8tBsL"></ext-link></element-citation></ref><ref id="ref40"><label>40</label><element-citation publication-type="webpage"><source>The OpenScience Project</source><date-in-citation>2014-09-17</date-in-citation><comment><ext-link ext-link-type="uri" xlink:href="http://www.openscience.org/blog/">http://www.openscience.org/blog/</ext-link></comment><ext-link ext-link-type="webcite" xlink:href="6SdxGg1YN"></ext-link></element-citation></ref><ref id="ref41"><label>41</label><element-citation publication-type="webpage"><source>Stanford School of Medicine</source><date-in-citation>2014-09-17</date-in-citation><comment>METRICS<ext-link ext-link-type="uri" xlink:href="http://med.stanford.edu/metrics/">http://med.stanford.edu/metrics/</ext-link></comment><ext-link ext-link-type="webcite" xlink:href="6SdxOQNyT"></ext-link></element-citation></ref></ref-list></back></pmc></record>Pour manipuler ce document sous Unix (Dilib)
EXPLOR_STEP=$WICRI_ROOT/Ticri/CIDE/explor/TelematiV1/Data/Pmc/Corpus
HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 000304 | SxmlIndent | more
Ou
HfdSelect -h $EXPLOR_AREA/Data/Pmc/Corpus/biblio.hfd -nk 000304 | SxmlIndent | more
Pour mettre un lien sur cette page dans le réseau Wicri
{{Explor lien
|wiki= Ticri/CIDE
|area= TelematiV1
|flux= Pmc
|étape= Corpus
|type= RBID
|clé=
|texte=
}}
| This area was generated with Dilib version V0.6.31. |  |