Tools and techniques for computational reproducibility
Identifieur interne : 000741 ( Ncbi/Merge ); précédent : 000740; suivant : 000742Tools and techniques for computational reproducibility
Auteurs : Stephen R. Piccolo [États-Unis] ; Michael B. Frampton [États-Unis]Source :
- GigaScience [ 2047-217X ] ; 2016.
Abstract
When reporting research findings, scientists document the steps they followed so that others can verify and build upon the research. When those steps have been described in sufficient detail that others can retrace the steps and obtain similar results, the research is said to be reproducible. Computers play a vital role in many research disciplines and present both opportunities and challenges for reproducibility. Computers can be programmed to execute analysis tasks, and those programs can be repeated and shared with others. The deterministic nature of most computer programs means that the same analysis tasks, applied to the same data, will often produce the same outputs. However, in practice, computational findings often cannot be reproduced because of complexities in how software is packaged, installed, and executed—and because of limitations associated with how scientists document analysis steps. Many tools and techniques are available to help overcome these challenges; here we describe seven such strategies. With a broad scientific audience in mind, we describe the strengths and limitations of each approach, as well as the circumstances under which each might be applied. No single strategy is sufficient for every scenario; thus we emphasize that it is often useful to combine approaches.
The online version of this article (doi:10.1186/s13742-016-0135-4) contains supplementary material, which is available to authorized users.
Url:
DOI: 10.1186/s13742-016-0135-4
PubMed: 27401684
PubMed Central: 4940747
Links toward previous steps (curation, corpus...)
- to stream Pmc, to step Corpus: 000183
- to stream Pmc, to step Curation: 000183
- to stream Pmc, to step Checkpoint: 000006
Links to Exploration step
PMC:4940747Le document en format XML
<record><TEI><teiHeader><fileDesc><titleStmt><title xml:lang="en">Tools and techniques for computational reproducibility</title>
<author><name sortKey="Piccolo, Stephen R" sort="Piccolo, Stephen R" uniqKey="Piccolo S" first="Stephen R." last="Piccolo">Stephen R. Piccolo</name>
<affiliation wicri:level="2"><nlm:aff id="Aff1">Department of Biology, Brigham Young University, Provo, UT 84602 USA</nlm:aff>
<country xml:lang="fr">États-Unis</country>
<placeName><region type="state">Utah</region>
</placeName>
<wicri:cityArea>Department of Biology, Brigham Young University, Provo</wicri:cityArea>
</affiliation>
</author>
<author><name sortKey="Frampton, Michael B" sort="Frampton, Michael B" uniqKey="Frampton M" first="Michael B." last="Frampton">Michael B. Frampton</name>
<affiliation wicri:level="2"><nlm:aff id="Aff2">Department of Computer Science, Brigham Young University, Provo, UT USA</nlm:aff>
<country>États-Unis</country>
<placeName><region type="state">Utah</region>
</placeName>
<wicri:cityArea>Department of Computer Science, Brigham Young University, Provo</wicri:cityArea>
</affiliation>
</author>
</titleStmt>
<publicationStmt><idno type="wicri:source">PMC</idno>
<idno type="pmid">27401684</idno>
<idno type="pmc">4940747</idno>
<idno type="url">http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4940747</idno>
<idno type="RBID">PMC:4940747</idno>
<idno type="doi">10.1186/s13742-016-0135-4</idno>
<date when="2016">2016</date>
<idno type="wicri:Area/Pmc/Corpus">000183</idno>
<idno type="wicri:Area/Pmc/Curation">000183</idno>
<idno type="wicri:Area/Pmc/Checkpoint">000006</idno>
<idno type="wicri:Area/Ncbi/Merge">000741</idno>
</publicationStmt>
<sourceDesc><biblStruct><analytic><title xml:lang="en" level="a" type="main">Tools and techniques for computational reproducibility</title>
<author><name sortKey="Piccolo, Stephen R" sort="Piccolo, Stephen R" uniqKey="Piccolo S" first="Stephen R." last="Piccolo">Stephen R. Piccolo</name>
<affiliation wicri:level="2"><nlm:aff id="Aff1">Department of Biology, Brigham Young University, Provo, UT 84602 USA</nlm:aff>
<country xml:lang="fr">États-Unis</country>
<placeName><region type="state">Utah</region>
</placeName>
<wicri:cityArea>Department of Biology, Brigham Young University, Provo</wicri:cityArea>
</affiliation>
</author>
<author><name sortKey="Frampton, Michael B" sort="Frampton, Michael B" uniqKey="Frampton M" first="Michael B." last="Frampton">Michael B. Frampton</name>
<affiliation wicri:level="2"><nlm:aff id="Aff2">Department of Computer Science, Brigham Young University, Provo, UT USA</nlm:aff>
<country>États-Unis</country>
<placeName><region type="state">Utah</region>
</placeName>
<wicri:cityArea>Department of Computer Science, Brigham Young University, Provo</wicri:cityArea>
</affiliation>
</author>
</analytic>
<series><title level="j">GigaScience</title>
<idno type="eISSN">2047-217X</idno>
<imprint><date when="2016">2016</date>
</imprint>
</series>
</biblStruct>
</sourceDesc>
</fileDesc>
<profileDesc><textClass></textClass>
</profileDesc>
</teiHeader>
<front><div type="abstract" xml:lang="en"><p>When reporting research findings, scientists document the steps they followed so that others can verify and build upon the research. When those steps have been described in sufficient detail that others can retrace the steps and obtain similar results, the research is said to be reproducible. Computers play a vital role in many research disciplines and present both opportunities and challenges for reproducibility. Computers can be programmed to execute analysis tasks, and those programs can be repeated and shared with others. The deterministic nature of most computer programs means that the same analysis tasks, applied to the same data, will often produce the same outputs. However, in practice, computational findings often cannot be reproduced because of complexities in how software is packaged, installed, and executed—and because of limitations associated with how scientists document analysis steps. Many tools and techniques are available to help overcome these challenges; here we describe seven such strategies. With a broad scientific audience in mind, we describe the strengths and limitations of each approach, as well as the circumstances under which each might be applied. No single strategy is sufficient for every scenario; thus we emphasize that it is often useful to combine approaches.</p>
<sec><title>Electronic supplementary material</title>
<p>The online version of this article (doi:10.1186/s13742-016-0135-4) contains supplementary material, which is available to authorized users.</p>
</sec>
</div>
</front>
<back><div1 type="bibliography"><listBibl><biblStruct><analytic><author><name sortKey="Fisher, Ra" uniqKey="Fisher R">RA Fisher</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Popper, Kr" uniqKey="Popper K">KR Popper</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Peng, Rd" uniqKey="Peng R">RD Peng</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Russell, Jf" uniqKey="Russell J">JF Russell</name>
</author>
</analytic>
</biblStruct>
<biblStruct></biblStruct>
<biblStruct></biblStruct>
<biblStruct></biblStruct>
<biblStruct></biblStruct>
<biblStruct><analytic><author><name sortKey="Wilson, G" uniqKey="Wilson G">G Wilson</name>
</author>
<author><name sortKey="Aruliah, Da" uniqKey="Aruliah D">DA Aruliah</name>
</author>
<author><name sortKey="Brown, Ct" uniqKey="Brown C">CT Brown</name>
</author>
<author><name sortKey="Chue Hong, Np" uniqKey="Chue Hong N">NP Chue Hong</name>
</author>
<author><name sortKey="Davis, M" uniqKey="Davis M">M Davis</name>
</author>
<author><name sortKey="Guy, Rt" uniqKey="Guy R">RT Guy</name>
</author>
</analytic>
</biblStruct>
<biblStruct></biblStruct>
<biblStruct></biblStruct>
<biblStruct><analytic><author><name sortKey="Sacks, J" uniqKey="Sacks J">J Sacks</name>
</author>
<author><name sortKey="Welch, Wj" uniqKey="Welch W">WJ Welch</name>
</author>
<author><name sortKey="Mitchell, Tj" uniqKey="Mitchell T">TJ Mitchell</name>
</author>
<author><name sortKey="Wynn, Hp" uniqKey="Wynn H">HP Wynn</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Garijo, D" uniqKey="Garijo D">D Garijo</name>
</author>
<author><name sortKey="Kinnings, S" uniqKey="Kinnings S">S Kinnings</name>
</author>
<author><name sortKey="Xie, L" uniqKey="Xie L">L Xie</name>
</author>
<author><name sortKey="Xie, L" uniqKey="Xie L">L Xie</name>
</author>
<author><name sortKey="Zhang, Y" uniqKey="Zhang Y">Y Zhang</name>
</author>
<author><name sortKey="Bourne, Pe" uniqKey="Bourne P">PE Bourne</name>
</author>
</analytic>
</biblStruct>
<biblStruct></biblStruct>
<biblStruct><analytic><author><name sortKey="Vandewalle, P" uniqKey="Vandewalle P">P Vandewalle</name>
</author>
<author><name sortKey="Barrenetxea, G" uniqKey="Barrenetxea G">G Barrenetxea</name>
</author>
<author><name sortKey="Jovanovic, I" uniqKey="Jovanovic I">I Jovanovic</name>
</author>
<author><name sortKey="Ridolfi, A" uniqKey="Ridolfi A">A Ridolfi</name>
</author>
<author><name sortKey="Vetterli, M" uniqKey="Vetterli M">M Vetterli</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Cassey, P" uniqKey="Cassey P">P Cassey</name>
</author>
<author><name sortKey="Cassey, P" uniqKey="Cassey P">P Cassey</name>
</author>
<author><name sortKey="Blackburn, T" uniqKey="Blackburn T">T Blackburn</name>
</author>
<author><name sortKey="Blackburn, T" uniqKey="Blackburn T">T Blackburn</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Murphy, Jm" uniqKey="Murphy J">JM Murphy</name>
</author>
<author><name sortKey="Sexton, Dmh" uniqKey="Sexton D">DMH Sexton</name>
</author>
<author><name sortKey="Barnett, Dn" uniqKey="Barnett D">DN Barnett</name>
</author>
<author><name sortKey="Jones, Gs" uniqKey="Jones G">GS Jones</name>
</author>
<author><name sortKey="Webb, Mj" uniqKey="Webb M">MJ Webb</name>
</author>
<author><name sortKey="Collins, M" uniqKey="Collins M">M Collins</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Mccarthy, Dj" uniqKey="Mccarthy D">DJ McCarthy</name>
</author>
<author><name sortKey="Humburg, P" uniqKey="Humburg P">P Humburg</name>
</author>
<author><name sortKey="Kanapin, A" uniqKey="Kanapin A">A Kanapin</name>
</author>
<author><name sortKey="Rivas, Ma" uniqKey="Rivas M">MA Rivas</name>
</author>
<author><name sortKey="Gaulton, K" uniqKey="Gaulton K">K Gaulton</name>
</author>
<author><name sortKey="Cazier, J B" uniqKey="Cazier J">J-B Cazier</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Neuman, Ja" uniqKey="Neuman J">JA Neuman</name>
</author>
<author><name sortKey="Isakov, O" uniqKey="Isakov O">O Isakov</name>
</author>
<author><name sortKey="Shomron, N" uniqKey="Shomron N">N Shomron</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Bradnam, Kr" uniqKey="Bradnam K">KR Bradnam</name>
</author>
<author><name sortKey="Fass, Jn" uniqKey="Fass J">JN Fass</name>
</author>
<author><name sortKey="Alexandrov, A" uniqKey="Alexandrov A">A Alexandrov</name>
</author>
<author><name sortKey="Baranay, P" uniqKey="Baranay P">P Baranay</name>
</author>
<author><name sortKey="Bechner, M" uniqKey="Bechner M">M Bechner</name>
</author>
<author><name sortKey="Birol, I" uniqKey="Birol I">I Birol</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Bilal, E" uniqKey="Bilal E">E Bilal</name>
</author>
<author><name sortKey="Dutkowski, J" uniqKey="Dutkowski J">J Dutkowski</name>
</author>
<author><name sortKey="Guinney, J" uniqKey="Guinney J">J Guinney</name>
</author>
<author><name sortKey="Jang, Is" uniqKey="Jang I">IS Jang</name>
</author>
<author><name sortKey="Logsdon, Ba" uniqKey="Logsdon B">BA Logsdon</name>
</author>
<author><name sortKey="Pandey, G" uniqKey="Pandey G">G Pandey</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Gronenschild, Ehbm" uniqKey="Gronenschild E">EHBM Gronenschild</name>
</author>
<author><name sortKey="Habets, P" uniqKey="Habets P">P Habets</name>
</author>
<author><name sortKey="Jacobs, Hil" uniqKey="Jacobs H">HIL Jacobs</name>
</author>
<author><name sortKey="Mengelers, R" uniqKey="Mengelers R">R Mengelers</name>
</author>
<author><name sortKey="Rozendaal, N" uniqKey="Rozendaal N">N Rozendaal</name>
</author>
<author><name sortKey="Van Os, J" uniqKey="Van Os J">J van Os</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Moskvin, Ov" uniqKey="Moskvin O">OV Moskvin</name>
</author>
<author><name sortKey="Mcilwain, S" uniqKey="Mcilwain S">S McIlwain</name>
</author>
<author><name sortKey="Ong, Im" uniqKey="Ong I">IM Ong</name>
</author>
</analytic>
</biblStruct>
<biblStruct></biblStruct>
<biblStruct><analytic><author><name sortKey="Michael, Cm" uniqKey="Michael C">CM Michael</name>
</author>
<author><name sortKey="Nass, Sj" uniqKey="Nass S">SJ Nass</name>
</author>
<author><name sortKey="Omenn, Gs" uniqKey="Omenn G">GS Omenn</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Collins, Fs" uniqKey="Collins F">FS Collins</name>
</author>
<author><name sortKey="Tabak, L A" uniqKey="Tabak L">L a Tabak</name>
</author>
</analytic>
</biblStruct>
<biblStruct></biblStruct>
<biblStruct><analytic><author><name sortKey="Leveque, Rj" uniqKey="Leveque R">RJ LeVeque</name>
</author>
<author><name sortKey="Mitchell, Im" uniqKey="Mitchell I">IM Mitchell</name>
</author>
<author><name sortKey="Stodden, V" uniqKey="Stodden V">V Stodden</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Stodden, V" uniqKey="Stodden V">V Stodden</name>
</author>
<author><name sortKey="Guo, P" uniqKey="Guo P">P Guo</name>
</author>
<author><name sortKey="Ma, Z" uniqKey="Ma Z">Z Ma</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Morin, A" uniqKey="Morin A">A Morin</name>
</author>
<author><name sortKey="Urban, J" uniqKey="Urban J">J Urban</name>
</author>
<author><name sortKey="Adams, Pd" uniqKey="Adams P">PD Adams</name>
</author>
<author><name sortKey="Foster, I" uniqKey="Foster I">I Foster</name>
</author>
<author><name sortKey="Sali, A" uniqKey="Sali A">A Sali</name>
</author>
<author><name sortKey="Baker, D" uniqKey="Baker D">D Baker</name>
</author>
</analytic>
</biblStruct>
<biblStruct></biblStruct>
<biblStruct><analytic><author><name sortKey="Ioannidis, Jp A" uniqKey="Ioannidis J">JP a Ioannidis</name>
</author>
<author><name sortKey="Allison, Db" uniqKey="Allison D">DB Allison</name>
</author>
<author><name sortKey="Ball, C A" uniqKey="Ball C">C a Ball</name>
</author>
<author><name sortKey="Coulibaly, I" uniqKey="Coulibaly I">I Coulibaly</name>
</author>
<author><name sortKey="Cui, X" uniqKey="Cui X">X Cui</name>
</author>
<author><name sortKey="Culhane, Ac" uniqKey="Culhane A">AC Culhane</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Nekrutenko, A" uniqKey="Nekrutenko A">A Nekrutenko</name>
</author>
<author><name sortKey="Taylor, J" uniqKey="Taylor J">J Taylor</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Baggerly, K A" uniqKey="Baggerly K">K a Baggerly</name>
</author>
<author><name sortKey="Coombes, Kr" uniqKey="Coombes K">KR Coombes</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Decullier, E" uniqKey="Decullier E">E Decullier</name>
</author>
<author><name sortKey="Huot, L" uniqKey="Huot L">L Huot</name>
</author>
<author><name sortKey="Samson, G" uniqKey="Samson G">G Samson</name>
</author>
<author><name sortKey="Maisonneuve, H" uniqKey="Maisonneuve H">H Maisonneuve</name>
</author>
</analytic>
</biblStruct>
<biblStruct></biblStruct>
<biblStruct><analytic><author><name sortKey="Stodden, V" uniqKey="Stodden V">V Stodden</name>
</author>
<author><name sortKey="Miguez, S" uniqKey="Miguez S">S Miguez</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Ravel, J" uniqKey="Ravel J">J Ravel</name>
</author>
<author><name sortKey="Wommack, Ke" uniqKey="Wommack K">KE Wommack</name>
</author>
</analytic>
</biblStruct>
<biblStruct></biblStruct>
<biblStruct><analytic><author><name sortKey="Birney, E" uniqKey="Birney E">E Birney</name>
</author>
<author><name sortKey="Hudson, Tj" uniqKey="Hudson T">TJ Hudson</name>
</author>
<author><name sortKey="Green, Ed" uniqKey="Green E">ED Green</name>
</author>
<author><name sortKey="Gunter, C" uniqKey="Gunter C">C Gunter</name>
</author>
<author><name sortKey="Eddy, S" uniqKey="Eddy S">S Eddy</name>
</author>
<author><name sortKey="Rogers, J" uniqKey="Rogers J">J Rogers</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Hothorn, T" uniqKey="Hothorn T">T Hothorn</name>
</author>
<author><name sortKey="Leisch, F" uniqKey="Leisch F">F Leisch</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Schofield, Pn" uniqKey="Schofield P">PN Schofield</name>
</author>
<author><name sortKey="Bubela, T" uniqKey="Bubela T">T Bubela</name>
</author>
<author><name sortKey="Weaver, T" uniqKey="Weaver T">T Weaver</name>
</author>
<author><name sortKey="Portilla, L" uniqKey="Portilla L">L Portilla</name>
</author>
<author><name sortKey="Brown, Sd" uniqKey="Brown S">SD Brown</name>
</author>
<author><name sortKey="Hancock, Jm" uniqKey="Hancock J">JM Hancock</name>
</author>
</analytic>
</biblStruct>
<biblStruct></biblStruct>
<biblStruct><analytic><author><name sortKey="Johnson, Ve" uniqKey="Johnson V">VE Johnson</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Halsey, Lg" uniqKey="Halsey L">LG Halsey</name>
</author>
<author><name sortKey="Curran Everett, D" uniqKey="Curran Everett D">D Curran-everett</name>
</author>
<author><name sortKey="Vowler, Sl" uniqKey="Vowler S">SL Vowler</name>
</author>
<author><name sortKey="Drummond, Gb" uniqKey="Drummond G">GB Drummond</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Wilson, G" uniqKey="Wilson G">G Wilson</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Sandve, Gk" uniqKey="Sandve G">GK Sandve</name>
</author>
<author><name sortKey="Nekrutenko, A" uniqKey="Nekrutenko A">A Nekrutenko</name>
</author>
<author><name sortKey="Taylor, J" uniqKey="Taylor J">J Taylor</name>
</author>
<author><name sortKey="Hovig, E" uniqKey="Hovig E">E Hovig</name>
</author>
</analytic>
</biblStruct>
<biblStruct></biblStruct>
<biblStruct></biblStruct>
<biblStruct></biblStruct>
<biblStruct></biblStruct>
<biblStruct><analytic><author><name sortKey="Blischak, Jd" uniqKey="Blischak J">JD Blischak</name>
</author>
<author><name sortKey="Davenport, Er" uniqKey="Davenport E">ER Davenport</name>
</author>
<author><name sortKey="Wilson, G" uniqKey="Wilson G">G Wilson</name>
</author>
</analytic>
</biblStruct>
<biblStruct></biblStruct>
<biblStruct><analytic><author><name sortKey="Huber, W" uniqKey="Huber W">W Huber</name>
</author>
<author><name sortKey="Carey, Vj" uniqKey="Carey V">VJ Carey</name>
</author>
<author><name sortKey="Gentleman, R" uniqKey="Gentleman R">R Gentleman</name>
</author>
<author><name sortKey="Anders, S" uniqKey="Anders S">S Anders</name>
</author>
<author><name sortKey="Carlson, M" uniqKey="Carlson M">M Carlson</name>
</author>
<author><name sortKey="Carvalho, Bs" uniqKey="Carvalho B">BS Carvalho</name>
</author>
</analytic>
</biblStruct>
<biblStruct></biblStruct>
<biblStruct><analytic><author><name sortKey="T Th, G" uniqKey="T Th G">G Tóth</name>
</author>
<author><name sortKey="Sokolov, Iv" uniqKey="Sokolov I">IV Sokolov</name>
</author>
<author><name sortKey="Gombosi, Ti" uniqKey="Gombosi T">TI Gombosi</name>
</author>
<author><name sortKey="Chesney, Dr" uniqKey="Chesney D">DR Chesney</name>
</author>
<author><name sortKey="Clauer, Cr" uniqKey="Clauer C">CR Clauer</name>
</author>
<author><name sortKey="De Zeeuw, Dl" uniqKey="De Zeeuw D">DL De Zeeuw</name>
</author>
</analytic>
</biblStruct>
<biblStruct></biblStruct>
<biblStruct></biblStruct>
<biblStruct><analytic><author><name sortKey="Schneider, Ca" uniqKey="Schneider C">CA Schneider</name>
</author>
<author><name sortKey="Rasband, Ws" uniqKey="Rasband W">WS Rasband</name>
</author>
<author><name sortKey="Eliceiri, Kw" uniqKey="Eliceiri K">KW Eliceiri</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Schindelin, J" uniqKey="Schindelin J">J Schindelin</name>
</author>
<author><name sortKey="Arganda Carreras, I" uniqKey="Arganda Carreras I">I Arganda-Carreras</name>
</author>
<author><name sortKey="Frise, E" uniqKey="Frise E">E Frise</name>
</author>
<author><name sortKey="Kaynig, V" uniqKey="Kaynig V">V Kaynig</name>
</author>
<author><name sortKey="Longair, M" uniqKey="Longair M">M Longair</name>
</author>
<author><name sortKey="Pietzsch, T" uniqKey="Pietzsch T">T Pietzsch</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Biasini, M" uniqKey="Biasini M">M Biasini</name>
</author>
<author><name sortKey="Schmidt, T" uniqKey="Schmidt T">T Schmidt</name>
</author>
<author><name sortKey="Bienert, S" uniqKey="Bienert S">S Bienert</name>
</author>
<author><name sortKey="Mariani, V" uniqKey="Mariani V">V Mariani</name>
</author>
<author><name sortKey="Studer, G" uniqKey="Studer G">G Studer</name>
</author>
<author><name sortKey="Haas, J" uniqKey="Haas J">J Haas</name>
</author>
</analytic>
</biblStruct>
<biblStruct></biblStruct>
<biblStruct></biblStruct>
<biblStruct><analytic><author><name sortKey="Martin, Rc" uniqKey="Martin R">RC Martin</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Knuth, De" uniqKey="Knuth D">DE Knuth</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Perez, F" uniqKey="Perez F">F Pérez</name>
</author>
<author><name sortKey="Granger, Be" uniqKey="Granger B">BE Granger</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Shen, H" uniqKey="Shen H">H Shen</name>
</author>
</analytic>
</biblStruct>
<biblStruct></biblStruct>
<biblStruct></biblStruct>
<biblStruct><analytic><author><name sortKey="Gross, Am" uniqKey="Gross A">AM Gross</name>
</author>
<author><name sortKey="Orosco, Rk" uniqKey="Orosco R">RK Orosco</name>
</author>
<author><name sortKey="Shen, Jp" uniqKey="Shen J">JP Shen</name>
</author>
<author><name sortKey="Egloff, Am" uniqKey="Egloff A">AM Egloff</name>
</author>
<author><name sortKey="Carter, H" uniqKey="Carter H">H Carter</name>
</author>
<author><name sortKey="Hofree, M" uniqKey="Hofree M">M Hofree</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Ding, T" uniqKey="Ding T">T Ding</name>
</author>
<author><name sortKey="Schloss, Pd" uniqKey="Schloss P">PD Schloss</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Ram, Y" uniqKey="Ram Y">Y Ram</name>
</author>
<author><name sortKey="Hadany, L" uniqKey="Hadany L">L Hadany</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Meadow, Jf" uniqKey="Meadow J">JF Meadow</name>
</author>
<author><name sortKey="Altrichter, Ae" uniqKey="Altrichter A">AE Altrichter</name>
</author>
<author><name sortKey="Kembel, Sw" uniqKey="Kembel S">SW Kembel</name>
</author>
<author><name sortKey="Moriyama, M" uniqKey="Moriyama M">M Moriyama</name>
</author>
<author><name sortKey="O Onnor, Tk" uniqKey="O Onnor T">TK O’Connor</name>
</author>
<author><name sortKey="Womack, Am" uniqKey="Womack A">AM Womack</name>
</author>
</analytic>
</biblStruct>
<biblStruct></biblStruct>
<biblStruct></biblStruct>
<biblStruct></biblStruct>
<biblStruct><analytic><author><name sortKey="Gil, Y" uniqKey="Gil Y">Y Gil</name>
</author>
<author><name sortKey="Deelman, E" uniqKey="Deelman E">E Deelman</name>
</author>
<author><name sortKey="Ellisman, M" uniqKey="Ellisman M">M Ellisman</name>
</author>
<author><name sortKey="Fahringer, T" uniqKey="Fahringer T">T Fahringer</name>
</author>
<author><name sortKey="Fox, G" uniqKey="Fox G">G Fox</name>
</author>
<author><name sortKey="Gannon, D" uniqKey="Gannon D">D Gannon</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Giardine, B" uniqKey="Giardine B">B Giardine</name>
</author>
<author><name sortKey="Riemer, C" uniqKey="Riemer C">C Riemer</name>
</author>
<author><name sortKey="Hardison, Rc" uniqKey="Hardison R">RC Hardison</name>
</author>
<author><name sortKey="Burhans, R" uniqKey="Burhans R">R Burhans</name>
</author>
<author><name sortKey="Elnitski, L" uniqKey="Elnitski L">L Elnitski</name>
</author>
<author><name sortKey="Shah, P" uniqKey="Shah P">P Shah</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Goecks, J" uniqKey="Goecks J">J Goecks</name>
</author>
<author><name sortKey="Nekrutenko, A" uniqKey="Nekrutenko A">A Nekrutenko</name>
</author>
<author><name sortKey="Taylor, J" uniqKey="Taylor J">J Taylor</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Afgan, E" uniqKey="Afgan E">E Afgan</name>
</author>
<author><name sortKey="Baker, D" uniqKey="Baker D">D Baker</name>
</author>
<author><name sortKey="Coraor, N" uniqKey="Coraor N">N Coraor</name>
</author>
<author><name sortKey="Goto, H" uniqKey="Goto H">H Goto</name>
</author>
<author><name sortKey="Paul, Im" uniqKey="Paul I">IM Paul</name>
</author>
<author><name sortKey="Makova, Kd" uniqKey="Makova K">KD Makova</name>
</author>
</analytic>
</biblStruct>
<biblStruct></biblStruct>
<biblStruct></biblStruct>
<biblStruct><analytic><author><name sortKey="Lazarus, R" uniqKey="Lazarus R">R Lazarus</name>
</author>
<author><name sortKey="Kaspi, A" uniqKey="Kaspi A">A Kaspi</name>
</author>
<author><name sortKey="Ziemann, M" uniqKey="Ziemann M">M Ziemann</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Dudley, Jt" uniqKey="Dudley J">JT Dudley</name>
</author>
<author><name sortKey="Butte, Aj" uniqKey="Butte A">AJ Butte</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Hurley, Dg" uniqKey="Hurley D">DG Hurley</name>
</author>
<author><name sortKey="Budden, Dm" uniqKey="Budden D">DM Budden</name>
</author>
<author><name sortKey="Crampin, Ej" uniqKey="Crampin E">EJ Crampin</name>
</author>
</analytic>
</biblStruct>
<biblStruct></biblStruct>
<biblStruct><analytic><author><name sortKey="Howe, B" uniqKey="Howe B">B Howe</name>
</author>
</analytic>
</biblStruct>
<biblStruct></biblStruct>
<biblStruct></biblStruct>
<biblStruct><analytic><author><name sortKey="Krampis, K" uniqKey="Krampis K">K Krampis</name>
</author>
<author><name sortKey="Booth, T" uniqKey="Booth T">T Booth</name>
</author>
<author><name sortKey="Chapman, B" uniqKey="Chapman B">B Chapman</name>
</author>
<author><name sortKey="Tiwari, B" uniqKey="Tiwari B">B Tiwari</name>
</author>
<author><name sortKey="Bicak, M" uniqKey="Bicak M">M Bicak</name>
</author>
<author><name sortKey="Field, D" uniqKey="Field D">D Field</name>
</author>
</analytic>
</biblStruct>
<biblStruct></biblStruct>
<biblStruct></biblStruct>
<biblStruct><analytic><author><name sortKey="Eglen, Sj" uniqKey="Eglen S">SJ Eglen</name>
</author>
<author><name sortKey="Weeks, M" uniqKey="Weeks M">M Weeks</name>
</author>
<author><name sortKey="Jessop, M" uniqKey="Jessop M">M Jessop</name>
</author>
<author><name sortKey="Simonotto, J" uniqKey="Simonotto J">J Simonotto</name>
</author>
<author><name sortKey="Jackson, T" uniqKey="Jackson T">T Jackson</name>
</author>
<author><name sortKey="Sernagor, E" uniqKey="Sernagor E">E Sernagor</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Eglen, Sj" uniqKey="Eglen S">SJ Eglen</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Bremges, A" uniqKey="Bremges A">A Bremges</name>
</author>
<author><name sortKey="Maus, I" uniqKey="Maus I">I Maus</name>
</author>
<author><name sortKey="Belmann, P" uniqKey="Belmann P">P Belmann</name>
</author>
<author><name sortKey="Eikmeyer, F" uniqKey="Eikmeyer F">F Eikmeyer</name>
</author>
<author><name sortKey="Winkler, A" uniqKey="Winkler A">A Winkler</name>
</author>
<author><name sortKey="Albersmeier, A" uniqKey="Albersmeier A">A Albersmeier</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Belmann, P" uniqKey="Belmann P">P Belmann</name>
</author>
<author><name sortKey="Droge, J" uniqKey="Droge J">J Dröge</name>
</author>
<author><name sortKey="Bremges, A" uniqKey="Bremges A">A Bremges</name>
</author>
<author><name sortKey="Mchardy, Ac" uniqKey="Mchardy A">AC McHardy</name>
</author>
<author><name sortKey="Sczyrba, A" uniqKey="Sczyrba A">A Sczyrba</name>
</author>
<author><name sortKey="Barton, Md" uniqKey="Barton M">MD Barton</name>
</author>
</analytic>
</biblStruct>
<biblStruct></biblStruct>
<biblStruct><analytic><author><name sortKey="Hones, Mj" uniqKey="Hones M">MJ Hones</name>
</author>
</analytic>
</biblStruct>
<biblStruct></biblStruct>
<biblStruct></biblStruct>
<biblStruct></biblStruct>
<biblStruct><analytic><author><name sortKey="Donoho, Dl" uniqKey="Donoho D">DL Donoho</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Goldberg, D" uniqKey="Goldberg D">D Goldberg</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Shirts, M" uniqKey="Shirts M">M Shirts</name>
</author>
<author><name sortKey="Pande, Vs" uniqKey="Pande V">VS Pande</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Bird, I" uniqKey="Bird I">I Bird</name>
</author>
</analytic>
</biblStruct>
<biblStruct></biblStruct>
<biblStruct><analytic><author><name sortKey="Ransohoff, Df" uniqKey="Ransohoff D">DF Ransohoff</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Bild, Ah" uniqKey="Bild A">AH Bild</name>
</author>
<author><name sortKey="Chang, Jt" uniqKey="Chang J">JT Chang</name>
</author>
<author><name sortKey="Johnson, We" uniqKey="Johnson W">WE Johnson</name>
</author>
<author><name sortKey="Piccolo, Sr" uniqKey="Piccolo S">SR Piccolo</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Koster, J" uniqKey="Koster J">J Köster</name>
</author>
<author><name sortKey="Rahmann, S" uniqKey="Rahmann S">S Rahmann</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Sadedin, Sp" uniqKey="Sadedin S">SP Sadedin</name>
</author>
<author><name sortKey="Pope, B" uniqKey="Pope B">B Pope</name>
</author>
<author><name sortKey="Oshlack, A" uniqKey="Oshlack A">A Oshlack</name>
</author>
</analytic>
</biblStruct>
<biblStruct></biblStruct>
<biblStruct></biblStruct>
<biblStruct></biblStruct>
<biblStruct></biblStruct>
<biblStruct><analytic><author><name sortKey="Goff, Sa" uniqKey="Goff S">SA Goff</name>
</author>
<author><name sortKey="Vaughn, M" uniqKey="Vaughn M">M Vaughn</name>
</author>
<author><name sortKey="Mckay, S" uniqKey="Mckay S">S McKay</name>
</author>
<author><name sortKey="Lyons, E" uniqKey="Lyons E">E Lyons</name>
</author>
<author><name sortKey="Stapleton, Ae" uniqKey="Stapleton A">AE Stapleton</name>
</author>
<author><name sortKey="Gessler, D" uniqKey="Gessler D">D Gessler</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Reich, M" uniqKey="Reich M">M Reich</name>
</author>
<author><name sortKey="Liefeld, T" uniqKey="Liefeld T">T Liefeld</name>
</author>
<author><name sortKey="Gould, J" uniqKey="Gould J">J Gould</name>
</author>
<author><name sortKey="Lerner, J" uniqKey="Lerner J">J Lerner</name>
</author>
<author><name sortKey="Tamayo, P" uniqKey="Tamayo P">P Tamayo</name>
</author>
<author><name sortKey="Mesirov, Jp" uniqKey="Mesirov J">JP Mesirov</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Reich, M" uniqKey="Reich M">M Reich</name>
</author>
<author><name sortKey="Liefeld, J" uniqKey="Liefeld J">J Liefeld</name>
</author>
<author><name sortKey="Thorvaldsdottir, H" uniqKey="Thorvaldsdottir H">H Thorvaldsdottir</name>
</author>
<author><name sortKey="Ocana, M" uniqKey="Ocana M">M Ocana</name>
</author>
<author><name sortKey="Polk, E" uniqKey="Polk E">E Polk</name>
</author>
<author><name sortKey="Jang, D" uniqKey="Jang D">D Jang</name>
</author>
</analytic>
</biblStruct>
<biblStruct></biblStruct>
<biblStruct><analytic><author><name sortKey="Wolstencroft, K" uniqKey="Wolstencroft K">K Wolstencroft</name>
</author>
<author><name sortKey="Haines, R" uniqKey="Haines R">R Haines</name>
</author>
<author><name sortKey="Fellows, D" uniqKey="Fellows D">D Fellows</name>
</author>
<author><name sortKey="Williams, A" uniqKey="Williams A">A Williams</name>
</author>
<author><name sortKey="Withers, D" uniqKey="Withers D">D Withers</name>
</author>
<author><name sortKey="Owen, S" uniqKey="Owen S">S Owen</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Rex, De" uniqKey="Rex D">DE Rex</name>
</author>
<author><name sortKey="Ma, Jq" uniqKey="Ma J">JQ Ma</name>
</author>
<author><name sortKey="Toga, Aw" uniqKey="Toga A">AW Toga</name>
</author>
</analytic>
</biblStruct>
<biblStruct></biblStruct>
<biblStruct></biblStruct>
<biblStruct></biblStruct>
<biblStruct></biblStruct>
<biblStruct></biblStruct>
<biblStruct></biblStruct>
<biblStruct></biblStruct>
</listBibl>
</div1>
</back>
</TEI>
<pmc article-type="research-article"><pmc-dir>properties open_access</pmc-dir>
<front><journal-meta><journal-id journal-id-type="nlm-ta">Gigascience</journal-id>
<journal-id journal-id-type="iso-abbrev">Gigascience</journal-id>
<journal-title-group><journal-title>GigaScience</journal-title>
</journal-title-group>
<issn pub-type="epub">2047-217X</issn>
<publisher><publisher-name>BioMed Central</publisher-name>
<publisher-loc>London</publisher-loc>
</publisher>
</journal-meta>
<article-meta><article-id pub-id-type="pmid">27401684</article-id>
<article-id pub-id-type="pmc">4940747</article-id>
<article-id pub-id-type="publisher-id">135</article-id>
<article-id pub-id-type="doi">10.1186/s13742-016-0135-4</article-id>
<article-categories><subj-group subj-group-type="heading"><subject>Review</subject>
</subj-group>
</article-categories>
<title-group><article-title>Tools and techniques for computational reproducibility</article-title>
</title-group>
<contrib-group><contrib contrib-type="author" corresp="yes"><name><surname>Piccolo</surname>
<given-names>Stephen R.</given-names>
</name>
<address><phone>+1 801-422-7116</phone>
<email>stephen_piccolo@byu.edu</email>
</address>
<xref ref-type="aff" rid="Aff1">1</xref>
</contrib>
<contrib contrib-type="author"><name><surname>Frampton</surname>
<given-names>Michael B.</given-names>
</name>
<xref ref-type="aff" rid="Aff2">2</xref>
</contrib>
<aff id="Aff1"><label>1</label>
Department of Biology, Brigham Young University, Provo, UT 84602 USA</aff>
<aff id="Aff2"><label>2</label>
Department of Computer Science, Brigham Young University, Provo, UT USA</aff>
</contrib-group>
<pub-date pub-type="epub"><day>11</day>
<month>7</month>
<year>2016</year>
</pub-date>
<pub-date pub-type="pmc-release"><day>11</day>
<month>7</month>
<year>2016</year>
</pub-date>
<pub-date pub-type="collection"><year>2016</year>
</pub-date>
<volume>5</volume>
<elocation-id>30</elocation-id>
<permissions><copyright-statement>© The Author(s). 2016</copyright-statement>
<license license-type="OpenAccess"><license-p><bold>Open Access</bold>
This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (<ext-link ext-link-type="uri" xlink:href="http://creativecommons.org/licenses/by/4.0/">http://creativecommons.org/licenses/by/4.0/</ext-link>
), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (<ext-link ext-link-type="uri" xlink:href="http://creativecommons.org/publicdomain/zero/1.0/">http://creativecommons.org/publicdomain/zero/1.0/</ext-link>
) applies to the data made available in this article, unless otherwise stated.</license-p>
</license>
</permissions>
<abstract id="Abs1"><p>When reporting research findings, scientists document the steps they followed so that others can verify and build upon the research. When those steps have been described in sufficient detail that others can retrace the steps and obtain similar results, the research is said to be reproducible. Computers play a vital role in many research disciplines and present both opportunities and challenges for reproducibility. Computers can be programmed to execute analysis tasks, and those programs can be repeated and shared with others. The deterministic nature of most computer programs means that the same analysis tasks, applied to the same data, will often produce the same outputs. However, in practice, computational findings often cannot be reproduced because of complexities in how software is packaged, installed, and executed—and because of limitations associated with how scientists document analysis steps. Many tools and techniques are available to help overcome these challenges; here we describe seven such strategies. With a broad scientific audience in mind, we describe the strengths and limitations of each approach, as well as the circumstances under which each might be applied. No single strategy is sufficient for every scenario; thus we emphasize that it is often useful to combine approaches.</p>
<sec><title>Electronic supplementary material</title>
<p>The online version of this article (doi:10.1186/s13742-016-0135-4) contains supplementary material, which is available to authorized users.</p>
</sec>
</abstract>
<kwd-group xml:lang="en"><title>Keywords</title>
<kwd>Computational reproducibility</kwd>
<kwd>Practice of science</kwd>
<kwd>Literate programming</kwd>
<kwd>Virtualization</kwd>
<kwd>Software containers</kwd>
<kwd>Software frameworks</kwd>
</kwd-group>
<custom-meta-group><custom-meta><meta-name>issue-copyright-statement</meta-name>
<meta-value>© The Author(s) 2016</meta-value>
</custom-meta>
</custom-meta-group>
</article-meta>
</front>
</pmc>
<affiliations><list><country><li>États-Unis</li>
</country>
<region><li>Utah</li>
</region>
</list>
<tree><country name="États-Unis"><region name="Utah"><name sortKey="Piccolo, Stephen R" sort="Piccolo, Stephen R" uniqKey="Piccolo S" first="Stephen R." last="Piccolo">Stephen R. Piccolo</name>
</region>
<name sortKey="Frampton, Michael B" sort="Frampton, Michael B" uniqKey="Frampton M" first="Michael B." last="Frampton">Michael B. Frampton</name>
</country>
</tree>
</affiliations>
</record>
Pour manipuler ce document sous Unix (Dilib)
EXPLOR_STEP=$WICRI_ROOT/Ticri/CIDE/explor/CyberinfraV1/Data/Ncbi/Merge
HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 000741 | SxmlIndent | more
Ou
HfdSelect -h $EXPLOR_AREA/Data/Ncbi/Merge/biblio.hfd -nk 000741 | SxmlIndent | more
Pour mettre un lien sur cette page dans le réseau Wicri
{{Explor lien |wiki= Ticri/CIDE |area= CyberinfraV1 |flux= Ncbi |étape= Merge |type= RBID |clé= PMC:4940747 |texte= Tools and techniques for computational reproducibility }}
Pour générer des pages wiki
HfdIndexSelect -h $EXPLOR_AREA/Data/Ncbi/Merge/RBID.i -Sk "pubmed:27401684" \ | HfdSelect -Kh $EXPLOR_AREA/Data/Ncbi/Merge/biblio.hfd \ | NlmPubMed2Wicri -a CyberinfraV1
![]() | This area was generated with Dilib version V0.6.25. | ![]() |