A tutorial on sensitivity analyses in clinical trials: the what, why, when and how
Identifieur interne : 000489 ( Pmc/Corpus ); précédent : 000488; suivant : 000490A tutorial on sensitivity analyses in clinical trials: the what, why, when and how
Auteurs : Lehana Thabane ; Lawrence Mbuagbaw ; Shiyuan Zhang ; Zainab Samaan ; Maura Marcucci ; Chenglin Ye ; Marroon Thabane ; Lora Giangregorio ; Brittany Dennis ; Daisy Kosa ; Victoria Borg Debono ; Rejane Dillenburg ; Vincent Fruci ; Monica Bawor ; Juneyoung Lee ; George Wells ; Charles H. GoldsmithSource :
- BMC Medical Research Methodology [ 1471-2288 ] ; 2013.
Abstract
Sensitivity analyses play a crucial role in assessing the robustness of the findings or conclusions based on primary analyses of data in clinical trials. They are a critical way to assess the impact, effect or influence of key assumptions or variations—such as different methods of analysis, definitions of outcomes, protocol deviations, missing data, and outliers—on the overall conclusions of a study.
The current paper is the second in a series of tutorial-type manuscripts intended to discuss and clarify aspects related to key methodological issues in the design and analysis of clinical trials.
In this paper we will provide a detailed exploration of the key aspects of sensitivity analyses including: 1) what sensitivity analyses are, why they are needed, and how often they are used in practice; 2) the different types of sensitivity analyses that one can do, with examples from the literature; 3) some frequently asked questions about sensitivity analyses; and 4) some suggestions on how to report the results of sensitivity analyses in clinical trials.
When reporting on a clinical trial, we recommend including planned or
Url:
DOI: 10.1186/1471-2288-13-92
PubMed: 23855337
PubMed Central: 3720188
Links to Exploration step
PMC:3720188Le document en format XML
<record><TEI><teiHeader><fileDesc><titleStmt><title xml:lang="en">A tutorial on sensitivity analyses in clinical trials: the what, why, when and how</title>
<author><name sortKey="Thabane, Lehana" sort="Thabane, Lehana" uniqKey="Thabane L" first="Lehana" last="Thabane">Lehana Thabane</name>
<affiliation><nlm:aff id="I1">Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, ON, Canada</nlm:aff>
</affiliation>
<affiliation><nlm:aff id="I2">Departments of Pediatrics and Anesthesia, McMaster University, Hamilton, ON, Canada</nlm:aff>
</affiliation>
<affiliation><nlm:aff id="I3">Center for Evaluation of Medicine, St Joseph’s Healthcare Hamilton, Hamilton, ON, Canada</nlm:aff>
</affiliation>
<affiliation><nlm:aff id="I4">Biostatistics Unit, Father Sean O’Sullivan Research Center, St Joseph’s Healthcare Hamilton, Hamilton, ON, Canada</nlm:aff>
</affiliation>
<affiliation><nlm:aff id="I5">Population Health Research Institute, Hamilton Health Sciences, Hamilton, ON, Canada</nlm:aff>
</affiliation>
</author>
<author><name sortKey="Mbuagbaw, Lawrence" sort="Mbuagbaw, Lawrence" uniqKey="Mbuagbaw L" first="Lawrence" last="Mbuagbaw">Lawrence Mbuagbaw</name>
<affiliation><nlm:aff id="I1">Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, ON, Canada</nlm:aff>
</affiliation>
<affiliation><nlm:aff id="I4">Biostatistics Unit, Father Sean O’Sullivan Research Center, St Joseph’s Healthcare Hamilton, Hamilton, ON, Canada</nlm:aff>
</affiliation>
</author>
<author><name sortKey="Zhang, Shiyuan" sort="Zhang, Shiyuan" uniqKey="Zhang S" first="Shiyuan" last="Zhang">Shiyuan Zhang</name>
<affiliation><nlm:aff id="I1">Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, ON, Canada</nlm:aff>
</affiliation>
<affiliation><nlm:aff id="I4">Biostatistics Unit, Father Sean O’Sullivan Research Center, St Joseph’s Healthcare Hamilton, Hamilton, ON, Canada</nlm:aff>
</affiliation>
</author>
<author><name sortKey="Samaan, Zainab" sort="Samaan, Zainab" uniqKey="Samaan Z" first="Zainab" last="Samaan">Zainab Samaan</name>
<affiliation><nlm:aff id="I1">Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, ON, Canada</nlm:aff>
</affiliation>
<affiliation><nlm:aff id="I6">Department of Psychiatry and Behavioral Neurosciences, McMaster University, Hamilton, ON, Canada</nlm:aff>
</affiliation>
<affiliation><nlm:aff id="I7">Population Genomics Program, McMaster University, Hamilton, ON, Canada</nlm:aff>
</affiliation>
</author>
<author><name sortKey="Marcucci, Maura" sort="Marcucci, Maura" uniqKey="Marcucci M" first="Maura" last="Marcucci">Maura Marcucci</name>
<affiliation><nlm:aff id="I1">Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, ON, Canada</nlm:aff>
</affiliation>
<affiliation><nlm:aff id="I4">Biostatistics Unit, Father Sean O’Sullivan Research Center, St Joseph’s Healthcare Hamilton, Hamilton, ON, Canada</nlm:aff>
</affiliation>
</author>
<author><name sortKey="Ye, Chenglin" sort="Ye, Chenglin" uniqKey="Ye C" first="Chenglin" last="Ye">Chenglin Ye</name>
<affiliation><nlm:aff id="I1">Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, ON, Canada</nlm:aff>
</affiliation>
<affiliation><nlm:aff id="I4">Biostatistics Unit, Father Sean O’Sullivan Research Center, St Joseph’s Healthcare Hamilton, Hamilton, ON, Canada</nlm:aff>
</affiliation>
</author>
<author><name sortKey="Thabane, Marroon" sort="Thabane, Marroon" uniqKey="Thabane M" first="Marroon" last="Thabane">Marroon Thabane</name>
<affiliation><nlm:aff id="I1">Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, ON, Canada</nlm:aff>
</affiliation>
<affiliation><nlm:aff id="I8">GSK, Mississauga, ON, Canada</nlm:aff>
</affiliation>
</author>
<author><name sortKey="Giangregorio, Lora" sort="Giangregorio, Lora" uniqKey="Giangregorio L" first="Lora" last="Giangregorio">Lora Giangregorio</name>
<affiliation><nlm:aff id="I9">Department of Kinesiology, University of Waterloo, Waterloo, ON, Canada</nlm:aff>
</affiliation>
</author>
<author><name sortKey="Dennis, Brittany" sort="Dennis, Brittany" uniqKey="Dennis B" first="Brittany" last="Dennis">Brittany Dennis</name>
<affiliation><nlm:aff id="I1">Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, ON, Canada</nlm:aff>
</affiliation>
<affiliation><nlm:aff id="I4">Biostatistics Unit, Father Sean O’Sullivan Research Center, St Joseph’s Healthcare Hamilton, Hamilton, ON, Canada</nlm:aff>
</affiliation>
</author>
<author><name sortKey="Kosa, Daisy" sort="Kosa, Daisy" uniqKey="Kosa D" first="Daisy" last="Kosa">Daisy Kosa</name>
<affiliation><nlm:aff id="I1">Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, ON, Canada</nlm:aff>
</affiliation>
<affiliation><nlm:aff id="I4">Biostatistics Unit, Father Sean O’Sullivan Research Center, St Joseph’s Healthcare Hamilton, Hamilton, ON, Canada</nlm:aff>
</affiliation>
<affiliation><nlm:aff id="I10">Department of Nephrology, Toronto General Hospital, Toronto, ON, Canada</nlm:aff>
</affiliation>
</author>
<author><name sortKey="Debono, Victoria Borg" sort="Debono, Victoria Borg" uniqKey="Debono V" first="Victoria Borg" last="Debono">Victoria Borg Debono</name>
<affiliation><nlm:aff id="I1">Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, ON, Canada</nlm:aff>
</affiliation>
<affiliation><nlm:aff id="I4">Biostatistics Unit, Father Sean O’Sullivan Research Center, St Joseph’s Healthcare Hamilton, Hamilton, ON, Canada</nlm:aff>
</affiliation>
</author>
<author><name sortKey="Dillenburg, Rejane" sort="Dillenburg, Rejane" uniqKey="Dillenburg R" first="Rejane" last="Dillenburg">Rejane Dillenburg</name>
<affiliation><nlm:aff id="I11">Department of Pediatrics, McMaster University, Hamilton, ON, Canada</nlm:aff>
</affiliation>
</author>
<author><name sortKey="Fruci, Vincent" sort="Fruci, Vincent" uniqKey="Fruci V" first="Vincent" last="Fruci">Vincent Fruci</name>
<affiliation><nlm:aff id="I12">Michael G. DeGroote School of Medicine, McMaster University, Hamilton, ON, Canada</nlm:aff>
</affiliation>
</author>
<author><name sortKey="Bawor, Monica" sort="Bawor, Monica" uniqKey="Bawor M" first="Monica" last="Bawor">Monica Bawor</name>
<affiliation><nlm:aff id="I13">McMaster Integrative Neuroscience Discovery & Study (MiNDS) Program, McMaster University, Hamilton, ON, Canada</nlm:aff>
</affiliation>
</author>
<author><name sortKey="Lee, Juneyoung" sort="Lee, Juneyoung" uniqKey="Lee J" first="Juneyoung" last="Lee">Juneyoung Lee</name>
<affiliation><nlm:aff id="I14">Department of Biostatistics, Korea University, Seoul, Korea</nlm:aff>
</affiliation>
</author>
<author><name sortKey="Wells, George" sort="Wells, George" uniqKey="Wells G" first="George" last="Wells">George Wells</name>
<affiliation><nlm:aff id="I15">Department of Clinical Epidemiology, University of Ottawa, Ottawa, ON, Canada</nlm:aff>
</affiliation>
</author>
<author><name sortKey="Goldsmith, Charles H" sort="Goldsmith, Charles H" uniqKey="Goldsmith C" first="Charles H" last="Goldsmith">Charles H. Goldsmith</name>
<affiliation><nlm:aff id="I1">Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, ON, Canada</nlm:aff>
</affiliation>
<affiliation><nlm:aff id="I4">Biostatistics Unit, Father Sean O’Sullivan Research Center, St Joseph’s Healthcare Hamilton, Hamilton, ON, Canada</nlm:aff>
</affiliation>
<affiliation><nlm:aff id="I16">Faculty of Health Sciences, Simon Fraser University, Burnaby, BC, Canada</nlm:aff>
</affiliation>
</author>
</titleStmt>
<publicationStmt><idno type="wicri:source">PMC</idno>
<idno type="pmid">23855337</idno>
<idno type="pmc">3720188</idno>
<idno type="url">http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3720188</idno>
<idno type="RBID">PMC:3720188</idno>
<idno type="doi">10.1186/1471-2288-13-92</idno>
<date when="2013">2013</date>
<idno type="wicri:Area/Pmc/Corpus">000489</idno>
<idno type="wicri:explorRef" wicri:stream="Pmc" wicri:step="Corpus" wicri:corpus="PMC">000489</idno>
</publicationStmt>
<sourceDesc><biblStruct><analytic><title xml:lang="en" level="a" type="main">A tutorial on sensitivity analyses in clinical trials: the what, why, when and how</title>
<author><name sortKey="Thabane, Lehana" sort="Thabane, Lehana" uniqKey="Thabane L" first="Lehana" last="Thabane">Lehana Thabane</name>
<affiliation><nlm:aff id="I1">Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, ON, Canada</nlm:aff>
</affiliation>
<affiliation><nlm:aff id="I2">Departments of Pediatrics and Anesthesia, McMaster University, Hamilton, ON, Canada</nlm:aff>
</affiliation>
<affiliation><nlm:aff id="I3">Center for Evaluation of Medicine, St Joseph’s Healthcare Hamilton, Hamilton, ON, Canada</nlm:aff>
</affiliation>
<affiliation><nlm:aff id="I4">Biostatistics Unit, Father Sean O’Sullivan Research Center, St Joseph’s Healthcare Hamilton, Hamilton, ON, Canada</nlm:aff>
</affiliation>
<affiliation><nlm:aff id="I5">Population Health Research Institute, Hamilton Health Sciences, Hamilton, ON, Canada</nlm:aff>
</affiliation>
</author>
<author><name sortKey="Mbuagbaw, Lawrence" sort="Mbuagbaw, Lawrence" uniqKey="Mbuagbaw L" first="Lawrence" last="Mbuagbaw">Lawrence Mbuagbaw</name>
<affiliation><nlm:aff id="I1">Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, ON, Canada</nlm:aff>
</affiliation>
<affiliation><nlm:aff id="I4">Biostatistics Unit, Father Sean O’Sullivan Research Center, St Joseph’s Healthcare Hamilton, Hamilton, ON, Canada</nlm:aff>
</affiliation>
</author>
<author><name sortKey="Zhang, Shiyuan" sort="Zhang, Shiyuan" uniqKey="Zhang S" first="Shiyuan" last="Zhang">Shiyuan Zhang</name>
<affiliation><nlm:aff id="I1">Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, ON, Canada</nlm:aff>
</affiliation>
<affiliation><nlm:aff id="I4">Biostatistics Unit, Father Sean O’Sullivan Research Center, St Joseph’s Healthcare Hamilton, Hamilton, ON, Canada</nlm:aff>
</affiliation>
</author>
<author><name sortKey="Samaan, Zainab" sort="Samaan, Zainab" uniqKey="Samaan Z" first="Zainab" last="Samaan">Zainab Samaan</name>
<affiliation><nlm:aff id="I1">Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, ON, Canada</nlm:aff>
</affiliation>
<affiliation><nlm:aff id="I6">Department of Psychiatry and Behavioral Neurosciences, McMaster University, Hamilton, ON, Canada</nlm:aff>
</affiliation>
<affiliation><nlm:aff id="I7">Population Genomics Program, McMaster University, Hamilton, ON, Canada</nlm:aff>
</affiliation>
</author>
<author><name sortKey="Marcucci, Maura" sort="Marcucci, Maura" uniqKey="Marcucci M" first="Maura" last="Marcucci">Maura Marcucci</name>
<affiliation><nlm:aff id="I1">Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, ON, Canada</nlm:aff>
</affiliation>
<affiliation><nlm:aff id="I4">Biostatistics Unit, Father Sean O’Sullivan Research Center, St Joseph’s Healthcare Hamilton, Hamilton, ON, Canada</nlm:aff>
</affiliation>
</author>
<author><name sortKey="Ye, Chenglin" sort="Ye, Chenglin" uniqKey="Ye C" first="Chenglin" last="Ye">Chenglin Ye</name>
<affiliation><nlm:aff id="I1">Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, ON, Canada</nlm:aff>
</affiliation>
<affiliation><nlm:aff id="I4">Biostatistics Unit, Father Sean O’Sullivan Research Center, St Joseph’s Healthcare Hamilton, Hamilton, ON, Canada</nlm:aff>
</affiliation>
</author>
<author><name sortKey="Thabane, Marroon" sort="Thabane, Marroon" uniqKey="Thabane M" first="Marroon" last="Thabane">Marroon Thabane</name>
<affiliation><nlm:aff id="I1">Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, ON, Canada</nlm:aff>
</affiliation>
<affiliation><nlm:aff id="I8">GSK, Mississauga, ON, Canada</nlm:aff>
</affiliation>
</author>
<author><name sortKey="Giangregorio, Lora" sort="Giangregorio, Lora" uniqKey="Giangregorio L" first="Lora" last="Giangregorio">Lora Giangregorio</name>
<affiliation><nlm:aff id="I9">Department of Kinesiology, University of Waterloo, Waterloo, ON, Canada</nlm:aff>
</affiliation>
</author>
<author><name sortKey="Dennis, Brittany" sort="Dennis, Brittany" uniqKey="Dennis B" first="Brittany" last="Dennis">Brittany Dennis</name>
<affiliation><nlm:aff id="I1">Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, ON, Canada</nlm:aff>
</affiliation>
<affiliation><nlm:aff id="I4">Biostatistics Unit, Father Sean O’Sullivan Research Center, St Joseph’s Healthcare Hamilton, Hamilton, ON, Canada</nlm:aff>
</affiliation>
</author>
<author><name sortKey="Kosa, Daisy" sort="Kosa, Daisy" uniqKey="Kosa D" first="Daisy" last="Kosa">Daisy Kosa</name>
<affiliation><nlm:aff id="I1">Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, ON, Canada</nlm:aff>
</affiliation>
<affiliation><nlm:aff id="I4">Biostatistics Unit, Father Sean O’Sullivan Research Center, St Joseph’s Healthcare Hamilton, Hamilton, ON, Canada</nlm:aff>
</affiliation>
<affiliation><nlm:aff id="I10">Department of Nephrology, Toronto General Hospital, Toronto, ON, Canada</nlm:aff>
</affiliation>
</author>
<author><name sortKey="Debono, Victoria Borg" sort="Debono, Victoria Borg" uniqKey="Debono V" first="Victoria Borg" last="Debono">Victoria Borg Debono</name>
<affiliation><nlm:aff id="I1">Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, ON, Canada</nlm:aff>
</affiliation>
<affiliation><nlm:aff id="I4">Biostatistics Unit, Father Sean O’Sullivan Research Center, St Joseph’s Healthcare Hamilton, Hamilton, ON, Canada</nlm:aff>
</affiliation>
</author>
<author><name sortKey="Dillenburg, Rejane" sort="Dillenburg, Rejane" uniqKey="Dillenburg R" first="Rejane" last="Dillenburg">Rejane Dillenburg</name>
<affiliation><nlm:aff id="I11">Department of Pediatrics, McMaster University, Hamilton, ON, Canada</nlm:aff>
</affiliation>
</author>
<author><name sortKey="Fruci, Vincent" sort="Fruci, Vincent" uniqKey="Fruci V" first="Vincent" last="Fruci">Vincent Fruci</name>
<affiliation><nlm:aff id="I12">Michael G. DeGroote School of Medicine, McMaster University, Hamilton, ON, Canada</nlm:aff>
</affiliation>
</author>
<author><name sortKey="Bawor, Monica" sort="Bawor, Monica" uniqKey="Bawor M" first="Monica" last="Bawor">Monica Bawor</name>
<affiliation><nlm:aff id="I13">McMaster Integrative Neuroscience Discovery & Study (MiNDS) Program, McMaster University, Hamilton, ON, Canada</nlm:aff>
</affiliation>
</author>
<author><name sortKey="Lee, Juneyoung" sort="Lee, Juneyoung" uniqKey="Lee J" first="Juneyoung" last="Lee">Juneyoung Lee</name>
<affiliation><nlm:aff id="I14">Department of Biostatistics, Korea University, Seoul, Korea</nlm:aff>
</affiliation>
</author>
<author><name sortKey="Wells, George" sort="Wells, George" uniqKey="Wells G" first="George" last="Wells">George Wells</name>
<affiliation><nlm:aff id="I15">Department of Clinical Epidemiology, University of Ottawa, Ottawa, ON, Canada</nlm:aff>
</affiliation>
</author>
<author><name sortKey="Goldsmith, Charles H" sort="Goldsmith, Charles H" uniqKey="Goldsmith C" first="Charles H" last="Goldsmith">Charles H. Goldsmith</name>
<affiliation><nlm:aff id="I1">Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, ON, Canada</nlm:aff>
</affiliation>
<affiliation><nlm:aff id="I4">Biostatistics Unit, Father Sean O’Sullivan Research Center, St Joseph’s Healthcare Hamilton, Hamilton, ON, Canada</nlm:aff>
</affiliation>
<affiliation><nlm:aff id="I16">Faculty of Health Sciences, Simon Fraser University, Burnaby, BC, Canada</nlm:aff>
</affiliation>
</author>
</analytic>
<series><title level="j">BMC Medical Research Methodology</title>
<idno type="eISSN">1471-2288</idno>
<imprint><date when="2013">2013</date>
</imprint>
</series>
</biblStruct>
</sourceDesc>
</fileDesc>
<profileDesc><textClass></textClass>
</profileDesc>
</teiHeader>
<front><div type="abstract" xml:lang="en"><sec><title>Background</title>
<p>Sensitivity analyses play a crucial role in assessing the robustness of the findings or conclusions based on primary analyses of data in clinical trials. They are a critical way to assess the impact, effect or influence of key assumptions or variations—such as different methods of analysis, definitions of outcomes, protocol deviations, missing data, and outliers—on the overall conclusions of a study.</p>
<p>The current paper is the second in a series of tutorial-type manuscripts intended to discuss and clarify aspects related to key methodological issues in the design and analysis of clinical trials.</p>
</sec>
<sec><title>Discussion</title>
<p>In this paper we will provide a detailed exploration of the key aspects of sensitivity analyses including: 1) what sensitivity analyses are, why they are needed, and how often they are used in practice; 2) the different types of sensitivity analyses that one can do, with examples from the literature; 3) some frequently asked questions about sensitivity analyses; and 4) some suggestions on how to report the results of sensitivity analyses in clinical trials.</p>
</sec>
<sec><title>Summary</title>
<p>When reporting on a clinical trial, we recommend including planned or <italic>posthoc</italic>
sensitivity analyses, the corresponding rationale and results along with the discussion of the consequences of these analyses on the overall findings of the study.</p>
</sec>
</div>
</front>
<back><div1 type="bibliography"><listBibl><biblStruct><analytic><author><name sortKey="Thabane, L" uniqKey="Thabane L">L Thabane</name>
</author>
<author><name sortKey="Ma, J" uniqKey="Ma J">J Ma</name>
</author>
<author><name sortKey="Chu, R" uniqKey="Chu R">R Chu</name>
</author>
<author><name sortKey="Cheng, J" uniqKey="Cheng J">J Cheng</name>
</author>
<author><name sortKey="Ismaila, A" uniqKey="Ismaila A">A Ismaila</name>
</author>
<author><name sortKey="Rios, Lp" uniqKey="Rios L">LP Rios</name>
</author>
<author><name sortKey="Robson, R" uniqKey="Robson R">R Robson</name>
</author>
<author><name sortKey="Thabane, M" uniqKey="Thabane M">M Thabane</name>
</author>
<author><name sortKey="Giangregorio, L" uniqKey="Giangregorio L">L Giangregorio</name>
</author>
<author><name sortKey="Goldsmith, Ch" uniqKey="Goldsmith C">CH Goldsmith</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Schneeweiss, S" uniqKey="Schneeweiss S">S Schneeweiss</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Viel, Jf" uniqKey="Viel J">JF Viel</name>
</author>
<author><name sortKey="Pobel, D" uniqKey="Pobel D">D Pobel</name>
</author>
<author><name sortKey="Carre, A" uniqKey="Carre A">A Carre</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Goldsmith, Ch" uniqKey="Goldsmith C">CH Goldsmith</name>
</author>
<author><name sortKey="Gafni, A" uniqKey="Gafni A">A Gafni</name>
</author>
<author><name sortKey="Drummond, Mf" uniqKey="Drummond M">MF Drummond</name>
</author>
<author><name sortKey="Torrance, Gw" uniqKey="Torrance G">GW Torrance</name>
</author>
<author><name sortKey="Stoddart, Gl" uniqKey="Stoddart G">GL Stoddart</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Saltelli, A" uniqKey="Saltelli A">A Saltelli</name>
</author>
<author><name sortKey="Tarantola, S" uniqKey="Tarantola S">S Tarantola</name>
</author>
<author><name sortKey="Campolongo, F" uniqKey="Campolongo F">F Campolongo</name>
</author>
<author><name sortKey="Ratto, M" uniqKey="Ratto M">M Ratto</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Saltelli, A" uniqKey="Saltelli A">A Saltelli</name>
</author>
<author><name sortKey="Ratto, M" uniqKey="Ratto M">M Ratto</name>
</author>
<author><name sortKey="Andres, T" uniqKey="Andres T">T Andres</name>
</author>
<author><name sortKey="Campolongo, F" uniqKey="Campolongo F">F Campolongo</name>
</author>
<author><name sortKey="Cariboni, J" uniqKey="Cariboni J">J Cariboni</name>
</author>
<author><name sortKey="Gatelli, D" uniqKey="Gatelli D">D Gatelli</name>
</author>
<author><name sortKey="Saisana, M" uniqKey="Saisana M">M Saisana</name>
</author>
<author><name sortKey="Tarantola, S" uniqKey="Tarantola S">S Tarantola</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Hunink, Mgm" uniqKey="Hunink M">MGM Hunink</name>
</author>
<author><name sortKey="Glasziou, Pp" uniqKey="Glasziou P">PP Glasziou</name>
</author>
<author><name sortKey="Siegel, Je" uniqKey="Siegel J">JE Siegel</name>
</author>
<author><name sortKey="Weeks, Jc" uniqKey="Weeks J">JC Weeks</name>
</author>
<author><name sortKey="Pliskin, Js" uniqKey="Pliskin J">JS Pliskin</name>
</author>
<author><name sortKey="Elstein, As" uniqKey="Elstein A">AS Elstein</name>
</author>
<author><name sortKey="Weinstein, Mc" uniqKey="Weinstein M">MC Weinstein</name>
</author>
</analytic>
</biblStruct>
<biblStruct></biblStruct>
<biblStruct></biblStruct>
<biblStruct><analytic><author><name sortKey="Ma, J" uniqKey="Ma J">J Ma</name>
</author>
<author><name sortKey="Thabane, L" uniqKey="Thabane L">L Thabane</name>
</author>
<author><name sortKey="Kaczorowski, J" uniqKey="Kaczorowski J">J Kaczorowski</name>
</author>
<author><name sortKey="Chambers, L" uniqKey="Chambers L">L Chambers</name>
</author>
<author><name sortKey="Dolovich, L" uniqKey="Dolovich L">L Dolovich</name>
</author>
<author><name sortKey="Karwalajtys, T" uniqKey="Karwalajtys T">T Karwalajtys</name>
</author>
<author><name sortKey="Levitt, C" uniqKey="Levitt C">C Levitt</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Peters, Tj" uniqKey="Peters T">TJ Peters</name>
</author>
<author><name sortKey="Richards, Sh" uniqKey="Richards S">SH Richards</name>
</author>
<author><name sortKey="Bankhead, Cr" uniqKey="Bankhead C">CR Bankhead</name>
</author>
<author><name sortKey="Ades, Ae" uniqKey="Ades A">AE Ades</name>
</author>
<author><name sortKey="Sterne, Ja" uniqKey="Sterne J">JA Sterne</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Chu, R" uniqKey="Chu R">R Chu</name>
</author>
<author><name sortKey="Thabane, L" uniqKey="Thabane L">L Thabane</name>
</author>
<author><name sortKey="Ma, J" uniqKey="Ma J">J Ma</name>
</author>
<author><name sortKey="Holbrook, A" uniqKey="Holbrook A">A Holbrook</name>
</author>
<author><name sortKey="Pullenayegum, E" uniqKey="Pullenayegum E">E Pullenayegum</name>
</author>
<author><name sortKey="Devereaux, Pj" uniqKey="Devereaux P">PJ Devereaux</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Kleinbaum, Dg" uniqKey="Kleinbaum D">DG Kleinbaum</name>
</author>
<author><name sortKey="Klein, M" uniqKey="Klein M">M Klein</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Barnett, V" uniqKey="Barnett V">V Barnett</name>
</author>
<author><name sortKey="Lewis, T" uniqKey="Lewis T">T Lewis</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Grubbs, Fe" uniqKey="Grubbs F">FE Grubbs</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Thabane, L" uniqKey="Thabane L">L Thabane</name>
</author>
<author><name sortKey="Akhtar Danesh, N" uniqKey="Akhtar Danesh N">N Akhtar-Danesh</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Williams, Nh" uniqKey="Williams N">NH Williams</name>
</author>
<author><name sortKey="Edwards, Rt" uniqKey="Edwards R">RT Edwards</name>
</author>
<author><name sortKey="Linck, P" uniqKey="Linck P">P Linck</name>
</author>
<author><name sortKey="Muntz, R" uniqKey="Muntz R">R Muntz</name>
</author>
<author><name sortKey="Hibbs, R" uniqKey="Hibbs R">R Hibbs</name>
</author>
<author><name sortKey="Wilkinson, C" uniqKey="Wilkinson C">C Wilkinson</name>
</author>
<author><name sortKey="Russell, I" uniqKey="Russell I">I Russell</name>
</author>
<author><name sortKey="Russell, D" uniqKey="Russell D">D Russell</name>
</author>
<author><name sortKey="Hounsome, B" uniqKey="Hounsome B">B Hounsome</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Zetta, S" uniqKey="Zetta S">S Zetta</name>
</author>
<author><name sortKey="Smith, K" uniqKey="Smith K">K Smith</name>
</author>
<author><name sortKey="Jones, M" uniqKey="Jones M">M Jones</name>
</author>
<author><name sortKey="Allcoat, P" uniqKey="Allcoat P">P Allcoat</name>
</author>
<author><name sortKey="Sullivan, F" uniqKey="Sullivan F">F Sullivan</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Morden, Jp" uniqKey="Morden J">JP Morden</name>
</author>
<author><name sortKey="Lambert, Pc" uniqKey="Lambert P">PC Lambert</name>
</author>
<author><name sortKey="Latimer, N" uniqKey="Latimer N">N Latimer</name>
</author>
<author><name sortKey="Abrams, Kr" uniqKey="Abrams K">KR Abrams</name>
</author>
<author><name sortKey="Wailoo, Aj" uniqKey="Wailoo A">AJ Wailoo</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="White, Ir" uniqKey="White I">IR White</name>
</author>
<author><name sortKey="Walker, S" uniqKey="Walker S">S Walker</name>
</author>
<author><name sortKey="Babiker, Ag" uniqKey="Babiker A">AG Babiker</name>
</author>
<author><name sortKey="Darbyshire, Jh" uniqKey="Darbyshire J">JH Darbyshire</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Borrelli, B" uniqKey="Borrelli B">B Borrelli</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Lawton, J" uniqKey="Lawton J">J Lawton</name>
</author>
<author><name sortKey="Jenkins, N" uniqKey="Jenkins N">N Jenkins</name>
</author>
<author><name sortKey="Darbyshire, Jl" uniqKey="Darbyshire J">JL Darbyshire</name>
</author>
<author><name sortKey="Holman, Rr" uniqKey="Holman R">RR Holman</name>
</author>
<author><name sortKey="Farmer, Aj" uniqKey="Farmer A">AJ Farmer</name>
</author>
<author><name sortKey="Hallowell, N" uniqKey="Hallowell N">N Hallowell</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Ye, C" uniqKey="Ye C">C Ye</name>
</author>
<author><name sortKey="Giangregorio, L" uniqKey="Giangregorio L">L Giangregorio</name>
</author>
<author><name sortKey="Holbrook, A" uniqKey="Holbrook A">A Holbrook</name>
</author>
<author><name sortKey="Pullenayegum, E" uniqKey="Pullenayegum E">E Pullenayegum</name>
</author>
<author><name sortKey="Goldsmith, Ch" uniqKey="Goldsmith C">CH Goldsmith</name>
</author>
<author><name sortKey="Thabane, L" uniqKey="Thabane L">L Thabane</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Horwitz, Ri" uniqKey="Horwitz R">RI Horwitz</name>
</author>
<author><name sortKey="Horwitz, Sm" uniqKey="Horwitz S">SM Horwitz</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Peduzzi, P" uniqKey="Peduzzi P">P Peduzzi</name>
</author>
<author><name sortKey="Wittes, J" uniqKey="Wittes J">J Wittes</name>
</author>
<author><name sortKey="Detre, K" uniqKey="Detre K">K Detre</name>
</author>
<author><name sortKey="Holford, T" uniqKey="Holford T">T Holford</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Montori, Vm" uniqKey="Montori V">VM Montori</name>
</author>
<author><name sortKey="Guyatt, Gh" uniqKey="Guyatt G">GH Guyatt</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Gibaldi, M" uniqKey="Gibaldi M">M Gibaldi</name>
</author>
<author><name sortKey="Sullivan, S" uniqKey="Sullivan S">S Sullivan</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Porta, M" uniqKey="Porta M">M Porta</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Everitt, B" uniqKey="Everitt B">B Everitt</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Sainani, Kl" uniqKey="Sainani K">KL Sainani</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Bendtsen, P" uniqKey="Bendtsen P">P Bendtsen</name>
</author>
<author><name sortKey="Mccambridge, J" uniqKey="Mccambridge J">J McCambridge</name>
</author>
<author><name sortKey="Bendtsen, M" uniqKey="Bendtsen M">M Bendtsen</name>
</author>
<author><name sortKey="Karlsson, N" uniqKey="Karlsson N">N Karlsson</name>
</author>
<author><name sortKey="Nilsen, P" uniqKey="Nilsen P">P Nilsen</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Brox, Ji" uniqKey="Brox J">JI Brox</name>
</author>
<author><name sortKey="Nygaard, Op" uniqKey="Nygaard O">OP Nygaard</name>
</author>
<author><name sortKey="Holm, I" uniqKey="Holm I">I Holm</name>
</author>
<author><name sortKey="Keller, A" uniqKey="Keller A">A Keller</name>
</author>
<author><name sortKey="Ingebrigtsen, T" uniqKey="Ingebrigtsen T">T Ingebrigtsen</name>
</author>
<author><name sortKey="Reikeras, O" uniqKey="Reikeras O">O Reikeras</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Mcknight, Pe" uniqKey="Mcknight P">PE McKnight</name>
</author>
<author><name sortKey="Mcknight, Km" uniqKey="Mcknight K">KM McKnight</name>
</author>
<author><name sortKey="Sidani, S" uniqKey="Sidani S">S Sidani</name>
</author>
<author><name sortKey="Figueredo, Aj" uniqKey="Figueredo A">AJ Figueredo</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Graham, Jw" uniqKey="Graham J">JW Graham</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Little, Rj" uniqKey="Little R">RJ Little</name>
</author>
<author><name sortKey="D Agostino, R" uniqKey="D Agostino R">R D'Agostino</name>
</author>
<author><name sortKey="Cohen, Ml" uniqKey="Cohen M">ML Cohen</name>
</author>
<author><name sortKey="Dickersin, K" uniqKey="Dickersin K">K Dickersin</name>
</author>
<author><name sortKey="Emerson, Ss" uniqKey="Emerson S">SS Emerson</name>
</author>
<author><name sortKey="Farrar, Jt" uniqKey="Farrar J">JT Farrar</name>
</author>
<author><name sortKey="Frangakis, C" uniqKey="Frangakis C">C Frangakis</name>
</author>
<author><name sortKey="Hogan, Jw" uniqKey="Hogan J">JW Hogan</name>
</author>
<author><name sortKey="Molenberghs, G" uniqKey="Molenberghs G">G Molenberghs</name>
</author>
<author><name sortKey="Murphy, Sa" uniqKey="Murphy S">SA Murphy</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Little, Rja" uniqKey="Little R">RJA Little</name>
</author>
<author><name sortKey="Rubin, Db" uniqKey="Rubin D">DB Rubin</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Rubin, Db" uniqKey="Rubin D">DB Rubin</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Schafer, Jl" uniqKey="Schafer J">JL Schafer</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Son, H" uniqKey="Son H">H Son</name>
</author>
<author><name sortKey="Friedmann, E" uniqKey="Friedmann E">E Friedmann</name>
</author>
<author><name sortKey="Thomas, Sa" uniqKey="Thomas S">SA Thomas</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Peters, Sa" uniqKey="Peters S">SA Peters</name>
</author>
<author><name sortKey="Bots, Ml" uniqKey="Bots M">ML Bots</name>
</author>
<author><name sortKey="Den Ruijter, Hm" uniqKey="Den Ruijter H">HM den Ruijter</name>
</author>
<author><name sortKey="Palmer, Mk" uniqKey="Palmer M">MK Palmer</name>
</author>
<author><name sortKey="Grobbee, De" uniqKey="Grobbee D">DE Grobbee</name>
</author>
<author><name sortKey="Crouse, Jr" uniqKey="Crouse J">JR Crouse</name>
</author>
<author><name sortKey="O Leary, Dh" uniqKey="O Leary D">DH O'Leary</name>
</author>
<author><name sortKey="Evans, Gw" uniqKey="Evans G">GW Evans</name>
</author>
<author><name sortKey="Raichlen, Js" uniqKey="Raichlen J">JS Raichlen</name>
</author>
<author><name sortKey="Moons, Kg" uniqKey="Moons K">KG Moons</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Zhang, H" uniqKey="Zhang H">H Zhang</name>
</author>
<author><name sortKey="Paik, Mc" uniqKey="Paik M">MC Paik</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Chen, Hy" uniqKey="Chen H">HY Chen</name>
</author>
<author><name sortKey="Gao, S" uniqKey="Gao S">S Gao</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Ma, J" uniqKey="Ma J">J Ma</name>
</author>
<author><name sortKey="Akhtar Danesh, N" uniqKey="Akhtar Danesh N">N Akhtar-Danesh</name>
</author>
<author><name sortKey="Dolovich, L" uniqKey="Dolovich L">L Dolovich</name>
</author>
<author><name sortKey="Thabane, L" uniqKey="Thabane L">L Thabane</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Kingsley, Gh" uniqKey="Kingsley G">GH Kingsley</name>
</author>
<author><name sortKey="Kowalczyk, A" uniqKey="Kowalczyk A">A Kowalczyk</name>
</author>
<author><name sortKey="Taylor, H" uniqKey="Taylor H">H Taylor</name>
</author>
<author><name sortKey="Ibrahim, F" uniqKey="Ibrahim F">F Ibrahim</name>
</author>
<author><name sortKey="Packham, Jc" uniqKey="Packham J">JC Packham</name>
</author>
<author><name sortKey="Mchugh, Nj" uniqKey="Mchugh N">NJ McHugh</name>
</author>
<author><name sortKey="Mulherin, Dm" uniqKey="Mulherin D">DM Mulherin</name>
</author>
<author><name sortKey="Kitas, Gd" uniqKey="Kitas G">GD Kitas</name>
</author>
<author><name sortKey="Chakravarty, K" uniqKey="Chakravarty K">K Chakravarty</name>
</author>
<author><name sortKey="Tom, Bd" uniqKey="Tom B">BD Tom</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="De Pauw, Be" uniqKey="De Pauw B">BE de Pauw</name>
</author>
<author><name sortKey="Sable, Ca" uniqKey="Sable C">CA Sable</name>
</author>
<author><name sortKey="Walsh, Tj" uniqKey="Walsh T">TJ Walsh</name>
</author>
<author><name sortKey="Lupinacci, Rj" uniqKey="Lupinacci R">RJ Lupinacci</name>
</author>
<author><name sortKey="Bourque, Mr" uniqKey="Bourque M">MR Bourque</name>
</author>
<author><name sortKey="Wise, Ba" uniqKey="Wise B">BA Wise</name>
</author>
<author><name sortKey="Nguyen, By" uniqKey="Nguyen B">BY Nguyen</name>
</author>
<author><name sortKey="Dinubile, Mj" uniqKey="Dinubile M">MJ DiNubile</name>
</author>
<author><name sortKey="Teppler, H" uniqKey="Teppler H">H Teppler</name>
</author>
</analytic>
</biblStruct>
<biblStruct></biblStruct>
<biblStruct><analytic><author><name sortKey="Song, P K" uniqKey="Song P">P-K Song</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Pintilie, M" uniqKey="Pintilie M">M Pintilie</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Tai, Bc" uniqKey="Tai B">BC Tai</name>
</author>
<author><name sortKey="Grundy, R" uniqKey="Grundy R">R Grundy</name>
</author>
<author><name sortKey="Machin, D" uniqKey="Machin D">D Machin</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Holbrook, Jt" uniqKey="Holbrook J">JT Holbrook</name>
</author>
<author><name sortKey="Wise, Ra" uniqKey="Wise R">RA Wise</name>
</author>
<author><name sortKey="Gold, Bd" uniqKey="Gold B">BD Gold</name>
</author>
<author><name sortKey="Blake, K" uniqKey="Blake K">K Blake</name>
</author>
<author><name sortKey="Brown, Ed" uniqKey="Brown E">ED Brown</name>
</author>
<author><name sortKey="Castro, M" uniqKey="Castro M">M Castro</name>
</author>
<author><name sortKey="Dozor, Aj" uniqKey="Dozor A">AJ Dozor</name>
</author>
<author><name sortKey="Lima, Jj" uniqKey="Lima J">JJ Lima</name>
</author>
<author><name sortKey="Mastronarde, Jg" uniqKey="Mastronarde J">JG Mastronarde</name>
</author>
<author><name sortKey="Sockrider, Mm" uniqKey="Sockrider M">MM Sockrider</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Holbrook, A" uniqKey="Holbrook A">A Holbrook</name>
</author>
<author><name sortKey="Thabane, L" uniqKey="Thabane L">L Thabane</name>
</author>
<author><name sortKey="Keshavjee, K" uniqKey="Keshavjee K">K Keshavjee</name>
</author>
<author><name sortKey="Dolovich, L" uniqKey="Dolovich L">L Dolovich</name>
</author>
<author><name sortKey="Bernstein, B" uniqKey="Bernstein B">B Bernstein</name>
</author>
<author><name sortKey="Chan, D" uniqKey="Chan D">D Chan</name>
</author>
<author><name sortKey="Troyan, S" uniqKey="Troyan S">S Troyan</name>
</author>
<author><name sortKey="Foster, G" uniqKey="Foster G">G Foster</name>
</author>
<author><name sortKey="Gerstein, H" uniqKey="Gerstein H">H Gerstein</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Hilbe, Jm" uniqKey="Hilbe J">JM Hilbe</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Forsblom, C" uniqKey="Forsblom C">C Forsblom</name>
</author>
<author><name sortKey="Harjutsalo, V" uniqKey="Harjutsalo V">V Harjutsalo</name>
</author>
<author><name sortKey="Thorn, Lm" uniqKey="Thorn L">LM Thorn</name>
</author>
<author><name sortKey="Waden, J" uniqKey="Waden J">J Waden</name>
</author>
<author><name sortKey="Tolonen, N" uniqKey="Tolonen N">N Tolonen</name>
</author>
<author><name sortKey="Saraheimo, M" uniqKey="Saraheimo M">M Saraheimo</name>
</author>
<author><name sortKey="Gordin, D" uniqKey="Gordin D">D Gordin</name>
</author>
<author><name sortKey="Moran, Jl" uniqKey="Moran J">JL Moran</name>
</author>
<author><name sortKey="Thomas, Mc" uniqKey="Thomas M">MC Thomas</name>
</author>
<author><name sortKey="Groop, Ph" uniqKey="Groop P">PH Groop</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Grams, Me" uniqKey="Grams M">ME Grams</name>
</author>
<author><name sortKey="Coresh, J" uniqKey="Coresh J">J Coresh</name>
</author>
<author><name sortKey="Segev, Dl" uniqKey="Segev D">DL Segev</name>
</author>
<author><name sortKey="Kucirka, Lm" uniqKey="Kucirka L">LM Kucirka</name>
</author>
<author><name sortKey="Tighiouart, H" uniqKey="Tighiouart H">H Tighiouart</name>
</author>
<author><name sortKey="Sarnak, Mj" uniqKey="Sarnak M">MJ Sarnak</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Lim, Hj" uniqKey="Lim H">HJ Lim</name>
</author>
<author><name sortKey="Zhang, X" uniqKey="Zhang X">X Zhang</name>
</author>
<author><name sortKey="Dyck, R" uniqKey="Dyck R">R Dyck</name>
</author>
<author><name sortKey="Osgood, N" uniqKey="Osgood N">N Osgood</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Chu, R" uniqKey="Chu R">R Chu</name>
</author>
<author><name sortKey="Walter, Sd" uniqKey="Walter S">SD Walter</name>
</author>
<author><name sortKey="Guyatt, G" uniqKey="Guyatt G">G Guyatt</name>
</author>
<author><name sortKey="Devereaux, Pj" uniqKey="Devereaux P">PJ Devereaux</name>
</author>
<author><name sortKey="Walsh, M" uniqKey="Walsh M">M Walsh</name>
</author>
<author><name sortKey="Thorlund, K" uniqKey="Thorlund K">K Thorlund</name>
</author>
<author><name sortKey="Thabane, L" uniqKey="Thabane L">L Thabane</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Bowen, A" uniqKey="Bowen A">A Bowen</name>
</author>
<author><name sortKey="Hesketh, A" uniqKey="Hesketh A">A Hesketh</name>
</author>
<author><name sortKey="Patchick, E" uniqKey="Patchick E">E Patchick</name>
</author>
<author><name sortKey="Young, A" uniqKey="Young A">A Young</name>
</author>
<author><name sortKey="Davies, L" uniqKey="Davies L">L Davies</name>
</author>
<author><name sortKey="Vail, A" uniqKey="Vail A">A Vail</name>
</author>
<author><name sortKey="Long, Af" uniqKey="Long A">AF Long</name>
</author>
<author><name sortKey="Watkins, C" uniqKey="Watkins C">C Watkins</name>
</author>
<author><name sortKey="Wilkinson, M" uniqKey="Wilkinson M">M Wilkinson</name>
</author>
<author><name sortKey="Pearl, G" uniqKey="Pearl G">G Pearl</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Spiegelhalter, Dj" uniqKey="Spiegelhalter D">DJ Spiegelhalter</name>
</author>
<author><name sortKey="Best, Ng" uniqKey="Best N">NG Best</name>
</author>
<author><name sortKey="Lunn, D" uniqKey="Lunn D">D Lunn</name>
</author>
<author><name sortKey="Thomas, A" uniqKey="Thomas A">A Thomas</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Byers, Al" uniqKey="Byers A">AL Byers</name>
</author>
<author><name sortKey="Allore, H" uniqKey="Allore H">H Allore</name>
</author>
<author><name sortKey="Gill, Tm" uniqKey="Gill T">TM Gill</name>
</author>
<author><name sortKey="Peduzzi, Pn" uniqKey="Peduzzi P">PN Peduzzi</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Yusuf, S" uniqKey="Yusuf S">S Yusuf</name>
</author>
<author><name sortKey="Wittes, J" uniqKey="Wittes J">J Wittes</name>
</author>
<author><name sortKey="Probstfield, J" uniqKey="Probstfield J">J Probstfield</name>
</author>
<author><name sortKey="Tyroler, Ha" uniqKey="Tyroler H">HA Tyroler</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Altman, Dg" uniqKey="Altman D">DG Altman</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Mauskopf, Ja" uniqKey="Mauskopf J">JA Mauskopf</name>
</author>
<author><name sortKey="Sullivan, Sd" uniqKey="Sullivan S">SD Sullivan</name>
</author>
<author><name sortKey="Annemans, L" uniqKey="Annemans L">L Annemans</name>
</author>
<author><name sortKey="Caro, J" uniqKey="Caro J">J Caro</name>
</author>
<author><name sortKey="Mullins, Cd" uniqKey="Mullins C">CD Mullins</name>
</author>
<author><name sortKey="Nuijten, M" uniqKey="Nuijten M">M Nuijten</name>
</author>
<author><name sortKey="Orlewska, E" uniqKey="Orlewska E">E Orlewska</name>
</author>
<author><name sortKey="Watkins, J" uniqKey="Watkins J">J Watkins</name>
</author>
<author><name sortKey="Trueman, P" uniqKey="Trueman P">P Trueman</name>
</author>
</analytic>
</biblStruct>
</listBibl>
</div1>
</back>
</TEI>
<pmc article-type="research-article" xml:lang="en"><pmc-dir>properties open_access</pmc-dir>
<front><journal-meta><journal-id journal-id-type="nlm-ta">BMC Med Res Methodol</journal-id>
<journal-id journal-id-type="iso-abbrev">BMC Med Res Methodol</journal-id>
<journal-title-group><journal-title>BMC Medical Research Methodology</journal-title>
</journal-title-group>
<issn pub-type="epub">1471-2288</issn>
<publisher><publisher-name>BioMed Central</publisher-name>
</publisher>
</journal-meta>
<article-meta><article-id pub-id-type="pmid">23855337</article-id>
<article-id pub-id-type="pmc">3720188</article-id>
<article-id pub-id-type="publisher-id">1471-2288-13-92</article-id>
<article-id pub-id-type="doi">10.1186/1471-2288-13-92</article-id>
<article-categories><subj-group subj-group-type="heading"><subject>Commentary</subject>
</subj-group>
</article-categories>
<title-group><article-title>A tutorial on sensitivity analyses in clinical trials: the what, why, when and how</article-title>
</title-group>
<contrib-group><contrib contrib-type="author" corresp="yes" id="A1"><name><surname>Thabane</surname>
<given-names>Lehana</given-names>
</name>
<xref ref-type="aff" rid="I1">1</xref>
<xref ref-type="aff" rid="I2">2</xref>
<xref ref-type="aff" rid="I3">3</xref>
<xref ref-type="aff" rid="I4">4</xref>
<xref ref-type="aff" rid="I5">5</xref>
<email>ThabanL@mcmaster.ca</email>
</contrib>
<contrib contrib-type="author" id="A2"><name><surname>Mbuagbaw</surname>
<given-names>Lawrence</given-names>
</name>
<xref ref-type="aff" rid="I1">1</xref>
<xref ref-type="aff" rid="I4">4</xref>
<email>mbuagblc@mcmaster.ca</email>
</contrib>
<contrib contrib-type="author" id="A3"><name><surname>Zhang</surname>
<given-names>Shiyuan</given-names>
</name>
<xref ref-type="aff" rid="I1">1</xref>
<xref ref-type="aff" rid="I4">4</xref>
<email>zhangs27@mcmaster.ca</email>
</contrib>
<contrib contrib-type="author" id="A4"><name><surname>Samaan</surname>
<given-names>Zainab</given-names>
</name>
<xref ref-type="aff" rid="I1">1</xref>
<xref ref-type="aff" rid="I6">6</xref>
<xref ref-type="aff" rid="I7">7</xref>
<email>samaanz@mcmaster.ca</email>
</contrib>
<contrib contrib-type="author" id="A5"><name><surname>Marcucci</surname>
<given-names>Maura</given-names>
</name>
<xref ref-type="aff" rid="I1">1</xref>
<xref ref-type="aff" rid="I4">4</xref>
<email>Maura.marcucci@gmail.com</email>
</contrib>
<contrib contrib-type="author" id="A6"><name><surname>Ye</surname>
<given-names>Chenglin</given-names>
</name>
<xref ref-type="aff" rid="I1">1</xref>
<xref ref-type="aff" rid="I4">4</xref>
<email>yecl@mcmaster.ca</email>
</contrib>
<contrib contrib-type="author" id="A7"><name><surname>Thabane</surname>
<given-names>Marroon</given-names>
</name>
<xref ref-type="aff" rid="I1">1</xref>
<xref ref-type="aff" rid="I8">8</xref>
<email>mthaban@mcmaster.ca</email>
</contrib>
<contrib contrib-type="author" id="A8"><name><surname>Giangregorio</surname>
<given-names>Lora</given-names>
</name>
<xref ref-type="aff" rid="I9">9</xref>
<email>lora.giangregorio@uwaterloo.ca</email>
</contrib>
<contrib contrib-type="author" id="A9"><name><surname>Dennis</surname>
<given-names>Brittany</given-names>
</name>
<xref ref-type="aff" rid="I1">1</xref>
<xref ref-type="aff" rid="I4">4</xref>
<email>dennisbb@mcmaster.ca</email>
</contrib>
<contrib contrib-type="author" id="A10"><name><surname>Kosa</surname>
<given-names>Daisy</given-names>
</name>
<xref ref-type="aff" rid="I1">1</xref>
<xref ref-type="aff" rid="I4">4</xref>
<xref ref-type="aff" rid="I10">10</xref>
<email>Sarah.kosa@uhn.ca</email>
</contrib>
<contrib contrib-type="author" id="A11"><name><surname>Debono</surname>
<given-names>Victoria Borg</given-names>
</name>
<xref ref-type="aff" rid="I1">1</xref>
<xref ref-type="aff" rid="I4">4</xref>
<email>borgdevj@mcmaster.ca</email>
</contrib>
<contrib contrib-type="author" id="A12"><name><surname>Dillenburg</surname>
<given-names>Rejane</given-names>
</name>
<xref ref-type="aff" rid="I11">11</xref>
<email>dillenburg@mcmaster.ca</email>
</contrib>
<contrib contrib-type="author" id="A13"><name><surname>Fruci</surname>
<given-names>Vincent</given-names>
</name>
<xref ref-type="aff" rid="I12">12</xref>
<email>vincent.fruci@medportal.ca</email>
</contrib>
<contrib contrib-type="author" id="A14"><name><surname>Bawor</surname>
<given-names>Monica</given-names>
</name>
<xref ref-type="aff" rid="I13">13</xref>
<email>baworm@mcmaster.ca</email>
</contrib>
<contrib contrib-type="author" id="A15"><name><surname>Lee</surname>
<given-names>Juneyoung</given-names>
</name>
<xref ref-type="aff" rid="I14">14</xref>
<email>jyleeuf@korea.ac.kr</email>
</contrib>
<contrib contrib-type="author" id="A16"><name><surname>Wells</surname>
<given-names>George</given-names>
</name>
<xref ref-type="aff" rid="I15">15</xref>
<email>gawells@ottawaheart.ca</email>
</contrib>
<contrib contrib-type="author" id="A17"><name><surname>Goldsmith</surname>
<given-names>Charles H</given-names>
</name>
<xref ref-type="aff" rid="I1">1</xref>
<xref ref-type="aff" rid="I4">4</xref>
<xref ref-type="aff" rid="I16">16</xref>
<email>cgoldsmi@sfu.ca</email>
</contrib>
</contrib-group>
<aff id="I1"><label>1</label>
Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, ON, Canada</aff>
<aff id="I2"><label>2</label>
Departments of Pediatrics and Anesthesia, McMaster University, Hamilton, ON, Canada</aff>
<aff id="I3"><label>3</label>
Center for Evaluation of Medicine, St Joseph’s Healthcare Hamilton, Hamilton, ON, Canada</aff>
<aff id="I4"><label>4</label>
Biostatistics Unit, Father Sean O’Sullivan Research Center, St Joseph’s Healthcare Hamilton, Hamilton, ON, Canada</aff>
<aff id="I5"><label>5</label>
Population Health Research Institute, Hamilton Health Sciences, Hamilton, ON, Canada</aff>
<aff id="I6"><label>6</label>
Department of Psychiatry and Behavioral Neurosciences, McMaster University, Hamilton, ON, Canada</aff>
<aff id="I7"><label>7</label>
Population Genomics Program, McMaster University, Hamilton, ON, Canada</aff>
<aff id="I8"><label>8</label>
GSK, Mississauga, ON, Canada</aff>
<aff id="I9"><label>9</label>
Department of Kinesiology, University of Waterloo, Waterloo, ON, Canada</aff>
<aff id="I10"><label>10</label>
Department of Nephrology, Toronto General Hospital, Toronto, ON, Canada</aff>
<aff id="I11"><label>11</label>
Department of Pediatrics, McMaster University, Hamilton, ON, Canada</aff>
<aff id="I12"><label>12</label>
Michael G. DeGroote School of Medicine, McMaster University, Hamilton, ON, Canada</aff>
<aff id="I13"><label>13</label>
McMaster Integrative Neuroscience Discovery & Study (MiNDS) Program, McMaster University, Hamilton, ON, Canada</aff>
<aff id="I14"><label>14</label>
Department of Biostatistics, Korea University, Seoul, Korea</aff>
<aff id="I15"><label>15</label>
Department of Clinical Epidemiology, University of Ottawa, Ottawa, ON, Canada</aff>
<aff id="I16"><label>16</label>
Faculty of Health Sciences, Simon Fraser University, Burnaby, BC, Canada</aff>
<pub-date pub-type="collection"><year>2013</year>
</pub-date>
<pub-date pub-type="epub"><day>16</day>
<month>7</month>
<year>2013</year>
</pub-date>
<volume>13</volume>
<fpage>92</fpage>
<lpage>92</lpage>
<history><date date-type="received"><day>11</day>
<month>12</month>
<year>2012</year>
</date>
<date date-type="accepted"><day>10</day>
<month>7</month>
<year>2013</year>
</date>
</history>
<permissions><copyright-statement>Copyright © 2013 Thabane et al.; licensee BioMed Central Ltd.</copyright-statement>
<copyright-year>2013</copyright-year>
<copyright-holder>Thabane et al.; licensee BioMed Central Ltd.</copyright-holder>
<license license-type="open-access" xlink:href="http://creativecommons.org/licenses/by/2.0"><license-p>This is an Open Access article distributed under the terms of the Creative Commons Attribution License (<ext-link ext-link-type="uri" xlink:href="http://creativecommons.org/licenses/by/2.0">http://creativecommons.org/licenses/by/2.0</ext-link>
), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.</license-p>
</license>
</permissions>
<self-uri xlink:href="http://www.biomedcentral.com/1471-2288/13/92"></self-uri>
<abstract><sec><title>Background</title>
<p>Sensitivity analyses play a crucial role in assessing the robustness of the findings or conclusions based on primary analyses of data in clinical trials. They are a critical way to assess the impact, effect or influence of key assumptions or variations—such as different methods of analysis, definitions of outcomes, protocol deviations, missing data, and outliers—on the overall conclusions of a study.</p>
<p>The current paper is the second in a series of tutorial-type manuscripts intended to discuss and clarify aspects related to key methodological issues in the design and analysis of clinical trials.</p>
</sec>
<sec><title>Discussion</title>
<p>In this paper we will provide a detailed exploration of the key aspects of sensitivity analyses including: 1) what sensitivity analyses are, why they are needed, and how often they are used in practice; 2) the different types of sensitivity analyses that one can do, with examples from the literature; 3) some frequently asked questions about sensitivity analyses; and 4) some suggestions on how to report the results of sensitivity analyses in clinical trials.</p>
</sec>
<sec><title>Summary</title>
<p>When reporting on a clinical trial, we recommend including planned or <italic>posthoc</italic>
sensitivity analyses, the corresponding rationale and results along with the discussion of the consequences of these analyses on the overall findings of the study.</p>
</sec>
</abstract>
<kwd-group><kwd>Sensitivity analysis</kwd>
<kwd>Clinical trials</kwd>
<kwd>Robustness</kwd>
</kwd-group>
</article-meta>
</front>
<body><sec><title>Background</title>
<p>The credibility or interpretation of the results of clinical trials relies on the validity of the methods of analysis or models used and their corresponding assumptions. An astute researcher or reader may be less confident in the findings of a study if they believe that the analysis or assumptions made were not appropriate. For a primary analysis of data from a prospective randomized controlled trial (RCT), the key questions for investigators (and for readers) include:</p>
<p>• How confident can I be about the results?</p>
<p>• Will the results change if I change the definition of the outcome (e.g., using different cut-off points)?</p>
<p>• Will the results change if I change the method of analysis?</p>
<p>• Will the results change if we take missing data into account? Will the method of handling missing data lead to different conclusions?</p>
<p>• How much influence will minor protocol deviations have on the conclusions?</p>
<p>• How will ignoring the serial correlation of measurements within a patient impact the results?</p>
<p>• What if the data were assumed to have a non-Normal distribution or there were outliers?</p>
<p>• Will the results change if one looks at subgroups of patients?</p>
<p>• Will the results change if the full intervention is received (i.e. degree of compliance)?</p>
<p>The above questions can be addressed by performing sensitivity analyses—testing the effect of these “changes” on the observed results. If, after performing sensitivity analyses the findings are consistent with those from the primary analysis and would lead to similar conclusions about treatment effect, the researcher is reassured that the underlying factor(s) had little or no influence or impact on the primary conclusions. In this situation, the results or the conclusions are said to be “robust”.</p>
<p>The objectives of this paper are to provide an overview of how to approach sensitivity analyses in clinical trials. This is the second in a series of tutorial-type manuscripts intended to discuss and clarify aspects related to some key methodological issues in the design and analysis of clinical trials. The first was on pilot studies
[<xref ref-type="bibr" rid="B1">1</xref>
]. We start by describing what sensitivity analysis is, why it is needed and how often it is done in practice. We then describe the different types of sensitivity analyses that one can do, with examples from the literature. We also address some of the commonly asked questions about sensitivity analysis and provide some guidance on how to report sensitivity analyses.</p>
</sec>
<sec sec-type="discussion"><title>Discussion</title>
<sec><title>Sensitivity Analysis</title>
<sec><title>What is a sensitivity analysis in clinical research?</title>
<p>Sensitivity Analysis (SA) is defined as “a method to determine the robustness of an assessment by examining the extent to which results are affected by changes in methods, models, values of unmeasured variables, or assumptions” with the aim of identifying “results that are most dependent on questionable or unsupported assumptions”
[<xref ref-type="bibr" rid="B2">2</xref>
]. It has also been defined as “a series of analyses of a data set to assess whether altering any of the assumptions made leads to different final interpretations or conclusions”
[<xref ref-type="bibr" rid="B3">3</xref>
]. Essentially, SA addresses the “what-if-the-key-inputs-or-assumptions-changed”-type of question. If we want to know whether the results change when something about the way we approach the data analysis changes, we can make the change in our analysis approach and document the changes in the results or conclusions. For more detailed coverage of SA, we refer the reader to these references
[<xref ref-type="bibr" rid="B4">4</xref>
-<xref ref-type="bibr" rid="B7">7</xref>
].</p>
</sec>
<sec><title>Why is sensitivity analysis necessary?</title>
<p>The design and analysis of clinical trials often rely on assumptions that may have some effect, influence or impact on the conclusions if they are not met. It is important to assess these effects through sensitivity analyses. Consistency between the results of primary analysis and the results of sensitivity analysis may strengthen the conclusions or credibility of the findings. However, it is important to note that the definition of consistency may depend in part on the area of investigation, the outcome of interest or even the implications of the findings or results.</p>
<p>It is equally important to assess the robustness to ensure appropriate interpretation of the results taking into account the things that may have an impact on them. Thus, it imperative for every analytic plan to have some sensitivity analyses built into it.</p>
<p>The United States (US) Food and Drug Administration (FDA) and the European Medicines Association (EMEA), which offer guidance on Statistical Principles for Clinical Trials, state that “it is important to evaluate the robustness of the results and primary conclusions of the trial.” Robustness refers to “the sensitivity of the overall conclusions to various limitations of the data, assumptions, and analytic approaches to data analysis”
[<xref ref-type="bibr" rid="B8">8</xref>
]. The United Kingdom (UK) National Institute of Health and Clinical Excellence (NICE) also recommends the use of sensitivity analysis in “exploring alternative scenarios and the uncertainty in cost-effectiveness results”
[<xref ref-type="bibr" rid="B9">9</xref>
].</p>
</sec>
<sec><title>How often is sensitivity analysis reported in practice?</title>
<p>To evaluate how often sensitivity analyses are used in medical and health research, we surveyed the January 2012 editions of major medical journals (British Medical Journal, New England Journal of Medicine, the Lancet, Journal of the American Medical Association and the Canadian Medical Association Journal) and major health economics journals (Pharmaco-economics, Medical Decision making, European Journal of Health Economics, Health Economics and the Journal of Health Economics). From every article that included some form of statistical analyses, we evaluated: i) the percentage of published articles that reported results of some sensitivity analyses; and ii) the types of sensitivity analyses that were performed. Table
<xref ref-type="table" rid="T1">1</xref>
provides a summary of the findings. Overall, the point prevalent use of sensitivity analyses is about 26.7% (36/135) —which seems very low. A higher percentage of papers published in health economics than in medical journals (30.8% vs. 20.3%) reported some sensitivity analyses. Among the papers in medical journals, 18 (28.1%) were RCTs, of which only 3 (16.6%) reported sensitivity analyses. Assessing robustness of the findings to different methods of analysis was the most common type of sensitivity analysis reported in both types of journals. Therefore despite their importance, sensitivity analyses are under-used in practice. Further, sensitivity analyses are more common in health economics research—for example in conducting cost-effectiveness analyses, cost-utility analyses or budget-impact analyses—than in other areas of health or medical research.</p>
<table-wrap position="float" id="T1"><label>Table 1</label>
<caption><p>Comparison of sensitivity analyses reported in medical and health economics journals in January 2012</p>
</caption>
<table frame="hsides" rules="groups" border="1"><colgroup><col align="left"></col>
<col align="left"></col>
<col align="left"></col>
</colgroup>
<thead valign="top"><tr><th align="left"><bold>Variable</bold>
</th>
<th align="left"><bold>Medical journals</bold>
</th>
<th align="left"><bold>Health economics journals</bold>
</th>
</tr>
</thead>
<tbody valign="top"><tr><td align="left" valign="bottom">Number with statistical analysis<hr></hr>
</td>
<td align="left" valign="bottom">64<sup>$</sup>
<hr></hr>
</td>
<td align="left" valign="bottom">71<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom">Number with sensitivity analysis (%)<hr></hr>
</td>
<td align="left" valign="bottom">13<sup>&</sup>
(20.3)<hr></hr>
</td>
<td align="left" valign="bottom">22 (30.9)<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom">Type of sensitivity analysis<hr></hr>
</td>
<td align="left" valign="bottom"> <hr></hr>
</td>
<td align="left" valign="bottom"> <hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom">• Methods of analysis<hr></hr>
</td>
<td align="left" valign="bottom">5<hr></hr>
</td>
<td align="left" valign="bottom">12<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom">• Outcome definitions<hr></hr>
</td>
<td align="left" valign="bottom">4<hr></hr>
</td>
<td align="left" valign="bottom">1<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom">• Distributional assumptions<hr></hr>
</td>
<td align="left" valign="bottom">1<hr></hr>
</td>
<td align="left" valign="bottom">0<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom">• Key assumptions*<hr></hr>
</td>
<td align="left" valign="bottom">2<hr></hr>
</td>
<td align="left" valign="bottom">4<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom">• Missing data<hr></hr>
</td>
<td align="left" valign="bottom">1<hr></hr>
</td>
<td align="left" valign="bottom">4<hr></hr>
</td>
</tr>
<tr><td align="left">• Baseline imbalances</td>
<td align="left">0</td>
<td align="left">1</td>
</tr>
</tbody>
</table>
<table-wrap-foot><p><sup>$</sup>
Eighteen (18) of these were randomized controlled trials, of which only 3 reported sensitivity analyses.</p>
<p><sup>&</sup>
Of which 3 were randomized controlled trials and 10 were observational studies.</p>
<p>*Assumptions related to the participants, interventions or outcomes that can affect the results of the trial. For example, considering a second episode of cancer as a relapse instead of a continuation of the first; in a cost-effectiveness analysis, modifying the anticipated frequency of the intervention.</p>
</table-wrap-foot>
</table-wrap>
</sec>
</sec>
<sec><title>Types of sensitivity analyses</title>
<p>In this section, we describe scenarios that may require sensitivity analyses, and how one could use sensitivity analyses to assess the robustness of the statistical analyses or findings of RCTs. These are not meant to be exhaustive, but rather to illustrate common situations where sensitivity analyses might be useful to consider (Table
<xref ref-type="table" rid="T2">2</xref>
). In each case, we provide examples of actual studies where sensitivity analyses were performed, and the implications of these sensitivity analyses.</p>
<table-wrap position="float" id="T2"><label>Table 2</label>
<caption><p>Examples of common scenarios for sensitivity analyses in clinical trials</p>
</caption>
<table frame="hsides" rules="groups" border="1"><colgroup><col align="left"></col>
<col align="left"></col>
</colgroup>
<thead valign="top"><tr><th align="left"><bold>Scenario</bold>
</th>
<th align="left"><bold>Sensitivity analysis options</bold>
</th>
</tr>
</thead>
<tbody valign="top"><tr><td rowspan="2" align="left" valign="bottom">Outliers<hr></hr>
</td>
<td align="left" valign="bottom">- Assess outlier by z-score or boxplot<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom">- Perform analyses with and without outliers<hr></hr>
</td>
</tr>
<tr><td rowspan="4" align="left" valign="bottom">Non-compliance or protocol violation in RCTs<hr></hr>
</td>
<td align="left" valign="bottom">Perform<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom">- Intention-to-treat analysis (as primary analysis)<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom">- As-treated analysis<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom">- Per-protocol analysis<hr></hr>
</td>
</tr>
<tr><td rowspan="2" align="left" valign="bottom">Missing data<hr></hr>
</td>
<td align="left" valign="bottom">- Analyze only complete cases<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom">- Impute the missing data using single or multiple imputation methods and redo the analysis<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom">Definitions of outcomes<hr></hr>
</td>
<td align="left" valign="bottom">- Perform analyses on outcomes of different cut-offs or definitions<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom">Clustering or correlation<hr></hr>
</td>
<td rowspan="2" align="left" valign="bottom">- Compare the analysis that ignores clustering with one primary method chosen to account for clustering<hr></hr>
</td>
</tr>
<tr><td rowspan="4" align="left" valign="bottom">and multi-center trials<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom">- Compare the analysis that ignores clustering with several methods of accounting for clustering
[<xref ref-type="bibr" rid="B10">10</xref>
,<xref ref-type="bibr" rid="B11">11</xref>
]<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom">- Perform analysis with and without adjusting for center<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom">- Use different methods of adjusting for center
[<xref ref-type="bibr" rid="B12">12</xref>
]<hr></hr>
</td>
</tr>
<tr><td rowspan="3" align="left" valign="bottom">Competing risks in RCTs<hr></hr>
</td>
<td align="left" valign="bottom">- Perform a survival analysis for each event separately<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom">- Use a proportional sub-distribution hazard model (Fine & Grey approach)<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom">- Fit one model by taking into account all the competing risks together
[<xref ref-type="bibr" rid="B13">13</xref>
]<hr></hr>
</td>
</tr>
<tr><td rowspan="3" align="left" valign="bottom">Baseline imbalance<hr></hr>
</td>
<td align="left" valign="bottom">Perform:<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom">- Analysis with and without adjustment for baseline characteristics<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom">- Analysis with different methods of adjusting for baseline imbalance. e.g. Multivariable regression vs. propensity score method<hr></hr>
</td>
</tr>
<tr><td rowspan="4" align="left" valign="bottom">Distributional assumptions<hr></hr>
</td>
<td align="left" valign="bottom">Perform analyses under different distributional assumptions<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom">- Different distributions (e.g. Poisson vs. Negative binomial)<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom">- Parametric vs. non-parametric methods<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom">- Classical vs. Bayesian methods<hr></hr>
</td>
</tr>
<tr><td align="left"> </td>
<td align="left">- Different prior distributions</td>
</tr>
</tbody>
</table>
</table-wrap>
<sec><title>Impact of outliers</title>
<p>An outlier is an observation that is numerically distant from the rest of the data. It deviates markedly from the rest of the sample from which it comes
[<xref ref-type="bibr" rid="B14">14</xref>
,<xref ref-type="bibr" rid="B15">15</xref>
]. Outliers are usually exceptional cases in a sample. The problem with outliers is that they can deflate or inflate the mean of a sample and therefore influence any estimates of treatment effect or association that are derived from the mean. To assess the potential impact of outliers, one would first assess whether or not any observations meet the definition of an outlier—using either a boxplot or z-scores
[<xref ref-type="bibr" rid="B16">16</xref>
]. Second, one could perform a sensitivity analysis with and without the outliers.</p>
<p>Examples:</p>
<p>• In a cost–utility analysis of a practice-based osteopathy clinic for subacute spinal pain, Williams et al. reported lower costs per quality of life year ratios when they excluded outliers
[<xref ref-type="bibr" rid="B17">17</xref>
]. In other words, there were certain participants in the trial whose costs were very high, and were making the average costs look higher than they probably were in reality. The observed cost per quality of life year was not robust to the exclusion of outliers, and changed when they were excluded.</p>
<p>• A primary analysis based on the intention-to-treat principle showed no statistically significant differences in reducing depression between a nurse-led cognitive self-help intervention program compared to standard care among 218 patients hospitalized with angina over 6 months. Some sensitivity analyses in this trial were performed by excluding participants with high baseline levels of depression (outliers) and showed a statistically significant reduction in depression in the intervention group compared to the control. This implies that the results of the primary analysis were affected by the presence of patients with baseline high depression
[<xref ref-type="bibr" rid="B18">18</xref>
].</p>
</sec>
<sec><title>Impact of non-compliance or protocol deviations</title>
<p>In clinical trials some participants may not adhere to the intervention they were allocated to receive or comply with the scheduled treatment visits. Non-adherence or non-compliance is a form of protocol deviation. Other types of protocol deviations include switching between intervention and control arms (i.e. treatment switching or crossovers)
[<xref ref-type="bibr" rid="B19">19</xref>
,<xref ref-type="bibr" rid="B20">20</xref>
], or not implementing the intervention as prescribed (i.e. intervention fidelity)
[<xref ref-type="bibr" rid="B21">21</xref>
,<xref ref-type="bibr" rid="B22">22</xref>
].</p>
<p>Protocol deviations are very common in interventional research
[<xref ref-type="bibr" rid="B23">23</xref>
-<xref ref-type="bibr" rid="B25">25</xref>
]. The potential impact of protocol deviations is the dilution of the treatment effect
[<xref ref-type="bibr" rid="B26">26</xref>
,<xref ref-type="bibr" rid="B27">27</xref>
]. Therefore, it is crucial to determine the robustness of the results to the inclusion of data from participants who deviate from the protocol. Typically, for RCTs the primary analysis is based on an intention-to-treat (ITT) principle—in which participants are analyzed according to the arm to which they were randomized, irrespective of whether they actually received the treatment or completed the prescribed regimen
[<xref ref-type="bibr" rid="B28">28</xref>
,<xref ref-type="bibr" rid="B29">29</xref>
]. Two common types of sensitivity analyses can be performed to assess the robustness of the results to protocol deviations: 1) per-protocol (PP) analysis—in which participants who violate the protocol are excluded from the analysis
[<xref ref-type="bibr" rid="B30">30</xref>
]; and 2) as-treated (AT) analysis—in which participants are analyzed according to the treatment they actually received
[<xref ref-type="bibr" rid="B30">30</xref>
]. The PP analysis provides the ideal scenario in which all the participants comply, and is more likely to show an effect; whereas the ITT analysis provides a “real life” scenario, in which some participants do not comply. It is more conservative, and less likely to show that the intervention is effective. For trials with repeated measures, some protocol violations which lead to missing data can be dealt with alternatively. This is covered in more detail in the next section.</p>
<p>Examples:</p>
<p>• A trial was designed to investigate the effects of an electronic screening and brief intervention to change risky drinking behaviour in university students. The results of the ITT analysis (on all 2336 participants who answered the follow-up survey) showed that the intervention had no significant effect. However, a sensitivity analysis based on the PP analysis (including only those with risky drinking at baseline and who answered the follow-up survey; n = 408) suggested a small beneficial effect on weekly alcohol consumption
[<xref ref-type="bibr" rid="B31">31</xref>
]. A reader might be less confident in the findings of the trial because of the inconsistency between the ITT and PP analyses—the ITT was not robust to sensitivity analyses. A researcher might choose to explore differences in the characteristics of the participants who were included in the ITT versus the PP analyses.</p>
<p>• A study compared the long-term effects of surgical versus non-surgical management of chronic back pain. Both the ITT and AT analyses showed no significant difference between the two management strategies
[<xref ref-type="bibr" rid="B32">32</xref>
]. A reader would be more confident in the findings because the ITT and AT analyses were consistent—the ITT was robust to sensitivity analyses.</p>
</sec>
<sec><title>Impact of missing data</title>
<p>Missing data are common in every research study. This is a problem that can be broadly defined as “missing some information on the phenomena in which we are interested”
[<xref ref-type="bibr" rid="B33">33</xref>
]. Data can be missing for different reasons including (1) non-response in surveys due to lack of interest, lack of time, nonsensical responses, and coding errors in data entry/transfer; (2) incompleteness of data in large data registries due to missing appointments, not everyone is captured in the database, and incomplete data; and (3) missingness in prospective studies as a result of loss to follow up, dropouts, non-adherence, missing doses, and data entry errors.</p>
<p>The choice of how to deal with missing data would depend on the mechanisms of missingness. In this regard, data can be missing at random (MAR), missing not at random (MNAR), or missing completely at random (MCAR). When data are MAR, the missing data are dependent on some other observed variables rather than any unobserved one. For example, consider a trial to investigate the effect of pre-pregnancy calcium supplementation on hypertensive disorders in pregnancy. Missing data on the hypertensive disorders is dependent (conditional) on being pregnant in the first place. When data are MCAR, the cases with missing data may be considered a random sample drawn from all the cases. In other words, there is no “cause” of missingness. Consider the example of a trial comparing a new cancer treatment to standard treatment in which participants are followed at 4, 8, 12 and 16 months. If a participant misses the follow up at the 8th and 16th months and these are unrelated to the outcome of interest, in this case mortality, then this missing data is MCAR. Reasons such as a clinic staff being ill or equipment failure are often unrelated to the outcome of interest. However, the MCAR assumption is often challenging to prove because the reason data is missing may not be known and therefore it is difficult to determine if it is related to the outcome of interest. When data are MNAR, missingness is dependent on some unobserved data. For example, in the case above, if the participant missed the 8th month appointment because he was feeling worse or the 16th month appointment because he was dead, the missingness is dependent on the data not observed because the participant was absent. When data are MAR or MCAR, they are often referred to as ignorable (provided the cause of MAR is taken into account). MNAR on the other hand, is nonignorable missingness. Ignoring the missingness in such data leads to biased parameter estimates
[<xref ref-type="bibr" rid="B34">34</xref>
]. Ignoring missing data in analyses can have implications on the reliability, validity and generalizability of research findings.</p>
<p>The best way to deal with missing data is prevention, by steps taken in the design and data collection stages, some of which have been described by Little et al.
[<xref ref-type="bibr" rid="B35">35</xref>
]. But this is difficult to achieve in most cases. There are two main approaches to handling missing data: i) ignore them—and use complete case analysis; and ii) impute them—using either single or multiple imputation techniques. Imputation is one of the most commonly used approaches to handling missing data. Examples of single imputation methods include hot deck, cold deck method, mean imputation, regression technique, last observation carried forward (LOCF) and composite methods—which uses a combination of the above methods to impute missing values. Single imputation methods often lead to biased estimates and under-estimation of the true variability in the data. Multiple imputation (MI) technique is currently the best available method of dealing with missing data under the assumption that data are missing at random (MAR)
[<xref ref-type="bibr" rid="B33">33</xref>
,<xref ref-type="bibr" rid="B36">36</xref>
-<xref ref-type="bibr" rid="B38">38</xref>
]. MI addresses the limitations of single imputation by using multiple imputed datasets which yield unbiased estimates, and also accounts for the within- and between-dataset variability. Bayesian methods using statistical models that assume a prior distribution for the missing data can also be used to impute data
[<xref ref-type="bibr" rid="B35">35</xref>
].</p>
<p>It is important to note that ignoring missing data in the analysis would be implicitly assuming that the data are MCAR, an assumption that is often hard to verify in reality.</p>
<p>There are some statistical approaches to dealing with missing data that do not necessarily require formal imputation methods. For example, in studies using continuous outcomes, linear mixed models for repeated measures are used for analyzing outcomes measured repeatedly over time
[<xref ref-type="bibr" rid="B39">39</xref>
,<xref ref-type="bibr" rid="B40">40</xref>
]. For categorical responses or count data, generalized estimating equations [GEE] and random-effects generalized linear mixed models [GLMM] methods may be used
[<xref ref-type="bibr" rid="B41">41</xref>
,<xref ref-type="bibr" rid="B42">42</xref>
]. In these models it is assumed that missing data are MAR. If this assumption is valid, then the complete-case analysis by including predictors of missing observations will provide consistent estimates of the parameter.</p>
<p>The choice of whether to ignore or impute missing data, and how to impute it, may affect the findings of the trial. Although one approach (ignore or impute, and if the latter, how to impute) should be made <italic>a priori,</italic>
a sensitivity analysis can be done with a different approach to see how “robust” the primary analysis is to the chosen method for handling missing data.</p>
<p>Examples:</p>
<p>• A 2011 paper reported the sensitivity analyses of different strategies for imputing missing data in cluster RCTs with a binary outcome using the community hypertension assessment trial (CHAT) as an example. They found that variance in the treatment effect was underestimated when the amount of missing data was large and the imputation strategy did not take into account the intra-cluster correlation. However, the effects of the intervention under various methods of imputation were similar. The CHAT intervention was not superior to usual care
[<xref ref-type="bibr" rid="B43">43</xref>
].</p>
<p>• In a trial comparing methotrexate with to placebo in the treatment of psoriatic arthritis, the authors reported both an intention-to-treat analysis (using multiple imputation techniques to account for missing data) and a complete case analysis (ignoring the missing data). The complete case analysis, which is less conservative, showed some borderline improvement in the primary outcome (psoriatic arthritis response criteria), while the intention-to-treat analysis did not
[<xref ref-type="bibr" rid="B44">44</xref>
]. A reader would be less confident about the effects of methotrexate on psoriatic arthritis, due to the discrepancy between the results with imputed data (ITT) and the complete case analysis.</p>
</sec>
<sec><title>Impact of different definitions of outcomes (e.g. different cut-off points for binary outcomes)</title>
<p>Often, an outcome is defined by achieving or not achieving a certain level or threshold of a measure. For example in a study measuring adherence rates to medication, levels of adherence can be dichotomized as achieving or not achieving at least 80%, 85% or 90% of pills taken. The choice of the level a participant has to achieve can affect the outcome—it might be harder to achieve 90% adherence than 80%. Therefore, a sensitivity analysis could be performed to see how redefining the threshold changes the observed effect of a given intervention.</p>
<p>Examples:</p>
<p>• In a trial comparing caspofungin to amphotericin B for febrile neutropoenic patients, a sensitivity analysis was conducted to investigate the impact of different definitions of fever resolution as part of a composite endpoint which included: resolution of any baseline invasive fungal infection, no breakthrough invasive fungal infection, survival, no premature discontinuation of study drug, and fever resolution for 48 hours during the period of neutropenia. They found that response rates were higher when less stringent fever resolution definitions were used, especially in low-risk patients. The modified definitions of fever resolution were: no fever for 24 hours before the resolution of neutropenia; no fever at the 7-day post-therapy follow-up visit; and removal of fever resolution completely from the composite endpoint. This implies that the efficacy of both medications depends somewhat on the definition of the outcomes
[<xref ref-type="bibr" rid="B45">45</xref>
].</p>
<p>• In a phase II trial comparing minocycline and creatinine to placebo for Parkinson’s disease, a sensitivity analysis was conducted based on another definition (threshold) for futility. In the primary analysis a predetermined futility threshold was set at 30% reduction in mean change in Unified Parkinson’s Disease Rating Scale (UPDRS) score, derived from historical control data. If minocycline or creatinine did not bring about at least a 30% reduction in UPDRS score, they would be considered as futile and no further testing will be conducted. Based on the data derived from the current control (placebo) group, a new threshold of 32.4% (more stringent) was used for the sensitivity analysis. The findings from the primary analysis and the sensitivity analysis both confirmed that that neither creatine nor minocycline could be rejected as futile and should both be tested in Phase III trials
[<xref ref-type="bibr" rid="B46">46</xref>
]. A reader would be more confident of these robust findings.</p>
</sec>
<sec><title>Impact of different methods of analysis to account for clustering or correlation</title>
<p>Interventions can be administered to individuals, but they can also be administered to clusters of individuals, or naturally occurring groups. For example, one might give an intervention to students in one class, and compare their outcomes to students in another class – the class is the cluster. Clusters can also be patients treated by the same physician, physicians in the same practice center or hospital, or participants living in the same community. Likewise, in the same trial, participants may be recruited from multiple sites or centers. Each of these centers will represent a cluster. Patients or elements within a cluster often have some appreciable degree of homogeneity as compared to patients between clusters. In other words, members of the same cluster are more likely to be similar to each other than they are to members of another cluster, and this similarity may then be reflected in the similarity or correlation measure, on the outcome of interest.</p>
<p>There are several methods of accounting or adjusting for similarities within clusters, or “clustering” in studies where this phenomenon is expected or exists as part of the design (e.g., in cluster randomization trials). Therefore, in assessing the impact of clustering one can build into the analytic plans two forms of sensitivity analyses: i) analysis with and without taking clustering into account—comparing the analysis that ignores clustering (i.e. assumes that the data are independent) to one primary method chosen to account for clustering; ii) analysis that compares several methods of accounting for clustering.</p>
<p>Correlated data may also occur in longitudinal studies through repeat or multiple measurements from the same patient, taken over time or based on multiple responses in a single survey. Ignoring the potential correlation between several measurements from an individual can lead to inaccurate conclusions
[<xref ref-type="bibr" rid="B47">47</xref>
].</p>
<p>Here are a few references to studies that compared the outcomes that resulted when different methods were/were not used to account for clustering. Noteworthy, is the fact that the analytical approaches for cluster-RCTs and multi-site RCTs are similar.</p>
<p>Examples:</p>
<p>• Ma et al. performed sensitivity analyses of different methods of analysing cluster RCTs
[<xref ref-type="bibr" rid="B48">48</xref>
]. In this paper they compared three cluster-level methods (un-weighted linear regression, weighted linear regression and random-effects meta-regression) to six individual level analysis methods (standard logistic regression, robust standard errors approach, GEE, random effects meta-analytic approach, random-effects logistic regression and Bayesian random-effects regression). Using data from the CHAT trial, in this analysis, all nine methods provided similar results, re-enforcing the hypothesis that the CHAT intervention was not superior to usual care.</p>
<p>• Peters et al. conducted sensitivity analyses to compare different methods—three cluster-level (un-weighted regression of practice log odds, regression of log odds weighted by their inverse variance and random-effects meta-regression of log odds with cluster as a random effect) and five individual-level methods (standard logistic regression ignoring clustering, robust standard errors, GEE, random-effects logistic regression and Bayesian random-effects logistic regression.)—for analyzing cluster randomized trials using an example involving a factorial design
[<xref ref-type="bibr" rid="B13">13</xref>
]. In this analysis, they demonstrated that the methods used in the analysis of cluster randomized trials could give varying results, with standard logistic regression ignoring clustering being the least conservative.</p>
<p>• Cheng et al. used sensitivity analyses to compare different methods (six models for clustered binary outcomes and three models for clustered nominal outcomes) of analysing correlated data in discrete choice surveys
[<xref ref-type="bibr" rid="B49">49</xref>
]. The results were robust to various statistical models, but showed more variability in the presence of a larger cluster effect (higher within-patient correlation).</p>
<p>• A trial evaluated the effects of lansoprazole on gastro-esophageal reflux disease in children from 19 clinics with asthma. The primary analysis was based on GEE to determine the effect of lansoprazole in reducing asthma symptoms. Subsequently they performed a sensitivity analysis by including the study site as a covariate. Their finding that lansoprazole did not significantly improve symptoms was robust to this sensitivity analysis
[<xref ref-type="bibr" rid="B50">50</xref>
].</p>
<p>• In addition to comparing the performance of different methods to estimate treatment effects on a continuous outcome in simulated multicenter randomized controlled trials
[<xref ref-type="bibr" rid="B12">12</xref>
], the authors used data from the Computerization of Medical Practices for the Enhancement of Therapeutic Effectiveness (COMPETE) II
[<xref ref-type="bibr" rid="B51">51</xref>
] to assess the robustness of the primary results (based on GEE to adjust for clustering by provider of care) under different methods of adjusting for clustering. The results, which showed that a shared electronic decision support system improved care and outcomes in diabetic patients, were robust under different methods of analysis.</p>
</sec>
<sec><title>Impact of competing risks in analysis of trials with composite outcomes</title>
<p>A competing risk event happens in situations where multiple events are likely to occur in a way that the occurrence of one event may prevent other events from being observed
[<xref ref-type="bibr" rid="B48">48</xref>
]. For example, in a trial using a composite of death, myocardial infarction or stroke, if someone dies, they cannot experience a subsequent event, or stroke or myocardial infarction—death can be a competing risk event. Similarly, death can be a competing risk in trials of patients with malignant diseases where thrombotic events are important. There are several options for dealing with competing risks in survival analyses: (1) to perform a survival analysis for each event separately, where the other competing event(s) is/are treated as censored; the common representation of survival curves using the Kaplan-Meier estimator is in this context replaced by the cumulative incidence function (CIF) which offers a better interpretation of the incidence curve for one risk, regardless of whether the competing risks are independent; (2) to use a proportional sub-distribution hazard model (Fine & Grey approach) in which subjects that experience other competing events are kept in the risk set for the event of interest (i.e. as if they could later experience the event); (3) to fit one model, rather than separate models, taking into account all the competing risks together (Lunn-McNeill approach)
[<xref ref-type="bibr" rid="B13">13</xref>
]. Therefore, the best approach to assessing the influence of a competing risk would be to plan for sensitivity analysis that adjusts for the competing risk event.</p>
<p>Examples:</p>
<p>• A previously-reported trial compared low molecular weight heparin (LMWH) with oral anticoagulant therapy for the prevention of recurrent venous thromboembolism (VTE) in patients with advanced cancer, and a subsequent study presented sensitivity analyses comparing the results from standard survival analysis (Kaplan-Meier method) with those from competing risk methods—namely, the cumulative incidence function (CIF) and Gray's test
[<xref ref-type="bibr" rid="B52">52</xref>
]. The results using both methods were similar. This strengthened their confidence in the conclusion that LMWH reduced the risk of recurrent VTE.</p>
<p>• For patients at increased risk of end stage renal disease (ESRD) but also of premature death not related to ESRD, such as patients with diabetes or with vascular disease, analyses considering the two events as different outcomes may be misleading if the possibility of dying before the development of ESRD is not taken into account
[<xref ref-type="bibr" rid="B49">49</xref>
]. Different studies performing sensitivity analyses demonstrated that the results on predictors of ESRD and death for any cause were dependent on whether the competing risks were taken into account or not
[<xref ref-type="bibr" rid="B53">53</xref>
,<xref ref-type="bibr" rid="B54">54</xref>
], and on which competing risk method was used
[<xref ref-type="bibr" rid="B55">55</xref>
]. These studies further highlight the need for a sensitivity analysis of competing risks when they are present in trials.</p>
</sec>
<sec><title>Impact of baseline imbalance in RCTs</title>
<p>In RCTs, randomization is used to balance the expected distribution of the baseline or prognostic characteristics of the patients in all treatment arms. Therefore the primary analysis is typically based on ITT approach unadjusted for baseline characteristics. However, some residual imbalance can still occur by chance. One can perform a sensitivity analysis by using a multivariable analysis to adjust for hypothesized residual baseline imbalances to assess their impact on effect estimates.</p>
<p>Examples:</p>
<p>• A paper presented a simulation study where the risk of the outcome, effect of the treatment, power and prevalence of the prognostic factors, and sample size were all varied to evaluate their effects on the treatment estimates. Logistic regression models were compared with and without adjustment for the prognostic factors. The study concluded that the probability of prognostic imbalance in small trials could be substantial. Also, covariate adjustment improved estimation accuracy and statistical power
[<xref ref-type="bibr" rid="B56">56</xref>
].</p>
<p>• In a trial testing the effectiveness of enhanced communication therapy for aphasia and dysarthria after stroke, the authors conducted a sensitivity analysis to adjust for baseline imbalances. Both primary and sensitivity analysis showed that enhanced communication therapy had no additional benefit
[<xref ref-type="bibr" rid="B57">57</xref>
].</p>
</sec>
<sec><title>Impact of distributional assumptions</title>
<p>Most statistical analyses rely on distributional assumptions for observed data (e.g. Normal distribution for continuous outcomes, Poisson distribution for count data, or binomial distribution for binary outcome data). It is important not only to test for goodness-of-fit for these distributions, but to also plan for sensitivity analyses using other suitable distributions. For example, for continuous data, one can redo the analysis assuming a Student-T distribution—which is symmetric, bell-shaped distribution like the Normal distribution, but with thicker tails; for count data, once can use the Negative-binomial distribution—which would be useful to assess the robustness of the results if over-dispersion is accounted for
[<xref ref-type="bibr" rid="B52">52</xref>
]. Bayesian analyses routinely include sensitivity analyses to assess the robustness of findings under different models for the data and prior distributions
[<xref ref-type="bibr" rid="B58">58</xref>
]. Analyses based on parametric methods—which often rely on strong distributional assumptions—may also need to be evaluated for robustness using non-parametric methods. The latter often make less stringent distributional assumptions. However, it is essential to note that in general non-parametric methods are less efficient (i.e. have less statistical power) than their parametric counter-parts if the data are Normally distributed.</p>
<p>Examples:</p>
<p>• Ma et al. performed sensitivity analyses based on Bayesian and classical methods for analysing cluster RCTs with a binary outcome in the CHAT trial. The similarities in the results after using the different methods confirmed the results of the primary analysis: the CHAT intervention was not superior to usual care
[<xref ref-type="bibr" rid="B10">10</xref>
].</p>
<p>• A negative binomial regression model was used
[<xref ref-type="bibr" rid="B52">52</xref>
] to analyze discrete outcome data from a clinical trial designed to evaluate the effectiveness of a pre-habilitation program in preventing functional decline among physically frail, community-living older persons. The negative binomial model provided an improved fit to the data than the Poisson regression model. The negative binomial model provides an alternative approach for analyzing discrete data where over-dispersion is a problem
[<xref ref-type="bibr" rid="B59">59</xref>
].</p>
</sec>
</sec>
<sec><title>Commonly asked questions about sensitivity analyses</title>
<p>• Q: Do I need to adjust the overall level of significance for performing sensitivity analyses?</p>
<p>A: No. Sensitivity analysis is typically a re-analysis of either the same outcome using different approaches, or different definitions of the outcome—with the primary goal of assessing how these changes impact the conclusions. Essentially everything else including the criterion for statistical significance needs to be kept constant so that we can assess whether any impact is attributable to underlying sensitivity analyses.</p>
<p>• Q: Do I have to report all the results of the sensitivity analyses?</p>
<p>A: Yes, especially if the results are different or lead to different a conclusion from the original results—whose sensitivity was being assessed. However, if the results remain robust (i.e. unchanged), then a brief statement to this effect may suffice.</p>
<p>• Q: Can I perform sensitivity analyses posthoc?</p>
<p>A: It is desirable to document all planned analyses including sensitivity analyses in the protocol <italic>a priori</italic>
. Sometimes, one cannot anticipate all the challenges that can occur during the conduct of a study that may require additional sensitivity analyses. In that case, one needs to incorporate the anticipated sensitivity analyses in the statistical analysis plan (SAP), which needs to be completed before analyzing the data. Clear rationale is needed for every sensitivity analysis. This may also occur <italic>posthoc</italic>
.</p>
<p>• Q: How do I choose between the results of different sensitivity analyses? (i.e. which results are the best?)</p>
<p>A: The goal of sensitivity analyses is not to select the “best” results. Rather, the aim is to assess the robustness or consistency of the results under different methods, subgroups, definitions, assumptions and so on. The assessment of robustness is often based on the magnitude, direction or statistical significance of the estimates. You cannot use the sensitivity analysis to choose an alternate conclusion to your study. Rather, you can state the conclusion based on your primary analysis, and present your sensitivity analysis as an example of how confident you are that it represents the truth. If the sensitivity analysis suggests that the primary analysis is not robust, it may point to the need for future research that might address the source of the inconsistency. Your study cannot answer the question <italic>which results are best?</italic>
To answer the question of <italic>which method is best and under what conditions,</italic>
simulation studies comparing the different approaches on the basis of bias, precision, coverage or efficiency may be necessary.</p>
<p>• Q: When should one perform sensitivity analysis?</p>
<p>A: The default position should be to plan for sensitivity analysis in every clinical trial. Thus, all studies need to include some sensitivity analysis to check the robustness of the primary findings. All statistical methods used to analyze data from clinical trials rely on assumptions—which need to either be tested whenever possible, with the results assessed for robustness through some sensitivity analyses. Similarly, missing data or protocol deviations are common occurrences in many trials and their impact on inferences needs to be assessed.</p>
<p>• Q: How many sensitivity analyses can one perform for a single primary analysis?</p>
<p>A: The number is not an important factor in determining what sensitivity analyses to perform. The most important factor is the rationale for doing any sensitivity analysis. Understanding the nature of the data, and having some content expertise are useful in determining which and how many sensitivity analyses to perform. For example, varying the ways of dealing with missing data is unlikely to change the results if 1% of data are missing. Likewise, understanding the distribution of certain variables can help to determine which cut points would be relevant. Typically, it is advisable to limit sensitivity analyses to the primary outcome. Conducting multiple sensitivity analysis on all outcomes is often neither practical, nor necessary.</p>
<p>• Q: How many factors can I vary in performing sensitivity analyses?</p>
<p>A: Ideally, one can study the impact of all key elements using a factorial design—which would allow the assessment of the impact of individual and joint factors. Alternatively, one can vary one factor at a time to be able to assess whether the factor is responsible for the resulting impact (if any). For example, in a sensitivity analysis to assess the impact of the Normality assumption (analysis assuming Normality e.g. T-test vs. analysis without assuming Normality e.g. Based on a sign test) and outlier (analysis with and without outlier), this can be achieved through 2x2 factorial design.</p>
<p>• Q: What is the difference between secondary analyses and sensitivity analyses?</p>
<p>A: Secondary analyses are typically analyses of secondary outcomes. Like primary analyses which deal with primary outcome(s), such analyses need to be documented in the protocol or SAP. In most studies such analyses are exploratory—because most studies are not powered for secondary outcomes. They serve to provide support that the effects reported in the primary outcome are consistent with underlying biology. They are different from sensitivity analyses as described above.</p>
<p>• Q: What is the difference between subgroup analyses and sensitivity analyses?</p>
<p>A: Subgroup analyses are intended to assess whether the effect is similar across specified groups of patients or modified by certain patient characteristics
[<xref ref-type="bibr" rid="B60">60</xref>
]. If the primary results are statistically significant, subgroup analyses are intended to assess whether the observed effect is consistent across the underlying patient subgroups—which may be viewed as some form of sensitivity analysis. In general, for subgroup analyses one is interested in the results for each subgroup, whereas in subgroup “sensitivity” analyses, one is interested in the similarity of results across subgroups (ie. robustness across subgroups). Typically subgroup analyses require specification of the subgroup hypothesis and rationale, and performed through inclusion of an interaction term (i.e. of the subgroup variable x main exposure variable) in the regression model. They may also require adjustment for alpha—the overall level of significance. Furthermore, most studies are not usually powered for subgroup analyses.</p>
</sec>
</sec>
<sec sec-type="conclusions"><title>Conclusion</title>
<sec><title>Reporting of sensitivity analyses</title>
<p>There has been considerable attention paid to enhancing the transparency of reporting of clinical trials. This has led to several reporting guidelines, starting with the CONSORT Statement
[<xref ref-type="bibr" rid="B61">61</xref>
] in 1996 and its extensions [<ext-link ext-link-type="uri" xlink:href="http://www.equator-network.org">http://www.equator-network.org</ext-link>
]. Not one of these guidelines specifically addresses how sensitivity analyses need to be reported. On the other hand, there is some guidance on how sensitivity analyses need to be reported in economic analyses
[<xref ref-type="bibr" rid="B62">62</xref>
]—which may partly explain the differential rates of reporting of sensitivity analyses shown in Table
<xref ref-type="table" rid="T1">1</xref>
. We strongly encourage some modifications of all reporting guidelines to include items on sensitivity analyses—as a way to enhance their use and reporting. The proposed reporting changes can be as follows:</p>
<p>• In Methods Section: Report the planned or <italic>posthoc</italic>
sensitivity analyses and rationale for each.</p>
<p>• In Results Section: Report whether or not the results of the sensitivity analyses or conclusions are similar to those based on primary analysis. If similar, just state that the results or conclusions remain robust. If different, report the results of the sensitivity analyses along with the primary results.</p>
<p>• In Discussion Section: Discuss the key limitations and implications of the results of the sensitivity analyses on the conclusions or findings. This can be done by describing what changes the sensitivity analyses bring to the interpretation of the data, and whether the sensitivity analyses are more stringent or more relaxed than the primary analysis.</p>
</sec>
<sec><title>Some concluding remarks</title>
<p>Sensitivity analyses play an important role is checking the robustness of the conclusions from clinical trials. They are important in interpreting or establishing the credibility of the findings. If the results remain robust under different assumptions, methods or scenarios, this can strengthen their credibility. The results of our brief survey of January 2012 editions of major medical and health economics journals that show that their use is very low. We recommend that some sensitivity analysis should be the default plan in statistical or economic analyses of any clinical trial. Investigators need to identify any key assumptions, variations, or methods that may impact or influence the findings, and plan to conduct some sensitivity analyses as part of their analytic strategy. The final report must include the documentation of the planned or <italic>posthoc</italic>
sensitivity analyses, rationale, corresponding results and a discussion of their consequences or repercussions on the overall findings.</p>
</sec>
</sec>
<sec><title>Abbreviations</title>
<p>SA: Sensitivity analysis; US: United States; FDA: Food and Drug Administration; EMEA: European Medicines Association; UK: United Kingdom; NICE: National Institute of Health and Clinical Excellence; RCT: Randomized controlled trial; ITT: Intention-to-treat; PP: Per-protocol; AT: As-treated; LOCF: Last observation carried forward; MI: Multiple imputation; MAR: Missing at random; GEE: Generalized estimating equations; GLMM: Generalized linear mixed models; CHAT: Community hypertension assessment trial; PSA: Prostate specific antigen; CIF: Cumulative incidence function; ESRD: End stage renal disease; IV: Instrumental variable; ANCOVA: Analysis of covariance; SAP: Statistical analysis plan; CONSORT: Consolidated Standards of Reporting Trials.</p>
</sec>
<sec><title>Competing interests</title>
<p>The authors declare that they have no competing interests.</p>
</sec>
<sec><title>Authors’ contributions</title>
<p>LT conceived the idea and drafted the outline and paper. GW, CHG and MT commented on the idea and draft outline. LM and SZ performed literature search and data abstraction. ZS, LG and CY edited and formatted the manuscript. MM, BD, DK, VBD, RD, VF, MB, JL reviewed and revised draft versions of the manuscript. All authors reviewed several draft versions of the manuscript and approved the final manuscript.</p>
</sec>
<sec><title>Pre-publication history</title>
<p>The pre-publication history for this paper can be accessed here:</p>
<p><ext-link ext-link-type="uri" xlink:href="http://www.biomedcentral.com/1471-2288/13/92/prepub">http://www.biomedcentral.com/1471-2288/13/92/prepub</ext-link>
</p>
</sec>
</body>
<back><sec><title>Acknowledgements</title>
<p>This work was supported in part by funds from the CANNeCTIN programme.</p>
</sec>
<ref-list><ref id="B1"><mixed-citation publication-type="journal"><name><surname>Thabane</surname>
<given-names>L</given-names>
</name>
<name><surname>Ma</surname>
<given-names>J</given-names>
</name>
<name><surname>Chu</surname>
<given-names>R</given-names>
</name>
<name><surname>Cheng</surname>
<given-names>J</given-names>
</name>
<name><surname>Ismaila</surname>
<given-names>A</given-names>
</name>
<name><surname>Rios</surname>
<given-names>LP</given-names>
</name>
<name><surname>Robson</surname>
<given-names>R</given-names>
</name>
<name><surname>Thabane</surname>
<given-names>M</given-names>
</name>
<name><surname>Giangregorio</surname>
<given-names>L</given-names>
</name>
<name><surname>Goldsmith</surname>
<given-names>CH</given-names>
</name>
<article-title>A tutorial on pilot studies: the what, why and how</article-title>
<source>BMC Med Res Methodol</source>
<year>2010</year>
<volume>10</volume>
<fpage>1</fpage>
<pub-id pub-id-type="doi">10.1186/1471-2288-10-1</pub-id>
<pub-id pub-id-type="pmid">20053272</pub-id>
</mixed-citation>
</ref>
<ref id="B2"><mixed-citation publication-type="journal"><name><surname>Schneeweiss</surname>
<given-names>S</given-names>
</name>
<article-title>Sensitivity analysis and external adjustment for unmeasured confounders in epidemiologic database studies of therapeutics</article-title>
<source>Pharmacoepidemiol Drug Saf</source>
<year>2006</year>
<volume>15</volume>
<issue>5</issue>
<fpage>291</fpage>
<lpage>303</lpage>
<pub-id pub-id-type="doi">10.1002/pds.1200</pub-id>
<pub-id pub-id-type="pmid">16447304</pub-id>
</mixed-citation>
</ref>
<ref id="B3"><mixed-citation publication-type="journal"><name><surname>Viel</surname>
<given-names>JF</given-names>
</name>
<name><surname>Pobel</surname>
<given-names>D</given-names>
</name>
<name><surname>Carre</surname>
<given-names>A</given-names>
</name>
<article-title>Incidence of leukaemia in young people around the La Hague nuclear waste reprocessing plant: a sensitivity analysis</article-title>
<source>Stat Med</source>
<year>1995</year>
<volume>14</volume>
<issue>21–22</issue>
<fpage>2459</fpage>
<lpage>2472</lpage>
<pub-id pub-id-type="pmid">8711281</pub-id>
</mixed-citation>
</ref>
<ref id="B4"><mixed-citation publication-type="book"><name><surname>Goldsmith</surname>
<given-names>CH</given-names>
</name>
<name><surname>Gafni</surname>
<given-names>A</given-names>
</name>
<name><surname>Drummond</surname>
<given-names>MF</given-names>
</name>
<name><surname>Torrance</surname>
<given-names>GW</given-names>
</name>
<name><surname>Stoddart</surname>
<given-names>GL</given-names>
</name>
<source>Sensitivity Analysis and Experimental Design: The Case of Economic Evaluation of Health Care Programmes. Proceedings of the Third Canadian Conference on Health Economics 1986</source>
<year>1987</year>
<publisher-name>Winnipeg MB: The University of Manitoba Press</publisher-name>
</mixed-citation>
</ref>
<ref id="B5"><mixed-citation publication-type="book"><name><surname>Saltelli</surname>
<given-names>A</given-names>
</name>
<name><surname>Tarantola</surname>
<given-names>S</given-names>
</name>
<name><surname>Campolongo</surname>
<given-names>F</given-names>
</name>
<name><surname>Ratto</surname>
<given-names>M</given-names>
</name>
<source>Sensitivity Analysis in Practice: A Guide to Assessing Scientific Models</source>
<year>2004</year>
<publisher-name>New York, NY: Willey</publisher-name>
</mixed-citation>
</ref>
<ref id="B6"><mixed-citation publication-type="book"><name><surname>Saltelli</surname>
<given-names>A</given-names>
</name>
<name><surname>Ratto</surname>
<given-names>M</given-names>
</name>
<name><surname>Andres</surname>
<given-names>T</given-names>
</name>
<name><surname>Campolongo</surname>
<given-names>F</given-names>
</name>
<name><surname>Cariboni</surname>
<given-names>J</given-names>
</name>
<name><surname>Gatelli</surname>
<given-names>D</given-names>
</name>
<name><surname>Saisana</surname>
<given-names>M</given-names>
</name>
<name><surname>Tarantola</surname>
<given-names>S</given-names>
</name>
<source>Global Sensitivity Analysis: The Primer</source>
<year>2008</year>
<publisher-name>New York, NY: Wiley-Interscience</publisher-name>
</mixed-citation>
</ref>
<ref id="B7"><mixed-citation publication-type="book"><name><surname>Hunink</surname>
<given-names>MGM</given-names>
</name>
<name><surname>Glasziou</surname>
<given-names>PP</given-names>
</name>
<name><surname>Siegel</surname>
<given-names>JE</given-names>
</name>
<name><surname>Weeks</surname>
<given-names>JC</given-names>
</name>
<name><surname>Pliskin</surname>
<given-names>JS</given-names>
</name>
<name><surname>Elstein</surname>
<given-names>AS</given-names>
</name>
<name><surname>Weinstein</surname>
<given-names>MC</given-names>
</name>
<source>Decision Making in Health and Medicine: Integrating Evidence and Values</source>
<year>2001</year>
<publisher-name>Cambridge: Cambridge University Press</publisher-name>
</mixed-citation>
</ref>
<ref id="B8"><mixed-citation publication-type="other"><collab>USFDA</collab>
<article-title>International Conference on Harmonisation; Guidance on Statistical Principles for Clinical Trials. Guideline E9. Statistical principles for clinical trials. Federal Register, 16 September 1998, Vol. 63, No. 179, p. 49583
</article-title>
<comment>[<ext-link ext-link-type="uri" xlink:href="http://www.fda.gov/downloads/RegulatoryInformation/Guidances/UCM129505.pdf">http://www.fda.gov/downloads/RegulatoryInformation/Guidances/UCM129505.pdf</ext-link>
]</comment>
</mixed-citation>
</ref>
<ref id="B9"><mixed-citation publication-type="other"><collab>NICE</collab>
<article-title>Guide to the methods of technology appraisal
</article-title>
<comment>[<ext-link ext-link-type="uri" xlink:href="http://www.nice.org.uk/media/b52/a7/tamethodsguideupdatedjune2008.pdf">http://www.nice.org.uk/media/b52/a7/tamethodsguideupdatedjune2008.pdf</ext-link>
]</comment>
</mixed-citation>
</ref>
<ref id="B10"><mixed-citation publication-type="journal"><name><surname>Ma</surname>
<given-names>J</given-names>
</name>
<name><surname>Thabane</surname>
<given-names>L</given-names>
</name>
<name><surname>Kaczorowski</surname>
<given-names>J</given-names>
</name>
<name><surname>Chambers</surname>
<given-names>L</given-names>
</name>
<name><surname>Dolovich</surname>
<given-names>L</given-names>
</name>
<name><surname>Karwalajtys</surname>
<given-names>T</given-names>
</name>
<name><surname>Levitt</surname>
<given-names>C</given-names>
</name>
<article-title>Comparison of Bayesian and classical methods in the analysis of cluster randomized controlled trials with a binary outcome: the Community Hypertension Assessment Trial (CHAT)</article-title>
<source>BMC Med Res Methodol</source>
<year>2009</year>
<volume>9</volume>
<fpage>37</fpage>
<pub-id pub-id-type="doi">10.1186/1471-2288-9-37</pub-id>
<pub-id pub-id-type="pmid">19531226</pub-id>
</mixed-citation>
</ref>
<ref id="B11"><mixed-citation publication-type="journal"><name><surname>Peters</surname>
<given-names>TJ</given-names>
</name>
<name><surname>Richards</surname>
<given-names>SH</given-names>
</name>
<name><surname>Bankhead</surname>
<given-names>CR</given-names>
</name>
<name><surname>Ades</surname>
<given-names>AE</given-names>
</name>
<name><surname>Sterne</surname>
<given-names>JA</given-names>
</name>
<article-title>Comparison of methods for analysing cluster randomized trials: an example involving a factorial design</article-title>
<source>Int J Epidemiol</source>
<year>2003</year>
<volume>32</volume>
<issue>5</issue>
<fpage>840</fpage>
<lpage>846</lpage>
<pub-id pub-id-type="doi">10.1093/ije/dyg228</pub-id>
<pub-id pub-id-type="pmid">14559762</pub-id>
</mixed-citation>
</ref>
<ref id="B12"><mixed-citation publication-type="journal"><name><surname>Chu</surname>
<given-names>R</given-names>
</name>
<name><surname>Thabane</surname>
<given-names>L</given-names>
</name>
<name><surname>Ma</surname>
<given-names>J</given-names>
</name>
<name><surname>Holbrook</surname>
<given-names>A</given-names>
</name>
<name><surname>Pullenayegum</surname>
<given-names>E</given-names>
</name>
<name><surname>Devereaux</surname>
<given-names>PJ</given-names>
</name>
<article-title>Comparing methods to estimate treatment effects on a continuous outcome in multicentre randomized controlled trials: a simulation study</article-title>
<source>BMC Med Res Methodol</source>
<year>2011</year>
<volume>11</volume>
<fpage>21</fpage>
<pub-id pub-id-type="doi">10.1186/1471-2288-11-21</pub-id>
<pub-id pub-id-type="pmid">21338524</pub-id>
</mixed-citation>
</ref>
<ref id="B13"><mixed-citation publication-type="book"><name><surname>Kleinbaum</surname>
<given-names>DG</given-names>
</name>
<name><surname>Klein</surname>
<given-names>M</given-names>
</name>
<source>Survival Analysis – A-Self Learning Text</source>
<year>2012</year>
<edition>3</edition>
<publisher-name>Springer</publisher-name>
</mixed-citation>
</ref>
<ref id="B14"><mixed-citation publication-type="book"><name><surname>Barnett</surname>
<given-names>V</given-names>
</name>
<name><surname>Lewis</surname>
<given-names>T</given-names>
</name>
<source>Outliers in Statistical Data</source>
<year>1994</year>
<edition>3</edition>
<publisher-name>John Wiley & Sons</publisher-name>
</mixed-citation>
</ref>
<ref id="B15"><mixed-citation publication-type="journal"><name><surname>Grubbs</surname>
<given-names>FE</given-names>
</name>
<article-title>Procedures for detecting outlying observations in samples</article-title>
<source>Technometrics</source>
<year>1969</year>
<volume>11</volume>
<fpage>1</fpage>
<lpage>21</lpage>
<pub-id pub-id-type="doi">10.1080/00401706.1969.10490657</pub-id>
</mixed-citation>
</ref>
<ref id="B16"><mixed-citation publication-type="journal"><name><surname>Thabane</surname>
<given-names>L</given-names>
</name>
<name><surname>Akhtar-Danesh</surname>
<given-names>N</given-names>
</name>
<article-title>Guidelines for reporting descriptive statistics in health research</article-title>
<source>Nurse Res</source>
<year>2008</year>
<volume>15</volume>
<issue>2</issue>
<fpage>72</fpage>
<lpage>81</lpage>
<pub-id pub-id-type="pmid">18283764</pub-id>
</mixed-citation>
</ref>
<ref id="B17"><mixed-citation publication-type="journal"><name><surname>Williams</surname>
<given-names>NH</given-names>
</name>
<name><surname>Edwards</surname>
<given-names>RT</given-names>
</name>
<name><surname>Linck</surname>
<given-names>P</given-names>
</name>
<name><surname>Muntz</surname>
<given-names>R</given-names>
</name>
<name><surname>Hibbs</surname>
<given-names>R</given-names>
</name>
<name><surname>Wilkinson</surname>
<given-names>C</given-names>
</name>
<name><surname>Russell</surname>
<given-names>I</given-names>
</name>
<name><surname>Russell</surname>
<given-names>D</given-names>
</name>
<name><surname>Hounsome</surname>
<given-names>B</given-names>
</name>
<article-title>Cost-utility analysis of osteopathy in primary care: results from a pragmatic randomized controlled trial</article-title>
<source>Fam Pract</source>
<year>2004</year>
<volume>21</volume>
<issue>6</issue>
<fpage>643</fpage>
<lpage>650</lpage>
<pub-id pub-id-type="doi">10.1093/fampra/cmh612</pub-id>
<pub-id pub-id-type="pmid">15531626</pub-id>
</mixed-citation>
</ref>
<ref id="B18"><mixed-citation publication-type="journal"><name><surname>Zetta</surname>
<given-names>S</given-names>
</name>
<name><surname>Smith</surname>
<given-names>K</given-names>
</name>
<name><surname>Jones</surname>
<given-names>M</given-names>
</name>
<name><surname>Allcoat</surname>
<given-names>P</given-names>
</name>
<name><surname>Sullivan</surname>
<given-names>F</given-names>
</name>
<article-title>Evaluating the Angina Plan in Patients Admitted to Hospital with Angina: A Randomized Controlled Trial</article-title>
<source>Cardiovascular Therapeutics</source>
<year>2011</year>
<volume>29</volume>
<issue>2</issue>
<fpage>112</fpage>
<lpage>124</lpage>
<pub-id pub-id-type="doi">10.1111/j.1755-5922.2009.00109.x</pub-id>
<pub-id pub-id-type="pmid">20041881</pub-id>
</mixed-citation>
</ref>
<ref id="B19"><mixed-citation publication-type="journal"><name><surname>Morden</surname>
<given-names>JP</given-names>
</name>
<name><surname>Lambert</surname>
<given-names>PC</given-names>
</name>
<name><surname>Latimer</surname>
<given-names>N</given-names>
</name>
<name><surname>Abrams</surname>
<given-names>KR</given-names>
</name>
<name><surname>Wailoo</surname>
<given-names>AJ</given-names>
</name>
<article-title>Assessing methods for dealing with treatment switching in randomised controlled trials: a simulation study</article-title>
<source>BMC Med Res Methodol</source>
<year>2011</year>
<volume>11</volume>
<fpage>4</fpage>
<pub-id pub-id-type="doi">10.1186/1471-2288-11-4</pub-id>
<pub-id pub-id-type="pmid">21223539</pub-id>
</mixed-citation>
</ref>
<ref id="B20"><mixed-citation publication-type="journal"><name><surname>White</surname>
<given-names>IR</given-names>
</name>
<name><surname>Walker</surname>
<given-names>S</given-names>
</name>
<name><surname>Babiker</surname>
<given-names>AG</given-names>
</name>
<name><surname>Darbyshire</surname>
<given-names>JH</given-names>
</name>
<article-title>Impact of treatment changes on the interpretation of the Concorde trial</article-title>
<source>AIDS</source>
<year>1997</year>
<volume>11</volume>
<issue>8</issue>
<fpage>999</fpage>
<lpage>1006</lpage>
<pub-id pub-id-type="doi">10.1097/00002030-199708000-00008</pub-id>
<pub-id pub-id-type="pmid">9223734</pub-id>
</mixed-citation>
</ref>
<ref id="B21"><mixed-citation publication-type="journal"><name><surname>Borrelli</surname>
<given-names>B</given-names>
</name>
<article-title>The assessment, monitoring, and enhancement of treatment fidelity in public health clinical trials</article-title>
<source>J Public Health Dent</source>
<year>2011</year>
<volume>71</volume>
<issue>Suppl 1</issue>
<fpage>S52</fpage>
<lpage>S63</lpage>
<pub-id pub-id-type="pmid">21656954</pub-id>
</mixed-citation>
</ref>
<ref id="B22"><mixed-citation publication-type="journal"><name><surname>Lawton</surname>
<given-names>J</given-names>
</name>
<name><surname>Jenkins</surname>
<given-names>N</given-names>
</name>
<name><surname>Darbyshire</surname>
<given-names>JL</given-names>
</name>
<name><surname>Holman</surname>
<given-names>RR</given-names>
</name>
<name><surname>Farmer</surname>
<given-names>AJ</given-names>
</name>
<name><surname>Hallowell</surname>
<given-names>N</given-names>
</name>
<article-title>Challenges of maintaining research protocol fidelity in a clinical care setting: a qualitative study of the experiences and views of patients and staff participating in a randomized controlled trial</article-title>
<source>Trials</source>
<year>2011</year>
<volume>12</volume>
<fpage>108</fpage>
<pub-id pub-id-type="doi">10.1186/1745-6215-12-108</pub-id>
<pub-id pub-id-type="pmid">21542916</pub-id>
</mixed-citation>
</ref>
<ref id="B23"><mixed-citation publication-type="journal"><name><surname>Ye</surname>
<given-names>C</given-names>
</name>
<name><surname>Giangregorio</surname>
<given-names>L</given-names>
</name>
<name><surname>Holbrook</surname>
<given-names>A</given-names>
</name>
<name><surname>Pullenayegum</surname>
<given-names>E</given-names>
</name>
<name><surname>Goldsmith</surname>
<given-names>CH</given-names>
</name>
<name><surname>Thabane</surname>
<given-names>L</given-names>
</name>
<article-title>Data withdrawal in randomized controlled trials: Defining the problem and proposing solutions: a commentary</article-title>
<source>Contemp Clin Trials</source>
<year>2011</year>
<volume>32</volume>
<issue>3</issue>
<fpage>318</fpage>
<lpage>322</lpage>
<pub-id pub-id-type="doi">10.1016/j.cct.2011.01.016</pub-id>
<pub-id pub-id-type="pmid">21300179</pub-id>
</mixed-citation>
</ref>
<ref id="B24"><mixed-citation publication-type="journal"><name><surname>Horwitz</surname>
<given-names>RI</given-names>
</name>
<name><surname>Horwitz</surname>
<given-names>SM</given-names>
</name>
<article-title>Adherence to treatment and health outcomes</article-title>
<source>Arch Intern Med</source>
<year>1993</year>
<volume>153</volume>
<issue>16</issue>
<fpage>1863</fpage>
<lpage>1868</lpage>
<pub-id pub-id-type="doi">10.1001/archinte.1993.00410160017001</pub-id>
<pub-id pub-id-type="pmid">8250647</pub-id>
</mixed-citation>
</ref>
<ref id="B25"><mixed-citation publication-type="journal"><name><surname>Peduzzi</surname>
<given-names>P</given-names>
</name>
<name><surname>Wittes</surname>
<given-names>J</given-names>
</name>
<name><surname>Detre</surname>
<given-names>K</given-names>
</name>
<name><surname>Holford</surname>
<given-names>T</given-names>
</name>
<article-title>Analysis as-randomized and the problem of non-adherence: an example from the Veterans Affairs Randomized Trial of Coronary Artery Bypass Surgery</article-title>
<source>Stat Med</source>
<year>1993</year>
<volume>12</volume>
<issue>13</issue>
<fpage>1185</fpage>
<lpage>1195</lpage>
<pub-id pub-id-type="doi">10.1002/sim.4780121302</pub-id>
<pub-id pub-id-type="pmid">8210821</pub-id>
</mixed-citation>
</ref>
<ref id="B26"><mixed-citation publication-type="journal"><name><surname>Montori</surname>
<given-names>VM</given-names>
</name>
<name><surname>Guyatt</surname>
<given-names>GH</given-names>
</name>
<article-title>Intention-to-treat principle</article-title>
<source>CMAJ</source>
<year>2001</year>
<volume>165</volume>
<issue>10</issue>
<fpage>1339</fpage>
<lpage>1341</lpage>
<pub-id pub-id-type="pmid">11760981</pub-id>
</mixed-citation>
</ref>
<ref id="B27"><mixed-citation publication-type="journal"><name><surname>Gibaldi</surname>
<given-names>M</given-names>
</name>
<name><surname>Sullivan</surname>
<given-names>S</given-names>
</name>
<article-title>Intention-to-treat analysis in randomized trials: who gets counted?</article-title>
<source>J Clin Pharmacol</source>
<year>1997</year>
<volume>37</volume>
<issue>8</issue>
<fpage>667</fpage>
<lpage>672</lpage>
<pub-id pub-id-type="doi">10.1002/j.1552-4604.1997.tb04353.x</pub-id>
<pub-id pub-id-type="pmid">9378838</pub-id>
</mixed-citation>
</ref>
<ref id="B28"><mixed-citation publication-type="book"><name><surname>Porta</surname>
<given-names>M</given-names>
</name>
<source>A dictionary of epidemiology</source>
<year>2008</year>
<edition>5</edition>
<publisher-name>Oxford: Oxford University Press, Inc</publisher-name>
</mixed-citation>
</ref>
<ref id="B29"><mixed-citation publication-type="book"><name><surname>Everitt</surname>
<given-names>B</given-names>
</name>
<source>Medical statistics from A to Z</source>
<year>2006</year>
<edition>2</edition>
<publisher-name>Cambridge: Cambridge University Press</publisher-name>
</mixed-citation>
</ref>
<ref id="B30"><mixed-citation publication-type="journal"><name><surname>Sainani</surname>
<given-names>KL</given-names>
</name>
<article-title>Making sense of intention-to-treat</article-title>
<source>PM R</source>
<year>2010</year>
<volume>2</volume>
<issue>3</issue>
<fpage>209</fpage>
<lpage>213</lpage>
<pub-id pub-id-type="doi">10.1016/j.pmrj.2010.01.004</pub-id>
<pub-id pub-id-type="pmid">20359686</pub-id>
</mixed-citation>
</ref>
<ref id="B31"><mixed-citation publication-type="journal"><name><surname>Bendtsen</surname>
<given-names>P</given-names>
</name>
<name><surname>McCambridge</surname>
<given-names>J</given-names>
</name>
<name><surname>Bendtsen</surname>
<given-names>M</given-names>
</name>
<name><surname>Karlsson</surname>
<given-names>N</given-names>
</name>
<name><surname>Nilsen</surname>
<given-names>P</given-names>
</name>
<article-title>Effectiveness of a proactive mail-based alcohol internet intervention for university students: dismantling the assessment and feedback components in a randomized controlled trial</article-title>
<source>J Med Internet Res</source>
<year>2012</year>
<volume>14</volume>
<issue>5</issue>
<fpage>e142</fpage>
<pub-id pub-id-type="doi">10.2196/jmir.2062</pub-id>
<pub-id pub-id-type="pmid">23113955</pub-id>
</mixed-citation>
</ref>
<ref id="B32"><mixed-citation publication-type="journal"><name><surname>Brox</surname>
<given-names>JI</given-names>
</name>
<name><surname>Nygaard</surname>
<given-names>OP</given-names>
</name>
<name><surname>Holm</surname>
<given-names>I</given-names>
</name>
<name><surname>Keller</surname>
<given-names>A</given-names>
</name>
<name><surname>Ingebrigtsen</surname>
<given-names>T</given-names>
</name>
<name><surname>Reikeras</surname>
<given-names>O</given-names>
</name>
<article-title>Four-year follow-up of surgical versus non-surgical therapy for chronic low back pain</article-title>
<source>Ann Rheum Dis</source>
<year>2010</year>
<volume>69</volume>
<issue>9</issue>
<fpage>1643</fpage>
<lpage>1648</lpage>
<pub-id pub-id-type="doi">10.1136/ard.2009.108902</pub-id>
<pub-id pub-id-type="pmid">19635718</pub-id>
</mixed-citation>
</ref>
<ref id="B33"><mixed-citation publication-type="book"><name><surname>McKnight</surname>
<given-names>PE</given-names>
</name>
<name><surname>McKnight</surname>
<given-names>KM</given-names>
</name>
<name><surname>Sidani</surname>
<given-names>S</given-names>
</name>
<name><surname>Figueredo</surname>
<given-names>AJ</given-names>
</name>
<source>Missing Data: A Gentle Introduction</source>
<year>2007</year>
<publisher-name>New York, NY: Guilford</publisher-name>
</mixed-citation>
</ref>
<ref id="B34"><mixed-citation publication-type="journal"><name><surname>Graham</surname>
<given-names>JW</given-names>
</name>
<article-title>Missing data analysis: making it work in the real world</article-title>
<source>Annu Rev Psychol</source>
<year>2009</year>
<volume>60</volume>
<fpage>549</fpage>
<lpage>576</lpage>
<pub-id pub-id-type="doi">10.1146/annurev.psych.58.110405.085530</pub-id>
<pub-id pub-id-type="pmid">18652544</pub-id>
</mixed-citation>
</ref>
<ref id="B35"><mixed-citation publication-type="journal"><name><surname>Little</surname>
<given-names>RJ</given-names>
</name>
<name><surname>D'Agostino</surname>
<given-names>R</given-names>
</name>
<name><surname>Cohen</surname>
<given-names>ML</given-names>
</name>
<name><surname>Dickersin</surname>
<given-names>K</given-names>
</name>
<name><surname>Emerson</surname>
<given-names>SS</given-names>
</name>
<name><surname>Farrar</surname>
<given-names>JT</given-names>
</name>
<name><surname>Frangakis</surname>
<given-names>C</given-names>
</name>
<name><surname>Hogan</surname>
<given-names>JW</given-names>
</name>
<name><surname>Molenberghs</surname>
<given-names>G</given-names>
</name>
<name><surname>Murphy</surname>
<given-names>SA</given-names>
</name>
<etal></etal>
<article-title>The Prevention and Treatment of Missing Data in Clinical Trials</article-title>
<source>New England Journal of Medicine</source>
<year>2012</year>
<volume>367</volume>
<issue>14</issue>
<fpage>1355</fpage>
<lpage>1360</lpage>
<pub-id pub-id-type="doi">10.1056/NEJMsr1203730</pub-id>
<pub-id pub-id-type="pmid">23034025</pub-id>
</mixed-citation>
</ref>
<ref id="B36"><mixed-citation publication-type="book"><name><surname>Little</surname>
<given-names>RJA</given-names>
</name>
<name><surname>Rubin</surname>
<given-names>DB</given-names>
</name>
<source>Statistical Analysis with Missing Data</source>
<year>2002</year>
<edition>2</edition>
<publisher-name>New York NY: Wiley</publisher-name>
</mixed-citation>
</ref>
<ref id="B37"><mixed-citation publication-type="book"><name><surname>Rubin</surname>
<given-names>DB</given-names>
</name>
<source>Multiple Imputation for Nonresponse in Surveys</source>
<year>1987</year>
<publisher-name>John Wiley & Sons, Inc: New York NY</publisher-name>
</mixed-citation>
</ref>
<ref id="B38"><mixed-citation publication-type="book"><name><surname>Schafer</surname>
<given-names>JL</given-names>
</name>
<source>Analysis of Incomplete Multivariate Data</source>
<year>1997</year>
<publisher-name>New York: Chapman and Hall</publisher-name>
</mixed-citation>
</ref>
<ref id="B39"><mixed-citation publication-type="journal"><name><surname>Son</surname>
<given-names>H</given-names>
</name>
<name><surname>Friedmann</surname>
<given-names>E</given-names>
</name>
<name><surname>Thomas</surname>
<given-names>SA</given-names>
</name>
<article-title>Application of pattern mixture models to address missing data in longitudinal data analysis using SPSS</article-title>
<source>Nursing research</source>
<year>2012</year>
<volume>61</volume>
<issue>3</issue>
<fpage>195</fpage>
<lpage>203</lpage>
<pub-id pub-id-type="doi">10.1097/NNR.0b013e3182541d8c</pub-id>
<pub-id pub-id-type="pmid">22551994</pub-id>
</mixed-citation>
</ref>
<ref id="B40"><mixed-citation publication-type="journal"><name><surname>Peters</surname>
<given-names>SA</given-names>
</name>
<name><surname>Bots</surname>
<given-names>ML</given-names>
</name>
<name><surname>den Ruijter</surname>
<given-names>HM</given-names>
</name>
<name><surname>Palmer</surname>
<given-names>MK</given-names>
</name>
<name><surname>Grobbee</surname>
<given-names>DE</given-names>
</name>
<name><surname>Crouse</surname>
<given-names>JR</given-names>
<suffix>3rd</suffix>
</name>
<name><surname>O'Leary</surname>
<given-names>DH</given-names>
</name>
<name><surname>Evans</surname>
<given-names>GW</given-names>
</name>
<name><surname>Raichlen</surname>
<given-names>JS</given-names>
</name>
<name><surname>Moons</surname>
<given-names>KG</given-names>
</name>
<etal></etal>
<article-title>Multiple imputation of missing repeated outcome measurements did not add to linear mixed-effects models</article-title>
<source>J Clin Epidemiol</source>
<year>2012</year>
<volume>65</volume>
<issue>6</issue>
<fpage>686</fpage>
<lpage>695</lpage>
<pub-id pub-id-type="doi">10.1016/j.jclinepi.2011.11.012</pub-id>
<pub-id pub-id-type="pmid">22459429</pub-id>
</mixed-citation>
</ref>
<ref id="B41"><mixed-citation publication-type="journal"><name><surname>Zhang</surname>
<given-names>H</given-names>
</name>
<name><surname>Paik</surname>
<given-names>MC</given-names>
</name>
<article-title>Handling missing responses in generalized linear mixed model without specifying missing mechanism</article-title>
<source>J Biopharm Stat</source>
<year>2009</year>
<volume>19</volume>
<issue>6</issue>
<fpage>1001</fpage>
<lpage>1017</lpage>
<pub-id pub-id-type="doi">10.1080/10543400903242761</pub-id>
<pub-id pub-id-type="pmid">20183461</pub-id>
</mixed-citation>
</ref>
<ref id="B42"><mixed-citation publication-type="journal"><name><surname>Chen</surname>
<given-names>HY</given-names>
</name>
<name><surname>Gao</surname>
<given-names>S</given-names>
</name>
<article-title>Estimation of average treatment effect with incompletely observed longitudinal data: application to a smoking cessation study</article-title>
<source>Statistics in medicine</source>
<year>2009</year>
<volume>28</volume>
<issue>19</issue>
<fpage>2451</fpage>
<lpage>2472</lpage>
<pub-id pub-id-type="doi">10.1002/sim.3617</pub-id>
<pub-id pub-id-type="pmid">19462416</pub-id>
</mixed-citation>
</ref>
<ref id="B43"><mixed-citation publication-type="journal"><name><surname>Ma</surname>
<given-names>J</given-names>
</name>
<name><surname>Akhtar-Danesh</surname>
<given-names>N</given-names>
</name>
<name><surname>Dolovich</surname>
<given-names>L</given-names>
</name>
<name><surname>Thabane</surname>
<given-names>L</given-names>
</name>
<article-title>Imputation strategies for missing binary outcomes in cluster randomized trials</article-title>
<source>BMC Med Res Methodol</source>
<year>2011</year>
<volume>11</volume>
<fpage>18</fpage>
<pub-id pub-id-type="doi">10.1186/1471-2288-11-18</pub-id>
<pub-id pub-id-type="pmid">21324148</pub-id>
</mixed-citation>
</ref>
<ref id="B44"><mixed-citation publication-type="journal"><name><surname>Kingsley</surname>
<given-names>GH</given-names>
</name>
<name><surname>Kowalczyk</surname>
<given-names>A</given-names>
</name>
<name><surname>Taylor</surname>
<given-names>H</given-names>
</name>
<name><surname>Ibrahim</surname>
<given-names>F</given-names>
</name>
<name><surname>Packham</surname>
<given-names>JC</given-names>
</name>
<name><surname>McHugh</surname>
<given-names>NJ</given-names>
</name>
<name><surname>Mulherin</surname>
<given-names>DM</given-names>
</name>
<name><surname>Kitas</surname>
<given-names>GD</given-names>
</name>
<name><surname>Chakravarty</surname>
<given-names>K</given-names>
</name>
<name><surname>Tom</surname>
<given-names>BD</given-names>
</name>
<etal></etal>
<article-title>A randomized placebo-controlled trial of methotrexate in psoriatic arthritis</article-title>
<source>Rheumatology (Oxford)</source>
<year>2012</year>
<volume>51</volume>
<issue>8</issue>
<fpage>1368</fpage>
<lpage>1377</lpage>
<pub-id pub-id-type="doi">10.1093/rheumatology/kes001</pub-id>
<pub-id pub-id-type="pmid">22344575</pub-id>
</mixed-citation>
</ref>
<ref id="B45"><mixed-citation publication-type="journal"><name><surname>de Pauw</surname>
<given-names>BE</given-names>
</name>
<name><surname>Sable</surname>
<given-names>CA</given-names>
</name>
<name><surname>Walsh</surname>
<given-names>TJ</given-names>
</name>
<name><surname>Lupinacci</surname>
<given-names>RJ</given-names>
</name>
<name><surname>Bourque</surname>
<given-names>MR</given-names>
</name>
<name><surname>Wise</surname>
<given-names>BA</given-names>
</name>
<name><surname>Nguyen</surname>
<given-names>BY</given-names>
</name>
<name><surname>DiNubile</surname>
<given-names>MJ</given-names>
</name>
<name><surname>Teppler</surname>
<given-names>H</given-names>
</name>
<article-title>Impact of alternate definitions of fever resolution on the composite endpoint in clinical trials of empirical antifungal therapy for neutropenic patients with persistent fever: analysis of results from the Caspofungin Empirical Therapy Study</article-title>
<source>Transpl Infect Dis</source>
<year>2006</year>
<volume>8</volume>
<issue>1</issue>
<fpage>31</fpage>
<lpage>37</lpage>
<pub-id pub-id-type="doi">10.1111/j.1399-3062.2006.00127.x</pub-id>
<pub-id pub-id-type="pmid">16623818</pub-id>
</mixed-citation>
</ref>
<ref id="B46"><mixed-citation publication-type="journal"><article-title>A randomized, double-blind, futility clinical trial of creatine and minocycline in early Parkinson disease</article-title>
<source>Neurology</source>
<year>2006</year>
<volume>66</volume>
<issue>5)</issue>
<fpage>664</fpage>
<lpage>671</lpage>
<pub-id pub-id-type="pmid">16481597</pub-id>
</mixed-citation>
</ref>
<ref id="B47"><mixed-citation publication-type="book"><name><surname>Song</surname>
<given-names>P-K</given-names>
</name>
<source>Correlated Data Analysis: Modeling, Analytics and Applications</source>
<year>2007</year>
<publisher-name>New York, NY: Springer Verlag</publisher-name>
</mixed-citation>
</ref>
<ref id="B48"><mixed-citation publication-type="book"><name><surname>Pintilie</surname>
<given-names>M</given-names>
</name>
<source>Competing Risks: A Practical Perspective</source>
<year>2006</year>
<publisher-name>New York, NY: John Wiley</publisher-name>
</mixed-citation>
</ref>
<ref id="B49"><mixed-citation publication-type="journal"><name><surname>Tai</surname>
<given-names>BC</given-names>
</name>
<name><surname>Grundy</surname>
<given-names>R</given-names>
</name>
<name><surname>Machin</surname>
<given-names>D</given-names>
</name>
<article-title>On the importance of accounting for competing risks in pediatric brain cancer: II. Regression modeling and sample size</article-title>
<source>Int J Radiat Oncol Biol Phys</source>
<year>2011</year>
<volume>79</volume>
<issue>4</issue>
<fpage>1139</fpage>
<lpage>1146</lpage>
<pub-id pub-id-type="doi">10.1016/j.ijrobp.2009.12.024</pub-id>
<pub-id pub-id-type="pmid">20472355</pub-id>
</mixed-citation>
</ref>
<ref id="B50"><mixed-citation publication-type="journal"><name><surname>Holbrook</surname>
<given-names>JT</given-names>
</name>
<name><surname>Wise</surname>
<given-names>RA</given-names>
</name>
<name><surname>Gold</surname>
<given-names>BD</given-names>
</name>
<name><surname>Blake</surname>
<given-names>K</given-names>
</name>
<name><surname>Brown</surname>
<given-names>ED</given-names>
</name>
<name><surname>Castro</surname>
<given-names>M</given-names>
</name>
<name><surname>Dozor</surname>
<given-names>AJ</given-names>
</name>
<name><surname>Lima</surname>
<given-names>JJ</given-names>
</name>
<name><surname>Mastronarde</surname>
<given-names>JG</given-names>
</name>
<name><surname>Sockrider</surname>
<given-names>MM</given-names>
</name>
<etal></etal>
<article-title>Lansoprazole for children with poorly controlled asthma: a randomized controlled trial</article-title>
<source>JAMA</source>
<year>2012</year>
<volume>307</volume>
<issue>4</issue>
<fpage>373</fpage>
<lpage>381</lpage>
<pub-id pub-id-type="pmid">22274684</pub-id>
</mixed-citation>
</ref>
<ref id="B51"><mixed-citation publication-type="journal"><name><surname>Holbrook</surname>
<given-names>A</given-names>
</name>
<name><surname>Thabane</surname>
<given-names>L</given-names>
</name>
<name><surname>Keshavjee</surname>
<given-names>K</given-names>
</name>
<name><surname>Dolovich</surname>
<given-names>L</given-names>
</name>
<name><surname>Bernstein</surname>
<given-names>B</given-names>
</name>
<name><surname>Chan</surname>
<given-names>D</given-names>
</name>
<name><surname>Troyan</surname>
<given-names>S</given-names>
</name>
<name><surname>Foster</surname>
<given-names>G</given-names>
</name>
<name><surname>Gerstein</surname>
<given-names>H</given-names>
</name>
<article-title>Individualized electronic decision support and reminders to improve diabetes care in the community: COMPETE II randomized trial</article-title>
<source>CMAJ: Canadian Medical Association journal = journal de l’Association medicale canadienne</source>
<year>2009</year>
<volume>181</volume>
<issue>1–2</issue>
<fpage>37</fpage>
<lpage>44</lpage>
</mixed-citation>
</ref>
<ref id="B52"><mixed-citation publication-type="book"><name><surname>Hilbe</surname>
<given-names>JM</given-names>
</name>
<source>Negative Binomial Regression</source>
<year>2011</year>
<edition>2</edition>
<publisher-name>Cambridge: Cambridge University Press</publisher-name>
</mixed-citation>
</ref>
<ref id="B53"><mixed-citation publication-type="journal"><name><surname>Forsblom</surname>
<given-names>C</given-names>
</name>
<name><surname>Harjutsalo</surname>
<given-names>V</given-names>
</name>
<name><surname>Thorn</surname>
<given-names>LM</given-names>
</name>
<name><surname>Waden</surname>
<given-names>J</given-names>
</name>
<name><surname>Tolonen</surname>
<given-names>N</given-names>
</name>
<name><surname>Saraheimo</surname>
<given-names>M</given-names>
</name>
<name><surname>Gordin</surname>
<given-names>D</given-names>
</name>
<name><surname>Moran</surname>
<given-names>JL</given-names>
</name>
<name><surname>Thomas</surname>
<given-names>MC</given-names>
</name>
<name><surname>Groop</surname>
<given-names>PH</given-names>
</name>
<article-title>Competing-risk analysis of ESRD and death among patients with type 1 diabetes and macroalbuminuria</article-title>
<source>J Am Soc Nephrol</source>
<year>2011</year>
<volume>22</volume>
<issue>3</issue>
<fpage>537</fpage>
<lpage>544</lpage>
<pub-id pub-id-type="doi">10.1681/ASN.2010020194</pub-id>
<pub-id pub-id-type="pmid">21335512</pub-id>
</mixed-citation>
</ref>
<ref id="B54"><mixed-citation publication-type="journal"><name><surname>Grams</surname>
<given-names>ME</given-names>
</name>
<name><surname>Coresh</surname>
<given-names>J</given-names>
</name>
<name><surname>Segev</surname>
<given-names>DL</given-names>
</name>
<name><surname>Kucirka</surname>
<given-names>LM</given-names>
</name>
<name><surname>Tighiouart</surname>
<given-names>H</given-names>
</name>
<name><surname>Sarnak</surname>
<given-names>MJ</given-names>
</name>
<article-title>Vascular disease, ESRD, and death: interpreting competing risk analyses</article-title>
<source>Clin J Am Soc Nephrol</source>
<year>2012</year>
<volume>7</volume>
<issue>10</issue>
<fpage>1606</fpage>
<lpage>1614</lpage>
<pub-id pub-id-type="doi">10.2215/CJN.03460412</pub-id>
<pub-id pub-id-type="pmid">22859747</pub-id>
</mixed-citation>
</ref>
<ref id="B55"><mixed-citation publication-type="journal"><name><surname>Lim</surname>
<given-names>HJ</given-names>
</name>
<name><surname>Zhang</surname>
<given-names>X</given-names>
</name>
<name><surname>Dyck</surname>
<given-names>R</given-names>
</name>
<name><surname>Osgood</surname>
<given-names>N</given-names>
</name>
<article-title>Methods of competing risks analysis of end-stage renal disease and mortality among people with diabetes</article-title>
<source>BMC Med Res Methodol</source>
<year>2010</year>
<volume>10</volume>
<fpage>97</fpage>
<pub-id pub-id-type="doi">10.1186/1471-2288-10-97</pub-id>
<pub-id pub-id-type="pmid">20964855</pub-id>
</mixed-citation>
</ref>
<ref id="B56"><mixed-citation publication-type="journal"><name><surname>Chu</surname>
<given-names>R</given-names>
</name>
<name><surname>Walter</surname>
<given-names>SD</given-names>
</name>
<name><surname>Guyatt</surname>
<given-names>G</given-names>
</name>
<name><surname>Devereaux</surname>
<given-names>PJ</given-names>
</name>
<name><surname>Walsh</surname>
<given-names>M</given-names>
</name>
<name><surname>Thorlund</surname>
<given-names>K</given-names>
</name>
<name><surname>Thabane</surname>
<given-names>L</given-names>
</name>
<article-title>Assessment and implication of prognostic imbalance in randomized controlled trials with a binary outcome–a simulation study</article-title>
<source>PLoS One</source>
<year>2012</year>
<volume>7</volume>
<issue>5</issue>
<fpage>e36677</fpage>
<pub-id pub-id-type="doi">10.1371/journal.pone.0036677</pub-id>
<pub-id pub-id-type="pmid">22629322</pub-id>
</mixed-citation>
</ref>
<ref id="B57"><mixed-citation publication-type="journal"><name><surname>Bowen</surname>
<given-names>A</given-names>
</name>
<name><surname>Hesketh</surname>
<given-names>A</given-names>
</name>
<name><surname>Patchick</surname>
<given-names>E</given-names>
</name>
<name><surname>Young</surname>
<given-names>A</given-names>
</name>
<name><surname>Davies</surname>
<given-names>L</given-names>
</name>
<name><surname>Vail</surname>
<given-names>A</given-names>
</name>
<name><surname>Long</surname>
<given-names>AF</given-names>
</name>
<name><surname>Watkins</surname>
<given-names>C</given-names>
</name>
<name><surname>Wilkinson</surname>
<given-names>M</given-names>
</name>
<name><surname>Pearl</surname>
<given-names>G</given-names>
</name>
<etal></etal>
<article-title>Effectiveness of enhanced communication therapy in the first four months after stroke for aphasia and dysarthria: a randomised controlled trial</article-title>
<source>BMJ</source>
<year>2012</year>
<volume>345</volume>
<fpage>e4407</fpage>
<pub-id pub-id-type="doi">10.1136/bmj.e4407</pub-id>
<pub-id pub-id-type="pmid">22797843</pub-id>
</mixed-citation>
</ref>
<ref id="B58"><mixed-citation publication-type="book"><name><surname>Spiegelhalter</surname>
<given-names>DJ</given-names>
</name>
<name><surname>Best</surname>
<given-names>NG</given-names>
</name>
<name><surname>Lunn</surname>
<given-names>D</given-names>
</name>
<name><surname>Thomas</surname>
<given-names>A</given-names>
</name>
<source>Bayesian Analysis using BUGS: A Practical Introduction</source>
<year>2009</year>
<publisher-name>New York, NY: Chapman and Hall</publisher-name>
</mixed-citation>
</ref>
<ref id="B59"><mixed-citation publication-type="journal"><name><surname>Byers</surname>
<given-names>AL</given-names>
</name>
<name><surname>Allore</surname>
<given-names>H</given-names>
</name>
<name><surname>Gill</surname>
<given-names>TM</given-names>
</name>
<name><surname>Peduzzi</surname>
<given-names>PN</given-names>
</name>
<article-title>Application of negative binomial modeling for discrete outcomes: a case study in aging research</article-title>
<source>J Clin Epidemiol</source>
<year>2003</year>
<volume>56</volume>
<issue>6</issue>
<fpage>559</fpage>
<lpage>564</lpage>
<pub-id pub-id-type="doi">10.1016/S0895-4356(03)00028-3</pub-id>
<pub-id pub-id-type="pmid">12873651</pub-id>
</mixed-citation>
</ref>
<ref id="B60"><mixed-citation publication-type="journal"><name><surname>Yusuf</surname>
<given-names>S</given-names>
</name>
<name><surname>Wittes</surname>
<given-names>J</given-names>
</name>
<name><surname>Probstfield</surname>
<given-names>J</given-names>
</name>
<name><surname>Tyroler</surname>
<given-names>HA</given-names>
</name>
<article-title>Analysis and interpretation of treatment effects in subgroups of patients in randomized clinical trials</article-title>
<source>JAMA: the journal of the American Medical Association</source>
<year>1991</year>
<volume>266</volume>
<issue>1</issue>
<fpage>93</fpage>
<lpage>98</lpage>
<pub-id pub-id-type="doi">10.1001/jama.1991.03470010097038</pub-id>
</mixed-citation>
</ref>
<ref id="B61"><mixed-citation publication-type="journal"><name><surname>Altman</surname>
<given-names>DG</given-names>
</name>
<article-title>Better reporting of randomised controlled trials: the CONSORT statement</article-title>
<source>BMJ</source>
<year>1996</year>
<volume>313</volume>
<issue>7057</issue>
<fpage>570</fpage>
<lpage>571</lpage>
<pub-id pub-id-type="doi">10.1136/bmj.313.7057.570</pub-id>
<pub-id pub-id-type="pmid">8806240</pub-id>
</mixed-citation>
</ref>
<ref id="B62"><mixed-citation publication-type="journal"><name><surname>Mauskopf</surname>
<given-names>JA</given-names>
</name>
<name><surname>Sullivan</surname>
<given-names>SD</given-names>
</name>
<name><surname>Annemans</surname>
<given-names>L</given-names>
</name>
<name><surname>Caro</surname>
<given-names>J</given-names>
</name>
<name><surname>Mullins</surname>
<given-names>CD</given-names>
</name>
<name><surname>Nuijten</surname>
<given-names>M</given-names>
</name>
<name><surname>Orlewska</surname>
<given-names>E</given-names>
</name>
<name><surname>Watkins</surname>
<given-names>J</given-names>
</name>
<name><surname>Trueman</surname>
<given-names>P</given-names>
</name>
<article-title>Principles of good practice for budget impact analysis: report of the ISPOR Task Force on good research practices–budget impact analysis</article-title>
<source>Value Health</source>
<year>2007</year>
<volume>10</volume>
<issue>5</issue>
<fpage>336</fpage>
<lpage>347</lpage>
<pub-id pub-id-type="doi">10.1111/j.1524-4733.2007.00187.x</pub-id>
<pub-id pub-id-type="pmid">17888098</pub-id>
</mixed-citation>
</ref>
</ref-list>
</back>
</pmc>
</record>
Pour manipuler ce document sous Unix (Dilib)
EXPLOR_STEP=$WICRI_ROOT/Wicri/Canada/explor/ParkinsonCanadaV1/Data/Pmc/Corpus
HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 000489 | SxmlIndent | more
Ou
HfdSelect -h $EXPLOR_AREA/Data/Pmc/Corpus/biblio.hfd -nk 000489 | SxmlIndent | more
Pour mettre un lien sur cette page dans le réseau Wicri
{{Explor lien |wiki= Wicri/Canada |area= ParkinsonCanadaV1 |flux= Pmc |étape= Corpus |type= RBID |clé= PMC:3720188 |texte= A tutorial on sensitivity analyses in clinical trials: the what, why, when and how }}
Pour générer des pages wiki
HfdIndexSelect -h $EXPLOR_AREA/Data/Pmc/Corpus/RBID.i -Sk "pubmed:23855337" \ | HfdSelect -Kh $EXPLOR_AREA/Data/Pmc/Corpus/biblio.hfd \ | NlmPubMed2Wicri -a ParkinsonCanadaV1
![]() | This area was generated with Dilib version V0.6.29. | ![]() |