Too good to be true: when overwhelming evidence fails to convince
Identifieur interne : 000485 ( Pmc/Corpus ); précédent : 000484; suivant : 000486Too good to be true: when overwhelming evidence fails to convince
Auteurs : Lachlan J. Gunn ; François Chapeau-Blondeau ; Mark D. Mcdonnell ; Bruce R. Davis ; Andrew Allison ; Derek AbbottSource :
- Proceedings. Mathematical, Physical, and Engineering Sciences / The Royal Society [ 1364-5021 ] ; 2016.
Abstract
Is it possible for a large sequence of measurements or observations, which support a hypothesis, to counterintuitively decrease our confidence? Can unanimous support be too good to be true? The assumption of independence is often made in good faith; however, rarely is consideration given to whether a systemic failure has occurred. Taking this into account can cause certainty in a hypothesis to decrease as the evidence for it becomes apparently stronger. We perform a probabilistic Bayesian analysis of this effect with examples based on (i) archaeological evidence, (ii) weighing of legal evidence and (iii) cryptographic primality testing. In this paper, we investigate the effects of small error rates in a set of measurements or observations. We find that even with very low systemic failure rates, high confidence is surprisingly difficult to achieve; in particular, we find that certain analyses of cryptographically important numerical tests are highly optimistic, underestimating their false-negative rate by as much as a factor of 280.
Url:
DOI: 10.1098/rspa.2015.0748
PubMed: 27118917
PubMed Central: 4841483
Links to Exploration step
PMC:4841483Le document en format XML
<record><TEI><teiHeader><fileDesc><titleStmt><title xml:lang="en">Too good to be true: when overwhelming evidence fails to convince</title><author><name sortKey="Gunn, Lachlan J" sort="Gunn, Lachlan J" uniqKey="Gunn L" first="Lachlan J." last="Gunn">Lachlan J. Gunn</name><affiliation><nlm:aff id="af1"><institution>School of Electrical and Electronic Engineering, The University of Adelaide</institution>,<addr-line>Adelaide 5005, Australia</addr-line></nlm:aff></affiliation></author><author><name sortKey="Chapeau Blondeau, Francois" sort="Chapeau Blondeau, Francois" uniqKey="Chapeau Blondeau F" first="François" last="Chapeau-Blondeau">François Chapeau-Blondeau</name><affiliation><nlm:aff id="af2"><addr-line>Laboratoire Angevin de Recherche en Ingénierie des Systèmes (LARIS)</addr-line>,<institution>University of Angers</institution>,<addr-line>62 avenue Notre Dame du Lac, Angers 49000, France</addr-line></nlm:aff></affiliation></author><author><name sortKey="Mcdonnell, Mark D" sort="Mcdonnell, Mark D" uniqKey="Mcdonnell M" first="Mark D." last="Mcdonnell">Mark D. Mcdonnell</name><affiliation><nlm:aff id="af1"><institution>School of Electrical and Electronic Engineering, The University of Adelaide</institution>,<addr-line>Adelaide 5005, Australia</addr-line></nlm:aff></affiliation><affiliation><nlm:aff id="af3"><institution>School of Information Technology and Mathematical Sciences, University of South Australia</institution>,<addr-line>Mawson Lakes, South Australia 5095, Australia</addr-line></nlm:aff></affiliation></author><author><name sortKey="Davis, Bruce R" sort="Davis, Bruce R" uniqKey="Davis B" first="Bruce R." last="Davis">Bruce R. Davis</name><affiliation><nlm:aff id="af1"><institution>School of Electrical and Electronic Engineering, The University of Adelaide</institution>,<addr-line>Adelaide 5005, Australia</addr-line></nlm:aff></affiliation></author><author><name sortKey="Allison, Andrew" sort="Allison, Andrew" uniqKey="Allison A" first="Andrew" last="Allison">Andrew Allison</name><affiliation><nlm:aff id="af1"><institution>School of Electrical and Electronic Engineering, The University of Adelaide</institution>,<addr-line>Adelaide 5005, Australia</addr-line></nlm:aff></affiliation></author><author><name sortKey="Abbott, Derek" sort="Abbott, Derek" uniqKey="Abbott D" first="Derek" last="Abbott">Derek Abbott</name><affiliation><nlm:aff id="af1"><institution>School of Electrical and Electronic Engineering, The University of Adelaide</institution>,<addr-line>Adelaide 5005, Australia</addr-line></nlm:aff></affiliation></author></titleStmt><publicationStmt><idno type="wicri:source">PMC</idno><idno type="pmid">27118917</idno><idno type="pmc">4841483</idno><idno type="url">http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4841483</idno><idno type="RBID">PMC:4841483</idno><idno type="doi">10.1098/rspa.2015.0748</idno><date when="2016">2016</date><idno type="wicri:Area/Pmc/Corpus">000485</idno><idno type="wicri:explorRef" wicri:stream="Pmc" wicri:step="Corpus" wicri:corpus="PMC">000485</idno></publicationStmt><sourceDesc><biblStruct><analytic><title xml:lang="en" level="a" type="main">Too good to be true: when overwhelming evidence fails to convince</title><author><name sortKey="Gunn, Lachlan J" sort="Gunn, Lachlan J" uniqKey="Gunn L" first="Lachlan J." last="Gunn">Lachlan J. Gunn</name><affiliation><nlm:aff id="af1"><institution>School of Electrical and Electronic Engineering, The University of Adelaide</institution>,<addr-line>Adelaide 5005, Australia</addr-line></nlm:aff></affiliation></author><author><name sortKey="Chapeau Blondeau, Francois" sort="Chapeau Blondeau, Francois" uniqKey="Chapeau Blondeau F" first="François" last="Chapeau-Blondeau">François Chapeau-Blondeau</name><affiliation><nlm:aff id="af2"><addr-line>Laboratoire Angevin de Recherche en Ingénierie des Systèmes (LARIS)</addr-line>,<institution>University of Angers</institution>,<addr-line>62 avenue Notre Dame du Lac, Angers 49000, France</addr-line></nlm:aff></affiliation></author><author><name sortKey="Mcdonnell, Mark D" sort="Mcdonnell, Mark D" uniqKey="Mcdonnell M" first="Mark D." last="Mcdonnell">Mark D. Mcdonnell</name><affiliation><nlm:aff id="af1"><institution>School of Electrical and Electronic Engineering, The University of Adelaide</institution>,<addr-line>Adelaide 5005, Australia</addr-line></nlm:aff></affiliation><affiliation><nlm:aff id="af3"><institution>School of Information Technology and Mathematical Sciences, University of South Australia</institution>,<addr-line>Mawson Lakes, South Australia 5095, Australia</addr-line></nlm:aff></affiliation></author><author><name sortKey="Davis, Bruce R" sort="Davis, Bruce R" uniqKey="Davis B" first="Bruce R." last="Davis">Bruce R. Davis</name><affiliation><nlm:aff id="af1"><institution>School of Electrical and Electronic Engineering, The University of Adelaide</institution>,<addr-line>Adelaide 5005, Australia</addr-line></nlm:aff></affiliation></author><author><name sortKey="Allison, Andrew" sort="Allison, Andrew" uniqKey="Allison A" first="Andrew" last="Allison">Andrew Allison</name><affiliation><nlm:aff id="af1"><institution>School of Electrical and Electronic Engineering, The University of Adelaide</institution>,<addr-line>Adelaide 5005, Australia</addr-line></nlm:aff></affiliation></author><author><name sortKey="Abbott, Derek" sort="Abbott, Derek" uniqKey="Abbott D" first="Derek" last="Abbott">Derek Abbott</name><affiliation><nlm:aff id="af1"><institution>School of Electrical and Electronic Engineering, The University of Adelaide</institution>,<addr-line>Adelaide 5005, Australia</addr-line></nlm:aff></affiliation></author></analytic><series><title level="j">Proceedings. Mathematical, Physical, and Engineering Sciences / The Royal Society</title><idno type="ISSN">1364-5021</idno><idno type="eISSN">1471-2946</idno><imprint><date when="2016">2016</date></imprint></series></biblStruct></sourceDesc></fileDesc><profileDesc><textClass></textClass></profileDesc></teiHeader><front><div type="abstract" xml:lang="en"><p>Is it possible for a large sequence of measurements or observations, which support a hypothesis, to counterintuitively decrease our confidence? Can unanimous support be too good to be true? The assumption of independence is often made in good faith; however, rarely is consideration given to whether a systemic failure has occurred. Taking this into account can cause certainty in a hypothesis to decrease as the evidence for it becomes apparently stronger. We perform a probabilistic Bayesian analysis of this effect with examples based on (i) archaeological evidence, (ii) weighing of legal evidence and (iii) cryptographic primality testing. In this paper, we investigate the effects of small error rates in a set of measurements or observations. We find that even with very low systemic failure rates, high confidence is surprisingly difficult to achieve; in particular, we find that certain analyses of cryptographically important numerical tests are highly optimistic, underestimating their false-negative rate by as much as a factor of 2<sup>80</sup>.</p></div></front></TEI><pmc article-type="research-article"><pmc-comment>The publisher of this article does not allow downloading of the full text in XML form.</pmc-comment>
<front><journal-meta><journal-id journal-id-type="nlm-ta">Proc Math Phys Eng Sci</journal-id><journal-id journal-id-type="iso-abbrev">Proc. Math. Phys. Eng. Sci</journal-id><journal-id journal-id-type="publisher-id">RSPA</journal-id><journal-id journal-id-type="hwp">royprsa</journal-id><journal-title-group><journal-title>Proceedings. Mathematical, Physical, and Engineering Sciences / The Royal Society</journal-title></journal-title-group><issn pub-type="ppub">1364-5021</issn><issn pub-type="epub">1471-2946</issn><publisher><publisher-name>The Royal Society Publishing</publisher-name></publisher></journal-meta><article-meta><article-id pub-id-type="pmid">27118917</article-id><article-id pub-id-type="pmc">4841483</article-id><article-id pub-id-type="doi">10.1098/rspa.2015.0748</article-id><article-id pub-id-type="publisher-id">rspa20150748</article-id><article-categories><subj-group subj-group-type="hwp-journal-coll"><subject>1003</subject><subject>54</subject><subject>182</subject><subject>1008</subject><subject>175</subject></subj-group><subj-group subj-group-type="heading"><subject>Research Articles</subject></subj-group></article-categories><title-group><article-title>Too good to be true: when overwhelming evidence fails to convince</article-title><alt-title alt-title-type="short">Too good to be true</alt-title></title-group><contrib-group><contrib contrib-type="author"><contrib-id contrib-id-type="orcid" authenticated="false">http://orcid.org/0000-0003-1767-7897</contrib-id><name><surname>Gunn</surname><given-names>Lachlan J.</given-names></name><xref ref-type="aff" rid="af1">1</xref><xref ref-type="corresp" rid="cor1"></xref></contrib><contrib contrib-type="author"><name><surname>Chapeau-Blondeau</surname><given-names>François</given-names></name><xref ref-type="aff" rid="af2">2</xref></contrib><contrib contrib-type="author"><name><surname>McDonnell</surname><given-names>Mark D.</given-names></name><xref ref-type="aff" rid="af1">1</xref><xref ref-type="aff" rid="af3">3</xref></contrib><contrib contrib-type="author"><name><surname>Davis</surname><given-names>Bruce R.</given-names></name><xref ref-type="aff" rid="af1">1</xref></contrib><contrib contrib-type="author"><name><surname>Allison</surname><given-names>Andrew</given-names></name><xref ref-type="aff" rid="af1">1</xref></contrib><contrib contrib-type="author"><name><surname>Abbott</surname><given-names>Derek</given-names></name><xref ref-type="aff" rid="af1">1</xref></contrib></contrib-group><aff id="af1"><label>1</label><institution>School of Electrical and Electronic Engineering, The University of Adelaide</institution>,<addr-line>Adelaide 5005, Australia</addr-line></aff><aff id="af2"><label>2</label><addr-line>Laboratoire Angevin de Recherche en Ingénierie des Systèmes (LARIS)</addr-line>,<institution>University of Angers</institution>,<addr-line>62 avenue Notre Dame du Lac, Angers 49000, France</addr-line></aff><aff id="af3"><label>3</label><institution>School of Information Technology and Mathematical Sciences, University of South Australia</institution>,<addr-line>Mawson Lakes, South Australia 5095, Australia</addr-line></aff><author-notes><corresp id="cor1">e-mail: <email>lachlan.gunn@adelaide.edu.au</email></corresp></author-notes><pub-date pub-type="ppub"><month>3</month><year>2016</year></pub-date><volume>472</volume><issue>2187</issue><elocation-id>20150748</elocation-id><history><date date-type="received"><day>28</day><month>10</month><year>2015</year></date><date date-type="accepted"><day>19</day><month>2</month><year>2016</year></date></history><permissions><copyright-statement>© 2016 The Author(s)</copyright-statement><copyright-year>2016</copyright-year><license specific-use="vor" xlink:href="http://royalsocietypublishing.org/licence"><license-p>Published by the Royal Society. All rights reserved.</license-p></license></permissions><self-uri content-type="pdf" xlink:href="rspa20150748.pdf"></self-uri><abstract><p>Is it possible for a large sequence of measurements or observations, which support a hypothesis, to counterintuitively decrease our confidence? Can unanimous support be too good to be true? The assumption of independence is often made in good faith; however, rarely is consideration given to whether a systemic failure has occurred. Taking this into account can cause certainty in a hypothesis to decrease as the evidence for it becomes apparently stronger. We perform a probabilistic Bayesian analysis of this effect with examples based on (i) archaeological evidence, (ii) weighing of legal evidence and (iii) cryptographic primality testing. In this paper, we investigate the effects of small error rates in a set of measurements or observations. We find that even with very low systemic failure rates, high confidence is surprisingly difficult to achieve; in particular, we find that certain analyses of cryptographically important numerical tests are highly optimistic, underestimating their false-negative rate by as much as a factor of 2<sup>80</sup>.</p></abstract><kwd-group><kwd>Bayesian</kwd><kwd>cryptography</kwd><kwd>criminology</kwd></kwd-group><funding-group specific-use="FundRef"><award-group><funding-source><institution-wrap><institution>Department of Education and Training, Australian Government</institution></institution-wrap></funding-source><award-id>Endeavour Research Fellowship (2015)</award-id></award-group></funding-group><funding-group specific-use="FundRef"><award-group><funding-source><institution-wrap><institution>Australian Research Council</institution><institution-id>http://dx.doi.org/10.13039/501100000923</institution-id></institution-wrap></funding-source><award-id>DP1093425</award-id><award-id>FT120100351</award-id></award-group></funding-group><custom-meta-group><custom-meta><meta-name>cover-date</meta-name><meta-value>March, 2016</meta-value></custom-meta></custom-meta-group></article-meta></front></pmc></record>Pour manipuler ce document sous Unix (Dilib)
EXPLOR_STEP=$WICRI_ROOT/Wicri/Asie/explor/AustralieFrV1/Data/Pmc/Corpus
HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 000485 | SxmlIndent | more
Ou
HfdSelect -h $EXPLOR_AREA/Data/Pmc/Corpus/biblio.hfd -nk 000485 | SxmlIndent | more
Pour mettre un lien sur cette page dans le réseau Wicri
{{Explor lien
|wiki= Wicri/Asie
|area= AustralieFrV1
|flux= Pmc
|étape= Corpus
|type= RBID
|clé= PMC:4841483
|texte= Too good to be true: when overwhelming evidence fails to convince
}}
Pour générer des pages wiki
HfdIndexSelect -h $EXPLOR_AREA/Data/Pmc/Corpus/RBID.i -Sk "pubmed:27118917" \
| HfdSelect -Kh $EXPLOR_AREA/Data/Pmc/Corpus/biblio.hfd \
| NlmPubMed2Wicri -a AustralieFrV1
| This area was generated with Dilib version V0.6.33. |  |