A novel measure and significance testing in data analysis of cell image segmentation
Identifieur interne : 000944 ( Pmc/Checkpoint ); précédent : 000943; suivant : 000945A novel measure and significance testing in data analysis of cell image segmentation
Auteurs : Jin Chu Wu ; Michael Halter ; Raghu N. Kacker ; John T. Elliott ; Anne L. PlantSource :
- BMC Bioinformatics [ 1471-2105 ] ; 2017.
Abstract
Cell image segmentation (CIS) is an essential part of quantitative imaging of biological cells. Designing a performance measure and conducting significance testing are critical for evaluating and comparing the CIS algorithms for image-based cell assays in cytometry. Many measures and methods have been proposed and implemented to evaluate segmentation methods. However, computing the standard errors (SE) of the measures and their correlation coefficient is not described, and thus the statistical significance of performance differences between CIS algorithms cannot be assessed.
We propose the total error rate (TER), a novel performance measure for segmenting all cells in the supervised evaluation. The TER statistically aggregates all misclassification error rates (MER) by taking cell sizes as weights. The MERs are for segmenting each single cell in the population. The TER is fully supported by the pairwise comparisons of MERs using 106 manually segmented ground-truth cells with different sizes and seven CIS algorithms taken from ImageJ. Further, the SE and 95% confidence interval (CI) of TER are computed based on the SE of MER that is calculated using the bootstrap method. An algorithm for computing the correlation coefficient of TERs between two CIS algorithms is also provided. Hence, the 95% CI error bars can be used to classify CIS algorithms. The SEs of TERs and their correlation coefficient can be employed to conduct the hypothesis testing, while the CIs overlap, to determine the statistical significance of the performance differences between CIS algorithms.
A novel measure TER of CIS is proposed. The TER’s SEs and correlation coefficient are computed. Thereafter, CIS algorithms can be evaluated and compared statistically by conducting the significance testing.
Url:
DOI: 10.1186/s12859-017-1527-x
PubMed: 28292256
PubMed Central: 5351215
Affiliations:
Links toward previous steps (curation, corpus...)
Links to Exploration step
PMC:5351215Le document en format XML
<record><TEI><teiHeader><fileDesc><titleStmt><title xml:lang="en">A novel measure and significance testing in data analysis of cell image segmentation</title><author><name sortKey="Wu, Jin Chu" sort="Wu, Jin Chu" uniqKey="Wu J" first="Jin Chu" last="Wu">Jin Chu Wu</name><affiliation><nlm:aff id="Aff1"></nlm:aff></affiliation></author><author><name sortKey="Halter, Michael" sort="Halter, Michael" uniqKey="Halter M" first="Michael" last="Halter">Michael Halter</name><affiliation><nlm:aff id="Aff1"></nlm:aff></affiliation></author><author><name sortKey="Kacker, Raghu N" sort="Kacker, Raghu N" uniqKey="Kacker R" first="Raghu N." last="Kacker">Raghu N. Kacker</name><affiliation><nlm:aff id="Aff1"></nlm:aff></affiliation></author><author><name sortKey="Elliott, John T" sort="Elliott, John T" uniqKey="Elliott J" first="John T." last="Elliott">John T. Elliott</name><affiliation><nlm:aff id="Aff1"></nlm:aff></affiliation></author><author><name sortKey="Plant, Anne L" sort="Plant, Anne L" uniqKey="Plant A" first="Anne L." last="Plant">Anne L. Plant</name><affiliation><nlm:aff id="Aff1"></nlm:aff></affiliation></author></titleStmt><publicationStmt><idno type="wicri:source">PMC</idno><idno type="pmid">28292256</idno><idno type="pmc">5351215</idno><idno type="url">http://www.ncbi.nlm.nih.gov/pmc/articles/PMC5351215</idno><idno type="RBID">PMC:5351215</idno><idno type="doi">10.1186/s12859-017-1527-x</idno><date when="2017">2017</date><idno type="wicri:Area/Pmc/Corpus">000265</idno><idno type="wicri:explorRef" wicri:stream="Pmc" wicri:step="Corpus" wicri:corpus="PMC">000265</idno><idno type="wicri:Area/Pmc/Curation">000265</idno><idno type="wicri:explorRef" wicri:stream="Pmc" wicri:step="Curation">000265</idno><idno type="wicri:Area/Pmc/Checkpoint">000944</idno><idno type="wicri:explorRef" wicri:stream="Pmc" wicri:step="Checkpoint">000944</idno></publicationStmt><sourceDesc><biblStruct><analytic><title xml:lang="en" level="a" type="main">A novel measure and significance testing in data analysis of cell image segmentation</title><author><name sortKey="Wu, Jin Chu" sort="Wu, Jin Chu" uniqKey="Wu J" first="Jin Chu" last="Wu">Jin Chu Wu</name><affiliation><nlm:aff id="Aff1"></nlm:aff></affiliation></author><author><name sortKey="Halter, Michael" sort="Halter, Michael" uniqKey="Halter M" first="Michael" last="Halter">Michael Halter</name><affiliation><nlm:aff id="Aff1"></nlm:aff></affiliation></author><author><name sortKey="Kacker, Raghu N" sort="Kacker, Raghu N" uniqKey="Kacker R" first="Raghu N." last="Kacker">Raghu N. Kacker</name><affiliation><nlm:aff id="Aff1"></nlm:aff></affiliation></author><author><name sortKey="Elliott, John T" sort="Elliott, John T" uniqKey="Elliott J" first="John T." last="Elliott">John T. Elliott</name><affiliation><nlm:aff id="Aff1"></nlm:aff></affiliation></author><author><name sortKey="Plant, Anne L" sort="Plant, Anne L" uniqKey="Plant A" first="Anne L." last="Plant">Anne L. Plant</name><affiliation><nlm:aff id="Aff1"></nlm:aff></affiliation></author></analytic><series><title level="j">BMC Bioinformatics</title><idno type="eISSN">1471-2105</idno><imprint><date when="2017">2017</date></imprint></series></biblStruct></sourceDesc></fileDesc><profileDesc><textClass></textClass></profileDesc></teiHeader><front><div type="abstract" xml:lang="en"><sec><title>Background</title><p>Cell image segmentation (CIS) is an essential part of quantitative imaging of biological cells. Designing a performance measure and conducting significance testing are critical for evaluating and comparing the CIS algorithms for image-based cell assays in cytometry. Many measures and methods have been proposed and implemented to evaluate segmentation methods. However, computing the standard errors (SE) of the measures and their correlation coefficient is not described, and thus the statistical significance of performance differences between CIS algorithms cannot be assessed.</p></sec><sec><title>Results</title><p>We propose the total error rate (TER), a novel performance measure for segmenting all cells in the supervised evaluation. The TER statistically aggregates all misclassification error rates (MER) by taking cell sizes as weights. The MERs are for segmenting each single cell in the population. The TER is fully supported by the pairwise comparisons of MERs using 106 manually segmented ground-truth cells with different sizes and seven CIS algorithms taken from ImageJ. Further, the SE and 95% confidence interval (CI) of TER are computed based on the SE of MER that is calculated using the bootstrap method. An algorithm for computing the correlation coefficient of TERs between two CIS algorithms is also provided. Hence, the 95% CI error bars can be used to classify CIS algorithms. The SEs of TERs and their correlation coefficient can be employed to conduct the hypothesis testing, while the CIs overlap, to determine the statistical significance of the performance differences between CIS algorithms.</p></sec><sec><title>Conclusions</title><p>A novel measure TER of CIS is proposed. The TER’s SEs and correlation coefficient are computed. Thereafter, CIS algorithms can be evaluated and compared statistically by conducting the significance testing.</p></sec></div></front><back><div1 type="bibliography"><listBibl><biblStruct><analytic><author><name sortKey="Hill, Aa" uniqKey="Hill A">AA Hill</name></author><author><name sortKey="Lapan, P" uniqKey="Lapan P">P LaPan</name></author><author><name sortKey="Li, Y" uniqKey="Li Y">Y Li</name></author><author><name sortKey="Haney, S" uniqKey="Haney S">S Haney</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Chalfoun, J" uniqKey="Chalfoun J">J Chalfoun</name></author><author><name sortKey="Kociolek, M" uniqKey="Kociolek M">M Kociolek</name></author><author><name sortKey="Dima, A" uniqKey="Dima A">A Dima</name></author><author><name sortKey="Cardone, A" uniqKey="Cardone A">A Cardone</name></author><author><name sortKey="Peskin, A" uniqKey="Peskin A">A Peskin</name></author><author><name sortKey="Bajcsy, P" uniqKey="Bajcsy P">P Bajcsy</name></author><author><name sortKey="Brady, M" uniqKey="Brady M">M Brady</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Bajcsy, P" uniqKey="Bajcsy P">P Bajcsy</name></author><author><name sortKey="Cardone, A" uniqKey="Cardone A">A Cardone</name></author><author><name sortKey="Chalfoun, J" uniqKey="Chalfoun J">J Chalfoun</name></author><author><name sortKey="Halter, M" uniqKey="Halter M">M Halter</name></author><author><name sortKey="Juba, D" uniqKey="Juba D">D Juba</name></author><author><name sortKey="Kociolek, M" uniqKey="Kociolek M">M Kociolek</name></author><author><name sortKey="Majurski, M" uniqKey="Majurski M">M Majurski</name></author><author><name sortKey="Peskin, A" uniqKey="Peskin A">A Peskin</name></author><author><name sortKey="Simon, Cg" uniqKey="Simon C">CG Simon</name></author><author><name sortKey="Simon, M" uniqKey="Simon M">M Simon</name></author><author><name sortKey="Vandecreme, A" uniqKey="Vandecreme A">A Vandecreme</name></author><author><name sortKey="Brady, M" uniqKey="Brady M">M Brady</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Ostle, B" uniqKey="Ostle B">B Ostle</name></author><author><name sortKey="Malone, Lc" uniqKey="Malone L">LC Malone</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Rand, Wm" uniqKey="Rand W">WM Rand</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Unnikrishnan, R" uniqKey="Unnikrishnan R">R Unnikrishnan</name></author><author><name sortKey="Pantofaru, C" uniqKey="Pantofaru C">C Pantofaru</name></author><author><name sortKey="Hebert, M" uniqKey="Hebert M">M Hebert</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Zijdenbos, Ap" uniqKey="Zijdenbos A">AP Zijdenbos</name></author><author><name sortKey="Dawant, Bm" uniqKey="Dawant B">BM Dawant</name></author><author><name sortKey="Margolin, Ra" uniqKey="Margolin R">RA Margolin</name></author><author><name sortKey="Palmer, Ac" uniqKey="Palmer A">AC Palmer</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Zhang, Yj" uniqKey="Zhang Y">YJ Zhang</name></author></analytic></biblStruct><biblStruct></biblStruct><biblStruct><analytic><author><name sortKey="Klar, N" uniqKey="Klar N">N Klar</name></author><author><name sortKey="Lipsitz, Sr" uniqKey="Lipsitz S">SR Lipsitz</name></author><author><name sortKey="Parzen, M" uniqKey="Parzen M">M Parzen</name></author><author><name sortKey="Leong, T" uniqKey="Leong T">T Leong</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Dimopoulos, S" uniqKey="Dimopoulos S">S Dimopoulos</name></author><author><name sortKey="Mayer, Ce" uniqKey="Mayer C">CE Mayer</name></author><author><name sortKey="Rudolf, F" uniqKey="Rudolf F">F Rudolf</name></author><author><name sortKey="Stelling, J" uniqKey="Stelling J">J Stelling</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Zhang, H" uniqKey="Zhang H">H Zhang</name></author><author><name sortKey="Fritts, Je" uniqKey="Fritts J">JE Fritts</name></author><author><name sortKey="Goldman, Sa" uniqKey="Goldman S">SA Goldman</name></author></analytic></biblStruct><biblStruct></biblStruct><biblStruct></biblStruct><biblStruct><analytic><author><name sortKey="Efron, B" uniqKey="Efron B">B Efron</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Efron, B" uniqKey="Efron B">B Efron</name></author><author><name sortKey="Tibshirani, Rj" uniqKey="Tibshirani R">RJ Tibshirani</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Wu, Jc" uniqKey="Wu J">JC Wu</name></author><author><name sortKey="Martin, Af" uniqKey="Martin A">AF Martin</name></author><author><name sortKey="Kacker, Rn" uniqKey="Kacker R">RN Kacker</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Wu, Jc" uniqKey="Wu J">JC Wu</name></author><author><name sortKey="Martin, Af" uniqKey="Martin A">AF Martin</name></author><author><name sortKey="Kacker, Rn" uniqKey="Kacker R">RN Kacker</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Wu, Jc" uniqKey="Wu J">JC Wu</name></author><author><name sortKey="Martin, Af" uniqKey="Martin A">AF Martin</name></author><author><name sortKey="Kacker, Rn" uniqKey="Kacker R">RN Kacker</name></author></analytic></biblStruct><biblStruct></biblStruct><biblStruct><analytic><author><name sortKey="Dresen, Img" uniqKey="Dresen I">IMG Dresen</name></author><author><name sortKey="Boes, T" uniqKey="Boes T">T Boes</name></author><author><name sortKey="Huesing, J" uniqKey="Huesing J">J Huesing</name></author><author><name sortKey="Neuhaeuser, M" uniqKey="Neuhaeuser M">M Neuhaeuser</name></author><author><name sortKey="Joeckel, Kh" uniqKey="Joeckel K">KH Joeckel</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Hand, Dj" uniqKey="Hand D">DJ Hand</name></author></analytic></biblStruct><biblStruct></biblStruct><biblStruct><analytic><author><name sortKey="Van Der Waerden, Bl" uniqKey="Van Der Waerden B">BL van der Waerden</name></author></analytic></biblStruct><biblStruct></biblStruct><biblStruct><analytic><author><name sortKey="Rice, Ja" uniqKey="Rice J">JA Rice</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Hyndman, Rj" uniqKey="Hyndman R">RJ Hyndman</name></author><author><name sortKey="Fan, Y" uniqKey="Fan Y">Y Fan</name></author></analytic></biblStruct></listBibl></div1></back></TEI><pmc article-type="research-article"><pmc-dir>properties open_access</pmc-dir>
<front><journal-meta><journal-id journal-id-type="nlm-ta">BMC Bioinformatics</journal-id><journal-id journal-id-type="iso-abbrev">BMC Bioinformatics</journal-id><journal-title-group><journal-title>BMC Bioinformatics</journal-title></journal-title-group><issn pub-type="epub">1471-2105</issn><publisher><publisher-name>BioMed Central</publisher-name><publisher-loc>London</publisher-loc></publisher></journal-meta><article-meta><article-id pub-id-type="pmid">28292256</article-id><article-id pub-id-type="pmc">5351215</article-id><article-id pub-id-type="publisher-id">1527</article-id><article-id pub-id-type="doi">10.1186/s12859-017-1527-x</article-id><article-categories><subj-group subj-group-type="heading"><subject>Research Article</subject></subj-group></article-categories><title-group><article-title>A novel measure and significance testing in data analysis of cell image segmentation</article-title></title-group><contrib-group><contrib contrib-type="author" corresp="yes"><contrib-id contrib-id-type="orcid">http://orcid.org/0000-0002-6340-2467</contrib-id><name><surname>Wu</surname><given-names>Jin Chu</given-names></name><address><email>jinchu.wu@nist.gov</email></address><xref ref-type="aff" rid="Aff1"></xref></contrib><contrib contrib-type="author"><name><surname>Halter</surname><given-names>Michael</given-names></name><address><email>michael.halter@nist.gov</email></address><xref ref-type="aff" rid="Aff1"></xref></contrib><contrib contrib-type="author"><name><surname>Kacker</surname><given-names>Raghu N.</given-names></name><address><email>raghu.kacker@nist.gov</email></address><xref ref-type="aff" rid="Aff1"></xref></contrib><contrib contrib-type="author"><name><surname>Elliott</surname><given-names>John T.</given-names></name><address><email>john.elliott@nist.gov</email></address><xref ref-type="aff" rid="Aff1"></xref></contrib><contrib contrib-type="author"><name><surname>Plant</surname><given-names>Anne L.</given-names></name><address><email>anne.plant@nist.gov</email></address><xref ref-type="aff" rid="Aff1"></xref></contrib><aff id="Aff1"><institution-wrap><institution-id institution-id-type="ISNI">000000012158463X</institution-id><institution-id institution-id-type="GRID">grid.94225.38</institution-id><institution></institution><institution>National Institute of Standards and Technology,</institution></institution-wrap>Gaithersburg, MD 20899 USA</aff></contrib-group><pub-date pub-type="epub"><day>14</day><month>3</month><year>2017</year></pub-date><pub-date pub-type="pmc-release"><day>14</day><month>3</month><year>2017</year></pub-date><pub-date pub-type="collection"><year>2017</year></pub-date><volume>18</volume><elocation-id>168</elocation-id><history><date date-type="received"><day>11</day><month>8</month><year>2016</year></date><date date-type="accepted"><day>6</day><month>2</month><year>2017</year></date></history><permissions><copyright-statement>© The Author(s). 2017</copyright-statement><license license-type="OpenAccess"><license-p><bold>Open Access</bold>This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (<ext-link ext-link-type="uri" xlink:href="http://creativecommons.org/licenses/by/4.0/">http://creativecommons.org/licenses/by/4.0/</ext-link>), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (<ext-link ext-link-type="uri" xlink:href="http://creativecommons.org/publicdomain/zero/1.0/">http://creativecommons.org/publicdomain/zero/1.0/</ext-link>) applies to the data made available in this article, unless otherwise stated.</license-p></license></permissions><abstract id="Abs1"><sec><title>Background</title><p>Cell image segmentation (CIS) is an essential part of quantitative imaging of biological cells. Designing a performance measure and conducting significance testing are critical for evaluating and comparing the CIS algorithms for image-based cell assays in cytometry. Many measures and methods have been proposed and implemented to evaluate segmentation methods. However, computing the standard errors (SE) of the measures and their correlation coefficient is not described, and thus the statistical significance of performance differences between CIS algorithms cannot be assessed.</p></sec><sec><title>Results</title><p>We propose the total error rate (TER), a novel performance measure for segmenting all cells in the supervised evaluation. The TER statistically aggregates all misclassification error rates (MER) by taking cell sizes as weights. The MERs are for segmenting each single cell in the population. The TER is fully supported by the pairwise comparisons of MERs using 106 manually segmented ground-truth cells with different sizes and seven CIS algorithms taken from ImageJ. Further, the SE and 95% confidence interval (CI) of TER are computed based on the SE of MER that is calculated using the bootstrap method. An algorithm for computing the correlation coefficient of TERs between two CIS algorithms is also provided. Hence, the 95% CI error bars can be used to classify CIS algorithms. The SEs of TERs and their correlation coefficient can be employed to conduct the hypothesis testing, while the CIs overlap, to determine the statistical significance of the performance differences between CIS algorithms.</p></sec><sec><title>Conclusions</title><p>A novel measure TER of CIS is proposed. The TER’s SEs and correlation coefficient are computed. Thereafter, CIS algorithms can be evaluated and compared statistically by conducting the significance testing.</p></sec></abstract><kwd-group xml:lang="en"><title>Keywords</title><kwd>Cell image segmentation</kwd><kwd>Cell assays</kwd><kwd>Performance measure</kwd><kwd>Misclassification error rate</kwd><kwd>Total error rate</kwd><kwd>Standard error</kwd><kwd>Confidence interval</kwd><kwd>Correlation coefficient</kwd><kwd>Significance testing</kwd><kwd>Bootstrap method</kwd></kwd-group><custom-meta-group><custom-meta><meta-name>issue-copyright-statement</meta-name><meta-value>© The Author(s) 2017</meta-value></custom-meta></custom-meta-group></article-meta></front></pmc><affiliations><list></list><tree><noCountry><name sortKey="Elliott, John T" sort="Elliott, John T" uniqKey="Elliott J" first="John T." last="Elliott">John T. Elliott</name><name sortKey="Halter, Michael" sort="Halter, Michael" uniqKey="Halter M" first="Michael" last="Halter">Michael Halter</name><name sortKey="Kacker, Raghu N" sort="Kacker, Raghu N" uniqKey="Kacker R" first="Raghu N." last="Kacker">Raghu N. Kacker</name><name sortKey="Plant, Anne L" sort="Plant, Anne L" uniqKey="Plant A" first="Anne L." last="Plant">Anne L. Plant</name><name sortKey="Wu, Jin Chu" sort="Wu, Jin Chu" uniqKey="Wu J" first="Jin Chu" last="Wu">Jin Chu Wu</name></noCountry></tree></affiliations></record>Pour manipuler ce document sous Unix (Dilib)
EXPLOR_STEP=$WICRI_ROOT/Sante/explor/MersV1/Data/Pmc/Checkpoint
HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 000944 | SxmlIndent | more
Ou
HfdSelect -h $EXPLOR_AREA/Data/Pmc/Checkpoint/biblio.hfd -nk 000944 | SxmlIndent | more
Pour mettre un lien sur cette page dans le réseau Wicri
{{Explor lien
|wiki= Sante
|area= MersV1
|flux= Pmc
|étape= Checkpoint
|type= RBID
|clé= PMC:5351215
|texte= A novel measure and significance testing in data analysis of cell image segmentation
}}
Pour générer des pages wiki
HfdIndexSelect -h $EXPLOR_AREA/Data/Pmc/Checkpoint/RBID.i -Sk "pubmed:28292256" \
| HfdSelect -Kh $EXPLOR_AREA/Data/Pmc/Checkpoint/biblio.hfd \
| NlmPubMed2Wicri -a MersV1
| This area was generated with Dilib version V0.6.33. |  |