Issues in Ground-Truthing Graphic Documents
Identifieur interne : 000126 ( Istex/Corpus ); précédent : 000125; suivant : 000127Issues in Ground-Truthing Graphic Documents
Auteurs : Daniel Lopresti ; George NagySource :
- Lecture Notes in Computer Science [ 0302-9743 ] ; 2002.
Abstract
Abstract: We examine the nature of ground-truth: whether it is always well-defined for a given task, or only relative and approximate. In the conventional scenario, reference data is produced by recording the interpretation of each test document using a chosen data-entry platform. Looking a little more closely at this process, we study its constituents and their interrelations. We provide examples from the literature and from our own experiments where non-trivial problems with each of the components appear to preclude the possibility of real progress in evaluating automated graphics recognition systems, and propose possible solutions. More specifically, for documents with complex structure we recommend multi-valued, layered, weighted, functional ground-truth supported by model-guided reference data-entry systems and protocols. Mostly, however, we raise far more questions than we currently have answers for.
Url:
DOI: 10.1007/3-540-45868-9_5
Links to Exploration step
ISTEX:3F3853C91990BECD53027DC72DA1372D1BD9B4BFLe document en format XML
<record><TEI wicri:istexFullTextTei="biblStruct"><teiHeader><fileDesc><titleStmt><title xml:lang="en">Issues in Ground-Truthing Graphic Documents</title><author><name sortKey="Lopresti, Daniel" sort="Lopresti, Daniel" uniqKey="Lopresti D" first="Daniel" last="Lopresti">Daniel Lopresti</name><affiliation><mods:affiliation>Lucent Technologies Inc., Bell Labs, 600 Mountain Ave. Room 2D-447, 07974, Murray Hill, NJ, USA</mods:affiliation></affiliation></author><author><name sortKey="Nagy, George" sort="Nagy, George" uniqKey="Nagy G" first="George" last="Nagy">George Nagy</name><affiliation><mods:affiliation>Rensselaer Polytechnic Institute Troy, Department of Electrical, Computer, and Systems Engineering, 12180, NY, USA</mods:affiliation></affiliation></author></titleStmt><publicationStmt><idno type="wicri:source">ISTEX</idno><idno type="RBID">ISTEX:3F3853C91990BECD53027DC72DA1372D1BD9B4BF</idno><date when="2002" year="2002">2002</date><idno type="doi">10.1007/3-540-45868-9_5</idno><idno type="url">https://api.istex.fr/document/3F3853C91990BECD53027DC72DA1372D1BD9B4BF/fulltext/pdf</idno><idno type="wicri:Area/Istex/Corpus">000126</idno></publicationStmt><sourceDesc><biblStruct><analytic><title level="a" type="main" xml:lang="en">Issues in Ground-Truthing Graphic Documents</title><author><name sortKey="Lopresti, Daniel" sort="Lopresti, Daniel" uniqKey="Lopresti D" first="Daniel" last="Lopresti">Daniel Lopresti</name><affiliation><mods:affiliation>Lucent Technologies Inc., Bell Labs, 600 Mountain Ave. Room 2D-447, 07974, Murray Hill, NJ, USA</mods:affiliation></affiliation></author><author><name sortKey="Nagy, George" sort="Nagy, George" uniqKey="Nagy G" first="George" last="Nagy">George Nagy</name><affiliation><mods:affiliation>Rensselaer Polytechnic Institute Troy, Department of Electrical, Computer, and Systems Engineering, 12180, NY, USA</mods:affiliation></affiliation></author></analytic><monogr></monogr><series><title level="s">Lecture Notes in Computer Science</title><imprint><date>2002</date></imprint><idno type="ISSN">0302-9743</idno><idno type="ISSN">0302-9743</idno></series><idno type="istex">3F3853C91990BECD53027DC72DA1372D1BD9B4BF</idno><idno type="DOI">10.1007/3-540-45868-9_5</idno><idno type="ChapterID">5</idno><idno type="ChapterID">Chap5</idno></biblStruct></sourceDesc><seriesStmt><idno type="ISSN">0302-9743</idno></seriesStmt></fileDesc><profileDesc><textClass></textClass><langUsage><language ident="en">en</language></langUsage></profileDesc></teiHeader><front><div type="abstract" xml:lang="en">Abstract: We examine the nature of ground-truth: whether it is always well-defined for a given task, or only relative and approximate. In the conventional scenario, reference data is produced by recording the interpretation of each test document using a chosen data-entry platform. Looking a little more closely at this process, we study its constituents and their interrelations. We provide examples from the literature and from our own experiments where non-trivial problems with each of the components appear to preclude the possibility of real progress in evaluating automated graphics recognition systems, and propose possible solutions. More specifically, for documents with complex structure we recommend multi-valued, layered, weighted, functional ground-truth supported by model-guided reference data-entry systems and protocols. Mostly, however, we raise far more questions than we currently have answers for.</div></front></TEI><istex><corpusName>springer</corpusName><author><json:item><name>Daniel Lopresti</name><affiliations><json:string>Lucent Technologies Inc., Bell Labs, 600 Mountain Ave. Room 2D-447, 07974, Murray Hill, NJ, USA</json:string></affiliations></json:item><json:item><name>George Nagy</name><affiliations><json:string>Rensselaer Polytechnic Institute Troy, Department of Electrical, Computer, and Systems Engineering, 12180, NY, USA</json:string></affiliations></json:item></author><language><json:string>eng</json:string></language><abstract>Abstract: We examine the nature of ground-truth: whether it is always well-defined for a given task, or only relative and approximate. In the conventional scenario, reference data is produced by recording the interpretation of each test document using a chosen data-entry platform. Looking a little more closely at this process, we study its constituents and their interrelations. We provide examples from the literature and from our own experiments where non-trivial problems with each of the components appear to preclude the possibility of real progress in evaluating automated graphics recognition systems, and propose possible solutions. More specifically, for documents with complex structure we recommend multi-valued, layered, weighted, functional ground-truth supported by model-guided reference data-entry systems and protocols. Mostly, however, we raise far more questions than we currently have answers for.</abstract><qualityIndicators><score>6.548</score><pdfVersion>1.3</pdfVersion><pdfPageSize>451 x 677.12 pts</pdfPageSize><refBibsNative>false</refBibsNative><keywordCount>0</keywordCount><abstractCharCount>919</abstractCharCount><pdfWordCount>7685</pdfWordCount><pdfCharCount>46871</pdfCharCount><pdfPageCount>22</pdfPageCount><abstractWordCount>129</abstractWordCount></qualityIndicators><title>Issues in Ground-Truthing Graphic Documents</title><genre.original><json:string>OriginalPaper</json:string></genre.original><chapterId><json:string>5</json:string><json:string>Chap5</json:string></chapterId><genre><json:string>conference [eBooks]</json:string></genre><serie><editor><json:item><name>Gerhard Goos</name><affiliations><json:string>Karlsruhe University, Germany</json:string></affiliations></json:item><json:item><name>Juris Hartmanis</name><affiliations><json:string>Cornell University, NY, USA</json:string></affiliations></json:item><json:item><name>Jan van Leeuwen</name><affiliations><json:string>Utrecht University, The Netherlands</json:string></affiliations></json:item></editor><issn><json:string>0302-9743</json:string></issn><language><json:string>unknown</json:string></language><title>Lecture Notes in Computer Science</title><copyrightDate>2002</copyrightDate></serie><host><editor><json:item><name>Dorothea Blostein</name><affiliations><json:string>Computing and Information Science, Queen’s University, K7L 3N6, Kingston, Ontario, Canada</json:string><json:string>E-mail: blostein@cs.queensu.ca</json:string></affiliations></json:item><json:item><name>Young-Bin Kwon</name><affiliations><json:string>Department of Computer Engineering, Chung-Ang University, 156-756, Seoul, South Korea</json:string><json:string>E-mail: ybkwon@visionnet.cse.cau.ac.kr</json:string></affiliations></json:item></editor><subject><json:item><value>Computer Science</value></json:item><json:item><value>Computer Science</value></json:item><json:item><value>Pattern Recognition</value></json:item><json:item><value>Image Processing and Computer Vision</value></json:item><json:item><value>Computer Graphics</value></json:item><json:item><value>Artificial Intelligence (incl. Robotics)</value></json:item><json:item><value>Discrete Mathematics in Computer Science</value></json:item><json:item><value>Algorithm Analysis and Problem Complexity</value></json:item></subject><isbn><json:string>978-3-540-44066-6</json:string></isbn><language><json:string>unknown</json:string></language><title>Graphics Recognition Algorithms and Applications</title><genre.original><json:string>Proceedings</json:string></genre.original><bookId><json:string>3-540-45868-9</json:string></bookId><volume>2390</volume><pages><last>67</last><first>46</first></pages><issn><json:string>0302-9743</json:string></issn><genre><json:string>Book Series</json:string></genre><eisbn><json:string>978-3-540-45868-5</json:string></eisbn><copyrightDate>2002</copyrightDate><doi><json:string>10.1007/3-540-45868-9</json:string></doi></host><publicationDate>2002</publicationDate><copyrightDate>2002</copyrightDate><doi><json:string>10.1007/3-540-45868-9_5</json:string></doi><id>3F3853C91990BECD53027DC72DA1372D1BD9B4BF</id><fulltext><json:item><original>true</original><mimetype>application/pdf</mimetype><extension>pdf</extension><uri>https://api.istex.fr/document/3F3853C91990BECD53027DC72DA1372D1BD9B4BF/fulltext/pdf</uri></json:item><json:item><original>false</original><mimetype>application/zip</mimetype><extension>zip</extension><uri>https://api.istex.fr/document/3F3853C91990BECD53027DC72DA1372D1BD9B4BF/fulltext/zip</uri></json:item><istex:fulltextTEI uri="https://api.istex.fr/document/3F3853C91990BECD53027DC72DA1372D1BD9B4BF/fulltext/tei"><teiHeader><fileDesc><titleStmt><title level="a" type="main" xml:lang="en">Issues in Ground-Truthing Graphic Documents</title><respStmt xml:id="ISTEX-API" resp="Références bibliographiques récupérées via GROBID" name="ISTEX-API (INIST-CNRS)"></respStmt></titleStmt><publicationStmt><authority>ISTEX</authority><publisher>Springer Berlin Heidelberg</publisher><pubPlace>Berlin, Heidelberg</pubPlace><availability><p>SPRINGER</p></availability><date>2002</date></publicationStmt><sourceDesc><biblStruct type="inbook"><analytic><title level="a" type="main" xml:lang="en">Issues in Ground-Truthing Graphic Documents</title><author><persName><forename type="first">Daniel</forename><surname>Lopresti</surname></persName><affiliation>Lucent Technologies Inc., Bell Labs, 600 Mountain Ave. Room 2D-447, 07974, Murray Hill, NJ, USA</affiliation></author><author><persName><forename type="first">George</forename><surname>Nagy</surname></persName><affiliation>Rensselaer Polytechnic Institute Troy, Department of Electrical, Computer, and Systems Engineering, 12180, NY, USA</affiliation></author></analytic><monogr><title level="m">Graphics Recognition Algorithms and Applications</title><title level="m" type="sub">4th International Workshop, GREC 2001 Kingston, Ontario, Canada, September 7–8, 2001 Selected Papers</title><idno type="pISBN">978-3-540-44066-6</idno><idno type="eISBN">978-3-540-45868-5</idno><idno type="pISSN">0302-9743</idno><idno type="DOI">10.1007/3-540-45868-9</idno><idno type="BookID">3-540-45868-9</idno><idno type="BookTitleID">72648</idno><idno type="BookSequenceNumber">2390</idno><idno type="BookVolumeNumber">2390</idno><idno type="BookChapterCount">33</idno><editor><persName><forename type="first">Dorothea</forename><surname>Blostein</surname></persName><email>blostein@cs.queensu.ca</email><affiliation>Computing and Information Science, Queen’s University, K7L 3N6, Kingston, Ontario, Canada</affiliation></editor><editor><persName><forename type="first">Young-Bin</forename><surname>Kwon</surname></persName><email>ybkwon@visionnet.cse.cau.ac.kr</email><affiliation>Department of Computer Engineering, Chung-Ang University, 156-756, Seoul, South Korea</affiliation></editor><imprint><publisher>Springer Berlin Heidelberg</publisher><pubPlace>Berlin, Heidelberg</pubPlace><date type="published" when="2002"></date><biblScope unit="volume">2390</biblScope><biblScope unit="page" from="46">46</biblScope><biblScope unit="page" to="67">67</biblScope></imprint></monogr><series><title level="s">Lecture Notes in Computer Science</title><editor><persName><forename type="first">Gerhard</forename><surname>Goos</surname></persName><affiliation>Karlsruhe University, Germany</affiliation></editor><editor><persName><forename type="first">Juris</forename><surname>Hartmanis</surname></persName><affiliation>Cornell University, NY, USA</affiliation></editor><editor><persName><forename type="first">Jan</forename><surname>van Leeuwen</surname></persName><affiliation>Utrecht University, The Netherlands</affiliation></editor><biblScope><date>2002</date></biblScope><idno type="pISSN">0302-9743</idno><idno type="seriesId">558</idno></series><idno type="istex">3F3853C91990BECD53027DC72DA1372D1BD9B4BF</idno><idno type="DOI">10.1007/3-540-45868-9_5</idno><idno type="ChapterID">5</idno><idno type="ChapterID">Chap5</idno></biblStruct></sourceDesc></fileDesc><profileDesc><creation><date>2002</date></creation><langUsage><language ident="en">en</language></langUsage><abstract xml:lang="en"><p>Abstract: We examine the nature of ground-truth: whether it is always well-defined for a given task, or only relative and approximate. In the conventional scenario, reference data is produced by recording the interpretation of each test document using a chosen data-entry platform. Looking a little more closely at this process, we study its constituents and their interrelations. We provide examples from the literature and from our own experiments where non-trivial problems with each of the components appear to preclude the possibility of real progress in evaluating automated graphics recognition systems, and propose possible solutions. More specifically, for documents with complex structure we recommend multi-valued, layered, weighted, functional ground-truth supported by model-guided reference data-entry systems and protocols. Mostly, however, we raise far more questions than we currently have answers for.</p></abstract><textClass><keywords scheme="Book Subject Collection"><list><label>SUCO11645</label><item><term>Computer Science</term></item></list></keywords></textClass><textClass><keywords scheme="Book Subject Group"><list><label>I</label><label>I2203X</label><label>I22021</label><label>I22013</label><label>I21017</label><label>I17028</label><label>I16021</label><item><term>Computer Science</term></item><item><term>Pattern Recognition</term></item><item><term>Image Processing and Computer Vision</term></item><item><term>Computer Graphics</term></item><item><term>Artificial Intelligence (incl. Robotics)</term></item><item><term>Discrete Mathematics in Computer Science</term></item><item><term>Algorithm Analysis and Problem Complexity</term></item></list></keywords></textClass></profileDesc><revisionDesc><change when="2002">Published</change><change xml:id="refBibs-istex" who="#ISTEX-API" when="2016-3-19">References added</change></revisionDesc></teiHeader></istex:fulltextTEI><json:item><original>false</original><mimetype>text/plain</mimetype><extension>txt</extension><uri>https://api.istex.fr/document/3F3853C91990BECD53027DC72DA1372D1BD9B4BF/fulltext/txt</uri></json:item></fulltext><metadata><istex:metadataXml wicri:clean="Springer, Publisher found" wicri:toSee="no header"><istex:xmlDeclaration>version="1.0" encoding="UTF-8"</istex:xmlDeclaration><istex:docType PUBLIC="-//Springer-Verlag//DTD A++ V2.4//EN" URI="http://devel.springer.de/A++/V2.4/DTD/A++V2.4.dtd" name="istex:docType"></istex:docType><istex:document><Publisher><PublisherInfo><PublisherName>Springer Berlin Heidelberg</PublisherName><PublisherLocation>Berlin, Heidelberg</PublisherLocation></PublisherInfo><Series><SeriesInfo SeriesType="Series" TocLevels="0"><SeriesID>558</SeriesID><SeriesPrintISSN>0302-9743</SeriesPrintISSN><SeriesTitle Language="En">Lecture Notes in Computer Science</SeriesTitle></SeriesInfo><SeriesHeader><EditorGroup><Editor AffiliationIDS="Aff1"><EditorName DisplayOrder="Western"><GivenName>Gerhard</GivenName><FamilyName>Goos</FamilyName></EditorName></Editor><Editor AffiliationIDS="Aff2"><EditorName DisplayOrder="Western"><GivenName>Juris</GivenName><FamilyName>Hartmanis</FamilyName></EditorName></Editor><Editor AffiliationIDS="Aff3"><EditorName DisplayOrder="Western"><GivenName>Jan</GivenName><Particle>van</Particle><FamilyName>Leeuwen</FamilyName></EditorName></Editor><Affiliation ID="Aff1"><OrgName>Karlsruhe University</OrgName><OrgAddress><Country>Germany</Country></OrgAddress></Affiliation><Affiliation ID="Aff2"><OrgName>Cornell University</OrgName><OrgAddress><State>NY</State><Country>USA</Country></OrgAddress></Affiliation><Affiliation ID="Aff3"><OrgName>Utrecht University</OrgName><OrgAddress><Country>The Netherlands</Country></OrgAddress></Affiliation></EditorGroup></SeriesHeader><Book Language="En"><BookInfo BookProductType="Proceedings" Language="En" MediaType="eBook" NumberingStyle="Unnumbered" TocLevels="0"><BookID>3-540-45868-9</BookID><BookTitle>Graphics Recognition Algorithms and Applications</BookTitle><BookSubTitle>4th International Workshop, GREC 2001 Kingston, Ontario, Canada, September 7–8, 2001 Selected Papers</BookSubTitle><BookVolumeNumber>2390</BookVolumeNumber><BookSequenceNumber>2390</BookSequenceNumber><BookDOI>10.1007/3-540-45868-9</BookDOI><BookTitleID>72648</BookTitleID><BookPrintISBN>978-3-540-44066-6</BookPrintISBN><BookElectronicISBN>978-3-540-45868-5</BookElectronicISBN><BookChapterCount>33</BookChapterCount><BookHistory><OnlineDate><Year>2002</Year><Month>10</Month><Day>4</Day></OnlineDate></BookHistory><BookCopyright><CopyrightHolderName>Springer-Verlag Berlin Heidelberg</CopyrightHolderName><CopyrightYear>2002</CopyrightYear></BookCopyright><BookSubjectGroup><BookSubject Code="I" Type="Primary">Computer Science</BookSubject><BookSubject Code="I2203X" Priority="1" Type="Secondary">Pattern Recognition</BookSubject><BookSubject Code="I22021" Priority="2" Type="Secondary">Image Processing and Computer Vision</BookSubject><BookSubject Code="I22013" Priority="3" Type="Secondary">Computer Graphics</BookSubject><BookSubject Code="I21017" Priority="4" Type="Secondary">Artificial Intelligence (incl. Robotics)</BookSubject><BookSubject Code="I17028" Priority="5" Type="Secondary">Discrete Mathematics in Computer Science</BookSubject><BookSubject Code="I16021" Priority="6" Type="Secondary">Algorithm Analysis and Problem Complexity</BookSubject><SubjectCollection Code="SUCO11645">Computer Science</SubjectCollection></BookSubjectGroup><BookContext><SeriesID>558</SeriesID></BookContext></BookInfo><BookHeader><EditorGroup><Editor AffiliationIDS="Aff4"><EditorName DisplayOrder="Western"><GivenName>Dorothea</GivenName><FamilyName>Blostein</FamilyName></EditorName><Contact><Email>blostein@cs.queensu.ca</Email></Contact></Editor><Editor AffiliationIDS="Aff5"><EditorName DisplayOrder="Western"><GivenName>Young-Bin</GivenName><FamilyName>Kwon</FamilyName></EditorName><Contact><Email>ybkwon@visionnet.cse.cau.ac.kr</Email></Contact></Editor><Affiliation ID="Aff4"><OrgDivision>Computing and Information Science</OrgDivision><OrgName>Queen’s University</OrgName><OrgAddress><City>Kingston</City><State>Ontario</State><Country>Canada</Country><Postcode>K7L 3N6</Postcode></OrgAddress></Affiliation><Affiliation ID="Aff5"><OrgDivision>Department of Computer Engineering</OrgDivision><OrgName>Chung-Ang University</OrgName><OrgAddress><City>Seoul</City><Postcode>156-756</Postcode><Country>South Korea</Country></OrgAddress></Affiliation></EditorGroup></BookHeader><Part ID="Part2"><PartInfo TocLevels="0"><PartID>2</PartID><PartSequenceNumber>2</PartSequenceNumber><PartTitle>Validation, User Interfaces</PartTitle><PartChapterCount>4</PartChapterCount><PartContext><SeriesID>558</SeriesID><BookID>3-540-45868-9</BookID><BookTitle>Graphics Recognition Algorithms and Applications</BookTitle></PartContext></PartInfo><Chapter ID="Chap5" Language="En"><ChapterInfo ChapterType="OriginalPaper" ContainsESM="No" Language="En" NumberingStyle="Unnumbered" TocLevels="0"><ChapterID>5</ChapterID><ChapterDOI>10.1007/3-540-45868-9_5</ChapterDOI><ChapterSequenceNumber>5</ChapterSequenceNumber><ChapterTitle Language="En">Issues in Ground-Truthing Graphic Documents</ChapterTitle><ChapterFirstPage>46</ChapterFirstPage><ChapterLastPage>67</ChapterLastPage><ChapterCopyright><CopyrightHolderName>Springer-Verlag Berlin Heidelberg</CopyrightHolderName><CopyrightYear>2002</CopyrightYear></ChapterCopyright><ChapterHistory><RegistrationDate><Year>2002</Year><Month>10</Month><Day>3</Day></RegistrationDate><OnlineDate><Year>2002</Year><Month>10</Month><Day>4</Day></OnlineDate></ChapterHistory><ChapterGrants Type="Regular"><MetadataGrant Grant="OpenAccess"></MetadataGrant><AbstractGrant Grant="OpenAccess"></AbstractGrant><BodyPDFGrant Grant="Restricted"></BodyPDFGrant><BodyHTMLGrant Grant="Restricted"></BodyHTMLGrant><BibliographyGrant Grant="Restricted"></BibliographyGrant><ESMGrant Grant="Restricted"></ESMGrant></ChapterGrants><ChapterContext><SeriesID>558</SeriesID><PartID>2</PartID><BookID>3-540-45868-9</BookID><BookTitle>Graphics Recognition Algorithms and Applications</BookTitle></ChapterContext></ChapterInfo><ChapterHeader><AuthorGroup><Author AffiliationIDS="Aff6"><AuthorName DisplayOrder="Western"><GivenName>Daniel</GivenName><FamilyName>Lopresti</FamilyName></AuthorName></Author><Author AffiliationIDS="Aff7"><AuthorName DisplayOrder="Western"><GivenName>George</GivenName><FamilyName>Nagy</FamilyName></AuthorName></Author><Affiliation ID="Aff6"><OrgDivision>Lucent Technologies Inc.</OrgDivision><OrgName>Bell Labs</OrgName><OrgAddress><Street>600 Mountain Ave. Room 2D-447</Street><City>Murray Hill</City><State>NJ</State><Postcode>07974</Postcode><Country>USA</Country></OrgAddress></Affiliation><Affiliation ID="Aff7"><OrgDivision>Rensselaer Polytechnic Institute Troy</OrgDivision><OrgName>Department of Electrical, Computer, and Systems Engineering</OrgName><OrgAddress><State>NY</State><Postcode>12180</Postcode><Country>USA</Country></OrgAddress></Affiliation></AuthorGroup><Abstract ID="Abs1" Language="En"><Heading>Abstract</Heading><Para>We examine the nature of ground-truth: whether it is always well-defined for a given task, or only relative and approximate. In the conventional scenario, reference data is produced by recording the interpretation of each test document using a chosen data-entry platform. Looking a little more closely at this process, we study its constituents and their interrelations. We provide examples from the literature and from our own experiments where non-trivial problems with each of the components appear to preclude the possibility of real progress in evaluating automated graphics recognition systems, and propose possible solutions. More specifically, for documents with complex structure we recommend multi-valued, layered, weighted, functional ground-truth supported by model-guided reference data-entry systems and protocols. Mostly, however, we raise far more questions than we currently have answers for.</Para></Abstract></ChapterHeader><NoBody></NoBody></Chapter></Part></Book></Series></Publisher></istex:document></istex:metadataXml><mods version="3.6"><titleInfo lang="en"><title>Issues in Ground-Truthing Graphic Documents</title></titleInfo><titleInfo type="alternative" contentType="CDATA" lang="en"><title>Issues in Ground-Truthing Graphic Documents</title></titleInfo><name type="personal"><namePart type="given">Daniel</namePart><namePart type="family">Lopresti</namePart><affiliation>Lucent Technologies Inc., Bell Labs, 600 Mountain Ave. Room 2D-447, 07974, Murray Hill, NJ, USA</affiliation><role><roleTerm type="text">author</roleTerm></role></name><name type="personal"><namePart type="given">George</namePart><namePart type="family">Nagy</namePart><affiliation>Rensselaer Polytechnic Institute Troy, Department of Electrical, Computer, and Systems Engineering, 12180, NY, USA</affiliation><role><roleTerm type="text">author</roleTerm></role></name><typeOfResource>text</typeOfResource><genre type="conference [eBooks]" displayLabel="OriginalPaper"></genre><originInfo><publisher>Springer Berlin Heidelberg</publisher><place><placeTerm type="text">Berlin, Heidelberg</placeTerm></place><dateIssued encoding="w3cdtf">2002</dateIssued><copyrightDate encoding="w3cdtf">2002</copyrightDate></originInfo><language><languageTerm type="code" authority="rfc3066">en</languageTerm><languageTerm type="code" authority="iso639-2b">eng</languageTerm></language><physicalDescription><internetMediaType>text/html</internetMediaType></physicalDescription><abstract lang="en">Abstract: We examine the nature of ground-truth: whether it is always well-defined for a given task, or only relative and approximate. In the conventional scenario, reference data is produced by recording the interpretation of each test document using a chosen data-entry platform. Looking a little more closely at this process, we study its constituents and their interrelations. We provide examples from the literature and from our own experiments where non-trivial problems with each of the components appear to preclude the possibility of real progress in evaluating automated graphics recognition systems, and propose possible solutions. More specifically, for documents with complex structure we recommend multi-valued, layered, weighted, functional ground-truth supported by model-guided reference data-entry systems and protocols. Mostly, however, we raise far more questions than we currently have answers for.</abstract><relatedItem type="host"><titleInfo><title>Graphics Recognition Algorithms and Applications</title><subTitle>4th International Workshop, GREC 2001 Kingston, Ontario, Canada, September 7–8, 2001 Selected Papers</subTitle></titleInfo><name type="personal"><namePart type="given">Dorothea</namePart><namePart type="family">Blostein</namePart><affiliation>Computing and Information Science, Queen’s University, K7L 3N6, Kingston, Ontario, Canada</affiliation><affiliation>E-mail: blostein@cs.queensu.ca</affiliation><role><roleTerm type="text">editor</roleTerm></role></name><name type="personal"><namePart type="given">Young-Bin</namePart><namePart type="family">Kwon</namePart><affiliation>Department of Computer Engineering, Chung-Ang University, 156-756, Seoul, South Korea</affiliation><affiliation>E-mail: ybkwon@visionnet.cse.cau.ac.kr</affiliation><role><roleTerm type="text">editor</roleTerm></role></name><genre type="Book Series" displayLabel="Proceedings"></genre><originInfo><copyrightDate encoding="w3cdtf">2002</copyrightDate><issuance>monographic</issuance></originInfo><subject><genre>Book Subject Collection</genre><topic authority="SpringerSubjectCodes" authorityURI="SUCO11645">Computer Science</topic></subject><subject><genre>Book Subject Group</genre><topic authority="SpringerSubjectCodes" authorityURI="I">Computer Science</topic><topic authority="SpringerSubjectCodes" authorityURI="I2203X">Pattern Recognition</topic><topic authority="SpringerSubjectCodes" authorityURI="I22021">Image Processing and Computer Vision</topic><topic authority="SpringerSubjectCodes" authorityURI="I22013">Computer Graphics</topic><topic authority="SpringerSubjectCodes" authorityURI="I21017">Artificial Intelligence (incl. Robotics)</topic><topic authority="SpringerSubjectCodes" authorityURI="I17028">Discrete Mathematics in Computer Science</topic><topic authority="SpringerSubjectCodes" authorityURI="I16021">Algorithm Analysis and Problem Complexity</topic></subject><identifier type="DOI">10.1007/3-540-45868-9</identifier><identifier type="ISBN">978-3-540-44066-6</identifier><identifier type="eISBN">978-3-540-45868-5</identifier><identifier type="ISSN">0302-9743</identifier><identifier type="BookTitleID">72648</identifier><identifier type="BookID">3-540-45868-9</identifier><identifier type="BookChapterCount">33</identifier><identifier type="BookVolumeNumber">2390</identifier><identifier type="BookSequenceNumber">2390</identifier><identifier type="PartChapterCount">4</identifier><part><date>2002</date><detail type="part"><title>Validation, User Interfaces</title></detail><detail type="volume"><number>2390</number><caption>vol.</caption></detail><extent unit="pages"><start>46</start><end>67</end></extent></part><recordInfo><recordOrigin>Springer-Verlag Berlin Heidelberg, 2002</recordOrigin></recordInfo></relatedItem><relatedItem type="series"><titleInfo><title>Lecture Notes in Computer Science</title></titleInfo><name type="personal"><namePart type="given">Gerhard</namePart><namePart type="family">Goos</namePart><affiliation>Karlsruhe University, Germany</affiliation><role><roleTerm type="text">editor</roleTerm></role></name><name type="personal"><namePart type="given">Juris</namePart><namePart type="family">Hartmanis</namePart><affiliation>Cornell University, NY, USA</affiliation><role><roleTerm type="text">editor</roleTerm></role></name><name type="personal"><namePart type="given">Jan</namePart><namePart type="family">van Leeuwen</namePart><affiliation>Utrecht University, The Netherlands</affiliation><role><roleTerm type="text">editor</roleTerm></role></name><originInfo><copyrightDate encoding="w3cdtf">2002</copyrightDate><issuance>serial</issuance></originInfo><identifier type="ISSN">0302-9743</identifier><identifier type="SeriesID">558</identifier><recordInfo><recordOrigin>Springer-Verlag Berlin Heidelberg, 2002</recordOrigin></recordInfo></relatedItem><identifier type="istex">3F3853C91990BECD53027DC72DA1372D1BD9B4BF</identifier><identifier type="DOI">10.1007/3-540-45868-9_5</identifier><identifier type="ChapterID">5</identifier><identifier type="ChapterID">Chap5</identifier><accessCondition type="use and reproduction" contentType="copyright">Springer-Verlag Berlin Heidelberg, 2002</accessCondition><recordInfo><recordContentSource>SPRINGER</recordContentSource><recordOrigin>Springer-Verlag Berlin Heidelberg, 2002</recordOrigin></recordInfo></mods></metadata><enrichments><istex:refBibTEI uri="https://api.istex.fr/document/3F3853C91990BECD53027DC72DA1372D1BD9B4BF/enrichments/refBib"><teiHeader></teiHeader><text><front></front><body></body><back><listBibl><biblStruct xml:id="b0"><monogr><title level="m" type="main">Table processing and understanding</title><author><persName><forename type="first">A</forename><forename type="middle">A</forename><surname>Abu-Tarif</surname></persName></author><imprint><date type="published" when="1998"></date><publisher>Rensselaer Polytechnic Institute</publisher><biblScope unit="page">55</biblScope></imprint></monogr></biblStruct><biblStruct xml:id="b1"><analytic><title level="a" type="main">Document image defect models</title><author><persName><forename type="first">H</forename><forename type="middle">S</forename><surname>Baird</surname></persName></author></analytic><monogr><title level="m">Structured Document Image Analysis</title><editor>H. S. Baird, H. Bunke, and K. Yamamoto</editor><imprint><biblScope unit="page" from="546" to="556"></biblScope></imprint></monogr></biblStruct><biblStruct xml:id="b2"><analytic><title level="a" type="main">Using diagram generation software to improve diagram recognition: A case study of music notation</title><author><persName><forename type="first">D</forename><surname>Blostein</surname></persName></author><author><persName><forename type="first">L</forename><surname>Haken</surname></persName></author></analytic><monogr><title level="j">IEEE Transactions on Pattern Analysis and Machine Intelligence</title><imprint><biblScope unit="volume">21</biblScope><biblScope unit="issue">50</biblScope><biblScope unit="page" from="1121" to="1136"></biblScope><date type="published" when="1999-11"></date></imprint></monogr></biblStruct><biblStruct xml:id="b3"><monogr><title level="m" type="main">The Art of Quartet Playing</title><author><persName><forename type="first">D</forename><surname>Blum</surname></persName></author><imprint><date type="published" when="1986"></date><publisher>Cornell University Press</publisher><biblScope unit="page">57</biblScope><pubPlace>Ithaca, NY</pubPlace></imprint></monogr></biblStruct><biblStruct xml:id="b4"><analytic><title level="a" type="main">An interpretation system for land register maps</title><author><persName><forename type="first">L</forename><surname>Boatto</surname></persName></author><author><persName><forename type="first">V</forename><surname>Consorti</surname></persName></author><author><persName><forename type="first">M</forename><forename type="middle">D</forename><surname>Bueno</surname></persName></author><author><persName><forename type="first">S</forename><forename type="middle">D</forename><surname>Zenzo</surname></persName></author><author><persName><forename type="first">V</forename><surname>Eramo</surname></persName></author><author><persName><forename type="first">A</forename><surname>Esposito</surname></persName></author><author><persName><forename type="first">F</forename><surname>Melcarne</surname></persName></author><author><persName><forename type="first">M</forename><surname>Meucci</surname></persName></author><author><persName><forename type="first">A</forename><surname>Morelli</surname></persName></author><author><persName><forename type="first">M</forename><surname>Mosciatti</surname></persName></author><author><persName><forename type="first">S</forename><surname>Scarci</surname></persName></author><author><persName><forename type="first">M</forename><surname>Tucci</surname></persName></author></analytic><monogr><title level="j">IEEE Computer</title><imprint><biblScope unit="volume">25</biblScope><biblScope unit="issue">7</biblScope><biblScope unit="page" from="25" to="33"></biblScope><date type="published" when="1992-07"></date></imprint></monogr></biblStruct><biblStruct xml:id="b5"><analytic><title level="a" type="main">The Second International Graphics Recognition Contest – raster to vector conversion: A report</title><author><persName><forename type="first">A</forename><forename type="middle">K</forename><surname>Chhabra</surname></persName></author><author><persName><forename type="first">I</forename><surname>Phillips</surname></persName></author></analytic><monogr><title level="m">Graphics Recognition: Algorithms and Systems</title><editor>K. Tombre and A. K. Chhabra</editor><meeting><address><addrLine>Berlin, Germany</addrLine></address></meeting><imprint><publisher>Springer-Verlag</publisher><date type="published" when="1998"></date><biblScope unit="page" from="390" to="410"></biblScope></imprint></monogr></biblStruct><biblStruct xml:id="b6"><analytic><title level="a" type="main">Issues in automatic OCR error classification</title><author><persName><forename type="first">J</forename><surname>Esakov</surname></persName></author><author><persName><forename type="first">D</forename><forename type="middle">P</forename><surname>Lopresti</surname></persName></author><author><persName><forename type="first">J</forename><forename type="middle">S</forename><surname>Sandberg</surname></persName></author><author><persName><forename type="first">J</forename><surname>Zhou</surname></persName></author></analytic><monogr><title level="m">Proceedings of the Third Annual Symposium on Document Analysis and Information Retrieval</title><meeting>the Third Annual Symposium on Document Analysis and Information Retrieval<address><addrLine>Las Vegas, NV</addrLine></address></meeting><imprint><date type="published" when="1994-04"></date><biblScope unit="page" from="401" to="412"></biblScope></imprint></monogr></biblStruct><biblStruct xml:id="b7"><analytic><title level="a" type="main">DAFS: A standard for document and image understanding</title><author><persName><forename type="first">T</forename><surname>Fruchterman</surname></persName></author></analytic><monogr><title level="m">Proceedings of the Symposium on Document Image Understanding Technology</title><meeting>the Symposium on Document Image Understanding Technology<address><addrLine>Bowie, MD</addrLine></address></meeting><imprint><date type="published" when="1995-10"></date><biblScope unit="page" from="94" to="100"></biblScope></imprint></monogr></biblStruct><biblStruct xml:id="b8"><analytic><title level="a" type="main">Document image recognition and retrieval: Where are we?</title><author><persName><forename type="first">M</forename><forename type="middle">D</forename><surname>Garris</surname></persName></author></analytic><monogr><title level="m">Proceedings of Document Recognition and Retrieval VI</title><meeting>Document Recognition and Retrieval VI<address><addrLine>San Jose, CA</addrLine></address></meeting><imprint><date type="published" when="1999-01"></date><biblScope unit="page" from="141" to="150"></biblScope></imprint></monogr></biblStruct><biblStruct xml:id="b9"><analytic><title level="a" type="main">Federal Register document image database</title><author><persName><forename type="first">M</forename><forename type="middle">D</forename><surname>Garris</surname></persName></author><author><persName><forename type="first">S</forename><forename type="middle">A</forename><surname>Janet</surname></persName></author><author><persName><forename type="first">W</forename><forename type="middle">W</forename><surname>Klein</surname></persName></author></analytic><monogr><title level="m">Proceedings of Document Recognition and Retrieval VI (IS&T/SPIE Electronic Imaging)</title><meeting>Document Recognition and Retrieval VI (IS&T/SPIE Electronic Imaging)<address><addrLine>San Jose, CA</addrLine></address></meeting><imprint><date type="published" when="1999-01"></date><biblScope unit="page" from="97" to="108"></biblScope></imprint></monogr></biblStruct><biblStruct xml:id="b10"><analytic><title level="a" type="main">Creating and validating a large image database for METTREC</title><author><persName><forename type="first">M</forename><forename type="middle">D</forename><surname>Garris</surname></persName></author><author><persName><forename type="first">W</forename><forename type="middle">W</forename><surname>Klein</surname></persName></author></analytic><monogr><title level="j">National Institute of Standards and Technology</title><imprint><biblScope unit="volume">51</biblScope><biblScope unit="page">57</biblScope><date type="published" when="1998-01"></date></imprint></monogr></biblStruct><biblStruct xml:id="b11"><analytic><title level="a" type="main">Estimating errors in document databases</title><author><persName><forename type="first">J</forename><surname>Ha</surname></persName></author><author><persName><forename type="first">R</forename><forename type="middle">M</forename><surname>Haralick</surname></persName></author><author><persName><forename type="first">S</forename><surname>Chen</surname></persName></author><author><persName><forename type="first">I</forename><forename type="middle">T</forename><surname>Phillips</surname></persName></author></analytic><monogr><title level="m">Proceedings of the Third Annual Symposium on Document Analysis and Information Retrieval</title><meeting>the Third Annual Symposium on Document Analysis and Information Retrieval<address><addrLine>Las Vegas, NV</addrLine></address></meeting><imprint><date type="published" when="1994-04"></date><biblScope unit="page" from="435" to="459"></biblScope></imprint></monogr></biblStruct><biblStruct xml:id="b12"><analytic><title level="a" type="main">Matching document images with ground truth</title><author><persName><forename type="first">J</forename><forename type="middle">D</forename><surname>Hobby</surname></persName></author></analytic><monogr><title level="j">International Journal on Document Analysis and Recognition</title><imprint><biblScope unit="volume">1</biblScope><biblScope unit="issue">1</biblScope><biblScope unit="page" from="52" to="61"></biblScope><date type="published" when="1998"></date></imprint></monogr></biblStruct><biblStruct xml:id="b13"><analytic><title level="a" type="main">Why table ground-truthing is hard</title><author><persName><forename type="first">J</forename><surname>Hu</surname></persName></author><author><persName><forename type="first">R</forename><surname>Kashi</surname></persName></author><author><persName><forename type="first">D</forename><surname>Lopresti</surname></persName></author><author><persName><forename type="first">G</forename><surname>Nagy</surname></persName></author><author><persName><forename type="first">G</forename><surname>Wilfong</surname></persName></author></analytic><monogr><title level="m">Proceedings of the Sixth International Conference on Document Analysis and Recognition</title><meeting>the Sixth International Conference on Document Analysis and Recognition<address><addrLine>Seattle, WA</addrLine></address></meeting><imprint><date type="published" when="2001-09"></date><biblScope unit="page" from="129" to="133"></biblScope></imprint></monogr></biblStruct><biblStruct xml:id="b14"><analytic><title level="a" type="main">An automatic closed-loop methodology for generating character groundtruth for scanned documents</title><author><persName><forename type="first">T</forename><surname>Kanungo</surname></persName></author><author><persName><forename type="first">R</forename><forename type="middle">M</forename><surname>Haralick</surname></persName></author></analytic><monogr><title level="j">IEEE Transactions on Pattern Analysis and Machine Intelligence</title><imprint><biblScope unit="volume">21</biblScope><biblScope unit="issue">2</biblScope><biblScope unit="page" from="179" to="183"></biblScope><date type="published" when="1999-02"></date></imprint></monogr></biblStruct><biblStruct xml:id="b15"><analytic><title level="a" type="main">TRUEVIZ: a groundtruth / metadata editing and visualizing toolkit for OCR</title><author><persName><forename type="first">T</forename><surname>Kanungo</surname></persName></author><author><persName><forename type="first">C</forename><forename type="middle">H</forename><surname>Lee</surname></persName></author><author><persName><forename type="first">J</forename><surname>Czorapinski</surname></persName></author><author><persName><forename type="first">I</forename><surname>Bella</surname></persName></author></analytic><monogr><title level="m">Proceedings of Document Recognition and Retrieval VIII (IS&T/SPIE Electronic Imaging)</title><meeting>Document Recognition and Retrieval VIII (IS&T/SPIE Electronic Imaging)<address><addrLine>San Jose, CA</addrLine></address></meeting><imprint><date type="published" when="2001-01"></date><biblScope unit="page" from="1" to="12"></biblScope></imprint></monogr></biblStruct><biblStruct xml:id="b16"><analytic><title level="a" type="main">A point matching algorithm for automatic generation of groundtruth for document images</title><author><persName><forename type="first">D.-W</forename><surname>Kim</surname></persName></author><author><persName><forename type="first">T</forename><surname>Kanungo</surname></persName></author></analytic><monogr><title level="m">Proceedings of the Fourth IAPR International Workshop on Document Analysis Systems</title><meeting>the Fourth IAPR International Workshop on Document Analysis Systems<address><addrLine>Rio de Janeiro, Brazil</addrLine></address></meeting><imprint><date type="published" when="1948"></date><biblScope unit="page" from="475" to="485"></biblScope></imprint></monogr></biblStruct><biblStruct xml:id="b17"><monogr><title level="m" type="main">The Texbook</title><author><persName><forename type="first">D</forename><surname>Knuth</surname></persName></author><imprint><date type="published" when="1984"></date><publisher>Addison-Wesley</publisher><biblScope unit="page">51</biblScope></imprint></monogr></biblStruct><biblStruct xml:id="b18"><analytic><title level="a" type="main">A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics</title><author><persName><forename type="first">D</forename><surname>Martin</surname></persName></author><author><persName><forename type="first">C</forename><surname>Fowlkes</surname></persName></author><author><persName><forename type="first">D</forename><surname>Tal</surname></persName></author><author><persName><forename type="first">J</forename><surname>Malik</surname></persName></author></analytic><monogr><title level="m">Proceedings of the International Conference on Computer Vision (ICCV), pages II</title><meeting>the International Conference on Computer Vision (ICCV), pages II<address><addrLine>Vancouver, Canada</addrLine></address></meeting><imprint><date type="published" when="2001-07"></date><biblScope unit="page" from="416" to="421"></biblScope></imprint></monogr></biblStruct><biblStruct xml:id="b19"><analytic><title level="a" type="main">Format of ground truth data used in the evaluation of the results of an optical music recognition system</title><author><persName><forename type="first">H</forename><surname>Miyao</surname></persName></author><author><persName><forename type="first">R</forename><forename type="middle">M</forename><surname>Haralick</surname></persName></author></analytic><monogr><title level="m">Proceedings of the Fourth IAPR International Workshop on Document Analysis Systems</title><meeting>the Fourth IAPR International Workshop on Document Analysis Systems<address><addrLine>Rio de Janeiro, Brazil</addrLine></address></meeting><imprint><date type="published" when="1957"></date><biblScope unit="page" from="497" to="506"></biblScope></imprint></monogr></biblStruct><biblStruct xml:id="b20"><analytic><title level="a" type="main">Hierarchical representation of optically scanned documents</title><author><persName><forename type="first">G</forename><surname>Nagy</surname></persName></author><author><persName><forename type="first">S</forename><surname>Seth</surname></persName></author></analytic><monogr><title level="m">Proceedings of the Seventh International Conference on Pattern Recognition</title><meeting>the Seventh International Conference on Pattern Recognition<address><addrLine>Montréal, Canada</addrLine></address></meeting><imprint><date type="published" when="1947"></date><biblScope unit="page" from="347" to="349"></biblScope></imprint></monogr></biblStruct><biblStruct xml:id="b21"><analytic><title level="a" type="main">An experimental implementation of a document recognition system for papers containing mathematical expressions</title><author><persName><forename type="first">M</forename><surname>Okamoto</surname></persName></author><author><persName><forename type="first">A</forename><surname>Miyazawa</surname></persName></author></analytic><monogr><title level="m">Structured Document Image Analysis</title><editor>H. S. Baird, H. Bunke, and K. Yamamoto</editor><meeting><address><addrLine>Berlin, Germany</addrLine></address></meeting><imprint><publisher>Springer-Verlag</publisher><date type="published" when="1992"></date><biblScope unit="page">52</biblScope></imprint></monogr></biblStruct><biblStruct xml:id="b22"><analytic><title level="a" type="main">The implementation methodology for the CD-ROM English document database</title><author><persName><forename type="first">I</forename><surname>Phillips</surname></persName></author><author><persName><forename type="first">J</forename><surname>Ha</surname></persName></author><author><persName><forename type="first">R</forename><surname>Haralick</surname></persName></author><author><persName><forename type="first">D</forename><surname>Dori</surname></persName></author></analytic><monogr><title level="m">Proceedings of Second International Conference on Document Analysis and Recognition</title><meeting>Second International Conference on Document Analysis and Recognition<address><addrLine>Tsukuba Science City, Japan</addrLine></address></meeting><imprint><date type="published" when="1956"></date><biblScope unit="page" from="484" to="487"></biblScope></imprint></monogr></biblStruct><biblStruct xml:id="b23"><analytic><title level="a" type="main">English document database design and implementation methodology</title><author><persName><forename type="first">I</forename><forename type="middle">T</forename><surname>Phillips</surname></persName></author><author><persName><forename type="first">S</forename><surname>Chen</surname></persName></author><author><persName><forename type="first">J</forename><surname>Ha</surname></persName></author><author><persName><forename type="first">R</forename><forename type="middle">M</forename><surname>Haralick</surname></persName></author></analytic><monogr><title level="m">Proceedings of the Second Annual Symposium on Document Analysis and Information Retrieval</title><meeting>the Second Annual Symposium on Document Analysis and Information Retrieval<address><addrLine>Las Vegas, NV</addrLine></address></meeting><imprint><date type="published" when="1993-04"></date><biblScope unit="page" from="65" to="104"></biblScope></imprint></monogr></biblStruct><biblStruct xml:id="b24"><analytic><title level="a" type="main">Implementation methodology and error analysis for the CD-ROM English document database</title><author><persName><forename type="first">I</forename><forename type="middle">T</forename><surname>Phillips</surname></persName></author><author><persName><forename type="first">J</forename><surname>Ha</surname></persName></author><author><persName><forename type="first">S</forename><surname>Chen</surname></persName></author><author><persName><forename type="first">R</forename><forename type="middle">M</forename><surname>Haralick</surname></persName></author></analytic><monogr><title level="m">Proceedings of the AIPR Workshop</title><meeting>the AIPR Workshop<address><addrLine>Washington DC</addrLine></address></meeting><imprint><date type="published" when="1956"></date></imprint></monogr></biblStruct><biblStruct xml:id="b25"><analytic><title level="a" type="main">Squaring the circle: Validation without ground truth</title><author><persName><forename type="first">P</forename><surname>Ratiu</surname></persName></author><author><persName><forename type="first">R</forename><surname>Kikinis</surname></persName></author></analytic><monogr><title level="m">Proceedings of the Third Visible Human Project Conference</title><meeting>the Third Visible Human Project Conference<address><addrLine>Bethesda, MD</addrLine></address></meeting><imprint><date type="published" when="2000-10"></date></imprint></monogr></biblStruct><biblStruct xml:id="b26"><monogr><title level="m" type="main">Preparing OCR test data</title><author><persName><forename type="first">S</forename><forename type="middle">V</forename><surname>Rice</surname></persName></author><author><persName><forename type="first">J</forename><surname>Kanai</surname></persName></author><author><persName><forename type="first">T</forename><forename type="middle">A</forename><surname>Nartker</surname></persName></author><imprint><date type="published" when="1993-06"></date><publisher>UNLV Information Science Research Institute</publisher><biblScope unit="page">54</biblScope><pubPlace>Las Vegas, NV</pubPlace></imprint></monogr></biblStruct><biblStruct xml:id="b27"><monogr><title level="m" type="main">Optical Character Recognition: An Illustrated Guide to the Frontier</title><author><persName><forename type="first">S</forename><forename type="middle">V</forename><surname>Rice</surname></persName></author><author><persName><forename type="first">G</forename><surname>Nagy</surname></persName></author><author><persName><forename type="first">T</forename><forename type="middle">A</forename><surname>Nartker</surname></persName></author><imprint><date type="published" when="1999"></date><publisher>Kluwer Academic Publishers</publisher><biblScope unit="page">59</biblScope><pubPlace>Norwell, MA</pubPlace></imprint></monogr></biblStruct><biblStruct xml:id="b28"><monogr><title></title><author><persName><forename type="first">Inc</forename><surname>Scansoft</surname></persName></author><author><persName><forename type="first">Ma</forename><surname>Peabody</surname></persName></author><author><persName><surname>Xdoc Data</surname></persName></author><author><persName><surname>Format</surname></persName></author><imprint><date type="published" when="1951"></date></imprint></monogr></biblStruct><biblStruct xml:id="b29"><analytic><title level="a" type="main">Truthing, testing and evaluation issues in complex systems</title><author><persName><forename type="first">S</forename><surname>Setlur</surname></persName></author><author><persName><forename type="first">V</forename><surname>Govindaraju</surname></persName></author><author><persName><forename type="first">S</forename><surname>Srihari</surname></persName></author></analytic><monogr><title level="m">Proceedings of the Symposium on Document Image Understanding Technology</title><meeting>the Symposium on Document Image Understanding Technology<address><addrLine>Annapolis, MD</addrLine></address></meeting><imprint><date type="published" when="1951"></date><biblScope unit="page" from="131" to="140"></biblScope></imprint></monogr></biblStruct><biblStruct xml:id="b30"><analytic><title level="a" type="main">Towards robust features for classifying audio in the CueVideo system</title><author><persName><forename type="first">S</forename><surname>Srinivasan</surname></persName></author><author><persName><forename type="first">D</forename><surname>Petkovic</surname></persName></author><author><persName><forename type="first">D</forename><surname>Ponceleon</surname></persName></author></analytic><monogr><title level="m">Proceedings of ACM Multimedia '99</title><meeting>ACM Multimedia '99<address><addrLine>Orlando FL</addrLine></address></meeting><imprint><date type="published" when="1958"></date><biblScope unit="page" from="393" to="400"></biblScope></imprint></monogr></biblStruct><biblStruct xml:id="b31"><analytic><title level="a" type="main">Experiences with high-volume, high accuracy document capture</title><author><persName><forename type="first">H</forename><forename type="middle">R</forename><surname>Stabler</surname></persName></author></analytic><monogr><title level="m">Document Analysis Systems</title><editor>A. L. Spitz and A. Dengel</editor><meeting><address><addrLine>Singapore</addrLine></address></meeting><imprint><date type="published" when="1995"></date><biblScope unit="page" from="38" to="51"></biblScope></imprint></monogr></biblStruct><biblStruct xml:id="b32"><monogr><title level="m" type="main">Tabular abstraction, editing, and formatting</title><author><persName><forename type="first">X</forename><surname>Wang</surname></persName></author><imprint><date type="published" when="1996"></date><biblScope unit="page">51</biblScope></imprint></monogr></biblStruct><biblStruct xml:id="b33"><analytic><title level="a" type="main">Automatic table ground truth generation and a background-analysis-based table structure extraction method</title><author><persName><forename type="first">Y</forename><surname>Wang</surname></persName></author><author><persName><forename type="first">I</forename><forename type="middle">T</forename><surname>Phillips</surname></persName></author><author><persName><forename type="first">R</forename><surname>Haralick</surname></persName></author></analytic><monogr><title level="m">Proceedings of the Sixth International Conference on Document Analysis and Recognition</title><meeting>the Sixth International Conference on Document Analysis and Recognition<address><addrLine>Seattle, WA</addrLine></address></meeting><imprint><date type="published" when="1950"></date><biblScope unit="page" from="528" to="532"></biblScope></imprint></monogr></biblStruct><biblStruct xml:id="b34"><analytic><title level="a" type="main">Pink Panther: a complete environment for ground-truthing and benchmarking document page segmentation</title><author><persName><forename type="first">B</forename><forename type="middle">A</forename><surname>Yanikoglu</surname></persName></author><author><persName><forename type="first">L</forename><surname>Vincent</surname></persName></author></analytic><monogr><title level="j">Pattern Recognition</title><imprint><biblScope unit="volume">31</biblScope><biblScope unit="issue">47</biblScope><biblScope unit="page" from="1191" to="1204"></biblScope><date type="published" when="1998"></date></imprint></monogr></biblStruct><biblStruct xml:id="b35"><analytic><title level="a" type="main">Towards a common validation methodology for segmentation and registration algorithms</title><author><persName><forename type="first">T</forename><forename type="middle">S</forename><surname>Yoo</surname></persName></author><author><persName><forename type="first">M</forename><forename type="middle">J</forename><surname>Ackerman</surname></persName></author><author><persName><forename type="first">M</forename><surname>Vannier</surname></persName></author></analytic><monogr><title level="m">Medical Image Computing and Computer-Assisted Intervention</title><editor>S. Delp, A. DiGioia, and B. Jaramaz</editor><imprint><date type="published" when="1935"></date><biblScope unit="page" from="422" to="431"></biblScope></imprint></monogr></biblStruct></listBibl></back></text></istex:refBibTEI></enrichments></istex></record>Pour manipuler ce document sous Unix (Dilib)
EXPLOR_STEP=$WICRI_ROOT/Ticri/CIDE/explor/OcrV1/Data/Istex/Corpus
HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 000126 | SxmlIndent | more
Ou
HfdSelect -h $EXPLOR_AREA/Data/Istex/Corpus/biblio.hfd -nk 000126 | SxmlIndent | more
Pour mettre un lien sur cette page dans le réseau Wicri
{{Explor lien
|wiki= Ticri/CIDE
|area= OcrV1
|flux= Istex
|étape= Corpus
|type= RBID
|clé= ISTEX:3F3853C91990BECD53027DC72DA1372D1BD9B4BF
|texte= Issues in Ground-Truthing Graphic Documents
}}
| This area was generated with Dilib version V0.6.32. |  |