Interpretation, Evaluation and the Semantic Gap ... What if we Were on a Side-Track?
Identifieur interne : 002C40 ( Hal/Corpus ); précédent : 002C39; suivant : 002C41Interpretation, Evaluation and the Semantic Gap ... What if we Were on a Side-Track?
Auteurs : Bart LamiroySource :
English descriptors
Abstract
A significant amount of research in Document Image Analysis, and Machine Perception in general, relies on the extraction and analysis of signal cues with the goal of interpreting them into higher level information. This paper gives an overview on how this interpretation process is usually considered, and how the research communities proceed in evaluating existing approaches and methods developed for realizing these processes. Evaluation being an essential part to measuring the quality of research and assessing the progress of the state-of-the art, our work aims at showing that classical evaluation methods are not necessarily well suited for interpretation problems, or, at least, that they introduce a strong bias, not necessarily visible at first sight, and that new ways of comparing methods and measuring performance are necessary. It also shows that the infamous {\em Semantic Gap} seems to be an inherent and unavoidable part of the general interpretation process, especially when considered within the framework of traditional evaluation. The use of Formal Concept Analysis is put forward to leverage these limitations into a new tool to the analysis and comparison of interpretation contexts.
Url:
Links to Exploration step
Hal:hal-01057362Le document en format XML
<record><TEI><teiHeader><fileDesc><titleStmt><title xml:lang="en">Interpretation, Evaluation and the Semantic Gap ... What if we Were on a Side-Track?</title><author><name sortKey="Lamiroy, Bart" sort="Lamiroy, Bart" uniqKey="Lamiroy B" first="Bart" last="Lamiroy">Bart Lamiroy</name><affiliation><hal:affiliation type="researchteam" xml:id="struct-206044" status="VALID"><orgName>Querying Graphics through Analysis and Recognition</orgName><orgName type="acronym">QGAR</orgName><desc><address><country key="FR"></country></address><ref type="url">http://qgar.loria.fr</ref></desc><listRelation><relation active="#struct-423086" type="direct"></relation><relation active="#struct-206040" type="indirect"></relation><relation active="#struct-300009" type="indirect"></relation><relation active="#struct-413289" type="indirect"></relation><relation name="UMR7503" active="#struct-441569" type="indirect"></relation></listRelation><tutelles><tutelle active="#struct-423086" type="direct"><org type="department" xml:id="struct-423086" status="VALID"><orgName>Department of Natural Language Processing & Knowledge Discovery</orgName><orgName type="acronym">LORIA - NLPKD</orgName><desc><address><country key="FR"></country></address><ref type="url">http://www.loria.fr/la-recherche-en/departements/Knowledge-and-Language-Management</ref></desc><listRelation><relation active="#struct-206040" type="direct"></relation><relation active="#struct-300009" type="indirect"></relation><relation active="#struct-413289" type="indirect"></relation><relation name="UMR7503" active="#struct-441569" type="indirect"></relation></listRelation></org></tutelle><tutelle active="#struct-206040" type="indirect"><org type="laboratory" xml:id="struct-206040" status="VALID"><idno type="IdRef">067077927</idno><idno type="RNSR">198912571S</idno><idno type="IdUnivLorraine">[UL]RSI--</idno><orgName>Laboratoire Lorrain de Recherche en Informatique et ses Applications</orgName><orgName type="acronym">LORIA</orgName><date type="start">2012-01-01</date><desc><address><addrLine>Campus Scientifique BP 239 54506 Vandoeuvre-lès-Nancy Cedex</addrLine><country key="FR"></country></address><ref type="url">http://www.loria.fr</ref></desc><listRelation><relation active="#struct-300009" type="direct"></relation><relation active="#struct-413289" type="direct"></relation><relation name="UMR7503" active="#struct-441569" type="direct"></relation></listRelation></org></tutelle><tutelle active="#struct-300009" type="indirect"><org type="institution" xml:id="struct-300009" status="VALID"><orgName>Institut National de Recherche en Informatique et en Automatique</orgName><orgName type="acronym">Inria</orgName><desc><address><addrLine>Domaine de VoluceauRocquencourt - BP 10578153 Le Chesnay Cedex</addrLine><country key="FR"></country></address><ref type="url">http://www.inria.fr/en/</ref></desc></org></tutelle><tutelle active="#struct-413289" type="indirect"><org type="institution" xml:id="struct-413289" status="VALID"><idno type="IdRef">157040569</idno><idno type="IdUnivLorraine">[UL]100--</idno><orgName>Université de Lorraine</orgName><orgName type="acronym">UL</orgName><date type="start">2012-01-01</date><desc><address><addrLine>34 cours Léopold - CS 25233 - 54052 Nancy cedex</addrLine><country key="FR"></country></address><ref type="url">http://www.univ-lorraine.fr/</ref></desc></org></tutelle><tutelle name="UMR7503" active="#struct-441569" type="indirect"><org type="institution" xml:id="struct-441569" status="VALID"><idno type="ISNI">0000000122597504</idno><idno type="IdRef">02636817X</idno><orgName>Centre National de la Recherche Scientifique</orgName><orgName type="acronym">CNRS</orgName><date type="start">1939-10-19</date><desc><address><country key="FR"></country></address><ref type="url">http://www.cnrs.fr/</ref></desc></org></tutelle></tutelles></hal:affiliation></affiliation></author></titleStmt><publicationStmt><idno type="wicri:source">HAL</idno><idno type="RBID">Hal:hal-01057362</idno><idno type="halId">hal-01057362</idno><idno type="halUri">https://hal.inria.fr/hal-01057362</idno><idno type="url">https://hal.inria.fr/hal-01057362</idno><date when="2013-08-20">2013-08-20</date><idno type="wicri:Area/Hal/Corpus">002C40</idno></publicationStmt><sourceDesc><biblStruct><analytic><title xml:lang="en">Interpretation, Evaluation and the Semantic Gap ... What if we Were on a Side-Track?</title><author><name sortKey="Lamiroy, Bart" sort="Lamiroy, Bart" uniqKey="Lamiroy B" first="Bart" last="Lamiroy">Bart Lamiroy</name><affiliation><hal:affiliation type="researchteam" xml:id="struct-206044" status="VALID"><orgName>Querying Graphics through Analysis and Recognition</orgName><orgName type="acronym">QGAR</orgName><desc><address><country key="FR"></country></address><ref type="url">http://qgar.loria.fr</ref></desc><listRelation><relation active="#struct-423086" type="direct"></relation><relation active="#struct-206040" type="indirect"></relation><relation active="#struct-300009" type="indirect"></relation><relation active="#struct-413289" type="indirect"></relation><relation name="UMR7503" active="#struct-441569" type="indirect"></relation></listRelation><tutelles><tutelle active="#struct-423086" type="direct"><org type="department" xml:id="struct-423086" status="VALID"><orgName>Department of Natural Language Processing & Knowledge Discovery</orgName><orgName type="acronym">LORIA - NLPKD</orgName><desc><address><country key="FR"></country></address><ref type="url">http://www.loria.fr/la-recherche-en/departements/Knowledge-and-Language-Management</ref></desc><listRelation><relation active="#struct-206040" type="direct"></relation><relation active="#struct-300009" type="indirect"></relation><relation active="#struct-413289" type="indirect"></relation><relation name="UMR7503" active="#struct-441569" type="indirect"></relation></listRelation></org></tutelle><tutelle active="#struct-206040" type="indirect"><org type="laboratory" xml:id="struct-206040" status="VALID"><idno type="IdRef">067077927</idno><idno type="RNSR">198912571S</idno><idno type="IdUnivLorraine">[UL]RSI--</idno><orgName>Laboratoire Lorrain de Recherche en Informatique et ses Applications</orgName><orgName type="acronym">LORIA</orgName><date type="start">2012-01-01</date><desc><address><addrLine>Campus Scientifique BP 239 54506 Vandoeuvre-lès-Nancy Cedex</addrLine><country key="FR"></country></address><ref type="url">http://www.loria.fr</ref></desc><listRelation><relation active="#struct-300009" type="direct"></relation><relation active="#struct-413289" type="direct"></relation><relation name="UMR7503" active="#struct-441569" type="direct"></relation></listRelation></org></tutelle><tutelle active="#struct-300009" type="indirect"><org type="institution" xml:id="struct-300009" status="VALID"><orgName>Institut National de Recherche en Informatique et en Automatique</orgName><orgName type="acronym">Inria</orgName><desc><address><addrLine>Domaine de VoluceauRocquencourt - BP 10578153 Le Chesnay Cedex</addrLine><country key="FR"></country></address><ref type="url">http://www.inria.fr/en/</ref></desc></org></tutelle><tutelle active="#struct-413289" type="indirect"><org type="institution" xml:id="struct-413289" status="VALID"><idno type="IdRef">157040569</idno><idno type="IdUnivLorraine">[UL]100--</idno><orgName>Université de Lorraine</orgName><orgName type="acronym">UL</orgName><date type="start">2012-01-01</date><desc><address><addrLine>34 cours Léopold - CS 25233 - 54052 Nancy cedex</addrLine><country key="FR"></country></address><ref type="url">http://www.univ-lorraine.fr/</ref></desc></org></tutelle><tutelle name="UMR7503" active="#struct-441569" type="indirect"><org type="institution" xml:id="struct-441569" status="VALID"><idno type="ISNI">0000000122597504</idno><idno type="IdRef">02636817X</idno><orgName>Centre National de la Recherche Scientifique</orgName><orgName type="acronym">CNRS</orgName><date type="start">1939-10-19</date><desc><address><country key="FR"></country></address><ref type="url">http://www.cnrs.fr/</ref></desc></org></tutelle></tutelles></hal:affiliation></affiliation></author></analytic></biblStruct></sourceDesc></fileDesc><profileDesc><textClass><keywords scheme="mix" xml:lang="en"><term>Document Image Analysis</term><term>Formal Concept Analysis</term><term>Performance Analysis</term><term>Semantic Gap</term></keywords></textClass></profileDesc></teiHeader><front><div type="abstract" xml:lang="en">A significant amount of research in Document Image Analysis, and Machine Perception in general, relies on the extraction and analysis of signal cues with the goal of interpreting them into higher level information. This paper gives an overview on how this interpretation process is usually considered, and how the research communities proceed in evaluating existing approaches and methods developed for realizing these processes. Evaluation being an essential part to measuring the quality of research and assessing the progress of the state-of-the art, our work aims at showing that classical evaluation methods are not necessarily well suited for interpretation problems, or, at least, that they introduce a strong bias, not necessarily visible at first sight, and that new ways of comparing methods and measuring performance are necessary. It also shows that the infamous {\em Semantic Gap} seems to be an inherent and unavoidable part of the general interpretation process, especially when considered within the framework of traditional evaluation. The use of Formal Concept Analysis is put forward to leverage these limitations into a new tool to the analysis and comparison of interpretation contexts.</div></front></TEI><hal api="V3"><titleStmt><title xml:lang="en">Interpretation, Evaluation and the Semantic Gap ... What if we Were on a Side-Track?</title><author role="aut"><persName><forename type="first">Bart</forename><surname>Lamiroy</surname></persName><email>Bart.Lamiroy@loria.fr</email><ptr type="url" target="http://www.loria.fr/~lamiroy"></ptr><idno type="idhal">bart-lamiroy</idno><idno type="halauthor">677314</idno><idno type="ResearcherId">http://www.researcherid.com/rid/A-7746-2010</idno><idno type="IdRef">http://www.idref.fr/111726980</idno><idno type="ORCID">http://orcid.org/0000-0003-0871-0149</idno><affiliation ref="#struct-206044"></affiliation></author><editor role="depositor"><persName><forename>Bart</forename><surname>Lamiroy</surname></persName><email>Bart.Lamiroy@loria.fr</email></editor></titleStmt><editionStmt><edition n="v1" type="current"><date type="whenSubmitted">2014-08-22 13:06:31</date><date type="whenModified">2015-09-21 11:15:43</date><date type="whenReleased">2014-09-03 09:26:05</date><date type="whenProduced">2013-08-20</date><date type="whenEndEmbargoed">2014-08-22</date><ref type="file" target="https://hal.inria.fr/hal-01057362/document"><date notBefore="2014-08-22"></date></ref><ref type="file" subtype="author" n="1" target="https://hal.inria.fr/hal-01057362/file/GREC_2013_LNCS.pdf"><date notBefore="2014-08-22"></date></ref></edition><respStmt><resp>contributor</resp><name key="113383"><persName><forename>Bart</forename><surname>Lamiroy</surname></persName><email>Bart.Lamiroy@loria.fr</email></name></respStmt></editionStmt><publicationStmt><distributor>CCSD</distributor><idno type="halId">hal-01057362</idno><idno type="halUri">https://hal.inria.fr/hal-01057362</idno><idno type="halBibtex">lamiroy:hal-01057362</idno><idno type="halRefHtml">Bart Lamiroy and Jean-Marc Ogier. 10th IAPR International Workshop on Graphics Recognition, GREC 2013, Aug 2013, Bethlehem, PA, United States. Springer, 8746, pp.213-226, 2014, LNCS; Graphics Recognition. Current Trends and Challenges</idno><idno type="halRef">Bart Lamiroy and Jean-Marc Ogier. 10th IAPR International Workshop on Graphics Recognition, GREC 2013, Aug 2013, Bethlehem, PA, United States. Springer, 8746, pp.213-226, 2014, LNCS; Graphics Recognition. Current Trends and Challenges</idno></publicationStmt><seriesStmt><idno type="stamp" n="CNRS">CNRS - Centre national de la recherche scientifique</idno><idno type="stamp" n="INRIA">INRIA - Institut National de Recherche en Informatique et en Automatique</idno><idno type="stamp" n="LORIA2">Publications du LORIA</idno><idno type="stamp" n="LORIA">LORIA - Laboratoire Lorrain de Recherche en Informatique et ses Applications</idno><idno type="stamp" n="LORIA-TALC" p="LORIA">Traitement automatique des langues et des connaissances</idno><idno type="stamp" n="UNIV-LORRAINE">Université de Lorraine</idno></seriesStmt><notesStmt><note type="audience" n="2">International</note><note type="invited" n="0">No</note><note type="popular" n="0">No</note><note type="peer" n="1">Yes</note><note type="proceedings" n="1">Yes</note></notesStmt><sourceDesc><biblStruct><analytic><title xml:lang="en">Interpretation, Evaluation and the Semantic Gap ... What if we Were on a Side-Track?</title><author role="aut"><persName><forename type="first">Bart</forename><surname>Lamiroy</surname></persName><email>Bart.Lamiroy@loria.fr</email><ptr type="url" target="http://www.loria.fr/~lamiroy"></ptr><idno type="idHal">bart-lamiroy</idno><idno type="halAuthorId">677314</idno><idno type="ResearcherId">http://www.researcherid.com/rid/A-7746-2010</idno><idno type="IdRef">http://www.idref.fr/111726980</idno><idno type="ORCID">http://orcid.org/0000-0003-0871-0149</idno><affiliation ref="#struct-206044"></affiliation></author></analytic><monogr><meeting><title>10th IAPR International Workshop on Graphics Recognition, GREC 2013</title><date type="start">2013-08-20</date><date type="end">2013-08-21</date><settlement>Bethlehem, PA</settlement><country key="US">United States</country></meeting><editor>Bart Lamiroy and Jean-Marc Ogier</editor><imprint><publisher>Springer</publisher><biblScope unit="serie"></biblScope><biblScope unit="volume">8746</biblScope><biblScope unit="pp">213-226</biblScope><date type="datePub">2014-11</date></imprint></monogr></biblStruct></sourceDesc><profileDesc><langUsage><language ident="en">English</language></langUsage><textClass><keywords scheme="author"><term xml:lang="en">Document Image Analysis</term><term xml:lang="en">Performance Analysis</term><term xml:lang="en">Semantic Gap</term><term xml:lang="en">Formal Concept Analysis</term></keywords><classCode scheme="halDomain" n="info.info-cv">Computer Science [cs]/Computer Vision and Pattern Recognition [cs.CV]</classCode><classCode scheme="halTypology" n="COMM">Conference papers</classCode></textClass><abstract xml:lang="en">A significant amount of research in Document Image Analysis, and Machine Perception in general, relies on the extraction and analysis of signal cues with the goal of interpreting them into higher level information. This paper gives an overview on how this interpretation process is usually considered, and how the research communities proceed in evaluating existing approaches and methods developed for realizing these processes. Evaluation being an essential part to measuring the quality of research and assessing the progress of the state-of-the art, our work aims at showing that classical evaluation methods are not necessarily well suited for interpretation problems, or, at least, that they introduce a strong bias, not necessarily visible at first sight, and that new ways of comparing methods and measuring performance are necessary. It also shows that the infamous {\em Semantic Gap} seems to be an inherent and unavoidable part of the general interpretation process, especially when considered within the framework of traditional evaluation. The use of Formal Concept Analysis is put forward to leverage these limitations into a new tool to the analysis and comparison of interpretation contexts.</abstract></profileDesc></hal></record>Pour manipuler ce document sous Unix (Dilib)
EXPLOR_STEP=$WICRI_ROOT/Wicri/Lorraine/explor/InforLorV4/Data/Hal/Corpus
HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 002C40 | SxmlIndent | more
Ou
HfdSelect -h $EXPLOR_AREA/Data/Hal/Corpus/biblio.hfd -nk 002C40 | SxmlIndent | more
Pour mettre un lien sur cette page dans le réseau Wicri
{{Explor lien
|wiki= Wicri/Lorraine
|area= InforLorV4
|flux= Hal
|étape= Corpus
|type= RBID
|clé= Hal:hal-01057362
|texte= Interpretation, Evaluation and the Semantic Gap ... What if we Were on a Side-Track?
}}
| This area was generated with Dilib version V0.6.33. |  |