Layout-aware text extraction from full-text PDF of scientific articles
Identifieur interne : 000131 ( Ncbi/Merge ); précédent : 000130; suivant : 000132Layout-aware text extraction from full-text PDF of scientific articles
Auteurs : Cartic Ramakrishnan [États-Unis] ; Abhishek Patnia [États-Unis] ; Eduard Hovy [États-Unis] ; Gully Apc Burns [États-Unis]Source :
- Source Code for Biology and Medicine [ 1751-0473 ] ; 2012.
Abstract
The Portable Document Format (PDF) is the most commonly used file format for online scientific publications. The absence of effective means to extract text from these PDF files in a layout-aware manner presents a significant challenge for developers of biomedical text mining or biocuration informatics systems that use published literature as an information source. In this paper we introduce the ‘Layout-Aware PDF Text Extraction’ (LA-PDFText) system to facilitate accurate extraction of text from PDF files of research articles for use in text mining applications.
Our paper describes the construction and performance of an open source system that extracts text blocks from PDF-formatted full-text research articles and classifies them into logical units based on rules that characterize specific sections. The LA-PDFText system focuses only on the textual content of the research articles and is meant as a baseline for further experiments into more advanced extraction methods that handle multi-modal content, such as images and graphs. The system works in a three-stage process: (1)
LA-PDFText is an open-source tool for accurately extracting text from full-text scientific articles. The release of the system is available at
Url:
DOI: 10.1186/1751-0473-7-7
PubMed: 22640904
PubMed Central: 3441580
Links toward previous steps (curation, corpus...)
- to stream Pmc, to step Corpus: 000081
- to stream Pmc, to step Curation: 000081
- to stream Pmc, to step Checkpoint: 000101
Links to Exploration step
PMC:3441580Le document en format XML
<record><TEI><teiHeader><fileDesc><titleStmt><title xml:lang="en">Layout-aware text extraction from full-text PDF of scientific articles</title>
<author><name sortKey="Ramakrishnan, Cartic" sort="Ramakrishnan, Cartic" uniqKey="Ramakrishnan C" first="Cartic" last="Ramakrishnan">Cartic Ramakrishnan</name>
<affiliation wicri:level="4"><nlm:aff id="I1">Information Sciences Institute, University of Southern California, 4676 Admiralty Way, Suite 1001, Marina del Rey, CA, 90292-6695, USA</nlm:aff>
<country xml:lang="fr">États-Unis</country>
<wicri:regionArea>Information Sciences Institute, University of Southern California, 4676 Admiralty Way, Suite 1001, Marina del Rey, CA, 90292-6695</wicri:regionArea>
<orgName type="university">Université de Californie du Sud</orgName>
<placeName><settlement type="city">Los Angeles</settlement>
<region type="state">Californie</region>
</placeName>
</affiliation>
</author>
<author><name sortKey="Patnia, Abhishek" sort="Patnia, Abhishek" uniqKey="Patnia A" first="Abhishek" last="Patnia">Abhishek Patnia</name>
<affiliation wicri:level="4"><nlm:aff id="I2">Computer Science Department, University of Southern California, 941 Bloom Walker, Los Angeles, CA, 90089-0781, USA</nlm:aff>
<country xml:lang="fr">États-Unis</country>
<wicri:regionArea>Computer Science Department, University of Southern California, 941 Bloom Walker, Los Angeles, CA, 90089-0781</wicri:regionArea>
<orgName type="university">Université de Californie du Sud</orgName>
<placeName><settlement type="city">Los Angeles</settlement>
<region type="state">Californie</region>
</placeName>
</affiliation>
</author>
<author><name sortKey="Hovy, Eduard" sort="Hovy, Eduard" uniqKey="Hovy E" first="Eduard" last="Hovy">Eduard Hovy</name>
<affiliation wicri:level="4"><nlm:aff id="I1">Information Sciences Institute, University of Southern California, 4676 Admiralty Way, Suite 1001, Marina del Rey, CA, 90292-6695, USA</nlm:aff>
<country xml:lang="fr">États-Unis</country>
<wicri:regionArea>Information Sciences Institute, University of Southern California, 4676 Admiralty Way, Suite 1001, Marina del Rey, CA, 90292-6695</wicri:regionArea>
<orgName type="university">Université de Californie du Sud</orgName>
<placeName><settlement type="city">Los Angeles</settlement>
<region type="state">Californie</region>
</placeName>
</affiliation>
</author>
<author><name sortKey="Burns, Gully Apc" sort="Burns, Gully Apc" uniqKey="Burns G" first="Gully Apc" last="Burns">Gully Apc Burns</name>
<affiliation wicri:level="4"><nlm:aff id="I1">Information Sciences Institute, University of Southern California, 4676 Admiralty Way, Suite 1001, Marina del Rey, CA, 90292-6695, USA</nlm:aff>
<country xml:lang="fr">États-Unis</country>
<wicri:regionArea>Information Sciences Institute, University of Southern California, 4676 Admiralty Way, Suite 1001, Marina del Rey, CA, 90292-6695</wicri:regionArea>
<orgName type="university">Université de Californie du Sud</orgName>
<placeName><settlement type="city">Los Angeles</settlement>
<region type="state">Californie</region>
</placeName>
</affiliation>
</author>
</titleStmt>
<publicationStmt><idno type="wicri:source">PMC</idno>
<idno type="pmid">22640904</idno>
<idno type="pmc">3441580</idno>
<idno type="url">http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3441580</idno>
<idno type="RBID">PMC:3441580</idno>
<idno type="doi">10.1186/1751-0473-7-7</idno>
<date when="2012">2012</date>
<idno type="wicri:Area/Pmc/Corpus">000081</idno>
<idno type="wicri:Area/Pmc/Curation">000081</idno>
<idno type="wicri:Area/Pmc/Checkpoint">000101</idno>
<idno type="wicri:Area/Ncbi/Merge">000131</idno>
</publicationStmt>
<sourceDesc><biblStruct><analytic><title xml:lang="en" level="a" type="main">Layout-aware text extraction from full-text PDF of scientific articles</title>
<author><name sortKey="Ramakrishnan, Cartic" sort="Ramakrishnan, Cartic" uniqKey="Ramakrishnan C" first="Cartic" last="Ramakrishnan">Cartic Ramakrishnan</name>
<affiliation wicri:level="4"><nlm:aff id="I1">Information Sciences Institute, University of Southern California, 4676 Admiralty Way, Suite 1001, Marina del Rey, CA, 90292-6695, USA</nlm:aff>
<country xml:lang="fr">États-Unis</country>
<wicri:regionArea>Information Sciences Institute, University of Southern California, 4676 Admiralty Way, Suite 1001, Marina del Rey, CA, 90292-6695</wicri:regionArea>
<orgName type="university">Université de Californie du Sud</orgName>
<placeName><settlement type="city">Los Angeles</settlement>
<region type="state">Californie</region>
</placeName>
</affiliation>
</author>
<author><name sortKey="Patnia, Abhishek" sort="Patnia, Abhishek" uniqKey="Patnia A" first="Abhishek" last="Patnia">Abhishek Patnia</name>
<affiliation wicri:level="4"><nlm:aff id="I2">Computer Science Department, University of Southern California, 941 Bloom Walker, Los Angeles, CA, 90089-0781, USA</nlm:aff>
<country xml:lang="fr">États-Unis</country>
<wicri:regionArea>Computer Science Department, University of Southern California, 941 Bloom Walker, Los Angeles, CA, 90089-0781</wicri:regionArea>
<orgName type="university">Université de Californie du Sud</orgName>
<placeName><settlement type="city">Los Angeles</settlement>
<region type="state">Californie</region>
</placeName>
</affiliation>
</author>
<author><name sortKey="Hovy, Eduard" sort="Hovy, Eduard" uniqKey="Hovy E" first="Eduard" last="Hovy">Eduard Hovy</name>
<affiliation wicri:level="4"><nlm:aff id="I1">Information Sciences Institute, University of Southern California, 4676 Admiralty Way, Suite 1001, Marina del Rey, CA, 90292-6695, USA</nlm:aff>
<country xml:lang="fr">États-Unis</country>
<wicri:regionArea>Information Sciences Institute, University of Southern California, 4676 Admiralty Way, Suite 1001, Marina del Rey, CA, 90292-6695</wicri:regionArea>
<orgName type="university">Université de Californie du Sud</orgName>
<placeName><settlement type="city">Los Angeles</settlement>
<region type="state">Californie</region>
</placeName>
</affiliation>
</author>
<author><name sortKey="Burns, Gully Apc" sort="Burns, Gully Apc" uniqKey="Burns G" first="Gully Apc" last="Burns">Gully Apc Burns</name>
<affiliation wicri:level="4"><nlm:aff id="I1">Information Sciences Institute, University of Southern California, 4676 Admiralty Way, Suite 1001, Marina del Rey, CA, 90292-6695, USA</nlm:aff>
<country xml:lang="fr">États-Unis</country>
<wicri:regionArea>Information Sciences Institute, University of Southern California, 4676 Admiralty Way, Suite 1001, Marina del Rey, CA, 90292-6695</wicri:regionArea>
<orgName type="university">Université de Californie du Sud</orgName>
<placeName><settlement type="city">Los Angeles</settlement>
<region type="state">Californie</region>
</placeName>
</affiliation>
</author>
</analytic>
<series><title level="j">Source Code for Biology and Medicine</title>
<idno type="eISSN">1751-0473</idno>
<imprint><date when="2012">2012</date>
</imprint>
</series>
</biblStruct>
</sourceDesc>
</fileDesc>
<profileDesc><textClass></textClass>
</profileDesc>
</teiHeader>
<front><div type="abstract" xml:lang="en"><sec><title>Background</title>
<p>The Portable Document Format (PDF) is the most commonly used file format for online scientific publications. The absence of effective means to extract text from these PDF files in a layout-aware manner presents a significant challenge for developers of biomedical text mining or biocuration informatics systems that use published literature as an information source. In this paper we introduce the ‘Layout-Aware PDF Text Extraction’ (LA-PDFText) system to facilitate accurate extraction of text from PDF files of research articles for use in text mining applications.</p>
</sec>
<sec><title>Results</title>
<p>Our paper describes the construction and performance of an open source system that extracts text blocks from PDF-formatted full-text research articles and classifies them into logical units based on rules that characterize specific sections. The LA-PDFText system focuses only on the textual content of the research articles and is meant as a baseline for further experiments into more advanced extraction methods that handle multi-modal content, such as images and graphs. The system works in a three-stage process: (1) <bold>Detecting contiguous text blocks</bold>
using spatial layout processing to locate and identify blocks of contiguous text, (2) <bold>Classifying text blocks into rhetorical categories</bold>
using a rule-based method and (3) <bold>Stitching classified text blocks together in the correct order</bold>
resulting in the extraction of text from section-wise grouped blocks. We show that our system can identify text blocks and classify them into rhetorical categories with Precision<sup>1</sup>
= 0.96% Recall = 0.89% and F1 = 0.91%. We also present an evaluation of the accuracy of the block detection algorithm used in step 2. Additionally, we have compared the accuracy of the text extracted by LA-PDFText to the text from the Open Access subset of PubMed Central. We then compared this accuracy with that of the text extracted by the PDF2Text system, <sup>2</sup>
commonly used to extract text from PDF. Finally, we discuss preliminary error analysis for our system and identify further areas of improvement.</p>
</sec>
<sec><title>Conclusions</title>
<p>LA-PDFText is an open-source tool for accurately extracting text from full-text scientific articles. The release of the system is available at <ext-link ext-link-type="uri" xlink:href="http://code.google.com/p/lapdftext/">http://code.google.com/p/lapdftext/</ext-link>
.</p>
</sec>
</div>
</front>
<back><div1 type="bibliography"><listBibl><biblStruct><analytic><author><name sortKey="Rebholz Schuhmann, D" uniqKey="Rebholz Schuhmann D">D Rebholz-Schuhmann</name>
</author>
<author><name sortKey="Kirsch, H" uniqKey="Kirsch H">H Kirsch</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Altman, Rb" uniqKey="Altman R">RB Altman</name>
</author>
<author><name sortKey="Bergman, Cm" uniqKey="Bergman C">CM Bergman</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Settles, B" uniqKey="Settles B">B Settles</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Rosario, B" uniqKey="Rosario B">B Rosario</name>
</author>
<author><name sortKey="Hearst, Ma" uniqKey="Hearst M">MA Hearst</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Krallinger, M" uniqKey="Krallinger M">M Krallinger</name>
</author>
<author><name sortKey="Vazquez, M" uniqKey="Vazquez M">M Vazquez</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Chun, Hw" uniqKey="Chun H">HW Chun</name>
</author>
<author><name sortKey="Tsuruoka, Y" uniqKey="Tsuruoka Y">Y Tsuruoka</name>
</author>
<author><name sortKey="Kim, Jd" uniqKey="Kim J">JD Kim</name>
</author>
<author><name sortKey="Shiba, R" uniqKey="Shiba R">R Shiba</name>
</author>
<author><name sortKey="Nagata, N" uniqKey="Nagata N">N Nagata</name>
</author>
<author><name sortKey="Hishiki, T" uniqKey="Hishiki T">T Hishiki</name>
</author>
<author><name sortKey="Tsujii, J" uniqKey="Tsujii J">J Tsujii</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Cohen, Kb" uniqKey="Cohen K">KB Cohen</name>
</author>
<author><name sortKey="Johnson, Hl" uniqKey="Johnson H">HL Johnson</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Alex, B" uniqKey="Alex B">B Alex</name>
</author>
<author><name sortKey="Grover, C" uniqKey="Grover C">C Grover</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Ramakrishnan, C" uniqKey="Ramakrishnan C">C Ramakrishnan</name>
</author>
<author><name sortKey="Mendes, Pn" uniqKey="Mendes P">PN Mendes</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Ramakrishnan, C" uniqKey="Ramakrishnan C">C Ramakrishnan</name>
</author>
<author><name sortKey="Mendes, Pn" uniqKey="Mendes P">PN Mendes</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Roy, S" uniqKey="Roy S">S Roy</name>
</author>
<author><name sortKey="Heinrich, K" uniqKey="Heinrich K">K Heinrich</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Cohen, Am" uniqKey="Cohen A">AM Cohen</name>
</author>
<author><name sortKey="Hersh, Wr" uniqKey="Hersh W">WR Hersh</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Bourne, P" uniqKey="Bourne P">P Bourne</name>
</author>
<author><name sortKey="Mcentyre, J" uniqKey="Mcentyre J">J McEntyre</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Krallinger, M" uniqKey="Krallinger M">M Krallinger</name>
</author>
<author><name sortKey="Morgan, A" uniqKey="Morgan A">A Morgan</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Morgan, Aa" uniqKey="Morgan A">AA Morgan</name>
</author>
<author><name sortKey="Lu, Z" uniqKey="Lu Z">Z Lu</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Dowell, Kg" uniqKey="Dowell K">KG Dowell</name>
</author>
<author><name sortKey="Mcandrews Hill, Ms" uniqKey="Mcandrews Hill M">MS McAndrews-Hill</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Forgy, Cl" uniqKey="Forgy C">CL Forgy</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Needleman, Sb" uniqKey="Needleman S">SB Needleman</name>
</author>
<author><name sortKey="Wunsch, Cd" uniqKey="Wunsch C">CD Wunsch</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Dengel, A" uniqKey="Dengel A">A Dengel</name>
</author>
<author><name sortKey="Dubiel, F" uniqKey="Dubiel F">F Dubiel</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Esposito, F" uniqKey="Esposito F">F Esposito</name>
</author>
<author><name sortKey="Malerba, D" uniqKey="Malerba D">D Malerba</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Summers, Kristen" uniqKey="Summers K">Kristen Summers</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Luong, M T" uniqKey="Luong M">M-T Luong</name>
</author>
<author><name sortKey="Nguyen, Td" uniqKey="Nguyen T">TD Nguyen</name>
</author>
<author><name sortKey="Kan, M Y" uniqKey="Kan M">M-Y Kan</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Lafferty, Jd" uniqKey="Lafferty J">JD Lafferty</name>
</author>
<author><name sortKey="Mccallum, A" uniqKey="Mccallum A">A McCallum</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Attwood, Tk" uniqKey="Attwood T">TK Attwood</name>
</author>
<author><name sortKey="Kell, Db" uniqKey="Kell D">DB Kell</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Vroling, B" uniqKey="Vroling B">B Vroling</name>
</author>
<author><name sortKey="Thorne, D" uniqKey="Thorne D">D Thorne</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Liu, Y" uniqKey="Liu Y">Y Liu</name>
</author>
<author><name sortKey="Mitra, P" uniqKey="Mitra P">P Mitra</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Murphy, Rf" uniqKey="Murphy R">RF Murphy</name>
</author>
<author><name sortKey="Velliste, M" uniqKey="Velliste M">M Velliste</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Lee Giles, C" uniqKey="Lee Giles C">C Lee Giles</name>
</author>
<author><name sortKey="Councill, I" uniqKey="Councill I">I Councill</name>
</author>
<author><name sortKey="Kan, M Y" uniqKey="Kan M">M-Y Kan</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Ciccarese, P" uniqKey="Ciccarese P">P Ciccarese</name>
</author>
<author><name sortKey="Attwood, T" uniqKey="Attwood T">T Attwood</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Ciccarese, P" uniqKey="Ciccarese P">P Ciccarese</name>
</author>
<author><name sortKey="Ocana, M" uniqKey="Ocana M">M Ocana</name>
</author>
<author><name sortKey="Garcia Castro, Lj" uniqKey="Garcia Castro L">LJ Garcia Castro</name>
</author>
<author><name sortKey="Das, S" uniqKey="Das S">S Das</name>
</author>
<author><name sortKey="Clark, T" uniqKey="Clark T">T Clark</name>
</author>
</analytic>
</biblStruct>
</listBibl>
</div1>
</back>
</TEI>
<pmc article-type="product-review" xml:lang="en"><pmc-dir>properties open_access</pmc-dir>
<front><journal-meta><journal-id journal-id-type="nlm-ta">Source Code Biol Med</journal-id>
<journal-id journal-id-type="iso-abbrev">Source Code Biol Med</journal-id>
<journal-title-group><journal-title>Source Code for Biology and Medicine</journal-title>
</journal-title-group>
<issn pub-type="epub">1751-0473</issn>
<publisher><publisher-name>BioMed Central</publisher-name>
</publisher>
</journal-meta>
<article-meta><article-id pub-id-type="pmid">22640904</article-id>
<article-id pub-id-type="pmc">3441580</article-id>
<article-id pub-id-type="publisher-id">1751-0473-7-7</article-id>
<article-id pub-id-type="doi">10.1186/1751-0473-7-7</article-id>
<article-categories><subj-group subj-group-type="heading"><subject>Software Review</subject>
</subj-group>
</article-categories>
<title-group><article-title>Layout-aware text extraction from full-text PDF of scientific articles</article-title>
</title-group>
<contrib-group><contrib contrib-type="author" corresp="yes" id="A1"><name><surname>Ramakrishnan</surname>
<given-names>Cartic</given-names>
</name>
<xref ref-type="aff" rid="I1">1</xref>
<email>cartic@isi.edu</email>
</contrib>
<contrib contrib-type="author" id="A2"><name><surname>Patnia</surname>
<given-names>Abhishek</given-names>
</name>
<xref ref-type="aff" rid="I2">2</xref>
<email>patnia@usc.edu</email>
</contrib>
<contrib contrib-type="author" id="A3"><name><surname>Hovy</surname>
<given-names>Eduard</given-names>
</name>
<xref ref-type="aff" rid="I1">1</xref>
<email>hovy@isi.edu</email>
</contrib>
<contrib contrib-type="author" id="A4"><name><surname>Burns</surname>
<given-names>Gully APC</given-names>
</name>
<xref ref-type="aff" rid="I1">1</xref>
<email>burns@isi.edu</email>
</contrib>
</contrib-group>
<aff id="I1"><label>1</label>
Information Sciences Institute, University of Southern California, 4676 Admiralty Way, Suite 1001, Marina del Rey, CA, 90292-6695, USA</aff>
<aff id="I2"><label>2</label>
Computer Science Department, University of Southern California, 941 Bloom Walker, Los Angeles, CA, 90089-0781, USA</aff>
<pub-date pub-type="collection"><year>2012</year>
</pub-date>
<pub-date pub-type="epub"><day>28</day>
<month>5</month>
<year>2012</year>
</pub-date>
<volume>7</volume>
<fpage>7</fpage>
<lpage>7</lpage>
<history><date date-type="received"><day>24</day>
<month>4</month>
<year>2012</year>
</date>
<date date-type="accepted"><day>28</day>
<month>5</month>
<year>2012</year>
</date>
</history>
<permissions><copyright-statement>Copyright ©2012 Ramakrishnan et al.; licensee BioMed Central Ltd.</copyright-statement>
<copyright-year>2012</copyright-year>
<copyright-holder>Ramakrishnan et al.; licensee BioMed Central Ltd.</copyright-holder>
<license license-type="open-access" xlink:href="http://creativecommons.org/licenses/by/2.0"><license-p>This is an Open Access article distributed under the terms of the Creative Commons Attribution License (<ext-link ext-link-type="uri" xlink:href="http://creativecommons.org/licenses/by/2.0">http://creativecommons.org/licenses/by/2.0</ext-link>
), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.</license-p>
</license>
</permissions>
<self-uri xlink:href="http://www.scfbm.org/content/7/1/7"></self-uri>
<abstract><sec><title>Background</title>
<p>The Portable Document Format (PDF) is the most commonly used file format for online scientific publications. The absence of effective means to extract text from these PDF files in a layout-aware manner presents a significant challenge for developers of biomedical text mining or biocuration informatics systems that use published literature as an information source. In this paper we introduce the ‘Layout-Aware PDF Text Extraction’ (LA-PDFText) system to facilitate accurate extraction of text from PDF files of research articles for use in text mining applications.</p>
</sec>
<sec><title>Results</title>
<p>Our paper describes the construction and performance of an open source system that extracts text blocks from PDF-formatted full-text research articles and classifies them into logical units based on rules that characterize specific sections. The LA-PDFText system focuses only on the textual content of the research articles and is meant as a baseline for further experiments into more advanced extraction methods that handle multi-modal content, such as images and graphs. The system works in a three-stage process: (1) <bold>Detecting contiguous text blocks</bold>
using spatial layout processing to locate and identify blocks of contiguous text, (2) <bold>Classifying text blocks into rhetorical categories</bold>
using a rule-based method and (3) <bold>Stitching classified text blocks together in the correct order</bold>
resulting in the extraction of text from section-wise grouped blocks. We show that our system can identify text blocks and classify them into rhetorical categories with Precision<sup>1</sup>
= 0.96% Recall = 0.89% and F1 = 0.91%. We also present an evaluation of the accuracy of the block detection algorithm used in step 2. Additionally, we have compared the accuracy of the text extracted by LA-PDFText to the text from the Open Access subset of PubMed Central. We then compared this accuracy with that of the text extracted by the PDF2Text system, <sup>2</sup>
commonly used to extract text from PDF. Finally, we discuss preliminary error analysis for our system and identify further areas of improvement.</p>
</sec>
<sec><title>Conclusions</title>
<p>LA-PDFText is an open-source tool for accurately extracting text from full-text scientific articles. The release of the system is available at <ext-link ext-link-type="uri" xlink:href="http://code.google.com/p/lapdftext/">http://code.google.com/p/lapdftext/</ext-link>
.</p>
</sec>
</abstract>
</article-meta>
</front>
<body><sec><title>Background and motivation</title>
<p>The field of Biomedical Natural Language Processing (BioNLP) is maturing, with specific fields of software development in response to user requirements: <italic>e.g.</italic>
, links between databases and literature, better tool interactivity and integration and the development of high-quality NLP resources [<xref ref-type="bibr" rid="B1">1</xref>
,<xref ref-type="bibr" rid="B2">2</xref>
]. NLP techniques such as Named Entity Recognition [<xref ref-type="bibr" rid="B3">3</xref>
] and Semantic Relation Extraction [<xref ref-type="bibr" rid="B4">4</xref>
] have been shown to be very useful to biologists studying protein-protein interactions [<xref ref-type="bibr" rid="B5">5</xref>
] and Gene-Disease-Phenotype relations [<xref ref-type="bibr" rid="B6">6</xref>
]. Given the ubiquity of the ‘Portable Document Format’ (PDF) as a means of distributing scientific publications and since access to information in full-text documents is vital for developing effective text-mining applications [<xref ref-type="bibr" rid="B7">7</xref>
], it is essential to the general BioNLP community that developers of such applications can extract the textual content from PDF files accurately with open-source tools. Many past biomedical text mining studies have used either the abstracts of scientific papers [<xref ref-type="bibr" rid="B8">8</xref>
-<xref ref-type="bibr" rid="B11">11</xref>
] or relatively small collections of full-text articles sampled from the Open Access subset of PubMed Central [<xref ref-type="bibr" rid="B12">12</xref>
]. It is likely that certain content of journals of interest in a particular task is not distributed as a part of the Open Access subset.</p>
<p>A long-standing promise of BioNLP has been to help accelerate the vital process of literature-based biocuration, where published information is carefully translated into the knowledge architecture of biomedical databases, using specific BioNLP tools [<xref ref-type="bibr" rid="B1">1</xref>
,<xref ref-type="bibr" rid="B8">8</xref>
,<xref ref-type="bibr" rid="B13">13</xref>
]. The identification of all papers relevant to the specific database being populated can be considered as a document classification problem [<xref ref-type="bibr" rid="B12">12</xref>
]. Subsequent steps have been cast as Information Extraction (IE) problems that leverage context dependent features [<xref ref-type="bibr" rid="B14">14</xref>
,<xref ref-type="bibr" rid="B15">15</xref>
]. A key consideration is that the well-crafted manual workflows, developed by expert curators in biomedical databases, typically use rules based on context and rhetorical structure-dependent clues found only in the full-text of an article. Thus, it is important for the developers of BioNLP applications to have access to an accurate representation of the full-text of papers derived from PDF files, see [<xref ref-type="bibr" rid="B16">16</xref>
].</p>
<p>Our goal is to provide an open-source software mechanism for automated decomposition and conversion of PDF files of research articles into a simple text format that other NLP groups can easily incorporate into their toolsets. In the most widely used text extraction programs (<italic>e.g.</italic>
, Adobe Acrobat, Grahl PDF Annotator, IntraPDF, PDFTron and PDF2Text), the flow of the main narrative from a file may be broken in mid sentence by errors derived from the reading order of individual text blocks and interruptions such as the inclusion of figure captions, footnotes and headers.</p>
<p>The variation in styles and formats of research articles (even within a single journal) can cause errors in terms of the ordering and splicing of text between pages and blocks. Any software that performs such decomposition and extraction should be adaptable with minimal human effort to new styles and formats. Driven by these needs, our system focuses on providing an open source PDF-to-text conversion capability meeting the following requirements: (1) the extraction mechanism should be able to adapt to single-column, two-column or mixed single and double column layouts, (2) extracted text should be error-free and grouped according to specific section headings used in the paper and (3) formatting artifacts such as, headers, footers, figures, tables and floating boxes (used in author summaries) should not interrupt the narrative-flow within each section. Thus, we have developed a three-step approach for extracting text from PDF files. The first step is the identification of contiguous text blocks. The second step is the classification of these text blocks into rhetorical categories (such as ‘Introduction’, ‘Results’ and ‘Discussion’) using logical rules that are easy to generate as ‘decision tables’ in a spreadsheet. The third step utilizes the classification results to ‘stitch’ appropriate text blocks together for extracting the text, while ignoring blocks that contain formatting embellishments so as to minimize flow-disruption of the extracted text. Our system provides programmatic, open-source access to each one (or to all three) of these capabilities for individual files or large collections of files.</p>
</sec>
<sec><title>Implementation</title>
<sec><title>Step 1 - Detecting contiguous text blocks</title>
<p>The first step in LA-PDFText is to identify contiguous text blocks. In addition to the frequently-used two-column and single-column formats, journals also often use a mixed format where the title, authors, affiliation and abstract span the entire page width (single-column format) while all other sections of the article use a two-column format. We have observed these changes in format by manually inspecting papers from all available issues of the journal <bold>Brain Research</bold>
. We denote these periodic changes in formatting over the lifetime of a given journal as ‘epochs’.</p>
<p>Our approach to detecting contiguous text blocks starts with detecting ‘word-blocks’ (bounding boxes of words). We use the GPL version of JPedal, an open-source Java PDF library to obtain the bounding boxes of each word in the PDF article (<ext-link ext-link-type="uri" xlink:href="http://www.jpedal.org/">http://www.jpedal.org/</ext-link>
). Using this as a starting point, LA-PDFText aggregates word-blocks systematically to build ‘chunk-blocks’ of text while respecting formatting constraints such as two-column <italic>vs.</italic>
one-column formatting. As shown in Figure <xref ref-type="fig" rid="F1">1</xref>
, the algorithm for identifying text blocks, functions by coalescing word blocks together that are close enough (based on the spatial statistics of the words’ layout on the page) and share font characteristics. The algorithm computes proximity automatically on a per-page basis giving it flexibility in dealing with varying formats both within a single page and across pages.</p>
<fig id="F1" position="float"><label>Figure 1 </label>
<caption><p><bold>Block detection per-page parameter computation algorithm.</bold>
The image shows the process by which the north–south and east–west parameters for neighboring block subsumption are computed. For an explanation of the symbols shown in the figure please see Table <xref ref-type="table" rid="T1">1</xref>
.</p>
</caption>
<graphic xlink:href="1751-0473-7-7-1"></graphic>
</fig>
<table-wrap position="float" id="T1"><label>Table 1</label>
<caption><p>Per page word block parameters symbols and their definitions</p>
</caption>
<table frame="hsides" rules="groups" border="1"><colgroup><col align="left"></col>
<col align="left"></col>
</colgroup>
<thead valign="top"><tr><th align="left"><bold>Parameter Symbols</bold>
</th>
<th align="left"><bold>Definitions</bold>
</th>
</tr>
</thead>
<tbody valign="top"><tr><td align="left" valign="bottom"><bold>w</bold>
<sub><bold>height</bold>
</sub>
<hr></hr>
</td>
<td align="left" valign="bottom">Word block height<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom"><bold>δw</bold>
<sub><bold>horizontal</bold>
</sub>
<hr></hr>
</td>
<td align="left" valign="bottom">Horizontal space between words<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom"><bold>δw</bold>
<sub><bold>vertical</bold>
</sub>
<hr></hr>
</td>
<td align="left" valign="bottom">Vertical space between words<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom"><bold>max</bold>
<sup><bold>i</bold>
</sup>
<sub><bold>f</bold>
</sub>
<bold>(δw</bold>
<sub><bold>height</bold>
</sub>
<bold>)</bold>
<hr></hr>
</td>
<td align="left" valign="bottom">Most Popular Word block height in a page <italic>i</italic>
<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom"><bold>max</bold>
<sup><bold>i</bold>
</sup>
<sub><bold>f</bold>
</sub>
<bold>(δw</bold>
<sub><bold>horizontal</bold>
</sub>
<bold>)</bold>
<hr></hr>
</td>
<td align="left" valign="bottom">Most Popular Horizontal space between word blocks in a page <italic>i</italic>
<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom"><bold>max</bold>
<sup><bold>i</bold>
</sup>
<sub><bold>f</bold>
</sub>
<bold>(δw</bold>
<sub><bold>vertical</bold>
</sub>
<bold>)</bold>
<hr></hr>
</td>
<td align="left" valign="bottom">Most Popular Vertical space between word blocks in a page <italic>i</italic>
<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom"><bold>ϕ</bold>
<sub><bold>EW</bold>
</sub>
<bold>= max</bold>
<sup><bold>i</bold>
</sup>
<sub><bold>f</bold>
</sub>
<bold>(δw</bold>
<sub><bold>height</bold>
</sub>
<bold>) + max</bold>
<sup><bold>i</bold>
</sup>
<sub><bold>f</bold>
</sub>
<bold>(δw</bold>
<sub><bold>horizontal</bold>
</sub>
<bold>)</bold>
<hr></hr>
</td>
<td align="left" valign="bottom">east–west word block expansion parameters in page<hr></hr>
</td>
</tr>
<tr><td align="left"><bold>ϕ</bold>
<sub><bold>NS</bold>
</sub>
<bold>= max</bold>
<sup><bold>i</bold>
</sup>
<sub><bold>f</bold>
</sub>
<bold>(δw</bold>
<sub><bold>height</bold>
</sub>
<bold>) + max</bold>
<sup><bold>i</bold>
</sup>
<sub><bold>f</bold>
</sub>
<bold>(δw</bold>
<sub><bold>vertical</bold>
</sub>
<bold>)</bold>
</td>
<td align="left">north–south word block expansion parameters in page</td>
</tr>
</tbody>
</table>
</table-wrap>
<p>Figure <xref ref-type="fig" rid="F1">1</xref>
is an example of how the block detection algorithm decides which word blocks to coalesce. Examples of the parameters δw<sub>horizontal</sub>
, δw<sub>vertical</sub>
and w<sub>height</sub>
are shown in Figure <xref ref-type="fig" rid="F1">1</xref>
. The distributions of these parameter values are calculated for each page and the most popular values for these parameters are chosen from these distributions to calculate ϕ<sub>EW</sub>
and ϕ<sub>NS</sub>
. We intentionally do not use most popular word width since biomedical text uses many long words and the most popular word width will make ϕ<sub>EW</sub>
too large thereby making the block subsumption algorithm too greedy. Consider the words ‘Introduction’ in the section heading, the word ‘antimicrobial’ in the first line of the first column and the word ‘the’ in the third line of the second column in Figure <xref ref-type="fig" rid="F1">1</xref>
. Each word-block (shown in red) is surrounded by an expanded bounding-box (shown using a blue dotted line). All word-blocks (shown in red) that intersect with this expanded bounding-box are treated as words blocks to be merged. The block merge procedure is a greedy algorithm and will combine a section heading, subheading and the sections content into a single block based on the ϕ<sub>EW</sub>
and ϕ<sub>NS</sub>
parameters.</p>
<p>To examine the flexibility of our block detection algorithm, we use a PDF file of the <bold>Nature</bold>
editorial in Volume 466 Issue no. 7303 (Figure <xref ref-type="fig" rid="F2">2</xref>
). This issue contains 3 editorials, and the second page contains part of the second editorial along with the third, separated by a horizontal line. LA-PDFText was able to accurately identify, classify and extract text from the editorial PDF file.</p>
<fig id="F2" position="float"><label>Figure 2 </label>
<caption><p><bold>Flexibility of the block identification algorithm.</bold>
The image shown on left of the figure is taken from page 2, with two distinct articles, of the <bold>Nature</bold>
editorial Volume 466 Issue no. 7303. The image on the right is an example of the debug output generated by LA-PDFText. Our block detection algorithm identifies the text blocks in the right column of the article page as distinct blocks allowing the subsequent block classification step of the system to apply rules that treat these blocks as parts of different articles.</p>
</caption>
<graphic xlink:href="1751-0473-7-7-2"></graphic>
</fig>
</sec>
<sec><title>Step 2 - Classifying text blocks into rhetorical categories</title>
<p>The next phase of LA-PDFText is based on ‘DROOLS’, a business rule management system and an enhanced Rules Engine implementation, ReteOO, based on the Rete algorithm [<xref ref-type="bibr" rid="B17">17</xref>
] tailored for the Java language distributed as part of the open-source JBoss Enterprise Platform (<ext-link ext-link-type="uri" xlink:href="http://labs.jboss.com/portal/jbossrules/">http://labs.jboss.com/portal/jbossrules/</ext-link>
). DROOLS provides a way for the LA-PDFText user to declaratively specify characteristics of a text block that make it a part of a particular section in the paper. We include the rule files for two epochs within the PLoS Biology dataset in both the DROOLS format as well as Microsoft Excel (Additional files <xref ref-type="supplementary-material" rid="S1">1</xref>
<xref ref-type="supplementary-material" rid="S2">2</xref>
and <xref ref-type="supplementary-material" rid="S3">3</xref>
).</p>
</sec>
<sec><title>Step 3 - Stitching classified text blocks together in the correct order</title>
<p>The final goal of LA-PDFText is to accurately extract the text of any given section(s) in the correct sequence. As an implementation of this capability the last component of the LA-PDFText iterates over the classified blocks and stitches the classified blocks together to produce contiguous sections along with section and sub-section headings appropriately demarcated. LA-PDFText provides mechanisms to output the text of these PDF as XML formatted using PubMed Central’s OpenAccess DTD.</p>
</sec>
</sec>
<sec sec-type="results"><title>Results</title>
<p>We have evaluated the three steps of our system independently of each other. In the following sections we will present our evaluation methods for each of the three steps of LA-PDFText and their results.</p>
<sec><title>Step 1 - Detecting contiguous text blocks - evaluation</title>
<p>In order to evaluate the effectiveness of spatial segmentation of each PDF page into text blocks, we manually segment each page in our experimental dataset to produce the ideal segmentation of each paper. We then count the number of edit operations (deleting and adding blocks) required to transform the manually segmented papers into to the segmentation predicted by our software. The ideal segmentation of a paper is one that does not require any deletion, addition or splitting of segments in order to retrieve the text from the segments in the correct order. We use the following guidelines in the manual segmentation process: (1) segments should be created in such a way as to facilitate sequence-preserving text extraction, (2) segments should be rectangular and (3) section headings and sub-headings should be marked as distinct segments from the body of their corresponding sections. Our algorithm creates images of each page of the input PDF showing the word block boundaries and the segment boundaries (Figure <xref ref-type="fig" rid="F2">2</xref>
). To explain the evaluation process further, we present the following sample situations (Table <xref ref-type="table" rid="T2">2</xref>
) that describe block configurations produced by LA-PDFText. In each case, we describe edit operations applied to the manually segmented page and their corresponding cost. The results of this evaluation are presented in Additional file <xref ref-type="supplementary-material" rid="S4">4</xref>
: Tables S4, S5, S6 and S7 under the column titled ‘Spatial Segmentation Score’. In the ideal case a paper segmented into blocks by LA-PDFText should have a spatial segmentation score of zero, indicating that it is perfectly segmented with respect to the manual segmentation.</p>
<table-wrap position="float" id="T2"><label>Table 2</label>
<caption><p>Example scenarios describing conversion operations and their corresponding costs</p>
</caption>
<table frame="hsides" rules="groups" border="1"><colgroup><col align="left"></col>
<col align="left"></col>
<col align="left"></col>
</colgroup>
<thead valign="top"><tr><th align="left"><bold>System Output</bold>
</th>
<th align="left"><bold>Operation in Gold Standard Representation</bold>
</th>
<th align="left"><bold>Cost</bold>
</th>
</tr>
</thead>
<tbody valign="top"><tr><td align="left" valign="bottom">Block is split<hr></hr>
</td>
<td align="left" valign="bottom">Split the gold standard block into the required number<hr></hr>
</td>
<td align="left" valign="bottom">1<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom">Big block is subsuming n small blocks<hr></hr>
</td>
<td align="left" valign="bottom">Delete the involved blocks in the gold standard and add one big block<hr></hr>
</td>
<td align="left" valign="bottom">n + 1<hr></hr>
</td>
</tr>
<tr><td align="left">n block are intersecting</td>
<td align="left">Delete all the blocks in the gold standard, whose area is common with the intersecting blocks, in the system output</td>
<td align="left">Number of blocks deleted from gold standard +1</td>
</tr>
</tbody>
</table>
</table-wrap>
</sec>
<sec><title>Step 2 - Classifying text blocks into rhetorical categories - evaluation</title>
<p>The rule based segment classifier component of our software is instrumented to produce color-coded segments depending upon the type of section to which each segment belongs. This color-coding is used in the manual evaluation to count the number of segments of each section that were correctly classified (true positives; TP), those that were incorrectly classified (false positives; FP) and those that were missed by the rule engine (false negatives; FN). Thus, we can calculate the Precision (P), Recall (R) and F1 metrics to evaluate the classification accuracy using the following metrics:</p>
<p><disp-formula id="bmcM1"><label>(1)</label>
<mml:math id="M1" name="1751-0473-7-7-i1" overflow="scroll"><mml:mrow><mml:mi>P</mml:mi>
<mml:mo>=</mml:mo>
<mml:mfrac><mml:mrow><mml:mi>T</mml:mi>
<mml:mi>P</mml:mi>
</mml:mrow>
<mml:mrow><mml:mi>T</mml:mi>
<mml:mi>P</mml:mi>
<mml:mo>+</mml:mo>
<mml:mi>F</mml:mi>
<mml:mi>P</mml:mi>
</mml:mrow>
</mml:mfrac>
<mml:mspace width="1em"></mml:mspace>
<mml:mi>R</mml:mi>
<mml:mo>=</mml:mo>
<mml:mfrac><mml:mrow><mml:mi>T</mml:mi>
<mml:mi>P</mml:mi>
</mml:mrow>
<mml:mrow><mml:mi>T</mml:mi>
<mml:mi>P</mml:mi>
<mml:mo>+</mml:mo>
<mml:mi>F</mml:mi>
<mml:mi>N</mml:mi>
</mml:mrow>
</mml:mfrac>
<mml:mspace width="1em"></mml:mspace>
<mml:mi>F</mml:mi>
<mml:mn>1</mml:mn>
<mml:mo>=</mml:mo>
<mml:mfrac><mml:mrow><mml:mn>2</mml:mn>
<mml:mo>*</mml:mo>
<mml:mi>P</mml:mi>
<mml:mo>*</mml:mo>
<mml:mi>R</mml:mi>
</mml:mrow>
<mml:mrow><mml:mi>P</mml:mi>
<mml:mo>+</mml:mo>
<mml:mi>R</mml:mi>
</mml:mrow>
</mml:mfrac>
</mml:mrow>
</mml:math>
</disp-formula>
</p>
<p>The results of this manual evaluation are reported in Additional file <xref ref-type="supplementary-material" rid="S4">4</xref>
: Tables S4, S5, S6 and S7 under the column titled ‘Block Classification Performance’. Classification Precision, Recall and F1 scores are averaged across all volumes and presented in Table <xref ref-type="table" rid="T3">3</xref>
in a per-section basis.</p>
<table-wrap position="float" id="T3"><label>Table 3</label>
<caption><p>Per-section Precision (P), Recall(R), and F1 scores for section classification</p>
</caption>
<table frame="hsides" rules="groups" border="1"><colgroup><col align="left"></col>
<col align="center"></col>
<col align="center"></col>
<col align="center"></col>
<col align="center"></col>
</colgroup>
<thead valign="top"><tr><th align="center"><bold>N</bold>
</th>
<th align="center"><bold>Section Parts</bold>
</th>
<th align="center"><bold>P</bold>
</th>
<th align="center"><bold>R</bold>
</th>
<th align="center"><bold>F1</bold>
</th>
</tr>
</thead>
<tbody valign="top"><tr><td align="left" valign="bottom"> Paper Title<hr></hr>
</td>
<td align="center" valign="bottom"> <hr></hr>
</td>
<td align="center" valign="bottom">1.000<hr></hr>
</td>
<td align="center" valign="bottom">0.966<hr></hr>
</td>
<td align="center" valign="bottom">0.983<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom"> Authors<hr></hr>
</td>
<td align="center" valign="bottom"> <hr></hr>
</td>
<td align="center" valign="bottom">0.987<hr></hr>
</td>
<td align="center" valign="bottom">0.906<hr></hr>
</td>
<td align="center" valign="bottom">0.945<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom">Abstract<hr></hr>
</td>
<td align="center" valign="bottom">Heading<hr></hr>
</td>
<td align="center" valign="bottom">1.000<hr></hr>
</td>
<td align="center" valign="bottom">1.000<hr></hr>
</td>
<td align="center" valign="bottom">1.000<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom"> <hr></hr>
</td>
<td align="center" valign="bottom">Body<hr></hr>
</td>
<td align="center" valign="bottom">0.988<hr></hr>
</td>
<td align="center" valign="bottom">0.883<hr></hr>
</td>
<td align="center" valign="bottom">0.933<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom">Introduction<hr></hr>
</td>
<td align="center" valign="bottom">Heading<hr></hr>
</td>
<td align="center" valign="bottom">1.000<hr></hr>
</td>
<td align="center" valign="bottom">0.988<hr></hr>
</td>
<td align="center" valign="bottom">0.994<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom"> <hr></hr>
</td>
<td align="center" valign="bottom">Body<hr></hr>
</td>
<td align="center" valign="bottom">0.876<hr></hr>
</td>
<td align="center" valign="bottom">0.915<hr></hr>
</td>
<td align="center" valign="bottom">0.895<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom">Results<hr></hr>
</td>
<td align="center" valign="bottom">Heading<hr></hr>
</td>
<td align="center" valign="bottom">1.000<hr></hr>
</td>
<td align="center" valign="bottom">1.000<hr></hr>
</td>
<td align="center" valign="bottom">1.000<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom"> <hr></hr>
</td>
<td align="center" valign="bottom">Body<hr></hr>
</td>
<td align="center" valign="bottom">0.948<hr></hr>
</td>
<td align="center" valign="bottom">0.912<hr></hr>
</td>
<td align="center" valign="bottom">0.930<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom"> <hr></hr>
</td>
<td align="center" valign="bottom">Sub-heading<hr></hr>
</td>
<td align="center" valign="bottom">0.947<hr></hr>
</td>
<td align="center" valign="bottom">0.843<hr></hr>
</td>
<td align="center" valign="bottom">0.892<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom">Methods<hr></hr>
</td>
<td align="center" valign="bottom">Heading<hr></hr>
</td>
<td align="center" valign="bottom">1.000<hr></hr>
</td>
<td align="center" valign="bottom">1.000<hr></hr>
</td>
<td align="center" valign="bottom">1.000<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom"> <hr></hr>
</td>
<td align="center" valign="bottom">Body<hr></hr>
</td>
<td align="center" valign="bottom">0.992<hr></hr>
</td>
<td align="center" valign="bottom">0.927<hr></hr>
</td>
<td align="center" valign="bottom">0.958<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom"> <hr></hr>
</td>
<td align="center" valign="bottom">Sub-heading<hr></hr>
</td>
<td align="center" valign="bottom">1.000<hr></hr>
</td>
<td align="center" valign="bottom">0.982<hr></hr>
</td>
<td align="center" valign="bottom">0.991<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom">Discussion<hr></hr>
</td>
<td align="center" valign="bottom">Heading<hr></hr>
</td>
<td align="center" valign="bottom">0.987<hr></hr>
</td>
<td align="center" valign="bottom">1.000<hr></hr>
</td>
<td align="center" valign="bottom">0.993<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom"> <hr></hr>
</td>
<td align="center" valign="bottom">Body<hr></hr>
</td>
<td align="center" valign="bottom">0.946<hr></hr>
</td>
<td align="center" valign="bottom">0.924<hr></hr>
</td>
<td align="center" valign="bottom">0.935<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom"> <hr></hr>
</td>
<td align="center" valign="bottom">Sub-heading<hr></hr>
</td>
<td align="center" valign="bottom">0.917<hr></hr>
</td>
<td align="center" valign="bottom">0.885<hr></hr>
</td>
<td align="center" valign="bottom">0.901<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom"> Figure Legend<hr></hr>
</td>
<td align="center" valign="bottom"> <hr></hr>
</td>
<td align="center" valign="bottom">0.986<hr></hr>
</td>
<td align="center" valign="bottom">0.840<hr></hr>
</td>
<td align="center" valign="bottom">0.907<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom">References<hr></hr>
</td>
<td align="center" valign="bottom">Heading<hr></hr>
</td>
<td align="center" valign="bottom">1.000<hr></hr>
</td>
<td align="center" valign="bottom">0.988<hr></hr>
</td>
<td align="center" valign="bottom">0.994<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom"> <hr></hr>
</td>
<td align="center" valign="bottom">Body<hr></hr>
</td>
<td align="center" valign="bottom">0.532<hr></hr>
</td>
<td align="center" valign="bottom">0.632<hr></hr>
</td>
<td align="center" valign="bottom">0.578<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom">Supporting Information<hr></hr>
</td>
<td align="center" valign="bottom">Heading<hr></hr>
</td>
<td align="center" valign="bottom">0.988<hr></hr>
</td>
<td align="center" valign="bottom">1.000<hr></hr>
</td>
<td align="center" valign="bottom">0.994<hr></hr>
</td>
</tr>
<tr><td align="left" valign="bottom"> <hr></hr>
</td>
<td align="center" valign="bottom">Body<hr></hr>
</td>
<td align="center" valign="bottom">0.946<hr></hr>
</td>
<td align="center" valign="bottom">0.224<hr></hr>
</td>
<td align="center" valign="bottom">0.362<hr></hr>
</td>
</tr>
<tr><td align="left"><bold> Macro Average</bold>
</td>
<td align="center"> </td>
<td align="center"><bold>0.956</bold>
</td>
<td align="center"><bold>0.888</bold>
</td>
<td align="center"><bold>0.910</bold>
</td>
</tr>
</tbody>
</table>
</table-wrap>
</sec>
<sec><title>Step 3 - Stitching classified text blocks together in the correct order - Evaluation</title>
<p>PDF2Text is a widely used approach to extract text from PDF files. However, it is unable to distinguish between formatting embellishments and the main narrative of a scientific article. PDF2Text treats the entire document as one string, introducing errors within individual sentences, at column breaks and page breaks. LA-PDFText classifies each text block and (provided the classification is accurate) stitches text blocks belonging to the same section together, in order to extract contiguous rhetorical sections of the input articles. We have compared the text extraction capabilities of both systems to evaluation step 3 of LA-PDFText. Although PDF2Text is a simpler tool to use, we evaluate LA-PDFText’s text extraction capability against that of PDF2Text to show the benefit of our three-stage approach to text extraction.</p>
<p>Figure <xref ref-type="fig" rid="F3">3</xref>
shows an example of the text extraction produced by PDF2Text where the string “PLoS Biology ∣ <ext-link ext-link-type="uri" xlink:href="http://www.plosbiology.org">http://www.plosbiology.org</ext-link>
1” interrupts the preceding sentence. The interruption is precisely the sort of error that is unacceptable in many applications of BioNLP, especially those contributing to biocuration. Our evaluation therefore seeks to quantitatively capture the notion of ‘flow-disruption’. Our strategy is based on comparing text extracted by PDF2Text and LA-PDFText for a given set of research articles, against the text extracted from the XML representation of that paper within the Open Access Subset. We chose PLoS Biology articles at random from volumes 5, 6, 7, and 8 for this evaluation. These XML files contain the full-text of their corresponding articles, along with the necessary markup that demarcates each section of the paper. The XML does not contain headers and footers present in the original PDF.</p>
<fig id="F3" position="float"><label>Figure 3 </label>
<caption><p><bold>Text Flow Interruptions.</bold>
The image (A) in the figure above is a snippet of text extracted from the corresponding PDF file (shown in image B) by PDF2Text. The red arrows on the extracted text mark a break in text flow generated by PDF2Text owing to its inability to discount formatting embellishments like footers. Our evaluation of text extraction accuracy quantifies the effect of such flow-interruption on the quality of the output text produced by both PDF2Text and LA-PDFText.</p>
</caption>
<graphic xlink:href="1751-0473-7-7-3"></graphic>
</fig>
<p>We use a variant of the Needleman-Wunsch algorithm [<xref ref-type="bibr" rid="B18">18</xref>
] to compute alignment costs for text extracted by both algorithms against text obtained from the Open Access XML for each paper. The Needleman-Wunsch algorithm uses dynamic programming to perform a global alignment on two sequences and using linear gap penalties. Our variant of this algorithm treats the Open Access text as a sequence of sentences and computes the cost of aligning sentences generated by LA-PDFText and PDF2Text with sentences in the Open Access text. The algorithm uses a gap penalty of −10, a mismatch penalty of −1 and a match reward of 5. Computed alignment costs for each paper are normalized by dividing them by the number of sentences in the Open Access version of the text for that paper. The resulting number can be interpreted as the ‘average per-sentence alignment cost’ for a given paper. The difference between normalized costs produced by both methods is plotted in the graph shown in Figure <xref ref-type="fig" rid="F4">4</xref>
. A number greater than zero indicates that LA-PDFText produced a higher alignment score with respect to the Open Access text than PDF2Text for a particular paper. A number less than zero indicates that PDF2Text produced a better alignment score. Figure <xref ref-type="fig" rid="F4">4</xref>
shows that only 7 out of 86 documents extracted by LA-PDFText (shown using +) produce a poorer alignment score with the Open Access text than PDF2Text (shown using -). In other words, in 91% of the cases LA-PDFText outperforms PDF2Text (p < 0.001). It should be noted that the text extracted by LA-PDFText used in this experiment still contain errors introduced due to sections that have not been classified into any rhetorical categories (recall errors). Despite these classification errors LA-PDFText extracts text with fewer flow interruptions resulting in higher accuracy of extracted text than PDF2Text.</p>
<fig id="F4" position="float"><label>Figure 4 </label>
<caption><p><bold>Text Flow Evaluations.</bold>
The graph above shows the relative alignment cost of LA-PDFText and PDF2Text with respect to the gold standard. Each green dot represents the difference between the normalized alignment scores of LA-PDFText and PDF2Text for one paper in the PLoS Biology dataset. + markers show normalized alignment scores produced by LA-PDFText and - markers show normalized alignment scores produced by PDF2text. Results indicated that LA-PDFText extracts text with better alignment scores with respect to the gold standard than PDF2Text for 91% of the documents tested (p < 0.001).</p>
</caption>
<graphic xlink:href="1751-0473-7-7-4"></graphic>
</fig>
</sec>
</sec>
<sec sec-type="discussion"><title>Discussion</title>
<p>LA-PDFText is designed to be a baseline system as a precursor for further improvements to the block detection, classification and text extraction stages. In this section, we discuss the results of each stage of LA-PDFText presenting error analyses and identify proposed future improvements.</p>
<sec><title>Step 1 - Detecting contiguous text blocks</title>
<p>LA-PDFText’s block detection algorithm is fairly accurate (see Spatial Segmentation Score in Additional file <xref ref-type="supplementary-material" rid="S4">4</xref>
: Tables S4, S5, S6 and S7). Over the PLoS Biology dataset, block detection results in alignment scores with mean (μ) = 9.5 and standard deviation (σ) = 5.7. The algorithm depends on the accuracy of JPedal at identifying word blocks. Although it is expected that using a commercial version of JPedal will reduce these scores and improve block detection, we want LA-PDFText to be available for use without the need for users to purchase the commercial version (although we may release a version of our systems that can also work with the commercial version of JPedal).</p>
</sec>
<sec><title>Step 2 - Classifying text blocks into rhetorical categories</title>
<p>We have designed the segment classification component of LA-PDFText using a rule-based approach so as to make the system more flexible and easily adaptable for use with various journal formats. The classification results (Table <xref ref-type="table" rid="T3">3</xref>
) are based on rule files (see Figure <xref ref-type="fig" rid="F5">5</xref>
) that we designed in roughly a single working day. The goal of our project is to provide a PDF-extraction library that can be customized for specific uses by BioNLP developers. Thus, we have provided a mechanism that requires a relatively small time investment from developers to classify PDF-based text blocks with suitable levels of accuracy. The software distribution includes a Microsoft Excel based ‘decision table’ which can be used to fill in values for features of blocks that cause rules to ‘fire’ and generate an appropriate labels for blocks. The ‘decision table’ mechanism will also allow non-programmers to specify rules for block classification.</p>
<fig id="F5" position="float"><label>Figure 5 </label>
<caption><p><bold>Sample Rule File Listing.</bold>
The figure shows examples of DROOLS Rules for block classification. DROOLS files meant for two epochs within the PLoS Biology dataset are available as a part of the software distribution accompanying this paper. They can also be downloaded from <ext-link ext-link-type="uri" xlink:href="http://code.google.com/p/lapdftext/">http://code.google.com/p/lapdftext/</ext-link>
. The two files included are named epoch_7Jun_8.drl and epoch_5_7May.drl and are located in a folder called `rules' in the base directory of the installation. Experiments reported in this paper have been conducted using these rules for the block classification stage. These files are also included as supplementary material for this paper.</p>
</caption>
<graphic xlink:href="1751-0473-7-7-5"></graphic>
</fig>
<p>We have identified specific errors in the rules that were responsible for poor performing categories (Table <xref ref-type="table" rid="T3">3</xref>
). Within PLoS Biology, the classification recall for the section titled ‘Supporting Information’ is only 0.224 (Table <xref ref-type="table" rid="T3">3</xref>
). Close inspection of our dataset reveals that most supporting information sections contain figure legends, which belong to two categories namely ‘Figure Legends’ and ‘Supporting Information’. The system correctly classifies the blocks as figure legends but not as supporting information. Both the precision and recall of the section titled ‘References’ are 0.532 and 0.632 respectively (Table <xref ref-type="table" rid="T3">3</xref>
). We attribute the low score to the fact that the font used in tables in many papers is the same as that used in references. Since our baseline rule-set did not contain a rule to identify tables they get wrongly identified as references resulting in poor recall and precision.</p>
</sec>
<sec><title>Step 3 - Stitching classified text blocks together in the correct order</title>
<p>The quality of text extraction is best determined by the usability of the text by downstream text mining applications. We have presented evaluations that show the ability of LA-PDFText to extract text with fewer flow interruptions than text extracted by PDF2Text. It should be noted that the evaluation of text extraction was done on full text of papers explicitly to contrast LA-PDFText with PDF2Text. LA-PDFText also provides the user with the additional capability to extract text on a per-section basis; a capability that PDF2Text does not support.</p>
</sec>
</sec>
<sec><title>Related work</title>
<p>Since the introduction of Portable Document Format in 1993 and the widespread development of online journals in the late 1990s, many archival documents published earlier have been scanned and converted into PDF. Furthermore, the scientific community and publishers have adopted PDF as the <italic>de facto</italic>
standard format for scientific communication. In this paper we therefore do not focus on the Optical Character Recognition (OCR) problem but instead assume that we are given PDF documents that include the text, fonts, images, and 2D vector graphics. We are primarily concerned with related work in development of PDF extraction systems that support BioNLP work in the academic community.</p>
<p>Discovering the logical structure of documents is a well-studied problem. However most past efforts were aimed a logical-structure discovery [<xref ref-type="bibr" rid="B19">19</xref>
,<xref ref-type="bibr" rid="B20">20</xref>
] and not explicitly aimed at text extraction from PDF documents. Furthermore, these past efforts used OCR to produce images of document pages, which are then segmented and the segments are classified to discover logical structure. Summers et al. present a survey of methods for the document-logical-structure discovery problem [<xref ref-type="bibr" rid="B21">21</xref>
]. While some methods surveyed by the author perform joint segmentation and classification, other methods separate these steps into distinct phases. Certain methods use a multi-level form of bounding boxes as the basis of their joint segmentation and decision-tree based classification [<xref ref-type="bibr" rid="B19">19</xref>
] for logical-structure discovery. All of the above methods are aimed at inducing some hierarchical representation of the document content from document images. The method presented in this paper uses bounding boxes as well but separates the segmentation and classification phases.</p>
<p>One recent effort aimed at recovering the logical structure of the scholarly articles using Nuance OmniPage 16 to identify bounding boxes of words [<xref ref-type="bibr" rid="B22">22</xref>
]. The bounding box information is represented in XML that includes markup indicating each line and paragraph within the input PDF. The words, lines and paragraph information along with font information of each word are used as features to train a Conditional Random Field (CRF) [<xref ref-type="bibr" rid="B23">23</xref>
] model to classify each line into one of 23 predetermined classes corresponding to rhetorical categories. The method proposed in [<xref ref-type="bibr" rid="B22">22</xref>
] relies on a commercial tool; a feature we seek to avoid here. The authors performed tests on two datasets: one comprising 40 scientific papers in the field of computer science and the other from their previous work comprising 211 <bold>Association of Computing Machinery</bold>
(<bold>ACM</bold>
) papers. We downloaded the second dataset<sup>3</sup>
and manually inspected the PDF documents. We observed that formatting across the 211 papers from <bold>ACM</bold>
is fairly regular using a two-column format. In contrast, we have tested LA-PDFText on articles from the journal <bold>Brain Research</bold>
spanning volumes 1 to 1155. Manually we have identified 10 significant formatting changes from 1966 to 2007. In order to deal with all articles within PubMed,<sup>4</sup>
a PDF extraction system will have to deal with these formatting variations. The system developed by [<xref ref-type="bibr" rid="B22">22</xref>
] also produces XML similar to the LA-PDFText system and can therefore produce text on per-section basis. Upon close inspection of their results, we observed that formatting embellishments interrupt the flow of text extracted by their system in much the same way as it is in PDF2Text’s results. We believe that this is due to the fact that their system does not use a rule-based classification of text blocks, and may not be flexible enough to incorporate this change without substantial effort in feature engineering and retraining.</p>
<p>PDF extraction was used in the Mouse Genome Informatics (MGI) system to generate text input for text-mining software <italic>in-situ</italic>
[<xref ref-type="bibr" rid="B16">16</xref>
]. They used a collection of commercial software (IntraPDF, PDFTron and specifically ProMiner) to extract text from PDF files but did not describe the process or outcome in detail, making it difficult to compare with our current work. Another toolset of particular interest is the Utopia documents platform [<xref ref-type="bibr" rid="B24">24</xref>
,<xref ref-type="bibr" rid="B25">25</xref>
]. Utopia uses PDF as the base framework for constructing an entire toolset within the familiar architecture of a paper. As a first step, the Utopia system performs the text extraction process with a high accuracy, but it does so directly within the rubric of the Utopia system. Our system is a library that provides low-level control of multiple components of the text extraction process and is designed specifically for use by other text mining developers.</p>
</sec>
<sec><title>Conclusion & future work</title>
<p>LA-PDFText is built using non-commercial components, making it freely available under the LGPL license. We believe that it is a very useful tool for the BioNLP community owing to its flexibility and adaptability to a variety of journal formats with minimal rule-development effort. We plan to extend this work by extracting text and structure from tables [<xref ref-type="bibr" rid="B26">26</xref>
], graphs, figures [<xref ref-type="bibr" rid="B27">27</xref>
] and citations (C [<xref ref-type="bibr" rid="B28">28</xref>
]). The systems framework is designed in a modular fashion and can incorporate different methods for block detection and block classification. LA-PDFText will be put to immediate use in the development of a variety of biocuration applications. The next version of LA-PDFText will output annotations in compliance with ontologies such as Annotation Ontology [<xref ref-type="bibr" rid="B29">29</xref>
,<xref ref-type="bibr" rid="B30">30</xref>
] and ontologies about bibliographic records, citations, evidence and discourse relationships.</p>
</sec>
<sec><title>Software verification</title>
<p>In addition to open-source software distribution of LA-PDFText, we also provide the data set that was used in the evaluation presented in this paper (see Additional file <xref ref-type="supplementary-material" rid="S4">4</xref>
). During our evaluation process each phase of our systems three-stage process produces intermediate files meant specifically for use by developers to monitor performance. For instance, the block classification phase produces images each page showing color-coded word blocks grouped using chunk block bounding boxes. This has been an invaluable tool for debugging rule files used in the classification process. Further details about verifying our systems output are forthcoming at the project page listed below. Our code contains unit tests that show how to programmatically invoke our system in all its modes of operation. We invite the reader to download the data set from the location indicated in the supplemental file and reconstruct our evaluation.</p>
</sec>
<sec><title>Availability and requirements</title>
<p>Project name: LA-PDFText – Layout-Aware Text Extraction from Full-text PDF of Scientific Articles</p>
<p>Project home page: <ext-link ext-link-type="uri" xlink:href="http://code.google.com/p/lapdftext/">http://code.google.com/p/lapdftext/</ext-link>
</p>
<p>Current Version: 1.7</p>
<p>Operating system: MacOSX 10.6.7, Linux and Windows XP</p>
<p>Programming language: Java 1.6</p>
<p>Other requirements: none.</p>
<p>License: GNU General Public License</p>
</sec>
<sec><title><bold>Endnotes</bold>
</title>
<p><sup>1</sup>
Average taken over all class labels in the section classification task</p>
<p><sup>2</sup>
<ext-link ext-link-type="uri" xlink:href="http://download.cnet.com/BatchConvert-PDF2Text/3000-2248_4-75147475.html">http://download.cnet.com/BatchConvert-PDF2Text/3000-2248_4-75147475.html</ext-link>
</p>
<p><sup>3</sup>
<ext-link ext-link-type="uri" xlink:href="http://wing.comp.nus.edu.sg/downloads/keyphraseCorpus/NUSkeyphraseCorpus.zip">http://wing.comp.nus.edu.sg/downloads/keyphraseCorpus/NUSkeyphraseCorpus.zip</ext-link>
</p>
<p><sup>4</sup>
<ext-link ext-link-type="uri" xlink:href="http://www.ncbi.nlm.nih.gov/pubmed/">http://www.ncbi.nlm.nih.gov/pubmed/</ext-link>
</p>
<p><sup>5</sup>
<ext-link ext-link-type="uri" xlink:href="https://wiki.birncommunity.org/display/NEWBIRNCC/SciKnowMine/">https://wiki.birncommunity.org/display/NEWBIRNCC/SciKnowMine/</ext-link>
</p>
</sec>
<sec><title>Competing interests</title>
<p>The authors declare that they have no competing interests.</p>
</sec>
<sec><title>Authors’ contributions</title>
<p>GAPCB formulated the idea behind LA-PDFText. CR, AP & GAPCB designed and created LA-PDFText. Authors CR and AP re-engineered and modularized the block detection algorithm. AP implemented the rule-based classification using the latest version of DROOLS. CR conducted the manual evaluation of the block detection and the block classification. CR designed the text accuracy evaluation scheme. Authors GAPCB and EH advised on the evaluation methodology. CR implemented and engineered the text evaluation. CR and GAPCB wrote the paper. All authors read and approved the final manuscript.</p>
</sec>
<sec sec-type="supplementary-material"><title>Supplementary Material</title>
<supplementary-material content-type="local-data" id="S1"><caption><title>Additional file 1 </title>
<p>Sample block classification rule file ‘epoch_5_7May.drl’. This file contains the rules for block classification for PLoS Biology articles in issue 5 to the May articles in issue 7 in DROOLS format. This rule-file can be used in conjunction with the LA-PDFText application available at <ext-link ext-link-type="uri" xlink:href="http://code.google.com/p/lapdftext/">http://code.google.com/p/lapdftext/</ext-link>
</p>
</caption>
<media xlink:href="1751-0473-7-7-S1.drl" mimetype="text" mime-subtype="plain"><caption><p>click here for file</p>
</caption>
</media>
</supplementary-material>
<supplementary-material content-type="local-data" id="S2"><caption><title>Additional file 2 </title>
<p>Sample block classification rule file ‘epoch_7Jun_8.drl’. This file contains the rules for block classification for PLoS Biology articles in issue 7 from June to those in issue 8 in DROOLS format. This rule-file can be used in conjunction with the LA-PDFText application available at <ext-link ext-link-type="uri" xlink:href="http://code.google.com/p/lapdftext/">http://code.google.com/p/lapdftext/</ext-link>
</p>
</caption>
<media xlink:href="1751-0473-7-7-S2.drl" mimetype="text" mime-subtype="plain"><caption><p>click here for file</p>
</caption>
</media>
</supplementary-material>
<supplementary-material content-type="local-data" id="S3"><caption><title>Additional file 3 </title>
<p>Sample block classification rule file ‘epoch_7Jun_8.csv’. This file contains the rules for block classification for PLoS Biology articles in issue 7 from June to those in issue 8 in CSV format. This rule-file can be used in conjunction with the LA-PDFText application available at <ext-link ext-link-type="uri" xlink:href="http://code.google.com/p/lapdftext/">http://code.google.com/p/lapdftext/</ext-link>
</p>
</caption>
<media xlink:href="1751-0473-7-7-S3.csv" mimetype="text" mime-subtype="plain"><caption><p>click here for file</p>
</caption>
</media>
</supplementary-material>
<supplementary-material content-type="local-data" id="S4"><caption><title>Additional file 4 </title>
<p>Contains supplemental Table 4, 5, 6 and 7</p>
</caption>
<media xlink:href="1751-0473-7-7-S4.doc" mimetype="application" mime-subtype="msword"><caption><p>Click here for file</p>
</caption>
</media>
</supplementary-material>
</sec>
</body>
<back><sec><title>Acknowledgements</title>
<p>This research is funded in part by:</p>
<p>· U.S. National Science Foundation under the SciKnowMine project<sup>5</sup>
(grant #0849977)</p>
<p>· NIGMS under the BioScholar project (NIGMS: RO1-GM083871)</p>
<p>· NIH under the NeuArt project (NIH: 1R01MH079068-01A2)</p>
<p>· BIRN project (U24 RR025736-01).</p>
<p>We wish to acknowledge Marcelo Tallis and Thomas Russ for the discussions regarding evaluations. We would also like to acknowledge the contributions of Mark Shirley in helping with the development of early proof-of-concept prototypes. The authors would like to specially thank Dr. Drashti Dave for her help in reviewing the manuscript.</p>
</sec>
<ref-list><ref id="B1"><mixed-citation publication-type="journal"><name><surname>Rebholz-Schuhmann</surname>
<given-names>D</given-names>
</name>
<name><surname>Kirsch</surname>
<given-names>H</given-names>
</name>
<etal></etal>
<article-title>Facts from text–is text mining ready to deliver?</article-title>
<source>PLoS Biol</source>
<year>2005</year>
<volume>3</volume>
<issue>2</issue>
<fpage>e65</fpage>
<pub-id pub-id-type="doi">10.1371/journal.pbio.0030065</pub-id>
<pub-id pub-id-type="pmid">15719064</pub-id>
</mixed-citation>
</ref>
<ref id="B2"><mixed-citation publication-type="journal"><name><surname>Altman</surname>
<given-names>RB</given-names>
</name>
<name><surname>Bergman</surname>
<given-names>CM</given-names>
</name>
<etal></etal>
<article-title>Text mining for biology–the way forward: opinions from leading scientists</article-title>
<source>Genome Biol</source>
<year>2008</year>
<volume>9</volume>
<issue>Suppl 2</issue>
<fpage>S7</fpage>
<pub-id pub-id-type="doi">10.1186/gb-2008-9-s2-s7</pub-id>
<pub-id pub-id-type="pmid">18834498</pub-id>
</mixed-citation>
</ref>
<ref id="B3"><mixed-citation publication-type="book"><name><surname>Settles</surname>
<given-names>B</given-names>
</name>
<source>Biomedical named entity recognition using conditional random fields and rich feature sets.Proceedings of the International Joint Workshop on Natural Language Processing in Biomedicine and its Applications</source>
<year>2004</year>
<publisher-name>Association for Computational Linguistics, Geneva</publisher-name>
<fpage>104</fpage>
<lpage>107</lpage>
</mixed-citation>
</ref>
<ref id="B4"><mixed-citation publication-type="book"><name><surname>Rosario</surname>
<given-names>B</given-names>
</name>
<name><surname>Hearst</surname>
<given-names>MA</given-names>
</name>
<article-title>Classifying semantic relations in bioscience texts</article-title>
<source>Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics</source>
<year>2004</year>
<publisher-name>Association for Computational Linguistics, Barcelona</publisher-name>
<fpage>430</fpage>
</mixed-citation>
</ref>
<ref id="B5"><mixed-citation publication-type="journal"><name><surname>Krallinger</surname>
<given-names>M</given-names>
</name>
<name><surname>Vazquez</surname>
<given-names>M</given-names>
</name>
<etal></etal>
<article-title>The Protein-Protein Interaction tasks of BioCreative III: classification/ranking of articles and linking bio-ontology concepts to full text</article-title>
<source>BMC Bioinformatics</source>
<year>2011</year>
<volume>12</volume>
<issue>Suppl 8</issue>
<fpage>S3</fpage>
<pub-id pub-id-type="doi">10.1186/1471-2105-12-S8-S3</pub-id>
<pub-id pub-id-type="pmid">22151929</pub-id>
</mixed-citation>
</ref>
<ref id="B6"><mixed-citation publication-type="journal"><name><surname>Chun</surname>
<given-names>HW</given-names>
</name>
<name><surname>Tsuruoka</surname>
<given-names>Y</given-names>
</name>
<name><surname>Kim</surname>
<given-names>JD</given-names>
</name>
<name><surname>Shiba</surname>
<given-names>R</given-names>
</name>
<name><surname>Nagata</surname>
<given-names>N</given-names>
</name>
<name><surname>Hishiki</surname>
<given-names>T</given-names>
</name>
<name><surname>Tsujii</surname>
<given-names>J</given-names>
</name>
<article-title>Extraction of gene-disease relations from Medline using domain dictionaries and machine learning</article-title>
<source>Pac Symp Biocomput</source>
<year>2006</year>
<volume>11</volume>
<fpage>4</fpage>
<lpage>15</lpage>
<pub-id pub-id-type="pmid">17094223</pub-id>
</mixed-citation>
</ref>
<ref id="B7"><mixed-citation publication-type="journal"><name><surname>Cohen</surname>
<given-names>KB</given-names>
</name>
<name><surname>Johnson</surname>
<given-names>HL</given-names>
</name>
<etal></etal>
<article-title>The structural and content aspects of abstracts versus bodies of full text journal articles are different</article-title>
<source>BMC Bioinformatics</source>
<year>2010</year>
<volume>11</volume>
<fpage>492</fpage>
<pub-id pub-id-type="doi">10.1186/1471-2105-11-492</pub-id>
<pub-id pub-id-type="pmid">20920264</pub-id>
</mixed-citation>
</ref>
<ref id="B8"><mixed-citation publication-type="journal"><name><surname>Alex</surname>
<given-names>B</given-names>
</name>
<name><surname>Grover</surname>
<given-names>C</given-names>
</name>
<etal></etal>
<article-title>Assisted curation: does text mining really help?</article-title>
<source>Pac Symp Biocomput</source>
<year>2008</year>
<volume>567</volume>
<fpage>556</fpage>
<lpage>567</lpage>
<pub-id pub-id-type="pmid">18229715</pub-id>
</mixed-citation>
</ref>
<ref id="B9"><mixed-citation publication-type="book"><name><surname>Ramakrishnan</surname>
<given-names>C</given-names>
</name>
<name><surname>Mendes</surname>
<given-names>PN</given-names>
</name>
<etal></etal>
<article-title>Joint Extraction of Compound Entities and Relationships from Biomedical Literature</article-title>
<source>Proceedings of the 2008 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology - Volume 01</source>
<year>2008</year>
<publisher-name>IEEE Computer Society, Sydney</publisher-name>
<fpage>398</fpage>
<lpage>401</lpage>
</mixed-citation>
</ref>
<ref id="B10"><mixed-citation publication-type="book"><name><surname>Ramakrishnan</surname>
<given-names>C</given-names>
</name>
<name><surname>Mendes</surname>
<given-names>PN</given-names>
</name>
<etal></etal>
<article-title>Unsupervised Discovery of Compound Entities for Relationship Extraction</article-title>
<source>Proceedings of the 16th international conference on Knowledge Engineering: Practice and Patterns</source>
<year>2008</year>
<publisher-name>Springer-Verlag, Acitrezza</publisher-name>
<fpage>146</fpage>
<lpage>155</lpage>
</mixed-citation>
</ref>
<ref id="B11"><mixed-citation publication-type="journal"><name><surname>Roy</surname>
<given-names>S</given-names>
</name>
<name><surname>Heinrich</surname>
<given-names>K</given-names>
</name>
<etal></etal>
<article-title>Latent Semantic Indexing of PubMed abstracts for identification of transcription factor candidates from microarray derived gene sets</article-title>
<source>BMC Bioinformatics</source>
<year>2011</year>
<volume>12</volume>
<issue>Suppl 10</issue>
<fpage>S19</fpage>
<pub-id pub-id-type="doi">10.1186/1471-2105-12-S10-S19</pub-id>
<pub-id pub-id-type="pmid">22165960</pub-id>
</mixed-citation>
</ref>
<ref id="B12"><mixed-citation publication-type="journal"><name><surname>Cohen</surname>
<given-names>AM</given-names>
</name>
<name><surname>Hersh</surname>
<given-names>WR</given-names>
</name>
<article-title>The TREC 2004 genomics track categorization task: classifying full text biomedical documents</article-title>
<source>J Biomed Discov Collab</source>
<year>2006</year>
<volume>1</volume>
<fpage>4</fpage>
<pub-id pub-id-type="doi">10.1186/1747-5333-1-4</pub-id>
<pub-id pub-id-type="pmid">16722582</pub-id>
</mixed-citation>
</ref>
<ref id="B13"><mixed-citation publication-type="journal"><name><surname>Bourne</surname>
<given-names>P</given-names>
</name>
<name><surname>McEntyre</surname>
<given-names>J</given-names>
</name>
<article-title>Biocurators: contributors to the world of science</article-title>
<source>PLoS Comput Biol</source>
<year>2006</year>
<volume>2</volume>
<issue>10</issue>
<fpage>e142</fpage>
<pub-id pub-id-type="doi">10.1371/journal.pcbi.0020142</pub-id>
<pub-id pub-id-type="pmid">17411327</pub-id>
</mixed-citation>
</ref>
<ref id="B14"><mixed-citation publication-type="journal"><name><surname>Krallinger</surname>
<given-names>M</given-names>
</name>
<name><surname>Morgan</surname>
<given-names>A</given-names>
</name>
<etal></etal>
<article-title>Evaluation of text-mining systems for biology: overview of the Second BioCreative community challenge</article-title>
<source>Genome Biol</source>
<year>2008</year>
<volume>9</volume>
<issue>Suppl 2</issue>
<fpage>S1</fpage>
<comment>Epub 2008 Sep 1</comment>
<pub-id pub-id-type="doi">10.1186/gb-2008-9-s2-s1</pub-id>
<pub-id pub-id-type="pmid">18834487</pub-id>
</mixed-citation>
</ref>
<ref id="B15"><mixed-citation publication-type="journal"><name><surname>Morgan</surname>
<given-names>AA</given-names>
</name>
<name><surname>Lu</surname>
<given-names>Z</given-names>
</name>
<etal></etal>
<article-title>Overview of BioCreative II gene normalization</article-title>
<source>Genome Biol</source>
<year>2008</year>
<volume>9</volume>
<issue>Suppl 2</issue>
<fpage>S3</fpage>
<pub-id pub-id-type="doi">10.1186/gb-2008-9-s2-s3</pub-id>
<pub-id pub-id-type="pmid">18834494</pub-id>
</mixed-citation>
</ref>
<ref id="B16"><mixed-citation publication-type="journal"><name><surname>Dowell</surname>
<given-names>KG</given-names>
</name>
<name><surname>McAndrews-Hill</surname>
<given-names>MS</given-names>
</name>
<etal></etal>
<article-title>Integrating text mining into the MGI biocuration workflow</article-title>
<source>Database</source>
<year>2009</year>
<volume>2009</volume>
<fpage>11</fpage>
</mixed-citation>
</ref>
<ref id="B17"><mixed-citation publication-type="journal"><name><surname>Forgy</surname>
<given-names>CL</given-names>
</name>
<article-title>Rete: a fast algorithm for the many pattern/many object pattern match problem</article-title>
<source>Artif Intell</source>
<year>1982</year>
<volume>19</volume>
<issue>1</issue>
<fpage>17</fpage>
<lpage>37</lpage>
<pub-id pub-id-type="doi">10.1016/0004-3702(82)90020-0</pub-id>
</mixed-citation>
</ref>
<ref id="B18"><mixed-citation publication-type="journal"><name><surname>Needleman</surname>
<given-names>SB</given-names>
</name>
<name><surname>Wunsch</surname>
<given-names>CD</given-names>
</name>
<article-title>A general method applicable to the search for similarities in the amino acid sequence of two proteins</article-title>
<source>J Mol Biol</source>
<year>1970</year>
<volume>48</volume>
<issue>3</issue>
<fpage>443</fpage>
<lpage>453</lpage>
<pub-id pub-id-type="doi">10.1016/0022-2836(70)90057-4</pub-id>
<pub-id pub-id-type="pmid">5420325</pub-id>
</mixed-citation>
</ref>
<ref id="B19"><mixed-citation publication-type="book"><name><surname>Dengel</surname>
<given-names>A</given-names>
</name>
<name><surname>Dubiel</surname>
<given-names>F</given-names>
</name>
<article-title>Clustering and classification of document structure-a machine learning approach</article-title>
<source>Proceedings of the Third International Conference on Document Analysis and Recognition (Volume 2) - Volume 2</source>
<year>1995</year>
<publisher-name>IEEE Computer Society, Washington</publisher-name>
<fpage>587</fpage>
</mixed-citation>
</ref>
<ref id="B20"><mixed-citation publication-type="book"><name><surname>Esposito</surname>
<given-names>F</given-names>
</name>
<name><surname>Malerba</surname>
<given-names>D</given-names>
</name>
<etal></etal>
<article-title>A Knowledge-Based Approach to the Layout Analysis</article-title>
<source>the Proceedings of the Third International Conference on Document Analysis and Recognition</source>
<year>1995</year>
<publisher-name>Society Press, Montreal</publisher-name>
<fpage>466</fpage>
<lpage>471</lpage>
</mixed-citation>
</ref>
<ref id="B21"><mixed-citation publication-type="book"><name><surname>Summers</surname>
<given-names>Kristen</given-names>
</name>
<source>Automatic Discovery of Logical Document Structure. Technical Report</source>
<year>1998</year>
<publisher-name>Cornell University, Ithaca</publisher-name>
</mixed-citation>
</ref>
<ref id="B22"><mixed-citation publication-type="journal"><name><surname>Luong</surname>
<given-names>M-T</given-names>
</name>
<name><surname>Nguyen</surname>
<given-names>TD</given-names>
</name>
<name><surname>Kan</surname>
<given-names>M-Y</given-names>
</name>
<article-title>Logical structure recovery in scholarly articles with rich document features</article-title>
<source>International Journal of Digital Library Systems (IJDLS)</source>
<year>2011</year>
<volume>1</volume>
<issue>4</issue>
<fpage>1</fpage>
<lpage>23</lpage>
</mixed-citation>
</ref>
<ref id="B23"><mixed-citation publication-type="book"><name><surname>Lafferty</surname>
<given-names>JD</given-names>
</name>
<name><surname>McCallum</surname>
<given-names>A</given-names>
</name>
<etal></etal>
<article-title>Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data</article-title>
<source>Proceedings of the Eighteenth International Conference on Machine Learning</source>
<year>2001</year>
<publisher-name>Morgan Kaufmann Publishers Inc, San Francisco</publisher-name>
<fpage>282</fpage>
<lpage>289</lpage>
</mixed-citation>
</ref>
<ref id="B24"><mixed-citation publication-type="journal"><name><surname>Attwood</surname>
<given-names>TK</given-names>
</name>
<name><surname>Kell</surname>
<given-names>DB</given-names>
</name>
<etal></etal>
<article-title>Utopia documents: linking scholarly literature with research data</article-title>
<source>Bioinformatics</source>
<year>2010</year>
<volume>26</volume>
<issue>18</issue>
<fpage>i568</fpage>
<lpage>i574</lpage>
<pub-id pub-id-type="doi">10.1093/bioinformatics/btq383</pub-id>
<pub-id pub-id-type="pmid">20823323</pub-id>
</mixed-citation>
</ref>
<ref id="B25"><mixed-citation publication-type="journal"><name><surname>Vroling</surname>
<given-names>B</given-names>
</name>
<name><surname>Thorne</surname>
<given-names>D</given-names>
</name>
<etal></etal>
<article-title>Integrating GPCR-specific information with full text articles</article-title>
<source>BMC Bioinformatics</source>
<year>2011</year>
<volume>12</volume>
<fpage>362</fpage>
<pub-id pub-id-type="doi">10.1186/1471-2105-12-362</pub-id>
<pub-id pub-id-type="pmid">21910883</pub-id>
</mixed-citation>
</ref>
<ref id="B26"><mixed-citation publication-type="book"><name><surname>Liu</surname>
<given-names>Y</given-names>
</name>
<name><surname>Mitra</surname>
<given-names>P</given-names>
</name>
<etal></etal>
<article-title>Identifying table boundaries in digital documents via sparse line detection</article-title>
<source>Proceeding of the 17th ACM conference on Information and knowledge management</source>
<year>2008</year>
<publisher-name>ACM, Napa Valley</publisher-name>
<fpage>1311</fpage>
<lpage>1320</lpage>
</mixed-citation>
</ref>
<ref id="B27"><mixed-citation publication-type="book"><name><surname>Murphy</surname>
<given-names>RF</given-names>
</name>
<name><surname>Velliste</surname>
<given-names>M</given-names>
</name>
<etal></etal>
<article-title>Searching Online Journals for Fluorescence Microscope Images Depicting Protein Subcellular Location Patterns</article-title>
<source>Proceedings of the 2nd IEEE International Symposium on Bioinformatics and Bioengineering</source>
<year>2001</year>
<publisher-name>IEEE Computer Society, Washington</publisher-name>
<fpage>119</fpage>
</mixed-citation>
</ref>
<ref id="B28"><mixed-citation publication-type="book"><name><surname>Lee Giles</surname>
<given-names>C</given-names>
</name>
<name><surname>Councill</surname>
<given-names>I</given-names>
</name>
<name><surname>Kan</surname>
<given-names>M-Y</given-names>
</name>
<article-title>ParsCit: an Open-source CRF Reference String Parsing Package</article-title>
<source>Proceedings of the Sixth International Language Resources and Evaluation (LREC'08)</source>
<year>2008</year>
<publisher-name>European Language Resources Association (ELRA), Marrakech</publisher-name>
</mixed-citation>
</ref>
<ref id="B29"><mixed-citation publication-type="other"><name><surname>Ciccarese</surname>
<given-names>P</given-names>
</name>
<name><surname>Attwood</surname>
<given-names>T</given-names>
</name>
<etal></etal>
<article-title>A Round-Trip to the Annotation Store: Open, Transferable Semantic Annotation of Biomedical Publications</article-title>
<source>Paper at Workshop Beyond the PDF</source>
<year>2011</year>
</mixed-citation>
</ref>
<ref id="B30"><mixed-citation publication-type="journal"><name><surname>Ciccarese</surname>
<given-names>P</given-names>
</name>
<name><surname>Ocana</surname>
<given-names>M</given-names>
</name>
<name><surname>Garcia Castro</surname>
<given-names>LJ</given-names>
</name>
<name><surname>Das</surname>
<given-names>S</given-names>
</name>
<name><surname>Clark</surname>
<given-names>T</given-names>
</name>
<article-title>An open annotation ontology for science on web 3.0</article-title>
<source>J Biomed Semantics</source>
<year>2011 May 17</year>
<volume>2</volume>
<issue>Suppl 2</issue>
<fpage>S4</fpage>
<pub-id pub-id-type="doi">10.1186/2041-1480-2-S2-S4</pub-id>
<pub-id pub-id-type="pmid">21624159</pub-id>
</mixed-citation>
</ref>
</ref-list>
</back>
</pmc>
<affiliations><list><country><li>États-Unis</li>
</country>
<region><li>Californie</li>
</region>
<settlement><li>Los Angeles</li>
</settlement>
<orgName><li>Université de Californie du Sud</li>
</orgName>
</list>
<tree><country name="États-Unis"><region name="Californie"><name sortKey="Ramakrishnan, Cartic" sort="Ramakrishnan, Cartic" uniqKey="Ramakrishnan C" first="Cartic" last="Ramakrishnan">Cartic Ramakrishnan</name>
</region>
<name sortKey="Burns, Gully Apc" sort="Burns, Gully Apc" uniqKey="Burns G" first="Gully Apc" last="Burns">Gully Apc Burns</name>
<name sortKey="Hovy, Eduard" sort="Hovy, Eduard" uniqKey="Hovy E" first="Eduard" last="Hovy">Eduard Hovy</name>
<name sortKey="Patnia, Abhishek" sort="Patnia, Abhishek" uniqKey="Patnia A" first="Abhishek" last="Patnia">Abhishek Patnia</name>
</country>
</tree>
</affiliations>
</record>
Pour manipuler ce document sous Unix (Dilib)
EXPLOR_STEP=$WICRI_ROOT/Ticri/CIDE/explor/OcrV1/Data/Ncbi/Merge
HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 000131 | SxmlIndent | more
Ou
HfdSelect -h $EXPLOR_AREA/Data/Ncbi/Merge/biblio.hfd -nk 000131 | SxmlIndent | more
Pour mettre un lien sur cette page dans le réseau Wicri
{{Explor lien |wiki= Ticri/CIDE |area= OcrV1 |flux= Ncbi |étape= Merge |type= RBID |clé= PMC:3441580 |texte= Layout-aware text extraction from full-text PDF of scientific articles }}
Pour générer des pages wiki
HfdIndexSelect -h $EXPLOR_AREA/Data/Ncbi/Merge/RBID.i -Sk "pubmed:22640904" \ | HfdSelect -Kh $EXPLOR_AREA/Data/Ncbi/Merge/biblio.hfd \ | NlmPubMed2Wicri -a OcrV1
![]() | This area was generated with Dilib version V0.6.32. | ![]() |