Links to Exploration step
Le document en format XML
<record><TEI><teiHeader><fileDesc><titleStmt><title xml:lang="en">A study on the application of topic models to motif finding algorithms</title><author><name sortKey="Basha Gutierrez, Josep" sort="Basha Gutierrez, Josep" uniqKey="Basha Gutierrez J" first="Josep" last="Basha Gutierrez">Josep Basha Gutierrez</name><affiliation><nlm:aff id="Aff1"><institution-wrap><institution-id institution-id-type="ISNI">0000 0001 2151 536X</institution-id><institution-id institution-id-type="GRID">grid.26999.3d</institution-id><institution>Department of Computational Biology and Medical Sciences, Graduate School of Frontier Sciences,</institution><institution>The University of Tokyo,</institution></institution-wrap>277-8561 Chiba, Japan</nlm:aff></affiliation><affiliation><nlm:aff id="Aff2"><institution-wrap><institution-id institution-id-type="ISNI">0000 0001 2151 536X</institution-id><institution-id institution-id-type="GRID">grid.26999.3d</institution-id><institution>Human Genome Center, The Institute of Medical Science,</institution><institution>The University of Tokyo,</institution></institution-wrap>4-6-1 Shirokane-dai, Minato-ku, 108-8639 Tokyo, Japan</nlm:aff></affiliation></author><author><name sortKey="Nakai, Kenta" sort="Nakai, Kenta" uniqKey="Nakai K" first="Kenta" last="Nakai">Kenta Nakai</name><affiliation><nlm:aff id="Aff1"><institution-wrap><institution-id institution-id-type="ISNI">0000 0001 2151 536X</institution-id><institution-id institution-id-type="GRID">grid.26999.3d</institution-id><institution>Department of Computational Biology and Medical Sciences, Graduate School of Frontier Sciences,</institution><institution>The University of Tokyo,</institution></institution-wrap>277-8561 Chiba, Japan</nlm:aff></affiliation><affiliation><nlm:aff id="Aff2"><institution-wrap><institution-id institution-id-type="ISNI">0000 0001 2151 536X</institution-id><institution-id institution-id-type="GRID">grid.26999.3d</institution-id><institution>Human Genome Center, The Institute of Medical Science,</institution><institution>The University of Tokyo,</institution></institution-wrap>4-6-1 Shirokane-dai, Minato-ku, 108-8639 Tokyo, Japan</nlm:aff></affiliation></author></titleStmt><publicationStmt><idno type="wicri:source">PMC</idno><idno type="pmid">28155646</idno><idno type="pmc">5259985</idno><idno type="url">http://www.ncbi.nlm.nih.gov/pmc/articles/PMC5259985</idno><idno type="RBID">PMC:5259985</idno><idno type="doi">10.1186/s12859-016-1364-3</idno><date when="2016">2016</date><idno type="wicri:Area/Pmc/Corpus">000263</idno><idno type="wicri:explorRef" wicri:stream="Pmc" wicri:step="Corpus" wicri:corpus="PMC">000263</idno></publicationStmt><sourceDesc><biblStruct><analytic><title xml:lang="en" level="a" type="main">A study on the application of topic models to motif finding algorithms</title><author><name sortKey="Basha Gutierrez, Josep" sort="Basha Gutierrez, Josep" uniqKey="Basha Gutierrez J" first="Josep" last="Basha Gutierrez">Josep Basha Gutierrez</name><affiliation><nlm:aff id="Aff1"><institution-wrap><institution-id institution-id-type="ISNI">0000 0001 2151 536X</institution-id><institution-id institution-id-type="GRID">grid.26999.3d</institution-id><institution>Department of Computational Biology and Medical Sciences, Graduate School of Frontier Sciences,</institution><institution>The University of Tokyo,</institution></institution-wrap>277-8561 Chiba, Japan</nlm:aff></affiliation><affiliation><nlm:aff id="Aff2"><institution-wrap><institution-id institution-id-type="ISNI">0000 0001 2151 536X</institution-id><institution-id institution-id-type="GRID">grid.26999.3d</institution-id><institution>Human Genome Center, The Institute of Medical Science,</institution><institution>The University of Tokyo,</institution></institution-wrap>4-6-1 Shirokane-dai, Minato-ku, 108-8639 Tokyo, Japan</nlm:aff></affiliation></author><author><name sortKey="Nakai, Kenta" sort="Nakai, Kenta" uniqKey="Nakai K" first="Kenta" last="Nakai">Kenta Nakai</name><affiliation><nlm:aff id="Aff1"><institution-wrap><institution-id institution-id-type="ISNI">0000 0001 2151 536X</institution-id><institution-id institution-id-type="GRID">grid.26999.3d</institution-id><institution>Department of Computational Biology and Medical Sciences, Graduate School of Frontier Sciences,</institution><institution>The University of Tokyo,</institution></institution-wrap>277-8561 Chiba, Japan</nlm:aff></affiliation><affiliation><nlm:aff id="Aff2"><institution-wrap><institution-id institution-id-type="ISNI">0000 0001 2151 536X</institution-id><institution-id institution-id-type="GRID">grid.26999.3d</institution-id><institution>Human Genome Center, The Institute of Medical Science,</institution><institution>The University of Tokyo,</institution></institution-wrap>4-6-1 Shirokane-dai, Minato-ku, 108-8639 Tokyo, Japan</nlm:aff></affiliation></author></analytic><series><title level="j">BMC Bioinformatics</title><idno type="eISSN">1471-2105</idno><imprint><date when="2016">2016</date></imprint></series></biblStruct></sourceDesc></fileDesc><profileDesc><textClass></textClass></profileDesc></teiHeader><front><div type="abstract" xml:lang="en"><sec><title>Background</title><p>Topic models are statistical algorithms which try to discover the structure of a set of documents according to the abstract topics contained in them. Here we try to apply this approach to the discovery of the structure of the transcription factor binding sites (TFBS) contained in a set of biological sequences, which is a fundamental problem in molecular biology research for the understanding of transcriptional regulation. Here we present two methods that make use of topic models for motif finding. First, we developed an algorithm in which first a set of biological sequences are treated as text documents, and the k-mers contained in them as words, to then build a correlated topic model (CTM) and iteratively reduce its perplexity. We also used the perplexity measurement of CTMs to improve our previous algorithm based on a genetic algorithm and several statistical coefficients.</p></sec><sec><title>Results</title><p>The algorithms were tested with 56 data sets from four different species and compared to 14 other methods by the use of several coefficients both at nucleotide and site level. The results of our first approach showed a performance comparable to the other methods studied, especially at site level and in sensitivity scores, in which it scored better than any of the 14 existing tools. In the case of our previous algorithm, the new approach with the addition of the perplexity measurement clearly outperformed all of the other methods in sensitivity, both at nucleotide and site level, and in overall performance at site level.</p></sec><sec><title>Conclusions</title><p>The statistics obtained show that the performance of a motif finding method based on the use of a CTM is satisfying enough to conclude that the application of topic models is a valid method for developing motif finding algorithms. Moreover, the addition of topic models to a previously developed method dramatically increased its performance, suggesting that this combined algorithm can be a useful tool to successfully predict motifs in different kinds of sets of DNA sequences.</p></sec><sec><title>Electronic supplementary material</title><p>The online version of this article (doi:10.1186/s12859-016-1364-3) contains supplementary material, which is available to authorized users.</p></sec></div></front><back><div1 type="bibliography"><listBibl><biblStruct><analytic><author><name sortKey="Tompa, M" uniqKey="Tompa M">M Tompa</name></author><author><name sortKey="Li, N" uniqKey="Li N">N Li</name></author><author><name sortKey="Bailey, Tl" uniqKey="Bailey T">TL Bailey</name></author><author><name sortKey="Church, Gm" uniqKey="Church G">GM Church</name></author><author><name sortKey="De Moor, B" uniqKey="De Moor B">B De Moor</name></author><author><name sortKey="Eskin, E" uniqKey="Eskin E">E Eskin</name></author><author><name sortKey="Favorov, Av" uniqKey="Favorov A">AV Favorov</name></author><author><name sortKey="Frith, Mc" uniqKey="Frith M">MC Frith</name></author><author><name sortKey="Fu, Y" uniqKey="Fu Y">Y Fu</name></author><author><name sortKey="Kent, Wj" uniqKey="Kent W">WJ Kent</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Das, Mk" uniqKey="Das M">MK Das</name></author><author><name sortKey="Dai, Hk" uniqKey="Dai H">HK Dai</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Blei, Dm" uniqKey="Blei D">DM Blei</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Blei, Dm" uniqKey="Blei D">DM Blei</name></author><author><name sortKey="Ng, Ay" uniqKey="Ng A">AY Ng</name></author><author><name sortKey="Jordan, Mi" uniqKey="Jordan M">MI Jordan</name></author></analytic></biblStruct><biblStruct></biblStruct><biblStruct><analytic><author><name sortKey="Mitchell, M" uniqKey="Mitchell M">M Mitchell</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Blei, D" uniqKey="Blei D">D Blei</name></author><author><name sortKey="Lafferty, J" uniqKey="Lafferty J">J Lafferty</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Aitchison, J" uniqKey="Aitchison J">J Aitchison</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Hornik, K" uniqKey="Hornik K">K Hornik</name></author><author><name sortKey="Grun, B" uniqKey="Grun B">B Grün</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Abnizova, I" uniqKey="Abnizova I">I Abnizova</name></author><author><name sortKey="Te Boekhorst, R" uniqKey="Te Boekhorst R">R te Boekhorst</name></author><author><name sortKey="Walter, K" uniqKey="Walter K">K Walter</name></author><author><name sortKey="Gilks, Wr" uniqKey="Gilks W">WR Gilks</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Shu, Jj" uniqKey="Shu J">JJ Shu</name></author><author><name sortKey="Li, Y" uniqKey="Li Y">Y Li</name></author></analytic></biblStruct><biblStruct></biblStruct><biblStruct><analytic><author><name sortKey="Favorov, Av" uniqKey="Favorov A">AV Favorov</name></author><author><name sortKey="Gelfand, Ms" uniqKey="Gelfand M">MS Gelfand</name></author><author><name sortKey="Gerasimova, Av" uniqKey="Gerasimova A">AV Gerasimova</name></author><author><name sortKey="Mironov, Aa" uniqKey="Mironov A">AA Mironov</name></author><author><name sortKey="Makeev, Vj" uniqKey="Makeev V">VJ Makeev</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Pavesi, G" uniqKey="Pavesi G">G Pavesi</name></author><author><name sortKey="Mereghetti, P" uniqKey="Mereghetti P">P Mereghetti</name></author><author><name sortKey="Mauri, G" uniqKey="Mauri G">G Mauri</name></author><author><name sortKey="Pesole, G" uniqKey="Pesole G">G Pesole</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Sinha, S" uniqKey="Sinha S">S Sinha</name></author><author><name sortKey="Tompa, M" uniqKey="Tompa M">M Tompa</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Pevzner, Pa" uniqKey="Pevzner P">PA Pevzner</name></author><author><name sortKey="Sze, Sh" uniqKey="Sze S">SH Sze</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Burset, M" uniqKey="Burset M">M Burset</name></author><author><name sortKey="Guigo, R" uniqKey="Guigo R">R Guigo</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Wingender, E" uniqKey="Wingender E">E Wingender</name></author><author><name sortKey="Dietze, P" uniqKey="Dietze P">P Dietze</name></author><author><name sortKey="Karas, H" uniqKey="Karas H">H Karas</name></author><author><name sortKey="Knuppel, R" uniqKey="Knuppel R">R Knüppel</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Hughes, Jd" uniqKey="Hughes J">JD Hughes</name></author><author><name sortKey="Estep, Pw" uniqKey="Estep P">PW Estep</name></author><author><name sortKey="Tavazoie, S" uniqKey="Tavazoie S">S Tavazoie</name></author><author><name sortKey="Church, Gm" uniqKey="Church G">GM Church</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Workman, Ct" uniqKey="Workman C">CT Workman</name></author><author><name sortKey="Stormo, Gd" uniqKey="Stormo G">GD Stormo</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Hertz, Gz" uniqKey="Hertz G">GZ Hertz</name></author><author><name sortKey="Stormo, Gd" uniqKey="Stormo G">GD Stormo</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Frith, Mc" uniqKey="Frith M">MC Frith</name></author><author><name sortKey="Hansen, U" uniqKey="Hansen U">U Hansen</name></author><author><name sortKey="Spouge, Jl" uniqKey="Spouge J">JL Spouge</name></author><author><name sortKey="Weng, Z" uniqKey="Weng Z">Z Weng</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Ao, W" uniqKey="Ao W">W Ao</name></author><author><name sortKey="Gaudet, J" uniqKey="Gaudet J">J Gaudet</name></author><author><name sortKey="Kent, Wj" uniqKey="Kent W">WJ Kent</name></author><author><name sortKey="Muttumu, S" uniqKey="Muttumu S">S Muttumu</name></author><author><name sortKey="Mango, Se" uniqKey="Mango S">SE Mango</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Bailey, Tl" uniqKey="Bailey T">TL Bailey</name></author><author><name sortKey="Elkan, C" uniqKey="Elkan C">C Elkan</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Eskin, E" uniqKey="Eskin E">E Eskin</name></author><author><name sortKey="Pevzner, P" uniqKey="Pevzner P">P Pevzner</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Thijs, G" uniqKey="Thijs G">G Thijs</name></author><author><name sortKey="Lescot, M" uniqKey="Lescot M">M Lescot</name></author><author><name sortKey="Marchal, K" uniqKey="Marchal K">K Marchal</name></author><author><name sortKey="Rombauts, S" uniqKey="Rombauts S">S Rombauts</name></author><author><name sortKey="De Moor, B" uniqKey="De Moor B">B De Moor</name></author><author><name sortKey="Rouze, P" uniqKey="Rouze P">P Rouze</name></author><author><name sortKey="Moreau, Y" uniqKey="Moreau Y">Y Moreau</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Van Helden, J" uniqKey="Van Helden J">J van Helden</name></author><author><name sortKey="Andre, B" uniqKey="Andre B">B Andre</name></author><author><name sortKey="Collado Vides, J" uniqKey="Collado Vides J">J Collado-Vides</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Van Helden, J" uniqKey="Van Helden J">J van Helden</name></author><author><name sortKey="Rios, Af" uniqKey="Rios A">AF Rios</name></author><author><name sortKey="Collado Vides, J" uniqKey="Collado Vides J">J Collado-Vides</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Regnier, M" uniqKey="Regnier M">M Régnier</name></author><author><name sortKey="Denise, A" uniqKey="Denise A">A Denise</name></author></analytic></biblStruct></listBibl></div1></back></TEI><pmc article-type="research-article"><pmc-dir>properties open_access</pmc-dir>
<front><journal-meta><journal-id journal-id-type="nlm-ta">BMC Bioinformatics</journal-id><journal-id journal-id-type="iso-abbrev">BMC Bioinformatics</journal-id><journal-title-group><journal-title>BMC Bioinformatics</journal-title></journal-title-group><issn pub-type="epub">1471-2105</issn><publisher><publisher-name>BioMed Central</publisher-name><publisher-loc>London</publisher-loc></publisher></journal-meta><article-meta><article-id pub-id-type="pmid">28155646</article-id><article-id pub-id-type="pmc">5259985</article-id><article-id pub-id-type="publisher-id">1364</article-id><article-id pub-id-type="doi">10.1186/s12859-016-1364-3</article-id><article-categories><subj-group subj-group-type="heading"><subject>Research</subject></subj-group></article-categories><title-group><article-title>A study on the application of topic models to motif finding algorithms</article-title></title-group><contrib-group><contrib contrib-type="author"><name><surname>Basha Gutierrez</surname><given-names>Josep</given-names></name><address><email>yusef@hgc.jp</email></address><xref ref-type="aff" rid="Aff1">1</xref><xref ref-type="aff" rid="Aff2">2</xref></contrib><contrib contrib-type="author" corresp="yes"><name><surname>Nakai</surname><given-names>Kenta</given-names></name><address><email>knakai@ims.u-tokyo.ac.jp</email></address><xref ref-type="aff" rid="Aff1">1</xref><xref ref-type="aff" rid="Aff2">2</xref></contrib><aff id="Aff1"><label>1</label><institution-wrap><institution-id institution-id-type="ISNI">0000 0001 2151 536X</institution-id><institution-id institution-id-type="GRID">grid.26999.3d</institution-id><institution>Department of Computational Biology and Medical Sciences, Graduate School of Frontier Sciences,</institution><institution>The University of Tokyo,</institution></institution-wrap>277-8561 Chiba, Japan</aff><aff id="Aff2"><label>2</label><institution-wrap><institution-id institution-id-type="ISNI">0000 0001 2151 536X</institution-id><institution-id institution-id-type="GRID">grid.26999.3d</institution-id><institution>Human Genome Center, The Institute of Medical Science,</institution><institution>The University of Tokyo,</institution></institution-wrap>4-6-1 Shirokane-dai, Minato-ku, 108-8639 Tokyo, Japan</aff></contrib-group><pub-date pub-type="epub"><day>22</day><month>12</month><year>2016</year></pub-date><pub-date pub-type="pmc-release"><day>22</day><month>12</month><year>2016</year></pub-date><pub-date pub-type="collection"><year>2016</year></pub-date><volume>17</volume><issue>Suppl 19</issue><issue-sponsor>Publication of this supplement has not been supported by sponsorship. Information about the source of funding for publication charges can be found in the individual articles. The articles have undergone the journal's standard peer review process for supplements. The Supplement Editors declare that they have no competing interests, except CS who was co-author of one article in this supplement, which was managed by the other Supplement Editors.</issue-sponsor><elocation-id>502</elocation-id><permissions><copyright-statement>© The Author(s). 2016</copyright-statement><license license-type="OpenAccess"><license-p><bold>Open Access</bold>This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (<ext-link ext-link-type="uri" xlink:href="http://creativecommons.org/licenses/by/4.0/">http://creativecommons.org/licenses/by/4.0/</ext-link>), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (<ext-link ext-link-type="uri" xlink:href="http://creativecommons.org/publicdomain/zero/1.0/">http://creativecommons.org/publicdomain/zero/1.0/</ext-link>) applies to the data made available in this article, unless otherwise stated.</license-p></license></permissions><abstract id="Abs1"><sec><title>Background</title><p>Topic models are statistical algorithms which try to discover the structure of a set of documents according to the abstract topics contained in them. Here we try to apply this approach to the discovery of the structure of the transcription factor binding sites (TFBS) contained in a set of biological sequences, which is a fundamental problem in molecular biology research for the understanding of transcriptional regulation. Here we present two methods that make use of topic models for motif finding. First, we developed an algorithm in which first a set of biological sequences are treated as text documents, and the k-mers contained in them as words, to then build a correlated topic model (CTM) and iteratively reduce its perplexity. We also used the perplexity measurement of CTMs to improve our previous algorithm based on a genetic algorithm and several statistical coefficients.</p></sec><sec><title>Results</title><p>The algorithms were tested with 56 data sets from four different species and compared to 14 other methods by the use of several coefficients both at nucleotide and site level. The results of our first approach showed a performance comparable to the other methods studied, especially at site level and in sensitivity scores, in which it scored better than any of the 14 existing tools. In the case of our previous algorithm, the new approach with the addition of the perplexity measurement clearly outperformed all of the other methods in sensitivity, both at nucleotide and site level, and in overall performance at site level.</p></sec><sec><title>Conclusions</title><p>The statistics obtained show that the performance of a motif finding method based on the use of a CTM is satisfying enough to conclude that the application of topic models is a valid method for developing motif finding algorithms. Moreover, the addition of topic models to a previously developed method dramatically increased its performance, suggesting that this combined algorithm can be a useful tool to successfully predict motifs in different kinds of sets of DNA sequences.</p></sec><sec><title>Electronic supplementary material</title><p>The online version of this article (doi:10.1186/s12859-016-1364-3) contains supplementary material, which is available to authorized users.</p></sec></abstract><conference><conf-name>15th International Conference On Bioinformatics (INCOB 2016)</conf-name><conf-acronym>InCOB 2016</conf-acronym><conf-loc>Queenstown, Singapore</conf-loc><conf-date>21-23 September 2016</conf-date><string-conf><uri>http://incob16.apbionet.org/</uri></string-conf></conference><custom-meta-group><custom-meta><meta-name>issue-copyright-statement</meta-name><meta-value>© The Author(s) 2016</meta-value></custom-meta></custom-meta-group></article-meta></front><body><sec id="Sec1" sec-type="introduction"><title>Background</title><p>Sequence motifs are short patterns that occur in DNA with certain frequency and that often have some sort of biological distinct function. In most cases, that function is to serve as a binding site for proteins. When these proteins are transcription factors (TF), the corresponding motifs are called transcription factor binding sites (TFBS). Knowing these TFBS gives a better understanding of how transcriptional regulation works, and therefore the discovery of TFBS is one of the most fundamental problems in molecular biology research [<xref ref-type="bibr" rid="CR1">1</xref>, <xref ref-type="bibr" rid="CR2">2</xref>]. Historically, a wide variety of methods have been applied to this problem, computational methods being currently the prevailing approach. The computational problem consists of discovering motifs by searching for overrepresented (and/or conserved) DNA patterns in sets of functionally related genes, such as genes with similar functional annotation or genes with similar expression patterns. The number of different computational approaches to tackle this problem is constantly growing as computational techniques evolve. One of the most recent techniques, which, to the best of our knowledge, to this date has not yet been applied to motif discovery, is known as <italic>topic models</italic> [<xref ref-type="bibr" rid="CR3">3</xref>].</p><sec id="Sec2"><title>Topic models</title><p>Topic models are statistical algorithms, based on natural language processing and machine learning, which try to discover the structure of a set of documents according to the abstract topics contained in them by hierarchical Bayesian analysis [<xref ref-type="bibr" rid="CR4">4</xref>]. These algorithms allow examining a set of documents and determining the existing topics and their distribution among the documents based on the statistical properties of the words of a specific vocabulary in each one of them. <italic>where L</italic> Application of topic models to the motif finding problem.</p><p>As far as we know, there is no literature about the application of topic models to motif finding algorithms. The first method here proposed tries to fill that gap and prove that topic models are a suitable method to the motif finding problem. In order to do so, it represents genetic sequences as documents and the k-mers contained in them as words, so that the patterns shown among these k-mers would be considered as motifs. Figure <xref rid="Fig1" ref-type="fig">1</xref> shows a graphic representation of a topic model and how our algorithm would adapt to it.<fig id="Fig1"><label>Fig. 1</label><caption><p>Representation of a topic model adapted to the motif finding problem. Representation of a topic model adapted to the motif finding problem. This figure shows the basic structure of a topic model (in this case, a LDA). The terms specific for the case of the motif finding problem are stated in red under the original ones in blue, showing that the motif finding problem can be represented by a topic model by describing the DNA sequences as documents, the instances of each given motif as words in those documents, and the motifs as clusters of words or topics</p></caption><graphic xlink:href="12859_2016_1364_Fig1_HTML" id="MO1"></graphic></fig></p><p>The algorithm, as a topic model, would therefore examine a set of sequences to determine the hidden structure of the patterns contained in it. As this is totally consistent with the motif finding problem, it seems likely that the algorithm should be able to correctly discover motifs.</p></sec><sec id="Sec3"><title>Addition of topic models to a previously developed algorithm (Statistical GA)</title><p>Previously to this study of topic models applied to the motif finding problem, we developed another algorithm with the structure of a GA, which used statistical coefficients as a fitness measurement [<xref ref-type="bibr" rid="CR5">5</xref>]. From this point, we will refer to this algorithm as the <italic>Statistical GA</italic> algorithm. The Statistical GA algorithm was proven to effectively find overrepresented motifs in sets of sequences. However, it had the main drawback of reporting an excessive number of false positives. Along with the algorithm based entirely on topic models, in this study we also research how the use of topic models can be applied to improve the previously developed algorithm and reduce the number of false positives.</p></sec></sec><sec id="Sec4" sec-type="materials|methods"><title>Methods</title><sec id="Sec5"><title>How topic models work</title><p>The main problem, from a computational point of view, of topic modeling is to infer a concealed topic structure from the examination of the documents.</p><p>A topic is formally defined as a multinomial distribution over a fixed vocabulary. In other words, topic models consider that a document could, conceptually, be generated from a set of topics, each one of them being a set of words related to that topic. So that, to create a document, the words would be selected iteratively from the topics that we desire to appear in it. For example, if we want a document that is two thirds about stem cells and one third about cancer, we would create two topics (stem cells and cancer) as sets of words typically related to them, and then construct the document by selecting two thirds of the words from the <italic>stem cells</italic> set and one third from the <italic>cancer</italic> set.</p><p>Topic modeling consists of reversing this conceptual approach, considering that the topics of a document (or a set of documents) can be inferred from the proportions of the words contained in them.</p><p>The intuition behind this algorithm is that all of the documents in the collection share the same set of topics, but each one of them in a different proportion, which is reflected in the distribution of the different words among them.</p><p>The inputs of a topic model are a set of <italic>N</italic> documents (<italic>d</italic><sub><italic>1</italic></sub><italic>, …, d</italic><sub><italic>N</italic></sub>) and the number of topics <italic>K</italic> that are expected to be contained in the documents. For each one of the documents <italic>d</italic><sub><italic>i</italic></sub> to be analyzed, the most basic algorithm would process the words in a two-stage process.<list list-type="order"><list-item><p>Choose a random distribution of the document over the topics (<italic>t</italic><sub><italic>1</italic></sub><italic>, …, t</italic><sub><italic>K</italic></sub>).</p></list-item><list-item><p>For each word <italic>w</italic><sub><italic>j</italic></sub> in the document:<list list-type="alpha-lower"><list-item><p>Choose a random topic <italic>t</italic><sub><italic>r</italic></sub> from the distribution over topics previously generated.</p></list-item><list-item><p>Once <italic>w</italic><sub><italic>j</italic></sub> is assigned, for each one of the topics <italic>t</italic><sub><italic>m</italic></sub> in the current set of topics, compute the proportion of words in the document <italic>d</italic><sub><italic>i</italic></sub> that are currently assigned to the topic <italic>t</italic><sub><italic>m</italic></sub>, <italic>P(t</italic><sub><italic>m</italic></sub><italic>|d</italic><sub><italic>i</italic></sub><italic>)</italic>, and the proportion of assignments to the topic <italic>t</italic><sub><italic>m</italic></sub> over all of the documents that come from the word <italic>w</italic><sub><italic>j</italic></sub> , <italic>P(w</italic><sub><italic>j</italic></sub><italic>|t</italic><sub><italic>m</italic></sub><italic>)</italic> and then reassign <italic>w</italic><sub><italic>j</italic></sub> to the topic that gives the best probability <italic>P(t</italic><sub><italic>m</italic></sub><italic>|d</italic><sub><italic>i</italic></sub><italic>) * P(w</italic><sub><italic>j</italic></sub><italic>|t</italic><sub><italic>m</italic></sub><italic>)</italic>.</p></list-item></list></p></list-item></list></p><p>A stable set of assignments will be reached after repeating the above steps for several iterations.</p><p>The benefit of the use of topic models is that they offer an automated solution to the organization and annotation of large text archives. However, this is not their only utility, and they can be applied to many other fields, such as the subject in question here, bioinformatics.</p></sec><sec id="Sec6"><title>Creating a motif finding algorithm based on topic models from scratch</title><sec id="Sec7"><title>Algorithm structure</title><p>The first problem that arises when adapting a topic model to the motif finding problem comes from the idea that, whereas the words in a text document are clearly separated by spaces, in the case of a genetic sequence a mechanism to select the k-mers that will form the vocabulary must be defined.</p><p>A typical topic model, as a first step, usually creates a vocabulary from the words in the documents by discarding meaningless words (in terms of determining a topic), such as “the” or “of” in documents written in English, as well as words that are not repeated frequently, since in both cases they would not help to find the hidden topics and they would instead add noise to the algorithm. Again, this is consistent with a motif finding algorithm, so in this case an initial vocabulary would need to be created similarly, but in this case by selecting k-mers that are overrepresented in the set of sequences.</p><p>From this a new problem arises, which is the impossibility to select all of the possible overrepresented patterns in a reduced amount of time. In order to deal with that, a genetic algorithm (GA) [<xref ref-type="bibr" rid="CR6">6</xref>] structure was chosen as the basis of the algorithm here presented, being the topic model the approach for selecting the best possible solutions in the fitness function.</p></sec><sec id="Sec8"><title>Algorithm implementation</title><p>The method here proposed is a heuristic algorithm, that is, it gives an approximate (not necessarily optimal) solution, and it is also stochastic, so that each time it is run with the same set of sequences it will likely produce different results. It searches only for ungapped motifs, so that patterns which contain gaps might be predicted split into several separate motifs. Also, in contrast with other motif finding algorithms, which usually suppose that there is at least an instance of the motifs in every sequence of the data set, it makes no assumptions about how the motifs are distributed among the sequences.</p><p>The algorithm is implemented as a classic GA. In other words, it starts by creating a population of possible solutions (individuals) for our problem and then it iterates over them, keeping the best (fittest) solutions of every iteration, discarding the worst ones, and creating new solutions based on the fittest ones for the following iteration, until an optimum solution is found or a given number of iterations is reached.</p><p>Therefore, the only aspects that need to be defined are how the population is represented, how it is evaluated (fitness), how the fittest individuals are selected in every iteration, how new individuals are generated by the surviving ones (crossover, mutation) and when the algorithm will stop iterating and report a final solution (or set of solutions).</p></sec><sec id="Sec9"><title>Representation</title><p>Each individual of the population is a set of <italic>m</italic> k-mers which can be contained in any of the sequences of the data set. The k-mers can be of any length between a minimum and a maximum passed as a parameter. The initialization works as follows:</p><p>Given a set of sequences, a minimum k-mer length <italic>k</italic><sub><italic>min</italic></sub>, a maximum k-mer length <italic>k</italic><sub><italic>max</italic></sub>, a minimum number of repetitions for each k-mer in the data set <italic>c</italic><sub><italic>min</italic></sub>, a population size <italic>N</italic>, and a number of words per individual in the population <italic>n</italic>. For each one of the <italic>N</italic> individuals, iterate until an initial set of <italic>n</italic> k-mers is reached:<list list-type="order"><list-item><p>Choose a random word length <italic>k</italic> within the range <italic>k</italic><sub><italic>min</italic></sub><italic>: k</italic><sub><italic>max</italic></sub>.</p></list-item><list-item><p>Choose a random sequence from the data set.</p></list-item><list-item><p>Choose a random position <italic>p</italic> in that sequence between 0 and <italic>l - k</italic>, being <italic>l</italic> the sequence length.</p></list-item><list-item><p>Select the word <italic>w</italic> starting in the position <italic>p</italic> with length <italic>k</italic>.</p></list-item><list-item><p>Count the number of occurrences <italic>c</italic> of the word <italic>w</italic> in the given sequence, allowing for 25% of mismatches.</p></list-item><list-item><p>Shuffle <italic>w</italic> into <italic>w</italic><sub>s</sub> and count the number of occurrences <italic>c</italic><sub><italic>s</italic></sub> of the word <italic>w</italic><sub><italic>s</italic></sub> in the given sequence, allowing for 25% of mismatches.</p></list-item><list-item><p>If <italic>c - c</italic><sub><italic>s</italic></sub> is greater or equal than <italic>c</italic><sub><italic>min</italic></sub>, add the k-mer to the set for the given individual.</p></list-item></list></p></sec><sec id="Sec10"><title>Evaluation</title><p>The fitness calculation is the more crucial step in a GA. It is at this moment when the topic models must be applied and provide a way to obtain solutions for the motif finding problem.</p><p>The type of topic model chosen was a correlated topic model (CTM) [<xref ref-type="bibr" rid="CR7">7</xref>], since it takes into consideration the correlation between topics, and, biologically speaking, motifs also usually show correlation, given that transcription factors which have correlated biological functions bind to them. A CTM makes use of a logistic normal distribution, which, through the transformation of a multivariate normal random variable, allows for a general pattern of variability between the components of the distribution [<xref ref-type="bibr" rid="CR8">8</xref>]. More specifically, the CTM contained in the R package <italic>topicmodels</italic> [<xref ref-type="bibr" rid="CR9">9</xref>] was the method used for the construction of the CTM in every iteration.</p><p>For each one of the individuals of the population, its set of k-mers, along with the original set of sequences, is fed to a CTM as the vocabulary and the documents respectively. Then the perplexity of the resulting model is measured and returned as the fitness of the given individual.</p></sec><sec id="Sec11"><title>Perplexity</title><p>The perplexity of a probabilistic model is a measure of the accuracy with which its distribution predicts a sample. It is the standard used in natural language processing to evaluate the accuracy of the model. The lower the perplexity, the better the model fits the data. The perplexity is calculated by splitting the dataset into two parts: one for training and one for testing, and then measuring the log-likelihood of the unseen documents. As the perplexity is calculated using the corresponding function provided by Hornik and Grün for their CTM implementation, the mathematical formula for perplexity used in this method follows their same definition [<xref ref-type="bibr" rid="CR9">9</xref>]:<disp-formula id="Equa"><alternatives><tex-math id="M1">\documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$ Perp\left(\omega \right)= exp\left\{-\frac{ \log \left(p\left(\omega \right)\right)}{{\displaystyle {\sum}_{d=1}^D}{\displaystyle {\sum}_{j=1}^V}{n}^{(jd)}}\right\} $$\end{document}</tex-math><mml:math id="M2"><mml:mi mathvariant="italic">Perp</mml:mi><mml:mfenced close=")" open="("><mml:mi>ω</mml:mi></mml:mfenced><mml:mo>=</mml:mo><mml:mi mathvariant="italic">exp</mml:mi><mml:mfenced close="}" open="{"><mml:mrow><mml:mo>−</mml:mo><mml:mfrac><mml:mrow><mml:mo>log</mml:mo><mml:mfenced close=")" open="("><mml:mrow><mml:mi>p</mml:mi><mml:mfenced close=")" open="("><mml:mi>ω</mml:mi></mml:mfenced></mml:mrow></mml:mfenced></mml:mrow><mml:mrow><mml:mstyle displaystyle="true"><mml:msubsup><mml:mo stretchy="true">∑</mml:mo><mml:mrow><mml:mi>d</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mi>D</mml:mi></mml:msubsup></mml:mstyle><mml:mstyle displaystyle="true"><mml:msubsup><mml:mo stretchy="true">∑</mml:mo><mml:mrow><mml:mi>j</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mi>V</mml:mi></mml:msubsup></mml:mstyle><mml:msup><mml:mi>n</mml:mi><mml:mfenced close=")" open="("><mml:mrow><mml:mi>j</mml:mi><mml:mi>d</mml:mi></mml:mrow></mml:mfenced></mml:msup></mml:mrow></mml:mfrac></mml:mrow></mml:mfenced></mml:math><graphic xlink:href="12859_2016_1364_Article_Equa.gif" position="anchor"></graphic></alternatives></disp-formula></p><p>where <italic>n</italic><sup><italic>(jd)</italic></sup> refers to the frequency with which the <italic>j</italic>th word appears in the document <italic>d</italic>.</p></sec><sec id="Sec12"><title>Selection</title><p>The algorithm tries to give an optimum set of solutions by minimizing the perplexity. Therefore, for the selection of the fittest candidates, an elitist approach is used. In other words, after all of the fitness measurements have been done for a specific generation of individuals, these are selected in random pairs, in which the fittest individual (lower perplexity) survives and the less fit individual is eliminated from the population. After this stage, <italic>N/2</italic> fit individuals remain in the population.</p><p>So the next step is generating new individuals by the use of the crossover function to create a new population of <italic>N</italic> individuals.</p></sec><sec id="Sec13"><title>Crossover</title><p>The Crossover step is performed after the Evaluation and Selection step to generate new individuals in the population for the next generation.</p><p>First, two individuals are randomly selected from the population to act as parents.</p><p>The crossover function in this case is a classic one-point crossover in which two children are generated by swapping the data beyond a randomly selected crossover point between both parents. In this case, the crossover point is an index in the array of k-mers of the individuals, which indicates which k-mers to select from each one of the parents (these k-mers are shuffled before this step).</p><p>The two newly formed children are added to the population and the process is repeated until the population contains <italic>N</italic> individuals again.</p></sec><sec id="Sec14"><title>Mutation</title><p>Mutation happens randomly, according to a parameter that defines the frequency. It is also applied to random individuals. The mutated individual will have a random number of its k-mers slightly shifted from their original position (the position in the sequence randomly increases or decreases by a number no longer than the length of the given k-mer).</p></sec><sec id="Sec15"><title>Post processing</title><p>Once the GA is terminated, the fittest individuals are sorted by perplexity (from lower to higher) and selected accordingly as solutions depending on a parameter set by the user that defines how many motifs are expected to be found in the data set. For each one of these solutions, the CTM is generated once again and each one of the resulting topics is returned as a motif of the data set.</p></sec></sec><sec id="Sec16"><title>Improving the statistical GA algorithm by the use of the perplexity measurement</title><p>The Statistical GA algorithm [<xref ref-type="bibr" rid="CR5">5</xref>] works as a GA in which the fitness function takes three steps to discard unfit solutions based on three different coefficients [<xref ref-type="bibr" rid="CR10">10</xref>, <xref ref-type="bibr" rid="CR11">11</xref>], in which the main method to select the final candidates is the Mann-Whitney U-Test [<xref ref-type="bibr" rid="CR12">12</xref>]. Each candidate is a k-mer of a fixed length defined as a parameter represented as a position in a supersequence, which is a concatenation in a random order of all the genetic sequences received as an input. To simplify the calculations, this supersequence is divided in a set of subsequences so that in each iteration the fitness is calculated for a given subsequence and the candidates which show no overrepresentation in the given segment are swiftly discarded without further computations.</p><sec id="Sec17"><title>Adding the use of topic models</title><p>The main drawback of the Statistical GA algorithm is that it reported a big amount of false positives, and one of the main reasons for this was that it had no way to measure the confidence of the results reported. Thus, it reported at least one motif for every data set, ballasting that way the overall performance of the algorithm. The solution here proposed for that problem consists of taking the final set of instances provided by the algorithm, creating a CTM with them, and measuring the perplexity. Then, only in the cases in which the perplexity is lower than certain threshold the motifs returned by the algorithm are reported (Fig. <xref rid="Fig2" ref-type="fig">2</xref>). The impact of this was that now the Statistical GA algorithm only reports motifs for those data sets in which there is a CTM that fits well the solutions found. That way, the algorithm is able to measure the confidence of the solutions obtained by the main GA. The threshold was set at 100 for experimental reasons, given that in the tests performed the motifs reported with a perplexity higher than 100 tended to be false positives.<fig id="Fig2"><label>Fig. 2</label><caption><p>Statistical GA algorithm workflow after including the use of topic models. This figure describes the updated flow of the Statistical GA algorithm after adding the perplexity measurement for the selection of solutions. There are four steps in this flow: the first one, in which the candidate instances are selected by the original Statistical GA; the second one, in which these instances are clustered attending to their similarity calculated by their hamming distance; the third one, which consists of building the CTM and measuring its perplexity, and the last step, which consists of reporting the motif if the perplexity calculated in the previous step is lower than 100</p></caption><graphic xlink:href="12859_2016_1364_Fig2_HTML" id="MO2"></graphic></fig></p></sec></sec><sec id="Sec18"><title>Assessment</title><p>Several studies [<xref ref-type="bibr" rid="CR1">1</xref>, <xref ref-type="bibr" rid="CR2">2</xref>] concluded that evaluating the performance of a motif finding tool has been proven to be a difficult task, and there is no method to compare tools that can give a definitive conclusion about which one is the best and which one is the worst. Keeping this idea in mind, both of the two methods here presented were tested making use of the assessment proposed by Tompa et al. [<xref ref-type="bibr" rid="CR1">1</xref>] in their study to evaluate the performance of several motif finding tools by the scores obtained in eight different statistical coefficients. It is worth mentioning that only the accuracy of the tools predicting binding sites is evaluated, and other aspects such as the running time of each method, are not measured. The benchmark provided by the assessment, which is the same one used in this study, is formed by 52 data sets, which belong to four different species (fly, human, mouse and yeast) and also 4 negative controls to sum a total of 56 data sets. These 56 data sets are also divided into three different categories: data sets of Type Real, which correspond to the real promoter sequences that contain the original sites that the different tools will try to locate; data sets of Type Generic, which correspond to promoter sequences generated randomly from the same genome, and data sets of Type Markov, which correspond to synthetic sequences generated by a Markov chain. The original assessment compared the efficiency of 14 different tools (Additional file <xref rid="MOESM1" ref-type="media">1</xref>: Table S1) [<xref ref-type="bibr" rid="CR13">13</xref>–<xref ref-type="bibr" rid="CR26">26</xref>]. Each one of those tools was allowed to report only one (or none) motif per data set. The format in which this motif was reported was as a list of instances and their corresponding positions in the sequences of the data set. Then, the accuracy of how well these instances match the real instances of the motif is studied both at nucleotide and site level. At site level, a predicted site is considered to match the known site if it overlaps at least one quarter of it. With this information, the following eight statistics are used to measure the accuracy of each one of the methods:<list list-type="bullet"><list-item><p><italic>nSn</italic> (Sensitivity, nucleotide level):<disp-formula id="Equb"><alternatives><tex-math id="M3">\documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$ nSn = \frac{nTP}{nTP+nFN} $$\end{document}</tex-math><mml:math id="M4"><mml:mi>n</mml:mi><mml:mi>S</mml:mi><mml:mi>n</mml:mi><mml:mo>=</mml:mo><mml:mspace width="0.25em"></mml:mspace><mml:mfrac><mml:mrow><mml:mi>n</mml:mi><mml:mi>T</mml:mi><mml:mi>P</mml:mi></mml:mrow><mml:mrow><mml:mi>n</mml:mi><mml:mi>T</mml:mi><mml:mi>P</mml:mi><mml:mo>+</mml:mo><mml:mi>n</mml:mi><mml:mi>F</mml:mi><mml:mi>N</mml:mi></mml:mrow></mml:mfrac></mml:math><graphic xlink:href="12859_2016_1364_Article_Equb.gif" position="anchor"></graphic></alternatives></disp-formula></p></list-item><list-item><p><italic>nPPV</italic> (Positive Predicted Value, nucleotide level):<disp-formula id="Equc"><alternatives><tex-math id="M5">\documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$ nPPV = \frac{nTP}{nTP+nFP} $$\end{document}</tex-math><mml:math id="M6"><mml:mi mathvariant="italic">nPPV</mml:mi><mml:mo>=</mml:mo><mml:mspace width="0.25em"></mml:mspace><mml:mfrac><mml:mrow><mml:mi>n</mml:mi><mml:mi>T</mml:mi><mml:mi>P</mml:mi></mml:mrow><mml:mrow><mml:mi>n</mml:mi><mml:mi>T</mml:mi><mml:mi>P</mml:mi><mml:mo>+</mml:mo><mml:mi>n</mml:mi><mml:mi>F</mml:mi><mml:mi>P</mml:mi></mml:mrow></mml:mfrac></mml:math><graphic xlink:href="12859_2016_1364_Article_Equc.gif" position="anchor"></graphic></alternatives></disp-formula></p></list-item><list-item><p><italic>nSp</italic> (Specificity):<disp-formula id="Equd"><alternatives><tex-math id="M7">\documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$ nSp = \frac{nTN}{nTN+nFP} $$\end{document}</tex-math><mml:math id="M8"><mml:mi>n</mml:mi><mml:mi>S</mml:mi><mml:mi>p</mml:mi><mml:mo>=</mml:mo><mml:mspace width="0.25em"></mml:mspace><mml:mfrac><mml:mrow><mml:mi>n</mml:mi><mml:mi>T</mml:mi><mml:mi>N</mml:mi></mml:mrow><mml:mrow><mml:mi>n</mml:mi><mml:mi>T</mml:mi><mml:mi>N</mml:mi><mml:mo>+</mml:mo><mml:mi>n</mml:mi><mml:mi>F</mml:mi><mml:mi>P</mml:mi></mml:mrow></mml:mfrac></mml:math><graphic xlink:href="12859_2016_1364_Article_Equd.gif" position="anchor"></graphic></alternatives></disp-formula></p></list-item><list-item><p><italic>nPC</italic> (Performance Coefficient, nucleotide level) [<xref ref-type="bibr" rid="CR27">27</xref>]:<disp-formula id="Eque"><alternatives><tex-math id="M9">\documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$ nPC = \frac{nTP}{nTP+nFN+nFP} $$\end{document}</tex-math><mml:math id="M10"><mml:mi>n</mml:mi><mml:mi>P</mml:mi><mml:mi>C</mml:mi><mml:mo>=</mml:mo><mml:mspace width="0.25em"></mml:mspace><mml:mfrac><mml:mrow><mml:mi>n</mml:mi><mml:mi>T</mml:mi><mml:mi>P</mml:mi></mml:mrow><mml:mrow><mml:mi>n</mml:mi><mml:mi>T</mml:mi><mml:mi>P</mml:mi><mml:mo>+</mml:mo><mml:mi>n</mml:mi><mml:mi>F</mml:mi><mml:mi>N</mml:mi><mml:mo>+</mml:mo><mml:mi>n</mml:mi><mml:mi>F</mml:mi><mml:mi>P</mml:mi></mml:mrow></mml:mfrac></mml:math><graphic xlink:href="12859_2016_1364_Article_Eque.gif" position="anchor"></graphic></alternatives></disp-formula></p></list-item><list-item><p><italic>nCC</italic> (Correlation Coefficient) [<xref ref-type="bibr" rid="CR28">28</xref>]:<disp-formula id="Equf"><alternatives><tex-math id="M11">\documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$ nCC = \frac{nTP \times nTN+nFN \times nFP}{\sqrt{\left(nTP+nFN\right)\left(nTN+nFP\right)\left(nTP+nFP\right)\left(nTN+nFN\right)}} $$\end{document}</tex-math><mml:math id="M12"><mml:mi>n</mml:mi><mml:mi>C</mml:mi><mml:mi>C</mml:mi><mml:mo>=</mml:mo><mml:mspace width="0.25em"></mml:mspace><mml:mfrac><mml:mrow><mml:mi>n</mml:mi><mml:mi>T</mml:mi><mml:mi>P</mml:mi><mml:mspace width="0.25em"></mml:mspace><mml:mo>×</mml:mo><mml:mspace width="0.25em"></mml:mspace><mml:mi>n</mml:mi><mml:mi>T</mml:mi><mml:mi>N</mml:mi><mml:mo>+</mml:mo><mml:mi>n</mml:mi><mml:mi>F</mml:mi><mml:mi>N</mml:mi><mml:mspace width="0.25em"></mml:mspace><mml:mo>×</mml:mo><mml:mspace width="0.25em"></mml:mspace><mml:mi>n</mml:mi><mml:mi>F</mml:mi><mml:mi>P</mml:mi></mml:mrow><mml:msqrt><mml:mrow><mml:mfenced close=")" open="("><mml:mrow><mml:mi>n</mml:mi><mml:mi>T</mml:mi><mml:mi>P</mml:mi><mml:mo>+</mml:mo><mml:mi>n</mml:mi><mml:mi>F</mml:mi><mml:mi>N</mml:mi></mml:mrow></mml:mfenced><mml:mfenced close=")" open="("><mml:mrow><mml:mi>n</mml:mi><mml:mi>T</mml:mi><mml:mi>N</mml:mi><mml:mo>+</mml:mo><mml:mi>n</mml:mi><mml:mi>F</mml:mi><mml:mi>P</mml:mi></mml:mrow></mml:mfenced><mml:mfenced close=")" open="("><mml:mrow><mml:mi>n</mml:mi><mml:mi>T</mml:mi><mml:mi>P</mml:mi><mml:mo>+</mml:mo><mml:mi>n</mml:mi><mml:mi>F</mml:mi><mml:mi>P</mml:mi></mml:mrow></mml:mfenced><mml:mfenced close=")" open="("><mml:mrow><mml:mi>n</mml:mi><mml:mi>T</mml:mi><mml:mi>N</mml:mi><mml:mo>+</mml:mo><mml:mi>n</mml:mi><mml:mi>F</mml:mi><mml:mi>N</mml:mi></mml:mrow></mml:mfenced></mml:mrow></mml:msqrt></mml:mfrac></mml:math><graphic xlink:href="12859_2016_1364_Article_Equf.gif" position="anchor"></graphic></alternatives></disp-formula></p></list-item><list-item><p><italic>sSn</italic> (Sensitivity, site level):<disp-formula id="Equg"><alternatives><tex-math id="M13">\documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$ sSn = \frac{sTP}{sTP+sFN} $$\end{document}</tex-math><mml:math id="M14"><mml:mi>s</mml:mi><mml:mi>S</mml:mi><mml:mi>n</mml:mi><mml:mo>=</mml:mo><mml:mspace width="0.25em"></mml:mspace><mml:mfrac><mml:mrow><mml:mi>s</mml:mi><mml:mi>T</mml:mi><mml:mi>P</mml:mi></mml:mrow><mml:mrow><mml:mi>s</mml:mi><mml:mi>T</mml:mi><mml:mi>P</mml:mi><mml:mo>+</mml:mo><mml:mi>s</mml:mi><mml:mi>F</mml:mi><mml:mi>N</mml:mi></mml:mrow></mml:mfrac></mml:math><graphic xlink:href="12859_2016_1364_Article_Equg.gif" position="anchor"></graphic></alternatives></disp-formula></p></list-item><list-item><p><italic>sPPV</italic> (Positive Predicted Value, site level):<disp-formula id="Equh"><alternatives><tex-math id="M15">\documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$ sPPV = \frac{sTP}{sTP+sFP} $$\end{document}</tex-math><mml:math id="M16"><mml:mi mathvariant="italic">sPPV</mml:mi><mml:mo>=</mml:mo><mml:mspace width="0.25em"></mml:mspace><mml:mfrac><mml:mrow><mml:mi>s</mml:mi><mml:mi>T</mml:mi><mml:mi>P</mml:mi></mml:mrow><mml:mrow><mml:mi>s</mml:mi><mml:mi>T</mml:mi><mml:mi>P</mml:mi><mml:mo>+</mml:mo><mml:mi>s</mml:mi><mml:mi>F</mml:mi><mml:mi>P</mml:mi></mml:mrow></mml:mfrac></mml:math><graphic xlink:href="12859_2016_1364_Article_Equh.gif" position="anchor"></graphic></alternatives></disp-formula></p></list-item><list-item><p>sASP (Average Site Performance) [<xref ref-type="bibr" rid="CR28">28</xref>]:<disp-formula id="Equi"><alternatives><tex-math id="M17">\documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$ sASP = \frac{sSn+ sPPV}{2} $$\end{document}</tex-math><mml:math id="M18"><mml:mi mathvariant="italic">sASP</mml:mi><mml:mo>=</mml:mo><mml:mspace width="0.25em"></mml:mspace><mml:mfrac><mml:mrow><mml:mi>s</mml:mi><mml:mi>S</mml:mi><mml:mi>n</mml:mi><mml:mo>+</mml:mo><mml:mi mathvariant="italic">sPPV</mml:mi></mml:mrow><mml:mn>2</mml:mn></mml:mfrac></mml:math><graphic xlink:href="12859_2016_1364_Article_Equi.gif" position="anchor"></graphic></alternatives></disp-formula></p></list-item></list></p><p>The coefficients starting by <italic>n</italic> are statistics at nucleotide level, and the coefficients starting by <italic>s</italic> are statistics at site level. <italic>TP</italic>, <italic>FP</italic>, <italic>TN</italic> and <italic>FN</italic> refer to the number of true positives, false positives, true negatives and false negatives respectively.</p><p>Both of the two methods here described were tested using the methodology presented in this assessment and compared to the 14 methods with which it was originally carried out.</p></sec></sec><sec id="Sec19" sec-type="results"><title>Results</title><p>The CTM algorithm was run with the following parameters:<list list-type="bullet"><list-item><p>Motif width between 6 and 30</p></list-item><list-item><p>Population size: 50</p></list-item><list-item><p>Number of generations: 90</p></list-item><list-item><p>Number of instances per individual: 1000</p></list-item><list-item><p>Maximum number of solutions: 10</p></list-item><list-item><p>Mutation rate: 0.1</p></list-item></list></p><p>As for the statistical GA algorithm, it was run with the same parameters as in the original study [<xref ref-type="bibr" rid="CR5">5</xref>]. After adding the perplexity measurement in the post processing stage, a new restriction was included: Only the motifs reported with a perplexity lower than 100 were considered as solutions.</p><p>All of the tests were run in a laptop computer with a 2.6 GHz Intel Core i5 processor and an 8 GB 1600 MHz DDR3 memory.</p><p>Figure <xref rid="Fig3" ref-type="fig">3</xref> summarizes the average values of the statistics previously defined for each one of the 14 tools originally analyzed in the assessment and for both of our proposed tools. Figure <xref rid="Fig4" ref-type="fig">4</xref> shows the average values grouped by organisms.<fig id="Fig3"><label>Fig. 3</label><caption><p>Average statistical values for all 56 data sets. This figure shows the average scores obtained by each one of the tools studied for each one of seven different statistics for all the 56 data sets of the benchmark. The Statistical GA method is shown as GA approach, and the CTM method as CTM approach</p></caption><graphic xlink:href="12859_2016_1364_Fig3_HTML" id="MO3"></graphic></fig><fig id="Fig4"><label>Fig. 4</label><caption><p>Average statistical values for each organism. This figure shows the average scores obtained by each one of the tools studied for each one of seven different statistics grouped by the four different species contained in the data sets. The Statistical GA method is shown as GA approach, and the CTM method as CTM approach</p></caption><graphic xlink:href="12859_2016_1364_Fig4_HTML" id="MO4"></graphic></fig></p><p>To calculate the average values, we followed the same process as in the original assessment. In a first step, the values of <italic>nTP</italic>, <italic>nFP</italic>, <italic>nFN</italic>, <italic>nTN</italic>, <italic>sTP</italic>, <italic>sFP</italic> and <italic>sFN</italic> obtained for each one of the data sets are summed. Then, each one of these summed values is considered as the given score of a large data set, and the eight statistics are calculated for that large data set , obtaining that way the average scores.</p></sec><sec id="Sec20" sec-type="discussion"><title>Discussion</title><p>As previously stated, none of the statistics analyzed should ever be taken as an absolute measurement of the quality of the methods. The authors of the assessment [<xref ref-type="bibr" rid="CR1">1</xref>] themselves indicate several factors that affect the results and might give a wrong impression about the performance of the different algorithms:<list list-type="bullet"><list-item><p>This assessment, as any other possible method, can never be considered a standard method to measure the biological significance of the studied tools, since it is still unknown how the subjacent biology works.</p></list-item><list-item><p>As each one of the algorithms was required to predict only one (or none) motif for each data set, there might be an arbitrary component in the candidate selected by each tool.</p></list-item><list-item><p>The assessment requires each tool to report only one (or none) motif for each data set. However, it is known that, especially in the case of the data sets of Type Real, they are likely to contain more than one motif.</p></list-item><list-item><p>Many of the known binding sites are longer than 30 bp. Our tools, as well as most of the others, were run for motifs no longer than 30 bp in the case of the CTM algorithm and 12 bp in the case of the statistical GA algorithm. This affects the performance at nucleotide level even if the performance at site level is high.</p></list-item><list-item><p>The assessment relies on TRANSFAC [<xref ref-type="bibr" rid="CR29">29</xref>] as its only source of known binding sites. As the information obtained from TRANSFAC is not contrasted with other sources, it might as well contain errors.</p></list-item><list-item><p>The above explained method used to compute the average scores of every tool tends to penalize those tools that make wrong predictions more than those that make no predictions at all, as 0 is the default value for the cases in which no motifs are reported.</p></list-item></list></p><p>As long as all these factors are not forgotten, some important conclusions regarding the performance of the different methods can still be inferred from the use of the benchmark proposed in the assessment.</p><p>First of all, the CTM method shows levels in Sensitivity (both at nucleotide level, <italic>nSn</italic>, and at site level, <italic>sSn</italic>) only outperformed by our other method, the Statistical GA (Figs. <xref rid="Fig3" ref-type="fig">3</xref> and <xref rid="Fig4" ref-type="fig">4</xref>). It also shows a remarkable Average Site Performance (<italic>sASP</italic>) and, regarding the rest of statistics, even though the numbers obtained are not especially satisfying, they are comparable to most of the other methods.</p><p>Thus, we can already reach the conclusion that topic models are a perfectly valid method to design motif finding algorithms.</p><p>As for the Statistical GA method, Fig. <xref rid="Fig5" ref-type="fig">5</xref> shows the improvement in all of the average statistics after narrowing down the results reported according to the perplexity shown in the CTM. All of the scores for the different statistics are practically doubled after filtering out the motifs for which the perplexity of the corresponding CTM is higher than 100.<fig id="Fig5"><label>Fig. 5</label><caption><p>Comparison of statistics for the Statistical GA method before and after filtering by perplexity. Here it is shown the improvement in the average scores of the Statistical GA method for seven different statistics obtained by the addition of the filtering of the results by their perplexity in the corresponding CTM</p></caption><graphic xlink:href="12859_2016_1364_Fig5_HTML" id="MO5"></graphic></fig></p><p>This tool now clearly outperforms most of the other methods, showing levels of <italic>nSn</italic>, <italic>sSn</italic>, and <italic>sASP</italic> to which any of the other tools can hardly be compared (Figs. <xref rid="Fig3" ref-type="fig">3</xref> and <xref rid="Fig4" ref-type="fig">4</xref>). This further proves the usefulness of topic models for motif discovery tools.</p><p>Given the nature of both methods, and the high number of true positives shown (especially at site level), it seems clear that both succeed in predicting many of the sites but lack of a mechanism to detect false positives. In other words, as the high scores in Sensitivity and Average Site Performance show, both methods can correctly report most of the known motifs, but they locate too many instances of them in the input sequences, so that the number of false positives reported in the assessment, especially at nucleotide level, appears too large, in spite of the correctness of the consensus or the score matrix given by the algorithms as a result. We therefore believe that the high number of false positives is due, to a large extent, to the nature of the assessment. This drawback was considerably reduced in the new version of the statistical GA algorithm, thanks to the use of the perplexity measurement to avoid predicting wrong motifs, but we believe the number of false positives in the assessment could still be shortened if some sort of method was used in the post processing step to obtain only the correct instances of the known site by the use of a weight matrix based on the consensus sequence, instead of simply reporting all of the candidate instances found as both tools currently do. For the CTM method, a way to filter out the results which are not reported with high confidence is required as well.</p><p>The method that gives the best overall statistics after the Statistical GA method is Weeder. As the authors of the assessment clarify [<xref ref-type="bibr" rid="CR1">1</xref>], one of the main reasons for that is the way in which it was run. The author of the tests decided to pick a cautious mode, that is, to predict a motif only if there is a high confidence of its existence. That explains, therefore, the great improvement in the statistics for our Statistical GA tool, and that it is mostly due to the way to calculate the average statistics proposed in the assessment.</p><p>As for the running time, as stated before, it is not an object of the assessment, which is focused on the accuracy of the sites predicted. However, it is worth mentioning that the CTM method slows down considerably when the number of input sequences is bigger than three. Therefore, some solution for this problem, such as dividing the data sets into subgroups of three or fewer sequences, will be required. The Statistical GA method, on the other hand, is able to report the results of data sets of any size in a matter of minutes.</p></sec><sec id="Sec21" sec-type="conclusion"><title>Conclusions</title><p>DNA motif finding still remains as one of the most challenging tasks for researchers, and so it is the task of comparing the performance of the different existing tools, given that each one of them has been designed using very heterogeneous algorithms and models, and that there is still little known about the subjacent biology. Therefore, we must insist on the fact that nowadays it is impossible to define a standard quality measurement to evaluate the performance of the different tools.</p><p>Most of the studies on the performance of motif finding algorithms [<xref ref-type="bibr" rid="CR2">2</xref>] conclude</p><p>that the best option for biologists to try to predict sites in a set of sequences is never to rely on a single tool, but better to use a few complementary tools and combine the top predicted results of each one of them.</p><p>In line with this, we believe that the methods here described, despite their drawbacks, can perfectly be part of those tools that biologists use in combination with others to predict <italic>de novo</italic> binding sites in sets of biological sequences. Especially in the case of the CTM method, there are still many improvements to be done. However, given the results, it can already be used as a ground for future tools based on the use of topic models as a reliable method for motif finding.</p></sec></body><back><app-group><app id="App1"><sec id="Sec22"><title>Additional file</title><p><media position="anchor" xlink:href="12859_2016_1364_MOESM1_ESM.docx" id="MOESM1"><label>Additional file 1:</label><caption><p>This table shows the tools that were studied in the original assessment by Tompa et al. [1] and the two methods presented in this study, each one with a short description of their underlying methodologies. (DOCX 15 kb)</p></caption></media></p></sec></app></app-group><glossary><title>Abbreviations</title><def-list><def-item><term>ASP</term><def><p>Average site performance</p></def></def-item><def-item><term>CTM</term><def><p>Correlated topic model</p></def></def-item><def-item><term>FN</term><def><p>False negative</p></def></def-item><def-item><term>FP</term><def><p>False positive</p></def></def-item><def-item><term>GA</term><def><p>Genetic algorithm</p></def></def-item><def-item><term>TFBS</term><def><p>Transcription factor binding site</p></def></def-item><def-item><term>TN</term><def><p>True negative</p></def></def-item><def-item><term>TP</term><def><p>True positive</p></def></def-item></def-list></glossary><ack><title>Acknowledgements</title><p>We would like to thank Martin Frith of the Computational Biology Research Center at the AIST Tokyo Waterfront Bio-IT Research Building for his contribution to the original development of the Statistical GA method used in this research. We also thank the reviewers of this manuscript for their comments, which certainly helped to improve it and will be useful for future developments.</p><sec id="FPar1"><title>Declarations</title><p>This article has been published as part of BMC Bioinformatics Volume 17 Supplement 19, 2016. 15th International Conference On Bioinformatics (INCOB 2016): bioinformatics. The full contents of the supplement are available online <ext-link ext-link-type="uri" xlink:href="https://bmcbioinformatics.biomedcentral.com/articles/supplements/volume-17-supplement-19">https://bmcbioinformatics.biomedcentral.com/articles/supplements/volume-17-supplement-19</ext-link>.</p></sec><sec id="FPar2"><title>Funding</title><p>Publication of this article was funded by the corresponding author’s institution.</p></sec><sec id="FPar3"><title>Availability of data and materials</title><p>The complete code of both of the methods presented in this article are available at <ext-link ext-link-type="uri" xlink:href="https://github.com/basha-u-tokyo/motif-finding-topic-models">https://github.com/basha-u-tokyo/motif-finding-topic-models</ext-link>.</p><p>The datasets supporting the conclusions of this article are available in the University of Washington repository, provided by Tompa et al. [<xref ref-type="bibr" rid="CR1">1</xref>] in their study to compare the performance of different motif finding methods using several statistical coefficients, <ext-link ext-link-type="uri" xlink:href="http://bio.cs.washington.edu/assessment/">http://bio.cs.washington.edu/assessment/</ext-link>.</p></sec><sec id="FPar4"><title>Authors’ contributions</title><p>JBG conceived and developed the methodology and drafted the manuscript, and KN supervised the whole research. All authors read and approved the final manuscript.</p></sec><sec id="FPar5"><title>Competing interests</title><p>The authors declare that they have no competing interests.</p></sec><sec id="FPar6"><title>Consent for publication</title><p>Not applicable.</p></sec><sec id="FPar7"><title>Ethics approval and consent to participate</title><p>Not applicable.</p></sec></ack><ref-list id="Bib1"><title>References</title><ref id="CR1"><label>1.</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Tompa</surname><given-names>M</given-names></name><name><surname>Li</surname><given-names>N</given-names></name><name><surname>Bailey</surname><given-names>TL</given-names></name><name><surname>Church</surname><given-names>GM</given-names></name><name><surname>De Moor</surname><given-names>B</given-names></name><name><surname>Eskin</surname><given-names>E</given-names></name><name><surname>Favorov</surname><given-names>AV</given-names></name><name><surname>Frith</surname><given-names>MC</given-names></name><name><surname>Fu</surname><given-names>Y</given-names></name><name><surname>Kent</surname><given-names>WJ</given-names></name><etal></etal></person-group><article-title>Assessing computational tools for the discovery of transcription factor binding sites</article-title><source>Nat Biotechnol</source><year>2005</year><volume>23</volume><issue>1</issue><fpage>137</fpage><lpage>47</lpage><pub-id pub-id-type="doi">10.1038/nbt1053</pub-id><pub-id pub-id-type="pmid">15637633</pub-id></element-citation></ref><ref id="CR2"><label>2.</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Das</surname><given-names>MK</given-names></name><name><surname>Dai</surname><given-names>HK</given-names></name></person-group><article-title>A survey of DNA motif finding algorithms</article-title><source>BMC Bioinf.</source><year>2007</year><volume>8</volume><issue>7</issue><fpage>1</fpage></element-citation></ref><ref id="CR3"><label>3.</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Blei</surname><given-names>DM</given-names></name></person-group><article-title>Probabilistic topic models</article-title><source>Commun. ACM</source><year>2012</year><volume>55</volume><issue>4</issue><fpage>77</fpage><lpage>84</lpage><pub-id pub-id-type="doi">10.1145/2133806.2133826</pub-id></element-citation></ref><ref id="CR4"><label>4.</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Blei</surname><given-names>DM</given-names></name><name><surname>Ng</surname><given-names>AY</given-names></name><name><surname>Jordan</surname><given-names>MI</given-names></name></person-group><article-title>Latent dirichlet allocation</article-title><source>J. Mach. Learn. Res.</source><year>2003</year><volume>3</volume><issue>Jan</issue><fpage>993</fpage><lpage>1022</lpage></element-citation></ref><ref id="CR5"><label>5.</label><mixed-citation publication-type="other">Gutierrez JB, Frith M, Nakai K. A Genetic Algorithm for Motif Finding Based on Statistical Significance. In: International Conference on Bioinformatics and Biomedical Engineering. Granada: Springer International Publishing; 2015. p. 438–49.</mixed-citation></ref><ref id="CR6"><label>6.</label><element-citation publication-type="book"><person-group person-group-type="author"><name><surname>Mitchell</surname><given-names>M</given-names></name></person-group><source>An introduction to genetic algorithms</source><year>1996</year><publisher-loc>Cambridge, MA</publisher-loc><publisher-name>MIT Press</publisher-name></element-citation></ref><ref id="CR7"><label>7.</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Blei</surname><given-names>D</given-names></name><name><surname>Lafferty</surname><given-names>J</given-names></name></person-group><article-title>Correlated topic models</article-title><source>Adv. Neural Inf. Proces. Syst.</source><year>2006</year><volume>18</volume><fpage>147</fpage></element-citation></ref><ref id="CR8"><label>8.</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Aitchison</surname><given-names>J</given-names></name></person-group><article-title>The statistical analysis of compositional data</article-title><source>J R Stat Soc B Methodol</source><year>1982</year><volume>44</volume><issue>2</issue><fpage>139</fpage><lpage>77</lpage></element-citation></ref><ref id="CR9"><label>9.</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Hornik</surname><given-names>K</given-names></name><name><surname>Grün</surname><given-names>B</given-names></name></person-group><article-title>topicmodels: An R package for fitting topic models</article-title><source>J. Stat. Softw.</source><year>2011</year><volume>40</volume><issue>13</issue><fpage>1</fpage><lpage>30</lpage></element-citation></ref><ref id="CR10"><label>10.</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Abnizova</surname><given-names>I</given-names></name><name><surname>te Boekhorst</surname><given-names>R</given-names></name><name><surname>Walter</surname><given-names>K</given-names></name><name><surname>Gilks</surname><given-names>WR</given-names></name></person-group><article-title>Some statistical properties of regulatory DNA sequences, and their use in predicting regulatory regions in the Drosophila genome: the fluffy-tail test</article-title><source>BMC Bioinf.</source><year>2005</year><volume>6</volume><issue>1</issue><fpage>109</fpage><pub-id pub-id-type="doi">10.1186/1471-2105-6-109</pub-id></element-citation></ref><ref id="CR11"><label>11.</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Shu</surname><given-names>JJ</given-names></name><name><surname>Li</surname><given-names>Y</given-names></name></person-group><article-title>A statistical thin-tail test of predicting regulatory regions in the Drosophila genome</article-title><source>Theor. Biol. Med. Model.</source><year>2013</year><volume>10</volume><issue>1</issue><fpage>11</fpage><pub-id pub-id-type="doi">10.1186/1742-4682-10-11</pub-id><pub-id pub-id-type="pmid">23409927</pub-id></element-citation></ref><ref id="CR12"><label>12.</label><mixed-citation publication-type="other">Mann HB, Whitney DR. On a test of whether one of two random variables is stochastically larger than the other. Ann Math Stat. 1947;50–60.</mixed-citation></ref><ref id="CR13"><label>13.</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Favorov</surname><given-names>AV</given-names></name><name><surname>Gelfand</surname><given-names>MS</given-names></name><name><surname>Gerasimova</surname><given-names>AV</given-names></name><name><surname>Mironov</surname><given-names>AA</given-names></name><name><surname>Makeev</surname><given-names>VJ</given-names></name></person-group><article-title>Gibbs sampler for identification of symmetrically structured, spaced DNA motifs with improved estimation of the signal length and its validation on the ArcA binding sites</article-title><source>Proc of BGRS</source><year>2004</year><volume>2004</volume><fpage>269</fpage><lpage>272</lpage></element-citation></ref><ref id="CR14"><label>14.</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Pavesi</surname><given-names>G</given-names></name><name><surname>Mereghetti</surname><given-names>P</given-names></name><name><surname>Mauri</surname><given-names>G</given-names></name><name><surname>Pesole</surname><given-names>G</given-names></name></person-group><article-title>Weeder Web: discovery of transcription factor binding sites in a set of sequences from co-regulated genes</article-title><source>Nucleic Acids Res</source><year>2004</year><volume>32</volume><fpage>W199</fpage><lpage>203</lpage><pub-id pub-id-type="doi">10.1093/nar/gkh465</pub-id><pub-id pub-id-type="pmid">15215380</pub-id></element-citation></ref><ref id="CR15"><label>15.</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Sinha</surname><given-names>S</given-names></name><name><surname>Tompa</surname><given-names>M</given-names></name></person-group><article-title>YMF: a program for discovery of novel transcription factor binding sites by statistical overrepresentation</article-title><source>Nucleic Acids Res</source><year>2003</year><volume>31</volume><fpage>3586</fpage><lpage>8</lpage><pub-id pub-id-type="doi">10.1093/nar/gkg618</pub-id><pub-id pub-id-type="pmid">12824371</pub-id></element-citation></ref><ref id="CR16"><label>16.</label><element-citation publication-type="book"><person-group person-group-type="author"><name><surname>Pevzner</surname><given-names>PA</given-names></name><name><surname>Sze</surname><given-names>SH</given-names></name></person-group><article-title>Combinatorial approaches to finding subtle signals in DNA sequences</article-title><source>ISMB</source><year>2000</year><fpage>269</fpage><lpage>278</lpage></element-citation></ref><ref id="CR17"><label>17.</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Burset</surname><given-names>M</given-names></name><name><surname>Guigo</surname><given-names>R</given-names></name></person-group><article-title>Evaluation of gene structure prediction programs</article-title><source>Genomics</source><year>1996</year><volume>34</volume><issue>3</issue><fpage>353</fpage><lpage>67</lpage><pub-id pub-id-type="doi">10.1006/geno.1996.0298</pub-id><pub-id pub-id-type="pmid">8786136</pub-id></element-citation></ref><ref id="CR18"><label>18.</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Wingender</surname><given-names>E</given-names></name><name><surname>Dietze</surname><given-names>P</given-names></name><name><surname>Karas</surname><given-names>H</given-names></name><name><surname>Knüppel</surname><given-names>R</given-names></name></person-group><article-title>TRANSFAC: a Database on transcription factors and their DNA binding sites</article-title><source>Nucleic Acids Res</source><year>1996</year><volume>24</volume><fpage>238</fpage><lpage>41</lpage><pub-id pub-id-type="doi">10.1093/nar/24.1.238</pub-id><pub-id pub-id-type="pmid">8594589</pub-id></element-citation></ref><ref id="CR19"><label>19.</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Hughes</surname><given-names>JD</given-names></name><name><surname>Estep</surname><given-names>PW</given-names></name><name><surname>Tavazoie</surname><given-names>S</given-names></name><name><surname>Church</surname><given-names>GM</given-names></name></person-group><article-title>Computational identification of cis-regulatory elements associated with functionally coherent groups of genes in Saccharomyces cerevisiae</article-title><source>J Mol Biol</source><year>2000</year><volume>296</volume><fpage>1205</fpage><lpage>14</lpage><pub-id pub-id-type="doi">10.1006/jmbi.2000.3519</pub-id><pub-id pub-id-type="pmid">10698627</pub-id></element-citation></ref><ref id="CR20"><label>20.</label><element-citation publication-type="book"><person-group person-group-type="author"><name><surname>Workman</surname><given-names>CT</given-names></name><name><surname>Stormo</surname><given-names>GD</given-names></name></person-group><article-title>ANN-Spec: a method for discovering transcription factor binding sites with improved specificity</article-title><source>Pac Symp Biocomput</source><year>2000</year><fpage>467</fpage><lpage>478</lpage></element-citation></ref><ref id="CR21"><label>21.</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Hertz</surname><given-names>GZ</given-names></name><name><surname>Stormo</surname><given-names>GD</given-names></name></person-group><article-title>Identifying DNA and protein patterns with statistically significant alignments of multiple sequences</article-title><source>Bioinformatics</source><year>1999</year><volume>15</volume><fpage>563</fpage><lpage>77</lpage><pub-id pub-id-type="doi">10.1093/bioinformatics/15.7.563</pub-id><pub-id pub-id-type="pmid">10487864</pub-id></element-citation></ref><ref id="CR22"><label>22.</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Frith</surname><given-names>MC</given-names></name><name><surname>Hansen</surname><given-names>U</given-names></name><name><surname>Spouge</surname><given-names>JL</given-names></name><name><surname>Weng</surname><given-names>Z</given-names></name></person-group><article-title>Finding functional sequence elements by multiple local alignment</article-title><source>Nucleic Acids Res</source><year>2004</year><volume>32</volume><fpage>189</fpage><lpage>200</lpage><pub-id pub-id-type="doi">10.1093/nar/gkh169</pub-id><pub-id pub-id-type="pmid">14704356</pub-id></element-citation></ref><ref id="CR23"><label>23.</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Ao</surname><given-names>W</given-names></name><name><surname>Gaudet</surname><given-names>J</given-names></name><name><surname>Kent</surname><given-names>WJ</given-names></name><name><surname>Muttumu</surname><given-names>S</given-names></name><name><surname>Mango</surname><given-names>SE</given-names></name></person-group><article-title>Environmentally induced foregut remodeling by PHA-4/FoxA and DAF-12/NHR</article-title><source>Science</source><year>2004</year><volume>305</volume><fpage>1743</fpage><lpage>6</lpage><pub-id pub-id-type="doi">10.1126/science.1102216</pub-id><pub-id pub-id-type="pmid">15375261</pub-id></element-citation></ref><ref id="CR24"><label>24.</label><element-citation publication-type="book"><person-group person-group-type="author"><name><surname>Bailey</surname><given-names>TL</given-names></name><name><surname>Elkan</surname><given-names>C</given-names></name></person-group><article-title>The value of prior knowledge in discovering motifs with MEME</article-title><source>Ismb</source><year>1995</year><fpage>21</fpage><lpage>29</lpage></element-citation></ref><ref id="CR25"><label>25.</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Eskin</surname><given-names>E</given-names></name><name><surname>Pevzner</surname><given-names>P</given-names></name></person-group><article-title>Finding composite regulatory patterns in DNA sequences</article-title><source>Bioinformatics</source><year>2002</year><volume>18</volume><issue>suppl 1</issue><fpage>S354</fpage><lpage>63</lpage><pub-id pub-id-type="doi">10.1093/bioinformatics/18.suppl_1.S354</pub-id><pub-id pub-id-type="pmid">12169566</pub-id></element-citation></ref><ref id="CR26"><label>26.</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Thijs</surname><given-names>G</given-names></name><name><surname>Lescot</surname><given-names>M</given-names></name><name><surname>Marchal</surname><given-names>K</given-names></name><name><surname>Rombauts</surname><given-names>S</given-names></name><name><surname>De Moor</surname><given-names>B</given-names></name><name><surname>Rouze</surname><given-names>P</given-names></name><name><surname>Moreau</surname><given-names>Y</given-names></name></person-group><article-title>A higher-order background model improves the detection of promoter regulatory elements by Gibbs sampling</article-title><source>Bioinformatics</source><year>2001</year><volume>17</volume><fpage>1113</fpage><lpage>22</lpage><pub-id pub-id-type="doi">10.1093/bioinformatics/17.12.1113</pub-id><pub-id pub-id-type="pmid">11751219</pub-id></element-citation></ref><ref id="CR27"><label>27.</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>van Helden</surname><given-names>J</given-names></name><name><surname>Andre</surname><given-names>B</given-names></name><name><surname>Collado-Vides</surname><given-names>J</given-names></name></person-group><article-title>Extracting regulatory sites from the upstream region of yeast genes by computational analysis of oligonucleotide frequencies</article-title><source>J Mol Biol</source><year>1998</year><volume>281</volume><fpage>827</fpage><lpage>42</lpage><pub-id pub-id-type="doi">10.1006/jmbi.1998.1947</pub-id><pub-id pub-id-type="pmid">9719638</pub-id></element-citation></ref><ref id="CR28"><label>28.</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>van Helden</surname><given-names>J</given-names></name><name><surname>Rios</surname><given-names>AF</given-names></name><name><surname>Collado-Vides</surname><given-names>J</given-names></name></person-group><article-title>Discovering regulatory elements in noncoding sequences by analysis of spaced dyads</article-title><source>Nucleic Acids Res</source><year>2000</year><volume>28</volume><fpage>1808</fpage><lpage>18</lpage><pub-id pub-id-type="doi">10.1093/nar/28.8.1808</pub-id><pub-id pub-id-type="pmid">10734201</pub-id></element-citation></ref><ref id="CR29"><label>29.</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Régnier</surname><given-names>M</given-names></name><name><surname>Denise</surname><given-names>A</given-names></name></person-group><article-title>Rare events and conditional events on random strings</article-title><source>Discrete Math Theor Comput Sci</source><year>2004</year><volume>6</volume><fpage>191</fpage><lpage>214</lpage></element-citation></ref></ref-list></back></pmc></record>Pour manipuler ce document sous Unix (Dilib)
EXPLOR_STEP=$WICRI_ROOT/Sante/explor/MersV1/Data/Pmc/Corpus
HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 000263 | SxmlIndent | more
Ou
HfdSelect -h $EXPLOR_AREA/Data/Pmc/Corpus/biblio.hfd -nk 000263 | SxmlIndent | more
Pour mettre un lien sur cette page dans le réseau Wicri
{{Explor lien
|wiki= Sante
|area= MersV1
|flux= Pmc
|étape= Corpus
|type= RBID
|clé=
|texte=
}}
| This area was generated with Dilib version V0.6.33. |  |