A New Method Based on Context for Combining Statistical Language Models
Identifieur interne : 002F06 ( Crin/Curation ); précédent : 002F05; suivant : 002F07A New Method Based on Context for Combining Statistical Language Models
Auteurs : David Langlois ; Kamel Smaïli ; Jean-Paul HatonSource :
English descriptors
Abstract
In this paper we propose a new method to extract from a corpus the histories for which a given language model is better than another one. The decision is based on a measure stemmed from perplexity. This measure allows, for a given history, to compare two language models, and then to choose the best one for this history. Using this principle, and with a 20K vocabulary words, we combined two language models : a bigram and a distant bigram. The contribution of a distant bigram is significant and outperforms a bigram model by 7.5%. Moreover, the performance in Shannon game are improved. We show through this article that we proposed a cheaper framework in comparison to the maximum entropy principle, for combining language models. In addition, the selected histories for which a model is better than another one, have been collected and studied. Almost, all of them are beginnings of very frequently used French phrases. Finally, by using this principle, we achieve a better trigram model in terms of parameters and perplexity. This model is a combination of a bigram and a trigram based on a selected history.
Links toward previous steps (curation, corpus...)
- to stream Crin, to step Corpus: Pour aller vers cette notice dans l'étape Curation :002F06
Links to Exploration step
CRIN:langlois01aLe document en format XML
<record><TEI><teiHeader><fileDesc><titleStmt><title xml:lang="en" wicri:score="438">A New Method Based on Context for Combining Statistical Language Models</title></titleStmt><publicationStmt><idno type="RBID">CRIN:langlois01a</idno><date when="2001" year="2001">2001</date><idno type="wicri:Area/Crin/Corpus">002F06</idno><idno type="wicri:Area/Crin/Curation">002F06</idno><idno type="wicri:explorRef" wicri:stream="Crin" wicri:step="Curation">002F06</idno></publicationStmt><sourceDesc><biblStruct><analytic><title xml:lang="en">A New Method Based on Context for Combining Statistical Language Models</title><author><name sortKey="Langlois, David" sort="Langlois, David" uniqKey="Langlois D" first="David" last="Langlois">David Langlois</name></author><author><name sortKey="Smaili, Kamel" sort="Smaili, Kamel" uniqKey="Smaili K" first="Kamel" last="Smaïli">Kamel Smaïli</name></author><author><name sortKey="Haton, Jean Paul" sort="Haton, Jean Paul" uniqKey="Haton J" first="Jean-Paul" last="Haton">Jean-Paul Haton</name></author></analytic></biblStruct></sourceDesc></fileDesc><profileDesc><textClass><keywords scheme="KwdEn" xml:lang="en"><term>combination</term><term>distant models</term><term>statistical language modeling</term></keywords></textClass></profileDesc></teiHeader><front><div type="abstract" xml:lang="en" wicri:score="3746">In this paper we propose a new method to extract from a corpus the histories for which a given language model is better than another one. The decision is based on a measure stemmed from perplexity. This measure allows, for a given history, to compare two language models, and then to choose the best one for this history. Using this principle, and with a 20K vocabulary words, we combined two language models : a bigram and a distant bigram. The contribution of a distant bigram is significant and outperforms a bigram model by 7.5%. Moreover, the performance in Shannon game are improved. We show through this article that we proposed a cheaper framework in comparison to the maximum entropy principle, for combining language models. In addition, the selected histories for which a model is better than another one, have been collected and studied. Almost, all of them are beginnings of very frequently used French phrases. Finally, by using this principle, we achieve a better trigram model in terms of parameters and perplexity. This model is a combination of a bigram and a trigram based on a selected history.</div></front></TEI><BibTex type="inproceedings"><ref>langlois01a</ref><crinnumber>A01-R-179</crinnumber><category>3</category><equipe>PAROLE</equipe><author><e>Langlois, David</e><e>Smaïli, Kamel</e><e>Haton, Jean-Paul</e></author><title>A New Method Based on Context for Combining Statistical Language Models</title><booktitle>{Third International Conference on Modeling and Using Context - CONTEXT 01, Dundee, Scotland}</booktitle><year>2001</year><editor>Varol Akman, Paolo Bouquet, Richmond Thomason, Roger A. Young</editor><volume>2116</volume><series>Lecture Notes in Artificial Intelligence</series><pages>235-247</pages><month>Jul</month><publisher>Springer</publisher><keywords><e>statistical language modeling</e><e>distant models</e><e>combination</e></keywords><abstract>In this paper we propose a new method to extract from a corpus the histories for which a given language model is better than another one. The decision is based on a measure stemmed from perplexity. This measure allows, for a given history, to compare two language models, and then to choose the best one for this history. Using this principle, and with a 20K vocabulary words, we combined two language models : a bigram and a distant bigram. The contribution of a distant bigram is significant and outperforms a bigram model by 7.5%. Moreover, the performance in Shannon game are improved. We show through this article that we proposed a cheaper framework in comparison to the maximum entropy principle, for combining language models. In addition, the selected histories for which a model is better than another one, have been collected and studied. Almost, all of them are beginnings of very frequently used French phrases. Finally, by using this principle, we achieve a better trigram model in terms of parameters and perplexity. This model is a combination of a bigram and a trigram based on a selected history.</abstract></BibTex></record>Pour manipuler ce document sous Unix (Dilib)
EXPLOR_STEP=$WICRI_ROOT/Wicri/Lorraine/explor/InforLorV4/Data/Crin/Curation
HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 002F06 | SxmlIndent | more
Ou
HfdSelect -h $EXPLOR_AREA/Data/Crin/Curation/biblio.hfd -nk 002F06 | SxmlIndent | more
Pour mettre un lien sur cette page dans le réseau Wicri
{{Explor lien
|wiki= Wicri/Lorraine
|area= InforLorV4
|flux= Crin
|étape= Curation
|type= RBID
|clé= CRIN:langlois01a
|texte= A New Method Based on Context for Combining Statistical Language Models
}}
| This area was generated with Dilib version V0.6.33. |  |