Statistical Language Modeling Based on Variable-Length Sequences
Identifieur interne : 007A78 ( Main/Merge ); précédent : 007A77; suivant : 007A79Statistical Language Modeling Based on Variable-Length Sequences
Auteurs : Imed Zitouni ; Kamel Smaili ; Jean-Paul Haton [France]Source :
- Computer Speech and Language ; 2003.
English descriptors
- KwdEn :
Abstract
In natural language and especially in spontaneous speech, people often group words in order to constitute phrases which become usual expressions. This is due to phonological (to make the pronunciation easier), or to semantic reasons (to remember more easily a phrase by assigning a meaning to a block of words). Classical language models do not adequately take into account such phrases. A better approach consists in modeling some word sequences as if they were individual dictionary elements. Sequences are considered as additional entries of the vocabulary, on which language models are computed. In this paper, we present a method for automatically retrieving the most relevant phrases from a corpus of written sentences. The originality of our approach resides in the fact that the extracted phrases are obtained from a linguistically tagged corpus. Therefore, the obtained phrases are linguistically viable. To measure the contribution of classes in retrieving phrases, we have implemented the same algorithm without using classes. The class-based method outperformed by 11% the other method. Our approach uses information theoretic criteria which insure a high statistical consistency and make the decision of selecting a potential sequence optimal in accordance with the language perplexity. We propose several variants of language model with and without word sequences. Among them, we present a model in which the trigger pairs are linguistically more significant. We show that the use of sequences decrease the word error rate and improve the normalized perplexity. For instance, the best sequence model improves the perplexity by 16%, and the accuracy of our dictation system (MAUD) by approximately 14%. Experiments, in terms of perplexity and recognition rate, have been carried out on a vocabulary of 20000 words extracted from a corpus of 43 million words made up of two years of the French newspaper Le Monde. The acoustic model (HMM) is trained with the Bref80 corpus.
Links toward previous steps (curation, corpus...)
- to stream Crin, to step Corpus: 003773
- to stream Crin, to step Curation: 003773
- to stream Crin, to step Checkpoint: 000C31
Links to Exploration step
CRIN:zitouni03aLe document en format XML
<record><TEI><teiHeader><fileDesc><titleStmt><title xml:lang="en" wicri:score="293">Statistical Language Modeling Based on Variable-Length Sequences</title></titleStmt><publicationStmt><idno type="RBID">CRIN:zitouni03a</idno><date when="2003" year="2003">2003</date><idno type="wicri:Area/Crin/Corpus">003773</idno><idno type="wicri:Area/Crin/Curation">003773</idno><idno type="wicri:explorRef" wicri:stream="Crin" wicri:step="Curation">003773</idno><idno type="wicri:Area/Crin/Checkpoint">000C31</idno><idno type="wicri:explorRef" wicri:stream="Crin" wicri:step="Checkpoint">000C31</idno><idno type="wicri:Area/Main/Merge">007A78</idno></publicationStmt><sourceDesc><biblStruct><analytic><title xml:lang="en">Statistical Language Modeling Based on Variable-Length Sequences</title><author><name sortKey="Zitouni, Imed" sort="Zitouni, Imed" uniqKey="Zitouni I" first="Imed" last="Zitouni">Imed Zitouni</name></author><author><name sortKey="Smaili, Kamel" sort="Smaili, Kamel" uniqKey="Smaili K" first="Kamel" last="Smaili">Kamel Smaili</name></author><author><name sortKey="Haton, Jean Paul" sort="Haton, Jean Paul" uniqKey="Haton J" first="Jean-Paul" last="Haton">Jean-Paul Haton</name><affiliation><country>France</country><placeName><settlement type="city">Nancy</settlement><region type="region" nuts="2">Grand Est</region><region type="region" nuts="2">Lorraine (région)</region></placeName><orgName type="laboratoire" n="5">Laboratoire lorrain de recherche en informatique et ses applications</orgName><orgName type="university">Université de Lorraine</orgName><orgName type="institution">Centre national de la recherche scientifique</orgName><orgName type="institution">Institut national de recherche en informatique et en automatique</orgName></affiliation></author></analytic><series><title level="j">Computer Speech and Language</title><imprint><date when="2003" type="published">2003</date></imprint></series></biblStruct></sourceDesc></fileDesc><profileDesc><textClass><keywords scheme="KwdEn" xml:lang="en"><term>cache</term><term>language modeling</term><term>normalized perplexity</term><term>phrases</term><term>sequences</term><term>triggres</term></keywords></textClass></profileDesc></teiHeader><front><div type="abstract" xml:lang="en" wicri:score="3856">In natural language and especially in spontaneous speech, people often group words in order to constitute phrases which become usual expressions. This is due to phonological (to make the pronunciation easier), or to semantic reasons (to remember more easily a phrase by assigning a meaning to a block of words). Classical language models do not adequately take into account such phrases. A better approach consists in modeling some word sequences as if they were individual dictionary elements. Sequences are considered as additional entries of the vocabulary, on which language models are computed. In this paper, we present a method for automatically retrieving the most relevant phrases from a corpus of written sentences. The originality of our approach resides in the fact that the extracted phrases are obtained from a linguistically tagged corpus. Therefore, the obtained phrases are linguistically viable. To measure the contribution of classes in retrieving phrases, we have implemented the same algorithm without using classes. The class-based method outperformed by 11% the other method. Our approach uses information theoretic criteria which insure a high statistical consistency and make the decision of selecting a potential sequence optimal in accordance with the language perplexity. We propose several variants of language model with and without word sequences. Among them, we present a model in which the trigger pairs are linguistically more significant. We show that the use of sequences decrease the word error rate and improve the normalized perplexity. For instance, the best sequence model improves the perplexity by 16%, and the accuracy of our dictation system (MAUD) by approximately 14%. Experiments, in terms of perplexity and recognition rate, have been carried out on a vocabulary of 20000 words extracted from a corpus of 43 million words made up of two years of the French newspaper Le Monde. The acoustic model (HMM) is trained with the Bref80 corpus.</div></front></TEI></record>Pour manipuler ce document sous Unix (Dilib)
EXPLOR_STEP=$WICRI_ROOT/Wicri/Lorraine/explor/InforLorV4/Data/Main/Merge
HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 007A78 | SxmlIndent | more
Ou
HfdSelect -h $EXPLOR_AREA/Data/Main/Merge/biblio.hfd -nk 007A78 | SxmlIndent | more
Pour mettre un lien sur cette page dans le réseau Wicri
{{Explor lien
|wiki= Wicri/Lorraine
|area= InforLorV4
|flux= Main
|étape= Merge
|type= RBID
|clé= CRIN:zitouni03a
|texte= Statistical Language Modeling Based on Variable-Length Sequences
}}
| This area was generated with Dilib version V0.6.33. |  |