Minimum Entropy Rate Simplification of Stochastic Processes.
Identifieur interne : 001108 ( Main/Exploration ); précédent : 001107; suivant : 001109Minimum Entropy Rate Simplification of Stochastic Processes.
Auteurs : Gustav Eje Henter ; W Bastiaan KleijnSource :
- IEEE transactions on pattern analysis and machine intelligence [ 1939-3539 ] ; 2016.
Descripteurs français
- KwdFr :
- MESH :
English descriptors
- KwdEn :
- MESH :
Abstract
We propose minimum entropy rate simplification (MERS), an information-theoretic, parameterization-independent framework for simplifying generative models of stochastic processes. Applications include improving model quality for sampling tasks by concentrating the probability mass on the most characteristic and accurately described behaviors while de-emphasizing the tails, and obtaining clean models from corrupted data (nonparametric denoising). This is the opposite of the smoothing step commonly applied to classification models. Drawing on rate-distortion theory, MERS seeks the minimum entropy-rate process under a constraint on the dissimilarity between the original and simplified processes. We particularly investigate the Kullback-Leibler divergence rate as a dissimilarity measure, where, compatible with our assumption that the starting model is disturbed or inaccurate, the simplification rather than the starting model is used for the reference distribution of the divergence. This leads to analytic solutions for stationary and ergodic Gaussian processes and Markov chains. The same formulas are also valid for maximum-entropy smoothing under the same divergence constraint. In experiments, MERS successfully simplifies and denoises models from audio, text, speech, and meteorology.
DOI: 10.1109/TPAMI.2016.2533382
PubMed: 26929031
Affiliations:
Links toward previous steps (curation, corpus...)
- to stream PubMed, to step Corpus: 001230
- to stream PubMed, to step Curation: 001230
- to stream PubMed, to step Checkpoint: 000F75
- to stream Ncbi, to step Merge: 001509
- to stream Ncbi, to step Curation: 001509
- to stream Ncbi, to step Checkpoint: 001509
- to stream Main, to step Merge: 001112
- to stream Main, to step Curation: 001108
Le document en format XML
<record><TEI><teiHeader><fileDesc><titleStmt><title xml:lang="en">Minimum Entropy Rate Simplification of Stochastic Processes.</title>
<author><name sortKey="Henter, Gustav Eje" sort="Henter, Gustav Eje" uniqKey="Henter G" first="Gustav Eje" last="Henter">Gustav Eje Henter</name>
</author>
<author><name sortKey="Kleijn, W Bastiaan" sort="Kleijn, W Bastiaan" uniqKey="Kleijn W" first="W Bastiaan" last="Kleijn">W Bastiaan Kleijn</name>
</author>
</titleStmt>
<publicationStmt><idno type="wicri:source">PubMed</idno>
<date when="2016">2016</date>
<idno type="RBID">pubmed:26929031</idno>
<idno type="pmid">26929031</idno>
<idno type="doi">10.1109/TPAMI.2016.2533382</idno>
<idno type="wicri:Area/PubMed/Corpus">001230</idno>
<idno type="wicri:explorRef" wicri:stream="PubMed" wicri:step="Corpus" wicri:corpus="PubMed">001230</idno>
<idno type="wicri:Area/PubMed/Curation">001230</idno>
<idno type="wicri:explorRef" wicri:stream="PubMed" wicri:step="Curation">001230</idno>
<idno type="wicri:Area/PubMed/Checkpoint">000F75</idno>
<idno type="wicri:explorRef" wicri:stream="Checkpoint" wicri:step="PubMed">000F75</idno>
<idno type="wicri:Area/Ncbi/Merge">001509</idno>
<idno type="wicri:Area/Ncbi/Curation">001509</idno>
<idno type="wicri:Area/Ncbi/Checkpoint">001509</idno>
<idno type="wicri:Area/Main/Merge">001112</idno>
<idno type="wicri:Area/Main/Curation">001108</idno>
<idno type="wicri:Area/Main/Exploration">001108</idno>
</publicationStmt>
<sourceDesc><biblStruct><analytic><title xml:lang="en">Minimum Entropy Rate Simplification of Stochastic Processes.</title>
<author><name sortKey="Henter, Gustav Eje" sort="Henter, Gustav Eje" uniqKey="Henter G" first="Gustav Eje" last="Henter">Gustav Eje Henter</name>
</author>
<author><name sortKey="Kleijn, W Bastiaan" sort="Kleijn, W Bastiaan" uniqKey="Kleijn W" first="W Bastiaan" last="Kleijn">W Bastiaan Kleijn</name>
</author>
</analytic>
<series><title level="j">IEEE transactions on pattern analysis and machine intelligence</title>
<idno type="eISSN">1939-3539</idno>
<imprint><date when="2016" type="published">2016</date>
</imprint>
</series>
</biblStruct>
</sourceDesc>
</fileDesc>
<profileDesc><textClass><keywords scheme="KwdEn" xml:lang="en"><term>Algorithms</term>
<term>Animals</term>
<term>Computer Simulation</term>
<term>Entropy</term>
<term>Humans</term>
<term>Machine Learning</term>
<term>Models, Biological</term>
<term>Models, Statistical</term>
<term>Pattern Recognition, Automated</term>
<term>Sound Spectrography (methods)</term>
<term>Speech Production Measurement (methods)</term>
</keywords>
<keywords scheme="KwdFr" xml:lang="fr"><term>Algorithmes</term>
<term>Animaux</term>
<term>Apprentissage machine</term>
<term>Entropie</term>
<term>Humains</term>
<term>Mesures de production de la parole ()</term>
<term>Modèles biologiques</term>
<term>Modèles statistiques</term>
<term>Reconnaissance automatique des formes</term>
<term>Simulation numérique</term>
<term>Spectrographie sonore ()</term>
</keywords>
<keywords scheme="MESH" qualifier="methods" xml:lang="en"><term>Sound Spectrography</term>
<term>Speech Production Measurement</term>
</keywords>
<keywords scheme="MESH" xml:lang="en"><term>Algorithms</term>
<term>Animals</term>
<term>Computer Simulation</term>
<term>Entropy</term>
<term>Humans</term>
<term>Machine Learning</term>
<term>Models, Biological</term>
<term>Models, Statistical</term>
<term>Pattern Recognition, Automated</term>
</keywords>
<keywords scheme="MESH" xml:lang="fr"><term>Algorithmes</term>
<term>Animaux</term>
<term>Apprentissage machine</term>
<term>Entropie</term>
<term>Humains</term>
<term>Mesures de production de la parole</term>
<term>Modèles biologiques</term>
<term>Modèles statistiques</term>
<term>Reconnaissance automatique des formes</term>
<term>Simulation numérique</term>
<term>Spectrographie sonore</term>
</keywords>
</textClass>
</profileDesc>
</teiHeader>
<front><div type="abstract" xml:lang="en">We propose minimum entropy rate simplification (MERS), an information-theoretic, parameterization-independent framework for simplifying generative models of stochastic processes. Applications include improving model quality for sampling tasks by concentrating the probability mass on the most characteristic and accurately described behaviors while de-emphasizing the tails, and obtaining clean models from corrupted data (nonparametric denoising). This is the opposite of the smoothing step commonly applied to classification models. Drawing on rate-distortion theory, MERS seeks the minimum entropy-rate process under a constraint on the dissimilarity between the original and simplified processes. We particularly investigate the Kullback-Leibler divergence rate as a dissimilarity measure, where, compatible with our assumption that the starting model is disturbed or inaccurate, the simplification rather than the starting model is used for the reference distribution of the divergence. This leads to analytic solutions for stationary and ergodic Gaussian processes and Markov chains. The same formulas are also valid for maximum-entropy smoothing under the same divergence constraint. In experiments, MERS successfully simplifies and denoises models from audio, text, speech, and meteorology.</div>
</front>
</TEI>
<affiliations><list></list>
<tree><noCountry><name sortKey="Henter, Gustav Eje" sort="Henter, Gustav Eje" uniqKey="Henter G" first="Gustav Eje" last="Henter">Gustav Eje Henter</name>
<name sortKey="Kleijn, W Bastiaan" sort="Kleijn, W Bastiaan" uniqKey="Kleijn W" first="W Bastiaan" last="Kleijn">W Bastiaan Kleijn</name>
</noCountry>
</tree>
</affiliations>
</record>
Pour manipuler ce document sous Unix (Dilib)
EXPLOR_STEP=$WICRI_ROOT/Sante/explor/MersV1/Data/Main/Exploration
HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 001108 | SxmlIndent | more
Ou
HfdSelect -h $EXPLOR_AREA/Data/Main/Exploration/biblio.hfd -nk 001108 | SxmlIndent | more
Pour mettre un lien sur cette page dans le réseau Wicri
{{Explor lien |wiki= Sante |area= MersV1 |flux= Main |étape= Exploration |type= RBID |clé= pubmed:26929031 |texte= Minimum Entropy Rate Simplification of Stochastic Processes. }}
Pour générer des pages wiki
HfdIndexSelect -h $EXPLOR_AREA/Data/Main/Exploration/RBID.i -Sk "pubmed:26929031" \ | HfdSelect -Kh $EXPLOR_AREA/Data/Main/Exploration/biblio.hfd \ | NlmPubMed2Wicri -a MersV1
![]() | This area was generated with Dilib version V0.6.33. | ![]() |