MersV1, Main, Exploration, bibRecord, 001108

Minimum Entropy Rate Simplification of Stochastic Processes.

Identifieur interne : 001108 ( Main/Exploration ); précédent : 001107; suivant : 001109

Minimum Entropy Rate Simplification of Stochastic Processes.

Auteurs : Gustav Eje Henter ; W Bastiaan Kleijn

Source :

IEEE transactions on pattern analysis and machine intelligence [ 1939-3539 ] ; 2016.

RBID : pubmed:26929031

Descripteurs français

KwdFr :
- Algorithmes, Animaux, Apprentissage machine, Entropie, Humains, Mesures de production de la parole (), Modèles biologiques, Modèles statistiques, Reconnaissance automatique des formes, Simulation numérique, Spectrographie sonore ().
MESH :
- Algorithmes, Animaux, Apprentissage machine, Entropie, Humains, Mesures de production de la parole, Modèles biologiques, Modèles statistiques, Reconnaissance automatique des formes, Simulation numérique, Spectrographie sonore.

English descriptors

KwdEn :
- Algorithms, Animals, Computer Simulation, Entropy, Humans, Machine Learning, Models, Biological, Models, Statistical, Pattern Recognition, Automated, Sound Spectrography (methods), Speech Production Measurement (methods).
MESH :
- methods : Sound Spectrography, Speech Production Measurement.
- Algorithms, Animals, Computer Simulation, Entropy, Humans, Machine Learning, Models, Biological, Models, Statistical, Pattern Recognition, Automated.

Abstract

We propose minimum entropy rate simplification (MERS), an information-theoretic, parameterization-independent framework for simplifying generative models of stochastic processes. Applications include improving model quality for sampling tasks by concentrating the probability mass on the most characteristic and accurately described behaviors while de-emphasizing the tails, and obtaining clean models from corrupted data (nonparametric denoising). This is the opposite of the smoothing step commonly applied to classification models. Drawing on rate-distortion theory, MERS seeks the minimum entropy-rate process under a constraint on the dissimilarity between the original and simplified processes. We particularly investigate the Kullback-Leibler divergence rate as a dissimilarity measure, where, compatible with our assumption that the starting model is disturbed or inaccurate, the simplification rather than the starting model is used for the reference distribution of the divergence. This leads to analytic solutions for stationary and ergodic Gaussian processes and Markov chains. The same formulas are also valid for maximum-entropy smoothing under the same divergence constraint. In experiments, MERS successfully simplifies and denoises models from audio, text, speech, and meteorology.

DOI: 10.1109/TPAMI.2016.2533382
PubMed: 26929031

Affiliations:

Le document en format XML

<record><TEI><teiHeader><fileDesc><titleStmt><title xml:lang="en">Minimum Entropy Rate Simplification of Stochastic Processes.</title>
<author><name sortKey="Henter, Gustav Eje" sort="Henter, Gustav Eje" uniqKey="Henter G" first="Gustav Eje" last="Henter">Gustav Eje Henter</name>
</author>
<author><name sortKey="Kleijn, W Bastiaan" sort="Kleijn, W Bastiaan" uniqKey="Kleijn W" first="W Bastiaan" last="Kleijn">W Bastiaan Kleijn</name>
</author>
</titleStmt>
<publicationStmt><idno type="wicri:source">PubMed</idno>
<date when="2016">2016</date>
<idno type="RBID">pubmed:26929031</idno>
<idno type="pmid">26929031</idno>
<idno type="doi">10.1109/TPAMI.2016.2533382</idno>
<idno type="wicri:Area/PubMed/Corpus">001230</idno>
<idno type="wicri:explorRef" wicri:stream="PubMed" wicri:step="Corpus" wicri:corpus="PubMed">001230</idno>
<idno type="wicri:Area/PubMed/Curation">001230</idno>
<idno type="wicri:explorRef" wicri:stream="PubMed" wicri:step="Curation">001230</idno>
<idno type="wicri:Area/PubMed/Checkpoint">000F75</idno>
<idno type="wicri:explorRef" wicri:stream="Checkpoint" wicri:step="PubMed">000F75</idno>
<idno type="wicri:Area/Ncbi/Merge">001509</idno>
<idno type="wicri:Area/Ncbi/Curation">001509</idno>
<idno type="wicri:Area/Ncbi/Checkpoint">001509</idno>
<idno type="wicri:Area/Main/Merge">001112</idno>
<idno type="wicri:Area/Main/Curation">001108</idno>
<idno type="wicri:Area/Main/Exploration">001108</idno>
</publicationStmt>
<sourceDesc><biblStruct><analytic><title xml:lang="en">Minimum Entropy Rate Simplification of Stochastic Processes.</title>
<author><name sortKey="Henter, Gustav Eje" sort="Henter, Gustav Eje" uniqKey="Henter G" first="Gustav Eje" last="Henter">Gustav Eje Henter</name>
</author>
<author><name sortKey="Kleijn, W Bastiaan" sort="Kleijn, W Bastiaan" uniqKey="Kleijn W" first="W Bastiaan" last="Kleijn">W Bastiaan Kleijn</name>
</author>
</analytic>
<series><title level="j">IEEE transactions on pattern analysis and machine intelligence</title>
<idno type="eISSN">1939-3539</idno>
<imprint><date when="2016" type="published">2016</date>
</imprint>
</series>
</biblStruct>
</sourceDesc>
</fileDesc>
<profileDesc><textClass><keywords scheme="KwdEn" xml:lang="en"><term>Algorithms</term>
<term>Animals</term>
<term>Computer Simulation</term>
<term>Entropy</term>
<term>Humans</term>
<term>Machine Learning</term>
<term>Models, Biological</term>
<term>Models, Statistical</term>
<term>Pattern Recognition, Automated</term>
<term>Sound Spectrography (methods)</term>
<term>Speech Production Measurement (methods)</term>
</keywords>
<keywords scheme="KwdFr" xml:lang="fr"><term>Algorithmes</term>
<term>Animaux</term>
<term>Apprentissage machine</term>
<term>Entropie</term>
<term>Humains</term>
<term>Mesures de production de la parole ()</term>
<term>Modèles biologiques</term>
<term>Modèles statistiques</term>
<term>Reconnaissance automatique des formes</term>
<term>Simulation numérique</term>
<term>Spectrographie sonore ()</term>
</keywords>
<keywords scheme="MESH" qualifier="methods" xml:lang="en"><term>Sound Spectrography</term>
<term>Speech Production Measurement</term>
</keywords>
<keywords scheme="MESH" xml:lang="en"><term>Algorithms</term>
<term>Animals</term>
<term>Computer Simulation</term>
<term>Entropy</term>
<term>Humans</term>
<term>Machine Learning</term>
<term>Models, Biological</term>
<term>Models, Statistical</term>
<term>Pattern Recognition, Automated</term>
</keywords>
<keywords scheme="MESH" xml:lang="fr"><term>Algorithmes</term>
<term>Animaux</term>
<term>Apprentissage machine</term>
<term>Entropie</term>
<term>Humains</term>
<term>Mesures de production de la parole</term>
<term>Modèles biologiques</term>
<term>Modèles statistiques</term>
<term>Reconnaissance automatique des formes</term>
<term>Simulation numérique</term>
<term>Spectrographie sonore</term>
</keywords>
</textClass>
</profileDesc>
</teiHeader>
<front><div type="abstract" xml:lang="en">We propose minimum entropy rate simplification (MERS), an information-theoretic, parameterization-independent framework for simplifying generative models of stochastic processes. Applications include improving model quality for sampling tasks by concentrating the probability mass on the most characteristic and accurately described behaviors while de-emphasizing the tails, and obtaining clean models from corrupted data (nonparametric denoising). This is the opposite of the smoothing step commonly applied to classification models. Drawing on rate-distortion theory, MERS seeks the minimum entropy-rate process under a constraint on the dissimilarity between the original and simplified processes. We particularly investigate the Kullback-Leibler divergence rate as a dissimilarity measure, where, compatible with our assumption that the starting model is disturbed or inaccurate, the simplification rather than the starting model is used for the reference distribution of the divergence. This leads to analytic solutions for stationary and ergodic Gaussian processes and Markov chains. The same formulas are also valid for maximum-entropy smoothing under the same divergence constraint. In experiments, MERS successfully simplifies and denoises models from audio, text, speech, and meteorology.</div>
</front>
</TEI>
<affiliations><list></list>
<tree><noCountry><name sortKey="Henter, Gustav Eje" sort="Henter, Gustav Eje" uniqKey="Henter G" first="Gustav Eje" last="Henter">Gustav Eje Henter</name>
<name sortKey="Kleijn, W Bastiaan" sort="Kleijn, W Bastiaan" uniqKey="Kleijn W" first="W Bastiaan" last="Kleijn">W Bastiaan Kleijn</name>
</noCountry>
</tree>
</affiliations>
</record>

Pour manipuler ce document sous Unix (Dilib)

EXPLOR_STEP=$WICRI_ROOT/Sante/explor/MersV1/Data/Main/Exploration

HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 001108 | SxmlIndent | more

HfdSelect -h $EXPLOR_AREA/Data/Main/Exploration/biblio.hfd -nk 001108 | SxmlIndent | more

Pour mettre un lien sur cette page dans le réseau Wicri

{{Explor lien
   |wiki=    Sante
   |area=    MersV1
   |flux=    Main
   |étape=   Exploration
   |type=    RBID
   |clé=     pubmed:26929031
   |texte=   Minimum Entropy Rate Simplification of Stochastic Processes.
}}

Pour générer des pages wiki

HfdIndexSelect -h $EXPLOR_AREA/Data/Main/Exploration/RBID.i   -Sk "pubmed:26929031" \
       | HfdSelect -Kh $EXPLOR_AREA/Data/Main/Exploration/biblio.hfd   \
       | NlmPubMed2Wicri -a MersV1

This area was generated with Dilib version V0.6.33.
Data generation: Mon Apr 20 23:26:43 2020. Site generation: Sat Mar 27 09:06:09 2021

Serveur d'exploration MERS

Minimum Entropy Rate Simplification of Stochastic Processes.

Minimum Entropy Rate Simplification of Stochastic Processes.

Source :

Descripteurs français

English descriptors

Abstract

Links toward previous steps (curation, corpus...)

Le document en format XML

Pour manipuler ce document sous Unix (Dilib)

Pour mettre un lien sur cette page dans le réseau Wicri

Pour générer des pages wiki

	Serveur d'exploration MERS
	Attention, ce site est en cours de développement ! Attention, site généré par des moyens informatiques à partir de corpus bruts. Les informations ne sont donc pas validées.