A tool for the synchronization of speech and mouth shapes : LIPS
Identifieur interne : 00A117 ( Main/Merge ); précédent : 00A116; suivant : 00A118A tool for the synchronization of speech and mouth shapes : LIPS
Auteurs : Odile Mella ; Dominique Fohr ; Laurent Martin ; Andreas CarlenSource :
English descriptors
- KwdEn :
Abstract
This paper presents a new approach to improve the phoneme-based lipsync process. The lipsync process is a step in the animation production pipelines of 2D and 3D cartoons. It consists in generatingthe mouth positions of a cartoon character from the dialogue recorded by an actor. The result of this step is a sequence of time markerswhich indicate the series of mouth shapes to be drawn : for instance an "opened" mouth when a "a" is uttered and a "closed" mouth for a "p". Until now, the lipsync phase has been done by hand : experts listen to the audio tape and write on an exposure sheet the shapes of the mouth and their timing. This traditional method is tedious and time consuming, almost one day for 4 minutes. We propose to speed up the lipsync process using tools coming from the field of automatic speech recognition. A standard tool of labelling cannot be used because the voices of the cartoon's characters are atypical : shouted voice, child voice, foreign accent, whispered speech, voice of animal characters, laugh... We describe the LIPS tool (LIPS Logiciel Interactif de PostSynchronisation : lipsync Interactive Software) and the generation of the acoustic models required by the tool.
Links toward previous steps (curation, corpus...)
- to stream Crin, to step Corpus: 002B75
- to stream Crin, to step Curation: 002B75
- to stream Crin, to step Checkpoint: 001A83
Links to Exploration step
CRIN:mella00aLe document en format XML
<record><TEI><teiHeader><fileDesc><titleStmt><title xml:lang="en" wicri:score="269">A tool for the synchronization of speech and mouth shapes : LIPS</title></titleStmt><publicationStmt><idno type="RBID">CRIN:mella00a</idno><date when="2000" year="2000">2000</date><idno type="wicri:Area/Crin/Corpus">002B75</idno><idno type="wicri:Area/Crin/Curation">002B75</idno><idno type="wicri:explorRef" wicri:stream="Crin" wicri:step="Curation">002B75</idno><idno type="wicri:Area/Crin/Checkpoint">001A83</idno><idno type="wicri:explorRef" wicri:stream="Crin" wicri:step="Checkpoint">001A83</idno><idno type="wicri:Area/Main/Merge">00A117</idno></publicationStmt><sourceDesc><biblStruct><analytic><title xml:lang="en">A tool for the synchronization of speech and mouth shapes : LIPS</title><author><name sortKey="Mella, Odile" sort="Mella, Odile" uniqKey="Mella O" first="Odile" last="Mella">Odile Mella</name></author><author><name sortKey="Fohr, Dominique" sort="Fohr, Dominique" uniqKey="Fohr D" first="Dominique" last="Fohr">Dominique Fohr</name></author><author><name sortKey="Martin, Laurent" sort="Martin, Laurent" uniqKey="Martin L" first="Laurent" last="Martin">Laurent Martin</name></author><author><name sortKey="Carlen, Andreas" sort="Carlen, Andreas" uniqKey="Carlen A" first="Andreas" last="Carlen">Andreas Carlen</name></author></analytic></biblStruct></sourceDesc></fileDesc><profileDesc><textClass><keywords scheme="KwdEn" xml:lang="en"><term>hmm</term><term>lipsync</term><term>speech alignment</term><term>speech recognition</term></keywords></textClass></profileDesc></teiHeader><front><div type="abstract" xml:lang="en" wicri:score="3369">This paper presents a new approach to improve the phoneme-based lipsync process. The lipsync process is a step in the animation production pipelines of 2D and 3D cartoons. It consists in generatingthe mouth positions of a cartoon character from the dialogue recorded by an actor. The result of this step is a sequence of time markerswhich indicate the series of mouth shapes to be drawn : for instance an "opened" mouth when a "a" is uttered and a "closed" mouth for a "p". Until now, the lipsync phase has been done by hand : experts listen to the audio tape and write on an exposure sheet the shapes of the mouth and their timing. This traditional method is tedious and time consuming, almost one day for 4 minutes. We propose to speed up the lipsync process using tools coming from the field of automatic speech recognition. A standard tool of labelling cannot be used because the voices of the cartoon's characters are atypical : shouted voice, child voice, foreign accent, whispered speech, voice of animal characters, laugh... We describe the LIPS tool (LIPS Logiciel Interactif de PostSynchronisation : lipsync Interactive Software) and the generation of the acoustic models required by the tool.</div></front></TEI></record>Pour manipuler ce document sous Unix (Dilib)
EXPLOR_STEP=$WICRI_ROOT/Wicri/Lorraine/explor/InforLorV4/Data/Main/Merge
HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 00A117 | SxmlIndent | more
Ou
HfdSelect -h $EXPLOR_AREA/Data/Main/Merge/biblio.hfd -nk 00A117 | SxmlIndent | more
Pour mettre un lien sur cette page dans le réseau Wicri
{{Explor lien
|wiki= Wicri/Lorraine
|area= InforLorV4
|flux= Main
|étape= Merge
|type= RBID
|clé= CRIN:mella00a
|texte= A tool for the synchronization of speech and mouth shapes : LIPS
}}
| This area was generated with Dilib version V0.6.33. |  |