< Back to previous page


Optimized Photorealistic Audiovisual Speech Synthesis Using Active Appearance Modeling

Book Contribution - Book Chapter Conference Contribution

Active appearance models can represent image information in terms of shape and texture parameters. This paper explains why this makes them highly suitable for data-based 2D audiovisual text-to-speech synthesis. We elaborate on how the differentiation between shape and texture information can be fully exploited to create appropriate unit-selection costs and to enhance the video concatenations. The latter is very important since for the synthetic visual speech a careful balancing between signal smoothness and articulation strength is required. Several optimization strategies to enhance the quality of the synthetic visual speech are proposed. By measuring the properties of each model parameter, an effective normalization of the visual speech database is feasible. In addition, the visual joins can be optimized by a parameter-specific concatenation smoothing. To further enhance the naturalness of the synthetic speech, a spectrum-based smoothing approach is introduced.
Book: International Conference on Auditory-visual Speech Processing 2010, Hakone, Japan
Pages: 148-153
Number of pages: 6
Publication year:2010
Keywords:audiovisual speech synthesis, AAM modeling
  • Scopus Id: 84865387939