SynFace-Speech-Driven Facial Animation for Virtual Speech-Reading Support
2009 (English)In: Eurasip Journal on Audio, Speech, and Music Processing, ISSN 1687-4714, Vol. 2009, 191940- p.Article in journal (Refereed) Published
This paper describes SynFace, a supportive technology that aims at enhancing audio-based spoken communication in adverse acoustic conditions by providing the missing visual information in the form of an animated talking head. Firstly, we describe the system architecture, consisting of a 3D animated face model controlled from the speech input by a specifically optimised phonetic recogniser. Secondly, we report on speech intelligibility experiments with focus on multilinguality and robustness to audio quality. The system, already available for Swedish, English, and Flemish, was optimised for German and for Swedish wide-band speech quality available in TV, radio, and Internet communication. Lastly, the paper covers experiments with nonverbal motions driven from the speech signal. It is shown that turn-taking gestures can be used to affect the flow of human-human dialogues. We have focused specifically on two categories of cues that may be extracted from the acoustic signal: prominence/emphasis and interactional cues (turn-taking/back-channelling).
Place, publisher, year, edition, pages
2009. Vol. 2009, 191940- p.
RECOGNITION, IMPLEMENTATION, THRESHOLD, MODELS
Specific Languages Computer and Information Science Fluid Mechanics and Acoustics
IdentifiersURN: urn:nbn:se:kth:diva-28195DOI: 10.1155/2009/191940ISI: 000285145100001ScopusID: 2-s2.0-76649097032OAI: oai:DiVA.org:kth-28195DiVA: diva2:384801
FunderEU, FP7, Seventh Framework Programme, IST-045089Swedish Research Council, 621-2005-3488
QC 201101102011-01-102011-01-102011-01-10Bibliographically approved