Effective Interaction with Talking Animated Agents in Dialogue Systems
2005 (English)In: Advances in Natural Multimodal Dialogue Systems / [ed] van Kuppevelt, J.; Dybkjaer, L.; Bernsen, N. O., Springer Netherlands, 2005, 215-243 p.Chapter in book (Refereed)
At the Centre for Speech Technology at KTH, we have for the past several years been developing spoken dialogue applications that include animated talking agents. Our motivation for moving into audiovisual output is to investigate the advantages of multimodality in human-system communication. While the mainstream character animation area has focussed on the naturalness and realism of the animated agents, our primary concern has been the possible increase of intelligibility and efficiency of interaction resulting from the addition of a talking face. In our first dialogue system, Waxholm, the agent used the deictic function of indicating specific information on the screen by eye gaze. In another project, Synface, we were specifically concerned with the advantages in intelligibility that a talking face could provide. In recent studies we have investigated the use of facial gesture cues to convey such dialogue-related functions as feedback and turn-taking as well as prosodic functions such as prominence. Results show that cues such as eyebrow and head movement can independently signal prominence. Current results also indicate that there can be considerable differences in cue strengths among visual cues such as smiling and nodding and that such cues can contribute in an additive manner together with auditory prosody as cues to different dialogue functions. Results from some of these studies are presented in the chapter along with examples of spoken dialogue applications using talking heads.
Place, publisher, year, edition, pages
Springer Netherlands, 2005. 215-243 p.
Audio-visual speech synthesis, talking heads, animated agents, spoken dialogue systems, visual prosody
Computer Science Language Technology (Computational Linguistics)
IdentifiersURN: urn:nbn:se:kth:diva-51888DOI: 10.1007/1-4020-3933-6_10OAI: oai:DiVA.org:kth-51888DiVA: diva2:465182
tmh_import_11_12_14. QC 201112292011-12-142011-12-142011-12-29Bibliographically approved