From acoustic cues to an expressive agent
2006 (English)In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) / [ed] Gibet, S; Courty, N; Kamp, JF, 2006, Vol. 3881, 280-291 p.Conference paper (Refereed)
This work proposes a new way for providing feedback to expressivity in music performance. Starting from studies on the expressivity of music performance we developed a system in which a visual feedback is given to the user using a graphical representation of a human face. The first part of the system, previously developed by researchers at KTH Stockholm and at the University of Uppsala, allows the real-time extraction and analysis of acoustic cues from the music performance. Cues extracted are: sound level, tempo, articulation, attack time, and spectrum energy. From these cues the system provides an high level interpretation of the emotional intention of the performer which will be classified into one basic emotion, such as happiness, sadness, or anger. We have implemented an interface between that system and the embodied conversational agent Greta, developed at the University of Rome "La Sapienza" and "University of Paris 8". We model expressivity of the facial animation of the agent with a set of six dimensions that characterize the manner of behavior execution. In this paper we will first describe a mapping between the acoustic cues and the expressivity dimensions of the face. Then we will show how to determine the facial expression corresponding to the emotional intention resulting from the acoustic analysis, using music sound level and tempo characteristics to control the intensity and the temporal variation of muscular activation.
Place, publisher, year, edition, pages
2006. Vol. 3881, 280-291 p.
, Lecture Notes In Artificial Intelligence, ISSN 0302-9743 ; 3881
Computer Science Human Computer Interaction Computer Vision and Robotics (Autonomous Systems) Psychology Music
IdentifiersURN: urn:nbn:se:kth:diva-42029DOI: 10.1007/11678816_31ISI: 000237042600031ScopusID: 2-s2.0-33745549699ISBN: 3-540-32624-3OAI: oai:DiVA.org:kth-42029DiVA: diva2:446085
6th International Workshop on Gesture in Human-Computer Interaction and Simulation Location: Berder Isl, France, Date: MAY 18-20, 2005
QC 201507082011-10-062011-10-052015-07-08Bibliographically approved