Change search
ReferencesLink to record
Permanent link

Direct link
From acoustic cues to an expressive agent
KTH, School of Computer Science and Communication (CSC), Speech, Music and Hearing, TMH.ORCID iD: 0000-0002-3086-0322
2006 (English)In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) / [ed] Gibet, S; Courty, N; Kamp, JF, 2006, Vol. 3881, 280-291 p.Conference paper (Refereed)
Abstract [en]

This work proposes a new way for providing feedback to expressivity in music performance. Starting from studies on the expressivity of music performance we developed a system in which a visual feedback is given to the user using a graphical representation of a human face. The first part of the system, previously developed by researchers at KTH Stockholm and at the University of Uppsala, allows the real-time extraction and analysis of acoustic cues from the music performance. Cues extracted are: sound level, tempo, articulation, attack time, and spectrum energy. From these cues the system provides an high level interpretation of the emotional intention of the performer which will be classified into one basic emotion, such as happiness, sadness, or anger. We have implemented an interface between that system and the embodied conversational agent Greta, developed at the University of Rome "La Sapienza" and "University of Paris 8". We model expressivity of the facial animation of the agent with a set of six dimensions that characterize the manner of behavior execution. In this paper we will first describe a mapping between the acoustic cues and the expressivity dimensions of the face. Then we will show how to determine the facial expression corresponding to the emotional intention resulting from the acoustic analysis, using music sound level and tempo characteristics to control the intensity and the temporal variation of muscular activation.

Place, publisher, year, edition, pages
2006. Vol. 3881, 280-291 p.
, Lecture Notes In Artificial Intelligence, ISSN 0302-9743 ; 3881
National Category
Computer Science Human Computer Interaction Computer Vision and Robotics (Autonomous Systems) Psychology Music
URN: urn:nbn:se:kth:diva-42029DOI: 10.1007/11678816_31ISI: 000237042600031ScopusID: 2-s2.0-33745549699ISBN: 3-540-32624-3OAI: diva2:446085
6th International Workshop on Gesture in Human-Computer Interaction and Simulation Location: Berder Isl, France, Date: MAY 18-20, 2005

QC 20150708

Available from: 2011-10-06 Created: 2011-10-05 Last updated: 2015-07-08Bibliographically approved

Open Access in DiVA

No full text

Other links

Publisher's full textScopus

Search in DiVA

By author/editor
Bresin, Roberto
By organisation
Speech, Music and Hearing, TMH
Computer ScienceHuman Computer InteractionComputer Vision and Robotics (Autonomous Systems)PsychologyMusic

Search outside of DiVA

GoogleGoogle Scholar
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

Altmetric score

Total: 24 hits
ReferencesLink to record
Permanent link

Direct link