Interactive sonification of expressive hand gestures on a handheld device
2012 (English)In: Journal on Multimodal User Interfaces, ISSN 1783-7677, Vol. 6, no 1-2, 49-57 p.Article in journal (Refereed) Published
We present here a mobile phone application called MoodifierLive which aims at using expressive music performances for the sonification of expressive gestures through the mapping of the phone’s accelerometer data to the performance parameters (i.e. tempo, sound level, and articulation). The application, and in particular the sonification principle, is described in detail. An experiment was carried out to evaluate the perceived matching between the gesture and the music performance that it produced, using two distinct mappings between gestures and performance. The results show that the application produces consistent performances, and that the mapping based on data collected from real gestures works better than one defined a priori by the authors.
Place, publisher, year, edition, pages
2012. Vol. 6, no 1-2, 49-57 p.
Automatic music performance, Emotional hand gestures, Mobile phone, Sonification
Computer and Information Science Human Computer Interaction Psychology
IdentifiersURN: urn:nbn:se:kth:diva-34084DOI: 10.1007/s12193-011-0076-2ISI: 000309998800006ScopusID: 2-s2.0-84863200318OAI: oai:DiVA.org:kth-34084DiVA: diva2:419080
FunderSwedish Research Council, 2010-4654EU, FP7, Seventh Framework Programme, FP7-ICT-STREP-215749
QC 20120809. Updated from submitted to published.2011-05-252011-05-252013-09-10Bibliographically approved