Open this publication in new window or tab >>2013 (English)In: Speech Communication, ISSN 0167-6393, E-ISSN 1872-7182, Vol. 55, no 3, p. 451-469Article in journal (Refereed) Published
Abstract [en]
This paper proposes methods for exploring acoustic correlates to feedback functions. A sub-language of Swedish, simple productive feedback, is introduced to facilitate investigations of the functional contributions of base tokens, phonological operations and prosody. The function of feedback is to convey the listeners' attention, understanding and affective states. In order to handle the large number of possible affective states, the current study starts by performing a listening experiment where humans annotated the functional similarity of feedback tokens with different prosodic realizations. By selecting a set of stimuli that had different prosodic distances from a reference token, it was possible to compute a generalised functional distance measure. The resulting generalised functional distance measure showed to be correlated to prosodic distance but the correlations varied as a function of base tokens and phonological operations. In a subsequent listening test, a small representative sample of feedback tokens were rated for understanding, agreement, interest, surprise and certainty. These ratings were found to explain a significant proportion of the generalised functional distance. By combining the acoustic analysis with an explorative visualisation of the prosody, we have established a map between human perception of similarity between feedback tokens, their measured distance in acoustic space, and the link to the perception of the function of feedback tokens with varying realisations.
Keywords
social signal processing, affective annotation, feedback modeling, grounding 2000 MSC
National Category
Language Technology (Computational Linguistics)
Identifiers
urn:nbn:se:kth:diva-102334 (URN)10.1016/j.specom.2012.12.007 (DOI)000316837000005 ()2-s2.0-84875460872 (Scopus ID)
Projects
SAMSYNTIURO
Funder
Swedish Research Council, 2009-4291EU, European Research Council, FP7 – 248314
Note
QC 20130508
2012-09-142012-09-142024-03-18Bibliographically approved