Semi-supervised methods for exploring the acoustics of simple productive feedback
2013 (English)In: Speech Communication, ISSN 0167-6393, Vol. 55, no 3, 451-469 p.Article in journal (Refereed) Published
This paper proposes methods for exploring acoustic correlates to feedback functions. A sub-language of Swedish, simple productive feedback, is introduced to facilitate investigations of the functional contributions of base tokens, phonological operations and prosody. The function of feedback is to convey the listeners' attention, understanding and affective states. In order to handle the large number of possible affective states, the current study starts by performing a listening experiment where humans annotated the functional similarity of feedback tokens with different prosodic realizations. By selecting a set of stimuli that had different prosodic distances from a reference token, it was possible to compute a generalised functional distance measure. The resulting generalised functional distance measure showed to be correlated to prosodic distance but the correlations varied as a function of base tokens and phonological operations. In a subsequent listening test, a small representative sample of feedback tokens were rated for understanding, agreement, interest, surprise and certainty. These ratings were found to explain a significant proportion of the generalised functional distance. By combining the acoustic analysis with an explorative visualisation of the prosody, we have established a map between human perception of similarity between feedback tokens, their measured distance in acoustic space, and the link to the perception of the function of feedback tokens with varying realisations.
Place, publisher, year, edition, pages
2013. Vol. 55, no 3, 451-469 p.
social signal processing, affective annotation, feedback modeling, grounding 2000 MSC
Language Technology (Computational Linguistics)
IdentifiersURN: urn:nbn:se:kth:diva-102334DOI: 10.1016/j.specom.2012.12.007ISI: 000316837000005ScopusID: 2-s2.0-84875460872OAI: oai:DiVA.org:kth-102334DiVA: diva2:552375
FunderSwedish Research Council, 2009-4291EU, European Research Council, FP7 – 248314
QC 201305082012-09-142012-09-142013-05-08Bibliographically approved