Change search
ReferencesLink to record
Permanent link

Direct link
Continuous Interaction with a Virtual Human
KTH, School of Computer Science and Communication (CSC), Speech, Music and Hearing, TMH, Speech Communication and Technology.
Show others and affiliations
2011 (English)In: Journal on Multimodal User Interfaces, ISSN 1783-7677, Vol. 4, no 2, 97-118 p.Article in journal (Refereed) Published
Abstract [en]

This paper presents our progress in developing a Virtual Human capable of being an attentive speaker. Such a Virtual Human should be able to attend to its interaction partner while it is speaking-and modify its communicative behavior on-the-fly based on what it observes in the behavior of its partner. We report new developments concerning a number of aspects, such as scheduling and interrupting multimodal behavior, automatic classification of listener responses, generation of response eliciting behavior, and strategies for generating appropriate reactions to listener responses. On the basis of this progress, a task-based setup for a responsive Virtual Human was implemented to carry out two user studies, the results of which are presented and discussed in this paper.

Place, publisher, year, edition, pages
2011. Vol. 4, no 2, 97-118 p.
Keyword [en]
Attentive speaking, Continuous interaction, Listener responses, Virtual humans
National Category
Computer Science Language Technology (Computational Linguistics)
URN: urn:nbn:se:kth:diva-52194DOI: 10.1007/s12193-011-0060-xISI: 000309997100004ScopusID: 2-s2.0-80955180056OAI: diva2:465492

tmh_import_11_12_14. QC 20111215

Available from: 2011-12-14 Created: 2011-12-14 Last updated: 2013-04-16Bibliographically approved
In thesis
1. Modelling Paralinguistic Conversational Interaction: Towards social awareness in spoken human-machine dialogue
Open this publication in new window or tab >>Modelling Paralinguistic Conversational Interaction: Towards social awareness in spoken human-machine dialogue
2012 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Parallel with the orthographic streams of words in conversation are multiple layered epiphenomena, short in duration and with a communicativepurpose. These paralinguistic events regulate the interaction flow via gaze,gestures and intonation. This thesis focus on how to compute, model, discoverand analyze prosody and it’s applications for spoken dialog systems.Specifically it addresses automatic classification and analysis of conversationalcues related to turn-taking, brief feedback, affective expressions, their crossrelationshipsas well as their cognitive and neurological basis. Techniques areproposed for instantaneous and suprasegmental parameterization of scalarand vector valued representations of fundamental frequency, but also intensity and voice quality. Examples are given for how to engineer supervised learned automata’s for off-line processing of conversational corpora as well as for incremental on-line processing with low-latency constraints suitable as detector modules in a responsive social interface. Specific attention is given to the communicative functions of vocal feedback like "mhm", "okay" and "yeah, that’s right" as postulated by the theories of grounding, emotion and a survey on laymen opinions. The potential functions and their prosodic cues are investigated via automatic decoding, data-mining, exploratory visualization and descriptive measurements.

Place, publisher, year, edition, pages
Stockholm: KTH Royal Institute of Technology, 2012. xiv, 86 p.
Trita-CSC-A, ISSN 1653-5723 ; 2012:08
National Category
Language Technology (Computational Linguistics)
urn:nbn:se:kth:diva-102335 (URN)978-91-7501-467-8 (ISBN)
Public defence
2012-09-28, Sal F3, Lindstedtsvägen 26, KTH, Stockholm, 13:00 (English)

QC 20120914

Available from: 2012-09-14 Created: 2012-09-14 Last updated: 2012-09-14Bibliographically approved

Open Access in DiVA

No full text

Other links

Publisher's full textScopus

Search in DiVA

By author/editor
Neiberg, Daniel
By organisation
Speech Communication and Technology
In the same journal
Journal on Multimodal User Interfaces
Computer ScienceLanguage Technology (Computational Linguistics)

Search outside of DiVA

GoogleGoogle Scholar
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

Altmetric score

Total: 71 hits
ReferencesLink to record
Permanent link

Direct link