Change search
ReferencesLink to record
Permanent link

Direct link
Investigating Communicative Feedback Phenomena across Languages and Modalities
KTH, School of Computer Science and Communication (CSC), Speech, Music and Hearing, TMH.
2007 (English)Doctoral thesis, monograph (Other scientific)
Abstract [en]

This thesis deals with human communicative behaviour related to feedback, analysed across languages (Italian and Swedish), modalities (auditory versus visual) and different communicative situations (human-human versus human-machine dialogues).

The aim of this study is to give more insight into how humans use communicative behaviour related to feedback and at the same time to suggest a method to collect valuable data that can be useful to control facial and head movements related to visual feedback in synthetic conversational agents. The study of human communicative behaviour necessitates the good quality of the materials under analysis, the support of reliable software packages for the audio-visual analysis and a specific coding scheme for the annotation of the phenomena under observation.

The materials used for the investigations presented in this thesis span from spontaneous conversations video recorded in real communicative situations, and semi-spontaneous dialogues obtained with different eliciting techniques, such as map-task and information-seeking scenarios, to a specific corpus of controlled interactive speech collected by means of a motion capture system. When motion caption is used it is possible to register facial and head movements with a high degree of precision, so as to obtain valuable data useful for the implementation of facial displays in talking heads.

A specific coding scheme has been developed, tested and used to annotate feedback. The annotation has been carried out with the support of different available software packages for audio-visual analysis.

The procedure followed in this thesis involves initial analyses of communicative phenomena in spontaneous human-human dialogues and human-machine interaction, in order to learn about regularities in human communicative behaviour that could be transferred to talking heads, then, for the sake of reproduction in talking heads, the investigation includes more detailed analyses of data collected in a lab environment with a novel acquisition set-up that allows capturing the dynamics of facial and head movements.

Finally the possibilities of transferring human communicative behaviour to a talking face are discussed and some evaluation paradigms are illustrated. The idea of reproducing human behaviour in talking heads is based on the assumption that the reproduction of facial displays related to communicative phenomena such as turn management, feedback production and expression of emotions in embodied conversational agents, might result in the design of advanced systems capable of effective multi-modal interactions with humans.

Place, publisher, year, edition, pages
Stockholm: KTH , 2007. , xx, 229 p.
Trita-CSC-A, ISSN 1653-5723 ; 2007:3
National Category
General Language Studies and Linguistics
URN: urn:nbn:se:kth:diva-4362ISBN: 978-91-7178-632-6OAI: diva2:11962
Public defence
2007-05-15, Sal F3, KTH, Lindstedtsvägen 26, Stockholm, 15:00
QC 20100819Available from: 2007-05-08 Created: 2007-05-08 Last updated: 2010-08-19Bibliographically approved

Open Access in DiVA

fulltext(2221 kB)270 downloads
File information
File name FULLTEXT01.pdfFile size 2221 kBChecksum SHA-1
Type fulltextMimetype application/pdf

Search in DiVA

By author/editor
Sundberg Cerrato, Loredana
By organisation
Speech, Music and Hearing, TMH
General Language Studies and Linguistics

Search outside of DiVA

GoogleGoogle Scholar
Total: 270 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

Total: 839 hits
ReferencesLink to record
Permanent link

Direct link