Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Investigating Communicative Feedback Phenomena across Languages and Modalities
KTH, School of Computer Science and Communication (CSC), Speech, Music and Hearing, TMH.
2007 (English)Doctoral thesis, monograph (Other scientific)
Abstract [en]

This thesis deals with human communicative behaviour related to feedback, analysed across languages (Italian and Swedish), modalities (auditory versus visual) and different communicative situations (human-human versus human-machine dialogues).

The aim of this study is to give more insight into how humans use communicative behaviour related to feedback and at the same time to suggest a method to collect valuable data that can be useful to control facial and head movements related to visual feedback in synthetic conversational agents. The study of human communicative behaviour necessitates the good quality of the materials under analysis, the support of reliable software packages for the audio-visual analysis and a specific coding scheme for the annotation of the phenomena under observation.

The materials used for the investigations presented in this thesis span from spontaneous conversations video recorded in real communicative situations, and semi-spontaneous dialogues obtained with different eliciting techniques, such as map-task and information-seeking scenarios, to a specific corpus of controlled interactive speech collected by means of a motion capture system. When motion caption is used it is possible to register facial and head movements with a high degree of precision, so as to obtain valuable data useful for the implementation of facial displays in talking heads.

A specific coding scheme has been developed, tested and used to annotate feedback. The annotation has been carried out with the support of different available software packages for audio-visual analysis.

The procedure followed in this thesis involves initial analyses of communicative phenomena in spontaneous human-human dialogues and human-machine interaction, in order to learn about regularities in human communicative behaviour that could be transferred to talking heads, then, for the sake of reproduction in talking heads, the investigation includes more detailed analyses of data collected in a lab environment with a novel acquisition set-up that allows capturing the dynamics of facial and head movements.

Finally the possibilities of transferring human communicative behaviour to a talking face are discussed and some evaluation paradigms are illustrated. The idea of reproducing human behaviour in talking heads is based on the assumption that the reproduction of facial displays related to communicative phenomena such as turn management, feedback production and expression of emotions in embodied conversational agents, might result in the design of advanced systems capable of effective multi-modal interactions with humans.

Place, publisher, year, edition, pages
Stockholm: KTH , 2007. , xx, 229 p.
Series
Trita-CSC-A, ISSN 1653-5723 ; 2007:3
National Category
General Language Studies and Linguistics
Identifiers
URN: urn:nbn:se:kth:diva-4362ISBN: 978-91-7178-632-6 (print)OAI: oai:DiVA.org:kth-4362DiVA: diva2:11962
Public defence
2007-05-15, Sal F3, KTH, Lindstedtsvägen 26, Stockholm, 15:00
Opponent
Supervisors
Note
QC 20100819Available from: 2007-05-08 Created: 2007-05-08 Last updated: 2010-08-19Bibliographically approved

Open Access in DiVA

fulltext(2221 kB)401 downloads
File information
File name FULLTEXT01.pdfFile size 2221 kBChecksum MD5
1452f604db8b2d390ff90d85a035d40204ab08195fac391460befbdf5e0395787801525c
Type fulltextMimetype application/pdf

Search in DiVA

By author/editor
Sundberg Cerrato, Loredana
By organisation
Speech, Music and Hearing, TMH
General Language Studies and Linguistics

Search outside of DiVA

GoogleGoogle Scholar
Total: 401 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

isbn
urn-nbn

Altmetric score

isbn
urn-nbn
Total: 901 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf