Change search
Link to record
Permanent link

Direct link
BETA
Publications (10 of 116) Show all publications
Székely, É., Henter, G. E., Beskow, J. & Gustafson, J. (2019). How to train your fillers: uh and um in spontaneous speech synthesis. In: : . Paper presented at The 10th ISCA Speech Synthesis Workshop.
Open this publication in new window or tab >>How to train your fillers: uh and um in spontaneous speech synthesis
2019 (English)Conference paper, Published paper (Refereed)
National Category
Engineering and Technology
Identifiers
urn:nbn:se:kth:diva-261693 (URN)
Conference
The 10th ISCA Speech Synthesis Workshop
Note

QC 20191011

Available from: 2019-10-10 Created: 2019-10-10 Last updated: 2019-10-11Bibliographically approved
Jonell, P., Kucherenko, T., Ekstedt, E. & Beskow, J. (2019). Learning Non-verbal Behavior for a Social Robot from YouTube Videos. In: : . Paper presented at ICDL-EpiRob Workshop on Naturalistic Non-Verbal and Affective Human-Robot Interactions, Oslo, Norway, August 19, 2019.
Open this publication in new window or tab >>Learning Non-verbal Behavior for a Social Robot from YouTube Videos
2019 (English)Conference paper, Poster (with or without abstract) (Refereed)
Abstract [en]

Non-verbal behavior is crucial for positive perception of humanoid robots. If modeled well it can improve the interaction and leave the user with a positive experience, on the other hand, if it is modelled poorly it may impede the interaction and become a source of distraction. Most of the existing work on modeling non-verbal behavior show limited variability due to the fact that the models employed are deterministic and the generated motion can be perceived as repetitive and predictable. In this paper, we present a novel method for generation of a limited set of facial expressions and head movements, based on a probabilistic generative deep learning architecture called Glow. We have implemented a workflow which takes videos directly from YouTube, extracts relevant features, and trains a model that generates gestures that can be realized in a robot without any post processing. A user study was conducted and illustrated the importance of having any kind of non-verbal behavior while most differences between the ground truth, the proposed method, and a random control were not significant (however, the differences that were significant were in favor of the proposed method).

Keywords
Facial expressions, non-verbal behavior, generative models, neural network, head movement, social robotics
National Category
Computer Systems
Research subject
Computer Science
Identifiers
urn:nbn:se:kth:diva-261242 (URN)
Conference
ICDL-EpiRob Workshop on Naturalistic Non-Verbal and Affective Human-Robot Interactions, Oslo, Norway, August 19, 2019
Funder
Swedish Foundation for Strategic Research , RIT15-0107
Note

QC 20191007

Available from: 2019-10-03 Created: 2019-10-03 Last updated: 2019-10-07Bibliographically approved
Stefanov, K., Salvi, G., Kontogiorgos, D., Kjellström, H. & Beskow, J. (2019). Modeling of Human Visual Attention in Multiparty Open-World Dialogues. ACM TRANSACTIONS ON HUMAN-ROBOT INTERACTION, 8(2), Article ID UNSP 8.
Open this publication in new window or tab >>Modeling of Human Visual Attention in Multiparty Open-World Dialogues
Show others...
2019 (English)In: ACM TRANSACTIONS ON HUMAN-ROBOT INTERACTION, ISSN 2573-9522, Vol. 8, no 2, article id UNSP 8Article in journal (Refereed) Published
Abstract [en]

This study proposes, develops, and evaluates methods for modeling the eye-gaze direction and head orientation of a person in multiparty open-world dialogues, as a function of low-level communicative signals generated by his/hers interlocutors. These signals include speech activity, eye-gaze direction, and head orientation, all of which can be estimated in real time during the interaction. By utilizing these signals and novel data representations suitable for the task and context, the developed methods can generate plausible candidate gaze targets in real time. The methods are based on Feedforward Neural Networks and Long Short-Term Memory Networks. The proposed methods are developed using several hours of unrestricted interaction data and their performance is compared with a heuristic baseline method. The study offers an extensive evaluation of the proposed methods that investigates the contribution of different predictors to the accurate generation of candidate gaze targets. The results show that the methods can accurately generate candidate gaze targets when the person being modeled is in a listening state. However, when the person being modeled is in a speaking state, the proposed methods yield significantly lower performance.

Place, publisher, year, edition, pages
ASSOC COMPUTING MACHINERY, 2019
Keywords
Human-human interaction, open-world dialogue, eye-gaze direction, head orientation, multiparty
National Category
Computer Vision and Robotics (Autonomous Systems)
Identifiers
urn:nbn:se:kth:diva-255203 (URN)10.1145/3323231 (DOI)000472066800003 ()
Note

QC 20190904

Available from: 2019-09-04 Created: 2019-09-04 Last updated: 2019-10-15Bibliographically approved
Malisz, Z., Henter, G. E., Valentini-Botinhao, C., Watts, O., Beskow, J. & Gustafson, J. (2019). Modern speech synthesis for phonetic sciences: A discussion and an evaluation. In: Proceedings of ICPhS: . Paper presented at International Congress of Phonetic Sciences.
Open this publication in new window or tab >>Modern speech synthesis for phonetic sciences: A discussion and an evaluation
Show others...
2019 (English)In: Proceedings of ICPhS, 2019Conference paper, Published paper (Refereed)
National Category
Language Technology (Computational Linguistics)
Identifiers
urn:nbn:se:kth:diva-260956 (URN)
Conference
International Congress of Phonetic Sciences
Note

QC 20191112

Available from: 2019-09-30 Created: 2019-09-30 Last updated: 2019-11-12Bibliographically approved
Skantze, G., Gustafson, J. & Beskow, J. (2019). Multimodal Conversational Interaction with Robots. In: Sharon Oviatt, Björn Schuller, Philip R. Cohen, Daniel Sonntag, Gerasimos Potamianos, Antonio Krüger (Ed.), The Handbook of Multimodal-Multisensor Interfaces, Volume 3: Language Processing, Software, Commercialization, and Emerging Directions. ACM Press
Open this publication in new window or tab >>Multimodal Conversational Interaction with Robots
2019 (English)In: The Handbook of Multimodal-Multisensor Interfaces, Volume 3: Language Processing, Software, Commercialization, and Emerging Directions / [ed] Sharon Oviatt, Björn Schuller, Philip R. Cohen, Daniel Sonntag, Gerasimos Potamianos, Antonio Krüger, ACM Press, 2019Chapter in book (Refereed)
Place, publisher, year, edition, pages
ACM Press, 2019
National Category
Human Computer Interaction
Identifiers
urn:nbn:se:kth:diva-254650 (URN)9781970001723 (ISBN)
Note

QC 20190821

Available from: 2019-07-02 Created: 2019-07-02 Last updated: 2019-08-21Bibliographically approved
Székely, É., Henter, G. E., Beskow, J. & Gustafson, J. (2019). Off the cuff: Exploring extemporaneous speech delivery with TTS. In: : . Paper presented at Interspeech.
Open this publication in new window or tab >>Off the cuff: Exploring extemporaneous speech delivery with TTS
2019 (English)Conference paper, Published paper (Refereed)
National Category
Engineering and Technology
Identifiers
urn:nbn:se:kth:diva-261691 (URN)
Conference
Interspeech
Note

QC 20191011

Available from: 2019-10-10 Created: 2019-10-10 Last updated: 2019-10-11Bibliographically approved
Székely, É., Henter, G. E., Beskow, J. & Gustafson, J. (2019). Off the cuff: Exploring extemporaneous speech delivery with TTS. In: : . Paper presented at The 20th Annual Conference of the International Speech Communication Association INTERSPEECH 2019 | Graz, Austria, Sep. 15-19, 2019. (pp. 3687-3688).
Open this publication in new window or tab >>Off the cuff: Exploring extemporaneous speech delivery with TTS
2019 (English)Conference paper, Published paper (Refereed)
National Category
Computer Sciences Language Technology (Computational Linguistics)
Identifiers
urn:nbn:se:kth:diva-260957 (URN)
Conference
The 20th Annual Conference of the International Speech Communication Association INTERSPEECH 2019 | Graz, Austria, Sep. 15-19, 2019.
Note

QC 20191113

Available from: 2019-09-30 Created: 2019-09-30 Last updated: 2019-11-13Bibliographically approved
Stefanov, K. (2019). Self-Supervised Vision-Based Detection of the Active Speaker as Support for Socially-Aware Language Acquisition. IEEE Transactions on Cognitive and Developmental Systems
Open this publication in new window or tab >>Self-Supervised Vision-Based Detection of the Active Speaker as Support for Socially-Aware Language Acquisition
2019 (English)In: IEEE Transactions on Cognitive and Developmental Systems, ISSN 2379-8920Article in journal (Refereed) Published
Abstract [en]

This paper presents a self-supervised method for visual detection of the active speaker in a multi-person spoken interaction scenario. Active speaker detection is a fundamental prerequisite for any artificial cognitive system attempting to acquire language in social settings. The proposed method is intended to complement the acoustic detection of the active speaker, thus improving the system robustness in noisy conditions. The method can detect an arbitrary number of possibly overlapping active speakers based exclusively on visual information about their face. Furthermore, the method does not rely on external annotations, thus complying with cognitive development. Instead, the method uses information from the auditory modality to support learning in the visual domain. This paper reports an extensive evaluation of the proposed method using a large multi-person face-to-face interaction dataset. The results show good performance in a speaker dependent setting. However, in a speaker independent setting the proposed method yields a significantly lower performance. We believe that the proposed method represents an essential component of any artificial cognitive system or robotic platform engaging in social interactions.

National Category
Computer Sciences
Identifiers
urn:nbn:se:kth:diva-260126 (URN)10.1109/TCDS.2019.2927941 (DOI)2-s2.0-85069908129 (Scopus ID)
Note

QC 20191011

Available from: 2019-09-25 Created: 2019-09-25 Last updated: 2019-10-11Bibliographically approved
Székely, É., Henter, G. E., Beskow, J. & Gustafson, J. (2019). Spontaneous conversational speech synthesis from found data. In: : . Paper presented at Interspeech.
Open this publication in new window or tab >>Spontaneous conversational speech synthesis from found data
2019 (English)Conference paper, Published paper (Refereed)
National Category
Engineering and Technology
Identifiers
urn:nbn:se:kth:diva-261689 (URN)
Conference
Interspeech
Note

QC 20191011

Available from: 2019-10-10 Created: 2019-10-10 Last updated: 2019-10-11Bibliographically approved
Székely, É., Henter, G. E., Beskow, J. & Gustafson, J. (2019). Spontaneous conversational speech synthesis from found data. In: : . Paper presented at The 20th Annual Conference of the International Speech Communication Association INTERSPEECH 2019 | Graz, Austria, Sep. 15-19, 2019..
Open this publication in new window or tab >>Spontaneous conversational speech synthesis from found data
2019 (English)Conference paper, Published paper (Refereed)
National Category
Language Technology (Computational Linguistics)
Identifiers
urn:nbn:se:kth:diva-260958 (URN)
Conference
The 20th Annual Conference of the International Speech Communication Association INTERSPEECH 2019 | Graz, Austria, Sep. 15-19, 2019.
Note

QC 20191113

Available from: 2019-09-30 Created: 2019-09-30 Last updated: 2019-11-13Bibliographically approved
Organisations
Identifiers
ORCID iD: ORCID iD iconorcid.org/0000-0003-1399-6604

Search in DiVA

Show all publications