kth.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Multimodal User Feedback During Adaptive Robot-Human Presentations
KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Speech, Music and Hearing, TMH.ORCID iD: 0000-0003-0112-6732
KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Speech, Music and Hearing, TMH.ORCID iD: 0000-0002-8579-1790
2022 (English)In: Frontiers in Computer Science, E-ISSN 2624-9898, Vol. 3Article in journal (Refereed) Published
Abstract [en]

Feedback is an essential part of all communication, and agents communicating with humans must be able to both give and receive feedback in order to ensure mutual understanding. In this paper, we analyse multimodal feedback given by humans towards a robot that is presenting a piece of art in a shared environment, similar to a museum setting. The data analysed contains both video and audio recordings of 28 participants, and the data has been richly annotated both in terms of multimodal cues (speech, gaze, head gestures, facial expressions, and body pose), as well as the polarity of any feedback (negative, positive, or neutral). We train statistical and machine learning models on the dataset, and find that random forest models and multinomial regression models perform well on predicting the polarity of the participants' reactions. An analysis of the different modalities shows that most information is found in the participants' speech and head gestures, while much less information is found in their facial expressions, body pose and gaze. An analysis of the timing of the feedback shows that most feedback is given when the robot makes pauses (and thereby invites feedback), but that the more exact timing of the feedback does not affect its meaning.

Place, publisher, year, edition, pages
Frontiers Media SA , 2022. Vol. 3
Keywords [en]
feedback, presentation, agent, robot, grounding, polarity, backchannel, multimodal
Keywords [sv]
återmatning, presentation, agent, robot, återkoppling, polaritet, reaktion, multimodal
National Category
Human Computer Interaction
Research subject
Speech and Music Communication
Identifiers
URN: urn:nbn:se:kth:diva-307105DOI: 10.3389/fcomp.2021.741148ISI: 000745131900001Scopus ID: 2-s2.0-85123110812OAI: oai:DiVA.org:kth-307105DiVA, id: diva2:1626622
Projects
Co-adaptive Human-Robot Interactive Systems
Funder
Swedish Foundation for Strategic Research, COIN
Note

QC 20220112 QC 20220216

Available from: 2022-01-11 Created: 2022-01-11 Last updated: 2022-06-25Bibliographically approved

Open Access in DiVA

fulltext(2648 kB)335 downloads
File information
File name FULLTEXT01.pdfFile size 2648 kBChecksum SHA-512
0014f7053392ab826a9b3634edb188bbe7ad3beedc951598c2e21d43d680c8d4a78aa21d3e1c9a0978017c4aa81ae7c3427562ab4a3f7c7fc8f0280533a708a7
Type fulltextMimetype application/pdf

Other links

Publisher's full textScopus

Authority records

Axelsson, AgnesSkantze, Gabriel

Search in DiVA

By author/editor
Axelsson, AgnesSkantze, Gabriel
By organisation
Speech, Music and Hearing, TMH
In the same journal
Frontiers in Computer Science
Human Computer Interaction

Search outside of DiVA

GoogleGoogle Scholar
Total: 335 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 580 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf