Endre søk
RefereraExporteraLink to record
Permanent link

Direct link
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annet format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annet språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf
Multimodal analysis of the predictability of hand-gesture properties
KTH, Skolan för elektroteknik och datavetenskap (EECS), Intelligenta system, Robotik, perception och lärande, RPL.ORCID-id: 0000-0001-9838-8848
KTH, Skolan för elektroteknik och datavetenskap (EECS), Intelligenta system, Robotik, perception och lärande, RPL.ORCID-id: 0000-0002-9653-6699
University of California, Davis, Davis, CA, USA.ORCID-id: 0000-0003-0226-2808
KTH, Skolan för elektroteknik och datavetenskap (EECS), Intelligenta system, Robotik, perception och lärande, RPL.ORCID-id: 0000-0002-5750-9655
Vise andre og tillknytning
2022 (engelsk)Inngår i: AAMAS '22: Proceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems, ACM Press, 2022, s. 770-779Konferansepaper, Publicerat paper (Fagfellevurdert)
Abstract [en]

Embodied conversational agents benefit from being able to accompany their speech with gestures. Although many data-driven approaches to gesture generation have been proposed in recent years, it is still unclear whether such systems can consistently generate gestures that convey meaning. We investigate which gesture properties (phase, category, and semantics) can be predicted from speech text and/or audio using contemporary deep learning. In extensive experiments, we show that gesture properties related to gesture meaning (semantics and category) are predictable from text features (time-aligned FastText embeddings) alone, but not from prosodic audio features, while rhythm-related gesture properties (phase) on the other hand can be predicted from audio features better than from text. These results are encouraging as they indicate that it is possible to equip an embodied agent with content-wise meaningful co-speech gestures using a machine-learning model.

sted, utgiver, år, opplag, sider
ACM Press, 2022. s. 770-779
Emneord [en]
embodied conversational agents, gesture generation, gesture analysis, gesture property
HSV kategori
Forskningsprogram
Datalogi; Människa-datorinteraktion
Identifikatorer
URN: urn:nbn:se:kth:diva-312470DOI: 10.5555/3535850.3535937Scopus ID: 2-s2.0-85134341889OAI: oai:DiVA.org:kth-312470DiVA, id: diva2:1659101
Konferanse
21st International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2022, Auckland, New Zealand, May 9-13, 2022
Forskningsfinansiär
Swedish Foundation for Strategic ResearchWallenberg AI, Autonomous Systems and Software Program (WASP)Knut and Alice Wallenberg Foundation
Merknad

Part of proceedings ISBN: 9781450392136

QC 20220621

Tilgjengelig fra: 2022-05-19 Laget: 2022-05-19 Sist oppdatert: 2023-04-26bibliografisk kontrollert

Open Access i DiVA

Fulltekst mangler i DiVA

Andre lenker

Forlagets fulltekstScopushttps://dl.acm.org/doi/abs/10.5555/3535850.3535937

Person

Kucherenko, TarasNagy, RajmundKjellström, HedvigHenter, Gustav Eje

Søk i DiVA

Av forfatter/redaktør
Kucherenko, TarasNagy, RajmundNeff, MichaelKjellström, HedvigHenter, Gustav Eje
Av organisasjonen

Søk utenfor DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric

doi
urn-nbn
Totalt: 91 treff
RefereraExporteraLink to record
Permanent link

Direct link
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annet format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annet språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf