kth.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Real-time emotion generation in human-robot dialogue using large language models
Furhat Robotics AB, Stockholm, Sweden; Max Planck Institute for Psycholinguistics, Nijmegen, Netherlands.ORCID iD: 0000-0002-9223-1230
Max Planck Institute for Psycholinguistics, Nijmegen, Netherlands.ORCID iD: 0000-0002-7124-4091
Max Planck Institute for Psycholinguistics, Nijmegen, Netherlands; Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, Netherlands.ORCID iD: 0000-0001-7280-7549
KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Speech, Music and Hearing, TMH. Furhat Robotics AB, Stockholm, Sweden.ORCID iD: 0000-0002-8579-1790
2023 (English)In: Frontiers in Robotics and AI, E-ISSN 2296-9144, Vol. 10Article in journal (Refereed) Published
Abstract [en]

Affective behaviors enable social robots to not only establish better connections with humans but also serve as a tool for the robots to express their internal states. It has been well established that emotions are important to signal understanding in Human-Robot Interaction (HRI). This work aims to harness the power of Large Language Models (LLM) and proposes an approach to control the affective behavior of robots. By interpreting emotion appraisal as an Emotion Recognition in Conversation (ERC) tasks, we used GPT-3.5 to predict the emotion of a robot’s turn in real-time, using the dialogue history of the ongoing conversation. The robot signaled the predicted emotion using facial expressions. The model was evaluated in a within-subjects user study (N = 47) where the model-driven emotion generation was compared against conditions where the robot did not display any emotions and where it displayed incongruent emotions. The participants interacted with the robot by playing a card sorting game that was specifically designed to evoke emotions. The results indicated that the emotions were reliably generated by the LLM and the participants were able to perceive the robot’s emotions. It was found that the robot expressing congruent model-driven facial emotion expressions were perceived to be significantly more human-like, emotionally appropriate, and elicit a more positive impression. Participants also scored significantly better in the card sorting game when the robot displayed congruent facial expressions. From a technical perspective, the study shows that LLMs can be used to control the affective behavior of robots reliably in real-time. Additionally, our results could be used in devising novel human-robot interactions, making robots more effective in roles where emotional interaction is important, such as therapy, companionship, or customer service.

Place, publisher, year, edition, pages
Frontiers Media SA , 2023. Vol. 10
National Category
Language Technology (Computational Linguistics)
Identifiers
URN: urn:nbn:se:kth:diva-341389DOI: 10.3389/frobt.2023.1271610ISI: 001125732400001Scopus ID: 2-s2.0-85179653946OAI: oai:DiVA.org:kth-341389DiVA, id: diva2:1821224
Note

QC 20231220

Available from: 2023-12-19 Created: 2023-12-19 Last updated: 2024-01-10Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Authority records

Skantze, Gabriel

Search in DiVA

By author/editor
Mishra, ChinmayaVerdonschot, RinusHagoort, PeterSkantze, Gabriel
By organisation
Speech, Music and Hearing, TMH
In the same journal
Frontiers in Robotics and AI
Language Technology (Computational Linguistics)

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 69 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf