kth.sePublications KTH
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Computational Approaches to Interaction-Shaping Robotics
KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.ORCID iD: 0000-0001-7130-0826
2024 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

The goal of this thesis is to develop computational approaches generating autonomous social robot behaviors that can interact with multiple people and dynamically adapt to shape their interactions. Positive interactions between people impact their well-being and are essential to a fulfilled and healthy life. In this thesis, we coin the term Interaction-Shaping Robotics (ISR) as the study of robots that shape interactions between other agents, e.g., people, and capture previous efforts from the Human-Robot Interaction (HRI) community and emphasize the potential positive or negative, intended or unintended effects of these robots. Previous efforts have explored phenomena that indicate interaction-shaping capabilities of social robots, however, how to de-velop autonomous social robots that can adapt to positively shape interactions between people based on perceived human-human dynamics remains largely unexplored. In this thesis, we contribute to the technical advancement of social interaction-shaping robots by developing heuristics and machine learning methods and demonstrating their effectiveness in studies with real users. We focus on shaping behaviors, i.e., balancing people’s participation in interactions to foster inclusion among newly-arrived and already present children in a music game and support adult second language learners and native speakers in a language game. Especially when leveraging learning techniques, an effective interaction-shaping robot needs to act socially appropriately. We design heuristics that are appropriate by design and establish the feasibility of autonomy for interaction-shaping robots through minimal perception of group dynamics and simple behavior rules. Allowing for learning behaviors for more complex interactions, we provide a formal definition of the problem of interaction-shaping and show that using imitation learning (IL) or offline reinforcement learning (RL) based on previously collected HRI data is feasible without compromising the interaction. To meet the challenge of acting appropriately, we explore techniques applied prior to deployment when learning offline from data and shielding - a technique from the safe RL community - to eventually allow for learning during deployment in interaction. Overall, this thesis demonstrates the feasibility and promise of computational methods for autonomous interaction-shaping robots and demonstrates that these methods generate effective and appropriate robot behavior when balancing participation to ensure the inclusion of all human group members.

Abstract [sv]

Målet med denna avhandling är att utveckla beräkningsbaserade meto-der för att generera autonoma sociala robotbeteenden som kan interagera med flera människor och dynamiskt anpassa sig för att forma deras interak-tioner. Positiva interaktioner mellan människor påverkar deras välbefinnande och är avgörande för ett meningsfullt och hälsosamt liv. I denna avhandling myntar vi termen "Interaction-Shaping Robotics"(ISR) som studerandet av robotar som formar interaktioner mellan andra aktörer, t.ex. människor, och sammanställer tidigare studier inom människ-robot-interaktion (eng. Human-Robot Interaction, HRI) samt betonar den potentiella positiva eller negativa, avsiktliga eller oavsiktliga, inverkan av dessa robotar. Tidigare studier har utforskat fenomen som indikerar på interaktionsformande förmågor hos sociala robotar, men utvecklandet av autonoma sociala robotar som kan anpassa sig för att positivt forma interaktioner mellan människor baserat på observerad människa-till-människa dynamik är fortfarande till stor del outforskat. I denna avhandling bidrar vi till den tekniska utvecklingen av sociala interaktionsformande robotar genom att utveckla heuristiker och maskininlärningsmetoder och demonstrera deras effektivitet i studier med användare. Vi fokuserar på att forma beteenden, d.v.s. balansera människors deltagande i interaktioner för att främja inkludering bland nyanlända och redan närvarande barn i ett musikspel och stödja vuxna andraspråksinlärare och modersmålstalare i ett språkspel. Särskilt när man utnyttjar maskininlärningsmetoder, behöver en effektiv interaktionsformande robot agera socialt korrekt. Vi designar heuristiker som är lämpliga by design” och fastställer genomförbarheten av autonomi för interaktionsformande robotar genom minimal perception av gruppdynamik och enkla beteenderegler. Genom att tillåta inlärning av beteenden för mer komplexa interaktioner, tillhandahåller vi en formell definition av problemet av interaktionsformande och visar att användning av imitationsinlärning (eng. imitation learning, IL) off-line förstärkningsinlärning (eng. reinforcement learning, RL), baserat på tidigare insamlad HRI-data är genomförbart utan att kompromissa med interaktionen. För att möta utmaningen att agera korrekt, utforskar vi tekniker som tillämpas innan implementering när man lär sig off-line från data och ”shielding” - en teknik inom säker RL - för att så småningom möjliggöra inlärning under implementering vid interaktion. Sammanfattningsvis visar denna avhandling genomförbarheten och utsikten av beräkningsbaserade metoder för autonoma interaktionsformande robotar och demonstrerar att dessa metoder genererar effektiva och lämpliga robotbeteenden när de balanserar deltagande för att säkerställa inkludering av alla mänskliga gruppmedlemmar.

Place, publisher, year, edition, pages
Stockholm: KTH Royal Institute of Technology, 2024. , p. 63
Series
TRITA-EECS-AVL ; 2024:60
Keywords [en]
Human-robot interaction, social robotics, behavior generation, multiparty interaction, human-human dynamics, machine learning
National Category
Computer graphics and computer vision
Research subject
Computer Science
Identifiers
URN: urn:nbn:se:kth:diva-350809ISBN: 978-91-8106-006-5 (print)OAI: oai:DiVA.org:kth-350809DiVA, id: diva2:1885026
Public defence
2024-09-05, https://kth-se.zoom.us/j/69226775403, F3 Flodis, Lindstedtsvägen 26 & 28, KTH Campus, Stockholm, 14:00 (English)
Opponent
Supervisors
Note

QC 20240722

Available from: 2024-07-22 Created: 2024-07-19 Last updated: 2025-12-02Bibliographically approved
List of papers
1. Interaction-Shaping Robotics: Robots That Influence Interactions between Other Agents
Open this publication in new window or tab >>Interaction-Shaping Robotics: Robots That Influence Interactions between Other Agents
Show others...
2024 (English)In: ACM Transactions on Human-Robot Interaction, E-ISSN 2573-9522, Vol. 13, no 1, article id 12Article in journal (Refereed) Published
Abstract [en]

Work in Human–Robot Interaction (HRI) has investigated interactions between one human and one robot as well as human–robot group interactions. Yet the field lacks a clear definition and understanding of the influence a robot can exert on interactions between other group members (e.g., human-to-human). In this article, we define Interaction-Shaping Robotics (ISR), a subfield of HRI that investigates robots that influence the behaviors and attitudes exchanged between two (or more) other agents. We highlight key factors of interaction-shaping robots that include the role of the robot, the robot-shaping outcome, the form of robot influence, the type of robot communication, and the timeline of the robot’s influence. We also describe three distinct structures of human–robot groups to highlight the potential of ISR in different group compositions and discuss targets for a robot’s interaction-shaping behavior. Finally, we propose areas of opportunity and challenges for future research in ISR.

Place, publisher, year, edition, pages
Association for Computing Machinery (ACM), 2024
Keywords
Human–robot interaction, interaction-shaping robotics, multiparty interactions, shaping interactions, social influence
National Category
Robotics and automation
Identifiers
urn:nbn:se:kth:diva-345236 (URN)10.1145/3643803 (DOI)001208571200012 ()2-s2.0-85189071275 (Scopus ID)
Note

QC 20240715

Available from: 2024-04-10 Created: 2024-04-10 Last updated: 2025-02-09Bibliographically approved
2. Robot Gaze Can Mediate Participation Imbalance in Groups with Different Skill Levels
Open this publication in new window or tab >>Robot Gaze Can Mediate Participation Imbalance in Groups with Different Skill Levels
Show others...
2021 (English)In: Proceedings of the 2021 ACM/IEEE International Conference on Human-Robot Interaction, Association for Computing Machinery , 2021, p. 303-311Conference paper, Published paper (Refereed)
Abstract [en]

Many small group activities, like working teams or study groups, have a high dependency on the skill of each group member. Differences in skill level among participants can affect not only the performance of a team but also influence the social interaction of its members. In these circumstances, an active member could balance individual participation without exerting direct pressure on specific members by using indirect means of communication, such as gaze behaviors. Similarly, in this study, we evaluate whether a social robot can balance the level of participation in a language skill-dependent game, played by a native speaker and a second language learner. In a between-subjects study (N = 72), we compared an adaptive robot gaze behavior, that was targeted to increase the level of contribution of the least active player, with a non-adaptive gaze behavior. Our results imply that, while overall levels of speech participation were influenced predominantly by personal traits of the participants, the robot’s adaptive gaze behavior could shape the interaction among participants which lead to more even participation during the game.

Place, publisher, year, edition, pages
Association for Computing Machinery, 2021
Series
HRI ’21
Keywords
language learning, gaze, multiparty interaction, group dynamics
National Category
Other Engineering and Technologies
Identifiers
urn:nbn:se:kth:diva-292043 (URN)10.1145/3434073.3444670 (DOI)001051690500035 ()2-s2.0-85102757966 (Scopus ID)
Conference
ACM/IEEE International Conference on Human-Robot Interaction March 09 –11
Funder
Swedish Foundation for Strategic Research , FFL18-0199Swedish Research Council, 2017-05189Swedish Research Council, 2016-03698
Note

QC 20210710

Available from: 2021-03-24 Created: 2021-03-24 Last updated: 2025-02-18Bibliographically approved
3. A social robot mediator to foster collaboration and inclusion among children
Open this publication in new window or tab >>A social robot mediator to foster collaboration and inclusion among children
2020 (English)In: Robotics: Science and systems XVI / [ed] Toussaint, M Bicchi, A Hermans, T, MIT Press, 2020Conference paper, Published paper (Refereed)
Abstract [en]

Formation of subgroups and thereby the problem of intergroup bias is well-studied in psychology. Already from the age of five, children can show ingroup preferences. We developed a social robot mediator to explore how a robot could help overcome these intergroup biases, especially for children newly arrived to a country. By utilizing an online evaluation of collaboration levels, we allow the robot to perceive and act upon the current group dynamics. We investigated the effectiveness of the robot’s mediating behavior in a between-subject study with 39 children, of whom 13 children had arrived in Sweden within the last 2 years. Results indicate that the robot could help the process of inclusion by mediating the activity. The robot succeeds in encouraging the newly arrived children to act more outgoing and in increasing collaboration among ingroup children. Further, children show a higher level of prosociality after interacting with the robot. In line with prior work, this study demonstrates the ability of social robotic technology to assist group processes.

Place, publisher, year, edition, pages
MIT Press, 2020
National Category
Pedagogy
Identifiers
urn:nbn:se:kth:diva-284305 (URN)10.15607/RSS.2020.XVI.103 (DOI)000570976900103 ()2-s2.0-85102742806 (Scopus ID)
Conference
16th Robotics: Science and Systems, RSS 2020, Virtual, Online, 12-16 July 2020
Note

QC 20220615

Part of proceedings: ISBN 978-099237476-1

Available from: 2020-10-28 Created: 2020-10-28 Last updated: 2024-07-19Bibliographically approved
4. Ice-Breakers, Turn-Takers and Fun-Makers: Exploring Robots for Groups with Teenagers
Open this publication in new window or tab >>Ice-Breakers, Turn-Takers and Fun-Makers: Exploring Robots for Groups with Teenagers
2022 (English)In: 2022 31st IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), Institute of Electrical and Electronics Engineers (IEEE) , 2022, p. 1474-1481Conference paper, Published paper (Refereed)
Abstract [en]

Successful, enjoyable group interactions are important in public and personal contexts, especially for teenagers whose peer groups are important for self-identity and selfesteem. Social robots seemingly have the potential to positively shape group interactions, but it seems difficult to effect such impact by designing robot behaviors solely based on related (human interaction) literature. In this article, we take a usercentered approach to explore how teenagers envisage a social robot <feminine ordinal indicator>group assistant degrees. We engaged 16 teenagers in focus groups, interviews, and robot testing to capture their views and reflections about robots for groups. Over the course of a two-week summer school, participants co-designed the action space for such a robot and experienced working with/wizarding it for 10+ hours. This experience further altered and deepened their insights into using robots as group assistants. We report results regarding teenagers' views on the applicability and use of a robot group assistant, how these expectations evolved throughout the study, and their repeat interactions with the robot. Our results indicate that each group moves on a spectrum of need for the robot, reflected in use of the robot more (or less) for ice-breaking, turn-taking, and fun-making as the situation demanded.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2022
National Category
Human Computer Interaction
Identifiers
urn:nbn:se:kth:diva-322317 (URN)10.1109/RO-MAN53752.2022.9900644 (DOI)000885903300209 ()2-s2.0-85140801159 (Scopus ID)
Conference
31st IEEE International Conference on Robot and Human Interactive Communication (RO-MAN) - Social, Asocial, and Antisocial Robots, AUG 29-SEP 02, 2022, Napoli, Italy
Note

QC 20221215

Part of proceedings: ISBN 978-1-7281-8859-1

Available from: 2022-12-15 Created: 2022-12-15 Last updated: 2024-07-23Bibliographically approved
5. Shielding for socially appropriate robot listening behaviors
Open this publication in new window or tab >>Shielding for socially appropriate robot listening behaviors
2024 (English)In: 2024 33rd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), 2024Conference paper, Published paper (Refereed)
Abstract [en]

A crucial part of traditional reinforcement learning (RL) is the initial exploration phase, in which trying available actions randomly is a critical element. As random behavior might be detrimental to a social interaction, this work proposes a novel paradigm for learning social robot behavior--the use of shielding to ensure socially appropriate behavior during exploration and learning. We explore how a data-driven approach for shielding could be used to generate listening behavior. In a video-based user study (N=110), we compare shielded exploration to two other exploration methods. We show that the shielded exploration is perceived as more comforting and appropriate than a straightforward random approach. Based on our findings, we discuss the potential for future work using shielded and socially guided approaches for learning idiosyncratic social robot behaviors through RL.   

National Category
Computer graphics and computer vision
Identifiers
urn:nbn:se:kth:diva-350432 (URN)
Conference
2024 33rd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), Pasadena, California, USA August 26th-30th, 2024
Note

Paper will be published later this year (accepted camera-ready version available).

QC 20240717

Available from: 2024-07-11 Created: 2024-07-11 Last updated: 2025-02-07Bibliographically approved
6. Learning Gaze Behaviors for Balancing Participation in Group Human-Robot Interactions
Open this publication in new window or tab >>Learning Gaze Behaviors for Balancing Participation in Group Human-Robot Interactions
2022 (English)In: HRI '22: Proceedings of the 2022 ACM/IEEE International Conference on Human-Robot Interaction, Institute of Electrical and Electronics Engineers (IEEE) , 2022, p. 265-274Conference paper, Published paper (Refereed)
Abstract [en]

Robots can affect group dynamics. In particular, prior work has shown that robots that use hand-crafted gaze heuristics can influence human participation in group interactions. However, hand-crafting robot behaviors can be difficult and might have unexpected results in groups. Thus, this work explores learning robot gaze behaviors that balance human participation in conversational interactions. More specifically, we examine two techniques for learning a gaze policy from data: imitation learning (IL) and batch reinforcement learning (RL). First, we formulate the problem of learning a gaze policy as a sequential decision-making task focused on human turn-taking. Second, we experimentally show that IL can be used to combine strategies from hand-crafted gaze behaviors, and we formulate a novel reward function to achieve a similar result using batch RL. Finally, we conduct an offline evaluation of IL and RL policies and compare them via a user study (N=50). The results from the study show that the learned behavior policies did not compromise the interaction. Interestingly, the proposed reward for the RL formulation enabled the robot to encourage participants to take more turns during group human-robot interactions than one of the gaze heuristic behaviors from prior work. Also, the imitation learning policy led to more active participation from human participants than another prior heuristic behavior. 

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2022
Series
ACM IEEE International Conference on Human-Robot Interaction, ISSN 2167-2121
Keywords
social robotics, nonverbal signals, learning
National Category
Human Computer Interaction
Identifiers
urn:nbn:se:kth:diva-316516 (URN)10.1109/HRI53351.2022.9889416 (DOI)000869793600031 ()2-s2.0-85140768966 (Scopus ID)
Conference
17th Annual ACM/IEEE International Conference on Human-Robot Interaction (HRI), MAR 07-10, 2022, ELECTR NETWORK
Note

Part of proceedings: ISBN 978-1-6654-0731-1

QC 20220905

Available from: 2022-08-19 Created: 2022-08-19 Last updated: 2024-07-19Bibliographically approved

Open Access in DiVA

Kappa(1815 kB)369 downloads
File information
File name SUMMARY01.pdfFile size 1815 kBChecksum SHA-512
5c61cce816a2824fd0366934bb86658edb4799d59a2120e9d741f9ba310b147dfc8993c3f833194d4a5d09989a0ed13b9ba3f57f76b617904dd5a609d2ea5698
Type fulltextMimetype application/pdf

Authority records

Gillet, Sarah

Search in DiVA

By author/editor
Gillet, Sarah
By organisation
Robotics, Perception and Learning, RPL
Computer graphics and computer vision

Search outside of DiVA

GoogleGoogle Scholar
Total: 0 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

isbn
urn-nbn

Altmetric score

isbn
urn-nbn
Total: 1147 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf