kth.sePublications
Change search
Link to record
Permanent link

Direct link
Publications (10 of 18) Show all publications
Cumbal, R. (2024). Robots Beyond Borders: The Role of Social Robots in Spoken Second Language Practice. (Doctoral dissertation). Stockholm: KTH Royal Institute of Technology
Open this publication in new window or tab >>Robots Beyond Borders: The Role of Social Robots in Spoken Second Language Practice
2024 (English)Doctoral thesis, comprehensive summary (Other academic)
Alternative title[sv]
Robotar bortom gränser : Sociala robotars roll i talat andraspråk
Abstract [en]

This thesis investigates how social robots can support adult second language (L2) learners in improving conversational skills. It recognizes the challenges inherent in adult L2 learning, including increased cognitive demands and the unique motivations driving adult education. While social robots hold potential for natural interactions and language education, research into conversational skill practice with adult learners remains underexplored. Thus, the thesis contributes to understanding these conversational dynamics, enhancing speaking practice, and examining cultural perspectives in this context.

To begin, this thesis investigates robot-led conversations with L2 learners, examining how learners respond to moments of uncertainty. The research reveals that when faced with uncertainty, learners frequently seek clarification, yet many remain unresponsive. As a result, effective strategies are required from robot conversational partners to address this challenge. These interactions are then used to evaluate the performance of off-the-shelf Automatic Speech Recognition (ASR) systems. The assessment highlights that speech recognition for L2 speakers is not as effective as for L1 speakers, with performance deteriorating for both groups during social conversations. Addressing these challenges is imperative for the successful integration of robots in conversational practice with L2 learners.

The thesis then explores the potential advantages of employing social robots in collaborative learning environments with multi-party interactions. It delves into strategies for improving speaking practice, including the use of non-verbal behaviors to encourage learners to speak. For instance, a robot's adaptive gazing behavior is used to effectively balance speaking contributions between L1 and L2 pairs of participants. Moreover, an adaptive use of encouraging backchannels significantly increases the speaking time of L2 learners.

Finally, the thesis highlights the importance of further research on cultural aspects in human-robot interactions. One study reveals distinct responses among various socio-cultural groups in interaction between L1 and L2 participants. For example, factors such as gender, age, extroversion, and familiarity with robots influence conversational engagement of L2 speakers. Additionally, another study investigates preconceptions related to the appearance and accents of nationality-encoded (virtual and physical) social robots. The results indicate that initial perceptions may lead to negative preconceptions, but that these perceptions diminish after actual interactions.

Despite technical limitations, social robots provide distinct benefits in supporting educational endeavors. This thesis emphasizes the potential of social robots as effective facilitators of spoken language practice for adult learners, advocating for continued exploration at the intersection of language education, human-robot interaction, and technology.

Abstract [sv]

Denna avhandling undersöker hur sociala robotar kan ge vuxna andraspråks\-inlärare stöd att förbättra sin konversationsförmåga på svenska. Andraspråks\-inlärning för vuxna, särskilt i migrationskontext, är mer komplext än för barn, bland annat på grund av att förutsättningarna för språkinlärning försämras med åren och att drivkrafterna ofta är andra. Sociala robotar har stor potential inom språkundervisning för att träna naturliga samtal, men fortfarande har lite forskning om hur robotar kan öva konversation med vuxna elever genomförts. Därför bidrar avhandlingen till att förstå samtal mellan andraspråksinlärare och robotar, förbättra dessa samtalsövningar och undersöka hur kulturella faktorer påverkar interaktionen.

Till att börja med undersöker avhandlingen hur andraspråkselever reagerar då de blir förbryllade eller osäkra i robotledda konversationsövningar. Resultaten visar att eleverna ofta försöker få roboten att ge förtydliganden när de är osäkra, men att de ibland helt enkelt inte svarar något alls, vilket innebär att roboten behöver kunna hantera sådana situationer. Konversationerna mellan andraspråksinlärare och en robot har även använts för att undersöka hur väl ledande system för taligenkänning kan tolka det adraspråkstalare säger. Det kan konstateras att systemen har väsentligt större svårigheter att känna igen andraspråkstalare än personer med svensk bakgrund, samt att de har utmananingar att tolka såväl svenska talare som andraspråkselever i friare sociala konversationer, vilket måste hanteras när robotar ska användas i samtalsövningar med andraspråkselever.

Avhandlingen undersöker sedan strategier för att uppmuntra andraspråks\-elever att prata mer och för att fördela ordet jämnare i trepartsövningar där två personer samtalar med roboten. Strategierna går ut på att modifiera hur roboten tittar på de två personerna eller ger icke-verbal återkoppling (hummanden) för att signalera förståelse och intresse för det eleverna säger.

Slutligen belyser avhandlingen vikten av ytterligare forskning om kulturella aspekter i interaktioner mellan människa och robot. En studie visar att faktorer som kön, ålder, tidigare erfarenhet av robotar och hur extrovert eleven är påverkar både hur mycket olika personer talar och hur de svarar på robotens försök att uppmuntra dem att tala mer genom icke-verbala signaler.

En andra studie undersöker om och hur förutfattade meningar relaterade till utseende och uttal påverkar hur människor uppfattar (virtuella och fysiska) sociala robotar som givits egenskaper (röst och ansikte) som kan kopplas till olika nationella bakgrunder. Resultaten visar att människors första intryck av en kulturellt färgad robot speglar förutfattade meningar, men att denna uppfattning inte alls får samma genomslag när personer faktiskt interagerat med roboten i ett realistiskt sammanhang.

En huvudsaklig slutsats i avhandlingen är att sociala robotar, trots att tekniska begränsningar finns kvar, har tydliga fördelar som kan utnyttjas inom utbildning. Specifikt betonar avhandlingen potentialen hos sociala robotar att leda samtalsövningar för vuxna andraspråkselever och förespråkar fortsatt forskning i skärningspunkten mellan språkundervisning, människa-robotinteraktion och teknik.

Place, publisher, year, edition, pages
Stockholm: KTH Royal Institute of Technology, 2024. p. 91
Series
TRITA-EECS-AVL ; 2024:23
Keywords
Conversations, gaze, backchannels, multi-party, accent, culture, Samtal, blick, återkoppling, gruppdynamik, brytning, kultur
National Category
Robotics and automation Natural Language Processing
Research subject
Speech and Music Communication
Identifiers
urn:nbn:se:kth:diva-343863 (URN)978-91-8040-858-5 (ISBN)
Public defence
2024-03-22, https://kth-se.zoom.us/j/65591848998, F3, Lindstedtsvägen 26, Stockholm, 10:00 (English)
Opponent
Supervisors
Note

QC 20240226

Available from: 2024-02-26 Created: 2024-02-26 Last updated: 2025-02-05Bibliographically approved
Cumbal, R. & Engwall, O. (2024). Speaking Transparently: Social Robots in Educational Settings. In: Companion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction (HRI '24 Companion), March 11--14, 2024, Boulder, CO, USA: . Paper presented at International Conference on Human-Robot Interaction.
Open this publication in new window or tab >>Speaking Transparently: Social Robots in Educational Settings
2024 (English)In: Companion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction (HRI '24 Companion), March 11--14, 2024, Boulder, CO, USA, 2024Conference paper, Published paper (Refereed)
Abstract [en]

The recent surge in popularity of Large Language Models, known for their inherent opacity, has increased the interest in fostering transparency in technology designed for human interaction. This concern is equally prevalent in the development of Social Robots, particularly when these are designed to engage in critical areas of our society, such as education or healthcare. In this paper we propose an experiment to investigate how users can be made aware of the automated decision processes when interacting in a discussion with a social robot. Our main objective is to assess the effectiveness of verbal expressions in fostering transparency within groups of individuals as they engage with a robot. We describe the proposed interactive settings, system design, and our approach to enhance the transparency in a robot's decision-making process for multi-party interactions.

Keywords
Dialogue system, transparency, multi-party, conversation
National Category
Robotics and automation
Research subject
Computer Science
Identifiers
urn:nbn:se:kth:diva-343862 (URN)10.1145/3610978.3640717 (DOI)001255070800079 ()2-s2.0-85188090310 (Scopus ID)
Conference
International Conference on Human-Robot Interaction
Note

QC 20240328

Available from: 2024-02-26 Created: 2024-02-26 Last updated: 2025-02-09Bibliographically approved
McMillan, D., Jaber, R., Cowan, B. R., Fischer, J. E., Irfan, B., Cumbal, R., . . . Lee, M. (2023). Human-Robot Conversational Interaction (HRCI). In: HRI 2023: Companion of the ACM/IEEE International Conference on Human-Robot Interaction. Paper presented at 18th Annual ACM/IEEE International Conference on Human-Robot Interaction, HRI 2023, Stockholm, Sweden, Mar 13 2023 - Mar 16 2023 (pp. 923-925). Association for Computing Machinery (ACM)
Open this publication in new window or tab >>Human-Robot Conversational Interaction (HRCI)
Show others...
2023 (English)In: HRI 2023: Companion of the ACM/IEEE International Conference on Human-Robot Interaction, Association for Computing Machinery (ACM) , 2023, p. 923-925Conference paper, Published paper (Refereed)
Abstract [en]

Conversation is one of the primary methods of interaction between humans and robots. It provides a natural way of communication with the robot, thereby reducing the obstacles that can be faced through other interfaces (e.g., text or touch) that may cause difficulties to certain populations, such as the elderly or those with disabilities, promoting inclusivity in Human-Robot Interaction (HRI).Work in HRI has contributed significantly to the design, understanding and evaluation of human-robot conversational interactions. Concurrently, the Conversational User Interfaces (CUI) community has developed with similar aims, though with a wider focus on conversational interactions across a range of devices and platforms. This workshop aims to bring together the CUI and HRI communities to outline key shared opportunities and challenges in developing conversational interactions with robots, resulting in collaborative publications targeted at the CUI 2023 provocations track.

Place, publisher, year, edition, pages
Association for Computing Machinery (ACM), 2023
Keywords
Conversational User Interaction, Embodied Interaction, HRI
National Category
Human Computer Interaction Robotics and automation
Identifiers
urn:nbn:se:kth:diva-333370 (URN)10.1145/3568294.3579954 (DOI)001054975700205 ()2-s2.0-85150450577 (Scopus ID)
Conference
18th Annual ACM/IEEE International Conference on Human-Robot Interaction, HRI 2023, Stockholm, Sweden, Mar 13 2023 - Mar 16 2023
Note

Part of ISBN 9781450399708

QC 20230801

Available from: 2023-08-01 Created: 2023-08-01 Last updated: 2025-02-05Bibliographically approved
Engwall, O., Cumbal, R. & Majlesi, A. R. (2023). Socio-cultural perception of robot backchannels. Frontiers in Robotics and AI, 10
Open this publication in new window or tab >>Socio-cultural perception of robot backchannels
2023 (English)In: Frontiers in Robotics and AI, E-ISSN 2296-9144, Vol. 10Article in journal (Refereed) Published
Abstract [en]

Introduction: Backchannels, i.e., short interjections by an interlocutor to indicate attention, understanding or agreement regarding utterances by another conversation participant, are fundamental in human-human interaction. Lack of backchannels or if they have unexpected timing or formulation may influence the conversation negatively, as misinterpretations regarding attention, understanding or agreement may occur. However, several studies over the years have shown that there may be cultural differences in how backchannels are provided and perceived and that these differences may affect intercultural conversations. Culturally aware robots must hence be endowed with the capability to detect and adapt to the way these conversational markers are used across different cultures. Traditionally, culture has been defined in terms of nationality, but this is more and more considered to be a stereotypic simplification. We therefore investigate several socio-cultural factors, such as the participants’ gender, age, first language, extroversion and familiarity with robots, that may be relevant for the perception of backchannels.

Methods: We first cover existing research on cultural influence on backchannel formulation and perception in human-human interaction and on backchannel implementation in Human-Robot Interaction. We then present an experiment on second language spoken practice, in which we investigate how backchannels from the social robot Furhat influence interaction (investigated through speaking time ratios and ethnomethodology and multimodal conversation analysis) and impression of the robot (measured by post-session ratings). The experiment, made in a triad word game setting, is focused on if activity-adaptive robot backchannels may redistribute the participants’ speaking time ratio, and/or if the participants’ assessment of the robot is influenced by the backchannel strategy. The goal is to explore how robot backchannels should be adapted to different language learners to encourage their participation while being perceived as socio-culturally appropriate.

Results: We find that a strategy that displays more backchannels towards a less active speaker may substantially decrease the difference in speaking time between the two speakers, that different socio-cultural groups respond differently to the robot’s backchannel strategy and that they also perceive the robot differently after the session.

Discussion: We conclude that the robot may need different backchanneling strategies towards speakers from different socio-cultural groups in order to encourage them to speak and have a positive perception of the robot.

 

Place, publisher, year, edition, pages
Frontiers Media S.A., 2023
Keywords
backchannels, multiparty HRI, robot-assisted language learning, spoken practice, cultural effects
National Category
Human Computer Interaction
Research subject
Computer Science
Identifiers
urn:nbn:se:kth:diva-323334 (URN)10.3389/frobt.2023.988042 (DOI)000935012900001 ()36777379 (PubMedID)2-s2.0-85147686864 (Scopus ID)
Funder
Marcus and Amalia Wallenberg Foundation, 2020.0052
Note

QC 20230130

Available from: 2023-01-26 Created: 2023-01-26 Last updated: 2024-02-26Bibliographically approved
Cumbal, R., Axelsson, A., Mehta, S. & Engwall, O. (2023). Stereotypical nationality representations in HRI: perspectives from international young adults. Frontiers in Robotics and AI, 10, Article ID 1264614.
Open this publication in new window or tab >>Stereotypical nationality representations in HRI: perspectives from international young adults
2023 (English)In: Frontiers in Robotics and AI, E-ISSN 2296-9144, Vol. 10, article id 1264614Article in journal (Refereed) Published
Abstract [en]

People often form immediate expectations about other people, or groups of people, based on visual appearance and characteristics of their voice and speech. These stereotypes, often inaccurate or overgeneralized, may translate to robots that carry human-like qualities. This study aims to explore if nationality-based preconceptions regarding appearance and accents can be found in people's perception of a virtual and a physical social robot. In an online survey with 80 subjects evaluating different first-language-influenced accents of English and nationality-influenced human-like faces for a virtual robot, we find that accents, in particular, lead to preconceptions on perceived competence and likeability that correspond to previous findings in social science research. In a physical interaction study with 74 participants, we then studied if the perception of competence and likeability is similar after interacting with a robot portraying one of four different nationality representations from the online survey. We find that preconceptions on national stereotypes that appeared in the online survey vanish or are overshadowed by factors related to general interaction quality. We do, however, find some effects of the robot's stereotypical alignment with the subject group, with Swedish subjects (the majority group in this study) rating the Swedish-accented robot as less competent than the international group, but, on the other hand, recalling more facts from the Swedish robot's presentation than the international group does. In an extension in which the physical robot was replaced by a virtual robot interacting in the same scenario online, we further found the same results that preconceptions are of less importance after actual interactions, hence demonstrating that the differences in the ratings of the robot between the online survey and the interaction is not due to the interaction medium. We hence conclude that attitudes towards stereotypical national representations in HRI have a weak effect, at least for the user group included in this study (primarily educated young students in an international setting).

Place, publisher, year, edition, pages
Frontiers Media SA, 2023
Keywords
accent, appearance, social robot, nationality, stereotype, impression, competence, likeability
National Category
Human Computer Interaction
Identifiers
urn:nbn:se:kth:diva-341526 (URN)10.3389/frobt.2023.1264614 (DOI)001115613500001 ()38077460 (PubMedID)2-s2.0-85178920101 (Scopus ID)
Note

QC 20231222

Available from: 2023-12-22 Created: 2023-12-22 Last updated: 2024-02-26Bibliographically approved
Cumbal, R. (2022). Adaptive Robot Discourse for Language Acquisition in Adulthood. In: Proceedings of the 2022 ACM/IEEE International Conference on Human-Robot Interaction: . Paper presented at 17th Annual ACM/IEEE International Conference on Human-Robot Interaction (HRI), MAR 07-10, 2022, ELECTR NETWORK (pp. 1158-1160). Institute of Electrical and Electronics Engineers (IEEE)
Open this publication in new window or tab >>Adaptive Robot Discourse for Language Acquisition in Adulthood
2022 (English)In: Proceedings of the 2022 ACM/IEEE International Conference on Human-Robot Interaction, Institute of Electrical and Electronics Engineers (IEEE) , 2022, p. 1158-1160Conference paper, Published paper (Refereed)
Abstract [en]

Acquiring a second language in adulthood differs considerably from the approach taken at younger ages. Learning rates tend to decrease during adolescence, and socio-emotional characteristics, like motivation and expectations, take a different perspective for adults. In particular, acquiring communicative competence is a stronger objective for older learners, as an appropriate use of language in social contexts ensures a better community immersion and well-being. This skill is best attained through interactions with proficient speakers, but if this option is not available, social robots present a good alternative for this purpose. However, to obtain optimal results, a robot companion should adapt to the learner's proficiency level and motivation continuously to encourage speech production and increase fluency. Our work attempts to achieve this goal by developing an adaptive robot that modifies its spoken dialogue strategy, and visual feedback, to reflect a student's knowledge, proficiency and engagement levels in situated interactions for long-term learning.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2022
Series
ACM IEEE International Conference on Human-Robot Interaction, ISSN 2167-2121
Keywords
social robot, communicative competence, dialogue system, situated interaction, second language learning
National Category
Interaction Technologies
Identifiers
urn:nbn:se:kth:diva-313359 (URN)10.1109/HRI53351.2022.9889517 (DOI)000869793600180 ()2-s2.0-85140431918 (Scopus ID)
Conference
17th Annual ACM/IEEE International Conference on Human-Robot Interaction (HRI), MAR 07-10, 2022, ELECTR NETWORK
Note

QC 20220607

Available from: 2022-06-02 Created: 2022-06-02 Last updated: 2022-12-16Bibliographically approved
Engwall, O., Cumbal, R., Lopes, J., Ljung, M. & Månsson, L. (2022). Identification of Low-engaged Learners in Robot-led Second Language Conversations with Adults. ACM Transactions on Human-Robot Interaction, 11(2), Article ID 18.
Open this publication in new window or tab >>Identification of Low-engaged Learners in Robot-led Second Language Conversations with Adults
Show others...
2022 (English)In: ACM Transactions on Human-Robot Interaction, E-ISSN 2573-9522, Vol. 11, no 2, article id 18Article in journal (Refereed) Published
Abstract [en]

The main aim of this study is to investigate if verbal, vocal, and facial information can be used to identify low-engaged second language learners in robot-led conversation practice. The experiments were performed on voice recordings and video data from 50 conversations, in which a robotic head talks with pairs of adult language learners using four different interaction strategies with varying robot-learner focus and initiative. It was found that these robot interaction strategies influenced learner activity and engagement. The verbal analysis indicated that learners with low activity rated the robot significantly lower on two out of four scales related to social competence. The acoustic vocal and video-based facial analysis, based on manual annotations or machine learning classification, both showed that learners with low engagement rated the robot's social competencies consistently, and in several cases significantly, lower, and in addition rated the learning effectiveness lower. The agreement between manual and automatic identification of low-engaged learners based on voice recordings or face videos was further found to be adequate for future use. These experiments constitute a first step towards enabling adaption to learners' activity and engagement through within- and between-strategy changes of the robot's interaction with learners.

Place, publisher, year, edition, pages
Association for Computing Machinery (ACM), 2022
Keywords
Robot-assisted language learning, user engagement, speech emotion recognition, facial emotion expressions
National Category
Human Computer Interaction
Identifiers
urn:nbn:se:kth:diva-311036 (URN)10.1145/3503799 (DOI)000774332200008 ()2-s2.0-85127492914 (Scopus ID)
Note

Not duplicate with DiVA 1612730

QC 20220420

Available from: 2022-04-20 Created: 2022-04-20 Last updated: 2023-01-09Bibliographically approved
Engwall, O., Águas Lopes, J. D. & Cumbal, R. (2022). Is a Wizard-of-Oz Required for Robot-Led Conversation Practice in a Second Language?. International Journal of Social Robotics
Open this publication in new window or tab >>Is a Wizard-of-Oz Required for Robot-Led Conversation Practice in a Second Language?
2022 (English)In: International Journal of Social Robotics, ISSN 1875-4791, E-ISSN 1875-4805Article in journal (Refereed) Published
Abstract [en]

The large majority of previous work on human-robot conversations in a second language has been performed with a human wizard-of-Oz. The reasons are that automatic speech recognition of non-native conversational speech is considered to be unreliable and that the dialogue management task of selecting robot utterances that are adequate at a given turn is complex in social conversations. This study therefore investigates if robot-led conversation practice in a second language with pairs of adult learners could potentially be managed by an autonomous robot. We first investigate how correct and understandable transcriptions of second language learner utterances are when made by a state-of-the-art speech recogniser. We find both a relatively high word error rate (41%) and that a substantial share (42%) of the utterances are judged to be incomprehensible or only partially understandable by a human reader. We then evaluate how adequate the robot utterance selection is, when performed manually based on the speech recognition transcriptions or autonomously using (a) predefined sequences of robot utterances, (b) a general state-of-the-art language model that selects utterances based on learner input or the preceding robot utterance, or (c) a custom-made statistical method that is trained on observations of the wizard’s choices in previous conversations. It is shown that adequate or at least acceptable robot utterances are selected by the human wizard in most cases (96%), even though the ASR transcriptions have a high word error rate. Further, the custom-made statistical method performs as well as manual selection of robot utterances based on ASR transcriptions. It was also found that the interaction strategy that the robot employed, which differed regarding how much the robot maintained the initiative in the conversation and if the focus of the conversation was on the robot or the learners, had marginal effects on the word error rate and understandability of the transcriptions but larger effects on the adequateness of the utterance selection. Autonomous robot-led conversations may hence work better with some robot interaction strategies.

Place, publisher, year, edition, pages
Springer Nature, 2022
Keywords
Robot-assisted language learning, Conversational practice, Non-native speech recognition, Dialogue management for spoken human-robot interaction
National Category
Natural Language Processing
Research subject
Computer Science
Identifiers
urn:nbn:se:kth:diva-306942 (URN)10.1007/s12369-021-00849-8 (DOI)000739285100001 ()2-s2.0-85122404446 (Scopus ID)
Funder
Swedish Research Council, 2016-03698Marcus and Amalia Wallenberg Foundation, MAW 2020.0052
Note

QC 20220112

Available from: 2022-01-05 Created: 2022-01-05 Last updated: 2025-02-07Bibliographically approved
Engwall, O., Lopes, J., Cumbal, R., Berndtson, G., Lindström, R., Ekman, P., . . . Mekonnen, M. (2022). Learner and teacher perspectives on robot-led L2 conversation practice. ReCALL, 34(3), 344-359
Open this publication in new window or tab >>Learner and teacher perspectives on robot-led L2 conversation practice
Show others...
2022 (English)In: ReCALL, ISSN 0958-3440, E-ISSN 1474-0109, Vol. 34, no 3, p. 344-359Article in journal (Refereed) Published
Abstract [en]

This article focuses on designing and evaluating conversation practice in a second language (L2) with a robot that employs human spoken and non-verbal interaction strategies. Based on an analysis of previous work and semi-structured interviews with L2 learners and teachers, recommendations for robot-led conversation practice for adult learners at intermediate level are first defined, focused on language learning, on the social context, on the conversational structure and on verbal and visual aspects of the robot moderation. Guided by these recommendations, an experiment is set up, in which 12 pairs of L2 learners of Swedish interact with a robot in short social conversations. These robot-learner interactions are evaluated through post-session interviews with the learners, teachers' ratings of the robot's behaviour and analyses of the video-recorded conversations, resulting in a set of guidelines for robot-led conversation practice: (1) societal and personal topics increase the practice's meaningfulness for learners; (2) strategies and methods for providing corrective feedback during conversation practice need to be explored further; (3) learners should be encouraged to support each other if the robot has difficulties adapting to their linguistic level; (4) the robot should establish a social relationship by contributing with its own story, remembering the participants' input, and making use of non-verbal communication signals; and (5) improvements are required regarding naturalness and intelligibility of text-to-speech synthesis, in particular its speed, if it is to be used for conversations with L2 learners. 

Place, publisher, year, edition, pages
Cambridge University Press (CUP), 2022
Keywords
conversation practice, educational robots, L2 speaking, multiparty interaction
National Category
Human Computer Interaction Learning Natural Language Processing
Identifiers
urn:nbn:se:kth:diva-323786 (URN)10.1017/S0958344022000027 (DOI)000792211500001 ()2-s2.0-85129635423 (Scopus ID)
Note

QC 20230213

Available from: 2023-02-13 Created: 2023-02-13 Last updated: 2025-02-01Bibliographically approved
Cumbal, R., Kazzi, D. A., Winberg, V. & Engwall, O. (2022). Shaping unbalanced multi-party interactions through adaptive robot backchannels. In: IVA 2022 - Proceedings of the 22nd ACM International Conference on Intelligent Virtual Agents: . Paper presented at IVA 2022 - Proceedings of the 22nd ACM International Conference on Intelligent Virtual Agents. Association for Computing Machinery, Inc
Open this publication in new window or tab >>Shaping unbalanced multi-party interactions through adaptive robot backchannels
2022 (English)In: IVA 2022 - Proceedings of the 22nd ACM International Conference on Intelligent Virtual Agents, Association for Computing Machinery, Inc , 2022Conference paper, Published paper (Refereed)
Abstract [en]

Non-verbal cues used in human communication have shown to be efficient in shaping speaking interactions. When applied to virtual agents or social robots, results imply that a similar effect is expected in dyad settings. In this study, we explore how encouraging, vocal and non-vocal, backchannels can stimulate speaking participation in a game-based multi-party interaction, where unbalanced contribution is expected. We design the study using a social robot, taking part in a language game with native speakers and language learners, to evaluate how an adaptive generation of backchannels, that targets the least speaking participant to encourage more speaking contribution, affects the group and individual participant's behavior. We report results from experiments with 30 subjects divided in pairs assigned to the adaptive (encouraging) and (neutral) control condition. Our results show that the speaking participation of the least active speaker increases significantly when the robot uses an adaptive backchanneling strategy. At the same time, the participation of the more active speaker slightly decreases, which causes the combined speaking time of both participants to be similar between the Control and Experimental conditions. The adaptive strategy further leads to a 50% decrease in the difference in speaker shares between the two participants (indicating a more balanced participation) compared to the Control condition. However, this distribution between speaker ratios is not significantly different from the Control.

Place, publisher, year, edition, pages
Association for Computing Machinery, Inc, 2022
Keywords
encouraging behavior, human-robot interaction, non-verbal behavior, second language learning, social robot, Social robots, Back channels, Condition, Humans-robot interactions, Language learning, Multiparty interaction, Non-verbal behaviours, Second language, Machine design
National Category
Human Computer Interaction
Identifiers
urn:nbn:se:kth:diva-327284 (URN)10.1145/3514197.3549680 (DOI)2-s2.0-85138738227 (Scopus ID)
Conference
IVA 2022 - Proceedings of the 22nd ACM International Conference on Intelligent Virtual Agents
Note

QC 20230524

Available from: 2023-05-24 Created: 2023-05-24 Last updated: 2024-02-26Bibliographically approved
Organisations
Identifiers
ORCID iD: ORCID iD iconorcid.org/0000-0003-4472-4732

Search in DiVA

Show all publications