kth.sePublications
Change search
Link to record
Permanent link

Direct link
Publications (10 of 48) Show all publications
Cumbal, R. & Engwall, O. (2024). Speaking Transparently: Social Robots in Educational Settings. In: Companion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction (HRI '24 Companion), March 11--14, 2024, Boulder, CO, USA: . Paper presented at International Conference on Human-Robot Interaction.
Open this publication in new window or tab >>Speaking Transparently: Social Robots in Educational Settings
2024 (English)In: Companion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction (HRI '24 Companion), March 11--14, 2024, Boulder, CO, USA, 2024Conference paper, Published paper (Refereed)
Abstract [en]

The recent surge in popularity of Large Language Models, known for their inherent opacity, has increased the interest in fostering transparency in technology designed for human interaction. This concern is equally prevalent in the development of Social Robots, particularly when these are designed to engage in critical areas of our society, such as education or healthcare. In this paper we propose an experiment to investigate how users can be made aware of the automated decision processes when interacting in a discussion with a social robot. Our main objective is to assess the effectiveness of verbal expressions in fostering transparency within groups of individuals as they engage with a robot. We describe the proposed interactive settings, system design, and our approach to enhance the transparency in a robot's decision-making process for multi-party interactions.

Keywords
Dialogue system, transparency, multi-party, conversation
National Category
Robotics
Research subject
Computer Science
Identifiers
urn:nbn:se:kth:diva-343862 (URN)10.1145/3610978.3640717 (DOI)2-s2.0-85188090310 (Scopus ID)
Conference
International Conference on Human-Robot Interaction
Note

QC 20240328

Available from: 2024-02-26 Created: 2024-02-26 Last updated: 2024-03-28Bibliographically approved
Engwall, O., Bandera Rubio, J. P., Bensch, S., Haring, K. S., Kanda, T., Núñez, P., . . . Sgorbissa, A. (2023). Editorial: Socially, culturally and contextually aware robots. Frontiers in Robotics and AI, 10, Article ID 1232215.
Open this publication in new window or tab >>Editorial: Socially, culturally and contextually aware robots
Show others...
2023 (English)In: Frontiers in Robotics and AI, E-ISSN 2296-9144, Vol. 10, article id 1232215Article in journal, Editorial material (Other academic) Published
Place, publisher, year, edition, pages
Frontiers Media SA, 2023
Keywords
context awareness, cultural awareness, original research, reviews-articles, social robots
National Category
Computer Vision and Robotics (Autonomous Systems)
Identifiers
urn:nbn:se:kth:diva-340355 (URN)10.3389/frobt.2023.1232215 (DOI)001105825000001 ()2-s2.0-85177180605 (Scopus ID)
Note

QC 20231204

Available from: 2023-12-04 Created: 2023-12-04 Last updated: 2023-12-11Bibliographically approved
Engwall, O., Cumbal, R. & Majlesi, A. R. (2023). Socio-cultural perception of robot backchannels. Frontiers in Robotics and AI, 10
Open this publication in new window or tab >>Socio-cultural perception of robot backchannels
2023 (English)In: Frontiers in Robotics and AI, E-ISSN 2296-9144, Vol. 10Article in journal (Refereed) Published
Abstract [en]

Introduction: Backchannels, i.e., short interjections by an interlocutor to indicate attention, understanding or agreement regarding utterances by another conversation participant, are fundamental in human-human interaction. Lack of backchannels or if they have unexpected timing or formulation may influence the conversation negatively, as misinterpretations regarding attention, understanding or agreement may occur. However, several studies over the years have shown that there may be cultural differences in how backchannels are provided and perceived and that these differences may affect intercultural conversations. Culturally aware robots must hence be endowed with the capability to detect and adapt to the way these conversational markers are used across different cultures. Traditionally, culture has been defined in terms of nationality, but this is more and more considered to be a stereotypic simplification. We therefore investigate several socio-cultural factors, such as the participants’ gender, age, first language, extroversion and familiarity with robots, that may be relevant for the perception of backchannels.

Methods: We first cover existing research on cultural influence on backchannel formulation and perception in human-human interaction and on backchannel implementation in Human-Robot Interaction. We then present an experiment on second language spoken practice, in which we investigate how backchannels from the social robot Furhat influence interaction (investigated through speaking time ratios and ethnomethodology and multimodal conversation analysis) and impression of the robot (measured by post-session ratings). The experiment, made in a triad word game setting, is focused on if activity-adaptive robot backchannels may redistribute the participants’ speaking time ratio, and/or if the participants’ assessment of the robot is influenced by the backchannel strategy. The goal is to explore how robot backchannels should be adapted to different language learners to encourage their participation while being perceived as socio-culturally appropriate.

Results: We find that a strategy that displays more backchannels towards a less active speaker may substantially decrease the difference in speaking time between the two speakers, that different socio-cultural groups respond differently to the robot’s backchannel strategy and that they also perceive the robot differently after the session.

Discussion: We conclude that the robot may need different backchanneling strategies towards speakers from different socio-cultural groups in order to encourage them to speak and have a positive perception of the robot.

 

Place, publisher, year, edition, pages
Frontiers Media S.A., 2023
Keywords
backchannels, multiparty HRI, robot-assisted language learning, spoken practice, cultural effects
National Category
Human Computer Interaction
Research subject
Computer Science
Identifiers
urn:nbn:se:kth:diva-323334 (URN)10.3389/frobt.2023.988042 (DOI)000935012900001 ()36777379 (PubMedID)2-s2.0-85147686864 (Scopus ID)
Funder
Marcus and Amalia Wallenberg Foundation, 2020.0052
Note

QC 20230130

Available from: 2023-01-26 Created: 2023-01-26 Last updated: 2024-02-26Bibliographically approved
Cumbal, R., Axelsson, A., Mehta, S. & Engwall, O. (2023). Stereotypical nationality representations in HRI: perspectives from international young adults. Frontiers in Robotics and AI, 10, Article ID 1264614.
Open this publication in new window or tab >>Stereotypical nationality representations in HRI: perspectives from international young adults
2023 (English)In: Frontiers in Robotics and AI, E-ISSN 2296-9144, Vol. 10, article id 1264614Article in journal (Refereed) Published
Abstract [en]

People often form immediate expectations about other people, or groups of people, based on visual appearance and characteristics of their voice and speech. These stereotypes, often inaccurate or overgeneralized, may translate to robots that carry human-like qualities. This study aims to explore if nationality-based preconceptions regarding appearance and accents can be found in people's perception of a virtual and a physical social robot. In an online survey with 80 subjects evaluating different first-language-influenced accents of English and nationality-influenced human-like faces for a virtual robot, we find that accents, in particular, lead to preconceptions on perceived competence and likeability that correspond to previous findings in social science research. In a physical interaction study with 74 participants, we then studied if the perception of competence and likeability is similar after interacting with a robot portraying one of four different nationality representations from the online survey. We find that preconceptions on national stereotypes that appeared in the online survey vanish or are overshadowed by factors related to general interaction quality. We do, however, find some effects of the robot's stereotypical alignment with the subject group, with Swedish subjects (the majority group in this study) rating the Swedish-accented robot as less competent than the international group, but, on the other hand, recalling more facts from the Swedish robot's presentation than the international group does. In an extension in which the physical robot was replaced by a virtual robot interacting in the same scenario online, we further found the same results that preconceptions are of less importance after actual interactions, hence demonstrating that the differences in the ratings of the robot between the online survey and the interaction is not due to the interaction medium. We hence conclude that attitudes towards stereotypical national representations in HRI have a weak effect, at least for the user group included in this study (primarily educated young students in an international setting).

Place, publisher, year, edition, pages
Frontiers Media SA, 2023
Keywords
accent, appearance, social robot, nationality, stereotype, impression, competence, likeability
National Category
Human Computer Interaction
Identifiers
urn:nbn:se:kth:diva-341526 (URN)10.3389/frobt.2023.1264614 (DOI)001115613500001 ()38077460 (PubMedID)2-s2.0-85178920101 (Scopus ID)
Note

QC 20231222

Available from: 2023-12-22 Created: 2023-12-22 Last updated: 2024-02-26Bibliographically approved
Engwall, O., Cumbal, R., Lopes, J., Ljung, M. & Månsson, L. (2022). Identification of Low-engaged Learners in Robot-led Second Language Conversations with Adults. ACM Transactions on Human-Robot Interaction, 11(2), Article ID 18.
Open this publication in new window or tab >>Identification of Low-engaged Learners in Robot-led Second Language Conversations with Adults
Show others...
2022 (English)In: ACM Transactions on Human-Robot Interaction, E-ISSN 2573-9522, Vol. 11, no 2, article id 18Article in journal (Refereed) Published
Abstract [en]

The main aim of this study is to investigate if verbal, vocal, and facial information can be used to identify low-engaged second language learners in robot-led conversation practice. The experiments were performed on voice recordings and video data from 50 conversations, in which a robotic head talks with pairs of adult language learners using four different interaction strategies with varying robot-learner focus and initiative. It was found that these robot interaction strategies influenced learner activity and engagement. The verbal analysis indicated that learners with low activity rated the robot significantly lower on two out of four scales related to social competence. The acoustic vocal and video-based facial analysis, based on manual annotations or machine learning classification, both showed that learners with low engagement rated the robot's social competencies consistently, and in several cases significantly, lower, and in addition rated the learning effectiveness lower. The agreement between manual and automatic identification of low-engaged learners based on voice recordings or face videos was further found to be adequate for future use. These experiments constitute a first step towards enabling adaption to learners' activity and engagement through within- and between-strategy changes of the robot's interaction with learners.

Place, publisher, year, edition, pages
Association for Computing Machinery (ACM), 2022
Keywords
Robot-assisted language learning, user engagement, speech emotion recognition, facial emotion expressions
National Category
Human Computer Interaction
Identifiers
urn:nbn:se:kth:diva-311036 (URN)10.1145/3503799 (DOI)000774332200008 ()2-s2.0-85127492914 (Scopus ID)
Note

Not duplicate with DiVA 1612730

QC 20220420

Available from: 2022-04-20 Created: 2022-04-20 Last updated: 2023-01-09Bibliographically approved
Engwall, O. & David Lopes, J. (2022). Interaction and collaboration in robot-assisted language learning for adults. Computer Assisted Language Learning, 35(5-6), 1273-1309
Open this publication in new window or tab >>Interaction and collaboration in robot-assisted language learning for adults
2022 (English)In: Computer Assisted Language Learning, ISSN 0958-8221, E-ISSN 1744-3210, Vol. 35, no 5-6, p. 1273-1309Article in journal (Refereed) Published
Abstract [en]

This article analyses how robot–learner interaction in robot-assisted language learning (RALL) is influenced by the interaction behaviour of the robot. Since the robot behaviour is to a large extent determined by the combination of teaching strategy, robot role and robot type, previous studies in RALL are first summarised with respect to which combinations that have been chosen, the rationale behind the choice and the effects on interaction and learning. The goal of the summary is to determine a suitable pedagogical set-up for RALL with adult learners, since previous RALL studies have almost exclusively been performed with children and youths. A user study in which 33 adult second language learners practice Swedish in three-party conversations with an anthropomorphic robot head is then presented. It is demonstrated how different robot interaction behaviours influence interaction between the robot and the learners and between the two learners. Through an analysis of learner interaction, collaboration and learner ratings for the different robot behaviours, it is observed that the learners were most positive towards the robot behaviour that focused on interviewing one learner at the time (highest average ratings), but that they were the most active in sessions when the robot encouraged learner–learner interaction. Moreover, the preferences and activity differed between learner pairs, depending on, e.g., their proficiency level and how well they knew the peer. It is therefore concluded that the robot behaviour needs to adapt to such factors. In addition, collaboration with the peer played an important part in conversation practice sessions to deal with linguistic difficulties or communication problems with the robot.

Place, publisher, year, edition, pages
Routledge, 2022
Keywords
Collaborative language learning; communicative language teaching; educational robots; human–robot interaction; spoken practice
National Category
Robotics
Research subject
Computer Science
Identifiers
urn:nbn:se:kth:diva-278975 (URN)10.1080/09588221.2020.1799821 (DOI)000557599700001 ()2-s2.0-85089256652 (Scopus ID)
Projects
Collaborative Robot-assisted Language Learning
Funder
Swedish Research Council, 2016-03698
Note

QC 20200818

Available from: 2020-08-08 Created: 2020-08-08 Last updated: 2023-10-02Bibliographically approved
Engwall, O., Águas Lopes, J. D. & Cumbal, R. (2022). Is a Wizard-of-Oz Required for Robot-Led Conversation Practice in a Second Language?. International Journal of Social Robotics
Open this publication in new window or tab >>Is a Wizard-of-Oz Required for Robot-Led Conversation Practice in a Second Language?
2022 (English)In: International Journal of Social Robotics, ISSN 1875-4791, E-ISSN 1875-4805Article in journal (Refereed) Published
Abstract [en]

The large majority of previous work on human-robot conversations in a second language has been performed with a human wizard-of-Oz. The reasons are that automatic speech recognition of non-native conversational speech is considered to be unreliable and that the dialogue management task of selecting robot utterances that are adequate at a given turn is complex in social conversations. This study therefore investigates if robot-led conversation practice in a second language with pairs of adult learners could potentially be managed by an autonomous robot. We first investigate how correct and understandable transcriptions of second language learner utterances are when made by a state-of-the-art speech recogniser. We find both a relatively high word error rate (41%) and that a substantial share (42%) of the utterances are judged to be incomprehensible or only partially understandable by a human reader. We then evaluate how adequate the robot utterance selection is, when performed manually based on the speech recognition transcriptions or autonomously using (a) predefined sequences of robot utterances, (b) a general state-of-the-art language model that selects utterances based on learner input or the preceding robot utterance, or (c) a custom-made statistical method that is trained on observations of the wizard’s choices in previous conversations. It is shown that adequate or at least acceptable robot utterances are selected by the human wizard in most cases (96%), even though the ASR transcriptions have a high word error rate. Further, the custom-made statistical method performs as well as manual selection of robot utterances based on ASR transcriptions. It was also found that the interaction strategy that the robot employed, which differed regarding how much the robot maintained the initiative in the conversation and if the focus of the conversation was on the robot or the learners, had marginal effects on the word error rate and understandability of the transcriptions but larger effects on the adequateness of the utterance selection. Autonomous robot-led conversations may hence work better with some robot interaction strategies.

Place, publisher, year, edition, pages
Springer Nature, 2022
Keywords
Robot-assisted language learning, Conversational practice, Non-native speech recognition, Dialogue management for spoken human-robot interaction
National Category
Language Technology (Computational Linguistics)
Research subject
Computer Science
Identifiers
urn:nbn:se:kth:diva-306942 (URN)10.1007/s12369-021-00849-8 (DOI)000739285100001 ()2-s2.0-85122404446 (Scopus ID)
Funder
Swedish Research Council, 2016-03698Marcus and Amalia Wallenberg Foundation, MAW 2020.0052
Note

QC 20220112

Available from: 2022-01-05 Created: 2022-01-05 Last updated: 2022-09-23Bibliographically approved
Engwall, O., Lopes, J., Cumbal, R., Berndtson, G., Lindström, R., Ekman, P., . . . Mekonnen, M. (2022). Learner and teacher perspectives on robot-led L2 conversation practice. ReCALL, 34(3), 344-359
Open this publication in new window or tab >>Learner and teacher perspectives on robot-led L2 conversation practice
Show others...
2022 (English)In: ReCALL, ISSN 0958-3440, E-ISSN 1474-0109, Vol. 34, no 3, p. 344-359Article in journal (Refereed) Published
Abstract [en]

This article focuses on designing and evaluating conversation practice in a second language (L2) with a robot that employs human spoken and non-verbal interaction strategies. Based on an analysis of previous work and semi-structured interviews with L2 learners and teachers, recommendations for robot-led conversation practice for adult learners at intermediate level are first defined, focused on language learning, on the social context, on the conversational structure and on verbal and visual aspects of the robot moderation. Guided by these recommendations, an experiment is set up, in which 12 pairs of L2 learners of Swedish interact with a robot in short social conversations. These robot-learner interactions are evaluated through post-session interviews with the learners, teachers' ratings of the robot's behaviour and analyses of the video-recorded conversations, resulting in a set of guidelines for robot-led conversation practice: (1) societal and personal topics increase the practice's meaningfulness for learners; (2) strategies and methods for providing corrective feedback during conversation practice need to be explored further; (3) learners should be encouraged to support each other if the robot has difficulties adapting to their linguistic level; (4) the robot should establish a social relationship by contributing with its own story, remembering the participants' input, and making use of non-verbal communication signals; and (5) improvements are required regarding naturalness and intelligibility of text-to-speech synthesis, in particular its speed, if it is to be used for conversations with L2 learners. 

Place, publisher, year, edition, pages
Cambridge University Press (CUP), 2022
Keywords
conversation practice, educational robots, L2 speaking, multiparty interaction
National Category
Human Computer Interaction Learning Language Technology (Computational Linguistics)
Identifiers
urn:nbn:se:kth:diva-323786 (URN)10.1017/S0958344022000027 (DOI)000792211500001 ()2-s2.0-85129635423 (Scopus ID)
Note

QC 20230213

Available from: 2023-02-13 Created: 2023-02-13 Last updated: 2023-02-13Bibliographically approved
Cumbal, R., Kazzi, D. A., Winberg, V. & Engwall, O. (2022). Shaping unbalanced multi-party interactions through adaptive robot backchannels. In: IVA 2022 - Proceedings of the 22nd ACM International Conference on Intelligent Virtual Agents: . Paper presented at IVA 2022 - Proceedings of the 22nd ACM International Conference on Intelligent Virtual Agents. Association for Computing Machinery, Inc
Open this publication in new window or tab >>Shaping unbalanced multi-party interactions through adaptive robot backchannels
2022 (English)In: IVA 2022 - Proceedings of the 22nd ACM International Conference on Intelligent Virtual Agents, Association for Computing Machinery, Inc , 2022Conference paper, Published paper (Refereed)
Abstract [en]

Non-verbal cues used in human communication have shown to be efficient in shaping speaking interactions. When applied to virtual agents or social robots, results imply that a similar effect is expected in dyad settings. In this study, we explore how encouraging, vocal and non-vocal, backchannels can stimulate speaking participation in a game-based multi-party interaction, where unbalanced contribution is expected. We design the study using a social robot, taking part in a language game with native speakers and language learners, to evaluate how an adaptive generation of backchannels, that targets the least speaking participant to encourage more speaking contribution, affects the group and individual participant's behavior. We report results from experiments with 30 subjects divided in pairs assigned to the adaptive (encouraging) and (neutral) control condition. Our results show that the speaking participation of the least active speaker increases significantly when the robot uses an adaptive backchanneling strategy. At the same time, the participation of the more active speaker slightly decreases, which causes the combined speaking time of both participants to be similar between the Control and Experimental conditions. The adaptive strategy further leads to a 50% decrease in the difference in speaker shares between the two participants (indicating a more balanced participation) compared to the Control condition. However, this distribution between speaker ratios is not significantly different from the Control.

Place, publisher, year, edition, pages
Association for Computing Machinery, Inc, 2022
Keywords
encouraging behavior, human-robot interaction, non-verbal behavior, second language learning, social robot, Social robots, Back channels, Condition, Humans-robot interactions, Language learning, Multiparty interaction, Non-verbal behaviours, Second language, Machine design
National Category
Human Computer Interaction
Identifiers
urn:nbn:se:kth:diva-327284 (URN)10.1145/3514197.3549680 (DOI)2-s2.0-85138738227 (Scopus ID)
Conference
IVA 2022 - Proceedings of the 22nd ACM International Conference on Intelligent Virtual Agents
Note

QC 20230524

Available from: 2023-05-24 Created: 2023-05-24 Last updated: 2024-02-26Bibliographically approved
Gillet, S., Cumbal, R., Abelho Pereira, A. T., Lopes, J., Engwall, O. & Leite, I. (2021). Robot Gaze Can Mediate Participation Imbalance in Groups with Different Skill Levels. In: Proceedings of the 2021 ACM/IEEE International Conference on Human-Robot Interaction: . Paper presented at ACM/IEEE International Conference on Human-Robot Interaction March 09 –11 (pp. 303-311). Association for Computing Machinery
Open this publication in new window or tab >>Robot Gaze Can Mediate Participation Imbalance in Groups with Different Skill Levels
Show others...
2021 (English)In: Proceedings of the 2021 ACM/IEEE International Conference on Human-Robot Interaction, Association for Computing Machinery , 2021, p. 303-311Conference paper, Published paper (Refereed)
Abstract [en]

Many small group activities, like working teams or study groups, have a high dependency on the skill of each group member. Differences in skill level among participants can affect not only the performance of a team but also influence the social interaction of its members. In these circumstances, an active member could balance individual participation without exerting direct pressure on specific members by using indirect means of communication, such as gaze behaviors. Similarly, in this study, we evaluate whether a social robot can balance the level of participation in a language skill-dependent game, played by a native speaker and a second language learner. In a between-subjects study (N = 72), we compared an adaptive robot gaze behavior, that was targeted to increase the level of contribution of the least active player, with a non-adaptive gaze behavior. Our results imply that, while overall levels of speech participation were influenced predominantly by personal traits of the participants, the robot’s adaptive gaze behavior could shape the interaction among participants which lead to more even participation during the game.

Place, publisher, year, edition, pages
Association for Computing Machinery, 2021
Series
HRI ’21
Keywords
language learning, gaze, multiparty interaction, group dynamics
National Category
Interaction Technologies
Identifiers
urn:nbn:se:kth:diva-292043 (URN)10.1145/3434073.3444670 (DOI)001051690500035 ()2-s2.0-85102757966 (Scopus ID)
Conference
ACM/IEEE International Conference on Human-Robot Interaction March 09 –11
Funder
Swedish Foundation for Strategic Research , FFL18-0199Swedish Research Council, 2017-05189Swedish Research Council, 2016-03698
Note

QC 20210710

Available from: 2021-03-24 Created: 2021-03-24 Last updated: 2024-03-18Bibliographically approved
Organisations
Identifiers
ORCID iD: ORCID iD iconorcid.org/0000-0003-4532-014X

Search in DiVA

Show all publications