kth.sePublications
Change search
Link to record
Permanent link

Direct link
Publications (10 of 51) Show all publications
Gonzalez Oliveras, P., Engwall, O. & Wilde, A. (2025). Social Educational Robotics and Learning Analytics: A Scoping Review of an Emerging Field. International Journal of Social Robotics
Open this publication in new window or tab >>Social Educational Robotics and Learning Analytics: A Scoping Review of an Emerging Field
2025 (English)In: International Journal of Social Robotics, ISSN 1875-4791, E-ISSN 1875-4805Article in journal (Refereed) Published
Abstract [en]

Social Educational Robotics and Learning Analytics (LA) are prominent fields in technology-enhanced learning, but their combined potential remains underexplored, despite methodological similarities. Increasingly, signs of joint interests have emerged, with a surge in publications mentioning both social robots and learning analytics in the last five years. We therefore conducted a scoping review to explore if a new research field is emerging. We identified 29 empirical studies that combine social robots and LA, but also found that few studies explicitly state that social educational robots and LA are used in combination. Several studies used social educational robots that adapted to the learners or the learning environment based on interaction data. This signifies that they are in fact employing the feedback cycle that is at the core of LA methodology, but as most of these studies update the learner model using post-session data (e.g., learner improvement or feedback), they are long-term studies with repeated interventions that are applying LA methodology inadvertently. There are also benefits for LA research to use social educational robots, since LA increasingly uses an array of equipment to collect multimodal data, and all studies in this review employ at least two input modalities (mu = 4.4). Social robots provide the possibility to collect this data non-intrusively with the robot itself, in addition to creating a pedagogically boosted interaction compared to traditional LA interventions (e.g., learning management systems). By raising researchers' awareness of how close the fields of social educational robotics and LA are, substantial synergy effects could therefore be gained.

Place, publisher, year, edition, pages
Springer Nature, 2025
Keywords
Human-robot interaction, Social robots, Educational robots, Learning analytics
National Category
Human Computer Interaction
Identifiers
urn:nbn:se:kth:diva-362926 (URN)10.1007/s12369-025-01235-4 (DOI)001457918300001 ()2-s2.0-105001712315 (Scopus ID)
Note

QC 20250430

Available from: 2025-04-30 Created: 2025-04-30 Last updated: 2025-04-30Bibliographically approved
Jansson, M., Tian, K., Hrastinski, S. & Engwall, O. (2024). An initial exploration of semi-automated tutoring: How AI could be used as support for online human tutors. In: Proceedings of the Fourteenth International Conference on Networked Learning: . Paper presented at The Fourteenth International Conference on Networked Learning, Valetta, Malta, 15-17 May, 2024. Aalborg University
Open this publication in new window or tab >>An initial exploration of semi-automated tutoring: How AI could be used as support for online human tutors
2024 (English)In: Proceedings of the Fourteenth International Conference on Networked Learning, Aalborg University , 2024Conference paper, Published paper (Refereed)
Abstract [en]

In this paper, we begin our process of incorporating an AI bot in an online chat tutoring setting as a support for the tutor. We explore how an AI bot could give suggestions for tutor messages, although the human tutor will control how to communicate with the student. Tutoring, an important dimension of networked learning, has long been seen as a beneficial approach to students’ learning. An AI bot has the potential to aid tutors in the tutoring process and contribute to the scalability. The present pilot study was conducted in the tutoring setting of the Math Coach program. In the program, teacher students aid students from upper primary school to upper secondary school in mathematics through an online text-based chat system. Llama2 was used as a large language model (LLM), fine-tuned for Swedish comprehension utilizing the Math Coach system's chat logs. Four coaches, teacher students at a technical Swedish university and active in the Math Coach program, were invited to interact with the AI bot and participate in a group discussion. The coaches interacted individually with the AI bot while the chat conversation was displayed on a monitor so all participants could discuss the interaction while it took place. A semi-structured interview approach was taken and the participants were also encouraged to 'think aloud' about their experience. In the discussions, the coaches expressed surprise by the AI's social aspect. They perceived the AI bot as friendly with a positive attitude and were especially surprised by its ability to correctly place appropriate emojis. The coaches agreed that the AI was able to ask both appropriate and helpful questions and share some good guidance for how to proceed in the problem-solving process. However, they felt that the AI bot was not able to offer sufficient mathematical guidance, oftentimes the AI bot was confidently wrong. It also wrote too long messages, which humans would typically separate into several chat messages, and did not wait for a response but instead moved too quickly towards the solution. Moving forward we plan to address the effects of improved prompts on the AI bot and continue finetuning the LLM. We will continue to conduct pilot studies and eventually conduct more large-scale empirical studies.

Place, publisher, year, edition, pages
Aalborg University, 2024
National Category
Educational Sciences
Identifiers
urn:nbn:se:kth:diva-352177 (URN)10.54337/nlc.v14i1.8070 (DOI)
Conference
The Fourteenth International Conference on Networked Learning, Valetta, Malta, 15-17 May, 2024
Note

QC 20240906

Available from: 2024-08-23 Created: 2024-08-23 Last updated: 2025-02-18Bibliographically approved
Kamelabad, A. M., Engwall, O. & Skantze, G. (2024). Conformity and Trust in Multi-party vs. Individual Human-Robot Interaction. In: Rachael Jack, Mathieu Chollet, Ruth Aylett, Timothy Bickmore, Stacy Marsella, Gale Lucas (Ed.), Proceedings of the 24th ACM International Conference on Intelligent Virtual Agents: . Paper presented at IVA '24: ACM International Conference on Intelligent Virtual Agents, GLASGOW United Kingdom, September 16-19, 2024. New York, NY United States: Association for Computing Machinery (ACM), Article ID 4.
Open this publication in new window or tab >>Conformity and Trust in Multi-party vs. Individual Human-Robot Interaction
2024 (English)In: Proceedings of the 24th ACM International Conference on Intelligent Virtual Agents / [ed] Rachael Jack, Mathieu Chollet, Ruth Aylett, Timothy Bickmore, Stacy Marsella, Gale Lucas, New York, NY United States: Association for Computing Machinery (ACM) , 2024, article id 4Conference paper, Published paper (Refereed)
Abstract [en]

In this study, we explored how conformity and trust vary in adolescent students’ interactions with a social robot. Specifically, we compared how this was influenced by whether the participants had individual or multi-party interaction with robot and whether the robot was portrayed as an adult or a child through appearance and voice. Our experiment involved 75 Swedish middle school students participating in a card sorting game with the Furhat robot, where the objective was to discuss and reach an agreement on the card sequence. The data analysis focused firstly on the participants’ willingness to rearrange cards following the robot’s suggestions and secondly their post-session subjective trust in the robot’s advice. Results indicated that individuals interacting with the robot individually were more likely to conform to its suggestions than those interacting with it together with a peer. Individuals interacting alone with the robot also showed higher post-session trust levels than those in multi-party settings, indicating group size impacts robot trustworthiness perceptions. However, the robot’s perceived age did not affect the level of conformity. Exploratory analyses also showed that mutual understanding was lower in the multi-party setting, while the child robot condition improved user experience, highlighting the complex influence of group dynamics and robot portrayal on human-robot interactions in education.

Place, publisher, year, edition, pages
New York, NY United States: Association for Computing Machinery (ACM), 2024
Keywords
Human-Robot Interaction, Conformity, Influential Agent, Multiparty Interaction, Child-robot Interaction, Education, Trust
National Category
Computer Sciences Human Computer Interaction Languages and Literature Sociology (Excluding Social Work, Social Anthropology, Demography and Criminology)
Research subject
Computer Science; Education and Communication in the Technological Sciences
Identifiers
urn:nbn:se:kth:diva-358521 (URN)10.1145/3652988.3673954 (DOI)001441957400004 ()2-s2.0-85215536524 (Scopus ID)
Conference
IVA '24: ACM International Conference on Intelligent Virtual Agents, GLASGOW United Kingdom, September 16-19, 2024
Projects
tmh_rall_convRALLe-laddaEarly Language Development in the Digital Age (e-LADDA)
Funder
EU, Horizon 2020, 857897
Note

Part of ISBN 9798400706257

QC 20250120

Available from: 2025-01-18 Created: 2025-01-18 Last updated: 2025-04-30Bibliographically approved
Cumbal, R. & Engwall, O. (2024). Speaking Transparently: Social Robots in Educational Settings. In: Companion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction (HRI '24 Companion), March 11--14, 2024, Boulder, CO, USA: . Paper presented at International Conference on Human-Robot Interaction.
Open this publication in new window or tab >>Speaking Transparently: Social Robots in Educational Settings
2024 (English)In: Companion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction (HRI '24 Companion), March 11--14, 2024, Boulder, CO, USA, 2024Conference paper, Published paper (Refereed)
Abstract [en]

The recent surge in popularity of Large Language Models, known for their inherent opacity, has increased the interest in fostering transparency in technology designed for human interaction. This concern is equally prevalent in the development of Social Robots, particularly when these are designed to engage in critical areas of our society, such as education or healthcare. In this paper we propose an experiment to investigate how users can be made aware of the automated decision processes when interacting in a discussion with a social robot. Our main objective is to assess the effectiveness of verbal expressions in fostering transparency within groups of individuals as they engage with a robot. We describe the proposed interactive settings, system design, and our approach to enhance the transparency in a robot's decision-making process for multi-party interactions.

Keywords
Dialogue system, transparency, multi-party, conversation
National Category
Robotics and automation
Research subject
Computer Science
Identifiers
urn:nbn:se:kth:diva-343862 (URN)10.1145/3610978.3640717 (DOI)001255070800079 ()2-s2.0-85188090310 (Scopus ID)
Conference
International Conference on Human-Robot Interaction
Note

QC 20240328

Available from: 2024-02-26 Created: 2024-02-26 Last updated: 2025-02-09Bibliographically approved
Engwall, O., Bandera Rubio, J. P., Bensch, S., Haring, K. S., Kanda, T., Núñez, P., . . . Sgorbissa, A. (2023). Editorial: Socially, culturally and contextually aware robots. Frontiers in Robotics and AI, 10, Article ID 1232215.
Open this publication in new window or tab >>Editorial: Socially, culturally and contextually aware robots
Show others...
2023 (English)In: Frontiers in Robotics and AI, E-ISSN 2296-9144, Vol. 10, article id 1232215Article in journal, Editorial material (Other academic) Published
Place, publisher, year, edition, pages
Frontiers Media SA, 2023
Keywords
context awareness, cultural awareness, original research, reviews-articles, social robots
National Category
Computer graphics and computer vision
Identifiers
urn:nbn:se:kth:diva-340355 (URN)10.3389/frobt.2023.1232215 (DOI)001105825000001 ()2-s2.0-85177180605 (Scopus ID)
Note

QC 20231204

Available from: 2023-12-04 Created: 2023-12-04 Last updated: 2025-02-07Bibliographically approved
Engwall, O., Cumbal, R. & Majlesi, A. R. (2023). Socio-cultural perception of robot backchannels. Frontiers in Robotics and AI, 10
Open this publication in new window or tab >>Socio-cultural perception of robot backchannels
2023 (English)In: Frontiers in Robotics and AI, E-ISSN 2296-9144, Vol. 10Article in journal (Refereed) Published
Abstract [en]

Introduction: Backchannels, i.e., short interjections by an interlocutor to indicate attention, understanding or agreement regarding utterances by another conversation participant, are fundamental in human-human interaction. Lack of backchannels or if they have unexpected timing or formulation may influence the conversation negatively, as misinterpretations regarding attention, understanding or agreement may occur. However, several studies over the years have shown that there may be cultural differences in how backchannels are provided and perceived and that these differences may affect intercultural conversations. Culturally aware robots must hence be endowed with the capability to detect and adapt to the way these conversational markers are used across different cultures. Traditionally, culture has been defined in terms of nationality, but this is more and more considered to be a stereotypic simplification. We therefore investigate several socio-cultural factors, such as the participants’ gender, age, first language, extroversion and familiarity with robots, that may be relevant for the perception of backchannels.

Methods: We first cover existing research on cultural influence on backchannel formulation and perception in human-human interaction and on backchannel implementation in Human-Robot Interaction. We then present an experiment on second language spoken practice, in which we investigate how backchannels from the social robot Furhat influence interaction (investigated through speaking time ratios and ethnomethodology and multimodal conversation analysis) and impression of the robot (measured by post-session ratings). The experiment, made in a triad word game setting, is focused on if activity-adaptive robot backchannels may redistribute the participants’ speaking time ratio, and/or if the participants’ assessment of the robot is influenced by the backchannel strategy. The goal is to explore how robot backchannels should be adapted to different language learners to encourage their participation while being perceived as socio-culturally appropriate.

Results: We find that a strategy that displays more backchannels towards a less active speaker may substantially decrease the difference in speaking time between the two speakers, that different socio-cultural groups respond differently to the robot’s backchannel strategy and that they also perceive the robot differently after the session.

Discussion: We conclude that the robot may need different backchanneling strategies towards speakers from different socio-cultural groups in order to encourage them to speak and have a positive perception of the robot.

 

Place, publisher, year, edition, pages
Frontiers Media S.A., 2023
Keywords
backchannels, multiparty HRI, robot-assisted language learning, spoken practice, cultural effects
National Category
Human Computer Interaction
Research subject
Computer Science
Identifiers
urn:nbn:se:kth:diva-323334 (URN)10.3389/frobt.2023.988042 (DOI)000935012900001 ()36777379 (PubMedID)2-s2.0-85147686864 (Scopus ID)
Funder
Marcus and Amalia Wallenberg Foundation, 2020.0052
Note

QC 20230130

Available from: 2023-01-26 Created: 2023-01-26 Last updated: 2024-02-26Bibliographically approved
Cumbal, R., Axelsson, A., Mehta, S. & Engwall, O. (2023). Stereotypical nationality representations in HRI: perspectives from international young adults. Frontiers in Robotics and AI, 10, Article ID 1264614.
Open this publication in new window or tab >>Stereotypical nationality representations in HRI: perspectives from international young adults
2023 (English)In: Frontiers in Robotics and AI, E-ISSN 2296-9144, Vol. 10, article id 1264614Article in journal (Refereed) Published
Abstract [en]

People often form immediate expectations about other people, or groups of people, based on visual appearance and characteristics of their voice and speech. These stereotypes, often inaccurate or overgeneralized, may translate to robots that carry human-like qualities. This study aims to explore if nationality-based preconceptions regarding appearance and accents can be found in people's perception of a virtual and a physical social robot. In an online survey with 80 subjects evaluating different first-language-influenced accents of English and nationality-influenced human-like faces for a virtual robot, we find that accents, in particular, lead to preconceptions on perceived competence and likeability that correspond to previous findings in social science research. In a physical interaction study with 74 participants, we then studied if the perception of competence and likeability is similar after interacting with a robot portraying one of four different nationality representations from the online survey. We find that preconceptions on national stereotypes that appeared in the online survey vanish or are overshadowed by factors related to general interaction quality. We do, however, find some effects of the robot's stereotypical alignment with the subject group, with Swedish subjects (the majority group in this study) rating the Swedish-accented robot as less competent than the international group, but, on the other hand, recalling more facts from the Swedish robot's presentation than the international group does. In an extension in which the physical robot was replaced by a virtual robot interacting in the same scenario online, we further found the same results that preconceptions are of less importance after actual interactions, hence demonstrating that the differences in the ratings of the robot between the online survey and the interaction is not due to the interaction medium. We hence conclude that attitudes towards stereotypical national representations in HRI have a weak effect, at least for the user group included in this study (primarily educated young students in an international setting).

Place, publisher, year, edition, pages
Frontiers Media SA, 2023
Keywords
accent, appearance, social robot, nationality, stereotype, impression, competence, likeability
National Category
Human Computer Interaction
Identifiers
urn:nbn:se:kth:diva-341526 (URN)10.3389/frobt.2023.1264614 (DOI)001115613500001 ()38077460 (PubMedID)2-s2.0-85178920101 (Scopus ID)
Note

QC 20231222

Available from: 2023-12-22 Created: 2023-12-22 Last updated: 2024-02-26Bibliographically approved
Engwall, O., Cumbal, R., Lopes, J., Ljung, M. & Månsson, L. (2022). Identification of Low-engaged Learners in Robot-led Second Language Conversations with Adults. ACM Transactions on Human-Robot Interaction, 11(2), Article ID 18.
Open this publication in new window or tab >>Identification of Low-engaged Learners in Robot-led Second Language Conversations with Adults
Show others...
2022 (English)In: ACM Transactions on Human-Robot Interaction, E-ISSN 2573-9522, Vol. 11, no 2, article id 18Article in journal (Refereed) Published
Abstract [en]

The main aim of this study is to investigate if verbal, vocal, and facial information can be used to identify low-engaged second language learners in robot-led conversation practice. The experiments were performed on voice recordings and video data from 50 conversations, in which a robotic head talks with pairs of adult language learners using four different interaction strategies with varying robot-learner focus and initiative. It was found that these robot interaction strategies influenced learner activity and engagement. The verbal analysis indicated that learners with low activity rated the robot significantly lower on two out of four scales related to social competence. The acoustic vocal and video-based facial analysis, based on manual annotations or machine learning classification, both showed that learners with low engagement rated the robot's social competencies consistently, and in several cases significantly, lower, and in addition rated the learning effectiveness lower. The agreement between manual and automatic identification of low-engaged learners based on voice recordings or face videos was further found to be adequate for future use. These experiments constitute a first step towards enabling adaption to learners' activity and engagement through within- and between-strategy changes of the robot's interaction with learners.

Place, publisher, year, edition, pages
Association for Computing Machinery (ACM), 2022
Keywords
Robot-assisted language learning, user engagement, speech emotion recognition, facial emotion expressions
National Category
Human Computer Interaction
Identifiers
urn:nbn:se:kth:diva-311036 (URN)10.1145/3503799 (DOI)000774332200008 ()2-s2.0-85127492914 (Scopus ID)
Note

Not duplicate with DiVA 1612730

QC 20220420

Available from: 2022-04-20 Created: 2022-04-20 Last updated: 2023-01-09Bibliographically approved
Engwall, O. & David Lopes, J. (2022). Interaction and collaboration in robot-assisted language learning for adults. Computer Assisted Language Learning, 35(5-6), 1273-1309
Open this publication in new window or tab >>Interaction and collaboration in robot-assisted language learning for adults
2022 (English)In: Computer Assisted Language Learning, ISSN 0958-8221, E-ISSN 1744-3210, Vol. 35, no 5-6, p. 1273-1309Article in journal (Refereed) Published
Abstract [en]

This article analyses how robot–learner interaction in robot-assisted language learning (RALL) is influenced by the interaction behaviour of the robot. Since the robot behaviour is to a large extent determined by the combination of teaching strategy, robot role and robot type, previous studies in RALL are first summarised with respect to which combinations that have been chosen, the rationale behind the choice and the effects on interaction and learning. The goal of the summary is to determine a suitable pedagogical set-up for RALL with adult learners, since previous RALL studies have almost exclusively been performed with children and youths. A user study in which 33 adult second language learners practice Swedish in three-party conversations with an anthropomorphic robot head is then presented. It is demonstrated how different robot interaction behaviours influence interaction between the robot and the learners and between the two learners. Through an analysis of learner interaction, collaboration and learner ratings for the different robot behaviours, it is observed that the learners were most positive towards the robot behaviour that focused on interviewing one learner at the time (highest average ratings), but that they were the most active in sessions when the robot encouraged learner–learner interaction. Moreover, the preferences and activity differed between learner pairs, depending on, e.g., their proficiency level and how well they knew the peer. It is therefore concluded that the robot behaviour needs to adapt to such factors. In addition, collaboration with the peer played an important part in conversation practice sessions to deal with linguistic difficulties or communication problems with the robot.

Place, publisher, year, edition, pages
Routledge, 2022
Keywords
Collaborative language learning; communicative language teaching; educational robots; human–robot interaction; spoken practice
National Category
Robotics and automation
Research subject
Computer Science
Identifiers
urn:nbn:se:kth:diva-278975 (URN)10.1080/09588221.2020.1799821 (DOI)000557599700001 ()2-s2.0-85089256652 (Scopus ID)
Projects
Collaborative Robot-assisted Language Learning
Funder
Swedish Research Council, 2016-03698
Note

QC 20200818

Available from: 2020-08-08 Created: 2020-08-08 Last updated: 2025-02-09Bibliographically approved
Engwall, O., Águas Lopes, J. D. & Cumbal, R. (2022). Is a Wizard-of-Oz Required for Robot-Led Conversation Practice in a Second Language?. International Journal of Social Robotics, 14(4), 1067-1085
Open this publication in new window or tab >>Is a Wizard-of-Oz Required for Robot-Led Conversation Practice in a Second Language?
2022 (English)In: International Journal of Social Robotics, ISSN 1875-4791, E-ISSN 1875-4805, Vol. 14, no 4, p. 1067-1085Article in journal (Refereed) Published
Abstract [en]

The large majority of previous work on human-robot conversations in a second language has been performed with a human wizard-of-Oz. The reasons are that automatic speech recognition of non-native conversational speech is considered to be unreliable and that the dialogue management task of selecting robot utterances that are adequate at a given turn is complex in social conversations. This study therefore investigates if robot-led conversation practice in a second language with pairs of adult learners could potentially be managed by an autonomous robot. We first investigate how correct and understandable transcriptions of second language learner utterances are when made by a state-of-the-art speech recogniser. We find both a relatively high word error rate (41%) and that a substantial share (42%) of the utterances are judged to be incomprehensible or only partially understandable by a human reader. We then evaluate how adequate the robot utterance selection is, when performed manually based on the speech recognition transcriptions or autonomously using (a) predefined sequences of robot utterances, (b) a general state-of-the-art language model that selects utterances based on learner input or the preceding robot utterance, or (c) a custom-made statistical method that is trained on observations of the wizard’s choices in previous conversations. It is shown that adequate or at least acceptable robot utterances are selected by the human wizard in most cases (96%), even though the ASR transcriptions have a high word error rate. Further, the custom-made statistical method performs as well as manual selection of robot utterances based on ASR transcriptions. It was also found that the interaction strategy that the robot employed, which differed regarding how much the robot maintained the initiative in the conversation and if the focus of the conversation was on the robot or the learners, had marginal effects on the word error rate and understandability of the transcriptions but larger effects on the adequateness of the utterance selection. Autonomous robot-led conversations may hence work better with some robot interaction strategies.

Place, publisher, year, edition, pages
Springer Nature, 2022
Keywords
Robot-assisted language learning, Conversational practice, Non-native speech recognition, Dialogue management for spoken human-robot interaction
National Category
Natural Language Processing
Research subject
Computer Science
Identifiers
urn:nbn:se:kth:diva-306942 (URN)10.1007/s12369-021-00849-8 (DOI)000739285100001 ()2-s2.0-85122404446 (Scopus ID)
Funder
Swedish Research Council, 2016-03698Marcus and Amalia Wallenberg Foundation, MAW 2020.0052
Note

QC 20250612

Available from: 2022-01-05 Created: 2022-01-05 Last updated: 2025-06-12Bibliographically approved
Organisations
Identifiers
ORCID iD: ORCID iD iconorcid.org/0000-0003-4532-014X

Search in DiVA

Show all publications