Social Educational Robotics and Learning Analytics (LA) are prominent fields in technology-enhanced learning, but their combined potential remains underexplored, despite methodological similarities. Increasingly, signs of joint interests have emerged, with a surge in publications mentioning both social robots and learning analytics in the last five years. We therefore conducted a scoping review to explore if a new research field is emerging. We identified 29 empirical studies that combine social robots and LA, but also found that few studies explicitly state that social educational robots and LA are used in combination. Several studies used social educational robots that adapted to the learners or the learning environment based on interaction data. This signifies that they are in fact employing the feedback cycle that is at the core of LA methodology, but as most of these studies update the learner model using post-session data (e.g., learner improvement or feedback), they are long-term studies with repeated interventions that are applying LA methodology inadvertently. There are also benefits for LA research to use social educational robots, since LA increasingly uses an array of equipment to collect multimodal data, and all studies in this review employ at least two input modalities (mu = 4.4). Social robots provide the possibility to collect this data non-intrusively with the robot itself, in addition to creating a pedagogically boosted interaction compared to traditional LA interventions (e.g., learning management systems). By raising researchers' awareness of how close the fields of social educational robotics and LA are, substantial synergy effects could therefore be gained.

In this paper, we begin our process of incorporating an AI bot in an online chat tutoring setting as a support for the tutor. We explore how an AI bot could give suggestions for tutor messages, although the human tutor will control how to communicate with the student. Tutoring, an important dimension of networked learning, has long been seen as a beneficial approach to students’ learning. An AI bot has the potential to aid tutors in the tutoring process and contribute to the scalability. The present pilot study was conducted in the tutoring setting of the Math Coach program. In the program, teacher students aid students from upper primary school to upper secondary school in mathematics through an online text-based chat system. Llama2 was used as a large language model (LLM), fine-tuned for Swedish comprehension utilizing the Math Coach system's chat logs. Four coaches, teacher students at a technical Swedish university and active in the Math Coach program, were invited to interact with the AI bot and participate in a group discussion. The coaches interacted individually with the AI bot while the chat conversation was displayed on a monitor so all participants could discuss the interaction while it took place. A semi-structured interview approach was taken and the participants were also encouraged to 'think aloud' about their experience. In the discussions, the coaches expressed surprise by the AI's social aspect. They perceived the AI bot as friendly with a positive attitude and were especially surprised by its ability to correctly place appropriate emojis. The coaches agreed that the AI was able to ask both appropriate and helpful questions and share some good guidance for how to proceed in the problem-solving process. However, they felt that the AI bot was not able to offer sufficient mathematical guidance, oftentimes the AI bot was confidently wrong. It also wrote too long messages, which humans would typically separate into several chat messages, and did not wait for a response but instead moved too quickly towards the solution. Moving forward we plan to address the effects of improved prompts on the AI bot and continue finetuning the LLM. We will continue to conduct pilot studies and eventually conduct more large-scale empirical studies.

In this study, we explored how conformity and trust vary in adolescent students’ interactions with a social robot. Specifically, we compared how this was influenced by whether the participants had individual or multi-party interaction with robot and whether the robot was portrayed as an adult or a child through appearance and voice. Our experiment involved 75 Swedish middle school students participating in a card sorting game with the Furhat robot, where the objective was to discuss and reach an agreement on the card sequence. The data analysis focused firstly on the participants’ willingness to rearrange cards following the robot’s suggestions and secondly their post-session subjective trust in the robot’s advice. Results indicated that individuals interacting with the robot individually were more likely to conform to its suggestions than those interacting with it together with a peer. Individuals interacting alone with the robot also showed higher post-session trust levels than those in multi-party settings, indicating group size impacts robot trustworthiness perceptions. However, the robot’s perceived age did not affect the level of conformity. Exploratory analyses also showed that mutual understanding was lower in the multi-party setting, while the child robot condition improved user experience, highlighting the complex influence of group dynamics and robot portrayal on human-robot interactions in education.

The recent surge in popularity of Large Language Models, known for their inherent opacity, has increased the interest in fostering transparency in technology designed for human interaction. This concern is equally prevalent in the development of Social Robots, particularly when these are designed to engage in critical areas of our society, such as education or healthcare. In this paper we propose an experiment to investigate how users can be made aware of the automated decision processes when interacting in a discussion with a social robot. Our main objective is to assess the effectiveness of verbal expressions in fostering transparency within groups of individuals as they engage with a robot. We describe the proposed interactive settings, system design, and our approach to enhance the transparency in a robot's decision-making process for multi-party interactions.

Introduction: Backchannels, i.e., short interjections by an interlocutor to indicate attention, understanding or agreement regarding utterances by another conversation participant, are fundamental in human-human interaction. Lack of backchannels or if they have unexpected timing or formulation may influence the conversation negatively, as misinterpretations regarding attention, understanding or agreement may occur. However, several studies over the years have shown that there may be cultural differences in how backchannels are provided and perceived and that these differences may affect intercultural conversations. Culturally aware robots must hence be endowed with the capability to detect and adapt to the way these conversational markers are used across different cultures. Traditionally, culture has been defined in terms of nationality, but this is more and more considered to be a stereotypic simplification. We therefore investigate several socio-cultural factors, such as the participants’ gender, age, first language, extroversion and familiarity with robots, that may be relevant for the perception of backchannels.

Methods: We first cover existing research on cultural influence on backchannel formulation and perception in human-human interaction and on backchannel implementation in Human-Robot Interaction. We then present an experiment on second language spoken practice, in which we investigate how backchannels from the social robot Furhat influence interaction (investigated through speaking time ratios and ethnomethodology and multimodal conversation analysis) and impression of the robot (measured by post-session ratings). The experiment, made in a triad word game setting, is focused on if activity-adaptive robot backchannels may redistribute the participants’ speaking time ratio, and/or if the participants’ assessment of the robot is influenced by the backchannel strategy. The goal is to explore how robot backchannels should be adapted to different language learners to encourage their participation while being perceived as socio-culturally appropriate.

Results: We find that a strategy that displays more backchannels towards a less active speaker may substantially decrease the difference in speaking time between the two speakers, that different socio-cultural groups respond differently to the robot’s backchannel strategy and that they also perceive the robot differently after the session.

Discussion: We conclude that the robot may need different backchanneling strategies towards speakers from different socio-cultural groups in order to encourage them to speak and have a positive perception of the robot.

People often form immediate expectations about other people, or groups of people, based on visual appearance and characteristics of their voice and speech. These stereotypes, often inaccurate or overgeneralized, may translate to robots that carry human-like qualities. This study aims to explore if nationality-based preconceptions regarding appearance and accents can be found in people's perception of a virtual and a physical social robot. In an online survey with 80 subjects evaluating different first-language-influenced accents of English and nationality-influenced human-like faces for a virtual robot, we find that accents, in particular, lead to preconceptions on perceived competence and likeability that correspond to previous findings in social science research. In a physical interaction study with 74 participants, we then studied if the perception of competence and likeability is similar after interacting with a robot portraying one of four different nationality representations from the online survey. We find that preconceptions on national stereotypes that appeared in the online survey vanish or are overshadowed by factors related to general interaction quality. We do, however, find some effects of the robot's stereotypical alignment with the subject group, with Swedish subjects (the majority group in this study) rating the Swedish-accented robot as less competent than the international group, but, on the other hand, recalling more facts from the Swedish robot's presentation than the international group does. In an extension in which the physical robot was replaced by a virtual robot interacting in the same scenario online, we further found the same results that preconceptions are of less importance after actual interactions, hence demonstrating that the differences in the ratings of the robot between the online survey and the interaction is not due to the interaction medium. We hence conclude that attitudes towards stereotypical national representations in HRI have a weak effect, at least for the user group included in this study (primarily educated young students in an international setting).

The main aim of this study is to investigate if verbal, vocal, and facial information can be used to identify low-engaged second language learners in robot-led conversation practice. The experiments were performed on voice recordings and video data from 50 conversations, in which a robotic head talks with pairs of adult language learners using four different interaction strategies with varying robot-learner focus and initiative. It was found that these robot interaction strategies influenced learner activity and engagement. The verbal analysis indicated that learners with low activity rated the robot significantly lower on two out of four scales related to social competence. The acoustic vocal and video-based facial analysis, based on manual annotations or machine learning classification, both showed that learners with low engagement rated the robot's social competencies consistently, and in several cases significantly, lower, and in addition rated the learning effectiveness lower. The agreement between manual and automatic identification of low-engaged learners based on voice recordings or face videos was further found to be adequate for future use. These experiments constitute a first step towards enabling adaption to learners' activity and engagement through within- and between-strategy changes of the robot's interaction with learners.

This article analyses how robot–learner interaction in robot-assisted language learning (RALL) is influenced by the interaction behaviour of the robot. Since the robot behaviour is to a large extent determined by the combination of teaching strategy, robot role and robot type, previous studies in RALL are first summarised with respect to which combinations that have been chosen, the rationale behind the choice and the effects on interaction and learning. The goal of the summary is to determine a suitable pedagogical set-up for RALL with adult learners, since previous RALL studies have almost exclusively been performed with children and youths. A user study in which 33 adult second language learners practice Swedish in three-party conversations with an anthropomorphic robot head is then presented. It is demonstrated how different robot interaction behaviours influence interaction between the robot and the learners and between the two learners. Through an analysis of learner interaction, collaboration and learner ratings for the different robot behaviours, it is observed that the learners were most positive towards the robot behaviour that focused on interviewing one learner at the time (highest average ratings), but that they were the most active in sessions when the robot encouraged learner–learner interaction. Moreover, the preferences and activity differed between learner pairs, depending on, e.g., their proficiency level and how well they knew the peer. It is therefore concluded that the robot behaviour needs to adapt to such factors. In addition, collaboration with the peer played an important part in conversation practice sessions to deal with linguistic difficulties or communication problems with the robot.

The large majority of previous work on human-robot conversations in a second language has been performed with a human wizard-of-Oz. The reasons are that automatic speech recognition of non-native conversational speech is considered to be unreliable and that the dialogue management task of selecting robot utterances that are adequate at a given turn is complex in social conversations. This study therefore investigates if robot-led conversation practice in a second language with pairs of adult learners could potentially be managed by an autonomous robot. We first investigate how correct and understandable transcriptions of second language learner utterances are when made by a state-of-the-art speech recogniser. We find both a relatively high word error rate (41%) and that a substantial share (42%) of the utterances are judged to be incomprehensible or only partially understandable by a human reader. We then evaluate how adequate the robot utterance selection is, when performed manually based on the speech recognition transcriptions or autonomously using (a) predefined sequences of robot utterances, (b) a general state-of-the-art language model that selects utterances based on learner input or the preceding robot utterance, or (c) a custom-made statistical method that is trained on observations of the wizard’s choices in previous conversations. It is shown that adequate or at least acceptable robot utterances are selected by the human wizard in most cases (96%), even though the ASR transcriptions have a high word error rate. Further, the custom-made statistical method performs as well as manual selection of robot utterances based on ASR transcriptions. It was also found that the interaction strategy that the robot employed, which differed regarding how much the robot maintained the initiative in the conversation and if the focus of the conversation was on the robot or the learners, had marginal effects on the word error rate and understandability of the transcriptions but larger effects on the adequateness of the utterance selection. Autonomous robot-led conversations may hence work better with some robot interaction strategies.