To enable better human-machine and human-robot interaction in healthcare applications, there is a need to improve haptic devices, assistive and companion robots to touch humans in a way that can be perceived as comfortable, safe and trusting. Towards this vision, we developed a flexible, wearable matrix force-sensing garment capable of recording dynamic force behaviour of caring touch from four healthcare experts in their natural therapy environment, and enable the touch recipient to provide feedback on the perceived quality of the same touches. Instead of assuming particular touch gestures are caring, we explored the application of the sensing and analysing system to characterise a caring quality of touch across different gestures. We were able to characterise distinctive features for touch aimed for sensory aspects and for moving the body. Our findings demonstrate that caring touch is not limited to specific gestures or a single parameter but is rather characterised by how the touch is performed across multiple features, with deliberateness - the careful and smoothness in the increasing and decreasing of force, emerging as a shared characteristic of caring touch regardless of the specific gesture employed. The study also demonstrates the feasibility of using raw tactile sensor data to assess subjective touch quality.

We present a qualitative study with five healthcare experts specialised in different types of touch practice to gain insight in how caring touch can be enacted. Through our analysis we focus on how to transfer this learning into design considerations towards enacting caring touch from technologies. Despite the rapidly growing expectation for and design interest in touch from technologies intending to enhance care and well-being, the knowledge on how to design caring touch is still fragmented. How caring touch is enacted in inter-personal touch is under-explored and such expertise from healthcare practitioners has not been engaged from the perspective of HCI design research. We propose designers to consider caring as an experiential quality instead of a division between instrumental types of touch and caring types. We recommend when designing for a caring quality in technology-initiated touch that designers create a progression of touch with dynamic sensitivity and adapt the materiality of actuating devices to the plural dimensions of the body’s textures.

This dissertation investigates the role of sound design in social robotics, drawing inspiration from robot depictions in science-fiction films. It addresses the limitations of robots’ movements and expressive behavior by integrating principles from film sound design, seeking to improve human-robot interaction through expressive gestures and non-verbal sounds.

The compiled works are structured into two parts. The first part focuses on perceptual studies, exploring how people perceive non-verbal sounds displayed by a Pepper robot related to its movement. These studies highlighted preferences for more refined sound models, subtle sounds that blend with ambient sounds, and sound characteristics matching the robot’s visual attributes. This part also resulted in a programming interface connecting the Pepper robot with sound production tools.

The second part focuses on a structured analysis of robot sounds in films, revealing three narrative themes related to robot sounds in films with implications for social robotics. The first theme involves sounds associated with the physical attributes of robots, encompassing sub-themes of sound linked to robot size, exposed mechanisms, build quality, and anthropomorphic traits. The second theme delves into sounds accentuating robots’ internal workings, with sub-themes related to learning and decision-making processes. Lastly, the third theme revolves around sounds utilized in robots’ interactions with other characters within the film scenes.

Based on these works, the dissertation discusses sound design recommendations for social robotics inspired by practices in film sound design. These recommendations encompass selecting the appropriate sound materials and sonic characteristics such as pitch and timbre, employing movement sound for effective communication and emotional expression, and integrating narrative and context into the interaction.

Denna avhandling undersöker ljuddesignens roll i social robotik, med inspiration från robotskildringar i science fiction filmer. Avhandlingen diskuterar begränsningar i robotars uttrycksfulla beteenden genom att integrera principer från filmljuddesign. Arbetet syftar till att främja interaktionen mellan människa och robot genom att förse robotar med uttrycksfulla gester och icke-verbala ljud.

Denna sammanläggningsavhandling inkluderar ett antal artiklar som är strukturerade i två separata delar. Den första delen fokuserar på perceptuella studier och undersöker hur människor uppfattar de icke-verbala ljud som roboten Pepper producerar i samband med sina rörelser. Dessa studier belyste preferenser för mer förfinade ljudmodeller, subtila ljud som blandas med omgivande ljud, och ljudegenskaper som matchar robotens visuella attribut. Denna del resulterade också i ett programmeringsgränssnitt som sammankopplar Pepper-roboten och ljudproduktionsverktyg.

Den andra delen fokuserar på en strukturerad analys av robotljud i filmer och avslöjar tre narrativa teman relaterade till robotljud i filmer med implikationer för social robotik. Det första temat handlar om ljud som förknippas med robotarnas fysiska attribut och omfattar underteman av ljud som är kopplade till robotstorlek, exponerade mekanismer, byggkvalitet, och antropomorfa drag. Det andra temat fördjupar sig i ljud som betonar robotarnas interna arbete, med underteman relaterade till inlärnings- och beslutsprocesser. Slutligen kretsar det tredje temat kring ljud som används i robotarnas interaktion med andra karaktärer i filmscenerna.

Baserat på ovan beskrivna arbeten diskuterar denna avhandling rekommendationer för ljuddesign inom social robotik inspirerade av praxis inom filmljuddesign. Dessa rekommendationer omfattar att välja lämpliga ljudmaterial och ljudegenskaper såsom tonhöjd och klangfärg, att använda rörelseljud för effektiv kommunikation och känslomässiga uttryck, samt att integrera narrativ och sammanhang i interaktionen.

The sound design of different fantastical aspects can tell the audience much about characters and things. Robots are one of the common fantastical characters that need to be sonified to indicate different aspects of their character. Often, one or more of these traits are an indication of gender and behavior. We investigated these traits in a survey where we asked both quantitative and qualitative questions about the participants' perceptions. We found that participants indicated a bias towards certain robots depending on perceived femininity and masculinity.

This study explores if, and how, the choices made regarding a robot's speaking voice and characteristic body sounds influence viewers' perceptions of its intent i.e., whether the robot's intention is positive or negative. The analysis focuses on robot representations and sounds in three films: "Robots"(2005) [1], "NextGen"(2018) [2], and "Love, Death, and Robots - Three Robots"(2019) [3]. In eight qualitative interviews, five parameters (tonality, intonation, volume, pitch, and speed) were used to understand robot sounds and the participant's perception of a robot's attitude and intentions. The study culminates in a set of recommendations and considerations for human-robot interaction designers to consider while sound coding for body, physiology, and movement.

Politeness is at the core of the common set of behavioral norms that regulate human communication and is therefore of significant interest in the design of Human-Robot Interactions. In this paper, we investigate how the politeness behaviors of a humanoid robot impact human decisions about where to join a group of two robots. We also evaluate the resulting impact on the perception of the robot's politeness. In a study involving 59 participants, the main (Pepper) robot in the group invited participants to join using six politeness behaviors derived from Brown and Levinson's politeness theory. It requests participants to join the group at the furthest side of the group which involves more effort to reach than a closer side that is also available to the participant but would ignore the request of the robot. We evaluated the robot's effectiveness in terms of persuasiveness, politeness, and clarity. We found that more direct and explicit politeness strategies derived from the theory have a higher level of success in persuading participants to join at the furthest side of the group. We also evaluated participants' adherence to social norms i.e. not walking through the center, or \textit{o-space}, of the group when joining it. Our results showed that participants tended to adhere to social norms when joining at the furthest side by not walking through the center of the group of robots, even though they were informed that the robots were fully automated. 

This paper presents three studies where we probe aesthetics strategies of sound produced by movement sonification of a Pepper robot by mapping its movements to sound models.

We developed two sets of sound models. The first set was made by two sound models, a sawtooth-based one and another based on feedback chains, for investigating how the perception of synthesized robot sounds would depend on their design complexity. We implemented the second set of sound models for probing the “materiality” of sound made by a robot in motion. This set consisted of a sound synthesis based on an engine highlighting the robot’s internal mechanisms, a metallic sound synthesis highlighting the robot’s typical appearance, and a whoosh sound synthesis highlighting the movement.

We conducted three studies. The first study explores how the first set of sound models can influence the perception of expressive gestures of a Pepper robot through an online survey. In the second study, we carried out an experiment in a museum installation with a Pepper robot presented in two scenarios: (1) while welcoming patrons into a restaurant and (2) while providing information to visitors in a shopping center. Finally, in the third study, we conducted an online survey with stimuli similar to those used in the second study.

Our findings suggest that participants preferred more complex sound models for the sonification of robot movements. Concerning the materiality, participants liked better subtle sounds that blend well with the ambient sound (i.e., less distracting) and soundscapes in which sound sources can be identified. Also, sound preferences varied depending on the context in which participants experienced the robot-generated sounds (e.g., as a live museum installation vs. an online display).

The aim of the SONAO project is to establish new methods basedon sonification of expressive movements for achieving a robust interaction between users and humanoid robots. We want to achievethis by combining competences of the research team members inthe fields of social robotics, sound and music computing, affective computing, and body motion analysis. We want to engineersound models for implementing effective mappings between stylized body movements and sound parameters that will enable anagent to express high-level body motion qualities through sound.These mappings are paramount for supporting feedback to andunderstanding robot body motion. The project will result in thedevelopment of new theories, guidelines, models, and tools forthe sonic representation of high-level body motion qualities in interactive applications. This work is part of the growing researchfield known as data sonification, in which we combine methodsand knowledge from the fields of interactive sonification, embodied cognition, multisensory perception, non-verbal and gesturalcommunication in robots.

A class of master of science students and a group of preschool children codesigned new digital musical instruments based on workshop interviews involving vocal sketching, a method for imitating and portraying sounds. The aim of the study was to explore how the students and children would approach vocal sketching as one of several design methods. The children described musical instruments to the students using vocal sketching and other modalities (verbal, drawing, gestures). The resulting instruments built by the students were showcased at the Swedish Museum of Performing Arts in Stockholm. Although all the children tried vocal sketching during preparatory tasks, few employed the method during the workshop. However, the instruments seemed to meet the children’s expectations. Consequently, even though the vocal sketching method alone provided few design directives in the given context, we suggest that vocal sketching, under favorable circumstances, can be an engaging component that complements other modalities in codesign involving children.

Despite the fact that sounds produced by robots can affect the interaction with humans, sound design is often an overlooked aspect in Human-Robot Interaction (HRI). This paper explores how different sets of sounds designed for expressive robot gestures of a humanoid Pepper robot can influence the perception of emotional intentions. In the pilot study presented in this paper, it has been asked to rate different stimuli in terms of perceived affective states. The stimuli were audio, audio-video and video only and contained either Pepper’s original servomotors noises, sawtooth, or more complex designed sounds. The preliminary results show a preference for the use of more complex sounds, thus confirming the necessity of further exploration in sonic HRI.