Robots are increasingly becoming an integral part of our everyday life. Expectations on robots could be influenced by how robots are represented in science fiction films. We hypothesize that sonic interaction design for real-world robots may find inspiration from sound design of fictional robots. In this paper, we present an exploratory study focusing on sonic characteristics of robot sounds in films. We believe that findings from the current study could be of relevance for future robotic applications involving the communication of internal states through sounds, as well for sonification of expressive robot movements. Excerpts from five films were annotated and analysed using Long Time Average Spectrum (LTAS). As an overall observation, we found that robot sonic presence is highly related to the physical appearance of robots. Preliminary results show that most of the robots analysed in this study have “metallic” voice qualities, matching the material of their physical form. Characteristics of robot voices show significant differences compared to voices of human characters; fundamental frequency of robotic voices is either shifted to higher or lower values, and the voices span over a broader frequency band.

Despite the fact that sounds produced by robots can affect the interaction with humans, sound design is often an overlooked aspect in Human-Robot Interaction (HRI). This paper explores how different sets of sounds designed for expressive robot gestures of a humanoid Pepper robot can influence the perception of emotional intentions. In the pilot study presented in this paper, it has been asked to rate different stimuli in terms of perceived affective states. The stimuli were audio, audio-video and video only and contained either Pepper’s original servomotors noises, sawtooth, or more complex designed sounds. The preliminary results show a preference for the use of more complex sounds, thus confirming the necessity of further exploration in sonic HRI.

This paper presents three studies where we probe aesthetics strategies of sound produced by movement sonification of a Pepper robot by mapping its movements to sound models.

We developed two sets of sound models. The first set was made by two sound models, a sawtooth-based one and another based on feedback chains, for investigating how the perception of synthesized robot sounds would depend on their design complexity. We implemented the second set of sound models for probing the “materiality” of sound made by a robot in motion. This set consisted of a sound synthesis based on an engine highlighting the robot’s internal mechanisms, a metallic sound synthesis highlighting the robot’s typical appearance, and a whoosh sound synthesis highlighting the movement.

We conducted three studies. The first study explores how the first set of sound models can influence the perception of expressive gestures of a Pepper robot through an online survey. In the second study, we carried out an experiment in a museum installation with a Pepper robot presented in two scenarios: (1) while welcoming patrons into a restaurant and (2) while providing information to visitors in a shopping center. Finally, in the third study, we conducted an online survey with stimuli similar to those used in the second study.

Our findings suggest that participants preferred more complex sound models for the sonification of robot movements. Concerning the materiality, participants liked better subtle sounds that blend well with the ambient sound (i.e., less distracting) and soundscapes in which sound sources can be identified. Also, sound preferences varied depending on the context in which participants experienced the robot-generated sounds (e.g., as a live museum installation vs. an online display).

This paper presents the design and development of PepperOSC, an interface that connects Pepper and NAO robots with soundproduction tools to enable the development of interactive sonification in human-robot interaction (HRI). The interface usesOpen Sound Control (OSC) messages to stream kinematic data from robots to various sound design and music productiontools. The goals of PepperOSC are twofold: (i) to provide a tool for HRI researchers in developing multimodal user interfacesthrough sonification, and (ii) to lower the barrier for sound designers to contribute to HRI. To demonstrate the potential useof PepperOSC, this paper also presents two applications we have conducted: (i) a course project by two master’s studentswho created a robot sound model in Pure Data, and (ii) a museum installation of Pepper robot, employing sound modelsdeveloped by a sound designer and a composer/researcher in music technology usingMaxMSP and SuperCollider respectively.Furthermore, we discuss the potential use cases of PepperOSC in social robotics and artistic contexts. These applicationsdemonstrate the versatility of PepperOSC and its ability to explore diverse aesthetic strategies for robot movement sonification,offering a promising approach to enhance the effectiveness and appeal of human-robot interactions.

This paper investigates the sound design of robots in films and their potential influence on the field of social robotics. Cinematic robot portrayals have inspired researchers and practitioners in Human-Robot Interaction (HRI). While the non-verbal sounds of iconic film robots like R2-D2 and Wall-E have been explored, this study takes a more comprehensive approach. We explore a broader selection of 15 films featuring humanoid robots across decades through a semiotic analysis of their non-verbal communication sounds, including those related to movements and internal mechanisms. Our analysis, guided by Bateman and Schmidt’s multimodal film analysis framework following Saussure’s organization of signs through paradigmatic and syntagmatic relations, interprets the paradigmatic axis as the examination of the sound and the syntagmatic axis as the examination of the events surrounding the sound. The findings uncover two primary film robot sound materials: mechanical and synthetic. Additionally, contextual analysis reveals three narrative themes and several sub-themes related to the physical attributes of robots, their internal workings, and their interactions with other characters. The discussion section explores the implications of these findings for social robotics, including the importance of sound materials, the role of movement sounds in communication and emotional expression, and the significance of narrative and context in human-robot interaction. The paper also acknowledges the challenges in translating film sound design into practical applications in social robotics. This study provides valuable insights for HRI researchers, practitioners, and sound designers seeking to enhance non-verbal auditory expressions in social robots.