kth.sePublikationer
Ändra sökning
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf
From Motion Pictures to Robotic Features: Adopting film sound design practices to foster sonic expression in social robotics through interactive sonification
KTH, Skolan för elektroteknik och datavetenskap (EECS), Människocentrerad teknologi, Medieteknik och interaktionsdesign, MID. (Sound and Music Computing)ORCID-id: 0000-0003-3572-6429
2024 (Engelska)Doktorsavhandling, sammanläggning (Övrigt vetenskapligt) [Forskning på konstnärlig grund]Alternativ titel
Från filmer till robotfunktioner : Användning av praxis inom filmljuddesign för att främja ljuduttryck i social robotik genom interaktiv sonifiering (Svenska)
Abstract [en]

This dissertation investigates the role of sound design in social robotics, drawing inspiration from robot depictions in science-fiction films. It addresses the limitations of robots’ movements and expressive behavior by integrating principles from film sound design, seeking to improve human-robot interaction through expressive gestures and non-verbal sounds.

The compiled works are structured into two parts. The first part focuses on perceptual studies, exploring how people perceive non-verbal sounds displayed by a Pepper robot related to its movement. These studies highlighted preferences for more refined sound models, subtle sounds that blend with ambient sounds, and sound characteristics matching the robot’s visual attributes. This part also resulted in a programming interface connecting the Pepper robot with sound production tools.

The second part focuses on a structured analysis of robot sounds in films, revealing three narrative themes related to robot sounds in films with implications for social robotics. The first theme involves sounds associated with the physical attributes of robots, encompassing sub-themes of sound linked to robot size, exposed mechanisms, build quality, and anthropomorphic traits. The second theme delves into sounds accentuating robots’ internal workings, with sub-themes related to learning and decision-making processes. Lastly, the third theme revolves around sounds utilized in robots’ interactions with other characters within the film scenes.

Based on these works, the dissertation discusses sound design recommendations for social robotics inspired by practices in film sound design. These recommendations encompass selecting the appropriate sound materials and sonic characteristics such as pitch and timbre, employing movement sound for effective communication and emotional expression, and integrating narrative and context into the interaction.

Abstract [sv]

Denna avhandling undersöker ljuddesignens roll i social robotik, med inspiration från robotskildringar i science fiction filmer. Avhandlingen diskuterar begränsningar i robotars uttrycksfulla beteenden genom att integrera principer från filmljuddesign. Arbetet syftar till att främja interaktionen mellan människa och robot genom att förse robotar med uttrycksfulla gester och icke-verbala ljud.

Denna sammanläggningsavhandling inkluderar ett antal artiklar som är strukturerade i två separata delar. Den första delen fokuserar på perceptuella studier och undersöker hur människor uppfattar de icke-verbala ljud som roboten Pepper producerar i samband med sina rörelser. Dessa studier belyste preferenser för mer förfinade ljudmodeller, subtila ljud som blandas med omgivande ljud, och ljudegenskaper som matchar robotens visuella attribut. Denna del resulterade också i ett programmeringsgränssnitt som sammankopplar Pepper-roboten och ljudproduktionsverktyg.

Den andra delen fokuserar på en strukturerad analys av robotljud i filmer och avslöjar tre narrativa teman relaterade till robotljud i filmer med implikationer för social robotik. Det första temat handlar om ljud som förknippas med robotarnas fysiska attribut och omfattar underteman av ljud som är kopplade till robotstorlek, exponerade mekanismer, byggkvalitet, och antropomorfa drag. Det andra temat fördjupar sig i ljud som betonar robotarnas interna arbete, med underteman relaterade till inlärnings- och beslutsprocesser. Slutligen kretsar det tredje temat kring ljud som används i robotarnas interaktion med andra karaktärer i filmscenerna.

Baserat på ovan beskrivna arbeten diskuterar denna avhandling rekommendationer för ljuddesign inom social robotik inspirerade av praxis inom filmljuddesign. Dessa rekommendationer omfattar att välja lämpliga ljudmaterial och ljudegenskaper såsom tonhöjd och klangfärg, att använda rörelseljud för effektiv kommunikation och känslomässiga uttryck, samt att integrera narrativ och sammanhang i interaktionen.

Ort, förlag, år, upplaga, sidor
Stockholm, Sweden: KTH Royal Institute of Technology, 2024. , s. xiii, 54
Serie
TRITA-EECS-AVL ; 2024:13
Nyckelord [en]
human-robot interaction, social robotics, film sound design, robot sound, interactive sonification
Nyckelord [sv]
människa-robotinteraktion, social robotik, filmljuddesign, robotljud, interaktiv sonifiering
Nationell ämneskategori
Robotteknik och automation Människa-datorinteraktion (interaktionsdesign) Filmvetenskap
Forskningsämne
Medieteknik
Identifikatorer
URN: urn:nbn:se:kth:diva-342759ISBN: 978-91-8040-831-8 (tryckt)OAI: oai:DiVA.org:kth-342759DiVA, id: diva2:1832820
Disputation
2024-02-22, https://kth-se.zoom.us/j/61765490226, Kollegiesalen, Brinellvägen 8, Stockholm, 10:00 (Engelska)
Opponent
Handledare
Anmärkning

QC 20240131

Tillgänglig från: 2024-01-31 Skapad: 2024-01-30 Senast uppdaterad: 2024-02-05Bibliografiskt granskad
Delarbeten
1. Sonic characteristics of robots in films
Öppna denna publikation i ny flik eller fönster >>Sonic characteristics of robots in films
2019 (Engelska)Ingår i: Proceedings of the 16th Sound and Music Computing Conference, Malaga, Spain, 2019, s. 1-6, artikel-id P2.7Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

Robots are increasingly becoming an integral part of our everyday life. Expectations on robots could be influenced by how robots are represented in science fiction films. We hypothesize that sonic interaction design for real-world robots may find inspiration from sound design of fictional robots. In this paper, we present an exploratory study focusing on sonic characteristics of robot sounds in films. We believe that findings from the current study could be of relevance for future robotic applications involving the communication of internal states through sounds, as well for sonification of expressive robot movements. Excerpts from five films were annotated and analysed using Long Time Average Spectrum (LTAS). As an overall observation, we found that robot sonic presence is highly related to the physical appearance of robots. Preliminary results show that most of the robots analysed in this study have “metallic” voice qualities, matching the material of their physical form. Characteristics of robot voices show significant differences compared to voices of human characters; fundamental frequency of robotic voices is either shifted to higher or lower values, and the voices span over a broader frequency band.

Ort, förlag, år, upplaga, sidor
Malaga, Spain: , 2019
Nyckelord
robot sound, non-verbal communication, sonic interaction design, human-robot interaction, film sound design, LTAS
Nationell ämneskategori
Medieteknik Människa-datorinteraktion (interaktionsdesign) Mediateknik Interaktionsteknik Musik Datorseende och robotik (autonoma system)
Forskningsämne
Medieteknik; Människa-datorinteraktion; Konst, teknik och design
Identifikatorer
urn:nbn:se:kth:diva-250788 (URN)2-s2.0-85084391030 (Scopus ID)
Konferens
Sound and Music Computing Conference
Projekt
SONAO
Forskningsfinansiär
Vetenskapsrådet, 2017-03979NordForsk, 86892
Anmärkning

QC 20190625

Tillgänglig från: 2019-05-06 Skapad: 2019-05-06 Senast uppdaterad: 2024-03-15Bibliografiskt granskad
2. Exploring emotion perception in sonic HRI
Öppna denna publikation i ny flik eller fönster >>Exploring emotion perception in sonic HRI
2020 (Engelska)Ingår i: 17th Sound and Music Computing Conference, Torino: Zenodo , 2020, s. 434-441Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

Despite the fact that sounds produced by robots can affect the interaction with humans, sound design is often an overlooked aspect in Human-Robot Interaction (HRI). This paper explores how different sets of sounds designed for expressive robot gestures of a humanoid Pepper robot can influence the perception of emotional intentions. In the pilot study presented in this paper, it has been asked to rate different stimuli in terms of perceived affective states. The stimuli were audio, audio-video and video only and contained either Pepper’s original servomotors noises, sawtooth, or more complex designed sounds. The preliminary results show a preference for the use of more complex sounds, thus confirming the necessity of further exploration in sonic HRI.

Ort, förlag, år, upplaga, sidor
Torino: Zenodo, 2020
Nationell ämneskategori
Medieteknik Människa-datorinteraktion (interaktionsdesign) Datorseende och robotik (autonoma system) Annan data- och informationsvetenskap
Forskningsämne
Medieteknik; Konst, teknik och design; Människa-datorinteraktion
Identifikatorer
urn:nbn:se:kth:diva-277947 (URN)10.5281/ZENODO.3898928 (DOI)2-s2.0-85101259342 (Scopus ID)
Konferens
Sound and Music Computing Conference, Torino, 24-26 June 2020
Projekt
SONAO
Forskningsfinansiär
Vetenskapsrådet, 2017-03979
Anmärkning

QC 20200722

Tillgänglig från: 2020-07-02 Skapad: 2020-07-02 Senast uppdaterad: 2024-01-30Bibliografiskt granskad
3. Probing Aesthetics Strategies for Robot Sound: Complexity and Materiality in Movement Sonification
Öppna denna publikation i ny flik eller fönster >>Probing Aesthetics Strategies for Robot Sound: Complexity and Materiality in Movement Sonification
2023 (Engelska)Ingår i: ACM Transactions on Human-Robot Interaction, E-ISSN 2573-9522Artikel i tidskrift (Refereegranskat) Published
Abstract [en]

This paper presents three studies where we probe aesthetics strategies of sound produced by movement sonification of a Pepper robot by mapping its movements to sound models.

We developed two sets of sound models. The first set was made by two sound models, a sawtooth-based one and another based on feedback chains, for investigating how the perception of synthesized robot sounds would depend on their design complexity. We implemented the second set of sound models for probing the “materiality” of sound made by a robot in motion. This set consisted of a sound synthesis based on an engine highlighting the robot’s internal mechanisms, a metallic sound synthesis highlighting the robot’s typical appearance, and a whoosh sound synthesis highlighting the movement.

We conducted three studies. The first study explores how the first set of sound models can influence the perception of expressive gestures of a Pepper robot through an online survey. In the second study, we carried out an experiment in a museum installation with a Pepper robot presented in two scenarios: (1) while welcoming patrons into a restaurant and (2) while providing information to visitors in a shopping center. Finally, in the third study, we conducted an online survey with stimuli similar to those used in the second study.

Our findings suggest that participants preferred more complex sound models for the sonification of robot movements. Concerning the materiality, participants liked better subtle sounds that blend well with the ambient sound (i.e., less distracting) and soundscapes in which sound sources can be identified. Also, sound preferences varied depending on the context in which participants experienced the robot-generated sounds (e.g., as a live museum installation vs. an online display).

Ort, förlag, år, upplaga, sidor
Association for Computing Machinery (ACM), 2023
Nyckelord
SONAO
Nationell ämneskategori
Människa-datorinteraktion (interaktionsdesign) Robotteknik och automation
Identifikatorer
urn:nbn:se:kth:diva-324962 (URN)10.1145/3585277 (DOI)001153514400008 ()2-s2.0-85170233153 (Scopus ID)
Anmärkning

QC 20230328

Tillgänglig från: 2023-03-21 Skapad: 2023-03-21 Senast uppdaterad: 2024-03-05Bibliografiskt granskad
4. PepperOSC: enabling interactive sonification of a robot's expressive movement
Öppna denna publikation i ny flik eller fönster >>PepperOSC: enabling interactive sonification of a robot's expressive movement
2023 (Engelska)Ingår i: Journal on Multimodal User Interfaces, ISSN 1783-7677, E-ISSN 1783-8738, Vol. 17, nr 4, s. 231-239Artikel i tidskrift (Refereegranskat) Published
Abstract [en]

This paper presents the design and development of PepperOSC, an interface that connects Pepper and NAO robots with soundproduction tools to enable the development of interactive sonification in human-robot interaction (HRI). The interface usesOpen Sound Control (OSC) messages to stream kinematic data from robots to various sound design and music productiontools. The goals of PepperOSC are twofold: (i) to provide a tool for HRI researchers in developing multimodal user interfacesthrough sonification, and (ii) to lower the barrier for sound designers to contribute to HRI. To demonstrate the potential useof PepperOSC, this paper also presents two applications we have conducted: (i) a course project by two master’s studentswho created a robot sound model in Pure Data, and (ii) a museum installation of Pepper robot, employing sound modelsdeveloped by a sound designer and a composer/researcher in music technology usingMaxMSP and SuperCollider respectively.Furthermore, we discuss the potential use cases of PepperOSC in social robotics and artistic contexts. These applicationsdemonstrate the versatility of PepperOSC and its ability to explore diverse aesthetic strategies for robot movement sonification,offering a promising approach to enhance the effectiveness and appeal of human-robot interactions.

Ort, förlag, år, upplaga, sidor
Springer Nature, 2023
Nyckelord
Interactive sonification, human-robot interaction, robot sound design
Nationell ämneskategori
Robotteknik och automation Medieteknik Människa-datorinteraktion (interaktionsdesign)
Forskningsämne
Konst, teknik och design
Identifikatorer
urn:nbn:se:kth:diva-342756 (URN)10.1007/s12193-023-00414-9 (DOI)001061715800001 ()2-s2.0-85176385871 (Scopus ID)
Projekt
SONAO
Forskningsfinansiär
Vetenskapsrådet, 2017-03979NordForsk, 86892
Anmärkning

QC 20240130

Tillgänglig från: 2024-01-30 Skapad: 2024-01-30 Senast uppdaterad: 2024-01-30Bibliografiskt granskad
5. Semiotic analysis of robot sounds in films: implications for sound design in social robotics
Öppna denna publikation i ny flik eller fönster >>Semiotic analysis of robot sounds in films: implications for sound design in social robotics
(Engelska)Manuskript (preprint) (Övrigt vetenskapligt)
Abstract [en]

This paper investigates the sound design of robots in films and their potential influence on the field of social robotics. Cinematic robot portrayals have inspired researchers and practitioners in Human-Robot Interaction (HRI). While the non-verbal sounds of iconic film robots like R2-D2 and Wall-E have been explored, this study takes a more comprehensive approach. We explore a broader selection of 15 films featuring humanoid robots across decades through a semiotic analysis of their non-verbal communication sounds, including those related to movements and internal mechanisms. Our analysis, guided by Bateman and Schmidt’s multimodal film analysis framework following Saussure’s organization of signs through paradigmatic and syntagmatic relations, interprets the paradigmatic axis as the examination of the sound and the syntagmatic axis as the examination of the events surrounding the sound. The findings uncover two primary film robot sound materials: mechanical and synthetic. Additionally, contextual analysis reveals three narrative themes and several sub-themes related to the physical attributes of robots, their internal workings, and their interactions with other characters. The discussion section explores the implications of these findings for social robotics, including the importance of sound materials, the role of movement sounds in communication and emotional expression, and the significance of narrative and context in human-robot interaction. The paper also acknowledges the challenges in translating film sound design into practical applications in social robotics. This study provides valuable insights for HRI researchers, practitioners, and sound designers seeking to enhance non-verbal auditory expressions in social robots.

Nyckelord
robot sound, film sound design, human-robot interaction, semiotic analysis
Nationell ämneskategori
Människa-datorinteraktion (interaktionsdesign) Robotteknik och automation Filmvetenskap
Forskningsämne
Konst, teknik och design
Identifikatorer
urn:nbn:se:kth:diva-342757 (URN)
Anmärkning

QC 20240130

Tillgänglig från: 2024-01-30 Skapad: 2024-01-30 Senast uppdaterad: 2024-01-31Bibliografiskt granskad

Open Access i DiVA

Kappa(17516 kB)179 nedladdningar
Filinformation
Filnamn FULLTEXT01.pdfFilstorlek 17516 kBChecksumma SHA-512
96a12db7c5902f0045d084695a09925d714884e16ab9c5c08c51b43437dd6e20d356d82e67fb987ddc1b3a5c015de4d53afd05f9af333f9eba1400f2363cf097
Typ fulltextMimetyp application/pdf

Person

Latupeirissa, Adrian Benigno

Sök vidare i DiVA

Av författaren/redaktören
Latupeirissa, Adrian Benigno
Av organisationen
Medieteknik och interaktionsdesign, MID
Robotteknik och automationMänniska-datorinteraktion (interaktionsdesign)Filmvetenskap

Sök vidare utanför DiVA

GoogleGoogle Scholar
Totalt: 179 nedladdningar
Antalet nedladdningar är summan av nedladdningar för alla fulltexter. Det kan inkludera t.ex tidigare versioner som nu inte längre är tillgängliga.

isbn
urn-nbn

Altmetricpoäng

isbn
urn-nbn
Totalt: 1062 träffar
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf