kth.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Probing Aesthetics Strategies for Robot Sound: Complexity and Materiality in Movement Sonification
KTH, School of Electrical Engineering and Computer Science (EECS), Human Centered Technology, Media Technology and Interaction Design, MID.ORCID iD: 0000-0003-3572-6429
KTH, School of Electrical Engineering and Computer Science (EECS), Human Centered Technology, Media Technology and Interaction Design, MID.ORCID iD: 0000-0002-1244-881x
KTH, School of Electrical Engineering and Computer Science (EECS), Human Centered Technology, Media Technology and Interaction Design, MID.ORCID iD: 0000-0002-3086-0322
2023 (English)In: ACM Transactions on Human-Robot Interaction, E-ISSN 2573-9522Article in journal (Refereed) Published
Abstract [en]

This paper presents three studies where we probe aesthetics strategies of sound produced by movement sonification of a Pepper robot by mapping its movements to sound models.

We developed two sets of sound models. The first set was made by two sound models, a sawtooth-based one and another based on feedback chains, for investigating how the perception of synthesized robot sounds would depend on their design complexity. We implemented the second set of sound models for probing the “materiality” of sound made by a robot in motion. This set consisted of a sound synthesis based on an engine highlighting the robot’s internal mechanisms, a metallic sound synthesis highlighting the robot’s typical appearance, and a whoosh sound synthesis highlighting the movement.

We conducted three studies. The first study explores how the first set of sound models can influence the perception of expressive gestures of a Pepper robot through an online survey. In the second study, we carried out an experiment in a museum installation with a Pepper robot presented in two scenarios: (1) while welcoming patrons into a restaurant and (2) while providing information to visitors in a shopping center. Finally, in the third study, we conducted an online survey with stimuli similar to those used in the second study.

Our findings suggest that participants preferred more complex sound models for the sonification of robot movements. Concerning the materiality, participants liked better subtle sounds that blend well with the ambient sound (i.e., less distracting) and soundscapes in which sound sources can be identified. Also, sound preferences varied depending on the context in which participants experienced the robot-generated sounds (e.g., as a live museum installation vs. an online display).

Place, publisher, year, edition, pages
Association for Computing Machinery (ACM) , 2023.
Keywords [en]
SONAO
National Category
Human Computer Interaction Robotics
Identifiers
URN: urn:nbn:se:kth:diva-324962DOI: 10.1145/3585277ISI: 001153514400008Scopus ID: 2-s2.0-85170233153OAI: oai:DiVA.org:kth-324962DiVA, id: diva2:1745059
Note

QC 20230328

Available from: 2023-03-21 Created: 2023-03-21 Last updated: 2024-03-05Bibliographically approved
In thesis
1. From Motion Pictures to Robotic Features: Adopting film sound design practices to foster sonic expression in social robotics through interactive sonification
Open this publication in new window or tab >>From Motion Pictures to Robotic Features: Adopting film sound design practices to foster sonic expression in social robotics through interactive sonification
2024 (English)Doctoral thesis, comprehensive summary (Other academic) [Artistic work]
Alternative title[sv]
Från filmer till robotfunktioner : Användning av praxis inom filmljuddesign för att främja ljuduttryck i social robotik genom interaktiv sonifiering
Abstract [en]

This dissertation investigates the role of sound design in social robotics, drawing inspiration from robot depictions in science-fiction films. It addresses the limitations of robots’ movements and expressive behavior by integrating principles from film sound design, seeking to improve human-robot interaction through expressive gestures and non-verbal sounds.

The compiled works are structured into two parts. The first part focuses on perceptual studies, exploring how people perceive non-verbal sounds displayed by a Pepper robot related to its movement. These studies highlighted preferences for more refined sound models, subtle sounds that blend with ambient sounds, and sound characteristics matching the robot’s visual attributes. This part also resulted in a programming interface connecting the Pepper robot with sound production tools.

The second part focuses on a structured analysis of robot sounds in films, revealing three narrative themes related to robot sounds in films with implications for social robotics. The first theme involves sounds associated with the physical attributes of robots, encompassing sub-themes of sound linked to robot size, exposed mechanisms, build quality, and anthropomorphic traits. The second theme delves into sounds accentuating robots’ internal workings, with sub-themes related to learning and decision-making processes. Lastly, the third theme revolves around sounds utilized in robots’ interactions with other characters within the film scenes.

Based on these works, the dissertation discusses sound design recommendations for social robotics inspired by practices in film sound design. These recommendations encompass selecting the appropriate sound materials and sonic characteristics such as pitch and timbre, employing movement sound for effective communication and emotional expression, and integrating narrative and context into the interaction.

Abstract [sv]

Denna avhandling undersöker ljuddesignens roll i social robotik, med inspiration från robotskildringar i science fiction filmer. Avhandlingen diskuterar begränsningar i robotars uttrycksfulla beteenden genom att integrera principer från filmljuddesign. Arbetet syftar till att främja interaktionen mellan människa och robot genom att förse robotar med uttrycksfulla gester och icke-verbala ljud.

Denna sammanläggningsavhandling inkluderar ett antal artiklar som är strukturerade i två separata delar. Den första delen fokuserar på perceptuella studier och undersöker hur människor uppfattar de icke-verbala ljud som roboten Pepper producerar i samband med sina rörelser. Dessa studier belyste preferenser för mer förfinade ljudmodeller, subtila ljud som blandas med omgivande ljud, och ljudegenskaper som matchar robotens visuella attribut. Denna del resulterade också i ett programmeringsgränssnitt som sammankopplar Pepper-roboten och ljudproduktionsverktyg.

Den andra delen fokuserar på en strukturerad analys av robotljud i filmer och avslöjar tre narrativa teman relaterade till robotljud i filmer med implikationer för social robotik. Det första temat handlar om ljud som förknippas med robotarnas fysiska attribut och omfattar underteman av ljud som är kopplade till robotstorlek, exponerade mekanismer, byggkvalitet, och antropomorfa drag. Det andra temat fördjupar sig i ljud som betonar robotarnas interna arbete, med underteman relaterade till inlärnings- och beslutsprocesser. Slutligen kretsar det tredje temat kring ljud som används i robotarnas interaktion med andra karaktärer i filmscenerna.

Baserat på ovan beskrivna arbeten diskuterar denna avhandling rekommendationer för ljuddesign inom social robotik inspirerade av praxis inom filmljuddesign. Dessa rekommendationer omfattar att välja lämpliga ljudmaterial och ljudegenskaper såsom tonhöjd och klangfärg, att använda rörelseljud för effektiv kommunikation och känslomässiga uttryck, samt att integrera narrativ och sammanhang i interaktionen.

Place, publisher, year, edition, pages
Stockholm, Sweden: KTH Royal Institute of Technology, 2024. p. xiii, 54
Series
TRITA-EECS-AVL ; 2024:13
Keywords
human-robot interaction, social robotics, film sound design, robot sound, interactive sonification, människa-robotinteraktion, social robotik, filmljuddesign, robotljud, interaktiv sonifiering
National Category
Robotics Human Computer Interaction Studies on Film
Research subject
Media Technology
Identifiers
urn:nbn:se:kth:diva-342759 (URN)978-91-8040-831-8 (ISBN)
Public defence
2024-02-22, https://kth-se.zoom.us/j/61765490226, Kollegiesalen, Brinellvägen 8, Stockholm, 10:00 (English)
Opponent
Supervisors
Note

QC 20240131

Available from: 2024-01-31 Created: 2024-01-30 Last updated: 2024-02-05Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Authority records

Latupeirissa, Adrian BenignoPanariello, ClaudioBresin, Roberto

Search in DiVA

By author/editor
Latupeirissa, Adrian BenignoPanariello, ClaudioBresin, Roberto
By organisation
Media Technology and Interaction Design, MID
In the same journal
ACM Transactions on Human-Robot Interaction
Human Computer InteractionRobotics

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 243 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf