kth.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Interactive Sound and Music Technology for Everyone: Designing Inclusive Standards for Web Audio Applications
KTH, School of Electrical Engineering and Computer Science (EECS), Human Centered Technology, Media Technology and Interaction Design, MID. Royal College of Music, Stockholm, Sweden. (Sound and Music Computing)ORCID iD: 0000-0003-3399-8869
2024 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

In this compilation thesis, I examine how systems and formats can be designed to include more people in creating interactive sound and music applications. I contribute knowledge, aiming to include everyone, but focus specifically on musicians with no programming skills and little interest in technical challenges. 

I have designed, developed, and evaluated a system – WebAudioXML (WAXML) – for implementing interactive sound and music in web pages using native web technologies. The system’s design is novel, and the work contributes knowledge about how markup language and spreadsheet concepts can describe sound and music structures. The results give insights into how high-level musical representation can be structured, named, and designed to be understood by those without prior programming experience. 

I also use WAXML to address musical diversity in interactive applications. I identify and solve technical challenges where current systems struggle to implement traditionally performed music. Novel solutions are designed, evaluated, discussed, and presented in the included papers. 

The system is finally implemented in applications aimed at education and inclusion, where I evaluate them through a series of case studies. The results confirm Web Audio as a solid platform for accessible learning, sharing, and distribution of audio applications and suggest that collective efforts shaping an ecosystem with a universal format would enable even more creators to make interactive sound and music applications. 

I research FOR the art THROUGH design. The knowledge output is valid for any interactive sound and music system but specifically addresses the design of Web Audio applications. 

Abstract [sv]

I denna sammanläggningsavhandling undersöker jag hur interaktiva system och format kan utformas för att inkludera fler personer i skapandet av inter- aktiva ljud- och musikapplikationer. Jag bidrar med kunskap med målet att inkludera alla, men fokuserar särskilt på musiker utan programmeringskunskaper och med begränsat intresse för tekniska utmaningar.

Jag har designat, utvecklat och utvärderat ett system – WebAudioXML (WAXML) – för att implementera interaktivt ljud och musik på webbsidor med hjälp av standardiserade webbtekniker. Systemet är nyskapande i sin design och arbetet bidrar med kunskap om hur markup-kod and koncept från kalkylblad kan användas för att beskriva ljud- och musikstrukturer. Resultaten ger insikter i hur musikalisk representation på en övergripande nivå kan struktureras, namnges och utformas för att bli förstådd av av dem som inte har tidigare erfarenhet av programmering.

Jag använder också WAXML som plattform för att utforska möjligheter till större musikalisk mångfald i interaktiva applikationer. Jag identifierar och löser tekniska utmaningar där nuvarande system har problem med att implementera musik som spelas enligt traditionen. Jag designar, utvärderar, diskuterar och presenterar nya lösningar i de inkluderade artiklarna.

Jag tillämpar även WAXML på applikationer avsedda för utbildning och inkludering, där jag utvärderar dem genom en serie fallstudier. Dessa bekräftar Web Audio som en stabil plattform för tillgängligt lärande, delning och distribution och pekar på att gemensamma insatser för att skapa ett ekosystem med ett universellt format skulle möjliggöra för fler att kunna skapa interaktiva ljud- och musikapplikationer.

Jag bedriver forskning FÖR konsten GENOM design. Resultatet är relevant för interaktiva ljud- och musiksystem i allmänhet och Web Audio- applikationer i synnerhet.

Place, publisher, year, edition, pages
Stockholm: KTH Royal Institute of Technology, 2024. , p. 69
Series
TRITA-EECS-AVL ; 2024:52
Keywords [en]
Interactive Music, Web Audio, Markup Language, Inclusion, Open Source Standards
Keywords [sv]
Interaktiv musik, Web Audio, Markup-kod, Inkludering, Öppna Standarder
National Category
Human Computer Interaction
Research subject
Human-computer Interaction
Identifiers
URN: urn:nbn:se:kth:diva-346629ISBN: 978-91-8040-950-6 (print)OAI: oai:DiVA.org:kth-346629DiVA, id: diva2:1859183
Public defence
2024-06-11, https://kmh-se.zoom.us/j/62738893688?pwd=L3hQTWZqRVg3OExJenliTE1kbnhtUT09, Lilla salen, Royal College of Music, Valhallavägen105, Stockholm, 13:00 (English)
Opponent
Supervisors
Funder
Royal College of Music in Stockholm
Note

QC 20240521

Available from: 2024-05-21 Created: 2024-05-21 Last updated: 2024-06-10Bibliographically approved
List of papers
1. Audio Parameter Mapping Made Explicit Using WebAudioXML
Open this publication in new window or tab >>Audio Parameter Mapping Made Explicit Using WebAudioXML
2021 (English)In: Proceedings of the Sound and Music Computing Conference / [ed] Sound and Music Computing Conference, Torino, 2021Conference paper, Published paper (Refereed)
Abstract [en]

Sonification using audio parameter mapping involves both aesthetic and technical challenges and requires interdisciplinary skills on a high level to produce a successful result. With the aim to lower the barrier for students to enter the field of sonification, we developed and presented WebAudioXML at SMC2020. Since then, more than 40 student projects has successfully proven that the technology is highly beneficial for non-programmers to learn how to create interactive web audio applications. With this study, we present new feature for WebAudioXML that also makes advanced audio parameter mapping, data interpolation and value conversion more accessible and easy to assess. Three student projects act as base for the syntax definition and by using an annotated portfolio and video recorded interviews with experts from the sound and music computing community, we present important insights from the project. The participants contributed with critical feedback and questions that helped us to better understand the strengths and weaknesses with the proposed syntax. We conclude that the technology is robust and useful and present new ideas that emerged from this study.

Place, publisher, year, edition, pages
Torino: , 2021
National Category
Music Interaction Technologies
Identifiers
urn:nbn:se:kth:diva-299920 (URN)2-s2.0-85122066758 (Scopus ID)
Conference
Sound and Music Computing Conference 2021
Note

QC 20210901

Available from: 2021-08-19 Created: 2021-08-19 Last updated: 2024-05-21Bibliographically approved
2. Towards a Standard for Interactive Music
Open this publication in new window or tab >>Towards a Standard for Interactive Music
2024 (English)In: Innovation in Music: Technology and Creativity / [ed] Jan-Olof Gullö, Russ Hepworth-Sawyer, Justin Paterson, Rob Toulson, Mark Marrington, London: Taylor & Francis, 2024, 1st, p. 1-16Chapter in book (Refereed)
Abstract [en]

Interactive Media is a rapidly growing market and an increasingly important target for music production. The field has attracted a lot of focus from both the industry and academia, but there is still a great potential for further development of tools and formats for content creation and implementation.

Even if music is an important component in media production, there is still no open file format for delivering and sharing interactive musical content between different applications, and the terminology varies between different applications. This study aims at finding useful terminologies and requirements for such a format.

Over a period of eight years, students, teachers, and researchers from the Royal College of Music in Stockholm (KMH) have participated with artistic visions, prototyping and testing in the development of a JavaScript framework called “iMusicXML”. In this exploratory design study, the current state of iMusicXML is analyzed to reveal important key concepts and features drawn from more than 100 student projects. Several features and solutions that have been proven useful are presented but also critiqued due to limited perspectives. It is also suggested that a wide range of users, genres, and applications should be invited to a continuing discussion about standards for interactive music.

Place, publisher, year, edition, pages
London: Taylor & Francis, 2024 Edition: 1st
National Category
Human Computer Interaction
Research subject
Human-computer Interaction
Identifiers
urn:nbn:se:kth:diva-346628 (URN)10.4324/9781003118817-14 (DOI)
Note

QC 20240522

Part of ISBN 9781003118817

Available from: 2024-05-21 Created: 2024-05-21 Last updated: 2024-05-22Bibliographically approved
3. Solving the Problem of Expressive Musical Timing in Interactive Music Systems
Open this publication in new window or tab >>Solving the Problem of Expressive Musical Timing in Interactive Music Systems
(English)Manuscript (preprint) (Other academic)
Abstract [en]

Interactive music systems often implement music into games using approaches that tend to premiere music with a steady beat and quantized onsets to make seamless transitions. We identified a problem in which folk music and other genres with expressive musical timing cannot be used in these systems without violating musical traditions.

This design study explores a novel solution and extends current practices with Flexible Horizontal Re-sequencing to solve the problem. We designed a system to evaluate the concept by running a project with all the necessary steps, from recording the music to implementing it into an open, accessible web application.

We use notes from the development process, musicians’ evaluations, and interviews to validate the system and learn what features are needed to succeed and what musical factors contribute to the result. We conclude that the solution works well and that the system can perform seamless transitions if it has features for implementing sync points and phrase annotations to make individual crossfades for different instruments.

We found critical musical aspects of the recording session that can improve the result without violating genre traditions or musicians’ playing styles. We also suggest that a shared file format for MIR and interactive music systems would contribute to an efficient ecosystem where Flexible Horizontal Re-sequencing could complement current interactive music approaches to include expressive musical timing.

Keywords
Interactive Music, Folk Music, Expressive Musical Timing
National Category
Human Computer Interaction
Identifiers
urn:nbn:se:kth:diva-346626 (URN)
Note

QC 20240521

Available from: 2024-05-21 Created: 2024-05-21 Last updated: 2024-05-21Bibliographically approved
4. Evaluating Web Audio for learning, accessibility and distribution
Open this publication in new window or tab >>Evaluating Web Audio for learning, accessibility and distribution
2022 (English)In: Journal of The Audio Engineering Society, ISSN 1549-4950, Vol. 70, no 11, p. 962-978Article in journal (Refereed) Published
Abstract [en]

Web Audio has a great potential for interactive audio content in which an open standard and easy integration with other web-based tools makes it particularly interesting. From earlier studies, obstacles for students to materialize creative ideas through programming were identified; focus shifted from artistic ambition to solving technical issues. This study builds upon 20 years of experience from teaching sound and music computing and evaluates how Web Audio contributes to the learning experience. Data was collected from different student projects through analysis of source code, reflective texts, group discussions, and online self-evaluation forms. The result indicates that Web Audio serves well as a learning platform and that an XML abstraction of the API helped the students to stay focused on the artistic output. It is also concluded that an online tool can reduce the time for getting started with Web Audio to less than 1 h. Although many obstacles have been successfully removed, the authors argue that there is still a great potential for new online tools targeting audio application development in which the accessibility and sharing features contribute to an even better learning experience. 

Place, publisher, year, edition, pages
Audio Engineering Society, 2022
National Category
Human Computer Interaction
Research subject
Media Technology
Identifiers
urn:nbn:se:kth:diva-317058 (URN)10.17743/jaes.2022.0031 (DOI)001068184800005 ()2-s2.0-85143869197 (Scopus ID)
Note

QC 20230329

Available from: 2022-09-05 Created: 2022-09-05 Last updated: 2024-05-21Bibliographically approved
5. Sonification For Everyone Everywhere: Evaluating The WebAudioXML Sonification Toolkit For Browsers
Open this publication in new window or tab >>Sonification For Everyone Everywhere: Evaluating The WebAudioXML Sonification Toolkit For Browsers
2021 (English)Conference paper, Published paper (Refereed)
Abstract [en]

Creating an effective sonification is a challenging task that requires skills and knowledge on an expertise level in several disciplines. This study contributes with WebAudioXML Sonification Toolkit (WAST) that aims at reaching new groups who have not yet considered themselves to be part of the ICAD community. We have designed, built, and evaluated the toolkit by analysing ten student projects using it and conclude that WAST did meet our expectations and that it lead to students taking a deep approach to learning and successfully contributed to reaching the learning outcomes. The result indicates that WAST is both easy-to-use, highly accessible, extensively flexible and offers possibilities to share the sonification in any device’s web browser simply through a web link, and without installations. We also suggest that a sonification toolkit would become an even more creative environment with virtual instruments and mixing features typically found in Digital Audio Workstations.

Keywords
Music, Musik, Interaction Technologies, Interaktionsteknik
National Category
Music Interaction Technologies
Identifiers
urn:nbn:se:kth:diva-299922 (URN)
Conference
The 26th International Conference on Auditory Display (ICAD 2021)
Note

QC 20210901

Available from: 2021-08-19 Created: 2021-08-19 Last updated: 2024-05-21Bibliographically approved
6. Collaborative music-making: special educational needs school assistants as facilitators in performances with accessible digital musical instruments
Open this publication in new window or tab >>Collaborative music-making: special educational needs school assistants as facilitators in performances with accessible digital musical instruments
Show others...
2023 (English)In: Frontiers in Computer Science, E-ISSN 2624-9898, Vol. 5, article id 1165442Article in journal (Refereed) Published
Abstract [en]

The field of research dedicated to Accessible Digital Musical Instruments (ADMIs) is growing and there is an increased interest in promoting diversity and inclusion in music-making. We have designed a novel system built into previously tested ADMIs that aims at involving assistants, students with Profound and Multiple Learning Disabilities (PMLD), and a professional musician in playing music together. In this study the system is evaluated in a workshop setting using quantitative as well as qualitative methods. One of the main findings was that the sounds from the ADMIs added to the musical context without making errors that impacted the music negatively even when the assistants mentioned experiencing a split between attending to different tasks, and a feeling of insecurity toward their musical contribution. We discuss the results in terms of how we perceive them as drivers or barriers toward reaching our overarching goal of organizing a joint concert that brings together students from the SEN school with students from a music school with a specific focus on traditional orchestral instruments. Our study highlights how a system of networked and synchronized ADMIs could be conceptualized to include assistants more actively in collaborative music-making, as well as design considerations that support them as facilitators.

Place, publisher, year, edition, pages
Frontiers Media SA, 2023
Keywords
collaborative music-making, Web Audio, assistants as facilitators, accessible digital musical instruments, interactive musical systems
National Category
Musicology Computer Sciences
Identifiers
urn:nbn:se:kth:diva-334301 (URN)10.3389/fcomp.2023.1165442 (DOI)001039158900001 ()2-s2.0-85166582445 (Scopus ID)
Note

QC 20230818

Available from: 2023-08-18 Created: 2023-08-18 Last updated: 2024-05-21Bibliographically approved

Open Access in DiVA

PhD Thesis - Interactive Sound and Music Technology for Everyone(4523 kB)405 downloads
File information
File name FULLTEXT01.pdfFile size 4523 kBChecksum SHA-512
bc7bc5ebf97221d19c6d07db7bbfbd6cf95d7a9e0e6af22c5f29a3f8d44051049dffc361a29866bae7a7a7190fa56f54b7c8817086cb98195f8a168d6a3d618a
Type summaryMimetype application/pdf

Authority records

Lindetorp, Hans

Search in DiVA

By author/editor
Lindetorp, Hans
By organisation
Media Technology and Interaction Design, MID
Human Computer Interaction

Search outside of DiVA

GoogleGoogle Scholar
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

isbn
urn-nbn

Altmetric score

isbn
urn-nbn
Total: 796 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf