Ändra sökning
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf
Interactive computer-aided expressive music performance: Analysis, control, modification and synthesis
KTH, Skolan för datavetenskap och kommunikation (CSC), Tal, musik och hörsel, TMH, Musikakustik.
2011 (Engelska)Doktorsavhandling, sammanläggning (Övrigt vetenskapligt)
Abstract [en]

This thesis describes the design and implementation process of two applications (PerMORFer and MoodifierLive) for the interactive manipulation of music performance. Such applications aim at closing the gap between the musicians, who play the music, and the listeners, who passively listen to it. The goal was to create computer programs that allow the user to actively control how the music is performed. This is achieved by modifying such parameters as tempo, dynamics, and articulation, much like a musician does when playing an instrument. An overview of similar systems and the problems related to their development is given in the first of the included papers.

Four requirements were defined for the applications: (1) to produce a natural, high quality sound; (2) to allow for realistic modifications of the performance parameters; (3) to be easy to control, even for non-musicians; (4) to be portable. Although there are many similarities between PerMORFer and MoodifierLive, the two applications fulfill different requirements. The first two were addressed in PerMORFer, with which the user can manipulate pre-recorded audio performance. The last two were addressed in MoodifierLive, a mobile phone application for gesture-based control of a MIDI score file. The tone-by tone modifications in both applications are based on the KTH rule system for music performance. The included papers describe studies, methods, and algorithms used in the development of the two applications.

Audio recordings of real performance have been used in PerMORFer toachieve a natural sound. The tone-by-tone manipulations defined by the KTH rules first require an analysis of the original performance to separate the tones and estimate their parameters (IOI, duration, dynamics). Available methods were combined with novel solutions, such as an approach to the separation of two overlapping sinusoidal components. On the topic of performance analysis, ad-hoc algorithms were also developed to analyze DJ scratching recordings.

A particularly complex problem is the estimation of a tone’s dynamic level. A study was conducted to identify the perceptual cues that listeners use to determinethe dynamics of a tone. The results showed that timbre is as important as loudness. These findings were applied in a partly unsuccessful attempt to estimate dynamics from spectral features.

The manipulation of tempo is a relatively simple problem, as is that of articulation (i.e. legato-staccato) as long as the tone can be separated. The modification of dynamics on the other hand is more difficult, as was its estimation. Following the findings of the previously mentioned perceptual study, a method to modify both loudness and timbre using a database of spectral models was implemented.

MoodifierLive was used to experiment with performance control interfaces. In particular, the mobile phone’s built-in accelerometer was used to track, analyze, and interpret the movements of the user. Expressive gestures were then mapped to corresponding expressive music performances. Evaluation showed that modes based on natural gestures were easier to use than those created witha top-down approach.

Ort, förlag, år, upplaga, sidor
Stockholm: KTH Royal Institute of Technology , 2011. , s. 69
Serie
Trita-CSC-A, ISSN 1653-5723 ; 2011:12
Nationell ämneskategori
Strömningsmekanik och akustik
Identifikatorer
URN: urn:nbn:se:kth:diva-34099ISBN: 978-91-7501-031-1 (tryckt)OAI: oai:DiVA.org:kth-34099DiVA, id: diva2:419099
Disputation
2011-06-15, F3, Lindstedtsvägen 26, KTH, Stockholm, 10:00 (Engelska)
Opponent
Handledare
Anmärkning
QC 20110607Tillgänglig från: 2011-06-07 Skapad: 2011-05-25 Senast uppdaterad: 2012-03-22Bibliografiskt granskad
Delarbeten
1. Systems for Interactive Control of Computer Generated Music Performance
Öppna denna publikation i ny flik eller fönster >>Systems for Interactive Control of Computer Generated Music Performance
2013 (Engelska)Ingår i: Guide to Computing for Expressive Music Performance / [ed] Kirke, A., & Miranda, E., Springer Berlin/Heidelberg, 2013, s. 49-73Kapitel i bok, del av antologi (Refereegranskat)
Abstract [en]

This chapter is a literature survey of systems for real-time interactive control of automatic expressive music performance. A classification is proposed based on two initial design choices: the music material to interact with (i.e., MIDI or audio recordings) and the type of control (i.e., direct control of the low-level parameters such as tempo, intensity, and instrument balance or mapping from high-level parameters, such as emotions, to low-level parameters). Their pros and cons are briefly discussed. Then, a generic approach to interactive control is presented, comprising four steps: control data collection and analysis, mapping from control data to performance parameters, modification of the music material, and audiovisual feedback synthesis. Several systems are then described, focusing on different technical and expressive aspects. For many of the surveyed systems, a formal evaluation is missing. Possible methods for the evaluation of such systems are finally discussed.

Ort, förlag, år, upplaga, sidor
Springer Berlin/Heidelberg, 2013
Nationell ämneskategori
Datavetenskap (datalogi) Människa-datorinteraktion (interaktionsdesign) Musik
Identifikatorer
urn:nbn:se:kth:diva-34088 (URN)10.1007/978-1-4471-4123-5_2 (DOI)978-1-4471-4123-5 (ISBN)
Anmärkning

Updated from In Press to Published

Tillgänglig från: 2011-05-25 Skapad: 2011-05-25 Senast uppdaterad: 2018-01-12Bibliografiskt granskad
2. Frequency, phase and amplitude estimation of overlapping partials in monaural musical signals
Öppna denna publikation i ny flik eller fönster >>Frequency, phase and amplitude estimation of overlapping partials in monaural musical signals
2010 (Engelska)Ingår i: 13th International Conference on Digital Audio Effects, DAFx 2010 Proceedings, 2010, s. 1-8Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

A method is described that simultaneously estimates the frequency, phase and amplitude of two overlapping partials in a monaural musical signal from the amplitudes and phases in three frequency bins of the signal's Odd Discrete Fourier Transform (ODFT). From the transform of the analysis window in its analytical form, and given the frequencies of the two partials, an analytical solution for the amplitude and phase of the two overlapping partials was obtained. Furthermore, the frequencies are estimated numerically solving a system of two equations and two unknowns, since no analytical solution could be found. Although the estimation is done independently frame by frame, particular situations (e.g. extremely close frequencies, same phase in the time window) lead to errors, which can be partly corrected with a moving average filter over several time frames. Results are presented for artificial sinusoids with time varying frequencies and amplitudes, and with different levels of noise added. The system still performs well with a Signalto- Noise ratio of down to 30 dB, with moderately modulated frequencies, and time varying amplitudes.

Nyckelord
Amplitude estimation, Analysis windows, Frequency bins, Moving average filter, Musical signals, Noise ratio, Odd discrete fourier transforms, Time frame, Time varying, Time windows, Time-varying frequency, Two equation
Nationell ämneskategori
Strömningsmekanik och akustik
Identifikatorer
urn:nbn:se:kth:diva-34083 (URN)2-s2.0-84872727032 (Scopus ID)978-320001940-9 (ISBN)
Konferens
13th International Conference on Digital Audio Effects, DAFx 2010; Graz; Austria; 6 September 2010 through 10 September 2010
Anmärkning

QC 20110708

Tillgänglig från: 2011-07-08 Skapad: 2011-05-25 Senast uppdaterad: 2014-08-14Bibliografiskt granskad
3. Influence of pitch, loudness, and timbre on the perception of instrument dynamics
Öppna denna publikation i ny flik eller fönster >>Influence of pitch, loudness, and timbre on the perception of instrument dynamics
2011 (Engelska)Ingår i: Journal of the Acoustical Society of America, ISSN 0001-4966, E-ISSN 1520-8524, Vol. 130, nr 4, s. EL193-EL199Artikel i tidskrift (Refereegranskat) Published
Abstract [en]

The effect of variations in pitch, loudness, and timbre on the perception of the dynamics of isolated instrumental tones is investigated. A full factorial design was used in a listening experiment. The subjects were asked to indicate the perceived dynamics of each stimulus on a scale from pianissimo to fortissimo. Statistical analysis showed that for the instruments included (i.e., clarinet, flute, piano, trumpet, and violin) timbre and loudness had equally large effects, while pitch was relevant mostly for the first three. The results confirmed our hypothesis that loudness alone is not a reliable estimate of the dynamics of musical tones.

Nationell ämneskategori
Data- och informationsvetenskap
Forskningsämne
Tal- och musikkommunikation
Identifikatorer
urn:nbn:se:kth:diva-34090 (URN)10.1121/1.3633687 (DOI)000295799400011 ()2-s2.0-82255196536 (Scopus ID)
Anmärkning

QC 20111104 Updated from submitted to published.

Tillgänglig från: 2011-05-25 Skapad: 2011-05-25 Senast uppdaterad: 2018-09-13Bibliografiskt granskad
4. Analysis of the acoustics and playing strategies of turntable scratching
Öppna denna publikation i ny flik eller fönster >>Analysis of the acoustics and playing strategies of turntable scratching
2011 (Engelska)Ingår i: Acta Acoustica united with Acustica, ISSN 1610-1928, E-ISSN 1861-9959, Vol. 97, nr 2, s. 303-314Artikel i tidskrift (Refereegranskat) Published
Abstract [en]

Scratching performed by a DJ (disk jockey) is a skillful style of playingthe turntable with complex musical output. This study focuses on the description of some of the acoustical parameters and playing strategies of typical scratch improvisations, and how these parameters typically are used for expressive performance. Three professional DJs were instructed to express different emotions through improvisations, and both audio and gesturaldata were recorded. Feature extraction and analysis of the recordings are based on a combination of audio and gestural data, instrument characteristics, and playing techniques. The acoustical and performance parameters extracted from the recordings give a first approximation on the functional ranges within which DJs normally play. Results from the analysis show that parameters which are important for other solo instrument performances, suchas pitch, have less influence in scratching. Both differences and commonalities between the DJs’ playing styles were found. Impact that the findings of this work may have on constructing models for scratch performances arediscussed.

Ort, förlag, år, upplaga, sidor
S Hirzel verlag, 2011
Nyckelord
MUSIC PERFORMANCE; FREQUENCY; PERCEPTION; DURATION; PROSODY
Nationell ämneskategori
Datavetenskap (datalogi) Människa-datorinteraktion (interaktionsdesign) Musik Psykologi
Identifikatorer
urn:nbn:se:kth:diva-11923 (URN)10.3813/AAA.918410 (DOI)000288130700014 ()2-s2.0-79952119561 (Scopus ID)
Anmärkning

QC20100729 (Uppdaterad från submitted till published 20110328). QC 20160115

Tillgänglig från: 2010-01-26 Skapad: 2010-01-26 Senast uppdaterad: 2018-01-12Bibliografiskt granskad
5. A method for the modification of acoustic instrument tone dynamics
Öppna denna publikation i ny flik eller fönster >>A method for the modification of acoustic instrument tone dynamics
2009 (Engelska)Ingår i: Proceedings of the 12th International Conference on Digital Audio Effects, DAFx 2009, 2009, s. 359-364Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

A method is described for making natural sounding modifications of the dynamic level of tones produced by acoustic instruments. Each tone is first analyzed in the frequency domain and divided into a harmonic and a noise component. The two components are modified separately using filters based on spectral envelopes extracted from recordings of isolated tones played at different dynamic levels. When transforming from low to high dynamics, additional high frequency partials are added to the spectrum to enhance the brightness of the sound. Finally, the two modified components are summed and a time domain signal is synthesized.

Nyckelord
Dynamic levels, Frequency domains, High frequency, Low-to-high, Noise components, Spectral envelopes, Time-domain signal, Two-component
Nationell ämneskategori
Strömningsmekanik och akustik
Identifikatorer
urn:nbn:se:kth:diva-34093 (URN)2-s2.0-84872714140 (Scopus ID)
Konferens
12th International Conference on Digital Audio Effects, DAFx 2009; Como; Italy; 1 September 2009 through 4 September 2009
Anmärkning

QC 20110607

Tillgänglig från: 2011-05-25 Skapad: 2011-05-25 Senast uppdaterad: 2014-09-24Bibliografiskt granskad
6. Interactive sonification of expressive hand gestures on a handheld device
Öppna denna publikation i ny flik eller fönster >>Interactive sonification of expressive hand gestures on a handheld device
2012 (Engelska)Ingår i: Journal on Multimodal User Interfaces, ISSN 1783-7677, E-ISSN 1783-8738, Vol. 6, nr 1-2, s. 49-57Artikel i tidskrift (Refereegranskat) Published
Abstract [en]

We present here a mobile phone application called MoodifierLive which aims at using expressive music performances for the sonification of expressive gestures through the mapping of the phone’s accelerometer data to the performance parameters (i.e. tempo, sound level, and articulation). The application, and in particular the sonification principle, is described in detail. An experiment was carried out to evaluate the perceived matching between the gesture and the music performance that it produced, using two distinct mappings between gestures and performance. The results show that the application produces consistent performances, and that the mapping based on data collected from real gestures works better than one defined a priori by the authors.

Nyckelord
Automatic music performance, Emotional hand gestures, Mobile phone, Sonification
Nationell ämneskategori
Data- och informationsvetenskap Människa-datorinteraktion (interaktionsdesign) Psykologi
Identifikatorer
urn:nbn:se:kth:diva-34084 (URN)10.1007/s12193-011-0076-2 (DOI)000309998800006 ()2-s2.0-84863200318 (Scopus ID)
Projekt
SAME
Forskningsfinansiär
Vetenskapsrådet, 2010-4654EU, FP7, Sjunde ramprogrammet, FP7-ICT-STREP-215749
Anmärkning

QC 20120809. Updated from submitted to published.

Tillgänglig från: 2011-05-25 Skapad: 2011-05-25 Senast uppdaterad: 2018-01-12Bibliografiskt granskad
7. Permorfer: Interactive Rule-Based Modification of Audio Recordings
Öppna denna publikation i ny flik eller fönster >>Permorfer: Interactive Rule-Based Modification of Audio Recordings
2011 (Engelska)Ingår i: Computer music journal, ISSN 0148-9267, E-ISSN 1531-5169Artikel i tidskrift (Övrigt vetenskapligt) Submitted
Nationell ämneskategori
Datavetenskap (datalogi)
Identifikatorer
urn:nbn:se:kth:diva-34085 (URN)
Anmärkning
QS 20120316Tillgänglig från: 2011-05-25 Skapad: 2011-05-25 Senast uppdaterad: 2018-01-12Bibliografiskt granskad
8. MoodifierLive: Interactive and collaborative music performance on mobile devices
Öppna denna publikation i ny flik eller fönster >>MoodifierLive: Interactive and collaborative music performance on mobile devices
2011 (Engelska)Ingår i: Proceedings of the International Conference on New Interfaces for Musical Expression (NIME11), 2011Konferensbidrag, Publicerat paper (Refereegranskat)
Nationell ämneskategori
Strömningsmekanik och akustik
Identifikatorer
urn:nbn:se:kth:diva-34097 (URN)
Anmärkning
QC 20110607Tillgänglig från: 2011-05-25 Skapad: 2011-05-25 Senast uppdaterad: 2012-01-21Bibliografiskt granskad

Open Access i DiVA

Fabiani2011thesis(2059 kB)405 nedladdningar
Filinformation
Filnamn FULLTEXT01.pdfFilstorlek 2059 kBChecksumma SHA-512
3e5ade4a05536dce998181717efaaf7d60a8af66532f6c8567810ee9783de1aa0f4deb5d00f3ba59bc0c68980886ca67a4e0b965611a1f5deecf3b07a69405db
Typ fulltextMimetyp application/pdf

Sök vidare i DiVA

Av författaren/redaktören
Fabiani, Marco
Av organisationen
Musikakustik
Strömningsmekanik och akustik

Sök vidare utanför DiVA

GoogleGoogle Scholar
Totalt: 405 nedladdningar
Antalet nedladdningar är summan av nedladdningar för alla fulltexter. Det kan inkludera t.ex tidigare versioner som nu inte längre är tillgängliga.

isbn
urn-nbn

Altmetricpoäng

isbn
urn-nbn
Totalt: 732 träffar
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf