Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Colour Association with Music is Mediated by Emotion: Evidence from an Experiment using a CIE Lab Interface and Interviews
KTH, School of Computer Science and Communication (CSC), Speech, Music and Hearing, TMH, Music Acoustics. Nanyang Technological University, Singapore.ORCID iD: 0000-0001-8121-9263
KTH, School of Computer Science and Communication (CSC), Speech, Music and Hearing, TMH, Music Acoustics.ORCID iD: 0000-0003-2926-6518
2015 (English)In: PLoS ONE, ISSN 1932-6203, E-ISSN 1932-6203, Vol. 10, no 12, e0144013Article in journal (Refereed) Published
Abstract [en]

Crossmodal associations may arise at neurological, perceptual, cognitive, or emotional levels of brain processing. Higher-level modal correspondences between musical timbre and visual colour have been previously investigated, though with limited sets of colour. We developed a novel response method that employs a tablet interface to navigate the CIE Lab colour space. The method was used in an experiment where 27 film music excerpts were presented to participants (n = 22) who continuously manipulated the colour and size of an on-screen patch to match the music. Analysis of the data replicated and extended earlier research, for example, that happy music was associated with yellow, music expressing anger with large red colour patches, and sad music with smaller patches towards dark blue. Correlation analysis suggested patterns of relationships between audio features and colour patch parameters. Using partial least squares regression, we tested models for predicting colour patch responses from audio features and ratings of perceived emotion in the music. Parsimonious models that included emotion robustly explained between 60% and 75% of the variation in each of the colour patch parameters, as measured by cross-validated R2. To illuminate the quantitative findings, we performed a content analysis of structured spoken interviews with the participants. This provided further evidence of a significant emotion mediation mechanism, whereby people tended to match colour association with the perceived emotion in the music. The mixed method approach of our study gives strong evidence that emotion can mediate crossmodal association between music and visual colour. The CIE Lab interface promises to be a useful tool in perceptual ratings of music and other sounds.

Place, publisher, year, edition, pages
PLOS , 2015. Vol. 10, no 12, e0144013
Keyword [en]
crossmodal association; perception; emotion; music; colour, CIE Lab
National Category
Other Engineering and Technologies not elsewhere specified Music Applied Psychology
Research subject
Art, Technology and Design
Identifiers
URN: urn:nbn:se:kth:diva-177108DOI: 10.1371/journal.pone.0144013ISI: 000366902700044Scopus ID: 2-s2.0-84955589525OAI: oai:DiVA.org:kth-177108DiVA: diva2:871801
Note

QC 20160121

Available from: 2015-11-17 Created: 2015-11-14 Last updated: 2017-12-01Bibliographically approved
In thesis
1. Sound perception and design in multimodal environments
Open this publication in new window or tab >>Sound perception and design in multimodal environments
2015 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

This dissertation is about sound in context. Since sensory processing is inherently multimodal, research in sound is necessarily multidisciplinary. The present work has been guided by principles of systematicity, ecological validity, complementarity of  methods, and integration of science and art. The main tools to investigate the mediating relationship of people and environment through sound have been empiricism and psychophysics. Four of the seven included papers focus on perception. In paper A, urban soundscapes were reproduced in a 3D installation. Analysis of results from an experiment revealed correlations between acoustic features and physiological indicators of stress and relaxation. Paper B evaluated soundscapes of different type. Perceived quality was predicted not only by psychoacoustic descriptors but also personality traits. Sound reproduction quality was manipulated in paper D, causing two effects on source localisation which were explained by spatial and semantic crossmodal correspondences. Crossmodal correspondence was central in paper C, a study of colour association with music. A response interface employing CIE Lab colour space, a novelty in music emotion research, was developed. A mixed method approach supported an emotion mediation hypothesis, evidenced in regression models and participant interviews. Three papers focus on design. Field surveys and acoustic measurements were carried out in restaurants. Paper E charted relations between acoustic, physical, and perceptual features, focussing on designable elements and materials. This investigation was pursued in Paper F where a taxonomy of sound sources was developed. Analysis of questionnaire data revealed perceptual and crossmodal effects. Lastly, paper G discussed how crossmodal correspondences facilitated creation of meaning in music by infusing ecologically founded sonification parameters with visual and spatial metaphors. The seven papers constitute an investigation into how sound affects us, and what sound means to us.

Abstract [sv]

Denna doktorsavhandling handlar om ljud i sammanhang. Eftersom informationsbehandling genom sinnena alltid är multimodal så kräver ljudforskning en tvärvetenskaplig forskningsansats. Arbetet i denna avhandling har vägletts av principer såsom systematik, ekologisk validitet, samspel mellan metoder, och integration av vetenskap och konst. De viktigaste redskapen för att undersöka den ömsesidiga påverkan mellan människa och miljö genom ljud har varit empiri och psykofysik.Fyra artiklar handlar om perception. I artikel A återskapades urbana ljudlandskap i en 3D-ljudinstallation. Analys av experimentresultat avslöjade samband mellan akustiska mått och fysiologiska markörer av stress och avslappning.  Artikel B utvärderade olika typer av ljudlandskap. Upplevd kvalitet kunde prediceras inte bara av psykoakustiska mått utan även av personlighetsdrag. Ljudåtergivningskvalitet manipulerades i artikel D och orsakade två effekter på lokalisering av en ljudkälla vilka förklarades av rumslig och semantisk korsmodala kopplingar. Korsmodalitet var huvudpunkten i artikel C, en studie av färgassociation till musik. Ett användargränssnitt utvecklades som använder färgrymden CIE Lab, en nyhet i forskningfältet musik och känslor. En abduktiv metod stödde hypotesen att känslouttryck medierar korsmodala kopplingar, vilket framgick av regressionsmodeller och intervjuer med försökspersonerna.Tre artiklar handlar om design. Fältundersökningar och ljudmätningar utfördes i restauranger. Artikel E kartlade samband mellan akustiska, fysiska och perceptuella särdrag, med fokus på formbara element och material. Detta arbete fortsattes i artikel F varigenom en taxonomi av ljudkällor utvecklades. Analys av enkätdata avslöjade perceptuella och korsmodala effekter. Slutligen, artikel G diskuterade hur korsmodala kopplingar främjade meningsskapande i musik genom att ekologiskt motiverade sonifikationsparametrar samverkade med visuella och spatiala uttryck. De sju artiklarna utgör landmärken i avhandlingens utforskande av hur ljud påverkar oss, och vad ljud betyder för oss.

Place, publisher, year, edition, pages
Stockholm: KTH Royal Institute of Technology, 2015. 67 p.
Series
TRITA-CSC-A, ISSN 1653-5723 ; 2015:17
Keyword
sound, perception, design, multimodal, environment, soundscape, music, listening, crossmodal, psychoacoustics, psychophysiology, personality trait, emotion, appraisal, aesthetics, colour, correlation, regression, classification, sonification
National Category
Other Electrical Engineering, Electronic Engineering, Information Engineering Interaction Technologies Music Other Physics Topics Applied Psychology
Research subject
Speech and Music Communication
Identifiers
urn:nbn:se:kth:diva-177271 (URN)978-91-7595-771-5 (ISBN)
Public defence
2015-12-11, Kollegiesalen, Brinellvägen 8, KTH, Stockholm, 10:00 (English)
Opponent
Supervisors
Note

QC 20151118

Available from: 2015-11-18 Created: 2015-11-17 Last updated: 2015-11-19Bibliographically approved

Open Access in DiVA

LIndborg+Friberg.2015.PLoSone(8198 kB)127 downloads
File information
File name FULLTEXT01.pdfFile size 8198 kBChecksum SHA-512
9e37dfc60fbd246369907b4a5148753f2110a622f3f4968760ef477fa0fcecbd0f493ad63cda22ca804b30f46f90555934b12433e1f0b28c7ba956a285b5e00d
Type fulltextMimetype application/pdf

Other links

Publisher's full textScopus

Authority records BETA

Lindborg, PerMagnusFriberg, Anders K

Search in DiVA

By author/editor
Lindborg, PerMagnusFriberg, Anders K
By organisation
Music Acoustics
In the same journal
PLoS ONE
Other Engineering and Technologies not elsewhere specifiedMusicApplied Psychology

Search outside of DiVA

GoogleGoogle Scholar
Total: 127 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 279 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf