kth.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Auditory vs. audiovisual prominence ratings of speech involving spontaneously produced head movements
Linnaeus University, Växjö, Sweden.
Lund University Humanities Lab, Sweden.
KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Speech, Music and Hearing, TMH.ORCID iD: 0000-0002-4628-3769
2022 (English)In: Proceedings of the 11th International Conference on Speech Prosody, Speech Prosody 2022, International Speech Communication Association , 2022, p. 352-356Conference paper, Published paper (Refereed)
Abstract [en]

Visual information can be integrated in prominence perception, but most available evidence stems from controlled experimental settings, often involving synthetic stimuli. The present study provides evidence from spontaneously produced head gestures that occurred in Swedish television news readings. Sixteen short clips (containing 218 words in total) were rated for word prominence by 85 adult volunteers in a between-subjects design (44 in an audio-visual vs. 41 in an audio-only condition) using a web-based rating task. As an initial test of overall rating behavior, average prominence across all 218 words was compared between the two conditions, revealing no significant difference. In a second step, we compared normalized prominence ratings between the two conditions for all 218 words individually. These results displayed significant (or near significant, p<.08) differences for 28 out of 218 words, with higher ratings in either the audiovisual (13 words) or the audio-only-condition (15 words). A detailed examination revealed that the presence of head movements (previously annotated) can boost prominence ratings in the audiovisual condition, while words with low prominence tend to be rated slightly higher in the audio-only condition. The study suggests that visual prominence signals are integrated in speech processing even in a relatively uncontrolled, naturalistic setting.

Place, publisher, year, edition, pages
International Speech Communication Association , 2022. p. 352-356
Keywords [en]
beat gesture, head movement, multimodality, pitch accent, prominence perception, visual prosody
National Category
General Language Studies and Linguistics
Identifiers
URN: urn:nbn:se:kth:diva-335753DOI: 10.21437/SpeechProsody.2022-72Scopus ID: 2-s2.0-85147200055OAI: oai:DiVA.org:kth-335753DiVA, id: diva2:1795813
Conference
11th International Conference on Speech Prosody, Speech Prosody 2022, Lisbon, Portugal, May 23 2022 - May 26 2022
Note

QC 20230911

Available from: 2023-09-11 Created: 2023-09-11 Last updated: 2023-09-11Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Authority records

House, David

Search in DiVA

By author/editor
House, David
By organisation
Speech, Music and Hearing, TMH
General Language Studies and Linguistics

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 56 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf