Endre søk
RefereraExporteraLink to record
Permanent link

Direct link
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annet format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annet språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf
Improved Dysarthric Speech to Text Conversion via TTS Personalization
Hungarian Research Centre for Linguistics, HUN-REN, Hungary; Department of Telecommunications and Artificial Intelligence, Budapest University of Technology, Hungary.
KTH, Skolan för elektroteknik och datavetenskap (EECS), Intelligenta system, Tal, musik och hörsel, TMH.ORCID-id: 0000-0003-1175-840X
Department of Telecommunications and Artificial Intelligence, Budapest University of Technology, Hungary.
SpeechTex Ltd., Hungary; Hungarian Research Centre for Linguistics, HUN-REN, Hungary.
Vise andre og tillknytning
2025 (engelsk)Inngår i: 2025 33rd European Signal Processing Conference, EUSIPCO 2025 - Proceedings, Institute of Electrical and Electronics Engineers (IEEE) , 2025, s. 521-525Konferansepaper, Publicerat paper (Fagfellevurdert)
Abstract [en]

We present a case study on developing a customized speech-to-text system for a Hungarian speaker with severe dysarthria. State-of-the-art automatic speech recognition (ASR) models struggle with zero-shot transcription of dysarthric speech, yielding high error rates. To improve performance with limited real dysarthric data, we fine-tune an ASR model using synthetic speech generated via a personalized text-to-speech (TTS) system. We introduce a method for generating synthetic dysarthric speech with controlled severity by leveraging premorbidity recordings of the given speaker and speaker embedding interpolation, enabling ASR fine-tuning on a continuum of impairments. Fine-tuning on both real and synthetic dysarthric speech reduces the character error rate (CER) from 36-51% (zero-shot) to 7.3%. Our monolingual FastConformer_Hu ASR model significantly outperforms Whisper-turbo when fine-tuned on the same data, and the inclusion of synthetic speech contributes to an 18% relative CER reduction. These results highlight the potential of personalized ASR systems for improving accessibility for individuals with severe speech impairments.

sted, utgiver, år, opplag, sider
Institute of Electrical and Electronics Engineers (IEEE) , 2025. s. 521-525
Emneord [en]
automatic speech recognition, dysarthric speech, few-shot learning, text to speech synthesis
HSV kategori
Identifikatorer
URN: urn:nbn:se:kth:diva-385592DOI: 10.23919/EUSIPCO63237.2025.11226704Scopus ID: 2-s2.0-105029819672OAI: oai:DiVA.org:kth-385592DiVA, id: diva2:2086794
Konferanse
33rd European Signal Processing Conference, EUSIPCO 2025, Palermo, Italy, September 8-12, 2025
Merknad

Part of ISBN 9789464593624

QC 20260716

Tilgjengelig fra: 2026-07-16 Laget: 2026-07-16 Sist oppdatert: 2026-11-13bibliografisk kontrollert

Open Access i DiVA

fulltext(214 kB)1 nedlastinger
Filinformasjon
Fil FULLTEXT01.pdfFilstørrelse 214 kBChecksum SHA-512
e7b1ec2473b6868fdbb11c4329985336198ddb4045b183c4a9ef076a00f1357932c2fa4472a206a81335e3e9b45a1497215219ac6c604351c368deee294fda7f
Type fulltextMimetype application/pdf

Andre lenker

Forlagets fulltekstScopus

Person

Székely, Éva

Søk i DiVA

Av forfatter/redaktør
Székely, Éva
Av organisasjonen

Søk utenfor DiVA

GoogleGoogle Scholar
Totalt: 1 nedlastinger
Antall nedlastinger er summen av alle nedlastinger av alle fulltekster. Det kan for eksempel være tidligere versjoner som er ikke lenger tilgjengelige

doi
urn-nbn

Altmetric

doi
urn-nbn
Totalt: 22 treff
RefereraExporteraLink to record
Permanent link

Direct link
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annet format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annet språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf