Improved Dysarthric Speech to Text Conversion via TTS PersonalizationVise andre og tillknytning
2025 (engelsk)Inngår i: 2025 33rd European Signal Processing Conference, EUSIPCO 2025 - Proceedings, Institute of Electrical and Electronics Engineers (IEEE) , 2025, s. 521-525Konferansepaper, Publicerat paper (Fagfellevurdert)
Abstract [en]
We present a case study on developing a customized speech-to-text system for a Hungarian speaker with severe dysarthria. State-of-the-art automatic speech recognition (ASR) models struggle with zero-shot transcription of dysarthric speech, yielding high error rates. To improve performance with limited real dysarthric data, we fine-tune an ASR model using synthetic speech generated via a personalized text-to-speech (TTS) system. We introduce a method for generating synthetic dysarthric speech with controlled severity by leveraging premorbidity recordings of the given speaker and speaker embedding interpolation, enabling ASR fine-tuning on a continuum of impairments. Fine-tuning on both real and synthetic dysarthric speech reduces the character error rate (CER) from 36-51% (zero-shot) to 7.3%. Our monolingual FastConformer_Hu ASR model significantly outperforms Whisper-turbo when fine-tuned on the same data, and the inclusion of synthetic speech contributes to an 18% relative CER reduction. These results highlight the potential of personalized ASR systems for improving accessibility for individuals with severe speech impairments.
sted, utgiver, år, opplag, sider
Institute of Electrical and Electronics Engineers (IEEE) , 2025. s. 521-525
Emneord [en]
automatic speech recognition, dysarthric speech, few-shot learning, text to speech synthesis
HSV kategori
Identifikatorer
URN: urn:nbn:se:kth:diva-385592DOI: 10.23919/EUSIPCO63237.2025.11226704Scopus ID: 2-s2.0-105029819672OAI: oai:DiVA.org:kth-385592DiVA, id: diva2:2086794
Konferanse
33rd European Signal Processing Conference, EUSIPCO 2025, Palermo, Italy, September 8-12, 2025
Merknad
Part of ISBN 9789464593624
QC 20260716
2026-07-162026-07-162026-11-13bibliografisk kontrollert