Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Gaussian process dynamical models for nonparametric speech representation and synthesis
KTH, School of Electrical Engineering (EES), Communication Theory.
School of Engineering and Computer Science, Victoria University of Wellington, New Zealand.
KTH, School of Electrical Engineering (EES), Communication Theory.
2012 (English)In: Acoustics, Speech and Signal Processing (ICASSP), 2012 IEEE International Conference on, IEEE , 2012, 4505-4508 p.Conference paper, Published paper (Refereed)
Abstract [en]

We propose Gaussian process dynamical models (GPDMs) as a new, nonparametric paradigm in acoustic models of speech. These use multidimensional, continuous state-spaces to overcome familiar issues with discrete-state, HMM-based speech models. The added dimensions allow the state to represent and describe more than just temporal structure as systematic differences in mean, rather than as mere correlations in a residual (which dynamic features or AR-HMMs do). Being based on Gaussian processes, the models avoid restrictive parametric or linearity assumptions on signal structure. We outline GPDM theory, and describe model setup and initialization schemes relevant to speech applications. Experiments demonstrate subjectively better quality of synthesized speech than from comparable HMMs. In addition, there is evidence for unsupervised discovery of salient speech structure.

Place, publisher, year, edition, pages
IEEE , 2012. 4505-4508 p.
Series
IEEE International Conference on Acoustics, Speech and Signal Processing. Proceedings, ISSN 1520-6149
Keyword [en]
acoustic models, stochastic models, nonparametric speech synthesis, sampling
National Category
Other Electrical Engineering, Electronic Engineering, Information Engineering
Identifiers
URN: urn:nbn:se:kth:diva-66403DOI: 10.1109/ICASSP.2012.6288919ISI: 000312381404144Scopus ID: 2-s2.0-84867596846ISBN: 978-1-4673-0046-9 (print)OAI: oai:DiVA.org:kth-66403DiVA: diva2:483922
Conference
ICASSP 2012, IEEE International Conference on Acoustics, Speech, and Signal Processing, March 25-30, 2012, Kyoto International Conference Center, Kyoto, Japan
Projects
LISTA
Funder
EU, FP7, Seventh Framework Programme, 256230ICT - The Next Generation
Note

QC 20120308

Available from: 2012-03-08 Created: 2012-01-26 Last updated: 2013-11-28Bibliographically approved
In thesis
1. Probabilistic Sequence Models with Speech and Language Applications
Open this publication in new window or tab >>Probabilistic Sequence Models with Speech and Language Applications
2013 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Series data, sequences of measured values, are ubiquitous. Whenever observations are made along a path in space or time, a data sequence results. To comprehend nature and shape it to our will, or to make informed decisions based on what we know, we need methods to make sense of such data. Of particular interest are probabilistic descriptions, which enable us to represent uncertainty and random variation inherent to the world around us.

This thesis presents and expands upon some tools for creating probabilistic models of sequences, with an eye towards applications involving speech and language. Modelling speech and language is not only of use for creating listening, reading, talking, and writing machines---for instance allowing human-friendly interfaces to future computational intelligences and smart devices of today---but probabilistic models may also ultimately tell us something about ourselves and the world we occupy.

The central theme of the thesis is the creation of new or improved models more appropriate for our intended applications, by weakening limiting and questionable assumptions made by standard modelling techniques. One contribution of this thesis examines causal-state splitting reconstruction (CSSR), an algorithm for learning discrete-valued sequence models whose states are minimal sufficient statistics for prediction. Unlike many traditional techniques, CSSR does not require the number of process states to be specified a priori, but builds a pattern vocabulary from data alone, making it applicable for language acquisition and the identification of stochastic grammars. A paper in the thesis shows that CSSR handles noise and errors expected in natural data poorly, but that the learner can be extended in a simple manner to yield more robust and stable results also in the presence of corruptions.

Even when the complexities of language are put aside, challenges remain. The seemingly simple task of accurately describing human speech signals, so that natural synthetic speech can be generated, has proved difficult, as humans are highly attuned to what speech should sound like. Two papers in the thesis therefore study nonparametric techniques suitable for improved acoustic modelling of speech for synthesis applications. Each of the two papers targets a known-incorrect assumption of established methods, based on the hypothesis that nonparametric techniques can better represent and recreate essential characteristics of natural speech.

In the first paper of the pair, Gaussian process dynamical models (GPDMs), nonlinear, continuous state-space dynamical models based on Gaussian processes, are shown to better replicate voiced speech, without traditional dynamical features or assumptions that cepstral parameters follow linear autoregressive processes. Additional dimensions of the state-space are able to represent other salient signal aspects such as prosodic variation. The second paper, meanwhile, introduces KDE-HMMs, asymptotically-consistent Markov models for continuous-valued data based on kernel density estimation, that additionally have been extended with a fixed-cardinality discrete hidden state. This construction is shown to provide improved probabilistic descriptions of nonlinear time series, compared to reference models from different paradigms. The hidden state can be used to control process output, making KDE-HMMs compelling as a probabilistic alternative to hybrid speech-synthesis approaches.

A final paper of the thesis discusses how models can be improved even when one is restricted to a fundamentally imperfect model class. Minimum entropy rate simplification (MERS), an information-theoretic scheme for postprocessing models for generative applications involving both speech and text, is introduced. MERS reduces the entropy rate of a model while remaining as close as possible to the starting model. This is shown to produce simplified models that concentrate on the most common and characteristic behaviours, and provides a continuum of simplifications between the original model and zero-entropy, completely predictable output. As the tails of fitted distributions may be inflated by noise or empirical variability that a model has failed to capture, MERS's ability to concentrate on high-probability output is also demonstrated to be useful for denoising models trained on disturbed data.

Place, publisher, year, edition, pages
Stockholm: KTH Royal Institute of Technology, 2013. xviii, 80 p.
Series
Trita-EE, ISSN 1653-5146 ; 2013:042
Keyword
Time series, acoustic modelling, speech synthesis, stochastic processes, causal-state splitting reconstruction, robust causal states, pattern discovery, Markov models, HMMs, nonparametric models, Gaussian processes, Gaussian process dynamical models, nonlinear Kalman filters, information theory, minimum entropy rate simplification, kernel density estimation, time-series bootstrap
National Category
Electrical Engineering, Electronic Engineering, Information Engineering
Identifiers
urn:nbn:se:kth:diva-134693 (URN)978-91-7501-932-1 (ISBN)
Public defence
2013-12-17, D3, Lindstedtsvägen 5, KTH, Stockholm, 09:00 (English)
Opponent
Supervisors
Projects
ACORNS: Acquisition of Communication and Recognition SkillsLISTA – The Listening Talker
Funder
EU, European Research Council, FP6-034362EU, FP7, Seventh Framework Programme, 256230
Note

QC 20131128

Available from: 2013-11-28 Created: 2013-11-27 Last updated: 2013-11-28Bibliographically approved

Open Access in DiVA

gpdm_speech_synthesis.pdf(363 kB)204 downloads
File information
File name FULLTEXT01.pdfFile size 363 kBChecksum SHA-512
4468c05eab5b9aac0ca1ec1de2db11c3c47fe22a7a28e89ec8a28f38084c4565a3413c2eb4f14093032de06055e515e4892a5b55ac40ae2f57de7d413e0cbe6a
Type fulltextMimetype application/pdf

Other links

Publisher's full textScopusConference website

Search in DiVA

By author/editor
Henter, Gustav EjeKleijn, W. Bastiaan
By organisation
Communication Theory
Other Electrical Engineering, Electronic Engineering, Information Engineering

Search outside of DiVA

GoogleGoogle Scholar
Total: 204 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

doi
isbn
urn-nbn

Altmetric score

doi
isbn
urn-nbn
Total: 170 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf