kth.sePublikationer
Ändra sökning
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf
Approximating the predictive distribution of the beta distribution with the local variational method
KTH, Skolan för elektro- och systemteknik (EES), Ljud- och bildbehandling.
KTH, Skolan för elektro- och systemteknik (EES), Ljud- och bildbehandling.ORCID-id: 0000-0001-7957-5103
2011 (Engelska)Ingår i: IEEE Intl. Workshop on Machine Learning for Signal Processing, 2011Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

In the Bayesian framework, the predictive distribution is obtained by averaging over the posterior parameter distribution. When there is a small amount of data, the uncertainty of the parameters is high. Thus with the predictive distribution, a more reliable result can be obtained in the applications as classification, recognition, etc. In the previous works, we have utilized the variational inference framework to approximate the posterior distribution of the parameters in the beta distribution by minimizing the Kullback-Leibler divergence of the true posterior distribution from the approximating one. However, the predictive distribution of the beta distribution was approximated by a plug-in approximation with the posterior mean, regardless of the parameter uncertainty. In this paper, we carry on the factorized approximation introduced in the previous work and approximate the beta function by its first order Taylor expansion. Then the upper bound of the predictive distribution is derived by exploiting the local variational method. By minimizing the upper bound of the predictive distribution and after normalization, we approximate the predictive distribution by a probability density function in a closed form. Experimental results shows the accuracy and efficiency of the proposed approximation method.

Ort, förlag, år, upplaga, sidor
2011.
Nyckelord [en]
Bayesian Estimation, Beta Distribution, Local Variational Method, Predictive Distribution
Nationell ämneskategori
Elektroteknik och elektronik Data- och informationsvetenskap
Identifikatorer
URN: urn:nbn:se:kth:diva-47401DOI: 10.1109/MLSP.2011.6064567ISI: 000298259900022Scopus ID: 2-s2.0-82455198854ISBN: 978-1-4577-1623-2 (tryckt)OAI: oai:DiVA.org:kth-47401DiVA, id: diva2:454927
Konferens
21st IEEE International Workshop on Machine Learning for Signal Processing (MLSP)
Anmärkning
Trita-EE 2011:28. QC 20120403Tillgänglig från: 2011-11-08 Skapad: 2011-11-08 Senast uppdaterad: 2024-03-18Bibliografiskt granskad
Ingår i avhandling
1. Non-Gaussian Statistical Modelsand Their Applications
Öppna denna publikation i ny flik eller fönster >>Non-Gaussian Statistical Modelsand Their Applications
2011 (Engelska)Doktorsavhandling, sammanläggning (Övrigt vetenskapligt)
Abstract [en]

Statistical modeling plays an important role in various research areas. It provides away to connect the data with the statistics. Based on the statistical properties of theobserved data, an appropriate model can be chosen that leads to a promising practicalperformance. The Gaussian distribution is the most popular and dominant probabilitydistribution used in statistics, since it has an analytically tractable Probability DensityFunction (PDF) and analysis based on it can be derived in an explicit form. However,various data in real applications have bounded support or semi-bounded support. As the support of the Gaussian distribution is unbounded, such type of data is obviously notGaussian distributed. Thus we can apply some non-Gaussian distributions, e.g., the betadistribution, the Dirichlet distribution, to model the distribution of this type of data.The choice of a suitable distribution is favorable for modeling efficiency. Furthermore,the practical performance based on the statistical model can also be improved by a bettermodeling.

An essential part in statistical modeling is to estimate the values of the parametersin the distribution or to estimate the distribution of the parameters, if we consider themas random variables. Unlike the Gaussian distribution or the corresponding GaussianMixture Model (GMM), a non-Gaussian distribution or a mixture of non-Gaussian dis-tributions does not have an analytically tractable solution, in general. In this dissertation,we study several estimation methods for the non-Gaussian distributions. For the Maxi-mum Likelihood (ML) estimation, a numerical method is utilized to search for the optimalsolution in the estimation of Dirichlet Mixture Model (DMM). For the Bayesian analysis,we utilize some approximations to derive an analytically tractable solution to approxi-mate the distribution of the parameters. The Variational Inference (VI) framework basedmethod has been shown to be efficient for approximating the parameter distribution byseveral researchers. Under this framework, we adapt the conventional Factorized Approx-imation (FA) method to the Extended Factorized Approximation (EFA) method and useit to approximate the parameter distribution in the beta distribution. Also, the LocalVariational Inference (LVI) method is applied to approximate the predictive distributionof the beta distribution. Finally, by assigning a beta distribution to each element in thematrix, we proposed a variational Bayesian Nonnegative Matrix Factorization (NMF) forbounded support data.

The performances of the proposed non-Gaussian model based methods are evaluatedby several experiments. The beta distribution and the Dirichlet distribution are appliedto model the Line Spectral Frequency (LSF) representation of the Linear Prediction (LP)model for statistical model based speech coding. For some image processing applications,the beta distribution is also applied. The proposed beta distribution based variationalBayesian NMF is applied for image restoration and collaborative filtering. Comparedto some conventional statistical model based methods, the non-Gaussian model basedmethods show a promising improvement.

Ort, förlag, år, upplaga, sidor
Stockholm: KTH Royal Institute of Technology, 2011. s. xii, 49
Serie
Trita-EE, ISSN 1653-5146
Nationell ämneskategori
Telekommunikation Elektroteknik och elektronik Data- och informationsvetenskap
Identifikatorer
urn:nbn:se:kth:diva-47408 (URN)978-91-7501-158-5 (ISBN)
Disputation
2011-12-05, E1, Lindstedsvägen 3, KTH, Stockholm, 09:00 (Engelska)
Opponent
Handledare
Anmärkning
QC 20111115Tillgänglig från: 2011-11-15 Skapad: 2011-11-08 Senast uppdaterad: 2024-01-08Bibliografiskt granskad

Open Access i DiVA

Fulltext saknas i DiVA

Övriga länkar

Förlagets fulltextScopus

Person

Leijon, Arne

Sök vidare i DiVA

Av författaren/redaktören
Ma, ZhanyuLeijon, Arne
Av organisationen
Ljud- och bildbehandling
Elektroteknik och elektronikData- och informationsvetenskap

Sök vidare utanför DiVA

GoogleGoogle Scholar

doi
isbn
urn-nbn

Altmetricpoäng

doi
isbn
urn-nbn
Totalt: 92 träffar
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf