Neural Network based Explicit Mixture Models and Expectation-maximization based Learning
2020 (English)In: Proceedings of the International Joint Conference on Neural Networks, Institute of Electrical and Electronics Engineers Inc. , 2020Conference paper, Published paper (Refereed)
Abstract [en]
We propose two neural network based mixture models in this work. The proposed mixture models are explicit. The explicit models have analytical forms with the advantages of computing likelihood and efficiency of generating samples. Expectation-maximization based algorithms are developed for learning parameters of the proposed models. We provide sufficient conditions to realize the expectation-maximization based learning. The main requirements are invertibility of neural networks that are used as generators and Jacobian computation of functional form of the neural networks. The requirements are practically realized using a flow-based neural network. In our first mixture model, we use multiple flow-based neural networks as generators. Naturally the model is complex. A single latent variable is used as the common input to all the neural networks. The second mixture model uses a single flow-based neural network as a generator to reduce complexity. The single generator has a latent variable input that follows a Gaussian mixture distribution. The proposed models are verified via training with expectation-maximization based algorithms on practical datasets. We demonstrate efficiency of proposed mixture models through extensive experiments for generating samples and maximum likelihood based classification.
Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers Inc. , 2020.
Keywords [en]
classification, expectation maximization, Generative model, mixture models, neural network, Complex networks, Efficiency, Learning systems, Maximum likelihood, Maximum principle, Analytical forms, Expectation - maximizations, Explicit models, Functional forms, Gaussian mixture distribution, Generating samples, Latent variable, Learning parameters, Neural networks
National Category
Control Engineering
Identifiers
URN: urn:nbn:se:kth:diva-291297DOI: 10.1109/IJCNN48605.2020.9207086ISI: 000626021403121Scopus ID: 2-s2.0-85093853732OAI: oai:DiVA.org:kth-291297DiVA, id: diva2:1538888
Conference
2020 International Joint Conference on Neural Networks, IJCNN 2020, 19 July 2020 through 24 July 2020
Note
QC 20210322QC 20210721
2021-03-222021-03-222023-04-03Bibliographically approved