Change search
ReferencesLink to record
Permanent link

Direct link
Sparse Kernel Reduced-Rank Regression for Bimodal Emotion Recognition From Facial Expression and Speech
Show others and affiliations
2016 (English)In: IEEE transactions on multimedia, ISSN 1520-9210, E-ISSN 1941-0077, Vol. 18, no 7, 1319-1329 p.Article in journal (Refereed) PublishedText
Abstract [en]

A novel bimodal emotion recognition approach from facial expression and speech based on the sparse kernel reduced-rank regression (SKRRR) fusion method is proposed in this paper. In this method, we use the openSMILE feature extractor and the scale invariant feature transform feature descriptor to respectively extract effective features from speech modality and facial expression modality, and then propose the SKRRR fusion approach to fuse the emotion features of two modalities. The proposed SKRRR method is a nonlinear extension of the traditional reduced-rank regression (RRR), where both predictor and response feature vectors in RRR are kernelized by being mapped onto two high-dimensional feature space via two nonlinear mappings, respectively. To solve the SKRRR problem, we propose a sparse representation (SR)-based approach to find the optimal solution of the coefficient matrices of SKRRR, where the introduction of the SR technique aims to fully consider the different contributions of training data samples to the derivation of optimal solution of SKRRR. Finally, we utilize the eNTERFACE '05 and AFEW4.0 bimodal emotion database to conduct the experiments of monomodal emotion recognition and bimodal emotion recognition, and the results indicate that our presented approach acquires the highest or comparable bimodal emotion recognition rate among some state-of-the-art approaches.

Place, publisher, year, edition, pages
2016. Vol. 18, no 7, 1319-1329 p.
Keyword [en]
Bimodal emotion recognition, facial expression, feature fusion, sparse kernel reduced-rank regression (SKRRR), speech
National Category
Computer and Information Science
URN: urn:nbn:se:kth:diva-190497DOI: 10.1109/TMM.2016.2557721ISI: 000379752600008ScopusID: 2-s2.0-84976556360OAI: diva2:954041

QC 20160819

Available from: 2016-08-19 Created: 2016-08-12 Last updated: 2016-08-19Bibliographically approved

Open Access in DiVA

No full text

Other links

Publisher's full textScopus

Search in DiVA

By author/editor
Li, Haibo
By organisation
Media Technology and Interaction Design, MID
In the same journal
IEEE transactions on multimedia
Computer and Information Science

Search outside of DiVA

GoogleGoogle Scholar
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

Altmetric score

Total: 3 hits
ReferencesLink to record
Permanent link

Direct link