Change search
ReferencesLink to record
Permanent link

Direct link
Covariance structure approximation via gLasso in high-dimensional supervised classification
KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Mathematical Statistics.
Stockholm Univ, Stockholm, Sweden.
Stockholm Univ, Stockholm, Sweden.
2012 (English)In: Journal of Applied Statistics, ISSN 0266-4763, E-ISSN 1360-0532, Vol. 39, no 8, 1643-1666 p.Article in journal (Refereed) Published
Abstract [en]

Recent work has shown that the Lasso-based regularization is very useful for estimating the high-dimensional inverse covariance matrix. A particularly useful scheme is based on penalizing the l(1) norm of the off-diagonal elements to encourage sparsity. We embed this type of regularization into high-dimensional classification. A two-stage estimation procedure is proposed which first recovers structural zeros of the inverse covariance matrix and then enforces block sparsity by moving non-zeros closer to the main diagonal. We show that the block-diagonal approximation of the inverse covariance matrix leads to an additive classifier, and demonstrate that accounting for the structure can yield better performance accuracy. Effect of the block size on classification is explored, and a class of as ymptotically equivalent structure approximations in a high-dimensional setting is specified. We suggest a variable selection at the block level and investigate properties of this procedure in growing dimension asymptotics. We present a consistency result on the feature selection procedure, establish asymptotic lower an upper bounds for the fraction of separative blocks and specify constraints under which the reliable classification with block-wise feature selection can be performed. The relevance and benefits of the proposed approach are illustrated on both simulated and real data.

Place, publisher, year, edition, pages
2012. Vol. 39, no 8, 1643-1666 p.
Keyword [en]
high dimensionality; classification accuracy; sparsity; block-diagonal covariance structure; graphical Lasso; separation strength
National Category
Probability Theory and Statistics
URN: urn:nbn:se:kth:diva-75454DOI: 10.1080/02664763.2012.663346ISI: 000305486300002ScopusID: 2-s2.0-84862594316OAI: diva2:490500
Swedish Research Council, 421-2008-1966
QC 20120717Available from: 2012-02-05 Created: 2012-02-05 Last updated: 2012-07-17Bibliographically approved

Open Access in DiVA

No full text

Other links

Publisher's full textScopus

Search in DiVA

By author/editor
Pavlenko, Tatjana
By organisation
Mathematical Statistics
In the same journal
Journal of Applied Statistics
Probability Theory and Statistics

Search outside of DiVA

GoogleGoogle Scholar
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

Altmetric score

Total: 47 hits
ReferencesLink to record
Permanent link

Direct link