Change search
ReferencesLink to record
Permanent link

Direct link
Fast and memory optimal low-rank matrix approximation
KTH, School of Electrical Engineering (EES), Automatic Control.
2015 (English)In: Advances in Neural Information Processing Systems, Neural information processing systems foundation , 2015, 3177-3185 p.Conference paper (Refereed)
Abstract [en]

In this paper, we revisit the problem of constructing a near-optimal rank k approximation of a matrix M ∈ [0,1]m×n under the streaming data model where the columns of M are revealed sequentially. We present SLA (Streaming Low-rank Approximation), an algorithm that is asymptotically accurate, when ksk+1(M) =o(√mn) where sk+1(M) is the (k + 1)-th largest singular value of M. This means that its average mean-square error converges to 0 as m and n grow large (i.e., || M(k)-M(k)||2 F = o(mn) with high probability, where M(k) and M(k) denote the output of SLA and the optimal rank k approximation of M, respectively). Our algorithm makes one pass on the data if the columns of M are revealed in a random order, and two passes if the columns of M arrive in an arbitrary order. To reduce its memory footprint and complexity, SLA uses random sparsification, and samples each entry of M with a small probability δ. In turn, SLA is memory optimal as its required memory space scales as k(m+n), the dimension of its output. Furthermore, SLA is computationally efficient as it runs in O(δkmn) time (a constant number of operations is made for each observed entry of M), which can be as small as O(k log(m)4n) for an appropriate choice of δ and if n ≥ m.

Place, publisher, year, edition, pages
Neural information processing systems foundation , 2015. 3177-3185 p.
Keyword [en]
Approximation theory, Information science, Matrix algebra, Mean square error, Arbitrary order, Computationally efficient, High probability, Low rank approximations, Low-rank matrix approximations, Memory footprint, Singular values, Sparsification, Approximation algorithms
National Category
Control Engineering
URN: urn:nbn:se:kth:diva-194718ScopusID: 2-s2.0-84965132645OAI: diva2:1048893
29th Annual Conference on Neural Information Processing Systems, NIPS 2015, 7 December 2015 through 12 December 2015

QC 20161122

Available from: 2016-11-22 Created: 2016-10-31 Last updated: 2016-11-22Bibliographically approved

Open Access in DiVA

No full text


Search in DiVA

By author/editor
Proutiere, Alexandre
By organisation
Automatic Control
Control Engineering

Search outside of DiVA

GoogleGoogle Scholar

ReferencesLink to record
Permanent link

Direct link