Change search
ReferencesLink to record
Permanent link

Direct link
A delayed proximal gradient method with linear convergence rate
KTH, School of Electrical Engineering (EES), Automatic Control.
KTH, School of Electrical Engineering (EES), Automatic Control.ORCID iD: 0000-0003-1725-2901
KTH, School of Electrical Engineering (EES), Automatic Control.
2014 (English)In: IEEE International Workshop on Machine Learning for Signal Processing (MLSP), IEEE conference proceedings, 2014, 1-6 p.Conference paper (Refereed)
Abstract [en]

This paper presents a new incremental gradient algorithm for minimizing the average of a large number of smooth component functions based on delayed partial gradients. Even with a constant step size, which can be chosen independently of the maximum delay bound and the number of objective function components, the expected objective value is guaranteed to converge linearly to within some ball around the optimum. We derive an explicit expression that quantifies how the convergence rate depends on objective function properties and algorithm parameters such as step-size and the maximum delay. An associated upper bound on the asymptotic error reveals the trade-off between convergence speed and residual error. Numerical examples confirm the validity of our results.

Place, publisher, year, edition, pages
IEEE conference proceedings, 2014. 1-6 p.
Keyword [en]
convergence, gradient methods, learning (artificial intelligence), algorithm parameter, asymptotic error, constant step size, convergence speed, delayed partial gradient, delayed proximal gradient method, incremental gradient algorithm, linear convergence rate, objective function component, objective function property, residual error, smooth component function, Convergence, Delays, Linear programming, Optimization, Radio frequency, Upper bound, Vectors, Incremental gradient, asynchronous parallelism, machine learning
National Category
Control Engineering
URN: urn:nbn:se:kth:diva-164290DOI: 10.1109/MLSP.2014.6958872ScopusID: 2-s2.0-84912569451OAI: diva2:805303
IEEE International Workshop on Machine Learning for Signal Processing (MLSP),21-24 Sept. 2014, Reims, France

QC 20150511

Available from: 2015-04-15 Created: 2015-04-15 Last updated: 2015-05-11Bibliographically approved

Open Access in DiVA

fulltext(346 kB)65 downloads
File information
File name FULLTEXT01.pdfFile size 346 kBChecksum SHA-512
Type fulltextMimetype application/pdf

Other links

Publisher's full textScopusIEEEXplore

Search in DiVA

By author/editor
Feyzmahdavian, Hamid RezaAytekin, ArdaJohansson, Mikael
By organisation
Automatic Control
Control Engineering

Search outside of DiVA

GoogleGoogle Scholar
Total: 65 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

Altmetric score

Total: 78 hits
ReferencesLink to record
Permanent link

Direct link