Change search
ReferencesLink to record
Permanent link

Direct link
An Asynchronous Mini-Batch Algorithm for Regularized Stochastic Optimization
KTH, School of Electrical Engineering (EES), Automatic Control.ORCID iD: 0000-0003-1149-4715
KTH, School of Electrical Engineering (EES), Automatic Control.ORCID iD: 0000-0003-1725-2901
KTH, School of Electrical Engineering (EES), Automatic Control.
2015 (English)In: 2015 54th IEEE Conference on Decision and Control (CDC), IEEE , 2015, 1384-1389 p.Conference paper (Refereed)
Abstract [en]

Mini-batch optimization has proven to be a powerful paradigm for large-scale learning. However, the state of the art mini-batch algorithms assume synchronous operation or cyclic update orders. When worker nodes are heterogeneous (due to different computational capabilities, or different communication delays), synchronous and cyclic operations are inefficient since they will leave workers idle waiting for the slower nodes to complete their work. We propose an asynchronous mini-batch algorithm for regularized stochastic optimization problems that eliminates idle waiting and allows workers to run at their maximal update rates. We show that the time necessary to compute an ϵ-optimal solution is asymptotically O(1/ϵ2), and the algorithm enjoys near-linear speedup if the number of workers is O(1/√ϵ). Theoretical results are confirmed in real implementations on a distributed computing infrastructure.

Place, publisher, year, edition, pages
IEEE , 2015. 1384-1389 p.
Keyword [en]
Optimization, Convex, Asynchronous optimization, Delay, Stochastic optimization
National Category
Engineering and Technology
Identifiers
URN: urn:nbn:se:kth:diva-183457DOI: 10.1109/CDC.2015.7402404ScopusID: 2-s2.0-84962032572ISBN: 978-1-4799-7884-7OAI: oai:DiVA.org:kth-183457DiVA: diva2:911486
Conference
CDC 2015
Note

QC 20160314

Available from: 2016-03-12 Created: 2016-03-12 Last updated: 2016-03-14Bibliographically approved

Open Access in DiVA

No full text

Other links

Publisher's full textScopus

Search in DiVA

By author/editor
Feyzmahdavian, Hamid RezaAytekin, ArdaJohansson, Mikael
By organisation
Automatic Control
Engineering and Technology

Search outside of DiVA

GoogleGoogle Scholar
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

Altmetric score

Total: 23 hits
ReferencesLink to record
Permanent link

Direct link