kth.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Asynchronous Iterations in Optimization: New Sequence Results and Sharper Algorithmic Guarantees
ABB Corp Res, Västerås, Sweden..
KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Decision and Control Systems (Automatic Control).ORCID iD: 0000-0002-2237-2580
2023 (English)In: Journal of machine learning research, ISSN 1532-4435, E-ISSN 1533-7928, Vol. 24, article id 158Article in journal (Refereed) Published
Abstract [en]

We introduce novel convergence results for asynchronous iterations that appear in the analysis of parallel and distributed optimization algorithms. The results are simple to apply and give explicit estimates for how the degree of asynchrony impacts the convergence rates of the iterates. Our results shorten, streamline and strengthen existing convergence proofs for several asynchronous optimization methods and allow us to establish convergence guarantees for popular algorithms that were thus far lacking a complete theoretical under-standing. Specifically, we use our results to derive better iteration complexity bounds for proximal incremental aggregated gradient methods, to obtain tighter guarantees depending on the average rather than maximum delay for the asynchronous stochastic gradient descent method, to provide less conservative analyses of the speedup conditions for asynchronous block-co ordinate implementations of Krasnosel'skii-Mann iterations, and to quantify the convergence rates for totally asynchronous iterations under various assumptions on communication delays and update rates.

Place, publisher, year, edition, pages
MICROTOME PUBL , 2023. Vol. 24, article id 158
Keywords [en]
asynchronous algorithms, parallel methods, incremental methods, coordinate descent, stochastic gradient descent
National Category
Control Engineering
Identifiers
URN: urn:nbn:se:kth:diva-340879ISI: 001111697000001OAI: oai:DiVA.org:kth-340879DiVA, id: diva2:1819842
Note

QC 20231215

Available from: 2023-12-15 Created: 2023-12-15 Last updated: 2023-12-15Bibliographically approved

Open Access in DiVA

No full text in DiVA

Authority records

Johansson, Mikael

Search in DiVA

By author/editor
Johansson, Mikael
By organisation
Decision and Control Systems (Automatic Control)
In the same journal
Journal of machine learning research
Control Engineering

Search outside of DiVA

GoogleGoogle Scholar

urn-nbn

Altmetric score

urn-nbn
Total: 23 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf