kth.sePublications KTH
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Distributed Learning Based on 1-Bit Gradient Coding in the Presence of Stragglers
KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Information Science and Engineering.ORCID iD: 0000-0003-1649-1943
KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Information Science and Engineering.ORCID iD: 0000-0002-7926-5081
2024 (English)In: IEEE Transactions on Communications, ISSN 0090-6778, E-ISSN 1558-0857, Vol. 72, no 8, p. 4903-4916Article in journal (Refereed) Published
Abstract [en]

This paper considers the problem of distributed learning (DL) in the presence of stragglers. For this problem, DL methods based on gradient coding have been widely investigated, which redundantly distribute the training data to the workers to guarantee convergence when some workers are stragglers. However, these methods require the workers to transmit real-valued vectors during the process of learning, which induces very high communication burden. To overcome this drawback, we propose a novel DL method based on 1-bit gradient coding (1-bit GC-DL), where 1-bit data encoded from the locally computed gradients are transmitted by the workers to reduce the communication overhead. We theoretically provide the convergence guarantees of the proposed method for both the convex loss functions and non-convex loss functions. It is shown empirically that 1-bit GC-DL outperforms the baseline methods, which attains better learning performance under the same communication overhead.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE) , 2024. Vol. 72, no 8, p. 4903-4916
Keywords [en]
Vectors, Quantization (signal), Convergence, Training data, Training, Encoding, Costs, Distributed learning, 1-bit quantization, stragglers, communication overhead, convergence analysis
National Category
Communication Systems
Identifiers
URN: urn:nbn:se:kth:diva-354084DOI: 10.1109/TCOMM.2024.3377715ISI: 001294594400036Scopus ID: 2-s2.0-85188541783OAI: oai:DiVA.org:kth-354084DiVA, id: diva2:1903411
Note

QC 20241004

Available from: 2024-10-04 Created: 2024-10-04 Last updated: 2024-10-04Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Authority records

Li, ChengxiSkoglund, Mikael

Search in DiVA

By author/editor
Li, ChengxiSkoglund, Mikael
By organisation
Information Science and Engineering
In the same journal
IEEE Transactions on Communications
Communication Systems

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 114 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf