Private Stochastic Dual Averaging for Decentralized Empirical Risk Minimization
2022 (English)In: 9th IFAC Conference on Networked Systems NECSYS 2022Zürich, Switzerland, 5–7 July 2022, Elsevier BV , 2022, Vol. 55, no 13, p. 43-48Conference paper, Published paper (Refereed)
Abstract [en]
In this work, we study the decentralized empirical risk minimization problem under the constraint of differential privacy (DP). Based on the algorithmic framework of dual averaging, we develop a novel decentralized stochastic optimization algorithm to solve the problem. The proposed algorithm features the following: i) it perturbs the stochastic subgradient evaluated over individual data samples, with which the information about the dataset can be released in a differentially private manner; ii) it employs hyperparameters that are more aggressive than conventional decentralized dual averaging algorithms to speed up convergence. The upper bound for the utility loss of the proposed algorithm is proven to be smaller than that of existing methods to achieve the same level of DP. As a by-product, when removing the perturbation, the non-private version of the proposed algorithm attains the optimal O(1/t) convergence rate for smooth stochastic optimization. Finally, experimental results are presented to demonstrate the effectiveness of the algorithm.
Place, publisher, year, edition, pages
Elsevier BV , 2022. Vol. 55, no 13, p. 43-48
Series
IFAC-PapersOnLine, ISSN 2405-8963 ; 55
Keywords [en]
Dual averaging, differential privacy, distributed optimization, convex optimization, large scale optimization problems
National Category
Other Engineering and Technologies
Identifiers
URN: urn:nbn:se:kth:diva-319443DOI: 10.1016/j.ifacol.2022.07.233ISI: 000852734000008Scopus ID: 2-s2.0-85137167981OAI: oai:DiVA.org:kth-319443DiVA, id: diva2:1699966
Conference
9th IFAC Conference on Networked Systems, NECSYS 2022, Zurich, 5 July 2022, through 7 July 2022
Note
QC 20220929
2022-09-292022-09-292025-02-10Bibliographically approved