kth.sePublications
Change search
Link to record
Permanent link

Direct link
Alternative names
Publications (10 of 608) Show all publications
Lenninger, M., Skoglund, M., Herman, P. & Kumar, A. (2023). Are single-peaked tuning curves tuned for speed rather than accuracy?. eLIFE, 12, Article ID e84531.
Open this publication in new window or tab >>Are single-peaked tuning curves tuned for speed rather than accuracy?
2023 (English)In: eLIFE, E-ISSN 2050-084X, Vol. 12, article id e84531Article in journal (Refereed) Published
Abstract [en]

According to the efficient coding hypothesis, sensory neurons are adapted to provide maximal information about the environment, given some biophysical constraints. In early visual areas, stimulus-induced modulations of neural activity (or tunings) are predominantly single-peaked. However, periodic tuning, as exhibited by grid cells, has been linked to a significant increase in decoding performance. Does this imply that the tuning curves in early visual areas are sub-optimal? We argue that the time scale at which neurons encode information is imperative to understand the advantages of single-peaked and periodic tuning curves, respectively. Here, we show that the possibility of catastrophic (large) errors creates a trade-off between decoding time and decoding ability. We investigate how decoding time and stimulus dimensionality affect the optimal shape of tuning curves for removing catastrophic errors. In particular, we focus on the spatial periods of the tuning curves for a class of circular tuning curves. We show an overall trend for minimal decoding time to increase with increasing Fisher information, implying a trade-off between accuracy and speed. This trade-off is reinforced whenever the stimulus dimensionality is high, or there is ongoing activity. Thus, given constraints on processing speed, we present normative arguments for the existence of the single-peaked tuning organization observed in early visual areas.

Place, publisher, year, edition, pages
eLife Sciences Publications, Ltd, 2023
Keywords
neural coding, tuning curves, decoding time, high-dimensional stimuli, spiking activity, None
National Category
Computer Sciences
Identifiers
urn:nbn:se:kth:diva-330512 (URN)10.7554/eLife.84531 (DOI)001006600800001 ()37191292 (PubMedID)2-s2.0-85161573273 (Scopus ID)
Note

QC 20230630

Available from: 2023-06-30 Created: 2023-06-30 Last updated: 2023-06-30Bibliographically approved
Chen, H., Ye, Y., Xiao, M. & Skoglund, M. (2023). Asynchronous Parallel Incremental Block-Coordinate Descent for Decentralized Machine Learning. IEEE Transactions on Big Data, 9(4), 1252-1259
Open this publication in new window or tab >>Asynchronous Parallel Incremental Block-Coordinate Descent for Decentralized Machine Learning
2023 (English)In: IEEE Transactions on Big Data, E-ISSN 2332-7790, Vol. 9, no 4, p. 1252-1259Article in journal (Refereed) Published
Abstract [en]

Machine learning (ML) is a key technique for big-data-driven modelling and analysis of massive Internet of Things (IoT) based intelligent and ubiquitous computing. For fast-increasing applications and data amounts, distributed learning is a promising emerging paradigm since it is often impractical or inefficient to share/aggregate data to a centralized location from distinct ones. This paper studies the problem of training an ML model over decentralized systems, where data are distributed over many user devices and the learning algorithm run on-device, with the aim of relaxing the burden at a central entity/server. Although gossip-based approaches have been used for this purpose in different use cases, they suffer from high communication costs, especially when the number of devices is large. To mitigate this, incremental-based methods are proposed. We first introduce incremental block-coordinate descent (I-BCD) for the decentralized ML, which can reduce communication costs at the expense of running time. To accelerate the convergence speed, an asynchronous parallel incremental BCD (API-BCD) method is proposed, where multiple devices/agents are active in an asynchronous fashion. We derive convergence properties for the proposed methods. Simulation results also show that our API-BCD method outperforms state of the art in terms of running time and communication costs.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2023
Keywords
Decentralized learning, block-coordinate descent, incremental method, asynchronous machine learning
National Category
Computer and Information Sciences
Identifiers
urn:nbn:se:kth:diva-334292 (URN)10.1109/TBDATA.2022.3230335 (DOI)001029182700017 ()2-s2.0-85146220732 (Scopus ID)
Note

QC 20230818

Available from: 2023-08-18 Created: 2023-08-18 Last updated: 2023-08-18Bibliographically approved
Daei, S., Razavikia, S., Kountouris, M., Skoglund, M., Fodor, G. & Fischione, C. (2023). Blind Asynchronous Goal-Oriented Detection for Massive Connectivity. In: 2023 21st International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks, WiOpt 2023: . Paper presented at 21st International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks, WiOpt 2023, Singapore, Singapore, Aug 24 2023 - Aug 27 2023 (pp. 167-174). Institute of Electrical and Electronics Engineers Inc.
Open this publication in new window or tab >>Blind Asynchronous Goal-Oriented Detection for Massive Connectivity
Show others...
2023 (English)In: 2023 21st International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks, WiOpt 2023, Institute of Electrical and Electronics Engineers Inc. , 2023, p. 167-174Conference paper, Published paper (Refereed)
Abstract [en]

Resource allocation and multiple access schemes are instrumental for the success of communication networks, which facilitate seamless wireless connectivity among a growing population of uncoordinated and non-synchronized users. In this paper, we present a novel random access scheme that addresses one of the most severe barriers of current strategies to achieve massive connectivity and ultra reliable and low latency communications for 6G. The proposed scheme utilizes wireless channels’ angular continuous group-sparsity feature to provide low latency, high reliability, and massive access features in the face of limited time-bandwidth resources, asynchronous transmissions, and preamble errors. Specifically, a reconstruction-free goal oriented optimization problem is proposed which preserves the angular information of active devices and is then complemented by a clustering algorithm to assign active users to specific groups. This allows to identify active stationary devices according to their line of sight angles. Additionally, for mobile devices, an alternating minimization algorithm is proposed to recover their preamble, data, and channel gains simultaneously, enabling the identification of active mobile users. Simulation results show that the proposed algorithm provides excellent performance and supports a massive number of devices. Moreover, the performance of the proposed scheme is independent of the total number of devices, distinguishing it from other random access schemes. The proposed method provides a unified solution to meet the requirements of machine-type communications and ultra reliable and low latency communications, making it an important contribution to the emerging 6G networks.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers Inc., 2023
Keywords
atomic norm minimization, goal-oriented optimization, Internet of Things, MIMO communications systems, Random access, reconstruction-free inference
National Category
Communication Systems
Identifiers
urn:nbn:se:kth:diva-343751 (URN)10.23919/WiOpt58741.2023.10349818 (DOI)2-s2.0-85184668805 (Scopus ID)
Conference
21st International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks, WiOpt 2023, Singapore, Singapore, Aug 24 2023 - Aug 27 2023
Note

QC 20240222

Part of ISBN 978-390317655-3

Available from: 2024-02-22 Created: 2024-02-22 Last updated: 2024-02-22Bibliographically approved
Zamani, A., Oechtering, T. J., Gündüz, D. & Skoglund, M. (2023). Cache-Aided Private Variable-Length Coding with Zero and Non-Zero Leakage. In: 2023 21st International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks, WiOpt 2023: . Paper presented at 21st International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks, WiOpt 2023, Singapore, Singapore, Aug 24 2023 - Aug 27 2023 (pp. 247-254). Institute of Electrical and Electronics Engineers (IEEE)
Open this publication in new window or tab >>Cache-Aided Private Variable-Length Coding with Zero and Non-Zero Leakage
2023 (English)In: 2023 21st International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks, WiOpt 2023, Institute of Electrical and Electronics Engineers (IEEE) , 2023, p. 247-254Conference paper, Published paper (Refereed)
Abstract [en]

A private cache-aided compression problem is studied, where a server has access to a database of N files, (Y1,...,YN), each of size F bits and is connected through a shared link to K users, each equipped with a local cache of size MF bits. In the placement phase, the server fills the users0 caches without knowing their demands, while the delivery phase takes place after the users send their demands to the server. We assume that each file Yi is arbitrarily correlated with a private attribute X, and an adversary is assumed to have access to the shared link. The users and the server have access to a shared key W. The goal is to design the cache contents and the delivered message C such that the average length of C is minimized, while satisfying: i. The response C does not reveal any information about X, i.e., X and C are independent, which corresponds to the perfect privacy constraint; ii. User i is able to decode its demand, Ydi, by using C, its local cache Zi, and the shared key W. Since the database is correlated with X, existing codes for cache-aided delivery do not satisfy the perfect privacy condition. Indeed, we propose a variable-length coding scheme that combines privacy-aware compression with coded caching techniques. In particular, we use two-part code construction and Functional Representation Lemma. Finally, we extend the results to the case, where X and C can be correlated, i.e., non-zero leakage is allowed.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2023
National Category
Telecommunications
Identifiers
urn:nbn:se:kth:diva-343750 (URN)10.23919/WiOpt58741.2023.10349822 (DOI)2-s2.0-85184656872 (Scopus ID)
Conference
21st International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks, WiOpt 2023, Singapore, Singapore, Aug 24 2023 - Aug 27 2023
Note

Part of proceedings ISBN 9783903176553

QC 20240222

Available from: 2024-02-22 Created: 2024-02-22 Last updated: 2024-02-22Bibliographically approved
Haghifam, M., Rodríguez Gálvez, B., Thobaben, R., Skoglund, M., Roy, D. M. & Dziugaite, G. K. (2023). Limitations of information: theoretic generalization bounds for gradient descent methods in stochastic convex optimization. In: Shipra Agrawal, Francesco Orabona (Ed.), Proceedings of ALT 2023: . Paper presented at 34th International Conference on Algorithmic Learning Theory, ALT 2023, Singapore, 20 - 23 February 2023 (pp. 663-706). ML Research Press
Open this publication in new window or tab >>Limitations of information: theoretic generalization bounds for gradient descent methods in stochastic convex optimization
Show others...
2023 (English)In: Proceedings of ALT 2023 / [ed] Shipra Agrawal, Francesco Orabona, ML Research Press , 2023, p. 663-706Conference paper, Published paper (Refereed)
Abstract [en]

To date, no “information-theoretic” frameworks for reasoning about generalization error have been shown to establish minimax rates for gradient descent in the setting of stochastic convex optimization. In this work, we consider the prospect of establishing such rates via several existing information-theoretic frameworks: input-output mutual information bounds, conditional mutual information bounds and variants, PAC-Bayes bounds, and recent conditional variants thereof. We prove that none of these bounds are able to establish minimax rates. We then consider a common tactic employed in studying gradient methods, whereby the final iterate is corrupted by Gaussian noise, producing a noisy “surrogate” algorithm. We prove that minimax rates cannot be established via the analysis of such surrogates. Our results suggest that new ideas are required to analyze gradient descent using information-theoretic techniques. 

Place, publisher, year, edition, pages
ML Research Press, 2023
Series
Proceedings of Machine Learning Research, ISSN 26403498
National Category
Computer Sciences
Identifiers
urn:nbn:se:kth:diva-328375 (URN)2-s2.0-85161238002 (Scopus ID)
Conference
34th International Conference on Algorithmic Learning Theory, ALT 2023, Singapore, 20 - 23 February 2023
Note

QC 20231204

Available from: 2023-06-08 Created: 2023-06-08 Last updated: 2023-12-04Bibliographically approved
Zamani, A., Oechtering, T. J. & Skoglund, M. (2023). Multi-User Privacy Mechanism Design with Non-zero Leakage. In: 2023 IEEE Information Theory Workshop, ITW 2023: . Paper presented at 2023 IEEE Information Theory Workshop, ITW 2023, Saint-Malo, France, Apr 23 2023 - Apr 28 2023 (pp. 401-405). Institute of Electrical and Electronics Engineers (IEEE)
Open this publication in new window or tab >>Multi-User Privacy Mechanism Design with Non-zero Leakage
2023 (English)In: 2023 IEEE Information Theory Workshop, ITW 2023, Institute of Electrical and Electronics Engineers (IEEE) , 2023, p. 401-405Conference paper, Published paper (Refereed)
Abstract [en]

A privacy mechanism design problem is studied through the lens of information theory. In this work, an agent observes useful data Y = (Y1,...,YN) that is correlated with private data X = (X1,...,XN) which is assumed to be also accessible by the agent. Here, we consider K users where user i demands a sub-vector of Y, denoted by Ci. The agent wishes to disclose Ci to user i. A privacy mechanism is designed to generate disclosed data U which maximizes a linear combinations of the users utilities while satisfying a bounded privacy constraint in terms of mutual information. In a similar work it has been assumed that Xi is a deterministic function of Yi, however in this work we let Xi and Yi be arbitrarily correlated.First, an upper bound on the privacy-utility trade-off is obtained by using a specific transformation, Functional Representation Lemma and Strong Functional Representation Lemma, then we show that the upper bound can be decomposed into N parallel problems. Next, lower bounds on privacy-utility tradeoff are derived using Functional Representation Lemma and Strong Functional Representation Lemma. The upper bound is tight within a constant and the lower bounds assert that the disclosed data is independent of all {Xj}i = 1N except one which we allocate the maximum allowed leakage to it. Finally, the obtained bounds are studied in special cases.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2023
National Category
Communication Systems
Identifiers
urn:nbn:se:kth:diva-334438 (URN)10.1109/ITW55543.2023.10161670 (DOI)001031733100071 ()2-s2.0-85165063293 (Scopus ID)
Conference
2023 IEEE Information Theory Workshop, ITW 2023, Saint-Malo, France, Apr 23 2023 - Apr 28 2023
Note

Part of ISBN 9798350301496

QC 20230821

Available from: 2023-08-21 Created: 2023-08-21 Last updated: 2023-09-01Bibliographically approved
Chen, Y., Oechtering, T. J., Skoglund, M. & Luo, Y. (2023). On Strong Secrecy for Multiple Access Channel with States and causal CSI. In: 2023 IEEE International Symposium on Information Theory, ISIT 2023: . Paper presented at 2023 IEEE International Symposium on Information Theory, ISIT 2023, Taipei, Taiwan, Jun 25 2023 - Jun 30 2023 (pp. 2744-2749). Institute of Electrical and Electronics Engineers (IEEE)
Open this publication in new window or tab >>On Strong Secrecy for Multiple Access Channel with States and causal CSI
2023 (English)In: 2023 IEEE International Symposium on Information Theory, ISIT 2023, Institute of Electrical and Electronics Engineers (IEEE) , 2023, p. 2744-2749Conference paper, Published paper (Refereed)
Abstract [en]

Strong secrecy communication over a discrete memoryless state-dependent multiple access channel (SD-MAC) with an external eavesdropper is investigated. The channel is governed by discrete memoryless and i.i.d. channel states and the channel state information (CSI) is revealed to the encoders in a causal manner. An inner bound of the capacity is provided. To establish the inner bound, we investigate coding schemes incorporating wiretap coding and secret key agreement between the sender and the legitimate receiver. Two kinds of block Markov coding schemes are studied. The first one uses backward decoding and Wyner-Ziv coding and the secret key is constructed from a lossy reproduction of the CSI. The other one is an extended version of the existing coding scheme for point-to-point wiretap channels with causal CSI. We further investigate some capacity-achieving cases for state-dependent multiple access wiretap channels (SD-MAWCs) with degraded message sets. It turns out that the two coding schemes are both optimal in these cases.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2023
National Category
Telecommunications Communication Systems
Identifiers
urn:nbn:se:kth:diva-337881 (URN)10.1109/ISIT54713.2023.10207008 (DOI)2-s2.0-85171454501 (Scopus ID)
Conference
2023 IEEE International Symposium on Information Theory, ISIT 2023, Taipei, Taiwan, Jun 25 2023 - Jun 30 2023
Note

Part of ISBN 9781665475549

QC 20231010

Available from: 2023-10-10 Created: 2023-10-10 Last updated: 2023-10-10Bibliographically approved
Zhang, D., Xiao, M. & Skoglund, M. (2023). Over-the-Air Computation Empowered Federated Learning: A Joint Uplink-Downlink Design. In: 98th IEEE Vehicular Technology Conference, VTC 2023-Fal: . Paper presented at 2023 IEEE 98th Vehicular Technology Conference, Hong Kong, 10-13 October 2023. Institute of Electrical and Electronics Engineers (IEEE)
Open this publication in new window or tab >>Over-the-Air Computation Empowered Federated Learning: A Joint Uplink-Downlink Design
2023 (English)In: 98th IEEE Vehicular Technology Conference, VTC 2023-Fal, Institute of Electrical and Electronics Engineers (IEEE) , 2023Conference paper, Published paper (Refereed)
Abstract [en]

In this paper, we investigate the communication designs of over-the-air computation (AirComp) empowered federated learning (FL) systems considering uplink model aggregation and downlink model dissemination jointly. We first derive an upper bound on the expected difference between the training loss and the optimal loss, which reveals that optimizing the FL performance is equivalent to minimizing the distortion in the received global gradient vector at each edge node. As such, we jointly optimize each edge node transmit and receive equalization coefficients along with the edge server forwarding matrix to minimize the maximum gradient distortion across all edge nodes. We further utilize the MNIST dataset to evaluate the performance of the considered FL system in the context of the handwritten digit recognition task. Experiment results show that deploying multiple antennas at the edge server significantly reduces the distortion in the received global gradient vector, leading to a notable improvement in recognition accuracy compared to the single antenna case.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2023
National Category
Telecommunications
Identifiers
urn:nbn:se:kth:diva-338819 (URN)2-s2.0-85181165722 (Scopus ID)
Conference
2023 IEEE 98th Vehicular Technology Conference, Hong Kong, 10-13 October 2023
Note

QC 20240112

Part of ISBN 979-835032928-5

Available from: 2023-10-26 Created: 2023-10-26 Last updated: 2024-01-12Bibliographically approved
Saeidian, S., Cervia, G., Oechtering, T. J. & Skoglund, M. (2023). Pointwise Maximal Leakage on General Alphabets. In: 2023 IEEE International Symposium on Information Theory, ISIT 2023: . Paper presented at 2023 IEEE International Symposium on Information Theory, ISIT 2023, Taipei, Taiwan, Jun 25 2023 - Jun 30 2023 (pp. 388-393). Institute of Electrical and Electronics Engineers (IEEE)
Open this publication in new window or tab >>Pointwise Maximal Leakage on General Alphabets
2023 (English)In: 2023 IEEE International Symposium on Information Theory, ISIT 2023, Institute of Electrical and Electronics Engineers (IEEE) , 2023, p. 388-393Conference paper, Published paper (Refereed)
Abstract [en]

Pointwise maximal leakage (PML) is an operationally meaningful privacy measure that quantifies the amount of information leaking about a secret X to a single outcome of a related random variable Y. In this paper, we extend the notion of PML to random variables on arbitrary probability spaces. We develop two new definitions: First, we extend PML to countably infinite random variables by considering adversaries who aim to guess the value of discrete (finite or countably infinite) functions of X. Then, we consider adversaries who construct estimates of X that maximize the expected value of their corresponding gain functions. We use this latter setup to introduce a highly versatile form of PML that captures many scenarios of practical interest whose definition requires no assumptions about the underlying probability spaces.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2023
National Category
Probability Theory and Statistics
Identifiers
urn:nbn:se:kth:diva-337885 (URN)10.1109/ISIT54713.2023.10206975 (DOI)2-s2.0-85171473626 (Scopus ID)
Conference
2023 IEEE International Symposium on Information Theory, ISIT 2023, Taipei, Taiwan, Jun 25 2023 - Jun 30 2023
Note

Part of ISBN 9781665475549

QC 20231010

Available from: 2023-10-10 Created: 2023-10-10 Last updated: 2023-10-10Bibliographically approved
Zamani, A., Oechtering, T. J. & Skoglund, M. (2023). Private Variable-Length Coding with Non-Zero Leakage. In: WIFS 2023 - IEEE Workshop on Information Forensics and Security: . Paper presented at 2023 IEEE International Workshop on Information Forensics and Security, WIFS 2023, Nurnberg, Germany, Dec 4 2023 - Dec 7 2023. Institute of Electrical and Electronics Engineers (IEEE)
Open this publication in new window or tab >>Private Variable-Length Coding with Non-Zero Leakage
2023 (English)In: WIFS 2023 - IEEE Workshop on Information Forensics and Security, Institute of Electrical and Electronics Engineers (IEEE) , 2023Conference paper, Published paper (Refereed)
Abstract [en]

A private compression design problem is studied, where an encoder observes useful data Y, wishes to compress it using variable length code and communicates it through an unsecured channel. Since Y is correlated with private data X, the encoder uses a private compression mechanism to design encoded message C and sends it over the channel. An adversary is assumed to have access to the output of the encoder, i.e., C and tries to estimate X. Furthermore, it is assumed that both encoder and decoder have access to a shared secret key W. In this work, we generalize the perfect privacy (secrecy) assumption and consider a non-zero leakage between the private data X and encoded message C. The design goal is to encode message C with minimum possible average length that satisfies non-perfect privacy constraints. We find upper and lower bounds on the average length of the encoded message using different privacy metrics and study them in special cases. For the achievability we use two-part construction coding and extended versions of Functional Representation Lemma. Lastly, in an example we show that the bounds can be asymptotically tight.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2023
National Category
Communication Systems Computer Sciences
Identifiers
urn:nbn:se:kth:diva-343179 (URN)10.1109/WIFS58808.2023.10374604 (DOI)001156967300005 ()2-s2.0-85183470148 (Scopus ID)
Conference
2023 IEEE International Workshop on Information Forensics and Security, WIFS 2023, Nurnberg, Germany, Dec 4 2023 - Dec 7 2023
Note

Part of proceedings ISBN 9798350324914

QC 20240212

Available from: 2024-02-08 Created: 2024-02-08 Last updated: 2024-03-12Bibliographically approved
Organisations
Identifiers
ORCID iD: ORCID iD iconorcid.org/0000-0002-7926-5081

Search in DiVA

Show all publications