We consider correlated equilibria in strategic games in an adversarial environment, where an adversary can compromise the public signal used by the players for choosing their strategies, while players aim at detecting a potential attack as soon as possible to avoid loss of utility. We model the interaction between the adversary and the players as a zero-sum game and we derive the maxmin strategies for both the defender and the attacker using the framework of quickest change detection. We define a class of adversarial strategies that achieve the optimal trade-off between attack impact and attack detectability and show that a generalized CUSUM scheme is asymptotically optimal for the detection of the attacks. Our numerical results on the Sioux-Falls benchmark traffic routing game show that the proposed detection scheme can effectively limit the utility loss by a potential adversary. Code - https://github.com/kiarashkaz/Detection-of-Adversarial-Attacks-against-CE

Automatic generation control (AGC) is an essential functionality for ensuring the stability of power systems, and its secure operation is thus of utmost importance to power system operators. In this paper, we investigate the vulnerability of AGC to false data injection attacks that could remain undetected by traditional detection methods based on the area control error (ACE) and the recently proposed unknown input observer (UIO). We formulate the problem of computing undetectable attacks as a multi-objective partially observable Markov decision process. We propose a flexible reward function that allows to explore the trade-off between attack impact and detectability, and use the proximal policy optimization (PPO) algorithm for learning efficient attack policies. Through extensive simulations of a 3-area power system, we show that the proposed attacks can drive the frequency beyond critical limits, while remaining undetectable by state-of-the-art algorithms employed for fault and attack detection in AGC. Our results also show that detectors trained using supervised and unsupervised machine learning can both significantly outperform existing detectors.

Secondary control of voltage magnitude and frequency is essential to the stable and secure operation of microgrids (MGs). Recent years have witnessed an increasing interest in developing secondary controllers based on multi-agent reinforcement learning (MARL), in order to replace existing model-based controllers. Nonetheless, unlike the vulnerabilities of model-based controllers, the vulnerability of MARLbased MG secondary controllers has so far not been addressed. In this paper, we investigate the vulnerability of MARL controllers to false data injection attacks (FDIAs). Based on a formulation of MG secondary control as a partially observable stochastic game (POSG), we propose to formulate the problem of computing FDIAs as a partially observable Markov decision process (POMDP), and we use state-of-the-art RL algorithms for solving the resulting problem. Based on extensive simulations of a MG with 4 distributed generators (DGs), our results show that MARL-based secondary controllers are more resilient to FDIAs compared to state of the art model-based controllers, both in terms of attack impact and in terms of the effort needed for computing impactful attacks. Our results can serve as additional arguments for employing MARL in future MG control.

We consider the problem of detecting adversarial attacks against cooperative multi-agent reinforcement learning. We propose a decentralized scheme that allows agents to detect the abnormal behavior of one compromised agent. Our approach is based on a recurrent neural network (RNN) trained during cooperative learning to predict the action distribution of other agents based on local observations. The predicted distribution is used for computing a normality score for the agents, which allows the detection of the misbehavior of other agents. To explore the robustness of the proposed detection scheme, we formulate the worst-case attack against our scheme as a constrained reinforcement learning problem. We propose to compute an attack policy via optimizing the corresponding dual function using reinforcement learning. Extensive simulations on various multi-agent benchmarks show the effectiveness of the proposed detection scheme in detecting state of the art attacks and in limiting the impact of undetectable attacks.

This paper presents a novel market clearing mechanism for manual Frequency Restoration Reserve (mFRR) capacity markets, focusing on the Nordic market setup. The proposed market clearing mechanism accounts for both expected energy activation costs and mFRR capacity costs. The market clearing problem is formulated as a two-stage stochastic Mixed Integer Linear Program (MILP). To efficiently solve the resulting optimization problem, we introduce a nested decomposition algorithm that combines the Benders Decomposition (BD) and Surrogate Absolute Value Lagrangian Relaxation(SAVLR) methods. For practical implementation, input data scenarios aregenerated using day-ahead (DA) price data modeled by a Bayesian Neural Network (BNN). A real-world case study in Sweden demonstrates that theproposed mechanism can reduce daily mFRR costs by 600-14,500 €. Simulation results further show that the nested decomposition algorithm converges to a near-optimal solution more quickly than standard BD.