Automatic generation control (AGC) is an essential functionality for ensuring the stability of power systems, and its secure operation is thus of utmost importance to power system operators. In this paper, we investigate the vulnerability of AGC to false data injection attacks that could remain undetected by traditional detection methods based on the area control error (ACE) and the recently proposed unknown input observer (UIO). We formulate the problem of computing undetectable attacks as a multi-objective partially observable Markov decision process. We propose a flexible reward function that allows to explore the trade-off between attack impact and detectability, and use the proximal policy optimization (PPO) algorithm for learning efficient attack policies. Through extensive simulations of a 3-area power system, we show that the proposed attacks can drive the frequency beyond critical limits, while remaining undetectable by state-of-the-art algorithms employed for fault and attack detection in AGC. Our results also show that detectors trained using supervised and unsupervised machine learning can both significantly outperform existing detectors.

Time synchronization attacks constitute a major threat to phasor measurement unit-based smart grid applications, and their cost-efficient detection and mitigation is thus of utmost importance. In this article, we propose a mitigation approach based on authenticated network-based time synchronization. Our approach relies on the observation that a time synchronization attack can be undetectable if and only if it targets at least three time references in the power system, and such attacks need to be mitigated through appropriate security controls. We first provide a formal proof of this result, including a characterization of the degrees of freedom of the attacker in constructing an attack. We then formulate the problem of mitigating undetectable attacks at minimum cost as an integer linear program, and prove that it is NP-hard. To solve the problem, we propose two approximation algorithms based on computing shortest paths and solving a linear relaxation of the problem. Extensive simulations suggest the superiority of the proposed algorithms on IEEE benchmark power system graphs compared with baseline solutions. We report mitigation cost savings of at least 76% compared with a naive approach for mitigation and at least 30% compared with the state-of-the-art approach.

Secondary control of voltage magnitude and frequency is essential to the stable and secure operation of microgrids (MGs). Recent years have witnessed an increasing interest in developing secondary controllers based on multi-agent reinforcement learning (MARL), in order to replace existing model-based controllers. Nonetheless, unlike the vulnerabilities of model-based controllers, the vulnerability of MARLbased MG secondary controllers has so far not been addressed. In this paper, we investigate the vulnerability of MARL controllers to false data injection attacks (FDIAs). Based on a formulation of MG secondary control as a partially observable stochastic game (POSG), we propose to formulate the problem of computing FDIAs as a partially observable Markov decision process (POMDP), and we use state-of-the-art RL algorithms for solving the resulting problem. Based on extensive simulations of a MG with 4 distributed generators (DGs), our results show that MARL-based secondary controllers are more resilient to FDIAs compared to state of the art model-based controllers, both in terms of attack impact and in terms of the effort needed for computing impactful attacks. Our results can serve as additional arguments for employing MARL in future MG control.

We consider the problem of detecting adversarial attacks against cooperative multi-agent reinforcement learning. We propose a decentralized scheme that allows agents to detect the abnormal behavior of one compromised agent. Our approach is based on a recurrent neural network (RNN) trained during cooperative learning to predict the action distribution of other agents based on local observations. The predicted distribution is used for computing a normality score for the agents, which allows the detection of the misbehavior of other agents. To explore the robustness of the proposed detection scheme, we formulate the worst-case attack against our scheme as a constrained reinforcement learning problem. We propose to compute an attack policy via optimizing the corresponding dual function using reinforcement learning. Extensive simulations on various multi-agent benchmarks show the effectiveness of the proposed detection scheme in detecting state of the art attacks and in limiting the impact of undetectable attacks.

Dynamic application security testing (DAST) scanning consists of automated requests to web applications with the goal of uncovering exploitable vulnerabilities. While the legitimate use of scanners aids development teams in improving security postures, they are often used by malicious actors in a brute-force manner for attack reconnaissance with a view to eventual compromise. Despite this threat from misuse of DAST to web applications and the critical data they handle, security mechanisms are lacking, with threshold-based classifiers suffering from being overly sensitive, causing excessive false positives. This paper demonstrates the first application of machine learning to specifically detect DAST attacks that augments a next-generation web application firewall implementing OWASP's AppSensor framework. Avoiding the brittle threshold approach and using tumbling windows of time to generate aggregated event features from source IPs, twelve random forest models are trained on millions of real-world events. Results show an optimal window size of 60 seconds achieves an F1 score of 0.94 and a miss rate of 6% on average across three production-grade web applications.

Time Synchronization Attacks (TSAs) against Phasor Measurement Units (PMUs) constitute a major threat to modern smart grid applications. By compromising the time reference of a set of PMUs, an attacker can change the phase angle of their measured phasors, with potentially detrimental impact on grid operation and control. Going beyond traditional residual-based techniques in detecting TSAs, in this paper we propose the use of Graph Signal Processing (GSP) to model the power grid so as to facilitate the detection and localization of TSAs. We analytically show that modeling the state of the power system as a low-pass graph signal can significantly improve the resilience of the grid against TSAs. We propose TSA detection and localization methods based on GSP, leveraging state-of-the-art machine learning algorithms. We provide empirical evidence for the efficiency of the proposed methods based on extensive simulations on five IEEE benchmark systems. In fact, our methods can detect at least 77% more TSAs of significant impact and localize an additional 70% of the attacked PMUs compared to state-of-the-art techniques.

Phasor Measurement Units (PMUs) constitute an emerging technology that is essential for various smart grid applications such as phase angle monitoring, power oscillation damping, fault localization, and linear state estimation. To obtain precise PMU measurements of voltage and current phasors, time synchronization in the order of 1 microsecond is typically required. Nevertheless, time synchronization sources for PMUs, such as GPS satellites and Precision Time Protocol (PTP), are vulnerable to Time Synchronization Attacks (TSAs). A TSA can disrupt time synchronization, resulting in malicious phase angle measurements, potentially leading to serious consequences to the stability of the power grid. Moreover, sophisticated attackers may be able to develop undetectable TSAs that would lead to incorrect but credible estimates of the system state, which will bypass traditional Bad Data Detection (BDD) algorithms employed in the grid. Therefore, the detection and mitigation of such undetectable TSAs is of utmost importance for power system operators.​

The first part of this thesis explores the threat of undetectable TSAs by investigating their practical feasibility. We provide necessary and sufficient conditions for a set of PMUs to be vulnerable to undetectable TSAs and provide an efficient algorithm to compute attacks against any number of vulnerable PMUs. Furthermore, we show that the set of undetectable TSAs forms a continuum if at least three vulnerable PMUs are targeted by the attack. This fact can be exploited by an attacker to develop low-rate attacks that would adapt to the clock servo that controls the PMU clock, and would bypass typical change detection-based security solutions. The feasibility of computing undetectable TSAs was demonstrated using realistic PMU data and a widely-used clock servo implementation.

The second part of this thesis considers the detection of TSAs. To this end, we proposed three detection approaches focusing on various aspects of PMU and power grid operations. The first proposed approach is decentralized, and attempts to detect TSAs at every PMU individually by leveraging the dependence between the PMU clock state and the measured phasor. The approach is based on the observation that a TSA changes the correlation between the PMU clock frequency adjustments and the change in the measured phase angle. We proposed model-based and data-driven machine learning-based TSA detectors exploiting the change in correlation. Using extensive simulations and realistic PMU clock models, the proposed detectors were shown to perform well even for relatively low-rate attacks. The second proposed approach is centralized and is based on performing state estimation using the complete three-phase model instead of the simpler and more widely-used direct-sequence equivalent model. Our analytical results and extensive simulations showed that three-phase state estimators are significantly more resilient to TSAs compared to single-phase state estimators in unbalanced three phase systems. The third proposed approach is based on the framework of Graph Signal Processing (GSP) in power systems. We showed that by regarding the system state as a graph signal, the low-dimensional structure of the PMU measurements and the system state can be exploited for TSA detection. Based on GSP, we proposed a high-pass graph filter as well as machine learning classifiers utilizing GSP features, both showing superior performance not only in detecting the presence of a TSA, but also in localizing the attacked PMUs.

The third and final part of the thesis considers the mitigation of TSAs, with special focus on PTP networks. In this regard, we investigated recently standardized authentication schemes in PTPv2.1 and their effect on both the synchronization accuracy and network latency in an experimental testbed. The results showed that the authentication schemes pose no significant overhead on the synchronization accuracy or the network latency. Moreover, the cost considerations of PTP authentication were investigated by considering the partial application of the authentication schemes to a PTP network only in the parts that are vulnerable to undetectable TSAs, thus combining TSA mitigation and detection. We showed that the problem of mitigating undetectable TSAs at minimum cost is NP-hard. We formulated the problem as an integer linear program and proposed two approximation algorithms based on linear relaxation and a greedy heuristic. Through extensive simulations on both synthetic graphs and realistic IEEE benchmark power system graphs, we showed that our proposed algorithms, combining both state estimation and PTP authentication, can dramatically reduce the cost of mitigating TSAs.

The TSA detection and mitigation approaches presented in this thesis constitute a step towards secure and reliable time synchronization for PMU applications and a more resilient smart grid infrastructure.

Phasor measurement units (PMU) rely on an accurate time-synchronization to phase-align the phasors and timestamp the voltage and current phasor measurements. Among the symmetrical components computed from the phasors in three-phase systems, the standard practice only uses the direct-sequence component for state estimation and bad data detection (BDD). Time-synchronization attacks (TSAs) can compromise the measured phasors and can, thus, significantly alter the state estimate in a manner that is undetectable by widely used power-system BDD algorithms. In this paper we investigate the potential of utilizing the three-phase model instead of the direct-sequence model for mitigating the vulnerability of state estimation to undetectable TSAs. We show analytically that if the power system is unbalanced then the use of the three-phase model as input to BDD algorithms enables to detect attacks that would be undetectable if only the direct-sequence model was used. Simulations performed on the IEEE 39-bus benchmark using real load profiles recorded on the grid of the city of Lausanne confirm our analytical results. Our results provide a new argument for the adoption of three-phase models for BDD, as their use is a simple, yet effective measure for reducing the vulnerability of PMU measurements to TSAs.

The emerging measurement technology of phasor measurement units (PMUs) makes it possible to estimate the state of electrical grids in real time, thus opening the way to new protection and control applications. PMUs rely on precise time synchronization; therefore, they are vulnerable to time-synchronization attacks (TSAs), which alter the measured voltage and current phases. In particular, undetectable TSAs pose a significant threat as they lead to an incorrect but credible estimate of the system state. Prior work has shown that such attacks exist against pairs of PMUs, but they do not take into consideration the clock adjustment performed by the clock servo, which can modify the attack angles and make the attacks detectable. This cannot be easily addressed with the existing attacks, as the undetectable angle values form a discrete set and cannot be continuously adjusted as would be required to address the problems posed to the attacker by the clock servo. Going beyond prior work, this article first shows how to perform undetectable attacks against more than two PMUs, so that the set of undetectable attacks forms a continuum and supports small adjustments. Second, it shows how an attacker can anticipate the operation of the clock servo while achieving her attack goal and remaining undetectable. Third, this article shows how to identify vulnerable sets of PMUs. Numerical results on the 39-bus IEEE benchmark system illustrate the feasibility of the proposed attack strategies.