This paper considers the problem of identifying the profiles and capabilities of attackers injecting adversarial inputs to a cyber-physical system. The system in question interacts with attackers of different levels of intelligence, each employing different feedback controllers against the system. Principles of behavioral game theory – specifically the concept of level-k thinking – is employed to construct a database of potential attack vectors. By observing the state trajectories under sequential interactions with different adversaries, the defender adaptively estimates both the number and profiles of the different attack signals using an online deterministic annealing approach. This information is used to dynamically estimate the level of intelligence of the attackers. Simulation results showcase the efficacy of the proposed method.

We consider correlated equilibria in strategic games in an adversarial environment, where an adversary can compromise the public signal used by the players for choosing their strategies, while players aim at detecting a potential attack as soon as possible to avoid loss of utility. We model the interaction between the adversary and the players as a zero-sum game and we derive the maxmin strategies for both the defender and the attacker using the framework of quickest change detection. We define a class of adversarial strategies that achieve the optimal trade-off between attack impact and attack detectability and show that a generalized CUSUM scheme is asymptotically optimal for the detection of the attacks. Our numerical results on the Sioux-Falls benchmark traffic routing game show that the proposed detection scheme can effectively limit the utility loss by a potential adversary. Code - https://github.com/kiarashkaz/Detection-of-Adversarial-Attacks-against-CE

In this paper we propose an unpredictability-based jamming defense framework based on the principles of Moving Target Defense for a wireless communication problem. Taking advantage of the complex nature of large-scale cyber-physical systems, we consider a platform consisting of a single receiving component but multiple potential transmitting components, each equipped with a multi-antenna phased array. We formulate an optimization problem over the probability simplex that characterizes a randomized receiving angle which seeks to balance between the estimated performance of the transmission and an entropy-based unpredictability measure. Furthermore, we explore the effect of an intelligent adversary that has knowledge of the derived probabilities and optimally places a single-antenna jamming device to disrupt the communication links. Finally, simulation results showcase the efficacy of the proposed algorithm.

Control-Theoretic Methods for Cyber-Physical Security presents novel results on security and defense methodologies applied to cyber-physical systems. This book takes a control and game theory perspective, treating autonomous platforms as dynamic systems. It introduces algorithmic frameworks designed to proactively and reactively safeguard these systems against catastrophic failures. The algorithms showcased encompass a wide spectrum of security techniques, from model-free detection mechanisms to unpredictability-based defense strategies, combining both model-based and data-driven approaches.

Fictitious play is a popular learning algorithm in which players that utilize the history of actions played by the players and the knowledge of their own payoff matrix can converge to the Nash equilibrium under certain conditions on the game. We consider the presence of an intelligent player that has access to the entire payoff matrix for the game. We show that by not conforming to fictitious play, such a player can achieve a better payoff than the one at the Nash Equilibrium. This result can be viewed both as a fragility of the fictitious play algorithm to a strategic intelligent player and an indication that players should not throw away additional information they may have, as suggested by classical fictitious play.

In this work, we consider systems whose components suffer from clock offsets and quantization and study the effect of those on a reinforcement learning (RL) algorithm. Specifically, we consider an off-policy iterative RL algorithm for continuous-time systems, which uses input and state data to approximate the Nash-equilibrium of a zero-sum game. However, the data used by this algorithm are not consistent with one another, in that each of them originates from a slightly different time instant of the past, hence putting the convergence of the algorithm in question. We prove that, given that these timing inconsistencies remain below a certain threshold, the iterative off-policy RL algorithm will still converge epsilon-closely to the desired Nash policy. However, this result is conditional to a certain Lipschitz continuity and differentiability condition on the input-state data collected, which is indispensable in the presence of clock offsets. A similar result is also derived when quantization of the measured state is considered. Finally, unlike prior work, we provide a sufficiently rich data condition for the execution of the iterative RL algorithm, which can be verified a priori across all iteration indices. Simulations are performed, which verify and clarify theoretical findings.

In this paper, we study the problem of poisoning the learning of an optimal controller by means of an actuation attack. We specifically consider a user who is gathering data from a linear system in the form of input and state measurements, and who uses these data to learn an optimal controller. Nevertheless, these measurements are corrupted by an attacker who has access to the system's actuators, and who is using them to launch an actuation attack during the learning process. We design this actuation attack so that it optimally corrupts the data used by the user: it forces the user to learn as closely as possible a gain that the attacker has selected, and which is unrelated to the actual optimal control gain. We prove that this poisoning actuation attack design boils down to the solution of certain coupled matrix equations, which we solve using the block successive over-relaxation (SOR) iterative procedure. Simulations on an aircraft model demonstrate theoretical findings, showing how the poisoning attack is effective in misleading the user towards learning an incorrect gain for the system.

In this paper, we present a comprehensive defense framework for cyber-physical systems comprising proactive and reactive mechanisms in a multi-agent scenario. The scenario is compounded of three crazyflie nano quadcopters – open-source flying development platforms – connected to a centralized controller via radio communication operating in uncertain and adversarial environments. The proactive mechanism, based on the principles of moving target defense, utilizes a stochastic switching structure to dynamically and continuously alter the altitude of hovering of the agents to minimize the risk of attack while the attacker’s effect is modeled as time-varying and unknown. The reactive mechanism, on the other side, detects potentially attacked components, namely sensors and actuators, by leveraging online data to compute an integral Bellman error. The attack detection relies on the optimality property as well as on data measured along the trajectories of the system; when an attack is detected by an agent at a certain altitude of hovering, the information is shared with all agents via a centralized controller to classify the region as unsafe. The efficacy of the proposed defense framework is shown by experimental trials in different scenarios of cyber-physical attacks.

In this paper, we present a comprehensive defense framework for cyber-physical systems comprising proactive and reactive mechanisms in a multi-agent scenario. The scenario is compounded of three crazyflie nano quadcopters - open-source flying development platforms - connected to a centralized controller via radio communication operating in uncertain and adversarial environments. The proactive mechanism, based on the principles of moving target defense, utilizes a stochastic switching structure to dynamically and continuously alter the altitude of hovering of the agents to minimize the risk of attack while the attacker's effect is modeled as time-varying and unknown. The reactive mechanism, on the other side, detects potentially attacked components, namely sensors and actuators, by leveraging online data to compute an integral Bellman error. The attack detection relies on the optimality property as well as on data measured along the trajectories of the system; when an attack is detected by an agent at a certain altitude of hovering, the information is shared with all agents via a centralized controller to classify the region as unsafe. The efficacy of the proposed defense framework is shown by experimental trials in different scenarios of cyber-physical attacks.

We propose an online identification scheme for discrete-time piece-wise affine state-space models based on a system of adaptive algorithms running in two timescales. A stochastic approximation algorithm implements an online deterministic annealing scheme at a slow timescale, estimating the partition of the augmented state-input space that defines the switching signal. At the same time, an adaptive identification algorithm, running at a higher timescale, updates the parameters of the local models based on the estimate of the switching signal. Identifiability conditions for the switched system are discussed and convergence results are given based on the theory of two-timescale stochastic approximation. In contrast to standard identification algorithms for piece-wise affine systems, the proposed approach progressively estimates the number of modes needed and is appropriate for online system identification using sequential data acquisition. This progressive nature of the algorithm improves computational efficiency and provides real-time control over the performance-complexity trade-off, desired in practical applications. Experimental results validate the efficacy of the proposed methodology.