In this dissertation, we study two related classes of problems within the field of systems theory: the analysis and optimization of networked dynamical systems, and the reconstruction of unknown cost functions in control and learning frameworks. These problems naturally arise in a wide range of applications, from engineering systems to natural phenomenon.
The first class of problems concerns control and optimization over large-scale networked systems. Achieving controllability efficiently is an essential topic. The problem of ensuring controllability with a minimal number of control inputs is investigated first. Moreover, optimal control placement with limited controls are studied to enhance energy efficiency, and reduce computational and implementation costs. The second class focuses on inverse problems of optimal control, which aim to reconstruct unknown cost functions from observed behaviour. These problems are valuable for uncovering the objectives underlying complex systems in nature and society. Both cases are considered: when the system dynamics are known a priori and when they are unknown.
Specifically, Paper A investigates the minimal control placement problem for networked systems derived from Turing's reaction-diffusion model, a classical framework for understanding self-organization and pattern formation in biological systems. The eigenstructure of the diffusion matrix is fully characterized, and by introducing symmetric control sets, we establish the necessary and sufficient graph-theoretic condition that guarantees controllability of diffusion systems over networks of arbitrary size and parameters. These results are further extended to the reaction-diffusion systems.
After studying network controllability, Paper B extends the analysis to energy-efficient control placement in networked systems. By classifying network symmetries and exploiting symmetric control combinations, we develop a method that enables efficient computation of the spectrum of the controllability Gramian through lower-dimensional representations. This approach is further generalized to non-symmetric cases, where upper and lower spectral bounds are derived. Moreover, by utilizing the trace of the controllability Gramian as the objective, we propose a closed-form algorithm for optimizing control placement under constraints of limited control inputs and system controllability, with simulations validating its effectiveness.
Paper C addresses inverse optimal control for continuous-time linear quadratic regulators over finite-time horizons, namely the reconstruction of unknown cost matrices R, Q, and F in the objective function from observed optimal control trajectories. The underlying linear system is assumed to be known. Both problem settings where R is either unknown or given are investigated. Firstly, two methods are developed to reconstruct R: one that leverages the full trajectory of the optimal feedback matrix and provides the necessary and sufficient conditions for uniqueness, and another that relies only on selected time points to reduce computational burden, which is particularly effective if F is given as positive definite. Secondly, when R is given, we investigate the role of system controllability in determining the well-posedness of the inverse problems. This assumption is subsequently relaxed, and sufficient conditions are established to ensure well-posedness, along with explicit analytical expressions for Q and F. Finally, the structural equivalence between IOC problems with unknown and given R is characterized under certain circumstances.
Paper D investigates inverse reinforcement learning (IRL) to reconstruct the unknown cost function in a model-free setting, where system dynamics are also unknown. Conventional IRL algorithms often require on-policy data collection and bi-level optimization, which impose potential practical limitations. To overcome these challenges, we propose a direct and adaptive IRL algorithm that learns from off-policy data satisfying only a mild persistence of excitation condition. By employing Nesterov-Todd (NT) step primal-dual interior-point iterations, the cost parameter is updated through simple one-step recursions, avoiding repeated forward RL computations. Theoretical analysis quantifies the impact of system noise and establishes sublinear convergence of the proposed algorithm. This method is further generalized to nonlinear objective functions via differential dynamic programming, where gradients of the loss function are computed through a backward pass. Numerical simulations demonstrate the efficiency and effectiveness of the proposed approach.