Deep reinforcement learning (DRL) is a promising solution for the coordination of power systems with high penetration of inverter-based renewable energy sources (RESs). Yet, when adopting the DRL-based control method, the safe and optimal operation of the system cannot be guaranteed at the same time, as the conventional DRL agent is not designed to solve the hard constraint problem. To address this challenge, a deep neural network (DNN) assisted projection-based DRL method for safe control of distribution grids is proposed in this section. First, a finite iteration projection algorithm is proposed to guarantee hard constraints by converting a non-convex optimization problem into a finite iteration problem. Next, a DNN-assisted projection method is proposed to accelerate the calculation of projection and achieve the practical implementation of hard constraints in the DRL problem. Finally, a DNN Projection embedded twin-delayed deep deterministic policy gradient (DPe-TD3) method is proposed to achieve optimal operation of distribution grids with guaranteed 100% safety of the distribution grid. The safety of the DRL training is guaranteed via the embedded Projection DNN in TD3 with participation in the gradient return process, which could smoothly and effectively project the DRL agent actions into the feasible area, thus guaranteeing the safety of data-driven control and the optimal operation at the same time. The case studies and comparisons are conducted in the IEEE 33 bus system to show the effectiveness of the proposed method.
Part of ISBN 9783031778322, 9783031778339
QC 20251111