kth.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
DNN Assisted Projection based Deep Reinforcement Learning for Safe Control of Distribution Grids
KTH, School of Electrical Engineering and Computer Science (EECS), Electrical Engineering, Electric Power and Energy Systems.ORCID iD: 0000-0003-0746-0221
State Grid Economic and Technological Research Institute Co., Ltd., Beijing, China.
KTH, School of Electrical Engineering and Computer Science (EECS), Electrical Engineering, Electric Power and Energy Systems.ORCID iD: 0000-0003-0184-6553
KTH, School of Electrical Engineering and Computer Science (EECS), Electrical Engineering, Electric Power and Energy Systems.ORCID iD: 0000-0002-2793-9048
2024 (English)In: IEEE Transactions on Power Systems, ISSN 0885-8950, E-ISSN 1558-0679, Vol. 39, no 4, p. 5687-5698Article in journal (Refereed) Published
Abstract [en]

Deep reinforcement learning (DRL) is a promising solution for voltage control of distribution grids with high penetration of inverter-based renewable energy sources (RESs). Yet, when adopting the DRL-based control method, the safe and optimal operation of the system cannot be guaranteed at the same time, as the conventional DRL agent is not designed to solve the hard constraint problem. To address this challenge, this paper proposes a deep neural network (DNN) assisted projection based DRL method for safe control of distribution grids. First, a finite iteration projection algorithm is proposed to guarantee hard constraints by converting a non-convex optimization problem into a finite iteration problem. Next, a DNN assisted projection method is proposed to accelerate the calculation of projection and achieve the practical implementation of hard constraints in DRL problem. Finally, a DNN Projection embedded twin-delayed deep deterministic policy gradient (DPe-TD3) method is proposed to achieve optimal operation of distribution grids with guaranteed 100% safety of the distribution grid. The safety of the DRL training is guaranteed via the embedded Projection DNN in TD3 with participation in gradient return process, which could smoothly and effectively project the DRL agent actions into the feasible area, thus guaranteeing the safety of data driven control and the optimal operation at the same time. The case studies and comparisons are conducted in the IEEE 33 bus system to show the effectiveness of the proposed method.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE) , 2024. Vol. 39, no 4, p. 5687-5698
Keywords [en]
Artificial neural networks, deep neural network, deep reinforcement learning, distribution grid, inverter interfaced RESs, Inverters, Optimization, projection, safety, Safety, Security, Training, Voltage control
National Category
Electrical Engineering, Electronic Engineering, Information Engineering
Identifiers
URN: urn:nbn:se:kth:diva-350174DOI: 10.1109/TPWRS.2023.3336614ISI: 001252602200041Scopus ID: 2-s2.0-85179111088OAI: oai:DiVA.org:kth-350174DiVA, id: diva2:1883207
Note

QC 20240709

Available from: 2024-07-09 Created: 2024-07-09 Last updated: 2024-07-15Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Authority records

Zhang, MengfanZhao, TianyangXu, Qianwen

Search in DiVA

By author/editor
Zhang, MengfanZhao, TianyangXu, Qianwen
By organisation
Electric Power and Energy Systems
In the same journal
IEEE Transactions on Power Systems
Electrical Engineering, Electronic Engineering, Information Engineering

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 12 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf