A Two-stage Multi-agent Deep Reinforcement Learning Method for Urban Distribution Network Reconfiguration Considering Switch Contribution
Loading...
Access rights
openAccess
acceptedVersion
URL
Journal Title
Journal ISSN
Volume Title
A1 Alkuperäisartikkeli tieteellisessä aikakauslehdessä
This publication is imported from Aalto University research portal.
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Other link related to publication (opens in new window)
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Other link related to publication (opens in new window)
Date
2024
Major/Subject
Mcode
Degree programme
Language
en
Pages
13
Series
IEEE Transactions on Power Systems, Volume 39, issue 6, pp. 7064-7076
Abstract
With the ever-escalating scale of urban distribution networks (UDNs), the traditional model-based reconfiguration methods are becoming inadequate for smart system control. On the contrary, the data-driven deep reinforcement learning method can facilitate the swift decision-making but the large action space would adversely affect the learning performance of its agents. Consequently, this paper presents a novel multi-agent deep reinforcement learning method for the reconfiguration of UDNs by introducing the concept of 'switch contribution'. First, a quantification method is proposed based on the mathematical UDN reconfiguration model. The contributions of controllable switches are effective quantified. By excluding the controllable switches with low contributions during network reconfiguration, the dimensionality of action space can be significantly reduced. Then, an improved QMIX algorithm is introduced to improve the policy of multiple agents by assigning the weights. Besides, a novel two-stage learning structure based on a reward-sharing mechanism is presented to further decompose tasks and enhance the learning efficiency of multiple agents. In the first stage, agents control the switches with higher contributions while switches with lower contributions will be controlled in the second stage. During the two-stage process, the proposed reward-sharing mechanism could guarantee a reliable UND reconfiguration and the convergence of our learning method. Finally, numerical results based on a practical 297-node system are performed to validate our method's effectiveness.Description
Publisher Copyright: IEEE
Keywords
Aerospace electronics, Control systems, Deep reinforcement learning, Distribution networks, enhanced QMIX algorithm, multi-agent deep reinforcement learning (MADRL), reconfiguration, Substations, switch contribution, Switches, two-stage learning structure, Urban distribution network (UDN), Voltage
Other note
Citation
Gao, H, Jiang, S, Li, Z, Wang, R, Liu, Y & Liu, J 2024, ' A Two-stage Multi-agent Deep Reinforcement Learning Method for Urban Distribution Network Reconfiguration Considering Switch Contribution ', IEEE Transactions on Power Systems, vol. 39, no. 6, pp. 7064-7076 . https://doi.org/10.1109/TPWRS.2024.3371093