The integration of distributed energy resources(DERs)has escalated the challenge of voltage magnitude regulation in distribution networks.Model-based approaches,which rely on complex sequential mathematical formulatio...The integration of distributed energy resources(DERs)has escalated the challenge of voltage magnitude regulation in distribution networks.Model-based approaches,which rely on complex sequential mathematical formulations,cannot meet the real-time demand.Deep reinforcement learning(DRL)offers an alternative by utilizing offline training with distribution network simulators and then executing online without computation.However,DRL algorithms fail to enforce voltage magnitude constraints during training and testing,potentially leading to serious operational violations.To tackle these challenges,we introduce a novel safe-guaranteed reinforcement learning algorithm,the Dist Flow safe reinforcement learning(DF-SRL),designed specifically for real-time voltage magnitude regulation in distribution networks.The DF-SRL algorithm incorporates a Dist Flow linearization to construct an expert-knowledge-based safety layer.Subsequently,the DF-SRL algorithm overlays this safety layer on top of the agent policy,recalibrating unsafe actions to safe domains through a quadratic programming formulation.Simulation results show the DF-SRL algorithm consistently ensures voltage magnitude constraints during training and real-time operation(test)phases,achieving faster convergence and higher performance,which differentiates it apart from(safe)DRL benchmark algorithms.展开更多
基金part of the DATALESs project(with project number 482.20.602)jointly financed by the Netherlands Organization for Scientific Research(NWO)the National Natural Science Foundation of China(NSFC)。
文摘The integration of distributed energy resources(DERs)has escalated the challenge of voltage magnitude regulation in distribution networks.Model-based approaches,which rely on complex sequential mathematical formulations,cannot meet the real-time demand.Deep reinforcement learning(DRL)offers an alternative by utilizing offline training with distribution network simulators and then executing online without computation.However,DRL algorithms fail to enforce voltage magnitude constraints during training and testing,potentially leading to serious operational violations.To tackle these challenges,we introduce a novel safe-guaranteed reinforcement learning algorithm,the Dist Flow safe reinforcement learning(DF-SRL),designed specifically for real-time voltage magnitude regulation in distribution networks.The DF-SRL algorithm incorporates a Dist Flow linearization to construct an expert-knowledge-based safety layer.Subsequently,the DF-SRL algorithm overlays this safety layer on top of the agent policy,recalibrating unsafe actions to safe domains through a quadratic programming formulation.Simulation results show the DF-SRL algorithm consistently ensures voltage magnitude constraints during training and real-time operation(test)phases,achieving faster convergence and higher performance,which differentiates it apart from(safe)DRL benchmark algorithms.
