Peer-to-peer(P2P)energy trading in active distribution networks(ADNs)plays a pivotal role in promoting the efficient consumption of renewable energy sources.However,it is challenging to effectively coordinate the powe...Peer-to-peer(P2P)energy trading in active distribution networks(ADNs)plays a pivotal role in promoting the efficient consumption of renewable energy sources.However,it is challenging to effectively coordinate the power dispatch of ADNs and P2P energy trading while preserving the privacy of different physical interests.Hence,this paper proposes a soft actor-critic algorithm incorporating distributed trading control(SAC-DTC)to tackle the optimal power dispatch of ADNs and the P2P energy trading considering privacy preservation among prosumers.First,the soft actor-critic(SAC)algorithm is used to optimize the control strategy of device in ADNs to minimize the operation cost,and the primary environmental information of the ADN at this point is published to prosumers.Then,a distributed generalized fast dual ascent method is used to iterate the trading process of prosumers and maximize their revenues.Subsequently,the results of trading are encrypted based on the differential privacy technique and returned to the ADN.Finally,the social welfare value consisting of ADN operation cost and P2P market revenue is utilized as a reward value to update network parameters and control strategies of the deep reinforcement learning.Simulation results show that the proposed SAC-DTC algorithm reduces the ADN operation cost,boosts the P2P market revenue,maximizes the social welfare,and exhibits high computational accuracy,demonstrating its practical application to the operation of power systems and power markets.展开更多
虚拟电厂(virtual power plant,VPP)作为多能流互联的综合能源网络,已成为中国加速实现双碳目标的重要角色。但VPP内部资源协同低碳调度面临多能流的耦合程度紧密、传统碳交易模型参数主观性强、含高维动态参数的优化目标在线求解困难...虚拟电厂(virtual power plant,VPP)作为多能流互联的综合能源网络,已成为中国加速实现双碳目标的重要角色。但VPP内部资源协同低碳调度面临多能流的耦合程度紧密、传统碳交易模型参数主观性强、含高维动态参数的优化目标在线求解困难等问题。针对这些问题,文中提出一种融合注意力机制(attention mechanism,AM)与柔性动作评价(soft actor-critic,SAC)算法的VPP多能流低碳调度方法。首先,根据VPP的随机碳流特性,面向动态参数建立基于贝叶斯优化的改进阶梯型碳交易机制。接着,以经济效益和碳排放量为目标函数构建含氢VPP多能流解耦模型。然后,考虑到该模型具有高维非线性与权重参数实时更新的特征,利用融合AM的改进SAC深度强化学习算法在连续动作空间对模型进行求解。最后,对多能流调度结果进行仿真分析和对比实验,验证了文中方法的可行性及其相较于原SAC算法较高的决策准确性。展开更多
为解决由于固定温度SAC(Soft Actor Critic)算法中存在的Q函数高估可能会导致算法陷入局部最优的问题,通过深入分析提出了一个稳定且受限的SAC算法(SCSAC:Stable Constrained Soft Actor Critic)。该算法通过改进最大熵目标函数修复固...为解决由于固定温度SAC(Soft Actor Critic)算法中存在的Q函数高估可能会导致算法陷入局部最优的问题,通过深入分析提出了一个稳定且受限的SAC算法(SCSAC:Stable Constrained Soft Actor Critic)。该算法通过改进最大熵目标函数修复固定温度SAC算法中的Q函数高估问题,同时增强算法在测试过程中稳定性的效果。最后,在4个OpenAI Gym Mujoco环境下对SCSAC算法进行了验证,实验结果表明,稳定且受限的SAC算法相比固定温度SAC算法可以有效减小Q函数高估出现的次数并能在测试中获得更加稳定的结果。展开更多
With the increasing integration of large-scale distributed energy resources into the grid,traditional distribution network optimization and dispatch methods struggle to address the challenges posed by both generation ...With the increasing integration of large-scale distributed energy resources into the grid,traditional distribution network optimization and dispatch methods struggle to address the challenges posed by both generation and load.Accounting for these issues,this paper proposes a multi-timescale coordinated optimization dispatch method for distribution networks.First,the probability box theory was employed to determine the uncertainty intervals of generation and load forecasts,based on which,the requirements for flexibility dispatch and capacity constraints of the grid were calculated and analyzed.Subsequently,a multi-timescale optimization framework was constructed,incorporating the generation and load forecast uncertainties.This framework included optimization models for dayahead scheduling,intra-day optimization,and real-time adjustments,aiming to meet flexibility needs across different timescales and improve the economic efficiency of the grid.Furthermore,an improved soft actor-critic algorithm was introduced to enhance the uncertainty exploration capability.Utilizing a centralized training and decentralized execution framework,a multi-agent SAC network model was developed to improve the decision-making efficiency of the agents.Finally,the effectiveness and superiority of the proposed method were validated using a modified IEEE-33 bus test system.展开更多
基金supported by the National Natural Science Foundation of China(No.52177085).
文摘Peer-to-peer(P2P)energy trading in active distribution networks(ADNs)plays a pivotal role in promoting the efficient consumption of renewable energy sources.However,it is challenging to effectively coordinate the power dispatch of ADNs and P2P energy trading while preserving the privacy of different physical interests.Hence,this paper proposes a soft actor-critic algorithm incorporating distributed trading control(SAC-DTC)to tackle the optimal power dispatch of ADNs and the P2P energy trading considering privacy preservation among prosumers.First,the soft actor-critic(SAC)algorithm is used to optimize the control strategy of device in ADNs to minimize the operation cost,and the primary environmental information of the ADN at this point is published to prosumers.Then,a distributed generalized fast dual ascent method is used to iterate the trading process of prosumers and maximize their revenues.Subsequently,the results of trading are encrypted based on the differential privacy technique and returned to the ADN.Finally,the social welfare value consisting of ADN operation cost and P2P market revenue is utilized as a reward value to update network parameters and control strategies of the deep reinforcement learning.Simulation results show that the proposed SAC-DTC algorithm reduces the ADN operation cost,boosts the P2P market revenue,maximizes the social welfare,and exhibits high computational accuracy,demonstrating its practical application to the operation of power systems and power markets.
文摘虚拟电厂(virtual power plant,VPP)作为多能流互联的综合能源网络,已成为中国加速实现双碳目标的重要角色。但VPP内部资源协同低碳调度面临多能流的耦合程度紧密、传统碳交易模型参数主观性强、含高维动态参数的优化目标在线求解困难等问题。针对这些问题,文中提出一种融合注意力机制(attention mechanism,AM)与柔性动作评价(soft actor-critic,SAC)算法的VPP多能流低碳调度方法。首先,根据VPP的随机碳流特性,面向动态参数建立基于贝叶斯优化的改进阶梯型碳交易机制。接着,以经济效益和碳排放量为目标函数构建含氢VPP多能流解耦模型。然后,考虑到该模型具有高维非线性与权重参数实时更新的特征,利用融合AM的改进SAC深度强化学习算法在连续动作空间对模型进行求解。最后,对多能流调度结果进行仿真分析和对比实验,验证了文中方法的可行性及其相较于原SAC算法较高的决策准确性。
文摘为解决由于固定温度SAC(Soft Actor Critic)算法中存在的Q函数高估可能会导致算法陷入局部最优的问题,通过深入分析提出了一个稳定且受限的SAC算法(SCSAC:Stable Constrained Soft Actor Critic)。该算法通过改进最大熵目标函数修复固定温度SAC算法中的Q函数高估问题,同时增强算法在测试过程中稳定性的效果。最后,在4个OpenAI Gym Mujoco环境下对SCSAC算法进行了验证,实验结果表明,稳定且受限的SAC算法相比固定温度SAC算法可以有效减小Q函数高估出现的次数并能在测试中获得更加稳定的结果。
基金funded by Jilin Province Science and Technology Development Plan Project,grant number 20220203163SF.
文摘With the increasing integration of large-scale distributed energy resources into the grid,traditional distribution network optimization and dispatch methods struggle to address the challenges posed by both generation and load.Accounting for these issues,this paper proposes a multi-timescale coordinated optimization dispatch method for distribution networks.First,the probability box theory was employed to determine the uncertainty intervals of generation and load forecasts,based on which,the requirements for flexibility dispatch and capacity constraints of the grid were calculated and analyzed.Subsequently,a multi-timescale optimization framework was constructed,incorporating the generation and load forecast uncertainties.This framework included optimization models for dayahead scheduling,intra-day optimization,and real-time adjustments,aiming to meet flexibility needs across different timescales and improve the economic efficiency of the grid.Furthermore,an improved soft actor-critic algorithm was introduced to enhance the uncertainty exploration capability.Utilizing a centralized training and decentralized execution framework,a multi-agent SAC network model was developed to improve the decision-making efficiency of the agents.Finally,the effectiveness and superiority of the proposed method were validated using a modified IEEE-33 bus test system.
