The integration of large-scale-distributed new energy resources has led to heightened source‒load uncertainty.As energy prosumers,microgrids urgently require enhanced real-time regulation capabilities over controllabl...The integration of large-scale-distributed new energy resources has led to heightened source‒load uncertainty.As energy prosumers,microgrids urgently require enhanced real-time regulation capabilities over controllable resources amid uncertain environments,rendering real-time and rapid decision-making a critical issue.This paper proposes a tailored twin delayed deep deterministic policy gradient(TD3)reinforcement learning algorithm that explicitly accounts for source‒load uncertainty.First,following an expert experience-based methodology,Gaussian process regression was implemented using the radial basis function covariance with historical source and load data.The parameters were adaptively adjusted by maximum likelihood estimation to generate the expected curves of demand and wind‒solar power generation,along with their 95%confidence regions,which were treated as representative uncertainty scenarios.Second,the traditional scheduling model was transformed into a deep reinforcement learning(DRL)environment through a Markov process.To minimize the total operational cost of the microgrid,the tailored TD3 algorithm was applied to formulate rapid intraday scheduling decisions.Finally,simulations were conducted using real historical data from an actual region in Zhejiang province,China,to verify the efficacy of the proposed method.The results demonstrate the potential of the algorithm for achieving economic scheduling for microgrids.展开更多
面对多障碍、大尺寸障碍、狭窄通道等特殊环境下的USV路径规划问题,快速扩展随机树算法(rapidly-exploring random trees,RRT)存在采样基数大、规划成功率低、规划路径曲折等缺点。基于双延迟深度确定性策略梯度(twin delayed deep dete...面对多障碍、大尺寸障碍、狭窄通道等特殊环境下的USV路径规划问题,快速扩展随机树算法(rapidly-exploring random trees,RRT)存在采样基数大、规划成功率低、规划路径曲折等缺点。基于双延迟深度确定性策略梯度(twin delayed deep deterministic policy gradient,TD3)提出一种全局路径规划算法(TD3-RRT)。结合RRT算法与深度强化学习建立USV路径搜索模型,利用前视探测感知环境以自适应调整扩展步长,通过策略网络输出路径搜索方向,解决RRT算法扩展盲目的问题;改进后见经验回放策略,通过重选虚拟目标、双经验回放池采样等策略以增强复杂环境下路径搜索能力;通过奖励函数提高规划路径质量,加快路径搜索速度。实验结果表明:不同环境下TD3-RRT相比当前主流算法能够有效提高规划成功率,优化转向角度、路径长度和规划时间,证明了改进算法能有效加快路径搜索速度并提高路径质量,且对不同环境具有良好适应性。展开更多
基金supported in part by Science and Technology Project of State Grid Corporation of China(No.5400-202319829A-4-1-KJ).
文摘The integration of large-scale-distributed new energy resources has led to heightened source‒load uncertainty.As energy prosumers,microgrids urgently require enhanced real-time regulation capabilities over controllable resources amid uncertain environments,rendering real-time and rapid decision-making a critical issue.This paper proposes a tailored twin delayed deep deterministic policy gradient(TD3)reinforcement learning algorithm that explicitly accounts for source‒load uncertainty.First,following an expert experience-based methodology,Gaussian process regression was implemented using the radial basis function covariance with historical source and load data.The parameters were adaptively adjusted by maximum likelihood estimation to generate the expected curves of demand and wind‒solar power generation,along with their 95%confidence regions,which were treated as representative uncertainty scenarios.Second,the traditional scheduling model was transformed into a deep reinforcement learning(DRL)environment through a Markov process.To minimize the total operational cost of the microgrid,the tailored TD3 algorithm was applied to formulate rapid intraday scheduling decisions.Finally,simulations were conducted using real historical data from an actual region in Zhejiang province,China,to verify the efficacy of the proposed method.The results demonstrate the potential of the algorithm for achieving economic scheduling for microgrids.
文摘面对多障碍、大尺寸障碍、狭窄通道等特殊环境下的USV路径规划问题,快速扩展随机树算法(rapidly-exploring random trees,RRT)存在采样基数大、规划成功率低、规划路径曲折等缺点。基于双延迟深度确定性策略梯度(twin delayed deep deterministic policy gradient,TD3)提出一种全局路径规划算法(TD3-RRT)。结合RRT算法与深度强化学习建立USV路径搜索模型,利用前视探测感知环境以自适应调整扩展步长,通过策略网络输出路径搜索方向,解决RRT算法扩展盲目的问题;改进后见经验回放策略,通过重选虚拟目标、双经验回放池采样等策略以增强复杂环境下路径搜索能力;通过奖励函数提高规划路径质量,加快路径搜索速度。实验结果表明:不同环境下TD3-RRT相比当前主流算法能够有效提高规划成功率,优化转向角度、路径长度和规划时间,证明了改进算法能有效加快路径搜索速度并提高路径质量,且对不同环境具有良好适应性。
