The integration of large-scale-distributed new energy resources has led to heightened source‒load uncertainty.As energy prosumers,microgrids urgently require enhanced real-time regulation capabilities over controllabl...The integration of large-scale-distributed new energy resources has led to heightened source‒load uncertainty.As energy prosumers,microgrids urgently require enhanced real-time regulation capabilities over controllable resources amid uncertain environments,rendering real-time and rapid decision-making a critical issue.This paper proposes a tailored twin delayed deep deterministic policy gradient(TD3)reinforcement learning algorithm that explicitly accounts for source‒load uncertainty.First,following an expert experience-based methodology,Gaussian process regression was implemented using the radial basis function covariance with historical source and load data.The parameters were adaptively adjusted by maximum likelihood estimation to generate the expected curves of demand and wind‒solar power generation,along with their 95%confidence regions,which were treated as representative uncertainty scenarios.Second,the traditional scheduling model was transformed into a deep reinforcement learning(DRL)environment through a Markov process.To minimize the total operational cost of the microgrid,the tailored TD3 algorithm was applied to formulate rapid intraday scheduling decisions.Finally,simulations were conducted using real historical data from an actual region in Zhejiang province,China,to verify the efficacy of the proposed method.The results demonstrate the potential of the algorithm for achieving economic scheduling for microgrids.展开更多
面对多障碍、大尺寸障碍、狭窄通道等特殊环境下的USV路径规划问题,快速扩展随机树算法(rapidly-exploring random trees,RRT)存在采样基数大、规划成功率低、规划路径曲折等缺点。基于双延迟深度确定性策略梯度(twin delayed deep dete...面对多障碍、大尺寸障碍、狭窄通道等特殊环境下的USV路径规划问题,快速扩展随机树算法(rapidly-exploring random trees,RRT)存在采样基数大、规划成功率低、规划路径曲折等缺点。基于双延迟深度确定性策略梯度(twin delayed deep deterministic policy gradient,TD3)提出一种全局路径规划算法(TD3-RRT)。结合RRT算法与深度强化学习建立USV路径搜索模型,利用前视探测感知环境以自适应调整扩展步长,通过策略网络输出路径搜索方向,解决RRT算法扩展盲目的问题;改进后见经验回放策略,通过重选虚拟目标、双经验回放池采样等策略以增强复杂环境下路径搜索能力;通过奖励函数提高规划路径质量,加快路径搜索速度。实验结果表明:不同环境下TD3-RRT相比当前主流算法能够有效提高规划成功率,优化转向角度、路径长度和规划时间,证明了改进算法能有效加快路径搜索速度并提高路径质量,且对不同环境具有良好适应性。展开更多
传统的PID(proportional integral differential)算法在用于控制一些模型复杂、参数时变的对象时存在参数整定过程繁琐、控制性能不佳、无法解决控制对象实时变化状态的影响等问题。针对上述问题,提出了一种基于双延迟深度确定性策略梯...传统的PID(proportional integral differential)算法在用于控制一些模型复杂、参数时变的对象时存在参数整定过程繁琐、控制性能不佳、无法解决控制对象实时变化状态的影响等问题。针对上述问题,提出了一种基于双延迟深度确定性策略梯度(twin delayed deep deterministic policy gradient,TDDDPG,以下简称TD3)算法的PID参数自整定算法。该算法将TD3算法与PID算法相结合,对TD3算法中的神经网络结构、奖励函数进行设计,能够实现控制器参数的自整定。以两轮直立车为实验对象,针对直立车的角度PID控制器进行参数整定实验。实验结果表明,与传统的参数整定算法(Z-N(Ziegler-Nichols)参数整定法)和基于强化学习的动态PID参数自整定算法相比,所提出的算法具有更优的控制效果,能够通过神经网络学习拟合更优的控制策略,提升控制器的动态响应性能和鲁棒性。展开更多
基金supported in part by Science and Technology Project of State Grid Corporation of China(No.5400-202319829A-4-1-KJ).
文摘The integration of large-scale-distributed new energy resources has led to heightened source‒load uncertainty.As energy prosumers,microgrids urgently require enhanced real-time regulation capabilities over controllable resources amid uncertain environments,rendering real-time and rapid decision-making a critical issue.This paper proposes a tailored twin delayed deep deterministic policy gradient(TD3)reinforcement learning algorithm that explicitly accounts for source‒load uncertainty.First,following an expert experience-based methodology,Gaussian process regression was implemented using the radial basis function covariance with historical source and load data.The parameters were adaptively adjusted by maximum likelihood estimation to generate the expected curves of demand and wind‒solar power generation,along with their 95%confidence regions,which were treated as representative uncertainty scenarios.Second,the traditional scheduling model was transformed into a deep reinforcement learning(DRL)environment through a Markov process.To minimize the total operational cost of the microgrid,the tailored TD3 algorithm was applied to formulate rapid intraday scheduling decisions.Finally,simulations were conducted using real historical data from an actual region in Zhejiang province,China,to verify the efficacy of the proposed method.The results demonstrate the potential of the algorithm for achieving economic scheduling for microgrids.
文摘面对多障碍、大尺寸障碍、狭窄通道等特殊环境下的USV路径规划问题,快速扩展随机树算法(rapidly-exploring random trees,RRT)存在采样基数大、规划成功率低、规划路径曲折等缺点。基于双延迟深度确定性策略梯度(twin delayed deep deterministic policy gradient,TD3)提出一种全局路径规划算法(TD3-RRT)。结合RRT算法与深度强化学习建立USV路径搜索模型,利用前视探测感知环境以自适应调整扩展步长,通过策略网络输出路径搜索方向,解决RRT算法扩展盲目的问题;改进后见经验回放策略,通过重选虚拟目标、双经验回放池采样等策略以增强复杂环境下路径搜索能力;通过奖励函数提高规划路径质量,加快路径搜索速度。实验结果表明:不同环境下TD3-RRT相比当前主流算法能够有效提高规划成功率,优化转向角度、路径长度和规划时间,证明了改进算法能有效加快路径搜索速度并提高路径质量,且对不同环境具有良好适应性。
文摘传统的PID(proportional integral differential)算法在用于控制一些模型复杂、参数时变的对象时存在参数整定过程繁琐、控制性能不佳、无法解决控制对象实时变化状态的影响等问题。针对上述问题,提出了一种基于双延迟深度确定性策略梯度(twin delayed deep deterministic policy gradient,TDDDPG,以下简称TD3)算法的PID参数自整定算法。该算法将TD3算法与PID算法相结合,对TD3算法中的神经网络结构、奖励函数进行设计,能够实现控制器参数的自整定。以两轮直立车为实验对象,针对直立车的角度PID控制器进行参数整定实验。实验结果表明,与传统的参数整定算法(Z-N(Ziegler-Nichols)参数整定法)和基于强化学习的动态PID参数自整定算法相比,所提出的算法具有更优的控制效果,能够通过神经网络学习拟合更优的控制策略,提升控制器的动态响应性能和鲁棒性。
