Trajectory tracking for nonlinear robotic systems remains a fundamental yet challenging problem in control engineering,particularly when both precision and efficiency must be ensured.Conventional control methods are o...Trajectory tracking for nonlinear robotic systems remains a fundamental yet challenging problem in control engineering,particularly when both precision and efficiency must be ensured.Conventional control methods are often effective for stabilization but may not directly optimize long-term performance.To address this limitation,this study develops an integrated framework that combines optimal control principles with reinforcement learning for a single-link robotic manipulator.The proposed scheme adopts an actor–critic structure,where the critic network approximates the value function associated with the Hamilton–Jacobi–Bellman equation,and the actor network generates near-optimal control signals in real time.This dual adaptation enables the controller to refine its policy online without explicit system knowledge.Stability of the closed-loop system is analyzed through Lyapunov theory,ensuring boundedness of the tracking error.Numerical simulations on the single-link manipulator demonstrate that themethod achieves accurate trajectory followingwhile maintaining lowcontrol effort.The results further showthat the actor–critic learning mechanism accelerates convergence of the control policy compared with conventional optimization-based strategies.This work highlights the potential of reinforcement learning integrated with optimal control for robotic manipulators and provides a foundation for future extensions to more complex multi-degree-of-freedom systems.The proposed controller is further validated in a physics-based virtual Gazebo environment,demonstrating stable adaptation and real-time feasibility.展开更多
随着可再生能源并网发电量的不断增加,由电力电子设备引发的电力系统次同步振荡问题逐渐凸显,给电力系统的安全稳定运行带来了新的挑战。除此之外,当目标电力系统规模较大时,常用的基于线性化模型的分析方法面临着维数灾难。为了解决上...随着可再生能源并网发电量的不断增加,由电力电子设备引发的电力系统次同步振荡问题逐渐凸显,给电力系统的安全稳定运行带来了新的挑战。除此之外,当目标电力系统规模较大时,常用的基于线性化模型的分析方法面临着维数灾难。为了解决上述问题,根据强化学习原理,通过动作-评价(Actor-Critic)学习框架提出一种对风机换流器控制参数的优化策略。通过搜集永磁直驱风机(permanent magnetic synchronous generator,PMSG)运行状态数据,训练强化学习代理(Agent),以此评估风机运行状态及其稳定性,并寻找优化风机换流器参数的最优策略。该训练方法得到的代理能够基于时域采样数据对风机换流器参数进行优化,从而有效抑制由于换流器诱发的振荡现象,在没有建立线性化分析模型的情况下,能够有效优化并增强电力系统的稳定性。经实验验证,该优化策略在采样数据有噪声干扰的情况下仍然具有良好的优化性能。展开更多
Objective: To know about the current situation of hypertension in some areas of Hubei Province and analyze the influencing factors. Methods: According to the principle of Stratified sampling, we conducted health exami...Objective: To know about the current situation of hypertension in some areas of Hubei Province and analyze the influencing factors. Methods: According to the principle of Stratified sampling, we conducted health examination and questionnaire survey for 1500 residents over the age of 18 at observation point of chronic noninfectious diseases at 10 sub-districts (towns) of Wuhan City, Jingzhou City, Huanggang City, Shiyan City, which used descriptive statistics and logistics to regressively analyze Current situation of hypertension and its influencing factors of residents. Results: Prevalence of hypertension of rural and urban residents over 18 is 27.44% in Hubei province. There are many differences among prevalence of hypertension of male and female, distribution of BMI of rural and urban residents and prevalence of hypertension and so on, and it has statistical significance (P-value < 0.05 averagely);logistic regressive analysis result shows that different age, gender, education level, dieting habits (high salt and high oil), family per capita monthly income, BMI have statistical significance on the prevalence of hypertension in urban and rural residents of Hubei Province. Conclusions: The prevalence of hypertension in the residents of five cities and prefectures in Hubei Province is on the trend of rising in ladder form and at a much earlier age. The health education, monitoring and intervention of chronic diseases need to be widely carried out, with emphasis on the intervention of the residents’ eating habits of high salt and oil, smoking, drinking and other bad lifestyle.展开更多
基金supported in part by the National Science and Technology Council under Grant NSTC 114-2221-E-027-104.
文摘Trajectory tracking for nonlinear robotic systems remains a fundamental yet challenging problem in control engineering,particularly when both precision and efficiency must be ensured.Conventional control methods are often effective for stabilization but may not directly optimize long-term performance.To address this limitation,this study develops an integrated framework that combines optimal control principles with reinforcement learning for a single-link robotic manipulator.The proposed scheme adopts an actor–critic structure,where the critic network approximates the value function associated with the Hamilton–Jacobi–Bellman equation,and the actor network generates near-optimal control signals in real time.This dual adaptation enables the controller to refine its policy online without explicit system knowledge.Stability of the closed-loop system is analyzed through Lyapunov theory,ensuring boundedness of the tracking error.Numerical simulations on the single-link manipulator demonstrate that themethod achieves accurate trajectory followingwhile maintaining lowcontrol effort.The results further showthat the actor–critic learning mechanism accelerates convergence of the control policy compared with conventional optimization-based strategies.This work highlights the potential of reinforcement learning integrated with optimal control for robotic manipulators and provides a foundation for future extensions to more complex multi-degree-of-freedom systems.The proposed controller is further validated in a physics-based virtual Gazebo environment,demonstrating stable adaptation and real-time feasibility.
文摘提出一种基于模糊RBF网络的自适应模糊A ctor-C ritic学习.采用一个模糊RBF神经网络同时逼近A ctor的动作函数和C ritic的值函数,解决状态空间泛化中易出现的“维数灾”问题.模糊RBF网络能够根据环境状态和被控对象特性的变化进行网络结构和参数的自适应学习,使得网络结构更加紧凑,整个模糊A ctor-C ritic学习具有泛化性能好、控制结构简单和学习效率高的特点.M oun ta in C ar的仿真结果验证了所提方法的有效性.
文摘随着可再生能源并网发电量的不断增加,由电力电子设备引发的电力系统次同步振荡问题逐渐凸显,给电力系统的安全稳定运行带来了新的挑战。除此之外,当目标电力系统规模较大时,常用的基于线性化模型的分析方法面临着维数灾难。为了解决上述问题,根据强化学习原理,通过动作-评价(Actor-Critic)学习框架提出一种对风机换流器控制参数的优化策略。通过搜集永磁直驱风机(permanent magnetic synchronous generator,PMSG)运行状态数据,训练强化学习代理(Agent),以此评估风机运行状态及其稳定性,并寻找优化风机换流器参数的最优策略。该训练方法得到的代理能够基于时域采样数据对风机换流器参数进行优化,从而有效抑制由于换流器诱发的振荡现象,在没有建立线性化分析模型的情况下,能够有效优化并增强电力系统的稳定性。经实验验证,该优化策略在采样数据有噪声干扰的情况下仍然具有良好的优化性能。
文摘Objective: To know about the current situation of hypertension in some areas of Hubei Province and analyze the influencing factors. Methods: According to the principle of Stratified sampling, we conducted health examination and questionnaire survey for 1500 residents over the age of 18 at observation point of chronic noninfectious diseases at 10 sub-districts (towns) of Wuhan City, Jingzhou City, Huanggang City, Shiyan City, which used descriptive statistics and logistics to regressively analyze Current situation of hypertension and its influencing factors of residents. Results: Prevalence of hypertension of rural and urban residents over 18 is 27.44% in Hubei province. There are many differences among prevalence of hypertension of male and female, distribution of BMI of rural and urban residents and prevalence of hypertension and so on, and it has statistical significance (P-value < 0.05 averagely);logistic regressive analysis result shows that different age, gender, education level, dieting habits (high salt and high oil), family per capita monthly income, BMI have statistical significance on the prevalence of hypertension in urban and rural residents of Hubei Province. Conclusions: The prevalence of hypertension in the residents of five cities and prefectures in Hubei Province is on the trend of rising in ladder form and at a much earlier age. The health education, monitoring and intervention of chronic diseases need to be widely carried out, with emphasis on the intervention of the residents’ eating habits of high salt and oil, smoking, drinking and other bad lifestyle.
