Trajectory tracking for nonlinear robotic systems remains a fundamental yet challenging problem in control engineering,particularly when both precision and efficiency must be ensured.Conventional control methods are o...Trajectory tracking for nonlinear robotic systems remains a fundamental yet challenging problem in control engineering,particularly when both precision and efficiency must be ensured.Conventional control methods are often effective for stabilization but may not directly optimize long-term performance.To address this limitation,this study develops an integrated framework that combines optimal control principles with reinforcement learning for a single-link robotic manipulator.The proposed scheme adopts an actor–critic structure,where the critic network approximates the value function associated with the Hamilton–Jacobi–Bellman equation,and the actor network generates near-optimal control signals in real time.This dual adaptation enables the controller to refine its policy online without explicit system knowledge.Stability of the closed-loop system is analyzed through Lyapunov theory,ensuring boundedness of the tracking error.Numerical simulations on the single-link manipulator demonstrate that themethod achieves accurate trajectory followingwhile maintaining lowcontrol effort.The results further showthat the actor–critic learning mechanism accelerates convergence of the control policy compared with conventional optimization-based strategies.This work highlights the potential of reinforcement learning integrated with optimal control for robotic manipulators and provides a foundation for future extensions to more complex multi-degree-of-freedom systems.The proposed controller is further validated in a physics-based virtual Gazebo environment,demonstrating stable adaptation and real-time feasibility.展开更多
The cemented tailings backfill(CTB)with initial defects is more prone to destabilization damage under the influence of various unfavorable factors during the mining process.In order to investigate its influence on the...The cemented tailings backfill(CTB)with initial defects is more prone to destabilization damage under the influence of various unfavorable factors during the mining process.In order to investigate its influence on the stability of underground mining engineering,this paper simulates the generation of different degrees of initial defects inside the CTB by adding different contents of air-entraining agent(AEA),investigates the acoustic emission RA/AF eigenvalues of CTB with different contents of AEA under uniaxial compression,and adopts various denoising algorithms(e.g.,moving average smoothing,median filtering,and outlier detection)to improve the accuracy of the data.The variance and autocorrelation coefficients of RA/AF parameters were analyzed in conjunction with the critical slowing down(CSD)theory.The results show that the acoustic emission RA/AF values can be used to characterize the progressive damage evolution of CTB.The denoising algorithm processed the AE signals to reduce the effects of extraneous noise and anomalous spikes.Changes in the variance curves provide clear precursor information,while abrupt changes in the autocorrelation coefficient can be used as an auxiliary localization warning signal.The phenomenon of dramatic increase in the variance and autocorrelation coefficient curves during the compression-tightening stage,which is influenced by the initial defects,can lead to false warnings.As the initial defects of the CTB increase,its instability precursor time and instability time are prolonged,the peak stress decreases,and the time difference between the CTB and the instability damage is smaller.The results provide a new method for real-time monitoring and early warning of CTB instability damage.展开更多
This paper describes a real-time beam tuning method with an improved asynchronous advantage actor–critic(A3C)algorithm for accelerator systems.The operating parameters of devices are usually inconsistent with the pre...This paper describes a real-time beam tuning method with an improved asynchronous advantage actor–critic(A3C)algorithm for accelerator systems.The operating parameters of devices are usually inconsistent with the predictions of physical designs because of errors in mechanical matching and installation.Therefore,parameter optimization methods such as pointwise scanning,evolutionary algorithms(EAs),and robust conjugate direction search are widely used in beam tuning to compensate for this inconsistency.However,it is difficult for them to deal with a large number of discrete local optima.The A3C algorithm,which has been applied in the automated control field,provides an approach for improving multi-dimensional optimization.The A3C algorithm is introduced and improved for the real-time beam tuning code for accelerators.Experiments in which optimization is achieved by using pointwise scanning,the genetic algorithm(one kind of EAs),and the A3C-algorithm are conducted and compared to optimize the currents of four steering magnets and two solenoids in the low-energy beam transport section(LEBT)of the Xi’an Proton Application Facility.Optimal currents are determined when the highest transmission of a radio frequency quadrupole(RFQ)accelerator downstream of the LEBT is achieved.The optimal work points of the tuned accelerator were obtained with currents of 0 A,0 A,0 A,and 0.1 A,for the four steering magnets,and 107 A and 96 A for the two solenoids.Furthermore,the highest transmission of the RFQ was 91.2%.Meanwhile,the lower time required for the optimization with the A3C algorithm was successfully verified.Optimization with the A3C algorithm consumed 42%and 78%less time than pointwise scanning with random initialization and pre-trained initialization of weights,respectively.展开更多
随着可再生能源并网发电量的不断增加,由电力电子设备引发的电力系统次同步振荡问题逐渐凸显,给电力系统的安全稳定运行带来了新的挑战。除此之外,当目标电力系统规模较大时,常用的基于线性化模型的分析方法面临着维数灾难。为了解决上...随着可再生能源并网发电量的不断增加,由电力电子设备引发的电力系统次同步振荡问题逐渐凸显,给电力系统的安全稳定运行带来了新的挑战。除此之外,当目标电力系统规模较大时,常用的基于线性化模型的分析方法面临着维数灾难。为了解决上述问题,根据强化学习原理,通过动作-评价(Actor-Critic)学习框架提出一种对风机换流器控制参数的优化策略。通过搜集永磁直驱风机(permanent magnetic synchronous generator,PMSG)运行状态数据,训练强化学习代理(Agent),以此评估风机运行状态及其稳定性,并寻找优化风机换流器参数的最优策略。该训练方法得到的代理能够基于时域采样数据对风机换流器参数进行优化,从而有效抑制由于换流器诱发的振荡现象,在没有建立线性化分析模型的情况下,能够有效优化并增强电力系统的稳定性。经实验验证,该优化策略在采样数据有噪声干扰的情况下仍然具有良好的优化性能。展开更多
The dual challenges of critical speed prediction inaccuracies and ambiguous vibration behaviors are present in high-speed flexible rotors,particularly in free turbine rotors in turboshaft engine systems.The study begi...The dual challenges of critical speed prediction inaccuracies and ambiguous vibration behaviors are present in high-speed flexible rotors,particularly in free turbine rotors in turboshaft engine systems.The study begins with an examination of the rotor-bearing bidirectional coupling mechanism,with a primary focus on the nonlinear characteristics of the bearing.An investigation is carried out on the mechanical modeling methodologies for four-point contact ball bearings(FPCBBs)and cylindrical roller bearings(CRBs).To address the issue of excessive computational time in traditional bearing calculation methods,the sled dog optimization(SDO)algorithm is substituted for the conventional Newton-Raphson method.A rotor-bearing coupling dynamics model is developed by the finite element and lumped mass methods,with experimental validation achieved through a simulator test rig.The effects of three internal bearing parameters in FPCBBs(arching width and raceway groove curvature coefficient)and CRBs(initial radial clearance)on the critical speed characteristics and vibrational behavior of rotorbearing coupled systems are examined.The numerical simulation results show some interesting conclusions.展开更多
基金supported in part by the National Science and Technology Council under Grant NSTC 114-2221-E-027-104.
文摘Trajectory tracking for nonlinear robotic systems remains a fundamental yet challenging problem in control engineering,particularly when both precision and efficiency must be ensured.Conventional control methods are often effective for stabilization but may not directly optimize long-term performance.To address this limitation,this study develops an integrated framework that combines optimal control principles with reinforcement learning for a single-link robotic manipulator.The proposed scheme adopts an actor–critic structure,where the critic network approximates the value function associated with the Hamilton–Jacobi–Bellman equation,and the actor network generates near-optimal control signals in real time.This dual adaptation enables the controller to refine its policy online without explicit system knowledge.Stability of the closed-loop system is analyzed through Lyapunov theory,ensuring boundedness of the tracking error.Numerical simulations on the single-link manipulator demonstrate that themethod achieves accurate trajectory followingwhile maintaining lowcontrol effort.The results further showthat the actor–critic learning mechanism accelerates convergence of the control policy compared with conventional optimization-based strategies.This work highlights the potential of reinforcement learning integrated with optimal control for robotic manipulators and provides a foundation for future extensions to more complex multi-degree-of-freedom systems.The proposed controller is further validated in a physics-based virtual Gazebo environment,demonstrating stable adaptation and real-time feasibility.
基金Projects(52374138,51764013)supported by the National Natural Science Foundation of ChinaProject(20204BCJ22005)supported by the Training Plan for Academic and Technical Leaders of Major Disciplines of Jiangxi Province,China+1 种基金Project(2019M652277)supported by the China Postdoctoral Science FoundationProject(20192ACBL21014)supported by the Natural Science Youth Foundation Key Projects of Jiangxi Province,China。
文摘The cemented tailings backfill(CTB)with initial defects is more prone to destabilization damage under the influence of various unfavorable factors during the mining process.In order to investigate its influence on the stability of underground mining engineering,this paper simulates the generation of different degrees of initial defects inside the CTB by adding different contents of air-entraining agent(AEA),investigates the acoustic emission RA/AF eigenvalues of CTB with different contents of AEA under uniaxial compression,and adopts various denoising algorithms(e.g.,moving average smoothing,median filtering,and outlier detection)to improve the accuracy of the data.The variance and autocorrelation coefficients of RA/AF parameters were analyzed in conjunction with the critical slowing down(CSD)theory.The results show that the acoustic emission RA/AF values can be used to characterize the progressive damage evolution of CTB.The denoising algorithm processed the AE signals to reduce the effects of extraneous noise and anomalous spikes.Changes in the variance curves provide clear precursor information,while abrupt changes in the autocorrelation coefficient can be used as an auxiliary localization warning signal.The phenomenon of dramatic increase in the variance and autocorrelation coefficient curves during the compression-tightening stage,which is influenced by the initial defects,can lead to false warnings.As the initial defects of the CTB increase,its instability precursor time and instability time are prolonged,the peak stress decreases,and the time difference between the CTB and the instability damage is smaller.The results provide a new method for real-time monitoring and early warning of CTB instability damage.
文摘This paper describes a real-time beam tuning method with an improved asynchronous advantage actor–critic(A3C)algorithm for accelerator systems.The operating parameters of devices are usually inconsistent with the predictions of physical designs because of errors in mechanical matching and installation.Therefore,parameter optimization methods such as pointwise scanning,evolutionary algorithms(EAs),and robust conjugate direction search are widely used in beam tuning to compensate for this inconsistency.However,it is difficult for them to deal with a large number of discrete local optima.The A3C algorithm,which has been applied in the automated control field,provides an approach for improving multi-dimensional optimization.The A3C algorithm is introduced and improved for the real-time beam tuning code for accelerators.Experiments in which optimization is achieved by using pointwise scanning,the genetic algorithm(one kind of EAs),and the A3C-algorithm are conducted and compared to optimize the currents of four steering magnets and two solenoids in the low-energy beam transport section(LEBT)of the Xi’an Proton Application Facility.Optimal currents are determined when the highest transmission of a radio frequency quadrupole(RFQ)accelerator downstream of the LEBT is achieved.The optimal work points of the tuned accelerator were obtained with currents of 0 A,0 A,0 A,and 0.1 A,for the four steering magnets,and 107 A and 96 A for the two solenoids.Furthermore,the highest transmission of the RFQ was 91.2%.Meanwhile,the lower time required for the optimization with the A3C algorithm was successfully verified.Optimization with the A3C algorithm consumed 42%and 78%less time than pointwise scanning with random initialization and pre-trained initialization of weights,respectively.
文摘随着可再生能源并网发电量的不断增加,由电力电子设备引发的电力系统次同步振荡问题逐渐凸显,给电力系统的安全稳定运行带来了新的挑战。除此之外,当目标电力系统规模较大时,常用的基于线性化模型的分析方法面临着维数灾难。为了解决上述问题,根据强化学习原理,通过动作-评价(Actor-Critic)学习框架提出一种对风机换流器控制参数的优化策略。通过搜集永磁直驱风机(permanent magnetic synchronous generator,PMSG)运行状态数据,训练强化学习代理(Agent),以此评估风机运行状态及其稳定性,并寻找优化风机换流器参数的最优策略。该训练方法得到的代理能够基于时域采样数据对风机换流器参数进行优化,从而有效抑制由于换流器诱发的振荡现象,在没有建立线性化分析模型的情况下,能够有效优化并增强电力系统的稳定性。经实验验证,该优化策略在采样数据有噪声干扰的情况下仍然具有良好的优化性能。
文摘The dual challenges of critical speed prediction inaccuracies and ambiguous vibration behaviors are present in high-speed flexible rotors,particularly in free turbine rotors in turboshaft engine systems.The study begins with an examination of the rotor-bearing bidirectional coupling mechanism,with a primary focus on the nonlinear characteristics of the bearing.An investigation is carried out on the mechanical modeling methodologies for four-point contact ball bearings(FPCBBs)and cylindrical roller bearings(CRBs).To address the issue of excessive computational time in traditional bearing calculation methods,the sled dog optimization(SDO)algorithm is substituted for the conventional Newton-Raphson method.A rotor-bearing coupling dynamics model is developed by the finite element and lumped mass methods,with experimental validation achieved through a simulator test rig.The effects of three internal bearing parameters in FPCBBs(arching width and raceway groove curvature coefficient)and CRBs(initial radial clearance)on the critical speed characteristics and vibrational behavior of rotorbearing coupled systems are examined.The numerical simulation results show some interesting conclusions.
文摘提出一种基于模糊RBF网络的自适应模糊A ctor-C ritic学习.采用一个模糊RBF神经网络同时逼近A ctor的动作函数和C ritic的值函数,解决状态空间泛化中易出现的“维数灾”问题.模糊RBF网络能够根据环境状态和被控对象特性的变化进行网络结构和参数的自适应学习,使得网络结构更加紧凑,整个模糊A ctor-C ritic学习具有泛化性能好、控制结构简单和学习效率高的特点.M oun ta in C ar的仿真结果验证了所提方法的有效性.
