Aiming at the tracking problem of a class of discrete nonaffine nonlinear multi-input multi-output(MIMO) repetitive systems subjected to separable and nonseparable disturbances, a novel data-driven iterative learning ...Aiming at the tracking problem of a class of discrete nonaffine nonlinear multi-input multi-output(MIMO) repetitive systems subjected to separable and nonseparable disturbances, a novel data-driven iterative learning control(ILC) scheme based on the zeroing neural networks(ZNNs) is proposed. First, the equivalent dynamic linearization data model is obtained by means of dynamic linearization technology, which exists theoretically in the iteration domain. Then, the iterative extended state observer(IESO) is developed to estimate the disturbance and the coupling between systems, and the decoupled dynamic linearization model is obtained for the purpose of controller synthesis. To solve the zero-seeking tracking problem with inherent tolerance of noise,an ILC based on noise-tolerant modified ZNN is proposed. The strict assumptions imposed on the initialization conditions of each iteration in the existing ILC methods can be absolutely removed with our method. In addition, theoretical analysis indicates that the modified ZNN can converge to the exact solution of the zero-seeking tracking problem. Finally, a generalized example and an application-oriented example are presented to verify the effectiveness and superiority of the proposed process.展开更多
Norm optimal iterative learning control(NOILC) has recently been applied to iterative learning control(ILC) problems in which tracking is only required at a subset of isolated time points along the trial duration. Thi...Norm optimal iterative learning control(NOILC) has recently been applied to iterative learning control(ILC) problems in which tracking is only required at a subset of isolated time points along the trial duration. This problem addresses the practical needs of many applications, including industrial automation, crane control, satellite positioning and motion control within a medical stroke rehabilitation context. This paper provides a substantial generalization of this framework by providing a solution to the problem of convergence at intermediate points with simultaneous tracking of subsets of outputs to reference trajectories on subintervals. This formulation enables the NOILC paradigm to tackle tasks which mix "point to point" movements with linear tracking requirements and hence substantially broadens the application domain to include automation tasks which include welding or cutting movements, or human motion control where the movement is restricted by the task to straight line and/or planar segments. A solution to the problem is presented in the framework of NOILC and inherits NOILC s well-defined convergence properties. Design guidelines and supporting experimental results are included.展开更多
This paper analyses the concept of a Limit Set in Parameter Optimal Iterative Learning Control （ILC）. We investigate the existence of stable and unstable parts of Limit Set and demonstrates that they will often exis...This paper analyses the concept of a Limit Set in Parameter Optimal Iterative Learning Control （ILC）. We investigate the existence of stable and unstable parts of Limit Set and demonstrates that they will often exist in practice. This is illustrated via a 2-dimensional example where the convergence of the learning algorithm is analyzed from the error＇s dynamic behaviour. These ideas are extended to the N-dimensional cases by analogy and example.展开更多
In this paper, a quasi-Newton-type optimized iterative learning control （ILC） algorithm is investigated for a class of discrete linear time-invariant systems. The proposed learning algorithm is to update the learnin...In this paper, a quasi-Newton-type optimized iterative learning control （ILC） algorithm is investigated for a class of discrete linear time-invariant systems. The proposed learning algorithm is to update the learning gain matrix by a quasi-Newton-type matrix instead of the inversion of the plant. By means of the mathematical inductive method, the monotone convergence of the proposed algorithm is analyzed, which shows that the tracking error monotonously converges to zero after a finite number of iterations. Compared with the existing optimized ILC algorithms, due to the superlinear convergence of quasi-Newton method, the proposed learning law operates with a faster convergent rate and is robust to the ill-condition of the system model, and thus owns a wide range of applications. Numerical simulations demonstrate the validity and effectiveness.展开更多
For unachievable tracking problems, where the system output cannot precisely track a given reference, achieving the best possible approximation for the reference trajectory becomes the objective. This study aims to in...For unachievable tracking problems, where the system output cannot precisely track a given reference, achieving the best possible approximation for the reference trajectory becomes the objective. This study aims to investigate solutions using the Ptype learning control scheme. Initially, we demonstrate the necessity of gradient information for achieving the best approximation.Subsequently, we propose an input-output-driven learning gain design to handle the imprecise gradients of a class of uncertain systems. However, it is discovered that the desired performance may not be attainable when faced with incomplete information.To address this issue, an extended iterative learning control scheme is introduced. In this scheme, the tracking errors are modified through output data sampling, which incorporates lowmemory footprints and offers flexibility in learning gain design.The input sequence is shown to converge towards the desired input, resulting in an output that is closest to the given reference in the least square sense. Numerical simulations are provided to validate the theoretical findings.展开更多
In the procedure of the steady-state hierarchical optimization with feedback for large-scale industrial processes, a sequence of set-point changes with different magnitudes is carried out on the optimization layer. To...In the procedure of the steady-state hierarchical optimization with feedback for large-scale industrial processes, a sequence of set-point changes with different magnitudes is carried out on the optimization layer. To improve the dynamic performance of transient response driven by the set-point changes, a filter-based iterative learning control strategy is proposed. In the proposed updating law, a local-symmetric-integral operator is adopted for eliminating the measurement noise of output information,a set of desired trajectories are specified according to the set-point changes sequence, the current control input is iteratively achieved by utilizing smoothed output error to modify its control input at previous iteration, to which the amplified coefficients related to the different magnitudes of set-point changes are introduced. The convergence of the algorithm is conducted by incorporating frequency-domain technique into time-domain analysis. Numerical simulation demonstrates the effectiveness of the proposed strategy,展开更多
Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and ...Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.展开更多
The goal of this paper is to design a model-free optimal controller for the multirate system based on reinforcement learning.Sampled-data control systems are widely used in the industrial production process and multir...The goal of this paper is to design a model-free optimal controller for the multirate system based on reinforcement learning.Sampled-data control systems are widely used in the industrial production process and multirate sampling has attracted much attention in the study of the sampled-data control theory.In this paper,we assume the sampling periods for state variables are different from periods for system inputs.Under this condition,we can obtain an equivalent discrete-time system using the lifting technique.Then,we provide an algorithm to solve the linear quadratic regulator(LQR)control problem of multirate systems with the utilization of matrix substitutions.Based on a reinforcement learning method,we use online policy iteration and off-policy algorithms to optimize the controller for multirate systems.By using the least squares method,we convert the off-policy algorithm into a model-free reinforcement learning algorithm,which only requires the input and output data of the system.Finally,we use an example to illustrate the applicability and efficiency of the model-free algorithm above mentioned.展开更多
In the stability framework of model predictive control(MPC),the size of the stabilizable set(also known as the region of attraction)is dependent on the terminal constraint region.This article aims to investigate the o...In the stability framework of model predictive control(MPC),the size of the stabilizable set(also known as the region of attraction)is dependent on the terminal constraint region.This article aims to investigate the optimization of the terminal region for predictive control of a class of systems with multiplicative uncertainty,aiming to expand the attraction region in MPC.By utilizing a coordinate transformation,we initially develop a structured design for terminal ingredients while considering uncertainties in parameters.Subsequently,we propose novel methods to convert the original nonlinear problem into a linear matrix inequality(LMI)problem with minimal conservatism in the formulation.We propose an iterative learning optimization approach to compute the polytopic terminal region,and its incremental volume is theoretically proven.The efectiveness of the proposed approaches is demonstrated using a benchmark academic example and vehicle lateral dynamics.Through real-time simulation experiments,we demonstrate that the proposed approach can enlarge the domain of attraction as well as reduce the computational complexity of robust MPC systems under parameter uncertainty.展开更多
This paper presents several neural network based modelling, reliable optimal control, and iterative learning control methods for batch processes. In order to overcome the lack of robustness of a single neural network,...This paper presents several neural network based modelling, reliable optimal control, and iterative learning control methods for batch processes. In order to overcome the lack of robustness of a single neural network, bootstrap aggregated neural networks are used to build reliable data based empirical models. Apart from improving the model generalisation capability, a bootstrap aggregated neural network can also provide model prediction confidence bounds. A reliable optimal control method by incorporating model prediction confidence bounds into the optimisation objective function is presented. A neural network based iterative learning control strategy is presented to overcome the problem due to unknown disturbances and model-plant mismatches. The proposed methods are demonstrated on a simulated batch polymerisation process.展开更多
Considering the wheeled mobile robot(WMR)tracking problem with velocity saturation,we developed a data‐driven iterative learning double loop control method with constraints.First,the authors designed an outer loop co...Considering the wheeled mobile robot(WMR)tracking problem with velocity saturation,we developed a data‐driven iterative learning double loop control method with constraints.First,the authors designed an outer loop controller to provide virtual velocity for the inner loop according to the position and pose tracking error of the WMR kinematic model.Second,the authors employed dynamic linearisation to transform the dynamic model into an online data‐driven model along the iterative domain.Based on the measured input and output data of the dynamic model,the authors identified the parameters of the inner loop controller.The authors considered the velocity saturation constraints;we adjusted the output velocity of the WMR online,providing effective solutions to the problem of velocity saltation and the saturation constraint in the tracking process.Notably,the inner loop controller only uses the output data and input of the dynamic model,which not only enables the reliable control of WMR trajectory tracking,but also avoids the influence of inaccurate model identification processes on the tracking performance.The authors analysed the algorithm's convergence in theory,and the results show that the tracking errors of position,angle and velocity can converge to zero in the iterative domain.Finally,the authors used a simulation to demonstrate the effectiveness of the algorithm.展开更多
Batch processes have been increasingly used in the production of low volume and high value added products. Consequently, optimization control in batch processes is crucial in order to derive the maximum benefit. In th...Batch processes have been increasingly used in the production of low volume and high value added products. Consequently, optimization control in batch processes is crucial in order to derive the maximum benefit. In this paper, a run-to-run product quality control based on iterative learning optimization control is developed. Moreover, a rigorous theorem is proposed and proven in this paper, which states that the tracking error under the optimal iterative learning control （ILC） law can converge to zero. In this paper, a typical nonlinear batch continuous stirred tank reactor （CSTR） is considered, and the results show that the performance of trajectory tracking is gradually improved by the ILC.展开更多
基金supported by the National Natural Science Foundation of China(U21A20166)in part by the Science and Technology Development Foundation of Jilin Province (20230508095RC)+1 种基金in part by the Development and Reform Commission Foundation of Jilin Province (2023C034-3)in part by the Exploration Foundation of State Key Laboratory of Automotive Simulation and Control。
文摘Aiming at the tracking problem of a class of discrete nonaffine nonlinear multi-input multi-output(MIMO) repetitive systems subjected to separable and nonseparable disturbances, a novel data-driven iterative learning control(ILC) scheme based on the zeroing neural networks(ZNNs) is proposed. First, the equivalent dynamic linearization data model is obtained by means of dynamic linearization technology, which exists theoretically in the iteration domain. Then, the iterative extended state observer(IESO) is developed to estimate the disturbance and the coupling between systems, and the decoupled dynamic linearization model is obtained for the purpose of controller synthesis. To solve the zero-seeking tracking problem with inherent tolerance of noise,an ILC based on noise-tolerant modified ZNN is proposed. The strict assumptions imposed on the initialization conditions of each iteration in the existing ILC methods can be absolutely removed with our method. In addition, theoretical analysis indicates that the modified ZNN can converge to the exact solution of the zero-seeking tracking problem. Finally, a generalized example and an application-oriented example are presented to verify the effectiveness and superiority of the proposed process.
文摘Norm optimal iterative learning control(NOILC) has recently been applied to iterative learning control(ILC) problems in which tracking is only required at a subset of isolated time points along the trial duration. This problem addresses the practical needs of many applications, including industrial automation, crane control, satellite positioning and motion control within a medical stroke rehabilitation context. This paper provides a substantial generalization of this framework by providing a solution to the problem of convergence at intermediate points with simultaneous tracking of subsets of outputs to reference trajectories on subintervals. This formulation enables the NOILC paradigm to tackle tasks which mix "point to point" movements with linear tracking requirements and hence substantially broadens the application domain to include automation tasks which include welding or cutting movements, or human motion control where the movement is restricted by the task to straight line and/or planar segments. A solution to the problem is presented in the framework of NOILC and inherits NOILC s well-defined convergence properties. Design guidelines and supporting experimental results are included.
文摘This paper analyses the concept of a Limit Set in Parameter Optimal Iterative Learning Control （ILC）. We investigate the existence of stable and unstable parts of Limit Set and demonstrates that they will often exist in practice. This is illustrated via a 2-dimensional example where the convergence of the learning algorithm is analyzed from the error＇s dynamic behaviour. These ideas are extended to the N-dimensional cases by analogy and example.
基金supported by the National Natural Science Foundation of China(Nos.F010114-60974140,61273135)
文摘In this paper, a quasi-Newton-type optimized iterative learning control （ILC） algorithm is investigated for a class of discrete linear time-invariant systems. The proposed learning algorithm is to update the learning gain matrix by a quasi-Newton-type matrix instead of the inversion of the plant. By means of the mathematical inductive method, the monotone convergence of the proposed algorithm is analyzed, which shows that the tracking error monotonously converges to zero after a finite number of iterations. Compared with the existing optimized ILC algorithms, due to the superlinear convergence of quasi-Newton method, the proposed learning law operates with a faster convergent rate and is robust to the ill-condition of the system model, and thus owns a wide range of applications. Numerical simulations demonstrate the validity and effectiveness.
基金supported by the National Natural Science Foundation of China (62173333, 12271522)Beijing Natural Science Foundation (Z210002)the Research Fund of Renmin University of China (2021030187)。
文摘For unachievable tracking problems, where the system output cannot precisely track a given reference, achieving the best possible approximation for the reference trajectory becomes the objective. This study aims to investigate solutions using the Ptype learning control scheme. Initially, we demonstrate the necessity of gradient information for achieving the best approximation.Subsequently, we propose an input-output-driven learning gain design to handle the imprecise gradients of a class of uncertain systems. However, it is discovered that the desired performance may not be attainable when faced with incomplete information.To address this issue, an extended iterative learning control scheme is introduced. In this scheme, the tracking errors are modified through output data sampling, which incorporates lowmemory footprints and offers flexibility in learning gain design.The input sequence is shown to converge towards the desired input, resulting in an output that is closest to the given reference in the least square sense. Numerical simulations are provided to validate the theoretical findings.
基金This work was supported by the National Natural Science Foundation of China (No. 60274055)
文摘In the procedure of the steady-state hierarchical optimization with feedback for large-scale industrial processes, a sequence of set-point changes with different magnitudes is carried out on the optimization layer. To improve the dynamic performance of transient response driven by the set-point changes, a filter-based iterative learning control strategy is proposed. In the proposed updating law, a local-symmetric-integral operator is adopted for eliminating the measurement noise of output information,a set of desired trajectories are specified according to the set-point changes sequence, the current control input is iteratively achieved by utilizing smoothed output error to modify its control input at previous iteration, to which the amplified coefficients related to the different magnitudes of set-point changes are introduced. The convergence of the algorithm is conducted by incorporating frequency-domain technique into time-domain analysis. Numerical simulation demonstrates the effectiveness of the proposed strategy,
基金Supported by National Natural Science Foundation of China （F030101-60574021） and National ＂985＂ Project of China Executed in Xi＇an Jiaotong University
基金supported in part by the National Natural Science Foundation of China(62222301, 62073085, 62073158, 61890930-5, 62021003)the National Key Research and Development Program of China (2021ZD0112302, 2021ZD0112301, 2018YFC1900800-5)Beijing Natural Science Foundation (JQ19013)。
文摘Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.
基金This work was supported by National Key R&D Program of China(No.2018YFB1308404).
文摘The goal of this paper is to design a model-free optimal controller for the multirate system based on reinforcement learning.Sampled-data control systems are widely used in the industrial production process and multirate sampling has attracted much attention in the study of the sampled-data control theory.In this paper,we assume the sampling periods for state variables are different from periods for system inputs.Under this condition,we can obtain an equivalent discrete-time system using the lifting technique.Then,we provide an algorithm to solve the linear quadratic regulator(LQR)control problem of multirate systems with the utilization of matrix substitutions.Based on a reinforcement learning method,we use online policy iteration and off-policy algorithms to optimize the controller for multirate systems.By using the least squares method,we convert the off-policy algorithm into a model-free reinforcement learning algorithm,which only requires the input and output data of the system.Finally,we use an example to illustrate the applicability and efficiency of the model-free algorithm above mentioned.
基金supported by the the Fundamental Research Funds for the Central Universities(Grant No.JUSRP202501133)。
文摘In the stability framework of model predictive control(MPC),the size of the stabilizable set(also known as the region of attraction)is dependent on the terminal constraint region.This article aims to investigate the optimization of the terminal region for predictive control of a class of systems with multiplicative uncertainty,aiming to expand the attraction region in MPC.By utilizing a coordinate transformation,we initially develop a structured design for terminal ingredients while considering uncertainties in parameters.Subsequently,we propose novel methods to convert the original nonlinear problem into a linear matrix inequality(LMI)problem with minimal conservatism in the formulation.We propose an iterative learning optimization approach to compute the polytopic terminal region,and its incremental volume is theoretically proven.The efectiveness of the proposed approaches is demonstrated using a benchmark academic example and vehicle lateral dynamics.Through real-time simulation experiments,we demonstrate that the proposed approach can enlarge the domain of attraction as well as reduce the computational complexity of robust MPC systems under parameter uncertainty.
基金Supported by UK EPSRC (grants GR/N13319 and GR/R 10875)
文摘This paper presents several neural network based modelling, reliable optimal control, and iterative learning control methods for batch processes. In order to overcome the lack of robustness of a single neural network, bootstrap aggregated neural networks are used to build reliable data based empirical models. Apart from improving the model generalisation capability, a bootstrap aggregated neural network can also provide model prediction confidence bounds. A reliable optimal control method by incorporating model prediction confidence bounds into the optimisation objective function is presented. A neural network based iterative learning control strategy is presented to overcome the problem due to unknown disturbances and model-plant mismatches. The proposed methods are demonstrated on a simulated batch polymerisation process.
基金supported by the Innovation Project of Guangxi Graduate Education(Grant No.YCSW2022436).
文摘Considering the wheeled mobile robot(WMR)tracking problem with velocity saturation,we developed a data‐driven iterative learning double loop control method with constraints.First,the authors designed an outer loop controller to provide virtual velocity for the inner loop according to the position and pose tracking error of the WMR kinematic model.Second,the authors employed dynamic linearisation to transform the dynamic model into an online data‐driven model along the iterative domain.Based on the measured input and output data of the dynamic model,the authors identified the parameters of the inner loop controller.The authors considered the velocity saturation constraints;we adjusted the output velocity of the WMR online,providing effective solutions to the problem of velocity saltation and the saturation constraint in the tracking process.Notably,the inner loop controller only uses the output data and input of the dynamic model,which not only enables the reliable control of WMR trajectory tracking,but also avoids the influence of inaccurate model identification processes on the tracking performance.The authors analysed the algorithm's convergence in theory,and the results show that the tracking errors of position,angle and velocity can converge to zero in the iterative domain.Finally,the authors used a simulation to demonstrate the effectiveness of the algorithm.
基金supported by the Science Foundation of Shanghai Municipal Education Commission (Grant No.09Y208)the Science Foundation of Science and Technology Commission of Shanghai Municipality (Grant Nos.08DZ2272400, 09DZ2273400)the "11th Five-Year Plan" 211 Construction Project of Shanghai University
文摘Batch processes have been increasingly used in the production of low volume and high value added products. Consequently, optimization control in batch processes is crucial in order to derive the maximum benefit. In this paper, a run-to-run product quality control based on iterative learning optimization control is developed. Moreover, a rigorous theorem is proposed and proven in this paper, which states that the tracking error under the optimal iterative learning control （ILC） law can converge to zero. In this paper, a typical nonlinear batch continuous stirred tank reactor （CSTR） is considered, and the results show that the performance of trajectory tracking is gradually improved by the ILC.
