The goal of this paper is to design a model-free optimal controller for the multirate system based on reinforcement learning.Sampled-data control systems are widely used in the industrial production process and multir...The goal of this paper is to design a model-free optimal controller for the multirate system based on reinforcement learning.Sampled-data control systems are widely used in the industrial production process and multirate sampling has attracted much attention in the study of the sampled-data control theory.In this paper,we assume the sampling periods for state variables are different from periods for system inputs.Under this condition,we can obtain an equivalent discrete-time system using the lifting technique.Then,we provide an algorithm to solve the linear quadratic regulator(LQR)control problem of multirate systems with the utilization of matrix substitutions.Based on a reinforcement learning method,we use online policy iteration and off-policy algorithms to optimize the controller for multirate systems.By using the least squares method,we convert the off-policy algorithm into a model-free reinforcement learning algorithm,which only requires the input and output data of the system.Finally,we use an example to illustrate the applicability and efficiency of the model-free algorithm above mentioned.展开更多
基金This work was supported by National Key R&D Program of China(No.2018YFB1308404).
文摘The goal of this paper is to design a model-free optimal controller for the multirate system based on reinforcement learning.Sampled-data control systems are widely used in the industrial production process and multirate sampling has attracted much attention in the study of the sampled-data control theory.In this paper,we assume the sampling periods for state variables are different from periods for system inputs.Under this condition,we can obtain an equivalent discrete-time system using the lifting technique.Then,we provide an algorithm to solve the linear quadratic regulator(LQR)control problem of multirate systems with the utilization of matrix substitutions.Based on a reinforcement learning method,we use online policy iteration and off-policy algorithms to optimize the controller for multirate systems.By using the least squares method,we convert the off-policy algorithm into a model-free reinforcement learning algorithm,which only requires the input and output data of the system.Finally,we use an example to illustrate the applicability and efficiency of the model-free algorithm above mentioned.
