A new reinforcement method for learning fuzzy logiccontrollers is proposed.The reinforcement learningscheme is composed of two fuzzy logic rule bases:oneacts as an adaptive critic,the other server as a control-ler.The...A new reinforcement method for learning fuzzy logiccontrollers is proposed.The reinforcement learningscheme is composed of two fuzzy logic rule bases:oneacts as an adaptive critic,the other server as a control-ler.The proposed method is tested on the cart-pole sys-tem.Simulation results show that the method has betterlearning performance than Anderson’s neural network-based method.展开更多
Inverse reinforcement learning optimal control is under the framework of learner-expert.The learner system can imitate the expert system's demonstrated behaviors and does not require the predefined cost function,s...Inverse reinforcement learning optimal control is under the framework of learner-expert.The learner system can imitate the expert system's demonstrated behaviors and does not require the predefined cost function,so it can handle optimal control problems effectively.This paper proposes an inverse reinforcement learning optimal control method for Takagi-Sugeno(T-S)fuzzy systems.Based on learner systems,an expert system is constructed,where the learner system only knows the expert system's optimal control policy.To reconstruct the unknown cost function,we firstly develop a model-based inverse reinforcement learning algorithm for the case that systems dynamics are known.The developed model-based learning algorithm is consists of two learning stages:an inner reinforcement learning loop and an outer inverse optimal control loop.The inner loop desires to obtain optimal control policy via learner's cost function and the outer loop aims to update learner's state-penalty matrices via only using expert's optimal control policy.Then,to eliminate the requirement that the system dynamics must be known,a data-driven integral learning algorithm is presented.It is proved that the presented two algorithms are convergent and the developed inverse reinforcement learning optimal control scheme can ensure the controlled fuzzy learner systems to be asymptotically stable.Finally,we apply the proposed fuzzy optimal control to the truck-trailer system,and the computer simulation results verify the effectiveness of the presented approach.展开更多
This paper presents a novel method for constructing fuzzy controllers based on a real time reinforcement genetic algorithm. This methodology introduces the real-time learning capability of neural networks into globall...This paper presents a novel method for constructing fuzzy controllers based on a real time reinforcement genetic algorithm. This methodology introduces the real-time learning capability of neural networks into globally searching process of genetic algorithm, aiming to enhance the convergence rate and real-time learning ability of genetic algorithm, which is then used to construct fuzzy controllers for complex dynamic systems without any knowledge about system dynamics and prior control experience. The cart-pole system is employed as a test bed to demonstrate the effectiveness of the proposed control scheme, and the robustness of the acquired fuzzy controller with comparable result.展开更多
The paper presents development of a reinforcement learning(RL)and sliding mode control(SMC)algorithm for a 3-phase PV system integrated to a grid.The PV system is integrated to grid through a voltage source inverter(V...The paper presents development of a reinforcement learning(RL)and sliding mode control(SMC)algorithm for a 3-phase PV system integrated to a grid.The PV system is integrated to grid through a voltage source inverter(VSI),in which PVVSI combination supplies active power and compensates reactive power of the local non-linear load connected to the point of common coupling(PCC).For extraction of maximum power from the PV panel,we develop a RL based maximum power point tracking(MPPT)algorithm.The instantaneous power theory(IPT)is adopted for generation reference inverter current(RIC).An SMC algorithm has been developed for injecting current to the local non-linear load at a reference value.The RL-SMC scheme is implemented in both simulation using MATLAB/SIMULINK software and on a prototype PV experimental.The performance of the proposed RL-SMC scheme is compared with that of fuzzy logic-sliding mode control(FL-SMC)and incremental conductance-sliding mode control(IC-SMC)algorithms.From the obtained results,it is observed that the proposed RL-SMC scheme provides better maximum power extraction and active power control than the FL-SMC and IC-SMC schemes.展开更多
In this paper a learning mechanism for reactive fuzzy controller design of a mobile robot navigating in unknown environments is proposed. The fuzzy logical controller is constructed based on the kinematics model of a ...In this paper a learning mechanism for reactive fuzzy controller design of a mobile robot navigating in unknown environments is proposed. The fuzzy logical controller is constructed based on the kinematics model of a real robot. The approach to learning the fuzzy rule base by relatively simple and less computational Q-learning is described in detail. After analyzing the credit assignment problem caused by the rules collision, a remedy is presented. Furthermore, time-varying parameters are used to increase the learning speed. Simulation results prove the mechanism can learn fuzzy navigation rules successfully only using scalar reinforcement signal and the rule base learned is proved to be correct and feasible on real robot platforms.展开更多
Lane changing is common in driving.Thus,the possibility of traffic accidents occurring during lane changes is high given the complexity of this process.One of the primary objectives of intelligent driving is to increa...Lane changing is common in driving.Thus,the possibility of traffic accidents occurring during lane changes is high given the complexity of this process.One of the primary objectives of intelligent driving is to increase a vehicle’s behavior,making it more similar to that of a real driver.This study proposes a decision-making framework based on deep reinforcement learning(DRL)in a lane-changing scenario,which seeks to find a driving strategy that simultaneously considers the expected lane-changing risks and gains.First,a fuzzy logic lane-changing controller is designed.It outputs the corresponding safety and lane-change gain weights by inputting relevant driving parameters.Second,the obtained weights are brought into the constructed reward function of DRL.The model parameters are designed and trained on the basis of lane-changing behavior.Finally,we conducted experiments in a simulator to evaluate the performance of our developed algorithm in urban scenarios.To visualize and validate the estimated driving intentions,lane-changing strategies were tested under four scenarios.The results show that the average improvement in travel efficiency in the four scenarios is 19%.In addition,the average accident rate in the four scenarios increased by only 4%.We combine fuzzy logic and DRL reward functions to personify the lane-changing behavior of intelligent driving.Compared with conservative strategies that prioritize only safety,this method can considerably improve the number of lane changes and travel efficiency for autonomous vehicles(AVs)on the premise of ensuring safety.The approach provides an effective and explainable method designed for facilitating intelligent driving lane-changing behavior.展开更多
A novel approach to control the unpredictable behavior of chaotic systems is presented. The control algorithm is based on fuzzy logic control technique combined with genetic algorithm. The use of fuzzy logic allows fo...A novel approach to control the unpredictable behavior of chaotic systems is presented. The control algorithm is based on fuzzy logic control technique combined with genetic algorithm. The use of fuzzy logic allows for the implementation of human "rule-of-thumb" approach to decision making by employing linguistic variables. An improved Genetic Algorithm (GA) is used to learn to optimally select the fuzzy membership functions of the linguistic labels in the condition portion of each rule, and to automatically generate fuzzy control actions under each condition. Simulation results show that such an approach for the control of chaotic systems is both effective and robust.展开更多
Some typical structural schemes of Fuzzy control have been surveyed. Besides general structure of fuzzy logic controller (FLC), the structural schemes include PID fuzzy controller, self-organizing fuzzy controller, se...Some typical structural schemes of Fuzzy control have been surveyed. Besides general structure of fuzzy logic controller (FLC), the structural schemes include PID fuzzy controller, self-organizing fuzzy controller, selftuning fuzzy controller, self-learning fuzzy controller, and expect fuzzy controller, etc. This survey focuses on the control principle, and provides a basis for potential applications. Most of the structures have been used in various control fields, one of application areas is in the metallurgy industry, e. g., the temperature control of the electric furnace, the control of the aluminum smelting process, etc. According to the application requirements, one can choose a structural scheme for special use.展开更多
The paper presents a fuzzy Q-learning(FQL)and optical flow-based autonomous navigation approach.The FQL method takes decisions in an unknown environment and without mapping,using motion information and through a reinf...The paper presents a fuzzy Q-learning(FQL)and optical flow-based autonomous navigation approach.The FQL method takes decisions in an unknown environment and without mapping,using motion information and through a reinforcement signal into an evolutionary algorithm.The reinforcement signal is calculated by estimating the optical flow densities in areas of the camera to determine whether they are“dense”or“thin”which has a relationship with the proximity of objects.The results obtained show that the present approach improves the rate of learning compared with a method with a simple reward system and without the evolutionary component.The proposed system was implemented in a virtual robotics system using the CoppeliaSim software and in communication with Python.展开更多
A stable control scheme for a class of unknown nonlinear systems was presented. The control architecture is composed of two parts, the fuzzy sliding mode controller (FSMC) is applied to drive the state to a designed s...A stable control scheme for a class of unknown nonlinear systems was presented. The control architecture is composed of two parts, the fuzzy sliding mode controller (FSMC) is applied to drive the state to a designed switching hyperplane, and a reinforcement self organizing fuzzy CPN (RSOFCPN) as a feedforward compensator is used to reduce the influence of system uncertainties. The simulation results demonstrate the effectiveness of the proposed control scheme.展开更多
文摘A new reinforcement method for learning fuzzy logiccontrollers is proposed.The reinforcement learningscheme is composed of two fuzzy logic rule bases:oneacts as an adaptive critic,the other server as a control-ler.The proposed method is tested on the cart-pole sys-tem.Simulation results show that the method has betterlearning performance than Anderson’s neural network-based method.
基金The National Natural Science Foundation of China(62173172).
文摘Inverse reinforcement learning optimal control is under the framework of learner-expert.The learner system can imitate the expert system's demonstrated behaviors and does not require the predefined cost function,so it can handle optimal control problems effectively.This paper proposes an inverse reinforcement learning optimal control method for Takagi-Sugeno(T-S)fuzzy systems.Based on learner systems,an expert system is constructed,where the learner system only knows the expert system's optimal control policy.To reconstruct the unknown cost function,we firstly develop a model-based inverse reinforcement learning algorithm for the case that systems dynamics are known.The developed model-based learning algorithm is consists of two learning stages:an inner reinforcement learning loop and an outer inverse optimal control loop.The inner loop desires to obtain optimal control policy via learner's cost function and the outer loop aims to update learner's state-penalty matrices via only using expert's optimal control policy.Then,to eliminate the requirement that the system dynamics must be known,a data-driven integral learning algorithm is presented.It is proved that the presented two algorithms are convergent and the developed inverse reinforcement learning optimal control scheme can ensure the controlled fuzzy learner systems to be asymptotically stable.Finally,we apply the proposed fuzzy optimal control to the truck-trailer system,and the computer simulation results verify the effectiveness of the presented approach.
文摘This paper presents a novel method for constructing fuzzy controllers based on a real time reinforcement genetic algorithm. This methodology introduces the real-time learning capability of neural networks into globally searching process of genetic algorithm, aiming to enhance the convergence rate and real-time learning ability of genetic algorithm, which is then used to construct fuzzy controllers for complex dynamic systems without any knowledge about system dynamics and prior control experience. The cart-pole system is employed as a test bed to demonstrate the effectiveness of the proposed control scheme, and the robustness of the acquired fuzzy controller with comparable result.
文摘The paper presents development of a reinforcement learning(RL)and sliding mode control(SMC)algorithm for a 3-phase PV system integrated to a grid.The PV system is integrated to grid through a voltage source inverter(VSI),in which PVVSI combination supplies active power and compensates reactive power of the local non-linear load connected to the point of common coupling(PCC).For extraction of maximum power from the PV panel,we develop a RL based maximum power point tracking(MPPT)algorithm.The instantaneous power theory(IPT)is adopted for generation reference inverter current(RIC).An SMC algorithm has been developed for injecting current to the local non-linear load at a reference value.The RL-SMC scheme is implemented in both simulation using MATLAB/SIMULINK software and on a prototype PV experimental.The performance of the proposed RL-SMC scheme is compared with that of fuzzy logic-sliding mode control(FL-SMC)and incremental conductance-sliding mode control(IC-SMC)algorithms.From the obtained results,it is observed that the proposed RL-SMC scheme provides better maximum power extraction and active power control than the FL-SMC and IC-SMC schemes.
文摘In this paper a learning mechanism for reactive fuzzy controller design of a mobile robot navigating in unknown environments is proposed. The fuzzy logical controller is constructed based on the kinematics model of a real robot. The approach to learning the fuzzy rule base by relatively simple and less computational Q-learning is described in detail. After analyzing the credit assignment problem caused by the rules collision, a remedy is presented. Furthermore, time-varying parameters are used to increase the learning speed. Simulation results prove the mechanism can learn fuzzy navigation rules successfully only using scalar reinforcement signal and the rule base learned is proved to be correct and feasible on real robot platforms.
基金supported in part by the National Natural Science Foundation of China(No.52372407)the Jilin Provincial Science and Technology Development Plan Project(No.20230402064GH).
文摘Lane changing is common in driving.Thus,the possibility of traffic accidents occurring during lane changes is high given the complexity of this process.One of the primary objectives of intelligent driving is to increase a vehicle’s behavior,making it more similar to that of a real driver.This study proposes a decision-making framework based on deep reinforcement learning(DRL)in a lane-changing scenario,which seeks to find a driving strategy that simultaneously considers the expected lane-changing risks and gains.First,a fuzzy logic lane-changing controller is designed.It outputs the corresponding safety and lane-change gain weights by inputting relevant driving parameters.Second,the obtained weights are brought into the constructed reward function of DRL.The model parameters are designed and trained on the basis of lane-changing behavior.Finally,we conducted experiments in a simulator to evaluate the performance of our developed algorithm in urban scenarios.To visualize and validate the estimated driving intentions,lane-changing strategies were tested under four scenarios.The results show that the average improvement in travel efficiency in the four scenarios is 19%.In addition,the average accident rate in the four scenarios increased by only 4%.We combine fuzzy logic and DRL reward functions to personify the lane-changing behavior of intelligent driving.Compared with conservative strategies that prioritize only safety,this method can considerably improve the number of lane changes and travel efficiency for autonomous vehicles(AVs)on the premise of ensuring safety.The approach provides an effective and explainable method designed for facilitating intelligent driving lane-changing behavior.
文摘A novel approach to control the unpredictable behavior of chaotic systems is presented. The control algorithm is based on fuzzy logic control technique combined with genetic algorithm. The use of fuzzy logic allows for the implementation of human "rule-of-thumb" approach to decision making by employing linguistic variables. An improved Genetic Algorithm (GA) is used to learn to optimally select the fuzzy membership functions of the linguistic labels in the condition portion of each rule, and to automatically generate fuzzy control actions under each condition. Simulation results show that such an approach for the control of chaotic systems is both effective and robust.
文摘Some typical structural schemes of Fuzzy control have been surveyed. Besides general structure of fuzzy logic controller (FLC), the structural schemes include PID fuzzy controller, self-organizing fuzzy controller, selftuning fuzzy controller, self-learning fuzzy controller, and expect fuzzy controller, etc. This survey focuses on the control principle, and provides a basis for potential applications. Most of the structures have been used in various control fields, one of application areas is in the metallurgy industry, e. g., the temperature control of the electric furnace, the control of the aluminum smelting process, etc. According to the application requirements, one can choose a structural scheme for special use.
文摘The paper presents a fuzzy Q-learning(FQL)and optical flow-based autonomous navigation approach.The FQL method takes decisions in an unknown environment and without mapping,using motion information and through a reinforcement signal into an evolutionary algorithm.The reinforcement signal is calculated by estimating the optical flow densities in areas of the camera to determine whether they are“dense”or“thin”which has a relationship with the proximity of objects.The results obtained show that the present approach improves the rate of learning compared with a method with a simple reward system and without the evolutionary component.The proposed system was implemented in a virtual robotics system using the CoppeliaSim software and in communication with Python.
基金National Natural Science Foundation ofChina! under grant No.69674 0 2 3
文摘A stable control scheme for a class of unknown nonlinear systems was presented. The control architecture is composed of two parts, the fuzzy sliding mode controller (FSMC) is applied to drive the state to a designed switching hyperplane, and a reinforcement self organizing fuzzy CPN (RSOFCPN) as a feedforward compensator is used to reduce the influence of system uncertainties. The simulation results demonstrate the effectiveness of the proposed control scheme.
