Autonomy, a key property associated with the agent, is an important topic in the current research of the agent theory. Although no definition of the agent autonomy is universally accepted, an important aspect of the a...Autonomy, a key property associated with the agent, is an important topic in the current research of the agent theory. Although no definition of the agent autonomy is universally accepted, an important aspect of the agent autonomy is the decision-making capability of the agents. This paper investigates the autonomy of the agent, presents a framework for autonomous agent and discusses its decision-making process. Started with introducing a language for representing autonomous agent, a framework is proposed for modeling autonomous agent based on a BDI model and the situation calculus. Finally, a kind of decision-making process of the autonomous agent is presented.展开更多
Planning and decision-making technology at intersections is a comprehensive research problem in intelligent transportation systems due to the uncertainties caused by a variety of traffic participants.As wireless commu...Planning and decision-making technology at intersections is a comprehensive research problem in intelligent transportation systems due to the uncertainties caused by a variety of traffic participants.As wireless communication advances,vehicle infrastructure integrated algorithms designed for intersection planning and decision-making have received increasing attention.In this paper,the recent studies on the planning and decision-making technologies at intersections are primarily overviewed.The general planning and decision-making approaches are presented,which include graph-based approach,prediction base approach,optimization-based approach and machine learning based approach.Since connected autonomous vehicles(CAVs)is the future direction for the automated driving area,we summarized the evolving planning and decision-making methods based on vehicle infrastructure cooperative technologies.Both four-way signalized and unsignalized intersection(s)are investigated under purely automated driving traffic and mixed traffic.The study benefit from current strategies,protocols,and simulation tools to help researchers identify the presented approaches’challenges and determine the research gaps,and several remaining possible research problems that need to be solved in the future.展开更多
To improve the safety and driving stability of the autonomous heavy truck, it is necessary to consider the differences of driving behavior and drivable trajectories between the heavy trucks and passenger cars. This st...To improve the safety and driving stability of the autonomous heavy truck, it is necessary to consider the differences of driving behavior and drivable trajectories between the heavy trucks and passenger cars. This study proposes a probabilistic decision-making and trajectory planning framework for the autonomous heavy trucks. Firstly, the driving decision process is divided into intention generation and feasibility evaluations, which are realized using the utility theory and risk assessment, respectively. Subsequently the driving decision is made and sent to the trajectory planning module. In order to reflect the greater risks of the truck to other surrounding vehicles, the aggressiveness index(AI) is proposed and quantified to infer the asymmetrical risk level of lane-change maneuver. In the planning stage, the lateral and roll dynamics stability domains are developed as the constraints to exclude the candidate trajectories that would cause vehicle instability. Finally, the simulation results are compared between the proposed model and the artificial potential filed model in the scenarios extracted from the naturalistic driving data. It is shown that the proposed framework can provide the human-like lane-change decisions and truck-friendly trajectories, and performs well in dynamic driving environments.展开更多
Decision-making and motion planning are extremely important in autonomous driving to ensure safe driving in a real-world environment.This study proposes an online evolutionary decision-making and motion planning frame...Decision-making and motion planning are extremely important in autonomous driving to ensure safe driving in a real-world environment.This study proposes an online evolutionary decision-making and motion planning framework for autonomous driving based on a hybrid data-and model-driven method.First,a data-driven decision-making module based on deep reinforcement learning(DRL)is developed to pursue a rational driving performance as much as possible.Then,model predictive control(MPC)is employed to execute both longitudinal and lateral motion planning tasks.Multiple constraints are defined according to the vehicle’s physical limit to meet the driving task requirements.Finally,two principles of safety and rationality for the self-evolution of autonomous driving are proposed.A motion envelope is established and embedded into a rational exploration and exploitation scheme,which filters out unreasonable experiences by masking unsafe actions so as to collect high-quality training data for the DRL agent.Experiments with a high-fidelity vehicle model and MATLAB/Simulink co-simulation environment are conducted,and the results show that the proposed online-evolution framework is able to generate safer,more rational,and more efficient driving action in a real-world environment.展开更多
With the maturation of autonomous driving technology, the use of autonomous vehicles in a socially acceptable manner has become a growing demand of the public. Human-like autonomous driving is expected due to the impa...With the maturation of autonomous driving technology, the use of autonomous vehicles in a socially acceptable manner has become a growing demand of the public. Human-like autonomous driving is expected due to the impact of the differences between autonomous vehicles and human drivers on safety.Although human-like decision-making has become a research hotspot, a unified theory has not yet been formed, and there are significant differences in the implementation and performance of existing methods. This paper provides a comprehensive overview of human-like decision-making for autonomous vehicles. The following issues are discussed: 1) The intelligence level of most autonomous driving decision-making algorithms;2) The driving datasets and simulation platforms for testing and verifying human-like decision-making;3) The evaluation metrics of human-likeness;personalized driving;the application of decisionmaking in real traffic scenarios;and 4) The potential research direction of human-like driving. These research results are significant for creating interpretable human-like driving models and applying them in dynamic traffic scenarios. In the future, the combination of intuitive logical reasoning and hierarchical structure will be an important topic for further research. It is expected to meet the needs of human-like driving.展开更多
The decision-making under complex urban environment become one of the key issues that restricts the rapid development of the autonomous vehicles. The difficulty in making timely and accurate decisions like human being...The decision-making under complex urban environment become one of the key issues that restricts the rapid development of the autonomous vehicles. The difficulty in making timely and accurate decisions like human beings under highly dynamic traffic environment is a major challenge for autonomous driving. Car-following has been regarded as the simplest but essential driving behavior among driving tasks and has received extensive attention from researchers around the world. This work addresses this problem and proposes a novel method RSAN(rough-set artificial neural network) to learn the decisions from excellent human drivers. A virtual urban traffic environment was built by Pre Scan and driving simulation was conducted to obtain a broad set of relevant data such as experienced drivers' behavior data and surrounding vehicles' motion data. Then, rough set was used to preprocess these data to extract the key influential factors on decision and reduce the impact of uncertain data and noise data. And the car-following decision was learned by neural network in which key factor was the input and acceleration was the output. The result shows the better convergence speed and the better decision accuracy of RSAN than ANN. Findings of this work contributes to the empirical understanding of driver's decision-making process and it provides a theoretical basis for the study of car-following decision-making under complex and dynamic environment.展开更多
While autonomous vehicles are vital components of intelligent transportation systems,ensuring the trustworthiness of decision-making remains a substantial challenge in realizing autonomous driving.Therefore,we present...While autonomous vehicles are vital components of intelligent transportation systems,ensuring the trustworthiness of decision-making remains a substantial challenge in realizing autonomous driving.Therefore,we present a novel robust reinforcement learning approach with safety guarantees to attain trustworthy decision-making for autonomous vehicles.The proposed technique ensures decision trustworthiness in terms of policy robustness and collision safety.Specifically,an adversary model is learned online to simulate the worst-case uncertainty by approximating the optimal adversarial perturbations on the observed states and environmental dynamics.In addition,an adversarial robust actor-critic algorithm is developed to enable the agent to learn robust policies against perturbations in observations and dynamics.Moreover,we devise a safety mask to guarantee the collision safety of the autonomous driving agent during both the training and testing processes using an interpretable knowledge model known as the Responsibility-Sensitive Safety Model.Finally,the proposed approach is evaluated through both simulations and experiments.These results indicate that the autonomous driving agent can make trustworthy decisions and drastically reduce the number of collisions through robust safety policies.展开更多
This paper is to explore the problems of intelligent connected vehicles(ICVs)autonomous driving decision-making under a 5G-V2X structured road environment.Through literature review and interviews with autonomous drivi...This paper is to explore the problems of intelligent connected vehicles(ICVs)autonomous driving decision-making under a 5G-V2X structured road environment.Through literature review and interviews with autonomous driving practitioners,this paper firstly puts forward a logical framework for designing a cerebrum-like autonomous driving system.Secondly,situated on this framework,it builds a hierarchical finite state machine(HFSM)model as well as a TOPSIS-GRA algorithm for making ICV autonomous driving decisions by employing a data fusion approach between the entropy weight method(EWM)and analytic hierarchy process method(AHP)and by employing a model fusion approach between the technique for order preference by similarity to an ideal solution(TOPSIS)and grey relational analysis(GRA).The HFSM model is composed of two layers:the global FSM model and the local FSM model.The decision of the former acts as partial input information of the latter and the result of the latter is sent forward to the local pathplanning module,meanwhile pulsating feedback to the former as real-time refresh data.To identify different traffic scenarios in a cerebrum-like way,the global FSM model is designed as 7 driving behavior states and 17 driving characteristic events,and the local FSM model is designed as 16 states and 8 characteristic events.In respect to designing a cerebrum-like algorithm for state transition,this paper firstly fuses AHP weight and EWM weight at their output layer to generate a synthetic weight coefficient for each characteristic event;then,it further fuses TOPSIS method and GRA method at the model building layer to obtain the implementable order of state transition.To verify the feasibility,reliability,and safety of theHFSMmodel aswell as its TOPSISGRA state transition algorithm,this paper elaborates on a series of simulative experiments conducted on the PreScan8.50 platform.The results display that the accuracy of obstacle detection gets 98%,lane line prediction is beyond 70 m,the speed of collision avoidance is higher than 45 km/h,the distance of collision avoidance is less than 5 m,path planning time for obstacle avoidance is averagely less than 50 ms,and brake deceleration is controlled under 6 m/s2.These technical indexes support that the driving states set and characteristic events set for the HFSM model as well as its TOPSIS-GRA algorithm may bring about cerebrum-like decision-making effectiveness for ICV autonomous driving under 5G-V2X intelligent road infrastructure.展开更多
To solve problems of poor security guarantee and insufficient training efficiency in the conventional reinforcement learning methods for decision-making,this study proposes a hybrid framework to combine deep reinforce...To solve problems of poor security guarantee and insufficient training efficiency in the conventional reinforcement learning methods for decision-making,this study proposes a hybrid framework to combine deep reinforcement learning with rule-based decision-making methods.A risk assessment model for lane-change maneuvers considering uncertain predictions of surrounding vehicles is established as a safety filter to improve learning efficiency while correcting dangerous actions for safety enhancement.On this basis,a Risk-fused DDQN is constructed utilizing the model-based risk assessment and supervision mechanism.The proposed reinforcement learning algorithm sets up a separate experience buffer for dangerous trials and punishes such actions,which is shown to improve the sampling efficiency and training outcomes.Compared with conventional DDQN methods,the proposed algorithm improves the convergence value of cumulated reward by 7.6%and 2.2%in the two constructed scenarios in the simulation study and reduces the number of training episodes by 52.2%and 66.8%respectively.The success rate of lane change is improved by 57.3%while the time headway is increased at least by 16.5%in real vehicle tests,which confirms the higher training efficiency,scenario adaptability,and security of the proposed Risk-fused DDQN.展开更多
Intelligent robots are increasingly being deployed across industries ranging from manufacturing to household applications and outdoor exploration.Their autonomous obstacle avoidance capabilities in complex environment...Intelligent robots are increasingly being deployed across industries ranging from manufacturing to household applications and outdoor exploration.Their autonomous obstacle avoidance capabilities in complex environments have become a critical factor determining operational stability.Multimodal perception technology,which integrates visual,auditory,tactile,and LiDAR data,provides robots with comprehensive environmental awareness.By establishing efficient autonomous obstacle avoidance decision-making mechanisms based on this information,the system’s adaptability to challenging scenarios can be significantly enhanced.This study investigates the integration of multimodal perception with autonomous obstacle avoidance decision-making,analyzing the acquisition and processing of perceptual information,core modules and logic of decision-making mechanisms,and proposing optimization strategies for specific scenarios.The research aims to provide theoretical references for advancing autonomous obstacle avoidance technology in intelligent robots,enabling safer and more flexible movement in diverse environments.展开更多
The anthropomorphic intelligence of autonomous driving has been a research hotspot in the world.However,current studies have not been able to reveal the mechanism of drivers'natural driving behaviors.Therefore,thi...The anthropomorphic intelligence of autonomous driving has been a research hotspot in the world.However,current studies have not been able to reveal the mechanism of drivers'natural driving behaviors.Therefore,this thesis starts from the perspective of cognitive decision-making in the human brain,which is inspired by the regulation of dopamine feedback in the basal ganglia,and a reinforcement learning model is established to solve the brain-like intelligent decision-making problems in the process of interacting with the environment.In this thesis,first,a detailed bionic mechanism architecture based on basal ganglia was proposed by the consideration and analysis of its feedback regulation mechanism;second,the above mechanism was transformed into a reinforcement Q-learning model,so as to implement the learning and adaptation abilities of an intelligent vehicle for brain-like intelligent decision-making during car-following;finally,the feasibility and effectiveness of the proposed method were verified by the simulations and real vehicle tests.展开更多
Autonomous umanned aerial vehicle(UAV) manipulation is necessary for the defense department to execute tactical missions given by commanders in the future unmanned battlefield. A large amount of research has been devo...Autonomous umanned aerial vehicle(UAV) manipulation is necessary for the defense department to execute tactical missions given by commanders in the future unmanned battlefield. A large amount of research has been devoted to improving the autonomous decision-making ability of UAV in an interactive environment, where finding the optimal maneuvering decisionmaking policy became one of the key issues for enabling the intelligence of UAV. In this paper, we propose a maneuvering decision-making algorithm for autonomous air-delivery based on deep reinforcement learning under the guidance of expert experience. Specifically, we refine the guidance towards area and guidance towards specific point tasks for the air-delivery process based on the traditional air-to-surface fire control methods.Moreover, we construct the UAV maneuvering decision-making model based on Markov decision processes(MDPs). Specifically, we present a reward shaping method for the guidance towards area and guidance towards specific point tasks using potential-based function and expert-guided advice. The proposed algorithm could accelerate the convergence of the maneuvering decision-making policy and increase the stability of the policy in terms of the output during the later stage of training process. The effectiveness of the proposed maneuvering decision-making policy is illustrated by the curves of training parameters and extensive experimental results for testing the trained policy.展开更多
Learning-based algorithm attracts great attention in the autonomous driving control field,especially for decisionmaking,to meet the challenge in long-tail extreme scenarios,where traditional methods demonstrate poor a...Learning-based algorithm attracts great attention in the autonomous driving control field,especially for decisionmaking,to meet the challenge in long-tail extreme scenarios,where traditional methods demonstrate poor adaptability even with a significant effort.To improve the autonomous driving performance in extreme scenarios,specifically consecutive sharp turns,three deep reinforcement learning algorithms,i.e.Deep Deterministic Policy Gradient(DDPG),Twin Delayed Deep Deterministic policy gradient(TD3),and Soft Actor-Critic(SAC),based decision-making policies are proposed in this study.The role of the observation variable in agent training is discussed by comparing the driving stability,average speed,and consumed computational effort of the proposed algorithms in curves with various curvatures.In addition,a novel reward-setting method that combines the states of the environment and the vehicle is proposed to solve the sparse reward problem in the reward-guided algorithm.Simulation results from the road with consecutive sharp turns show that the DDPG,SAC,and TD3 algorithms-based vehicles take 367.2,359.6,and 302.1 s to finish the task,respectively,which match the training results,and verifies the observation variable role in agent quality improvement.展开更多
In order to improve the agility and applicability of trajectory planning algorithm for autonomous vehicles, this paper proposes a novel actor-critic based learning method for decision-making and planning in multi-vehi...In order to improve the agility and applicability of trajectory planning algorithm for autonomous vehicles, this paper proposes a novel actor-critic based learning method for decision-making and planning in multi-vehicle complex traffic. It is the coupling planning of vehicle’s path and speed thus to make the trajectory more flexible. First, generations from the decided action to the planned trajectory are described by the end-point of the trajectory. Then, the actor-critic based learning method is built to learn an optimal policy for the decision process. It can update the policy by the gradient of the current policy’s advantage. In this process,features of the real traffic are carefully extracted by time headway(TH) and speed distribution. Reward function is built by the safety, efficiency and driving comfort. Furthermore, to make the policy network have better convergency, the policy network is modularized in two parts: the lane-changing network and the lane-keeping network, which decide the optimal end-point of the path and speed candidates respectively. Finally, the curved overtaking scenario and the interaction process with human driver are conducted to illustrate the feasibility and superiority. The results show that the proposed method has better real-time performance and can make the planned coupling trajectory more continuous and smoother than the existing rule-based method.展开更多
Road intersection is one of the most complex and accident-prone traffic scenarios,so it’s challenging for autonomous vehicles(AVs)to make safe and efficient decisions at the intersections.Most of the related studies ...Road intersection is one of the most complex and accident-prone traffic scenarios,so it’s challenging for autonomous vehicles(AVs)to make safe and efficient decisions at the intersections.Most of the related studies focus on the solution to a single scenario or only guarantee safety without considering driving efficiency.To address these problems,this study proposed a deep reinforcement learning enabled decision-making framework for AVs to drive through intersections automatically,safely and efficiently.The mapping relationship between traffic images and vehicle operations was obtained by an end-to-end decision-making framework established by convolutional neural networks.Traffic images collected at two timesteps were used to calculate the relative velocity between vehicles.Markov decision process was employed to model the interaction between AVs and other vehicles,and the deep Q-network algorithm was utilized to obtain the optimal driving policy regarding safety and efficiency.To verify the effectiveness of the proposed decision-making framework,the top three accident-prone crossing path crash scenarios at intersections were simulated,when different initial vehicle states were adopted for better generalization capability.The results showed that the developed method could make AVs drive safely and efficiently through intersections in all of the tested scenarios.展开更多
This paper presents a rule-based framework for addressing decision-making problems within the context of the "UI-STRIVE"Competition.First,two distinct autonomous confrontation scenarios are described:autonom...This paper presents a rule-based framework for addressing decision-making problems within the context of the "UI-STRIVE"Competition.First,two distinct autonomous confrontation scenarios are described:autonomous air combat and cooperative interception.Second,a State-Event-Condition-Action(SECA)decision-making framework is developed,which integrates thefinite state machine and event-condition-action frameworks.This framework provides three products to describe rules,i.e.the SECA model,the SECA state chart,and the SECA rule description.Third,the situation assessment and target assignment during autonomous air combat are investigated,and the mathematical models are established.Finally,the decisionmaking model's rationality and feasibility are verified through data simulation and analysis.展开更多
The automotive sector is crucial in modern society,facilitating essential transportation needs across personal,commercial,and logistical domains while significantly contributing to national economic development and em...The automotive sector is crucial in modern society,facilitating essential transportation needs across personal,commercial,and logistical domains while significantly contributing to national economic development and employment generation.The transformative impact of Artificial Intelligence(AI)has revolutionised multiple facets of the automotive industry,encompassing intelligent manufacturing processes,diagnostic systems,control mechanisms,supply chain operations,customer service platforms,and traffic management solutions.While extensive research exists on the above aspects of AI applications in automotive contexts,there is a compelling need to synthesise this knowledge comprehensively to guide and inspire future research.This review introduces a novel taxonomic framework that provides a holistic perspective on AI integration into the automotive sector,focusing on next-generation AI methods and their critical implementation aspects.Additionally,the proposed conceptual framework for real-time condition monitoring of electric vehicle subsystems delivers actionable maintenance recommendations to stakeholders,addressing a critical gap in the field.The review highlights that AI has significantly expedited the development of autonomous vehicles regarding navigation,decision-making,and safety features through the use of advanced algorithms and deep learning structures.Furthermore,it identifies advanced driver assistance systems,vehicle health monitoring,and predictive maintenance as the most impactful AI applications,transforming operational safety and maintenance efficiency in modern automotive technologies.The work is beneficial to understanding the various use cases of AI in the different automotive domains,where AI maintains a state-of-the-art for sector-specific applications,providing a strong foundation for meeting Industry 4.0 needs and encouraging AI use among more nascent industry segments.The current work is intended to consolidate previous works while shedding some light on future research directions in promoting further growth of AI-based innovations in the scope of automotive applications.展开更多
Modern beyond visual range air game is characterized by high dynamic complexity and multi-factor interweaving,making it challenging for traditional decision-making methods.This paper constructs an autonomous decision-...Modern beyond visual range air game is characterized by high dynamic complexity and multi-factor interweaving,making it challenging for traditional decision-making methods.This paper constructs an autonomous decision-making framework for a single-plane beyond visual range air game that encompasses the entire game process.We design tactical rules with both offensive and defensive capabilities and formulate an autonomous air game decision-making method.This method is primarily driven by rules and is complemented by an intelligent maneuver decision-making approach based on the Double Deep Q-Network(DDQN).Simulation results demonstrate that our designed method effectively enhances the maneuvering decision-making ability of aircraft and has promising application prospects.展开更多
Improving the safety and efficiency of the autonomous driving decision-making process in complex traffic environments is a challenging issue.In this paper,a framework of decision-making algorithms that combines rules ...Improving the safety and efficiency of the autonomous driving decision-making process in complex traffic environments is a challenging issue.In this paper,a framework of decision-making algorithms that combines rules and reinforcement learning is proposed.The framework firstly applies rule-based and learning-based policies in parallel,and accurately evaluates the confidence level of both with the help of the data accumulated during the training process.Subsequently,through a activation function,the policy with the highest confidence level in the current context is automatically selected as the final execution plan.In order to verify the effectiveness of this scheme,a highway scenario is selected for this study to conduct simulation experiments.The results show that compared with the rule-based approach,the proposed method demonstrates a greater advantage in terms of reward,while compared with the learning-based approach,it achieves significant improvement in terms of decision stability.展开更多
文摘Autonomy, a key property associated with the agent, is an important topic in the current research of the agent theory. Although no definition of the agent autonomy is universally accepted, an important aspect of the agent autonomy is the decision-making capability of the agents. This paper investigates the autonomy of the agent, presents a framework for autonomous agent and discusses its decision-making process. Started with introducing a language for representing autonomous agent, a framework is proposed for modeling autonomous agent based on a BDI model and the situation calculus. Finally, a kind of decision-making process of the autonomous agent is presented.
文摘Planning and decision-making technology at intersections is a comprehensive research problem in intelligent transportation systems due to the uncertainties caused by a variety of traffic participants.As wireless communication advances,vehicle infrastructure integrated algorithms designed for intersection planning and decision-making have received increasing attention.In this paper,the recent studies on the planning and decision-making technologies at intersections are primarily overviewed.The general planning and decision-making approaches are presented,which include graph-based approach,prediction base approach,optimization-based approach and machine learning based approach.Since connected autonomous vehicles(CAVs)is the future direction for the automated driving area,we summarized the evolving planning and decision-making methods based on vehicle infrastructure cooperative technologies.Both four-way signalized and unsignalized intersection(s)are investigated under purely automated driving traffic and mixed traffic.The study benefit from current strategies,protocols,and simulation tools to help researchers identify the presented approaches’challenges and determine the research gaps,and several remaining possible research problems that need to be solved in the future.
基金supported by the National Natural Science Foundation of China(5187051675)。
文摘To improve the safety and driving stability of the autonomous heavy truck, it is necessary to consider the differences of driving behavior and drivable trajectories between the heavy trucks and passenger cars. This study proposes a probabilistic decision-making and trajectory planning framework for the autonomous heavy trucks. Firstly, the driving decision process is divided into intention generation and feasibility evaluations, which are realized using the utility theory and risk assessment, respectively. Subsequently the driving decision is made and sent to the trajectory planning module. In order to reflect the greater risks of the truck to other surrounding vehicles, the aggressiveness index(AI) is proposed and quantified to infer the asymmetrical risk level of lane-change maneuver. In the planning stage, the lateral and roll dynamics stability domains are developed as the constraints to exclude the candidate trajectories that would cause vehicle instability. Finally, the simulation results are compared between the proposed model and the artificial potential filed model in the scenarios extracted from the naturalistic driving data. It is shown that the proposed framework can provide the human-like lane-change decisions and truck-friendly trajectories, and performs well in dynamic driving environments.
基金the financial support of the National Key Research and Development Program of China(2020AAA0108100)the Shanghai Municipal Science and Technology Major Project(2021SHZDZX0100)the Shanghai Gaofeng and Gaoyuan Project for University Academic Program Development for funding。
文摘Decision-making and motion planning are extremely important in autonomous driving to ensure safe driving in a real-world environment.This study proposes an online evolutionary decision-making and motion planning framework for autonomous driving based on a hybrid data-and model-driven method.First,a data-driven decision-making module based on deep reinforcement learning(DRL)is developed to pursue a rational driving performance as much as possible.Then,model predictive control(MPC)is employed to execute both longitudinal and lateral motion planning tasks.Multiple constraints are defined according to the vehicle’s physical limit to meet the driving task requirements.Finally,two principles of safety and rationality for the self-evolution of autonomous driving are proposed.A motion envelope is established and embedded into a rational exploration and exploitation scheme,which filters out unreasonable experiences by masking unsafe actions so as to collect high-quality training data for the DRL agent.Experiments with a high-fidelity vehicle model and MATLAB/Simulink co-simulation environment are conducted,and the results show that the proposed online-evolution framework is able to generate safer,more rational,and more efficient driving action in a real-world environment.
基金supported by the National Key R&D Program of China (2022YFB2502900)the National Natural Science Foundation of China (62088102, 61790563)。
文摘With the maturation of autonomous driving technology, the use of autonomous vehicles in a socially acceptable manner has become a growing demand of the public. Human-like autonomous driving is expected due to the impact of the differences between autonomous vehicles and human drivers on safety.Although human-like decision-making has become a research hotspot, a unified theory has not yet been formed, and there are significant differences in the implementation and performance of existing methods. This paper provides a comprehensive overview of human-like decision-making for autonomous vehicles. The following issues are discussed: 1) The intelligence level of most autonomous driving decision-making algorithms;2) The driving datasets and simulation platforms for testing and verifying human-like decision-making;3) The evaluation metrics of human-likeness;personalized driving;the application of decisionmaking in real traffic scenarios;and 4) The potential research direction of human-like driving. These research results are significant for creating interpretable human-like driving models and applying them in dynamic traffic scenarios. In the future, the combination of intuitive logical reasoning and hierarchical structure will be an important topic for further research. It is expected to meet the needs of human-like driving.
基金Project(9142020013)support by the National Natural Science Foundation of China
文摘The decision-making under complex urban environment become one of the key issues that restricts the rapid development of the autonomous vehicles. The difficulty in making timely and accurate decisions like human beings under highly dynamic traffic environment is a major challenge for autonomous driving. Car-following has been regarded as the simplest but essential driving behavior among driving tasks and has received extensive attention from researchers around the world. This work addresses this problem and proposes a novel method RSAN(rough-set artificial neural network) to learn the decisions from excellent human drivers. A virtual urban traffic environment was built by Pre Scan and driving simulation was conducted to obtain a broad set of relevant data such as experienced drivers' behavior data and surrounding vehicles' motion data. Then, rough set was used to preprocess these data to extract the key influential factors on decision and reduce the impact of uncertain data and noise data. And the car-following decision was learned by neural network in which key factor was the input and acceleration was the output. The result shows the better convergence speed and the better decision accuracy of RSAN than ANN. Findings of this work contributes to the empirical understanding of driver's decision-making process and it provides a theoretical basis for the study of car-following decision-making under complex and dynamic environment.
基金supported in part by the Start-Up Grant-Nanyang Assistant Professorship Grant of Nanyang Technological Universitythe Agency for Science,Technology and Research(A*STAR)under Advanced Manufacturing and Engineering(AME)Young Individual Research under Grant(A2084c0156)+2 种基金the MTC Individual Research Grant(M22K2c0079)the ANR-NRF Joint Grant(NRF2021-NRF-ANR003 HM Science)the Ministry of Education(MOE)under the Tier 2 Grant(MOE-T2EP50222-0002)。
文摘While autonomous vehicles are vital components of intelligent transportation systems,ensuring the trustworthiness of decision-making remains a substantial challenge in realizing autonomous driving.Therefore,we present a novel robust reinforcement learning approach with safety guarantees to attain trustworthy decision-making for autonomous vehicles.The proposed technique ensures decision trustworthiness in terms of policy robustness and collision safety.Specifically,an adversary model is learned online to simulate the worst-case uncertainty by approximating the optimal adversarial perturbations on the observed states and environmental dynamics.In addition,an adversarial robust actor-critic algorithm is developed to enable the agent to learn robust policies against perturbations in observations and dynamics.Moreover,we devise a safety mask to guarantee the collision safety of the autonomous driving agent during both the training and testing processes using an interpretable knowledge model known as the Responsibility-Sensitive Safety Model.Finally,the proposed approach is evaluated through both simulations and experiments.These results indicate that the autonomous driving agent can make trustworthy decisions and drastically reduce the number of collisions through robust safety policies.
基金funded by Chongqing Science and Technology Bureau (No.cstc2021jsyj-yzysbAX0008)Chongqing University of Arts and Sciences (No.P2021JG13)2021 Humanities and Social Sciences Program of Chongqing Education Commission (No.21SKGH227).
文摘This paper is to explore the problems of intelligent connected vehicles(ICVs)autonomous driving decision-making under a 5G-V2X structured road environment.Through literature review and interviews with autonomous driving practitioners,this paper firstly puts forward a logical framework for designing a cerebrum-like autonomous driving system.Secondly,situated on this framework,it builds a hierarchical finite state machine(HFSM)model as well as a TOPSIS-GRA algorithm for making ICV autonomous driving decisions by employing a data fusion approach between the entropy weight method(EWM)and analytic hierarchy process method(AHP)and by employing a model fusion approach between the technique for order preference by similarity to an ideal solution(TOPSIS)and grey relational analysis(GRA).The HFSM model is composed of two layers:the global FSM model and the local FSM model.The decision of the former acts as partial input information of the latter and the result of the latter is sent forward to the local pathplanning module,meanwhile pulsating feedback to the former as real-time refresh data.To identify different traffic scenarios in a cerebrum-like way,the global FSM model is designed as 7 driving behavior states and 17 driving characteristic events,and the local FSM model is designed as 16 states and 8 characteristic events.In respect to designing a cerebrum-like algorithm for state transition,this paper firstly fuses AHP weight and EWM weight at their output layer to generate a synthetic weight coefficient for each characteristic event;then,it further fuses TOPSIS method and GRA method at the model building layer to obtain the implementable order of state transition.To verify the feasibility,reliability,and safety of theHFSMmodel aswell as its TOPSISGRA state transition algorithm,this paper elaborates on a series of simulative experiments conducted on the PreScan8.50 platform.The results display that the accuracy of obstacle detection gets 98%,lane line prediction is beyond 70 m,the speed of collision avoidance is higher than 45 km/h,the distance of collision avoidance is less than 5 m,path planning time for obstacle avoidance is averagely less than 50 ms,and brake deceleration is controlled under 6 m/s2.These technical indexes support that the driving states set and characteristic events set for the HFSM model as well as its TOPSIS-GRA algorithm may bring about cerebrum-like decision-making effectiveness for ICV autonomous driving under 5G-V2X intelligent road infrastructure.
基金Supported by National Key Research and Development Program of China(Grant No.2022YFE0117100)National Science Foundation of China(Grant No.52102468,52325212)Fundamental Research Funds for the Central Universities。
文摘To solve problems of poor security guarantee and insufficient training efficiency in the conventional reinforcement learning methods for decision-making,this study proposes a hybrid framework to combine deep reinforcement learning with rule-based decision-making methods.A risk assessment model for lane-change maneuvers considering uncertain predictions of surrounding vehicles is established as a safety filter to improve learning efficiency while correcting dangerous actions for safety enhancement.On this basis,a Risk-fused DDQN is constructed utilizing the model-based risk assessment and supervision mechanism.The proposed reinforcement learning algorithm sets up a separate experience buffer for dangerous trials and punishes such actions,which is shown to improve the sampling efficiency and training outcomes.Compared with conventional DDQN methods,the proposed algorithm improves the convergence value of cumulated reward by 7.6%and 2.2%in the two constructed scenarios in the simulation study and reduces the number of training episodes by 52.2%and 66.8%respectively.The success rate of lane change is improved by 57.3%while the time headway is increased at least by 16.5%in real vehicle tests,which confirms the higher training efficiency,scenario adaptability,and security of the proposed Risk-fused DDQN.
文摘Intelligent robots are increasingly being deployed across industries ranging from manufacturing to household applications and outdoor exploration.Their autonomous obstacle avoidance capabilities in complex environments have become a critical factor determining operational stability.Multimodal perception technology,which integrates visual,auditory,tactile,and LiDAR data,provides robots with comprehensive environmental awareness.By establishing efficient autonomous obstacle avoidance decision-making mechanisms based on this information,the system’s adaptability to challenging scenarios can be significantly enhanced.This study investigates the integration of multimodal perception with autonomous obstacle avoidance decision-making,analyzing the acquisition and processing of perceptual information,core modules and logic of decision-making mechanisms,and proposing optimization strategies for specific scenarios.The research aims to provide theoretical references for advancing autonomous obstacle avoidance technology in intelligent robots,enabling safer and more flexible movement in diverse environments.
基金supported by the National Key Research and Development Program of China(2017YFB0102601)the National Science Foundation of China(51775236).
文摘The anthropomorphic intelligence of autonomous driving has been a research hotspot in the world.However,current studies have not been able to reveal the mechanism of drivers'natural driving behaviors.Therefore,this thesis starts from the perspective of cognitive decision-making in the human brain,which is inspired by the regulation of dopamine feedback in the basal ganglia,and a reinforcement learning model is established to solve the brain-like intelligent decision-making problems in the process of interacting with the environment.In this thesis,first,a detailed bionic mechanism architecture based on basal ganglia was proposed by the consideration and analysis of its feedback regulation mechanism;second,the above mechanism was transformed into a reinforcement Q-learning model,so as to implement the learning and adaptation abilities of an intelligent vehicle for brain-like intelligent decision-making during car-following;finally,the feasibility and effectiveness of the proposed method were verified by the simulations and real vehicle tests.
基金supported by the Key Research and Development Program of Shaanxi (2022GXLH-02-09)the Aeronautical Science Foundation of China (20200051053001)the Natural Science Basic Research Program of Shaanxi (2020JM-147)。
文摘Autonomous umanned aerial vehicle(UAV) manipulation is necessary for the defense department to execute tactical missions given by commanders in the future unmanned battlefield. A large amount of research has been devoted to improving the autonomous decision-making ability of UAV in an interactive environment, where finding the optimal maneuvering decisionmaking policy became one of the key issues for enabling the intelligence of UAV. In this paper, we propose a maneuvering decision-making algorithm for autonomous air-delivery based on deep reinforcement learning under the guidance of expert experience. Specifically, we refine the guidance towards area and guidance towards specific point tasks for the air-delivery process based on the traditional air-to-surface fire control methods.Moreover, we construct the UAV maneuvering decision-making model based on Markov decision processes(MDPs). Specifically, we present a reward shaping method for the guidance towards area and guidance towards specific point tasks using potential-based function and expert-guided advice. The proposed algorithm could accelerate the convergence of the maneuvering decision-making policy and increase the stability of the policy in terms of the output during the later stage of training process. The effectiveness of the proposed maneuvering decision-making policy is illustrated by the curves of training parameters and extensive experimental results for testing the trained policy.
文摘Learning-based algorithm attracts great attention in the autonomous driving control field,especially for decisionmaking,to meet the challenge in long-tail extreme scenarios,where traditional methods demonstrate poor adaptability even with a significant effort.To improve the autonomous driving performance in extreme scenarios,specifically consecutive sharp turns,three deep reinforcement learning algorithms,i.e.Deep Deterministic Policy Gradient(DDPG),Twin Delayed Deep Deterministic policy gradient(TD3),and Soft Actor-Critic(SAC),based decision-making policies are proposed in this study.The role of the observation variable in agent training is discussed by comparing the driving stability,average speed,and consumed computational effort of the proposed algorithms in curves with various curvatures.In addition,a novel reward-setting method that combines the states of the environment and the vehicle is proposed to solve the sparse reward problem in the reward-guided algorithm.Simulation results from the road with consecutive sharp turns show that the DDPG,SAC,and TD3 algorithms-based vehicles take 367.2,359.6,and 302.1 s to finish the task,respectively,which match the training results,and verifies the observation variable role in agent quality improvement.
基金supported by the Jiangsu Key R&D Plan (Grant No.BE2018124)the National Natural Science Foundation of China (Grant Nos.51775007 and 51875279)the Postgraduate Research and Practice Innovation Program of Jiangsu Province (Grant No.KYCX190157)。
文摘In order to improve the agility and applicability of trajectory planning algorithm for autonomous vehicles, this paper proposes a novel actor-critic based learning method for decision-making and planning in multi-vehicle complex traffic. It is the coupling planning of vehicle’s path and speed thus to make the trajectory more flexible. First, generations from the decided action to the planned trajectory are described by the end-point of the trajectory. Then, the actor-critic based learning method is built to learn an optimal policy for the decision process. It can update the policy by the gradient of the current policy’s advantage. In this process,features of the real traffic are carefully extracted by time headway(TH) and speed distribution. Reward function is built by the safety, efficiency and driving comfort. Furthermore, to make the policy network have better convergency, the policy network is modularized in two parts: the lane-changing network and the lane-keeping network, which decide the optimal end-point of the path and speed candidates respectively. Finally, the curved overtaking scenario and the interaction process with human driver are conducted to illustrate the feasibility and superiority. The results show that the proposed method has better real-time performance and can make the planned coupling trajectory more continuous and smoother than the existing rule-based method.
基金supported by the National Natural Science Foundation of China(Grant No.51805332)the Young Elite Scientists Sponsorship Program funded by the China Society of Automotive Engineers,the Natural Science Foundation of Guangdong Province(Grant No.2018A030310532)the Shenzhen Fundamental Research Fund(Grant No.JCYJ20190808142613246).
文摘Road intersection is one of the most complex and accident-prone traffic scenarios,so it’s challenging for autonomous vehicles(AVs)to make safe and efficient decisions at the intersections.Most of the related studies focus on the solution to a single scenario or only guarantee safety without considering driving efficiency.To address these problems,this study proposed a deep reinforcement learning enabled decision-making framework for AVs to drive through intersections automatically,safely and efficiently.The mapping relationship between traffic images and vehicle operations was obtained by an end-to-end decision-making framework established by convolutional neural networks.Traffic images collected at two timesteps were used to calculate the relative velocity between vehicles.Markov decision process was employed to model the interaction between AVs and other vehicles,and the deep Q-network algorithm was utilized to obtain the optimal driving policy regarding safety and efficiency.To verify the effectiveness of the proposed decision-making framework,the top three accident-prone crossing path crash scenarios at intersections were simulated,when different initial vehicle states were adopted for better generalization capability.The results showed that the developed method could make AVs drive safely and efficiently through intersections in all of the tested scenarios.
文摘This paper presents a rule-based framework for addressing decision-making problems within the context of the "UI-STRIVE"Competition.First,two distinct autonomous confrontation scenarios are described:autonomous air combat and cooperative interception.Second,a State-Event-Condition-Action(SECA)decision-making framework is developed,which integrates thefinite state machine and event-condition-action frameworks.This framework provides three products to describe rules,i.e.the SECA model,the SECA state chart,and the SECA rule description.Third,the situation assessment and target assignment during autonomous air combat are investigated,and the mathematical models are established.Finally,the decisionmaking model's rationality and feasibility are verified through data simulation and analysis.
基金The authors are grateful to the Universiti Malaysia Pahang Al-Sultan Abdullah and the Malaysian Ministry of Higher Education for their generous support and funding provided through University Distinguished Research Grants(Project No.RDU223016)as well as financial assistance provided through the Fundamental Research Grant Scheme(No.FRGS/1/2022/TK10/UMP/02/35).
文摘The automotive sector is crucial in modern society,facilitating essential transportation needs across personal,commercial,and logistical domains while significantly contributing to national economic development and employment generation.The transformative impact of Artificial Intelligence(AI)has revolutionised multiple facets of the automotive industry,encompassing intelligent manufacturing processes,diagnostic systems,control mechanisms,supply chain operations,customer service platforms,and traffic management solutions.While extensive research exists on the above aspects of AI applications in automotive contexts,there is a compelling need to synthesise this knowledge comprehensively to guide and inspire future research.This review introduces a novel taxonomic framework that provides a holistic perspective on AI integration into the automotive sector,focusing on next-generation AI methods and their critical implementation aspects.Additionally,the proposed conceptual framework for real-time condition monitoring of electric vehicle subsystems delivers actionable maintenance recommendations to stakeholders,addressing a critical gap in the field.The review highlights that AI has significantly expedited the development of autonomous vehicles regarding navigation,decision-making,and safety features through the use of advanced algorithms and deep learning structures.Furthermore,it identifies advanced driver assistance systems,vehicle health monitoring,and predictive maintenance as the most impactful AI applications,transforming operational safety and maintenance efficiency in modern automotive technologies.The work is beneficial to understanding the various use cases of AI in the different automotive domains,where AI maintains a state-of-the-art for sector-specific applications,providing a strong foundation for meeting Industry 4.0 needs and encouraging AI use among more nascent industry segments.The current work is intended to consolidate previous works while shedding some light on future research directions in promoting further growth of AI-based innovations in the scope of automotive applications.
文摘Modern beyond visual range air game is characterized by high dynamic complexity and multi-factor interweaving,making it challenging for traditional decision-making methods.This paper constructs an autonomous decision-making framework for a single-plane beyond visual range air game that encompasses the entire game process.We design tactical rules with both offensive and defensive capabilities and formulate an autonomous air game decision-making method.This method is primarily driven by rules and is complemented by an intelligent maneuver decision-making approach based on the Double Deep Q-Network(DDQN).Simulation results demonstrate that our designed method effectively enhances the maneuvering decision-making ability of aircraft and has promising application prospects.
基金supported by the National Natural Science Foundation of China through Grant No.52202493.
文摘Improving the safety and efficiency of the autonomous driving decision-making process in complex traffic environments is a challenging issue.In this paper,a framework of decision-making algorithms that combines rules and reinforcement learning is proposed.The framework firstly applies rule-based and learning-based policies in parallel,and accurately evaluates the confidence level of both with the help of the data accumulated during the training process.Subsequently,through a activation function,the policy with the highest confidence level in the current context is automatically selected as the final execution plan.In order to verify the effectiveness of this scheme,a highway scenario is selected for this study to conduct simulation experiments.The results show that compared with the rule-based approach,the proposed method demonstrates a greater advantage in terms of reward,while compared with the learning-based approach,it achieves significant improvement in terms of decision stability.
