Formation control of multiple unmanned aerial vehicles(UAVs)is a fundamental challenge for advanced cooperative tasks.In dense obstacle environments,machine learning-based formation control algorithms face significant...Formation control of multiple unmanned aerial vehicles(UAVs)is a fundamental challenge for advanced cooperative tasks.In dense obstacle environments,machine learning-based formation control algorithms face significant challenges due to the high environmental complexity and UAV dynamics,particularly manifested as an explosion in state space dimensionality and poor obstacle avoidance robustness.To address these issues,this paper proposes a threat-aware subspace feature extraction mechanism and constraint learning algorithm within a deep reinforcement learning(DRL)framework.Our approach first reconstructs spatial distributions of high-threat obstacles and UAV kinematic characteristics from LiDAR point cloud data to reduce state space dimensionality.Then,we solve the obstacle avoidance problem using a novel constrained reinforcement learning framework.This framework employs a safety-oriented penalty function instead of conventional posterior penalties to explicitly enforce safety constraints,thereby preventing dangerous actions.We rigorously prove the algorithm's convergence and stability using Lyapunov stability theory.Comparative experiments carried out in the high-fidelity AirSim environment have demonstrated that the proposed algorithm outperforms the state-of-the-art methods,where the convergence speed improves by 36.36%,stability increases by18.22%,and mission success rate rises by 5.6%.展开更多
基金supported by the National Natural Science Foundation of China(Grant Nos.U20B2001,12102178)。
文摘Formation control of multiple unmanned aerial vehicles(UAVs)is a fundamental challenge for advanced cooperative tasks.In dense obstacle environments,machine learning-based formation control algorithms face significant challenges due to the high environmental complexity and UAV dynamics,particularly manifested as an explosion in state space dimensionality and poor obstacle avoidance robustness.To address these issues,this paper proposes a threat-aware subspace feature extraction mechanism and constraint learning algorithm within a deep reinforcement learning(DRL)framework.Our approach first reconstructs spatial distributions of high-threat obstacles and UAV kinematic characteristics from LiDAR point cloud data to reduce state space dimensionality.Then,we solve the obstacle avoidance problem using a novel constrained reinforcement learning framework.This framework employs a safety-oriented penalty function instead of conventional posterior penalties to explicitly enforce safety constraints,thereby preventing dangerous actions.We rigorously prove the algorithm's convergence and stability using Lyapunov stability theory.Comparative experiments carried out in the high-fidelity AirSim environment have demonstrated that the proposed algorithm outperforms the state-of-the-art methods,where the convergence speed improves by 36.36%,stability increases by18.22%,and mission success rate rises by 5.6%.
