This paper conducts a comprehensive study on the multi-constrained two-on-one impulsive orbital pursuit–evasion game(OPEG).Firstly,considering constraints such as maneuverability,fuel reserves,and mission duration,a ...This paper conducts a comprehensive study on the multi-constrained two-on-one impulsive orbital pursuit–evasion game(OPEG).Firstly,considering constraints such as maneuverability,fuel reserves,and mission duration,a mathematical game model for the two-on-one impulsive OPEG is established,which transforms the two-on-one impulsive OPEG,where cooperation and competition coexist,into a multi-constrained three-party optimization problem suitable for solving with multi-agent deep reinforcement learning.Then,an intelligent solution method for cooperative game strategies based on the Multi-Agent Deep Deterministic Policy Gradient(MADDPG)algorithm is proposed.In the reward function design section,a reward function based on fixed-time triggering is introduced to address the information loss problem caused by long impulse intervals.To ensure good convergence of the algorithm and guide the spacecraft to learn effective cooperative strategies during training,an immediate reward function is designed,incorporating outcome rewards,guidance rewards,and cooperative rewards.Numerical simulations validate the feasibility and effectiveness of the proposed method.To further analyze the cooperative mechanisms learned by the spacecraft during algorithm training,a comparative experiment with the one-on-one impulsive OPEG is designed.The experimental results demonstrate that the two pursuers in the two-on-one impulsive OPEG not only develop various strategies such as“pre-emptive interception”,“pincer interception”,and“trailing pursuit”during training,but also improve mission success rates and reduce mission durations through coordinated efforts.Additionally,this paper reveals the impact of the relative initial state distribution between the two pursuing spacecraft and the evading spacecraft on the effectiveness of cooperation.展开更多
基金supported by National Key R&D Program of China:Gravitational Wave Detection Project(No.2021YFC22026,No.2021YFC2202601,No.2021YFC2202603)National Natural Science Foundation of China(No.12172288 and No.12472046).
文摘This paper conducts a comprehensive study on the multi-constrained two-on-one impulsive orbital pursuit–evasion game(OPEG).Firstly,considering constraints such as maneuverability,fuel reserves,and mission duration,a mathematical game model for the two-on-one impulsive OPEG is established,which transforms the two-on-one impulsive OPEG,where cooperation and competition coexist,into a multi-constrained three-party optimization problem suitable for solving with multi-agent deep reinforcement learning.Then,an intelligent solution method for cooperative game strategies based on the Multi-Agent Deep Deterministic Policy Gradient(MADDPG)algorithm is proposed.In the reward function design section,a reward function based on fixed-time triggering is introduced to address the information loss problem caused by long impulse intervals.To ensure good convergence of the algorithm and guide the spacecraft to learn effective cooperative strategies during training,an immediate reward function is designed,incorporating outcome rewards,guidance rewards,and cooperative rewards.Numerical simulations validate the feasibility and effectiveness of the proposed method.To further analyze the cooperative mechanisms learned by the spacecraft during algorithm training,a comparative experiment with the one-on-one impulsive OPEG is designed.The experimental results demonstrate that the two pursuers in the two-on-one impulsive OPEG not only develop various strategies such as“pre-emptive interception”,“pincer interception”,and“trailing pursuit”during training,but also improve mission success rates and reduce mission durations through coordinated efforts.Additionally,this paper reveals the impact of the relative initial state distribution between the two pursuing spacecraft and the evading spacecraft on the effectiveness of cooperation.
