Presents the robot soccer software simulation platform to be firstly used at FIRA Robot World Cup China 2001, introduces the system’s purpose and design plan; discusses the system core server configuration and workin...Presents the robot soccer software simulation platform to be firstly used at FIRA Robot World Cup China 2001, introduces the system’s purpose and design plan; discusses the system core server configuration and working principle; describes the operating method and how to develop competition strategy, and refers to the teams to take part in FIRA Robot World Cup China 2001 and investigators who are interested in the distributed multi agent system.展开更多
Cooperative multi-agent reinforcement learning(MARL)is a key technology for enabling cooperation in complex multi-agent systems.It has achieved remarkable progress in areas such as gaming,autonomous driving,and multi-...Cooperative multi-agent reinforcement learning(MARL)is a key technology for enabling cooperation in complex multi-agent systems.It has achieved remarkable progress in areas such as gaming,autonomous driving,and multi-robot control.Empowering cooperative MARL with multi-task decision-making capabilities is expected to further broaden its application scope.In multi-task scenarios,cooperative MARL algorithms need to address 3 types of multi-task problems:reward-related multi-task,arising from different reward functions;multi-domain multi-task,caused by differences in state and action spaces,state transition functions;and scalability-related multi-task,resulting from the dynamic variation in the number of agents.Most existing studies focus on scalability-related multitask problems.However,with the increasing integration between large language models(LLMs)and multi-agent systems,a growing number of LLM-based multi-agent systems have emerged,enabling more complex multi-task cooperation.This paper provides a comprehensive review of the latest advances in this field.By combining multi-task reinforcement learning with cooperative MARL,we categorize and analyze the 3 major types of multi-task problems under multi-agent settings,offering more fine-grained classifications and summarizing key insights for each.In addition,we summarize commonly used benchmarks and discuss future directions of research in this area,which hold promise for further enhancing the multi-task cooperation capabilities of multi-agent systems and expanding their practical applications in the real world.展开更多
This paper addresses the consensus problem of nonlinear multi-agent systems subject to external disturbances and uncertainties under denial-ofservice(DoS)attacks.Firstly,an observer-based state feedback control method...This paper addresses the consensus problem of nonlinear multi-agent systems subject to external disturbances and uncertainties under denial-ofservice(DoS)attacks.Firstly,an observer-based state feedback control method is employed to achieve secure control by estimating the system's state in real time.Secondly,by combining a memory-based adaptive eventtriggered mechanism with neural networks,the paper aims to approximate the nonlinear terms in the networked system and efficiently conserve system resources.Finally,based on a two-degree-of-freedom model of a vehicle affected by crosswinds,this paper constructs a multi-unmanned ground vehicle(Multi-UGV)system to validate the effectiveness of the proposed method.Simulation results show that the proposed control strategy can effectively handle external disturbances such as crosswinds in practical applications,ensuring the stability and reliable operation of the Multi-UGV system.展开更多
This paper investigates the challenges associated with Unmanned Aerial Vehicle (UAV) collaborative search and target tracking in dynamic and unknown environments characterized by limited field of view. The primary obj...This paper investigates the challenges associated with Unmanned Aerial Vehicle (UAV) collaborative search and target tracking in dynamic and unknown environments characterized by limited field of view. The primary objective is to explore the unknown environments to locate and track targets effectively. To address this problem, we propose a novel Multi-Agent Reinforcement Learning (MARL) method based on Graph Neural Network (GNN). Firstly, a method is introduced for encoding continuous-space multi-UAV problem data into spatial graphs which establish essential relationships among agents, obstacles, and targets. Secondly, a Graph AttenTion network (GAT) model is presented, which focuses exclusively on adjacent nodes, learns attention weights adaptively and allows agents to better process information in dynamic environments. Reward functions are specifically designed to tackle exploration challenges in environments with sparse rewards. By introducing a framework that integrates centralized training and distributed execution, the advancement of models is facilitated. Simulation results show that the proposed method outperforms the existing MARL method in search rate and tracking performance with less collisions. The experiments show that the proposed method can be extended to applications with a larger number of agents, which provides a potential solution to the challenging problem of multi-UAV autonomous tracking in dynamic unknown environments.展开更多
An orderly competition mechanism is used to change unexpected competition into predictable competition so as to reduce access collision during access process.The scheme is realized by learning,queuing,and accessing.Qu...An orderly competition mechanism is used to change unexpected competition into predictable competition so as to reduce access collision during access process.The scheme is realized by learning,queuing,and accessing.Queuing is the key step to reduce random and realize orderly competition.Related parameters leading to access random including the arrival rate,the delay requirements,the number of devices,and so on,are defined as queue factors in this paper.The queue factors are obtained from the improved double deep Q network(DDQN)algorithm which is proposed here by setting asynchronous weights of two target networks.By learning,the queue factors will guide the devices with diverse delay requirements to queue.Then the queued devices start the access process according to their learning optimal access slot and preamble.Different from traditional competition solutions,markov decision process of the orderly competition mechanism has only two states,which remarkably cuts down the back-off rate and reduces the access delay.The simulation results show that the access success rate of this method can be close to 100%before the system capacity approaches the maximum value.展开更多
During the development of the nervous system,there is an overproduction of neurons and synapses.Hebbian competition between neighboring nerve endings and synapses performing different activity levels leads to their el...During the development of the nervous system,there is an overproduction of neurons and synapses.Hebbian competition between neighboring nerve endings and synapses performing different activity levels leads to their elimination or strengthening.We have extensively studied the involvement of the brain-derived neurotrophic factor-Tropomyosin-related kinase B receptor neurotrophic retrograde pathway,at the neuromuscular junction,in the axonal development and synapse elimination process versus the synapse consolidation.The purpose of this review is to describe the neurotrophic influence on developmental synapse elimination,in relation to other molecular pathways that we and others have found to regulate this process.In particular,we summarize our published results based on transmitter release analysis and axonal counts to show the different involvement of the presynaptic acetylcholine muscarinic autoreceptors,coupled to downstream serine-threonine protein kinases A and C(PKA and PKC)and voltage-gated calcium channels,at different nerve endings in developmental competition.The dynamic changes that occur simultaneously in several nerve terminals and synapses converge across a postsynaptic site,influence each other,and require careful studies to individualize the mechanisms of specific endings.We describe an activity-dependent balance(related to the extent of transmitter release)between the presynaptic muscarinic subtypes and the neurotrophin-mediated TrkB/p75NTR pathways that can influence the timing and fate of the competitive interactions between the different axon terminals.The downstream displacement of the PKA/PKC activity ratio to lower values,both in competing nerve terminals and at postsynaptic sites,plays a relevant role in controlling the elimination of supernumerary synapses.Finally,calcium entry through L-and P/Q-subtypes of voltage-gated calcium channels(both channels are present,together with the N-type channel in developing nerve terminals)contributes to reduce transmitter release and promote withdrawal of the most unfavorable nerve terminals during elimination(the weakest in acetylcholine release and those that have already become silent).The main findings contribute to a better understanding of punishment-rewarding interactions between nerve endings during development.Identifying the molecular targets and signaling pathways that allow synapse consolidation or withdrawal of synapses in different situations is important for potential therapies in neurodegenerative diseases.展开更多
This paper mainly focuses on the velocity-constrained consensus problem of discrete-time heterogeneous multi-agent systems with nonconvex constraints and arbitrarily switching topologies,where each agent has first-ord...This paper mainly focuses on the velocity-constrained consensus problem of discrete-time heterogeneous multi-agent systems with nonconvex constraints and arbitrarily switching topologies,where each agent has first-order or second-order dynamics.To solve this problem,a distributed algorithm is proposed based on a contraction operator.By employing the properties of the stochastic matrix,it is shown that all agents’position states could converge to a common point and second-order agents’velocity states could remain in corresponding nonconvex constraint sets and converge to zero as long as the joint communication topology has one directed spanning tree.Finally,the numerical simulation results are provided to verify the effectiveness of the proposed algorithms.展开更多
The development of chassis active safety control technology has improved vehicle stability under extreme conditions.However,its cross-system and multi-functional characteristics make the controller difficult to achiev...The development of chassis active safety control technology has improved vehicle stability under extreme conditions.However,its cross-system and multi-functional characteristics make the controller difficult to achieve cooperative goals.In addition,the chassis system,which has high complexity,numerous subsystems,and strong coupling,will also lead to low computing efficiency and poor control effect of the controller.Therefore,this paper proposes a scenario-driven hybrid distributed model predictive control algorithm with variable control topology.This algorithm divides multiple stability regions based on the vehicle’s β−γ phase plane,forming a mapping relationship between the control structure and the vehicle’s state.A control input fusion mechanism within the transition domain is designed to mitigate the problems of system state oscillation and control input jitter caused by switching control structures.Then,a distributed state-space equation with state coupling and input coupling characteristics is constructed,and a weighted local agent cost function in quadratic programming is derived.Through cost coupling,local agents can coordinate global performance goals.Finally,through Simulink/CarSim joint simulation and hardware-in-the-loop(HIL)test,the proposed algorithm is validated to improve vehicle stability while ensuring trajectory tracking accuracy and has good applicability for multi-objective coordinated control.This paper combines the advantages of distributed MPC and decentralized MPC,achieving a balance between approximating the global optimal results and the solution’s efficiency.展开更多
This paper focuses on the problem of leaderfollowing consensus for nonlinear cascaded multi-agent systems.The control strategies for these systems are transformed into successive control problem schemes for lower-orde...This paper focuses on the problem of leaderfollowing consensus for nonlinear cascaded multi-agent systems.The control strategies for these systems are transformed into successive control problem schemes for lower-order error subsystems.A distributed consensus analysis for the corresponding error systems is conducted by employing recursive methods and virtual controllers,accompanied by a series of Lyapunov functions devised throughout the iterative process,which solves the leaderfollowing consensus problem of a class of nonlinear cascaded multi-agent systems.Specific simulation examples illustrate the effectiveness of the proposed control algorithm.展开更多
In this paper,the distributed optimal formation control problem of heterogeneous Euler–Lagrange multi-agent systems with generic formation constraints and inequality constraints is investigated.Based on the primal–d...In this paper,the distributed optimal formation control problem of heterogeneous Euler–Lagrange multi-agent systems with generic formation constraints and inequality constraints is investigated.Based on the primal–dual dynamics and the adaptive control technique,a distributed optimal formation controller consists of a velocity reference signal generator and a velocity tracking controller is proposed.By using the optimality condition,the relationship between the equilibrium point of the closed-loop system and the optimal solution of the optimization problem is established.Then,by utilizing Lyapunov stability analysis,it is rigorously proved that the optimal formation is reached with the proposed controller.Lastly,simulation examples are provided to substantiate the theoretical results.展开更多
This article investigates the time-varying output group formation tracking control(GFTC)problem for heterogeneous multi-agent systems(HMASs)under switching topologies.The objective is to design a distributed control s...This article investigates the time-varying output group formation tracking control(GFTC)problem for heterogeneous multi-agent systems(HMASs)under switching topologies.The objective is to design a distributed control strategy that enables the outputs of the followers to form the desired sub-formations and track the outputs of the leader in each subgroup.Firstly,novel distributed observers are developed to estimate the states of the leaders under switching topologies.Then,GFTC protocols are designed based on the proposed observers.It is shown that with the distributed protocol,the GFTC problem for HMASs under switching topologies is solved if the average dwell time associated with the switching topologies is larger than a fixed threshold.Finally,an example is provided to illustrate the effectiveness of the proposed control strategy.展开更多
When a fire breaks out in a high-rise building,the occlusion of smoke and obstacles results in dearth of crucial information concerning people in distress,thereby creating a challenge in their detection.Given the rest...When a fire breaks out in a high-rise building,the occlusion of smoke and obstacles results in dearth of crucial information concerning people in distress,thereby creating a challenge in their detection.Given the restricted sensing range of a single unmanned aerial vehicle(UAV)cam-era,enhancing the target recognition rate becomes challenging without target information.To tackle this issue,this paper proposes a multi-agent autonomous collaborative detection method for multi-targets in complex fire environments.The objective is to achieve the fusion of multi-angle visual information,effectively increasing the target’s information dimension,and ultimately address-ing the problem of low target recognition rate caused by the lack of target information.The method steps are as follows:first,the you only look once version5(YOLOv5)is used to detect the target in the image;second,the detected targets are tracked to monitor their movements and trajectories;third,the person re-identification(ReID)model is employed to extract the appearance features of targets;finally,by fusing the visual information from multi-angle cameras,the method achieves multi-agent autonomous collaborative detection.The experimental results show that the method effectively combines the visual information from multi-angle cameras,resulting in improved detec-tion efficiency for people in distress.展开更多
This paper proposes a Multi-Agent Attention Proximal Policy Optimization(MA2PPO)algorithm aiming at the problems such as credit assignment,low collaboration efficiency and weak strategy generalization ability existing...This paper proposes a Multi-Agent Attention Proximal Policy Optimization(MA2PPO)algorithm aiming at the problems such as credit assignment,low collaboration efficiency and weak strategy generalization ability existing in the cooperative pursuit tasks of multiple unmanned aerial vehicles(UAVs).Traditional algorithms often fail to effectively identify critical cooperative relationships in such tasks,leading to low capture efficiency and a significant decline in performance when the scale expands.To tackle these issues,based on the proximal policy optimization(PPO)algorithm,MA2PPO adopts the centralized training with decentralized execution(CTDE)framework and introduces a dynamic decoupling mechanism,that is,sharing the multi-head attention(MHA)mechanism for critics during centralized training to solve the credit assignment problem.This method enables the pursuers to identify highly correlated interactions with their teammates,effectively eliminate irrelevant and weakly relevant interactions,and decompose large-scale cooperation problems into decoupled sub-problems,thereby enhancing the collaborative efficiency and policy stability among multiple agents.Furthermore,a reward function has been devised to facilitate the pursuers to encircle the escapee by combining a formation reward with a distance reward,which incentivizes UAVs to develop sophisticated cooperative pursuit strategies.Experimental results demonstrate the effectiveness of the proposed algorithm in achieving multi-UAV cooperative pursuit and inducing diverse cooperative pursuit behaviors among UAVs.Moreover,experiments on scalability have demonstrated that the algorithm is suitable for large-scale multi-UAV systems.展开更多
Droughts have caused tree growth decline and high tree mortality across temperate forests,however,how to manage planted forests to alleviate drought stress is still challenging.We used tree-ring and forest inventory d...Droughts have caused tree growth decline and high tree mortality across temperate forests,however,how to manage planted forests to alleviate drought stress is still challenging.We used tree-ring and forest inventory data from different density stands to investigate how competition,tree diameter at breast height(DBH),tree age,and their interactions influence drought sensitivity and resistance for three widely-distributed and planted conifer species(Larix principis-rupprechtii,Picea meyeri,and Pinus sylvestris var.mongolica).Our results showed that the drought sensitivity of the three species was influenced by competition,tree size,and their interactions.Large L.principis-rupprechtii trees were particularly sensitive to drought during the growing season in medium to high-density stands,while the growth of large P.sylvestris var.mongolica was most affected by precipitation at low to medium density stands.Drought resistance of L.principis-rupprechtii trees decreased as tree size increased.Large L.principis-rupprechtii trees had lower drought resistance than small trees in all stands.Drought resistance of large P.meyeri trees exhibited high resistance to drought only in high-density stands.However,drought resistance of P.sylvestris var.mongolica trees was affected by tree size,competition,and their interactions.These results indicated that targeted silvicultural interventions,such as thinning,can be implemented to enhance drought resistance specifically for large L.principis-rupprechtii trees and small P.sylvestris var.mongolica trees in medium and high competition stands,and small P.meyeri trees in high competition stands.Our results highlight that properly conducted thinning can in some cases enhance growth resistance to droughts,depending on stand density,tree size,and tree species.展开更多
Brain size varies dramatically across populations and species in anuran species.The differences in structure,function,or size of brains are linked to processing specific cognitive tasks by different behaviors.In parti...Brain size varies dramatically across populations and species in anuran species.The differences in structure,function,or size of brains are linked to processing specific cognitive tasks by different behaviors.In particular,the causes of how male-male competition promotes the increased cognitive abilities to increase brains are as yet unexplored in anurans.To evaluate the effect of male-male competition on variation in brain size in B.andrewsi,we compared the differences in relative brain size between mated males and unpaired males under natural and experimental conditions.We found that mated males had relatively larger brains than unpaired males in a natural population when controlling the effect of body size.Likewise,we also found that there were larger brains in mated males than in unpaired males in both experiment 1 where two males competed for a female and experiment 2 where three males competed for a female,suggesting that males with mating success during male-male competition possess increased brain size and cognitive abilities.When we compared difference in relative brain size in mated males between experiment2 and experiment 1 we found that males experiencing more intense competition did not display larger brains than males experiencing relatively weak competition,suggesting that low intensity competition is already enough to trigger the increase in relative brain size in B.andrewsi.展开更多
Continuous control protocols are extensively utilized in traditional MASs,in which information needs to be transmitted among agents consecutively,therefore resulting in excessive consumption of limited resources.To de...Continuous control protocols are extensively utilized in traditional MASs,in which information needs to be transmitted among agents consecutively,therefore resulting in excessive consumption of limited resources.To decrease the control cost,based on ISC,several LFC problems are investigated for second-order MASs without and with time delay,respectively.Firstly,an intermittent sampled controller is designed,and a sufficient and necessary condition is derived,under which state errors between the leader and all the followers approach zero asymptotically.Considering that time delay is inevitable,a new protocol is proposed to deal with the time-delay situation.The error system’s stability is analyzed using the Schur stability theorem,and sufficient and necessary conditions for LFC are obtained,which are closely associated with the coupling gain,the system parameters,and the network structure.Furthermore,for the case where the current position and velocity information are not available,a distributed protocol is designed that depends only on the sampled position information.The sufficient and necessary conditions for LFC are also given.The results show that second-order MASs can achieve the LFC if and only if the system parameters satisfy the inequalities proposed in the paper.Finally,the correctness of the obtained results is verified by numerical simulations.展开更多
Formation control in multi-agent systems has become a critical area of interest due to its wide-ranging applications in robotics,autonomous transportation,and surveillance.While various studies have explored distribut...Formation control in multi-agent systems has become a critical area of interest due to its wide-ranging applications in robotics,autonomous transportation,and surveillance.While various studies have explored distributed cooperative control,this review focuses on the theoretical foundations and recent developments in formation control strategies.The paper categorizes and analyzes key formation types,including formation maintenance,group or cluster formation,bipartite formations,event-triggered formations,finite-time convergence,and constrained formations.A significant portion of the review addresses formation control under constrained dynamics,presenting both modelbased and model-free approaches that consider practical limitations such as actuator bounds,communication delays,and nonholonomic constraints.Additionally,the paper discusses emerging trends,including the integration of eventdriven mechanisms and AI-enhanced coordination strategies.Comparative evaluations highlight the trade-offs among various methodologies regarding scalability,robustness,and real-world feasibility.Practical implementations are reviewed across diverse platforms,and the review identifies the current achievements and unresolved challenges in the field.The paper concludes by outlining promising research directions,such as adaptive control for dynamic environments,energy-efficient coordination,and using learning-based control under uncertainty.This review synthesizes the current state of the art and provides a road map for future investigation,making it a valuable reference for researchers and practitioners aiming to advance formation control in multi-agent systems.展开更多
In recent years,significant research attention has been directed towards swarm intelligence.The Milling behavior of fish schools,a prime example of swarm intelligence,shows how simple rules followed by individual agen...In recent years,significant research attention has been directed towards swarm intelligence.The Milling behavior of fish schools,a prime example of swarm intelligence,shows how simple rules followed by individual agents lead to complex collective behaviors.This paper studies Multi-Agent Reinforcement Learning to simulate fish schooling behavior,overcoming the challenges of tuning parameters in traditional models and addressing the limitations of single-agent methods in multi-agent environments.Based on this foundation,a novel Graph Convolutional Networks(GCN)-Critic MADDPG algorithm leveraging GCN is proposed to enhance cooperation among agents in a multi-agent system.Simulation experiments demonstrate that,compared to traditional single-agent algorithms,the proposed method not only exhibits significant advantages in terms of convergence speed and stability but also achieves tighter group formations and more naturally aligned Milling behavior.Additionally,a fish school self-organizing behavior research platform based on an event-triggered mechanism has been developed,providing a robust tool for exploring dynamic behavioral changes under various conditions.展开更多
The Internet of Unmanned Aerial Vehicles(I-UAVs)is expected to execute latency-sensitive tasks,but limited by co-channel interference and malicious jamming.In the face of unknown prior environmental knowledge,defendin...The Internet of Unmanned Aerial Vehicles(I-UAVs)is expected to execute latency-sensitive tasks,but limited by co-channel interference and malicious jamming.In the face of unknown prior environmental knowledge,defending against jamming and interference through spectrum allocation becomes challenging,especially when each UAV pair makes decisions independently.In this paper,we propose a cooperative multi-agent reinforcement learning(MARL)-based anti-jamming framework for I-UAVs,enabling UAV pairs to learn their own policies cooperatively.Specifically,we first model the problem as a modelfree multi-agent Markov decision process(MAMDP)to maximize the long-term expected system throughput.Then,for improving the exploration of the optimal policy,we resort to optimizing a MARL objective function with a mutual-information(MI)regularizer between states and actions,which can dynamically assign the probability for actions frequently used by the optimal policy.Next,through sharing their current channel selections and local learning experience(their soft Q-values),the UAV pairs can learn their own policies cooperatively relying on only preceding observed information and predicting others’actions.Our simulation results show that for both sweep jamming and Markov jamming patterns,the proposed scheme outperforms the benchmarkers in terms of throughput,convergence and stability for different numbers of jammers,channels and UAV pairs.展开更多
文摘Presents the robot soccer software simulation platform to be firstly used at FIRA Robot World Cup China 2001, introduces the system’s purpose and design plan; discusses the system core server configuration and working principle; describes the operating method and how to develop competition strategy, and refers to the teams to take part in FIRA Robot World Cup China 2001 and investigators who are interested in the distributed multi agent system.
基金The National Natural Science Foundation of China(62136008,62293541)The Beijing Natural Science Foundation(4232056)The Beijing Nova Program(20240484514).
文摘Cooperative multi-agent reinforcement learning(MARL)is a key technology for enabling cooperation in complex multi-agent systems.It has achieved remarkable progress in areas such as gaming,autonomous driving,and multi-robot control.Empowering cooperative MARL with multi-task decision-making capabilities is expected to further broaden its application scope.In multi-task scenarios,cooperative MARL algorithms need to address 3 types of multi-task problems:reward-related multi-task,arising from different reward functions;multi-domain multi-task,caused by differences in state and action spaces,state transition functions;and scalability-related multi-task,resulting from the dynamic variation in the number of agents.Most existing studies focus on scalability-related multitask problems.However,with the increasing integration between large language models(LLMs)and multi-agent systems,a growing number of LLM-based multi-agent systems have emerged,enabling more complex multi-task cooperation.This paper provides a comprehensive review of the latest advances in this field.By combining multi-task reinforcement learning with cooperative MARL,we categorize and analyze the 3 major types of multi-task problems under multi-agent settings,offering more fine-grained classifications and summarizing key insights for each.In addition,we summarize commonly used benchmarks and discuss future directions of research in this area,which hold promise for further enhancing the multi-task cooperation capabilities of multi-agent systems and expanding their practical applications in the real world.
基金The National Natural Science Foundation of China(W2431048)The Science and Technology Research Program of Chongqing Municipal Education Commission,China(KJZDK202300807)The Chongqing Natural Science Foundation,China(CSTB2024NSCQQCXMX0052).
文摘This paper addresses the consensus problem of nonlinear multi-agent systems subject to external disturbances and uncertainties under denial-ofservice(DoS)attacks.Firstly,an observer-based state feedback control method is employed to achieve secure control by estimating the system's state in real time.Secondly,by combining a memory-based adaptive eventtriggered mechanism with neural networks,the paper aims to approximate the nonlinear terms in the networked system and efficiently conserve system resources.Finally,based on a two-degree-of-freedom model of a vehicle affected by crosswinds,this paper constructs a multi-unmanned ground vehicle(Multi-UGV)system to validate the effectiveness of the proposed method.Simulation results show that the proposed control strategy can effectively handle external disturbances such as crosswinds in practical applications,ensuring the stability and reliable operation of the Multi-UGV system.
基金supported by the National Natural Science Foundation of China(Nos.12272104,U22B2013).
文摘This paper investigates the challenges associated with Unmanned Aerial Vehicle (UAV) collaborative search and target tracking in dynamic and unknown environments characterized by limited field of view. The primary objective is to explore the unknown environments to locate and track targets effectively. To address this problem, we propose a novel Multi-Agent Reinforcement Learning (MARL) method based on Graph Neural Network (GNN). Firstly, a method is introduced for encoding continuous-space multi-UAV problem data into spatial graphs which establish essential relationships among agents, obstacles, and targets. Secondly, a Graph AttenTion network (GAT) model is presented, which focuses exclusively on adjacent nodes, learns attention weights adaptively and allows agents to better process information in dynamic environments. Reward functions are specifically designed to tackle exploration challenges in environments with sparse rewards. By introducing a framework that integrates centralized training and distributed execution, the advancement of models is facilitated. Simulation results show that the proposed method outperforms the existing MARL method in search rate and tracking performance with less collisions. The experiments show that the proposed method can be extended to applications with a larger number of agents, which provides a potential solution to the challenging problem of multi-UAV autonomous tracking in dynamic unknown environments.
基金supported by Key Technologies R&D Program of Jiangsu(Prospective and Key Technologies for Industry)(BE2022067,BE2022067-1 and BE2022067-2)National Natural Science Foundation of China(No.61771255)Provincial and Ministerial Key Laboratory Open Project under Grant No.20190904.
文摘An orderly competition mechanism is used to change unexpected competition into predictable competition so as to reduce access collision during access process.The scheme is realized by learning,queuing,and accessing.Queuing is the key step to reduce random and realize orderly competition.Related parameters leading to access random including the arrival rate,the delay requirements,the number of devices,and so on,are defined as queue factors in this paper.The queue factors are obtained from the improved double deep Q network(DDQN)algorithm which is proposed here by setting asynchronous weights of two target networks.By learning,the queue factors will guide the devices with diverse delay requirements to queue.Then the queued devices start the access process according to their learning optimal access slot and preamble.Different from traditional competition solutions,markov decision process of the orderly competition mechanism has only two states,which remarkably cuts down the back-off rate and reduces the access delay.The simulation results show that the access success rate of this method can be close to 100%before the system capacity approaches the maximum value.
基金supported by Catalan Government,Nos.2014SGR344(to JT),2017SGR704(to JT),2021SGR01214(to MAL)MCIN/AEI/10.13039/501100011033/by“ERDF A way of making Europe,”Nos.SAF2015-67143(to JT),PID2019-106332GB-I00(to JT and MAL)and PID2022-141252NB-I00(to MAL).
文摘During the development of the nervous system,there is an overproduction of neurons and synapses.Hebbian competition between neighboring nerve endings and synapses performing different activity levels leads to their elimination or strengthening.We have extensively studied the involvement of the brain-derived neurotrophic factor-Tropomyosin-related kinase B receptor neurotrophic retrograde pathway,at the neuromuscular junction,in the axonal development and synapse elimination process versus the synapse consolidation.The purpose of this review is to describe the neurotrophic influence on developmental synapse elimination,in relation to other molecular pathways that we and others have found to regulate this process.In particular,we summarize our published results based on transmitter release analysis and axonal counts to show the different involvement of the presynaptic acetylcholine muscarinic autoreceptors,coupled to downstream serine-threonine protein kinases A and C(PKA and PKC)and voltage-gated calcium channels,at different nerve endings in developmental competition.The dynamic changes that occur simultaneously in several nerve terminals and synapses converge across a postsynaptic site,influence each other,and require careful studies to individualize the mechanisms of specific endings.We describe an activity-dependent balance(related to the extent of transmitter release)between the presynaptic muscarinic subtypes and the neurotrophin-mediated TrkB/p75NTR pathways that can influence the timing and fate of the competitive interactions between the different axon terminals.The downstream displacement of the PKA/PKC activity ratio to lower values,both in competing nerve terminals and at postsynaptic sites,plays a relevant role in controlling the elimination of supernumerary synapses.Finally,calcium entry through L-and P/Q-subtypes of voltage-gated calcium channels(both channels are present,together with the N-type channel in developing nerve terminals)contributes to reduce transmitter release and promote withdrawal of the most unfavorable nerve terminals during elimination(the weakest in acetylcholine release and those that have already become silent).The main findings contribute to a better understanding of punishment-rewarding interactions between nerve endings during development.Identifying the molecular targets and signaling pathways that allow synapse consolidation or withdrawal of synapses in different situations is important for potential therapies in neurodegenerative diseases.
基金2024 Jiangsu Province Youth Science and Technology Talent Support Project2024 Yancheng Key Research and Development Plan(Social Development)projects,“Research and Application of Multi Agent Offline Distributed Trust Perception Virtual Wireless Sensor Network Algorithm”and“Research and Application of a New Type of Fishery Ship Safety Production Monitoring Equipment”。
文摘This paper mainly focuses on the velocity-constrained consensus problem of discrete-time heterogeneous multi-agent systems with nonconvex constraints and arbitrarily switching topologies,where each agent has first-order or second-order dynamics.To solve this problem,a distributed algorithm is proposed based on a contraction operator.By employing the properties of the stochastic matrix,it is shown that all agents’position states could converge to a common point and second-order agents’velocity states could remain in corresponding nonconvex constraint sets and converge to zero as long as the joint communication topology has one directed spanning tree.Finally,the numerical simulation results are provided to verify the effectiveness of the proposed algorithms.
基金Supported by National Natural Science Foundation of China(Grant Nos.52225212,52272418,U22A20100)National Key Research and Development Program of China(Grant No.2022YFB2503302).
文摘The development of chassis active safety control technology has improved vehicle stability under extreme conditions.However,its cross-system and multi-functional characteristics make the controller difficult to achieve cooperative goals.In addition,the chassis system,which has high complexity,numerous subsystems,and strong coupling,will also lead to low computing efficiency and poor control effect of the controller.Therefore,this paper proposes a scenario-driven hybrid distributed model predictive control algorithm with variable control topology.This algorithm divides multiple stability regions based on the vehicle’s β−γ phase plane,forming a mapping relationship between the control structure and the vehicle’s state.A control input fusion mechanism within the transition domain is designed to mitigate the problems of system state oscillation and control input jitter caused by switching control structures.Then,a distributed state-space equation with state coupling and input coupling characteristics is constructed,and a weighted local agent cost function in quadratic programming is derived.Through cost coupling,local agents can coordinate global performance goals.Finally,through Simulink/CarSim joint simulation and hardware-in-the-loop(HIL)test,the proposed algorithm is validated to improve vehicle stability while ensuring trajectory tracking accuracy and has good applicability for multi-objective coordinated control.This paper combines the advantages of distributed MPC and decentralized MPC,achieving a balance between approximating the global optimal results and the solution’s efficiency.
基金National Natural Science Foundation of China(No.12071370)。
文摘This paper focuses on the problem of leaderfollowing consensus for nonlinear cascaded multi-agent systems.The control strategies for these systems are transformed into successive control problem schemes for lower-order error subsystems.A distributed consensus analysis for the corresponding error systems is conducted by employing recursive methods and virtual controllers,accompanied by a series of Lyapunov functions devised throughout the iterative process,which solves the leaderfollowing consensus problem of a class of nonlinear cascaded multi-agent systems.Specific simulation examples illustrate the effectiveness of the proposed control algorithm.
基金supported in part by the National Key Research and Development Program of China under Grant 2022YFB3303900in part by the National Natural Science Foundation of China under Grants 62103277 and 62025305。
文摘In this paper,the distributed optimal formation control problem of heterogeneous Euler–Lagrange multi-agent systems with generic formation constraints and inequality constraints is investigated.Based on the primal–dual dynamics and the adaptive control technique,a distributed optimal formation controller consists of a velocity reference signal generator and a velocity tracking controller is proposed.By using the optimality condition,the relationship between the equilibrium point of the closed-loop system and the optimal solution of the optimization problem is established.Then,by utilizing Lyapunov stability analysis,it is rigorously proved that the optimal formation is reached with the proposed controller.Lastly,simulation examples are provided to substantiate the theoretical results.
文摘This article investigates the time-varying output group formation tracking control(GFTC)problem for heterogeneous multi-agent systems(HMASs)under switching topologies.The objective is to design a distributed control strategy that enables the outputs of the followers to form the desired sub-formations and track the outputs of the leader in each subgroup.Firstly,novel distributed observers are developed to estimate the states of the leaders under switching topologies.Then,GFTC protocols are designed based on the proposed observers.It is shown that with the distributed protocol,the GFTC problem for HMASs under switching topologies is solved if the average dwell time associated with the switching topologies is larger than a fixed threshold.Finally,an example is provided to illustrate the effectiveness of the proposed control strategy.
文摘When a fire breaks out in a high-rise building,the occlusion of smoke and obstacles results in dearth of crucial information concerning people in distress,thereby creating a challenge in their detection.Given the restricted sensing range of a single unmanned aerial vehicle(UAV)cam-era,enhancing the target recognition rate becomes challenging without target information.To tackle this issue,this paper proposes a multi-agent autonomous collaborative detection method for multi-targets in complex fire environments.The objective is to achieve the fusion of multi-angle visual information,effectively increasing the target’s information dimension,and ultimately address-ing the problem of low target recognition rate caused by the lack of target information.The method steps are as follows:first,the you only look once version5(YOLOv5)is used to detect the target in the image;second,the detected targets are tracked to monitor their movements and trajectories;third,the person re-identification(ReID)model is employed to extract the appearance features of targets;finally,by fusing the visual information from multi-angle cameras,the method achieves multi-agent autonomous collaborative detection.The experimental results show that the method effectively combines the visual information from multi-angle cameras,resulting in improved detec-tion efficiency for people in distress.
基金supported by the National Research and Development Program of China under Grant JCKY2018607C019in part by the Key Laboratory Fund of UAV of Northwestern Polytechnical University under Grant 2021JCJQLB0710L.
文摘This paper proposes a Multi-Agent Attention Proximal Policy Optimization(MA2PPO)algorithm aiming at the problems such as credit assignment,low collaboration efficiency and weak strategy generalization ability existing in the cooperative pursuit tasks of multiple unmanned aerial vehicles(UAVs).Traditional algorithms often fail to effectively identify critical cooperative relationships in such tasks,leading to low capture efficiency and a significant decline in performance when the scale expands.To tackle these issues,based on the proximal policy optimization(PPO)algorithm,MA2PPO adopts the centralized training with decentralized execution(CTDE)framework and introduces a dynamic decoupling mechanism,that is,sharing the multi-head attention(MHA)mechanism for critics during centralized training to solve the credit assignment problem.This method enables the pursuers to identify highly correlated interactions with their teammates,effectively eliminate irrelevant and weakly relevant interactions,and decompose large-scale cooperation problems into decoupled sub-problems,thereby enhancing the collaborative efficiency and policy stability among multiple agents.Furthermore,a reward function has been devised to facilitate the pursuers to encircle the escapee by combining a formation reward with a distance reward,which incentivizes UAVs to develop sophisticated cooperative pursuit strategies.Experimental results demonstrate the effectiveness of the proposed algorithm in achieving multi-UAV cooperative pursuit and inducing diverse cooperative pursuit behaviors among UAVs.Moreover,experiments on scalability have demonstrated that the algorithm is suitable for large-scale multi-UAV systems.
基金funded by State Key Research and Development Program of China(No.2023YFD2200401)S&T Program of Hebei(Nos.226Z6801G,22326807D)Talent Introduction Program in Hebei Agricultural University(No.YJ201918).
文摘Droughts have caused tree growth decline and high tree mortality across temperate forests,however,how to manage planted forests to alleviate drought stress is still challenging.We used tree-ring and forest inventory data from different density stands to investigate how competition,tree diameter at breast height(DBH),tree age,and their interactions influence drought sensitivity and resistance for three widely-distributed and planted conifer species(Larix principis-rupprechtii,Picea meyeri,and Pinus sylvestris var.mongolica).Our results showed that the drought sensitivity of the three species was influenced by competition,tree size,and their interactions.Large L.principis-rupprechtii trees were particularly sensitive to drought during the growing season in medium to high-density stands,while the growth of large P.sylvestris var.mongolica was most affected by precipitation at low to medium density stands.Drought resistance of L.principis-rupprechtii trees decreased as tree size increased.Large L.principis-rupprechtii trees had lower drought resistance than small trees in all stands.Drought resistance of large P.meyeri trees exhibited high resistance to drought only in high-density stands.However,drought resistance of P.sylvestris var.mongolica trees was affected by tree size,competition,and their interactions.These results indicated that targeted silvicultural interventions,such as thinning,can be implemented to enhance drought resistance specifically for large L.principis-rupprechtii trees and small P.sylvestris var.mongolica trees in medium and high competition stands,and small P.meyeri trees in high competition stands.Our results highlight that properly conducted thinning can in some cases enhance growth resistance to droughts,depending on stand density,tree size,and tree species.
基金funded by the National Natural Science Foundation of China (32370456)the Key Project of Natural Sciences Foundation of Sichuan Province (22NSFSC0011)。
文摘Brain size varies dramatically across populations and species in anuran species.The differences in structure,function,or size of brains are linked to processing specific cognitive tasks by different behaviors.In particular,the causes of how male-male competition promotes the increased cognitive abilities to increase brains are as yet unexplored in anurans.To evaluate the effect of male-male competition on variation in brain size in B.andrewsi,we compared the differences in relative brain size between mated males and unpaired males under natural and experimental conditions.We found that mated males had relatively larger brains than unpaired males in a natural population when controlling the effect of body size.Likewise,we also found that there were larger brains in mated males than in unpaired males in both experiment 1 where two males competed for a female and experiment 2 where three males competed for a female,suggesting that males with mating success during male-male competition possess increased brain size and cognitive abilities.When we compared difference in relative brain size in mated males between experiment2 and experiment 1 we found that males experiencing more intense competition did not display larger brains than males experiencing relatively weak competition,suggesting that low intensity competition is already enough to trigger the increase in relative brain size in B.andrewsi.
基金supported by the National Natural Science Foundation of China under Grants 62476138 and 42375016.
文摘Continuous control protocols are extensively utilized in traditional MASs,in which information needs to be transmitted among agents consecutively,therefore resulting in excessive consumption of limited resources.To decrease the control cost,based on ISC,several LFC problems are investigated for second-order MASs without and with time delay,respectively.Firstly,an intermittent sampled controller is designed,and a sufficient and necessary condition is derived,under which state errors between the leader and all the followers approach zero asymptotically.Considering that time delay is inevitable,a new protocol is proposed to deal with the time-delay situation.The error system’s stability is analyzed using the Schur stability theorem,and sufficient and necessary conditions for LFC are obtained,which are closely associated with the coupling gain,the system parameters,and the network structure.Furthermore,for the case where the current position and velocity information are not available,a distributed protocol is designed that depends only on the sampled position information.The sufficient and necessary conditions for LFC are also given.The results show that second-order MASs can achieve the LFC if and only if the system parameters satisfy the inequalities proposed in the paper.Finally,the correctness of the obtained results is verified by numerical simulations.
基金supported in part by the National Natural Science Foundation of China under Grant 6237319in part by the Postgraduate Research and Practice Innovation Program of Jiangsu Province under Grant KYCX230479.
文摘Formation control in multi-agent systems has become a critical area of interest due to its wide-ranging applications in robotics,autonomous transportation,and surveillance.While various studies have explored distributed cooperative control,this review focuses on the theoretical foundations and recent developments in formation control strategies.The paper categorizes and analyzes key formation types,including formation maintenance,group or cluster formation,bipartite formations,event-triggered formations,finite-time convergence,and constrained formations.A significant portion of the review addresses formation control under constrained dynamics,presenting both modelbased and model-free approaches that consider practical limitations such as actuator bounds,communication delays,and nonholonomic constraints.Additionally,the paper discusses emerging trends,including the integration of eventdriven mechanisms and AI-enhanced coordination strategies.Comparative evaluations highlight the trade-offs among various methodologies regarding scalability,robustness,and real-world feasibility.Practical implementations are reviewed across diverse platforms,and the review identifies the current achievements and unresolved challenges in the field.The paper concludes by outlining promising research directions,such as adaptive control for dynamic environments,energy-efficient coordination,and using learning-based control under uncertainty.This review synthesizes the current state of the art and provides a road map for future investigation,making it a valuable reference for researchers and practitioners aiming to advance formation control in multi-agent systems.
基金supported by the National Natural Science Foundation of China under Grant 62273351 and Grant 62303020.
文摘In recent years,significant research attention has been directed towards swarm intelligence.The Milling behavior of fish schools,a prime example of swarm intelligence,shows how simple rules followed by individual agents lead to complex collective behaviors.This paper studies Multi-Agent Reinforcement Learning to simulate fish schooling behavior,overcoming the challenges of tuning parameters in traditional models and addressing the limitations of single-agent methods in multi-agent environments.Based on this foundation,a novel Graph Convolutional Networks(GCN)-Critic MADDPG algorithm leveraging GCN is proposed to enhance cooperation among agents in a multi-agent system.Simulation experiments demonstrate that,compared to traditional single-agent algorithms,the proposed method not only exhibits significant advantages in terms of convergence speed and stability but also achieves tighter group formations and more naturally aligned Milling behavior.Additionally,a fish school self-organizing behavior research platform based on an event-triggered mechanism has been developed,providing a robust tool for exploring dynamic behavioral changes under various conditions.
基金supported in part by the National Natural Science Foundation of China under Grants 62001225,62071236,62071234 and U22A2002in part by the Major Science and Technology plan of Hainan Province under Grant ZDKJ2021022+1 种基金in part by the Scientific Research Fund Project of Hainan University under Grant KYQD(ZR)-21008in part by the Key Technologies R&D Program of Jiangsu(Prospective and Key Technologies for Industry)under Grants BE2023022 and BE2023022-2.
文摘The Internet of Unmanned Aerial Vehicles(I-UAVs)is expected to execute latency-sensitive tasks,but limited by co-channel interference and malicious jamming.In the face of unknown prior environmental knowledge,defending against jamming and interference through spectrum allocation becomes challenging,especially when each UAV pair makes decisions independently.In this paper,we propose a cooperative multi-agent reinforcement learning(MARL)-based anti-jamming framework for I-UAVs,enabling UAV pairs to learn their own policies cooperatively.Specifically,we first model the problem as a modelfree multi-agent Markov decision process(MAMDP)to maximize the long-term expected system throughput.Then,for improving the exploration of the optimal policy,we resort to optimizing a MARL objective function with a mutual-information(MI)regularizer between states and actions,which can dynamically assign the probability for actions frequently used by the optimal policy.Next,through sharing their current channel selections and local learning experience(their soft Q-values),the UAV pairs can learn their own policies cooperatively relying on only preceding observed information and predicting others’actions.Our simulation results show that for both sweep jamming and Markov jamming patterns,the proposed scheme outperforms the benchmarkers in terms of throughput,convergence and stability for different numbers of jammers,channels and UAV pairs.
