The integration of substantial renewable energy and controllable resources disrupts the supply-demand balance in distribution grids.Secure operations are dependent on the participation of user-side resources in demand...The integration of substantial renewable energy and controllable resources disrupts the supply-demand balance in distribution grids.Secure operations are dependent on the participation of user-side resources in demand response at both the day-ahead and intraday levels.Current studies typically overlook the spatial--temporal variations and coordination between these timescales,leading to significant day-ahead optimization errors,high intraday costs,and slow convergence.To address these challenges,we developed a multiagent,multitimescale aggregated regulation method for spatial--temporal coordinated demand response of user-side resources.Firstly,we established a framework considering the spatial--temporal coordinated characteristics of user-side resources with the objective to min-imize the total regulation cost and weighted sum of distribution grid losses.The optimization problem was then solved for two different timescales:day-ahead and intraday.For the day-ahead timescale,we developed an improved particle swarm optimization(IPSO)algo-rithm that dynamically adjusts the number of particles based on intraday outcomes to optimize the regulation strategies.For the intraday timescale,we developed an improved alternating direction method of multipliers(IADMM)algorithm that distributes tasks across edge distribution stations,dynamically adjusting penalty factors by using historical day-ahead data to synchronize the regulations and enhance precision.The simulation results indicate that this method can fully achieve multitimescale spatial--temporal coordinated aggregated reg-ulation between day-ahead and intraday,effectively reduce the total regulation cost and distribution grid losses,and enhance smart grid resilience.展开更多
The increasing adoption of unmanned aerial vehicles(UAVs)in urban low-altitude logistics systems,particularly for time-sensitive applications like parcel delivery and supply distribution,necessitates sophisticated coo...The increasing adoption of unmanned aerial vehicles(UAVs)in urban low-altitude logistics systems,particularly for time-sensitive applications like parcel delivery and supply distribution,necessitates sophisticated coordination mechanisms to optimize operational efficiency.However,the limited capability of UAVs to extract stateaction information in complex environments poses significant challenges to achieving effective cooperation in dynamic and uncertain scenarios.To address this,we presents an Improved Multi-Agent Hybrid Attention Critic(IMAHAC)framework that advances multi-agent deep reinforcement learning(MADRL)through two key innovations.Firstly,a Temporal Difference Error and Time-based Prioritized Experience Replay(TT-PER)mechanism that dynamically adjusts sample weights based on temporal relevance and prediction error magnitude,effectively reducing the interference from obsolete collaborative experiences while maintaining training stability.Secondly,a hybrid attention mechanism is developed,integrating a sensor fusion layer—which aggregates features from multi-sensor data to enhance decision-making—and a dissimilarity layer that evaluates the similarity between key-value pairs and query values.By combining this hybrid attention mechanism with theMulti-Actor Attention Critic(MAAC)framework,our approach strengthens UAVs’capability to extract critical state-action features in diverse environments.Comprehensive simulations in urban air mobility scenarios demonstrate IMAHAC’s superiority over conventional MADRL baselines and MAAC,achieving higher cumulative rewards,fewer collisions,and enhanced cooperative capabilities.This work provides both algorithmic advancements and empirical validation for developing robust autonomous aerial systems in smart city infrastructures.展开更多
In this paper, distributed event-triggered performance constraint control is proposed for Heterogeneous Multiagent Systems (HMASs) including quadrotor unmanned aerial vehicles and unmanned ground vehicles in the prese...In this paper, distributed event-triggered performance constraint control is proposed for Heterogeneous Multiagent Systems (HMASs) including quadrotor unmanned aerial vehicles and unmanned ground vehicles in the presence of unknown external disturbances. To tackle the problem of different dynamic characteristics and facilitate the controller design, the virtual variable is introduced in the z axis of the nonlinear model of unmanned ground vehicles. By using this approach, a universal model is established for the HMAS. Moreover, a distributed disturbance observer is established to cope with the adverse influence of the external disturbances. Then, an Appointed-Time Prescribed Performance Function (ATPPF) is designed to restrict the tracking error in the predefined regions. On this basis, the distributed performance constraint controller is proposed for the HMAS based on the ATPPF and the distributed disturbance observer. Furthermore, the improved event-triggered mechanism is proposed with a dynamic threshold, which depends on the distance between the tracking error and the boundary of the ATPPF. Finally, the effectiveness of the proposed control method is verified by the comparative experiments on an HMAS.展开更多
Dear Editor,This letter studies output consensus problem of heterogeneous linear multiagent systems over directed graphs. A novel adaptive dynamic event-triggered controller is presented based only on the feedback com...Dear Editor,This letter studies output consensus problem of heterogeneous linear multiagent systems over directed graphs. A novel adaptive dynamic event-triggered controller is presented based only on the feedback combination of the agent's own state and neighbors' output,which can achieve exponential output consensus through intermittent communication. The controller is obtained by solving two linear matrix equations, and Zeno behavior is excluded.展开更多
In this paper, the containment control problem in nonlinear multi-agent systems(NMASs) under denial-of-service(DoS) attacks is addressed. Firstly, a prediction model is obtained using the broad learning technique to t...In this paper, the containment control problem in nonlinear multi-agent systems(NMASs) under denial-of-service(DoS) attacks is addressed. Firstly, a prediction model is obtained using the broad learning technique to train historical data generated by the system offline without DoS attacks. Secondly, the dynamic linearization method is used to obtain the equivalent linearization model of NMASs. Then, a novel model-free adaptive predictive control(MFAPC) framework based on historical and online data generated by the system is proposed, which combines the trained prediction model with the model-free adaptive control method. The development of the MFAPC method motivates a much simpler robust predictive control solution that is convenient to use in the case of DoS attacks. Meanwhile, the MFAPC algorithm provides a unified predictive framework for solving consensus tracking and containment control problems. The boundedness of the containment error can be proven by using the contraction mapping principle and the mathematical induction method. Finally, the proposed MFAPC is assessed through comparative experiments.展开更多
Recently,learning-based control for multi-robot systems(MRS)with obstacle avoidance has received increasing attention.The goals of formation control and obstacle avoidance could be intrinsically tied.As a result,devel...Recently,learning-based control for multi-robot systems(MRS)with obstacle avoidance has received increasing attention.The goals of formation control and obstacle avoidance could be intrinsically tied.As a result,developing a safe and near-optimal control policy with the actor-critic structure is challenging.Therefore,a hybrid distributed and decentralised asynchronous actor-critic reinforcement learning(Di-De-RL)technique is proposed to address this problem.First,we decompose the integrated formation control and collision avoidance problem into two successive ones.To solve them,we design a distributed reinforcement learning(Di-RL)algorithm that employs a neural network-based actor-critic structure for formation control,and a decentralised RL(De-RL)algorithm that incorporates a potential-field(PF)-based actor-critic structure for collision avoidance.In Di-RL,the actor-critic pairs are trained in a distributed manner to achieve near-optimal consensus formation control.With the trained policy of Di-RL fixed,the PF actor-critic pairs in De-RL are trained in a decentralised manner for safe collision avoidance.Such an asynchronous training design of the hybrid Di-RL and De-RL enables weight convergence and control safety in the learning process.The simulated and real-world experimental results demonstrate the effectiveness and enhanced performance of the approach in formation control with both static and dynamic obstacle avoidance,highlighting its advantages in resolving the conflict between the safety objective and optimal control.展开更多
Using the differences and complementarities between human intelligence and artificial intelligence(AI),a hybrid-augmented intelligence,that is both stronger than human intelligence and AI,is created through Human-AI C...Using the differences and complementarities between human intelligence and artificial intelligence(AI),a hybrid-augmented intelligence,that is both stronger than human intelligence and AI,is created through Human-AI Cooperation(HAC)for teaching and learning.Human-AI Cooperation is infiltrating into all links of education,and recent research has focused a lot on the impact of teaching,learning,management,and evaluation with Human-AI Cooperation.However,AI still has its limits of intelligence,and cannot cooperate as humans.Thus,it is critical to study the obstacles of Human-AI Cooperation in education,as AI plays a role as a partner,not a tool.This study discussed for the first time how teachers and AI cooperate based on Multiple Intelligences of AI proposed by Andrzej Cichocki and puts forward a new Human-AI Cooperation teaching mode:human in the loop and teaching as leadership.It is proposed that humans in the loop and teaching as leadership can solve the problem that AI cannot cope with complex and dynamic teaching tasks in open situations,as well as the limits of intelligence for AI.展开更多
The static nature of cyber defense systems gives attackers a sufficient amount of time to explore and further exploit the vulnerabilities of information technology systems.In this paper,we investigate a problem where ...The static nature of cyber defense systems gives attackers a sufficient amount of time to explore and further exploit the vulnerabilities of information technology systems.In this paper,we investigate a problem where multiagent sys-tems sensing and acting in an environment contribute to adaptive cyber defense.We present a learning strategy that enables multiple agents to learn optimal poli-cies using multiagent reinforcement learning(MARL).Our proposed approach is inspired by the multiarmed bandits(MAB)learning technique for multiple agents to cooperate in decision making or to work independently.We study a MAB approach in which defenders visit a system multiple times in an alternating fash-ion to maximize their rewards and protect their system.We find that this game can be modeled from an individual player’s perspective as a restless MAB problem.We discover further results when the MAB takes the form of a pure birth process,such as a myopic optimal policy,as well as providing environments that offer the necessary incentives required for cooperation in multiplayer projects.展开更多
Multi-agent systems(MASs)are typically composed of multiple smart entities with independent sensing,communication,computing,and decision-making capabilities.Nowadays,MASs have a wide range of applications in smart gri...Multi-agent systems(MASs)are typically composed of multiple smart entities with independent sensing,communication,computing,and decision-making capabilities.Nowadays,MASs have a wide range of applications in smart grids,smart manufacturing,sensor networks,and intelligent transportation systems.Control of the MASs are often coordinated through information interaction among agents,which is one of the most important factors affecting coordination and cooperation performance.However,unexpected physical faults and cyber attacks on a single agent may spread to other agents via information interaction very quickly,and thus could lead to severe degradation of the whole system performance and even destruction of MASs.This paper is concerned with the safety/security analysis and synthesis of MASs arising from physical faults and cyber attacks,and our goal is to present a comprehensive survey on recent results on fault estimation,detection,diagnosis and fault-tolerant control of MASs,and cyber attack detection and secure control of MASs subject to two typical cyber attacks.Finally,the paper concludes with some potential future research topics on the security issues of MASs.展开更多
基金supported by Science and Technology Program of China Southern Power Grid Corporation under grant number 036000KK52222004(GDKJXM20222117)National Key R&D Program of China for International S&T Cooperation Projects(2019YFE0118700).
文摘The integration of substantial renewable energy and controllable resources disrupts the supply-demand balance in distribution grids.Secure operations are dependent on the participation of user-side resources in demand response at both the day-ahead and intraday levels.Current studies typically overlook the spatial--temporal variations and coordination between these timescales,leading to significant day-ahead optimization errors,high intraday costs,and slow convergence.To address these challenges,we developed a multiagent,multitimescale aggregated regulation method for spatial--temporal coordinated demand response of user-side resources.Firstly,we established a framework considering the spatial--temporal coordinated characteristics of user-side resources with the objective to min-imize the total regulation cost and weighted sum of distribution grid losses.The optimization problem was then solved for two different timescales:day-ahead and intraday.For the day-ahead timescale,we developed an improved particle swarm optimization(IPSO)algo-rithm that dynamically adjusts the number of particles based on intraday outcomes to optimize the regulation strategies.For the intraday timescale,we developed an improved alternating direction method of multipliers(IADMM)algorithm that distributes tasks across edge distribution stations,dynamically adjusting penalty factors by using historical day-ahead data to synchronize the regulations and enhance precision.The simulation results indicate that this method can fully achieve multitimescale spatial--temporal coordinated aggregated reg-ulation between day-ahead and intraday,effectively reduce the total regulation cost and distribution grid losses,and enhance smart grid resilience.
基金supported by theHubei Provincial Technology Innovation Special Project and the Natural Science Foundation of Hubei Province under Grants 2023BEB024,2024AFC066,respectively.
文摘The increasing adoption of unmanned aerial vehicles(UAVs)in urban low-altitude logistics systems,particularly for time-sensitive applications like parcel delivery and supply distribution,necessitates sophisticated coordination mechanisms to optimize operational efficiency.However,the limited capability of UAVs to extract stateaction information in complex environments poses significant challenges to achieving effective cooperation in dynamic and uncertain scenarios.To address this,we presents an Improved Multi-Agent Hybrid Attention Critic(IMAHAC)framework that advances multi-agent deep reinforcement learning(MADRL)through two key innovations.Firstly,a Temporal Difference Error and Time-based Prioritized Experience Replay(TT-PER)mechanism that dynamically adjusts sample weights based on temporal relevance and prediction error magnitude,effectively reducing the interference from obsolete collaborative experiences while maintaining training stability.Secondly,a hybrid attention mechanism is developed,integrating a sensor fusion layer—which aggregates features from multi-sensor data to enhance decision-making—and a dissimilarity layer that evaluates the similarity between key-value pairs and query values.By combining this hybrid attention mechanism with theMulti-Actor Attention Critic(MAAC)framework,our approach strengthens UAVs’capability to extract critical state-action features in diverse environments.Comprehensive simulations in urban air mobility scenarios demonstrate IMAHAC’s superiority over conventional MADRL baselines and MAAC,achieving higher cumulative rewards,fewer collisions,and enhanced cooperative capabilities.This work provides both algorithmic advancements and empirical validation for developing robust autonomous aerial systems in smart city infrastructures.
基金supported in part by the National Natural Science Foundation of China(Nos.U23B2036,U2013201).
文摘In this paper, distributed event-triggered performance constraint control is proposed for Heterogeneous Multiagent Systems (HMASs) including quadrotor unmanned aerial vehicles and unmanned ground vehicles in the presence of unknown external disturbances. To tackle the problem of different dynamic characteristics and facilitate the controller design, the virtual variable is introduced in the z axis of the nonlinear model of unmanned ground vehicles. By using this approach, a universal model is established for the HMAS. Moreover, a distributed disturbance observer is established to cope with the adverse influence of the external disturbances. Then, an Appointed-Time Prescribed Performance Function (ATPPF) is designed to restrict the tracking error in the predefined regions. On this basis, the distributed performance constraint controller is proposed for the HMAS based on the ATPPF and the distributed disturbance observer. Furthermore, the improved event-triggered mechanism is proposed with a dynamic threshold, which depends on the distance between the tracking error and the boundary of the ATPPF. Finally, the effectiveness of the proposed control method is verified by the comparative experiments on an HMAS.
基金supported by the National Science and Technology Innovation 2030-Major Program(2022ZD 0115403)the National Natural Science Foundation of China(61991414)+1 种基金Chongqing Natural Science Foundation(CSTB2023NSCQJQX0018)Beijing Natural Science Foundation(L221005)
文摘Dear Editor,This letter studies output consensus problem of heterogeneous linear multiagent systems over directed graphs. A novel adaptive dynamic event-triggered controller is presented based only on the feedback combination of the agent's own state and neighbors' output,which can achieve exponential output consensus through intermittent communication. The controller is obtained by solving two linear matrix equations, and Zeno behavior is excluded.
基金supported in part by the National Natural Science Foundation of China(62403396,62433018,62373113)the Guangdong Basic and Applied Basic Research Foundation(2023A1515011527,2023B1515120010)the Postdoctoral Fellowship Program of CPSF(GZB20240621)
文摘In this paper, the containment control problem in nonlinear multi-agent systems(NMASs) under denial-of-service(DoS) attacks is addressed. Firstly, a prediction model is obtained using the broad learning technique to train historical data generated by the system offline without DoS attacks. Secondly, the dynamic linearization method is used to obtain the equivalent linearization model of NMASs. Then, a novel model-free adaptive predictive control(MFAPC) framework based on historical and online data generated by the system is proposed, which combines the trained prediction model with the model-free adaptive control method. The development of the MFAPC method motivates a much simpler robust predictive control solution that is convenient to use in the case of DoS attacks. Meanwhile, the MFAPC algorithm provides a unified predictive framework for solving consensus tracking and containment control problems. The boundedness of the containment error can be proven by using the contraction mapping principle and the mathematical induction method. Finally, the proposed MFAPC is assessed through comparative experiments.
基金supported by the National Natural Science Foundation of China(Grants Nos.61825305 and U21A20518)China Postdoctoral Science Foundation(Grants No.47680).
文摘Recently,learning-based control for multi-robot systems(MRS)with obstacle avoidance has received increasing attention.The goals of formation control and obstacle avoidance could be intrinsically tied.As a result,developing a safe and near-optimal control policy with the actor-critic structure is challenging.Therefore,a hybrid distributed and decentralised asynchronous actor-critic reinforcement learning(Di-De-RL)technique is proposed to address this problem.First,we decompose the integrated formation control and collision avoidance problem into two successive ones.To solve them,we design a distributed reinforcement learning(Di-RL)algorithm that employs a neural network-based actor-critic structure for formation control,and a decentralised RL(De-RL)algorithm that incorporates a potential-field(PF)-based actor-critic structure for collision avoidance.In Di-RL,the actor-critic pairs are trained in a distributed manner to achieve near-optimal consensus formation control.With the trained policy of Di-RL fixed,the PF actor-critic pairs in De-RL are trained in a decentralised manner for safe collision avoidance.Such an asynchronous training design of the hybrid Di-RL and De-RL enables weight convergence and control safety in the learning process.The simulated and real-world experimental results demonstrate the effectiveness and enhanced performance of the approach in formation control with both static and dynamic obstacle avoidance,highlighting its advantages in resolving the conflict between the safety objective and optimal control.
基金This research was supported by"Zhejiang Soft Science Research Program,Grant no:2021C35016".
文摘Using the differences and complementarities between human intelligence and artificial intelligence(AI),a hybrid-augmented intelligence,that is both stronger than human intelligence and AI,is created through Human-AI Cooperation(HAC)for teaching and learning.Human-AI Cooperation is infiltrating into all links of education,and recent research has focused a lot on the impact of teaching,learning,management,and evaluation with Human-AI Cooperation.However,AI still has its limits of intelligence,and cannot cooperate as humans.Thus,it is critical to study the obstacles of Human-AI Cooperation in education,as AI plays a role as a partner,not a tool.This study discussed for the first time how teachers and AI cooperate based on Multiple Intelligences of AI proposed by Andrzej Cichocki and puts forward a new Human-AI Cooperation teaching mode:human in the loop and teaching as leadership.It is proposed that humans in the loop and teaching as leadership can solve the problem that AI cannot cope with complex and dynamic teaching tasks in open situations,as well as the limits of intelligence for AI.
基金This work is funded by the Deanship of Scientific Research(DSR)the University of Jeddah,under Grant No.(UJ-22-DR-1).
文摘The static nature of cyber defense systems gives attackers a sufficient amount of time to explore and further exploit the vulnerabilities of information technology systems.In this paper,we investigate a problem where multiagent sys-tems sensing and acting in an environment contribute to adaptive cyber defense.We present a learning strategy that enables multiple agents to learn optimal poli-cies using multiagent reinforcement learning(MARL).Our proposed approach is inspired by the multiarmed bandits(MAB)learning technique for multiple agents to cooperate in decision making or to work independently.We study a MAB approach in which defenders visit a system multiple times in an alternating fash-ion to maximize their rewards and protect their system.We find that this game can be modeled from an individual player’s perspective as a restless MAB problem.We discover further results when the MAB takes the form of a pure birth process,such as a myopic optimal policy,as well as providing environments that offer the necessary incentives required for cooperation in multiplayer projects.
基金partially supported by the National Natural Science Foundation of China(61873237)the Fundamental Research Funds for the Central Universities+2 种基金the Fundamental Research Funds for the Provincial Universities of Zhejiang(RF-A2019003)the Research Grants Council of the Hong Kong Special Administrative Region of China(City U/11204315)the Hong Kong Scholars Program(XJ2016030)。
文摘Multi-agent systems(MASs)are typically composed of multiple smart entities with independent sensing,communication,computing,and decision-making capabilities.Nowadays,MASs have a wide range of applications in smart grids,smart manufacturing,sensor networks,and intelligent transportation systems.Control of the MASs are often coordinated through information interaction among agents,which is one of the most important factors affecting coordination and cooperation performance.However,unexpected physical faults and cyber attacks on a single agent may spread to other agents via information interaction very quickly,and thus could lead to severe degradation of the whole system performance and even destruction of MASs.This paper is concerned with the safety/security analysis and synthesis of MASs arising from physical faults and cyber attacks,and our goal is to present a comprehensive survey on recent results on fault estimation,detection,diagnosis and fault-tolerant control of MASs,and cyber attack detection and secure control of MASs subject to two typical cyber attacks.Finally,the paper concludes with some potential future research topics on the security issues of MASs.
