Reinforcement Learning(RL)techniques are being studied to solve the Demand and Capacity Balancing(DCB)problems to fully exploit their computational performance.A locally gen-eralised Multi-Agent Reinforcement Learning...Reinforcement Learning(RL)techniques are being studied to solve the Demand and Capacity Balancing(DCB)problems to fully exploit their computational performance.A locally gen-eralised Multi-Agent Reinforcement Learning(MARL)for real-world DCB problems is proposed.The proposed method can deploy trained agents directly to unseen scenarios in a specific Air Traffic Flow Management(ATFM)region to quickly obtain a satisfactory solution.In this method,agents of all flights in a scenario form a multi-agent decision-making system based on partial observation.The trained agent with the customised neural network can be deployed directly on the corresponding flight,allowing it to solve the DCB problem jointly.A cooperation coefficient is introduced in the reward function,which is used to adjust the agent’s cooperation preference in a multi-agent system,thereby controlling the distribution of flight delay time allocation.A multi-iteration mechanism is designed for the DCB decision-making framework to deal with problems arising from non-stationarity in MARL and to ensure that all hotspots are eliminated.Experiments based on large-scale high-complexity real-world scenarios are conducted to verify the effectiveness and efficiency of the method.From a statis-tical point of view,it is proven that the proposed method is generalised within the scope of the flights and sectors of interest,and its optimisation performance outperforms the standard computer-assisted slot allocation and state-of-the-art RL-based DCB methods.The sensitivity analysis preliminarily reveals the effect of the cooperation coefficient on delay time allocation.展开更多
This paper introduces an innovative approach to the synchronized demand-capacity balance with special focus on sector capacity uncertainty within a centrally controlled collaborative air traffic flow management(ATFM)f...This paper introduces an innovative approach to the synchronized demand-capacity balance with special focus on sector capacity uncertainty within a centrally controlled collaborative air traffic flow management(ATFM)framework.Further with previous study,the uncertainty in capacity is considered as a non-negligible issue regarding multiple reasons,like the impact of weather,the strike of air traffic controllers(ATCOs),the military use of airspace and the spatiotemporal distribution of nonscheduled flights,etc.These recessive factors affect the outcome of traffic flow optimization.In this research,the focus is placed on the impact of sector capacity uncertainty on demand and capacity balancing(DCB)optimization and ATFM,and multiple options,such as delay assignment and rerouting,are intended for regulating the traffic flow.A scenario optimization method for sector capacity in the presence of uncertainties is used to find the approximately optimal solution.The results show that the proposed approach can achieve better demand and capacity balancing and determine perfect integer solutions to ATFM problems,solving large-scale instances(24 h on seven capacity scenarios,with 6255 flights and 8949 trajectories)in 5-15 min.To the best of our knowledge,our experiment is the first to tackle large-scale instances of stochastic ATFM problems within the collaborative ATFM framework.展开更多
基金co-funded by the National Natural Science Foundation of China(No.61903187)the National Key R&D Program of China(No.2021YFB1600500)+2 种基金the China Scholarship Council(No.202006830095)the Natural Science Foundation of Jiangsu Province(No.BK20190414)the Jiangsu Province Postgraduate Innovation Fund(No.KYCX20_0213).
文摘Reinforcement Learning(RL)techniques are being studied to solve the Demand and Capacity Balancing(DCB)problems to fully exploit their computational performance.A locally gen-eralised Multi-Agent Reinforcement Learning(MARL)for real-world DCB problems is proposed.The proposed method can deploy trained agents directly to unseen scenarios in a specific Air Traffic Flow Management(ATFM)region to quickly obtain a satisfactory solution.In this method,agents of all flights in a scenario form a multi-agent decision-making system based on partial observation.The trained agent with the customised neural network can be deployed directly on the corresponding flight,allowing it to solve the DCB problem jointly.A cooperation coefficient is introduced in the reward function,which is used to adjust the agent’s cooperation preference in a multi-agent system,thereby controlling the distribution of flight delay time allocation.A multi-iteration mechanism is designed for the DCB decision-making framework to deal with problems arising from non-stationarity in MARL and to ensure that all hotspots are eliminated.Experiments based on large-scale high-complexity real-world scenarios are conducted to verify the effectiveness and efficiency of the method.From a statis-tical point of view,it is proven that the proposed method is generalised within the scope of the flights and sectors of interest,and its optimisation performance outperforms the standard computer-assisted slot allocation and state-of-the-art RL-based DCB methods.The sensitivity analysis preliminarily reveals the effect of the cooperation coefficient on delay time allocation.
文摘This paper introduces an innovative approach to the synchronized demand-capacity balance with special focus on sector capacity uncertainty within a centrally controlled collaborative air traffic flow management(ATFM)framework.Further with previous study,the uncertainty in capacity is considered as a non-negligible issue regarding multiple reasons,like the impact of weather,the strike of air traffic controllers(ATCOs),the military use of airspace and the spatiotemporal distribution of nonscheduled flights,etc.These recessive factors affect the outcome of traffic flow optimization.In this research,the focus is placed on the impact of sector capacity uncertainty on demand and capacity balancing(DCB)optimization and ATFM,and multiple options,such as delay assignment and rerouting,are intended for regulating the traffic flow.A scenario optimization method for sector capacity in the presence of uncertainties is used to find the approximately optimal solution.The results show that the proposed approach can achieve better demand and capacity balancing and determine perfect integer solutions to ATFM problems,solving large-scale instances(24 h on seven capacity scenarios,with 6255 flights and 8949 trajectories)in 5-15 min.To the best of our knowledge,our experiment is the first to tackle large-scale instances of stochastic ATFM problems within the collaborative ATFM framework.
