通过优化地铁时刻表可有效降低地铁牵引能耗。为解决客流波动和车辆延误对实际节能率影响的问题,提出列车牵引和供电系统实时潮流计算分析模型和基于Dueling Deep Q Network(Dueling DQN)深度强化学习算法相结合的运行图节能优化方法,...通过优化地铁时刻表可有效降低地铁牵引能耗。为解决客流波动和车辆延误对实际节能率影响的问题,提出列车牵引和供电系统实时潮流计算分析模型和基于Dueling Deep Q Network(Dueling DQN)深度强化学习算法相结合的运行图节能优化方法,建立基于区间动态客流概率统计的时刻表迭代优化模型,降低动态客流变化对节能率的影响。对预测Q网络和目标Q网络分别选取自适应时刻估计和均方根反向传播方法,提高模型收敛快速性,同时以时刻表优化前、后总运行时间不变、乘客换乘时间和等待时间最小为优化目标,实现节能时刻表无感切换。以苏州轨道交通4号线为例验证方法的有效性,节能对比试验结果表明:在到达换乘站时刻偏差不超过2 s和列车全周转运行时间不变的前提下,列车牵引节能率达5.27%,车公里能耗下降4.99%。展开更多
With the rapid development ofmobile Internet,spatial crowdsourcing has becomemore andmore popular.Spatial crowdsourcing consists of many different types of applications,such as spatial crowd-sensing services.In terms ...With the rapid development ofmobile Internet,spatial crowdsourcing has becomemore andmore popular.Spatial crowdsourcing consists of many different types of applications,such as spatial crowd-sensing services.In terms of spatial crowd-sensing,it collects and analyzes traffic sensing data from clients like vehicles and traffic lights to construct intelligent traffic prediction models.Besides collecting sensing data,spatial crowdsourcing also includes spatial delivery services like DiDi and Uber.Appropriate task assignment and worker selection dominate the service quality for spatial crowdsourcing applications.Previous research conducted task assignments via traditional matching approaches or using simple network models.However,advanced mining methods are lacking to explore the relationship between workers,task publishers,and the spatio-temporal attributes in tasks.Therefore,in this paper,we propose a Deep Double Dueling Spatial-temporal Q Network(D3SQN)to adaptively learn the spatialtemporal relationship between task,task publishers,and workers in a dynamic environment to achieve optimal allocation.Specifically,D3SQNis revised through reinforcement learning by adding a spatial-temporal transformer that can estimate the expected state values and action advantages so as to improve the accuracy of task assignments.Extensive experiments are conducted over real data collected fromDiDi and ELM,and the simulation results verify the effectiveness of our proposed models.展开更多
为了解决大型工程项目中文件的传输时间与成本问题,提出一个基于文件工作流的工程项目文件管理优化方法。首先,构建了工程项目文件管理环境和具有逻辑顺序的文件工作流模型,分析了文件的传输和缓存。在此基础上,将文件管理优化问题建模...为了解决大型工程项目中文件的传输时间与成本问题,提出一个基于文件工作流的工程项目文件管理优化方法。首先,构建了工程项目文件管理环境和具有逻辑顺序的文件工作流模型,分析了文件的传输和缓存。在此基础上,将文件管理优化问题建模为马尔可夫过程,通过设计状态空间、动作空间及奖励函数等实现文件工作流的任务完成时间与缓存成本的联合优化。其次,采用对抗式双重深度Q网络(dueling double deep Q network,D3QN)来降低训练时间,提高训练效率。仿真结果验证了提出方案在不同参数配置下文件传输的有效性,并且在任务体量增大时仍能保持较好的优化能力。展开更多
针对电极箔化成车间节能调度问题,研究提出了一种基于深度强化学习的方法进行求解。首先,将调度问题抽象为马尔可夫决策过程;其次,构建了3个静态矩阵和4个动态矩阵,用以描绘车间的环境状态,并根据电极箔车间实际生产情况,设计了由节能...针对电极箔化成车间节能调度问题,研究提出了一种基于深度强化学习的方法进行求解。首先,将调度问题抽象为马尔可夫决策过程;其次,构建了3个静态矩阵和4个动态矩阵,用以描绘车间的环境状态,并根据电极箔车间实际生产情况,设计了由节能策略的组合调度规则作为动作空间;同时,设计了用于描述最大完工时间和能耗的奖励函数。最终,利用Dueling Deep Q-Network算法进行模型训练,并与传统调度规则进行对比实验,结果表明所提方法能够有效缩短完工时间和降低化成车间总能耗。为了满足电极箔化成车间生产环境对系统的使用需求,开发电极箔化成车间调度系统进行应用。展开更多
Low Earth Orbit(LEO)mega-constellation networks,exemplified by Starlink,are poised to play a pivotal role in future mobile communication networks,due to their low latency and high capacity.With the massively deployed ...Low Earth Orbit(LEO)mega-constellation networks,exemplified by Starlink,are poised to play a pivotal role in future mobile communication networks,due to their low latency and high capacity.With the massively deployed satellites,ground users now can be covered by multiple visible satellites,but also face complex handover issues with such massive high-mobility satellites in multi-layer.The end-to-end routing is also affected by the handover behavior.In this paper,we propose an intelligent handover strategy dedicated to multi-layer LEO mega-constellation networks.Firstly,an analytic model is utilized to rapidly estimate the end-to-end propagation latency as a key handover factor to construct a multi-objective optimization model.Subsequently,an intelligent handover strategy is proposed by employing the Dueling Double Deep Q Network(D3QN)-based deep reinforcement learning algorithm for single-layer constellations.Moreover,an optimal crosslayer handover scheme is proposed by predicting the latency-jitter and minimizing the cross-layer overhead.Simulation results demonstrate the superior performance of the proposed method in the multi-layer LEO mega-constellation,showcasing reductions of up to 8.2%and 59.5%in end-to-end latency and jitter respectively,when compared to the existing handover strategies.展开更多
At present,energy consumption is one of the main bottlenecks in autonomous mobile robot development.To address the challenge of high energy consumption in path planning for autonomous mobile robots navigating unknown ...At present,energy consumption is one of the main bottlenecks in autonomous mobile robot development.To address the challenge of high energy consumption in path planning for autonomous mobile robots navigating unknown and complex environments,this paper proposes an Attention-Enhanced Dueling Deep Q-Network(ADDueling DQN),which integrates a multi-head attention mechanism and a prioritized experience replay strategy into a Dueling-DQN reinforcement learning framework.A multi-objective reward function,centered on energy efficiency,is designed to comprehensively consider path length,terrain slope,motion smoothness,and obstacle avoidance,enabling optimal low-energy trajectory generation in 3D space from the source.The incorporation of a multihead attention mechanism allows the model to dynamically focus on energy-critical state features—such as slope gradients and obstacle density—thereby significantly improving its ability to recognize and avoid energy-intensive paths.Additionally,the prioritized experience replay mechanism accelerates learning from key decision-making experiences,suppressing inefficient exploration and guiding the policy toward low-energy solutions more rapidly.The effectiveness of the proposed path planning algorithm is validated through simulation experiments conducted in multiple off-road scenarios.Results demonstrate that AD-Dueling DQN consistently achieves the lowest average energy consumption across all tested environments.Moreover,the proposed method exhibits faster convergence and greater training stability compared to baseline algorithms,highlighting its global optimization capability under energy-aware objectives in complex terrains.This study offers an efficient and scalable intelligent control strategy for the development of energy-conscious autonomous navigation systems.展开更多
Due to the high mileage and heavy load capabilities of hybrid electric vehicles(HEVs),energy management becomes crucial in improving energy efficiency.To avoid the over-dependence on the hard-crafted models,deep reinf...Due to the high mileage and heavy load capabilities of hybrid electric vehicles(HEVs),energy management becomes crucial in improving energy efficiency.To avoid the over-dependence on the hard-crafted models,deep reinforcement learning(DRL)is utilized to learn more precise energy management strategies(EMSs),but cannot generalize well to different driving situations in most cases.When driving cycles are changed,the neural network needs to be retrained,which is a time-consuming and laborious task.A more efficient transferable way is to combine DRL algorithms with transfer learning,which can utilize the knowledge of the driving cycles in other new driving situations,leading to better initial performance and a faster training process to convergence.In this paper,we propose a novel transferable EMS by incorporating the DRL method and dueling network architecture for HEVs.Simulation results indicate that the proposed method can generalize well to new driving cycles,with comparably initial performance and faster convergence in the training process.展开更多
基金supported in part by the Pioneer and Leading Goose R&D Program of Zhejiang Province under Grant 2022C01083 (Dr.Yu Li,https://zjnsf.kjt.zj.gov.cn/)Pioneer and Leading Goose R&D Program of Zhejiang Province under Grant 2023C01217 (Dr.Yu Li,https://zjnsf.kjt.zj.gov.cn/).
文摘With the rapid development ofmobile Internet,spatial crowdsourcing has becomemore andmore popular.Spatial crowdsourcing consists of many different types of applications,such as spatial crowd-sensing services.In terms of spatial crowd-sensing,it collects and analyzes traffic sensing data from clients like vehicles and traffic lights to construct intelligent traffic prediction models.Besides collecting sensing data,spatial crowdsourcing also includes spatial delivery services like DiDi and Uber.Appropriate task assignment and worker selection dominate the service quality for spatial crowdsourcing applications.Previous research conducted task assignments via traditional matching approaches or using simple network models.However,advanced mining methods are lacking to explore the relationship between workers,task publishers,and the spatio-temporal attributes in tasks.Therefore,in this paper,we propose a Deep Double Dueling Spatial-temporal Q Network(D3SQN)to adaptively learn the spatialtemporal relationship between task,task publishers,and workers in a dynamic environment to achieve optimal allocation.Specifically,D3SQNis revised through reinforcement learning by adding a spatial-temporal transformer that can estimate the expected state values and action advantages so as to improve the accuracy of task assignments.Extensive experiments are conducted over real data collected fromDiDi and ELM,and the simulation results verify the effectiveness of our proposed models.
文摘为了解决大型工程项目中文件的传输时间与成本问题,提出一个基于文件工作流的工程项目文件管理优化方法。首先,构建了工程项目文件管理环境和具有逻辑顺序的文件工作流模型,分析了文件的传输和缓存。在此基础上,将文件管理优化问题建模为马尔可夫过程,通过设计状态空间、动作空间及奖励函数等实现文件工作流的任务完成时间与缓存成本的联合优化。其次,采用对抗式双重深度Q网络(dueling double deep Q network,D3QN)来降低训练时间,提高训练效率。仿真结果验证了提出方案在不同参数配置下文件传输的有效性,并且在任务体量增大时仍能保持较好的优化能力。
文摘针对电极箔化成车间节能调度问题,研究提出了一种基于深度强化学习的方法进行求解。首先,将调度问题抽象为马尔可夫决策过程;其次,构建了3个静态矩阵和4个动态矩阵,用以描绘车间的环境状态,并根据电极箔车间实际生产情况,设计了由节能策略的组合调度规则作为动作空间;同时,设计了用于描述最大完工时间和能耗的奖励函数。最终,利用Dueling Deep Q-Network算法进行模型训练,并与传统调度规则进行对比实验,结果表明所提方法能够有效缩短完工时间和降低化成车间总能耗。为了满足电极箔化成车间生产环境对系统的使用需求,开发电极箔化成车间调度系统进行应用。
基金supported by the National Natural Science Foundation of China(No.62401597)Natural Science Foundation of Hunan Province,China(No.2024JJ6469)the Research Project of National University of Defense Technology,China(No.ZK22-02).
文摘Low Earth Orbit(LEO)mega-constellation networks,exemplified by Starlink,are poised to play a pivotal role in future mobile communication networks,due to their low latency and high capacity.With the massively deployed satellites,ground users now can be covered by multiple visible satellites,but also face complex handover issues with such massive high-mobility satellites in multi-layer.The end-to-end routing is also affected by the handover behavior.In this paper,we propose an intelligent handover strategy dedicated to multi-layer LEO mega-constellation networks.Firstly,an analytic model is utilized to rapidly estimate the end-to-end propagation latency as a key handover factor to construct a multi-objective optimization model.Subsequently,an intelligent handover strategy is proposed by employing the Dueling Double Deep Q Network(D3QN)-based deep reinforcement learning algorithm for single-layer constellations.Moreover,an optimal crosslayer handover scheme is proposed by predicting the latency-jitter and minimizing the cross-layer overhead.Simulation results demonstrate the superior performance of the proposed method in the multi-layer LEO mega-constellation,showcasing reductions of up to 8.2%and 59.5%in end-to-end latency and jitter respectively,when compared to the existing handover strategies.
文摘At present,energy consumption is one of the main bottlenecks in autonomous mobile robot development.To address the challenge of high energy consumption in path planning for autonomous mobile robots navigating unknown and complex environments,this paper proposes an Attention-Enhanced Dueling Deep Q-Network(ADDueling DQN),which integrates a multi-head attention mechanism and a prioritized experience replay strategy into a Dueling-DQN reinforcement learning framework.A multi-objective reward function,centered on energy efficiency,is designed to comprehensively consider path length,terrain slope,motion smoothness,and obstacle avoidance,enabling optimal low-energy trajectory generation in 3D space from the source.The incorporation of a multihead attention mechanism allows the model to dynamically focus on energy-critical state features—such as slope gradients and obstacle density—thereby significantly improving its ability to recognize and avoid energy-intensive paths.Additionally,the prioritized experience replay mechanism accelerates learning from key decision-making experiences,suppressing inefficient exploration and guiding the policy toward low-energy solutions more rapidly.The effectiveness of the proposed path planning algorithm is validated through simulation experiments conducted in multiple off-road scenarios.Results demonstrate that AD-Dueling DQN consistently achieves the lowest average energy consumption across all tested environments.Moreover,the proposed method exhibits faster convergence and greater training stability compared to baseline algorithms,highlighting its global optimization capability under energy-aware objectives in complex terrains.This study offers an efficient and scalable intelligent control strategy for the development of energy-conscious autonomous navigation systems.
文摘Due to the high mileage and heavy load capabilities of hybrid electric vehicles(HEVs),energy management becomes crucial in improving energy efficiency.To avoid the over-dependence on the hard-crafted models,deep reinforcement learning(DRL)is utilized to learn more precise energy management strategies(EMSs),but cannot generalize well to different driving situations in most cases.When driving cycles are changed,the neural network needs to be retrained,which is a time-consuming and laborious task.A more efficient transferable way is to combine DRL algorithms with transfer learning,which can utilize the knowledge of the driving cycles in other new driving situations,leading to better initial performance and a faster training process to convergence.In this paper,we propose a novel transferable EMS by incorporating the DRL method and dueling network architecture for HEVs.Simulation results indicate that the proposed method can generalize well to new driving cycles,with comparably initial performance and faster convergence in the training process.
