With the advent of sixth-generation mobile communications(6G),space-air-ground integrated networks have become mainstream.This paper focuses on collaborative scheduling for mobile edge computing(MEC)under a three-tier...With the advent of sixth-generation mobile communications(6G),space-air-ground integrated networks have become mainstream.This paper focuses on collaborative scheduling for mobile edge computing(MEC)under a three-tier heterogeneous architecture composed of mobile devices,unmanned aerial vehicles(UAVs),and macro base stations(BSs).This scenario typically faces fast channel fading,dynamic computational loads,and energy constraints,whereas classical queuing-theoretic or convex-optimization approaches struggle to yield robust solutions in highly dynamic settings.To address this issue,we formulate a multi-agent Markov decision process(MDP)for an air-ground-fused MEC system,unify link selection,bandwidth/power allocation,and task offloading into a continuous action space and propose a joint scheduling strategy that is based on an improved MATD3 algorithm.The improvements include Alternating Layer Normalization(ALN)in the actor to suppress gradient variance,Residual Orthogonalization(RO)in the critic to reduce the correlation between the twin Q-value estimates,and a dynamic-temperature reward to enable adaptive trade-offs during training.On a multi-user,dual-link simulation platform,we conduct ablation and baseline comparisons.The results reveal that the proposed method has better convergence and stability.Compared with MADDPG,TD3,and DSAC,our algorithm achieves more robust performance across key metrics.展开更多
无人机(unmanned aerial vehicle,UAV)搭载边缘服务器构成移动边缘服务器,可以在一些基站难以部署的场景下为用户设备(user equipment,UE)提供计算服务,借助深度强化学习对智能体进行训练,能够在连续复杂的状态空间中制定合理的卸载决策...无人机(unmanned aerial vehicle,UAV)搭载边缘服务器构成移动边缘服务器,可以在一些基站难以部署的场景下为用户设备(user equipment,UE)提供计算服务,借助深度强化学习对智能体进行训练,能够在连续复杂的状态空间中制定合理的卸载决策,将用户产生的计算密集型任务部分卸载至边缘服务器处执行,提高系统的续航和响应时间.但目前的深度强化学习算法所使用的全连接神经网络无法较好地处理UAV辅助移动边缘计算(mobile edge computing,MEC)场景下的时间序列数据,算法的智能体训练效率低,决策性能差.针对上述问题,本文以最小化UAV辅助MEC系统总时延为目标,提出了一种基于长短期记忆网络的双延迟深度确定性策略梯度算法(twin delayed deep deterministic policy gradient algorithm based on long short term memory,LSTM-TD3).利用LSTM改进TD3算法的Actor-Critic网络结构,将网络划分成3部分:包含LSTM的记忆提取单元,当前特征提取单元,以及感知整合单元;并在改进了经验池中的样本数据,定义了历史数据,使记忆提取单元能够得到更好的训练效果.仿真结果表明,与AC算法、DQN算法和DDPG算法相比,LSTM-TD3算法在以系统最小总时延为目标对卸载策略进行优化时具有最好的性能.展开更多
在任务计算密集型和延迟敏感型的场景下,无人机辅助的移动边缘计算由于其高机动性和放置成本低的特点而被广泛研究.然而,无人机的能耗限制导致其无法长时间工作并且卸载任务内的不同模块往往存在着依赖关系.针对这种情况,以有向无环图(d...在任务计算密集型和延迟敏感型的场景下,无人机辅助的移动边缘计算由于其高机动性和放置成本低的特点而被广泛研究.然而,无人机的能耗限制导致其无法长时间工作并且卸载任务内的不同模块往往存在着依赖关系.针对这种情况,以有向无环图(direct acyclic graph,DAG)为基础对任务内部模块的依赖关系进行建模,综合考虑系统时延和能耗的影响,以最小化系统成本为优化目标得到最优的卸载策略.为了解决这一优化问题,提出了一种基于亚群、高斯变异和反向学习的二进制灰狼优化算法(binary grey wolf optimization algorithm based on subpopulation,Gaussian mutation,and reverse learning,BGWOSGR).仿真结果表明,所提出算法计算出的系统成本比其他4种对比方法分别降低了约19%、27%、16%、13%,并且收敛速度更快.展开更多
文摘With the advent of sixth-generation mobile communications(6G),space-air-ground integrated networks have become mainstream.This paper focuses on collaborative scheduling for mobile edge computing(MEC)under a three-tier heterogeneous architecture composed of mobile devices,unmanned aerial vehicles(UAVs),and macro base stations(BSs).This scenario typically faces fast channel fading,dynamic computational loads,and energy constraints,whereas classical queuing-theoretic or convex-optimization approaches struggle to yield robust solutions in highly dynamic settings.To address this issue,we formulate a multi-agent Markov decision process(MDP)for an air-ground-fused MEC system,unify link selection,bandwidth/power allocation,and task offloading into a continuous action space and propose a joint scheduling strategy that is based on an improved MATD3 algorithm.The improvements include Alternating Layer Normalization(ALN)in the actor to suppress gradient variance,Residual Orthogonalization(RO)in the critic to reduce the correlation between the twin Q-value estimates,and a dynamic-temperature reward to enable adaptive trade-offs during training.On a multi-user,dual-link simulation platform,we conduct ablation and baseline comparisons.The results reveal that the proposed method has better convergence and stability.Compared with MADDPG,TD3,and DSAC,our algorithm achieves more robust performance across key metrics.
文摘无人机(unmanned aerial vehicle,UAV)搭载边缘服务器构成移动边缘服务器,可以在一些基站难以部署的场景下为用户设备(user equipment,UE)提供计算服务,借助深度强化学习对智能体进行训练,能够在连续复杂的状态空间中制定合理的卸载决策,将用户产生的计算密集型任务部分卸载至边缘服务器处执行,提高系统的续航和响应时间.但目前的深度强化学习算法所使用的全连接神经网络无法较好地处理UAV辅助移动边缘计算(mobile edge computing,MEC)场景下的时间序列数据,算法的智能体训练效率低,决策性能差.针对上述问题,本文以最小化UAV辅助MEC系统总时延为目标,提出了一种基于长短期记忆网络的双延迟深度确定性策略梯度算法(twin delayed deep deterministic policy gradient algorithm based on long short term memory,LSTM-TD3).利用LSTM改进TD3算法的Actor-Critic网络结构,将网络划分成3部分:包含LSTM的记忆提取单元,当前特征提取单元,以及感知整合单元;并在改进了经验池中的样本数据,定义了历史数据,使记忆提取单元能够得到更好的训练效果.仿真结果表明,与AC算法、DQN算法和DDPG算法相比,LSTM-TD3算法在以系统最小总时延为目标对卸载策略进行优化时具有最好的性能.
文摘在任务计算密集型和延迟敏感型的场景下,无人机辅助的移动边缘计算由于其高机动性和放置成本低的特点而被广泛研究.然而,无人机的能耗限制导致其无法长时间工作并且卸载任务内的不同模块往往存在着依赖关系.针对这种情况,以有向无环图(direct acyclic graph,DAG)为基础对任务内部模块的依赖关系进行建模,综合考虑系统时延和能耗的影响,以最小化系统成本为优化目标得到最优的卸载策略.为了解决这一优化问题,提出了一种基于亚群、高斯变异和反向学习的二进制灰狼优化算法(binary grey wolf optimization algorithm based on subpopulation,Gaussian mutation,and reverse learning,BGWOSGR).仿真结果表明,所提出算法计算出的系统成本比其他4种对比方法分别降低了约19%、27%、16%、13%,并且收敛速度更快.
