摘要
基于深度强化学习理论框架,提出分层递进式解决方案。首先,构建融合边缘计算节点的异构数据传输架构,建立具有时变特征的多维状态空间马尔可夫决策过程。其次,在传统深度Q网络(deep Qlearning network,DQN)算法中嵌入熵正则化约束项,结合同策略经验回放机制,形成增强型ESERDQN(improved DQN algorithm based on entropy and same-strategy experience replay)优化器。最终,设计五维评估指标体系(收敛速率、累积奖励值、能耗、传输时延、传输成本),开展多算法对比实验。仿真结果表明,ESERDQN在1500训练周期内达成稳定收敛,较基准贪心算法、随机算法、DDPG算法及PPO分别提升收敛速度49.2%、41.7%、30.1%和13.3%;在综合业务指标方面,其单位能耗成本降低27.8%,关键任务时延控制在12.3 ms以内,验证了所提方法在智慧城市复杂传输场景下的技术优越性。
Based on the theoretical framework of deep reinforcement learning,a hierarchical and progressive solution was proposed.Firstly,a heterogeneous data transmission architecture integrating edge computing nodes was constructed,and a multi-dimensional state space Markov decision process with time-varying characteristics was established.Secondly,the entropy regularization constraint term was embedded in the traditional deep Q-learning network(DQN)algorithm,and the experience replay mechanism of the same strategy was combined.An enhanced ESERDQN(improved DQN algorithm based on entropy and same-strategy experience replay)optimizer was formed.Finally,a five-dimensional evaluation index system(convergence rate,cumulative reward value,energy consumption,end-toend delay,transmission cost)was designed to carry out multi-algorithm comparison experiments.The simulation results show that ESERDQN achieves stable convergence within 1500 training cycles,which improves the convergence speed by 49.2%,41.7%,30.1%and 13.3%respectively compared with the benchmark greedy algorithm,random algorithm,DDPG algorithm and PPO.In terms of comprehensive business indicators,the unit energy cost was reduced by 27.8%,and the delay of key tasks is controlled within 12.3 ms,which verifies the technical superiority of the proposed method in complex transmission scenarios of smart cities.
作者
蒋守花
冯军
舒晖
黎佳宜
JIANG Shouhua;FENG Jun;SHU Hui;LI Jiayi(Modern Education Technology Center,Chengdu Medical College,Chengdu 610599,China;Beijing Normal University,Beijing 100875,China)
出处
《电信科学》
北大核心
2025年第8期148-162,共15页
Telecommunications Science
基金
四川省教育信息化应用与发展研究中心2024年度立项课题(No.JYXX2410)
四川省教育信息化与大数据中心项目(No.DSJZXKT256)
四川省教育数字化发展与评价重点实验室2025年度立项课题(No.JYSZH202514)。
关键词
智慧城市
数据传输
熵
同策略经验回放
深度强化学习
smart city
data transmission
entropy
same-strategy experience replay
deep reinforcement learning
