摘要
针对传统网络架构下覆盖组播缺乏对底层网络状态的感知、难以适应网络动态变化,以及现有强化学习方法因覆盖组播树路径耦合、面临问题规模大、动作空间维度高导致学习不稳定、收敛缓慢等问题,提出一种基于目标导向分层强化学习的智能覆盖组播路由方法GOHRL-OM。首先,利用SDN的全局感知能力,构建动态流量矩阵为路由决策提供全局信息支撑。其次,GOHRL-OM结合目标导向强化学习与分层强化学习优化覆盖组播树:目标导向机制引入任务目标,增强策略学习方向性;分层学习将任务分解为上下层子任务,通过协同策略和分层奖励实现任务解耦与分层优化,从而降低动作维度并提升学习稳定性。仿真实验表明,相较于现有方法,GOHRL-OM在优化吞吐量、时延与丢包率的同时,具备更加灵活的路由决策和网络适应能力。
Traditional overlay multicast lacks effective awareness of underlying network states and adaptability to dynamic network conditions.Meanwhile,existing reinforcement learning–based routing methods often suffer from unstable learning and slow convergence due to coupled multicast tree paths and high-dimensional action spaces.To address these issues,this paper proposes an intelligent overlay multicast routing method based on goal-oriented hierarchical reinforcement learning,termed GOHRL-OM.By exploiting the global network visibility provided by software-defined networking(SDN),a dynamic traffic matrix is constructed to support routing decisions with global information.GOHRL-OM integrates goaloriented learning with hierarchical task decomposition to decouple routing optimization,reduce action dimensionality,and enhance learning stability.Simulation results show that,compared with existing methods,GOHRL-OM achieves improved throughput,lower delay,and reduced packet loss,while adapting effectively to dynamic network environments.
作者
叶苗
李繁有
文鹏
蒋秋香
王勇
何倩
叶聪
Ye Miao;Li Fanyou;Wen Peng;Jiang Qiuxiang;Wang Yong;He Qian;Ye Cong(School of Information and Communication,Guilin University of Electronic Technology,Guilin 541004,China;Guangxi University Engineering Research Center of Cloud Network Convergence and Data Security,Guilin University of Electronic Technology,Guilin 541004,China)
出处
《通信学报》
北大核心
2026年第1期106-123,共18页
Journal on Communications
基金
国家自然科学基金资助项目(No.62161006,No.62372353)
广西研究生教育创新计划基金资助项目(No.YCSW2025351)
桂林电子科技大学云网融合与数据安全广西高校工程研究中心(No.C25KYS00RX01)
认知无线电与信息处理教育部重点实验室主任基金资助项目(No.CRKL220103)。
关键词
软件定义网络
目标导向
分层强化学习
覆盖组播
software defined network
goal-oriented
hierarchical reinforcement learning
overlay multicast
