This paper proposes a Deep Reinforcement Learning(DRL)algorithm for user scheduling in Millimeter Wave(mmWave)networks,which utilizes Channel Knowledge Map(CKM)for knowledge transfer to enhance the learning of schedul...This paper proposes a Deep Reinforcement Learning(DRL)algorithm for user scheduling in Millimeter Wave(mmWave)networks,which utilizes Channel Knowledge Map(CKM)for knowledge transfer to enhance the learning of scheduling strategies.The user scheduling and link configuration problems are modeled as a multiqueue system.Each queue represents the data demand of an individual user.This setup allows the base station to make dynamic scheduling decisions based on changing environmental conditions.This approach facilitates efficient management of user-specific requirements while addressing the challenges posed by dynamic network environments.Our model incorporates relay selection,codebook selection,and beam tracking to support flexible and efficient resource allocation.In contrast to traditional channel model-based optimization,we design algorithms for scheduling policy pre-training using CKMs,which provide information about the channel between specific pairs of locations.Specifically,we assume that the CKM is fully available to allow the complex scheduling network to have a better starting point or follow a more favorable gradient direction through knowledge migration.This integration of CKM with knowledge transfer significantly accelerates DRL convergence and enhances performance stability.Simulation results confirmed the effectiveness of the proposed approach.Relative to the baseline methods,integrating CKM with knowledge transfer accelerated the convergence of the DRL algorithm by approximately 20%,maintained the delay within 30 milliseconds,and reduced the average queue length by nearly 30%.展开更多
基金supported in part by the Shenzhen Basic Research Program under Grant JCYJ20220531103008018,Grants 20231120142345001 and 20231127144045001the Natural Science Foundation of China under Grant U20A20156。
文摘This paper proposes a Deep Reinforcement Learning(DRL)algorithm for user scheduling in Millimeter Wave(mmWave)networks,which utilizes Channel Knowledge Map(CKM)for knowledge transfer to enhance the learning of scheduling strategies.The user scheduling and link configuration problems are modeled as a multiqueue system.Each queue represents the data demand of an individual user.This setup allows the base station to make dynamic scheduling decisions based on changing environmental conditions.This approach facilitates efficient management of user-specific requirements while addressing the challenges posed by dynamic network environments.Our model incorporates relay selection,codebook selection,and beam tracking to support flexible and efficient resource allocation.In contrast to traditional channel model-based optimization,we design algorithms for scheduling policy pre-training using CKMs,which provide information about the channel between specific pairs of locations.Specifically,we assume that the CKM is fully available to allow the complex scheduling network to have a better starting point or follow a more favorable gradient direction through knowledge migration.This integration of CKM with knowledge transfer significantly accelerates DRL convergence and enhances performance stability.Simulation results confirmed the effectiveness of the proposed approach.Relative to the baseline methods,integrating CKM with knowledge transfer accelerated the convergence of the DRL algorithm by approximately 20%,maintained the delay within 30 milliseconds,and reduced the average queue length by nearly 30%.
