Cybertwin-enabled 6th Generation(6G)network is envisioned to support artificial intelligence-native management to meet changing demands of 6G applications.Multi-Agent Deep Reinforcement Learning(MADRL)technologies dri...Cybertwin-enabled 6th Generation(6G)network is envisioned to support artificial intelligence-native management to meet changing demands of 6G applications.Multi-Agent Deep Reinforcement Learning(MADRL)technologies driven by Cybertwins have been proposed for adaptive task offloading strategies.However,the existence of random transmission delay between Cybertwin-driven agents and underlying networks is not considered in related works,which destroys the standard Markov property and increases the decision reaction time to reduce the task offloading strategy performance.In order to address this problem,we propose a pipelining task offloading method to lower the decision reaction time and model it as a delay-aware Markov Decision Process(MDP).Then,we design a delay-aware MADRL algorithm to minimize the weighted sum of task execution latency and energy consumption.Firstly,the state space is augmented using the lastly-received state and historical actions to rebuild the Markov property.Secondly,Gate Transformer-XL is introduced to capture historical actions'importance and maintain the consistent input dimension dynamically changed due to random transmission delays.Thirdly,a sampling method and a new loss function with the difference between the current and target state value and the difference between real state-action value and augmented state-action value are designed to obtain state transition trajectories close to the real ones.Numerical results demonstrate that the proposed methods are effective in reducing reaction time and improving the task offloading performance in the random-delay Cybertwin-enabled 6G networks.展开更多
In vehicular Ad-hoc network（VANET）, many multi-hop broadcast schemes are employed to widely propagate the warning messages among vehicles and the key is to dynamically determine the optimal relay vehicle for retrans...In vehicular Ad-hoc network（VANET）, many multi-hop broadcast schemes are employed to widely propagate the warning messages among vehicles and the key is to dynamically determine the optimal relay vehicle for retransmission. In order to achieve reliable and fast delivery of warning messages, this paper proposes a delay-aware and reliable broadcast protocol（DR-BP） based on transmit power control technique. First, a comprehensive model is derived to evaluate the transmission in vehicle-to-vehicle communications. This model considers the wireless channel fading, transmission delay and retransmissions characters occurring in the physical layer/medium access control（PHY/MAC） layer. Then, a local optimal relay selection mechanism based on the above model is designed. In DR-BP scheme, only the vehicle selected as the optimal relays can forward warning messages and the transmit power is time-varying. Finally, extensive simulations verify the performance of DR-BP under different traffic scenarios. Simulation results show that DR-BP outperforms the traditional slotted 1-persistence（S1P） and flooding scheme in terms of packets delivery ratio and transmission delay.展开更多
With the popularity of variety delay-sensitive services, how to guarantee the delay requirements for mobile users （MUs） is a great challenge for downlink beamformer design in green cloud radio access networks （C-R...With the popularity of variety delay-sensitive services, how to guarantee the delay requirements for mobile users （MUs） is a great challenge for downlink beamformer design in green cloud radio access networks （C-RANs）. In this paper, we consider the problem of the delay-aware downlink beamforming with discrete rate adaptation to minimize the power consumption of C-RANs. We address the problem via a mixed integer nonlinear program （MINLP）, and then reformulate the MINLP problem as a mixed integer second-order cone program （MI-SOCP）, which is a convex program when the integer variables are relaxed as continuous ones. Based on this formulation, a deflation algorithm, whose computational complexity is polynomial, is proposed to derive the suboptimal solution. The simulation results are presented to validate the effectiveness of our proposed algorithm.展开更多
基金funded by the National Key Research and Development Program of China under Grant 2019YFB1803301Beijing Natural Science Foundation (L202002)。
文摘Cybertwin-enabled 6th Generation(6G)network is envisioned to support artificial intelligence-native management to meet changing demands of 6G applications.Multi-Agent Deep Reinforcement Learning(MADRL)technologies driven by Cybertwins have been proposed for adaptive task offloading strategies.However,the existence of random transmission delay between Cybertwin-driven agents and underlying networks is not considered in related works,which destroys the standard Markov property and increases the decision reaction time to reduce the task offloading strategy performance.In order to address this problem,we propose a pipelining task offloading method to lower the decision reaction time and model it as a delay-aware Markov Decision Process(MDP).Then,we design a delay-aware MADRL algorithm to minimize the weighted sum of task execution latency and energy consumption.Firstly,the state space is augmented using the lastly-received state and historical actions to rebuild the Markov property.Secondly,Gate Transformer-XL is introduced to capture historical actions'importance and maintain the consistent input dimension dynamically changed due to random transmission delays.Thirdly,a sampling method and a new loss function with the difference between the current and target state value and the difference between real state-action value and augmented state-action value are designed to obtain state transition trajectories close to the real ones.Numerical results demonstrate that the proposed methods are effective in reducing reaction time and improving the task offloading performance in the random-delay Cybertwin-enabled 6G networks.
基金supported by the National Basic Research Program of China(2011CB302905)the National Natural Science Foundation of China(61170058)+3 种基金National Science and Technology Major Project(2011ZX03005-004-04,2012ZX03005009)the Research Fund for the Doctoral Program of Higher Education of China(20103402110041,20123402110019)the Guangdong Province and CAS Comprehensive Strategic Cooperation Projects(2012B090400013)the Fundamental Research Project of Suzhou(SYG201143)
文摘In vehicular Ad-hoc network（VANET）, many multi-hop broadcast schemes are employed to widely propagate the warning messages among vehicles and the key is to dynamically determine the optimal relay vehicle for retransmission. In order to achieve reliable and fast delivery of warning messages, this paper proposes a delay-aware and reliable broadcast protocol（DR-BP） based on transmit power control technique. First, a comprehensive model is derived to evaluate the transmission in vehicle-to-vehicle communications. This model considers the wireless channel fading, transmission delay and retransmissions characters occurring in the physical layer/medium access control（PHY/MAC） layer. Then, a local optimal relay selection mechanism based on the above model is designed. In DR-BP scheme, only the vehicle selected as the optimal relays can forward warning messages and the transmit power is time-varying. Finally, extensive simulations verify the performance of DR-BP under different traffic scenarios. Simulation results show that DR-BP outperforms the traditional slotted 1-persistence（S1P） and flooding scheme in terms of packets delivery ratio and transmission delay.
基金supported by the National Natural Science Foundation of China(61501047,61671088)
文摘With the popularity of variety delay-sensitive services, how to guarantee the delay requirements for mobile users （MUs） is a great challenge for downlink beamformer design in green cloud radio access networks （C-RANs）. In this paper, we consider the problem of the delay-aware downlink beamforming with discrete rate adaptation to minimize the power consumption of C-RANs. We address the problem via a mixed integer nonlinear program （MINLP）, and then reformulate the MINLP problem as a mixed integer second-order cone program （MI-SOCP）, which is a convex program when the integer variables are relaxed as continuous ones. Based on this formulation, a deflation algorithm, whose computational complexity is polynomial, is proposed to derive the suboptimal solution. The simulation results are presented to validate the effectiveness of our proposed algorithm.
