In this study,a solution based on deep Q network(DQN)is proposed to address the relay selection problem in cooperative non-orthogonal multiple access(NOMA)systems.DQN is particularly effective in addressing problems w...In this study,a solution based on deep Q network(DQN)is proposed to address the relay selection problem in cooperative non-orthogonal multiple access(NOMA)systems.DQN is particularly effective in addressing problems within dynamic and complex communication environ-ments.By formulating the relay selection problem as a Markov decision process(MDP),the DQN algorithm employs deep neural networks(DNNs)to learn and make decisions through real-time interactions with the communication environment,aiming to minimize the system’s outage proba-bility.During the learning process,the DQN algorithm progressively acquires channel state infor-mation(CSI)between two nodes,thereby minimizing the system’s outage probability until a sta-ble level is reached.Simulation results show that the proposed method effectively reduces the out-age probability by 82%compared to the two-way relay selection scheme(Two-Way)when the sig-nal-to-noise ratio(SNR)is 30 dB.This study demonstrates the applicability and advantages of the DQN algorithm in cooperative NOMA systems,providing a novel approach to addressing real-time relay selection challenges in dynamic communication environments.展开更多
针对深度Q网络(DQN)算法因过估计导致收敛稳定性差的问题,在传统时序差分(TD)的基础上提出N阶TD误差的概念,设计基于二阶TD误差的双网络DQN算法。构造基于二阶TD误差的值函数更新公式,同时结合DQN算法建立双网络模型,得到两个同构的值...针对深度Q网络(DQN)算法因过估计导致收敛稳定性差的问题,在传统时序差分(TD)的基础上提出N阶TD误差的概念,设计基于二阶TD误差的双网络DQN算法。构造基于二阶TD误差的值函数更新公式,同时结合DQN算法建立双网络模型,得到两个同构的值函数网络分别用于表示先后两轮的值函数,协同更新网络参数,以提高DQN算法中值函数估计的稳定性。基于Open AI Gym平台的实验结果表明,在解决Mountain Car和Cart Pole问题方面,该算法较经典DQN算法具有更好的收敛稳定性。展开更多
基金supported by the National Natural Science Foundation of China(Nos.61841107 and 62061024)Gansu Natural Sci-ence Foundation(Nos.22JR5RA274 and 23YFGA0062)Gansu Innovation Foundation(No.2022A-215).
文摘In this study,a solution based on deep Q network(DQN)is proposed to address the relay selection problem in cooperative non-orthogonal multiple access(NOMA)systems.DQN is particularly effective in addressing problems within dynamic and complex communication environ-ments.By formulating the relay selection problem as a Markov decision process(MDP),the DQN algorithm employs deep neural networks(DNNs)to learn and make decisions through real-time interactions with the communication environment,aiming to minimize the system’s outage proba-bility.During the learning process,the DQN algorithm progressively acquires channel state infor-mation(CSI)between two nodes,thereby minimizing the system’s outage probability until a sta-ble level is reached.Simulation results show that the proposed method effectively reduces the out-age probability by 82%compared to the two-way relay selection scheme(Two-Way)when the sig-nal-to-noise ratio(SNR)is 30 dB.This study demonstrates the applicability and advantages of the DQN algorithm in cooperative NOMA systems,providing a novel approach to addressing real-time relay selection challenges in dynamic communication environments.
文摘针对深度Q网络(DQN)算法因过估计导致收敛稳定性差的问题,在传统时序差分(TD)的基础上提出N阶TD误差的概念,设计基于二阶TD误差的双网络DQN算法。构造基于二阶TD误差的值函数更新公式,同时结合DQN算法建立双网络模型,得到两个同构的值函数网络分别用于表示先后两轮的值函数,协同更新网络参数,以提高DQN算法中值函数估计的稳定性。基于Open AI Gym平台的实验结果表明,在解决Mountain Car和Cart Pole问题方面,该算法较经典DQN算法具有更好的收敛稳定性。
