In this study,a solution based on deep Q network(DQN)is proposed to address the relay selection problem in cooperative non-orthogonal multiple access(NOMA)systems.DQN is particularly effective in addressing problems w...In this study,a solution based on deep Q network(DQN)is proposed to address the relay selection problem in cooperative non-orthogonal multiple access(NOMA)systems.DQN is particularly effective in addressing problems within dynamic and complex communication environ-ments.By formulating the relay selection problem as a Markov decision process(MDP),the DQN algorithm employs deep neural networks(DNNs)to learn and make decisions through real-time interactions with the communication environment,aiming to minimize the system’s outage proba-bility.During the learning process,the DQN algorithm progressively acquires channel state infor-mation(CSI)between two nodes,thereby minimizing the system’s outage probability until a sta-ble level is reached.Simulation results show that the proposed method effectively reduces the out-age probability by 82%compared to the two-way relay selection scheme(Two-Way)when the sig-nal-to-noise ratio(SNR)is 30 dB.This study demonstrates the applicability and advantages of the DQN algorithm in cooperative NOMA systems,providing a novel approach to addressing real-time relay selection challenges in dynamic communication environments.展开更多
工业应用中,动态多变的流式数据特性使强化学习算法在训练过程中很难在模型收敛性与知识遗忘之间实现很好的平衡。考虑工业现场内容请求与当前生产任务具有高度相关性,提出一种基于集成深度Q网络算法(Integrated Deep Q-Network,IDQN)...工业应用中,动态多变的流式数据特性使强化学习算法在训练过程中很难在模型收敛性与知识遗忘之间实现很好的平衡。考虑工业现场内容请求与当前生产任务具有高度相关性,提出一种基于集成深度Q网络算法(Integrated Deep Q-Network,IDQN)的自适应缓存策略。算法在离线阶段利用不同历史任务数据,训练并保存多个历史任务模型。在线阶段每当检测到实时数据流的任务特征发生变化,则重新训练网络模型。如果实时数据流的特征隶属于历史任务,则向深度Q网络(Deep Q-Network,DQN)导入相应的历史任务模型进行网络训练。否则直接利用实时数据流训练并标记为新的任务模型。仿真实验结果表明,IDQN与参考算法相比,在内容请求流行度动态变化时能够有效减少模型收敛时间,提高缓存效率。展开更多
基金supported by the National Natural Science Foundation of China(Nos.61841107 and 62061024)Gansu Natural Sci-ence Foundation(Nos.22JR5RA274 and 23YFGA0062)Gansu Innovation Foundation(No.2022A-215).
文摘In this study,a solution based on deep Q network(DQN)is proposed to address the relay selection problem in cooperative non-orthogonal multiple access(NOMA)systems.DQN is particularly effective in addressing problems within dynamic and complex communication environ-ments.By formulating the relay selection problem as a Markov decision process(MDP),the DQN algorithm employs deep neural networks(DNNs)to learn and make decisions through real-time interactions with the communication environment,aiming to minimize the system’s outage proba-bility.During the learning process,the DQN algorithm progressively acquires channel state infor-mation(CSI)between two nodes,thereby minimizing the system’s outage probability until a sta-ble level is reached.Simulation results show that the proposed method effectively reduces the out-age probability by 82%compared to the two-way relay selection scheme(Two-Way)when the sig-nal-to-noise ratio(SNR)is 30 dB.This study demonstrates the applicability and advantages of the DQN algorithm in cooperative NOMA systems,providing a novel approach to addressing real-time relay selection challenges in dynamic communication environments.
文摘工业应用中,动态多变的流式数据特性使强化学习算法在训练过程中很难在模型收敛性与知识遗忘之间实现很好的平衡。考虑工业现场内容请求与当前生产任务具有高度相关性,提出一种基于集成深度Q网络算法(Integrated Deep Q-Network,IDQN)的自适应缓存策略。算法在离线阶段利用不同历史任务数据,训练并保存多个历史任务模型。在线阶段每当检测到实时数据流的任务特征发生变化,则重新训练网络模型。如果实时数据流的特征隶属于历史任务,则向深度Q网络(Deep Q-Network,DQN)导入相应的历史任务模型进行网络训练。否则直接利用实时数据流训练并标记为新的任务模型。仿真实验结果表明,IDQN与参考算法相比,在内容请求流行度动态变化时能够有效减少模型收敛时间,提高缓存效率。
文摘针对传统深度Q学习网络(deep Q-learning network,DQN)在具有动态障碍物的路径规划下,移动机器人在探索时频繁碰撞难以移动至目标点的问题,通过在探索策略和经验回放机制上进行改进,提出一种改进的DQN算法。在探索策略上,利用快速搜索随机树(rapidly-exploring random tree,RRT)算法自动生成静态先验知识来指导动作选取,替代ε-贪婪策略的随机动作,提高智能体到达目标的成功率;在经验利用上,使用K-means算法设计一种聚类经验回放机制,根据动态障碍物的位置信息进行聚类分簇,着重采样与当前智能体状态相似的经验进行回放,使智能体更有效地避免碰撞动态障碍物。二维栅格化环境下的仿真实验表明,在动态环境下,该算法可以避开静态和动态障碍物,成功移动至目标点,验证了该算法在应对动态避障路径规划的可行性。
