This study introduces the Smart Exponential-Threshold-Linear with Double Deep Q-learning Network(SETL-DDQN)and an extended Gumbel distribution method,designed to optimize the Contention Window(CW)in IEEE 802.11 networ...This study introduces the Smart Exponential-Threshold-Linear with Double Deep Q-learning Network(SETL-DDQN)and an extended Gumbel distribution method,designed to optimize the Contention Window(CW)in IEEE 802.11 networks.Unlike conventional Deep Reinforcement Learning(DRL)-based approaches for CW size adjustment,which often suffer from overestimation bias and limited exploration diversity,leading to suboptimal throughput and collision performance.Our framework integrates the Gumbel distribution and extreme value theory to systematically enhance action selection under varying network conditions.First,SETL adopts a DDQN architecture(SETL-DDQN)to improve Q-value estimation accuracy and enhance training stability.Second,we incorporate a Gumbel distribution-driven exploration mechanism,forming SETL-DDQN(Gumbel),which employs the extreme value theory to promote diverse action selection,replacing the conventional-greedy exploration that undergoes early convergence to suboptimal solutions.Both models are evaluated through extensive simulations in static and time-varying IEEE 802.11 network scenarios.The results demonstrate that our approach consistently achieves higher throughput,lower collision rates,and improved adaptability,even under abrupt fluctuations in traffic load and network conditions.In particular,the Gumbel-based mechanism enhances the balance between exploration and exploitation,facilitating faster adaptation to varying congestion levels.These findings position Gumbel-enhanced DRL as an effective and robust solution for CW optimization in wireless networks,offering notable gains in efficiency and reliability over existing methods.展开更多
文摘This study introduces the Smart Exponential-Threshold-Linear with Double Deep Q-learning Network(SETL-DDQN)and an extended Gumbel distribution method,designed to optimize the Contention Window(CW)in IEEE 802.11 networks.Unlike conventional Deep Reinforcement Learning(DRL)-based approaches for CW size adjustment,which often suffer from overestimation bias and limited exploration diversity,leading to suboptimal throughput and collision performance.Our framework integrates the Gumbel distribution and extreme value theory to systematically enhance action selection under varying network conditions.First,SETL adopts a DDQN architecture(SETL-DDQN)to improve Q-value estimation accuracy and enhance training stability.Second,we incorporate a Gumbel distribution-driven exploration mechanism,forming SETL-DDQN(Gumbel),which employs the extreme value theory to promote diverse action selection,replacing the conventional-greedy exploration that undergoes early convergence to suboptimal solutions.Both models are evaluated through extensive simulations in static and time-varying IEEE 802.11 network scenarios.The results demonstrate that our approach consistently achieves higher throughput,lower collision rates,and improved adaptability,even under abrupt fluctuations in traffic load and network conditions.In particular,the Gumbel-based mechanism enhances the balance between exploration and exploitation,facilitating faster adaptation to varying congestion levels.These findings position Gumbel-enhanced DRL as an effective and robust solution for CW optimization in wireless networks,offering notable gains in efficiency and reliability over existing methods.
