摘要
自从Alphago战胜人类职业围棋选手以来,科研领域就掀起了深度神经网络研究的热潮,而优化器是神经网络训练各神经元之间参数学习的核心算法。经过分析目前多个最具影响力的神经网络优化算法,总结各算法的优缺点,并通过主流的神经网络框架Pytorch搭建神经网络对不同的优化算法进行测试,对比参数收敛速度及误差率。测试表明在相同学习率、相同批量大小、用相同数据训练同一神经网络的情况下,不同的优化算法对神经网络参数收敛速度和误差率影响很大。
Since Alphago defeated human professional Go players,the research field has set off a wave of deep neural network research,and the optimizer is the core algorithm for neural network training parameters between neurons.After analyzing several current most influential neural network optimization algorithms,the advantages and disadvantages of each algorithm are summarized.The neural network is built by the mainstream neural network framework Pytorch to test different optimization algorithms,and the parameter convergence speed and error rate are compared.Tests show that different optimization algorithms have a great influence on the convergence speed and error rate of neural network parameters under the same learning rate,the same batch size,the same data training and the same neural network.
作者
姬壮伟
JI Zhuang-wei(Department of Computer Science,Changzhi University,Changzhi Shanxi,046011)
出处
《山西大同大学学报（自然科学版）》
2020年第6期51-53,58,共4页
Journal of Shanxi Datong University(Natural Science Edition)
关键词
神经网络
优化算法
梯度下降
参数收敛
neural network
optimization algorithm
gradient descent
parameter convergence
