摘要
针对目前神经网络中的Levenberg-Marquardt反向传播(LMBP)算法在训练过程中有可能迭代到鞍点的问题,提出一种能有效克服鞍点的LMBP改进算法。计算鞍点处雅克比矩阵的正特征值对应的特征向量并将其作为新的搜索方向。通过实例对比传统LMBP算法与改进LMBP算法的效果,证明改进的算法能有效地脱离鞍点并进一步收敛到极小点处。
Aiming at the Levenberg-Marquardt Back Propagation（LMBP） algorithm of neural network sometimes converges to the saddle point during training process,an improved LMBP algorithm which can overcome the saddle point effectively is proposed.All eigenvectors for all the positive eigenvalue of Jacobi matrix are calculated as new searching directions.The improved LMBP algorithm is proved that it can get out of saddle point,and it iterates to minima effectively by an example of comparing with the traditional LMBP algorithm and the improved one.
出处
《计算机工程》
CAS
CSCD
2012年第23期173-176,180,共5页
Computer Engineering
基金
广东省教育部产学研结合基金资助项目(2009B090300393)
广州市软件(动漫)产业发展基金资助项目(2060404)
