Focused on various BP algorithms with variable learning rate based on network system error gradient, a modified learning strategy for training non-linear network models is developed with both the incremental and the d...Focused on various BP algorithms with variable learning rate based on network system error gradient, a modified learning strategy for training non-linear network models is developed with both the incremental and the decremental factors of network learning rate being adjusted adaptively and dynamically. The golden section law is put forward to build a relationship between the network training parameters, and a series of data from an existing model is used to train and test the network parameters. By means of the evaluation of network performance in respect to convergent speed and predicting precision, the effectiveness of the proposed learning strategy can be illustrated.展开更多
This paper presents an improved BP algorithm. The approach can reduce the amount of computation by using the logarithmic objective function. The learning rate μ(k) per iteration is determined by dynamic o...This paper presents an improved BP algorithm. The approach can reduce the amount of computation by using the logarithmic objective function. The learning rate μ(k) per iteration is determined by dynamic optimization method to accelerate the convergence rate. Since the determination of the learning rate in the proposed BP algorithm only uses the obtained first order derivatives in standard BP algorithm(SBP), the scale of computational and storage burden is like that of SBP algorithm,and the convergence rate is remarkably accelerated. Computer simulations demonstrate the effectiveness of the proposed algorithm展开更多
For accelerating the supervised learning by the SpikeProp algorithm with the temporal coding paradigm in spiking neural networks (SNNs), three learning rate adaptation methods (heuristic rule, delta-delta rule, and de...For accelerating the supervised learning by the SpikeProp algorithm with the temporal coding paradigm in spiking neural networks (SNNs), three learning rate adaptation methods (heuristic rule, delta-delta rule, and delta-bar-delta rule), which are used to speed up training in artificial neural networks, are used to develop the training algorithms for feedforward SNN. The performance of these algorithms is investigated by four experiments: classical XOR (exclusive or) problem, Iris dataset, fault diagnosis in the Tennessee Eastman process, and Poisson trains of discrete spikes. The results demonstrate that all the three learning rate adaptation methods are able to speed up convergence of SNN compared with the original SpikeProp algorithm. Furthermore, if the adaptive learning rate is used in combination with the momentum term, the two modifications will balance each other in a beneficial way to accomplish rapid and steady convergence. In the three learning rate adaptation methods, delta-bar-delta rule performs the best. The delta-bar-delta method with momentum has the fastest convergence rate, the greatest stability of training process, and the maximum accuracy of network learning. The proposed algorithms in this paper are simple and efficient, and consequently valuable for practical applications of SNN.展开更多
A multilayer perceptron neural network system is established to support the diagnosis for five most common heart diseases (coronary heart disease, rheumatic valvular heart disease, hypertension, chronic cor pulmonale ...A multilayer perceptron neural network system is established to support the diagnosis for five most common heart diseases (coronary heart disease, rheumatic valvular heart disease, hypertension, chronic cor pulmonale and congenital heart disease). Momentum term, adaptive learning rate, the forgetting mechanics, and conjugate gradients method are introduced to improve the basic BP algorithm aiming to speed up the convergence of the BP algorithm and enhance the accuracy for diagnosis. A heart disease database consisting of 352 samples is applied to the training and testing courses of the system. The performance of the system is assessed by cross-validation method. It is found that as the basic BP algorithm is improved step by step, the convergence speed and the classification accuracy of the network are enhanced, and the system has great application prospect in supporting heart diseases diagnosis.展开更多
针对捕鱼优化算法(catch fish optimization algorithm,CFOA)容易陷入局部最优、迭代后期种群多样性单一等问题,提出一种多策略融合改进的捕鱼优化算法。首先,通过反向学习策略进行种群初始化,以提高初始种群的质量;其次,引入组长趋同...针对捕鱼优化算法(catch fish optimization algorithm,CFOA)容易陷入局部最优、迭代后期种群多样性单一等问题,提出一种多策略融合改进的捕鱼优化算法。首先,通过反向学习策略进行种群初始化,以提高初始种群的质量;其次,引入组长趋同自适应组队策略,强化算法优势经验的学习;最后,通过引入Lévy飞行螺旋搜索策略,改善集体捕获阶段算法跳出局部最优值的能力;改进算法与灰狼优化(grey wolf optimization,GWO)算法、麻雀优化算法(sparrow search algorithm,SSA)、鲸鱼优化算法(whale optimization algorithm,WOA)、正弦余弦优化算法(sine cosine algorithm,SCA)等7种算法在15个基准测试函数上进行了仿真对比分析。试验结果表明,改进算法在求解精度和收敛速度等方面有较好提升。此外,3个工程设计优化问题的仿真试验进一步验证了改进算法在处理工程优化问题上的优越性。展开更多
在船舶设计过程中经常会出现随机新设计任务,为船舶设计任务调度方案的制订带来一定的困难。基于反向传播(Back Propagation, BP)算法,引入动量-自适应学习率反向传播(Momentum and Self-Adaptive Learning Rate Back Propagation, MSBP...在船舶设计过程中经常会出现随机新设计任务,为船舶设计任务调度方案的制订带来一定的困难。基于反向传播(Back Propagation, BP)算法,引入动量-自适应学习率反向传播(Momentum and Self-Adaptive Learning Rate Back Propagation, MSBP)算法预测随机新设计任务是否可加入制订的船舶设计任务调度方案,以解决扰动情况下的船舶设计任务动态调度(Dynamic Scheduling of Ship Design Tasks, DSSDT)问题。为减小求解空间和训练难度,选择对调度结果具有重大影响的属性作为MSBP算法的特征值。基于抽取的特征值构建MSBP算法模型,并采用大量数据完成对模型的训练。对比试验结果表明,MSBP算法的准确性优于未改进的BP算法,某项随机新设计任务的可调度性与其优先级最为密切。展开更多
文摘Focused on various BP algorithms with variable learning rate based on network system error gradient, a modified learning strategy for training non-linear network models is developed with both the incremental and the decremental factors of network learning rate being adjusted adaptively and dynamically. The golden section law is put forward to build a relationship between the network training parameters, and a series of data from an existing model is used to train and test the network parameters. By means of the evaluation of network performance in respect to convergent speed and predicting precision, the effectiveness of the proposed learning strategy can be illustrated.
文摘This paper presents an improved BP algorithm. The approach can reduce the amount of computation by using the logarithmic objective function. The learning rate μ(k) per iteration is determined by dynamic optimization method to accelerate the convergence rate. Since the determination of the learning rate in the proposed BP algorithm only uses the obtained first order derivatives in standard BP algorithm(SBP), the scale of computational and storage burden is like that of SBP algorithm,and the convergence rate is remarkably accelerated. Computer simulations demonstrate the effectiveness of the proposed algorithm
基金Supported by the National Natural Science Foundation of China (60904018, 61203040)the Natural Science Foundation of Fujian Province of China (2009J05147, 2011J01352)+1 种基金the Foundation for Distinguished Young Scholars of Higher Education of Fujian Province of China (JA10004)the Science Research Foundation of Huaqiao University (09BS617)
文摘For accelerating the supervised learning by the SpikeProp algorithm with the temporal coding paradigm in spiking neural networks (SNNs), three learning rate adaptation methods (heuristic rule, delta-delta rule, and delta-bar-delta rule), which are used to speed up training in artificial neural networks, are used to develop the training algorithms for feedforward SNN. The performance of these algorithms is investigated by four experiments: classical XOR (exclusive or) problem, Iris dataset, fault diagnosis in the Tennessee Eastman process, and Poisson trains of discrete spikes. The results demonstrate that all the three learning rate adaptation methods are able to speed up convergence of SNN compared with the original SpikeProp algorithm. Furthermore, if the adaptive learning rate is used in combination with the momentum term, the two modifications will balance each other in a beneficial way to accomplish rapid and steady convergence. In the three learning rate adaptation methods, delta-bar-delta rule performs the best. The delta-bar-delta method with momentum has the fastest convergence rate, the greatest stability of training process, and the maximum accuracy of network learning. The proposed algorithms in this paper are simple and efficient, and consequently valuable for practical applications of SNN.
基金the Natural Science Foundation of China (No. 30070211).
文摘A multilayer perceptron neural network system is established to support the diagnosis for five most common heart diseases (coronary heart disease, rheumatic valvular heart disease, hypertension, chronic cor pulmonale and congenital heart disease). Momentum term, adaptive learning rate, the forgetting mechanics, and conjugate gradients method are introduced to improve the basic BP algorithm aiming to speed up the convergence of the BP algorithm and enhance the accuracy for diagnosis. A heart disease database consisting of 352 samples is applied to the training and testing courses of the system. The performance of the system is assessed by cross-validation method. It is found that as the basic BP algorithm is improved step by step, the convergence speed and the classification accuracy of the network are enhanced, and the system has great application prospect in supporting heart diseases diagnosis.
