摘要
提出了一种基于熵理论的BP神经网络结构设计修剪算法.其实质是依据Shannon熵原理,定义神经网络隐层节点输出的拟熵,该熵与Shannon熵对不确定性的描述具有相同的效果,但克服了Shannon熵固有的缺点.将神经网络实际输出与期望输出的交叉熵和隐节点输出拟熵作为代价函数,并采用熵周期的策略对网络参数进行寻优,最后通过删除合并隐层神经元达到简化神经网络结构的目的.通过逼近典型非线性函数进行仿真实验,结果表明,该修剪算法在保证其逼近性能的同时,可以简化BP神经网络结构.
A pruning algorithm based on entropy theory for designing BP neural network structure is proposed. The essence is to define the psuedo-entropy of the hidden node’s output of neural network based on the Shannon’s entropy principle. Both of the two different definitions of entropy have the same effect on the description of uncertainty, but the new definition of entropy overcomes the inherent drawbacks of Shannon’s entropy. The cross-entropy of neural network’s actual output and target output and the pseudo-entropy of the hidden node＇s output are used as cost function, and entropy cycle strategy is used to opimize the parameters of the network, and a simple neural network structure can be obtained by deleting and merging the hidden layer neurons at last. The simulation result of a typical non-linear function approximation shows that a simple architecture of BP neural networks can be achieved while the approximation performance is ensured.
出处
《信息与控制》
CSCD
北大核心
2009年第5期633-636,640,共5页
Information and Control
基金
国家自然科学基金资助项目(60736009)
辽宁省高校科研基金资助项目(2009S051)
关键词
交叉熵
拟熵
BP神经网络
修剪算法
cross-entropy
pseudo-entropy
back-propagation neural network
pruning algorithm
