摘要
虽然经典的渐进式神经网络(PNN)通过获取先前任务的经验知识来提高神经网络在当前任务中的性能,但忽略了在渐进任务较多时渐进任务间的相关性差异对网络性能的影响。针对该场景,提出一种基于相关性差异化迁移的渐进式神经网络(CDT-PNN)。首先使用基于余弦相似度的算法评估两个渐进任务的相关性;然后利用当前任务和先前任务之间的相关性来决定神经网络的知识参数传递,并删除与当前渐进任务呈负相关的先前渐进任务的知识参数;最后依据任务间相关性按一定比例随机抽取与当前渐进任务呈正相关的先前渐进任务的知识参数进行参数迁移。在添加了不同程度噪声的cifar-100数据集和mnist数据集上进行实验。结果显示,与PNN相比,CDT-PNN在cifar-100和mnist数据集上的实验任务平均分类精度(AA)提高了6.6个百分点和1.58个百分点。这说明,在复杂多任务场景下CDT-PNN能获得比PNN更好的性能。
Classical Progressive Neural Network(PNN)improves the performance of neural networks on the current task by acquiring empirical knowledge of previous tasks,but ignores the influence of the correlation differences between progressive tasks on the performance of the network when there are many progressive tasks.In this scenario,a Progressive Neural Network based on Correlation Differentiation Transfer(CDT-PNN)was proposed for the large number of progressive tasks and the different correlations between tasks.Firstly,the correlation of two progressive tasks was evaluated by using a cosine similarity-based algorithm.Then,the knowledge parameter transfer of the neural network was determined by exploiting the correlations between the current task and the previous tasks,and the knowledge parameters of the previous progressive tasks that are negatively correlated with the current progressive task were removed.Finally,the knowledge parameters of the previous progressive tasks that are positively correlated with the current progressive task were randomly selected for parameter transfer.Experiments on cifar-100 and mnist data sets with different levels of noise were carried out.The results show that,specifically,CDT-PNN improves the Average classification Accuracy(AA)of experimental tasks by 6.6 percentage points and 1.58 percentage points compared with PNN on cifar-100 and mnist data sets.Indicating that CDTPNN can obtain better performance than PNN in complex multi-task scenarios.
作者
蔡昌骁
王士同
CAI Changxiao;WANG Shitong(School of Artificial Intelligence and Computer Science,Jiangnan University,Wuxi Jiangsu 214122,China)
出处
《计算机应用》
CSCD
北大核心
2023年第7期2107-2115,共9页
journal of Computer Applications
基金
江苏省自然科学基金资助项目(BK20191331)。
关键词
渐进式神经网络
相关性差异
渐进任务
参数传递
持续学习
复杂多任务
Progressive Neural Network(PNN)
correlation differentiation
progressive task
parameter passing
continual learning
complex multi-task
