摘要
灾难性遗忘对联邦类增量学习(FCIL)构成了显著挑战,导致进行FCIL持续任务时性能下降的问题。针对此问题,提出一种融合多头自注意力(MHSA)的标签语义嵌入(LSE)的FCIL方法——ATTLSE(ATTention Label Semantic Embedding)。首先,融合MHSA的LSE和生成器;其次,在无数据知识蒸馏(DFKD)阶段,依靠融合MHSA的生成器生成更多有意义的数据样本,以指导用户端模型的训练,并降低灾难性遗忘问题在FCIL中的影响。实验结果表明,在CIFAR-100和Tiny_ImageNet数据集上,与LANDER(Label Text Centered Data-Free Knowledge Transfer)方法相比,ATTLSE的平均准确率提升了0.06~6.45个百分点,缓解了持续任务在联邦类增量学习上的灾难性遗忘问题。
Catastrophic forgetting poses a significant challenge to Federated Class-Incremental Learning(FCIL),leading to performance degradation of continuous tasks in FCIL.To address this issue,an FCIL method of Label Semantic Embedding(LSE)with Multi-Head Self-Attention(MHSA)—ATTLSE(ATTention Label Semantic Embedding)was proposed.Firstly,an LSE with MHSA was integrated with a generator.Secondly,during the stage of Data-Free Knowledge Distillation(DFKD),the generator with MHSA was used to produce more meaningful data samples,which guided the training of client models and reduced the influence of catastrophic forgetting problem in FCIL.Experiments were carried out on the CIFAR-100 and Tiny_ImageNet datasets.The results demonstrate that the average accuracy of ATTLSE is improved by 0.06 to 6.45 percentage points compared to LANDER(Label Text Centered Data-Free Knowledge Transfer)method,so as to solve the catastrophic forgetting problem to certain extent of continuous tasks in FCIL.
作者
王虎
王晓峰
李可
马云洁
WANG Hu;WANG Xiaofeng;LI Ke;MA Yunjie(School of Computer Science and Engineering,North Minzu University,Yinchuan Ningxia 750021,China;The Key Laboratory of Images and Graphics Intelligent Processing of State Ethnic Affairs Commission(North Minzu University),Yinchuan Ningxia 750021,China;School of Mathematics and Information Science,North Minzu University,Yinchuan Ningxia 750021,China)
出处
《计算机应用》
北大核心
2025年第10期3083-3090,共8页
journal of Computer Applications
基金
宁夏自然科学基金资助项目(2024AAC03165,2024AAC03169)
宁夏青年拔尖人才项目(2021)。
关键词
灾难性遗忘
联邦类增量学习
多头自注意力
标签语义嵌入
无数据知识蒸馏
catastrophic forgetting
Federated Class-Incremental Learning(FCIL)
Multi-Head Self-Attention(MHSA)
Label Semantic Embedding(LSE)
Data-Free Knowledge Distillation(DFKD)
