摘要
针对预训练模型参数多且推理时间长导致在真实旅游场景应用受限的问题,提出一种知识蒸馏和领域知识融合的文本情感分类轻量模型(lightweight model for knowledge distillation and domain knowledge fusion,KD-DKF)。构建了旅游领域词典;在BERT-WWM-EXT模型的基础上加入改进的词性因子向量和位置信息相似度矩阵得到融入领域信息的BERT-WWM-EXT模型(domain information BERT-WWM-EXT,DI-BERT-WWM-EXT);考虑旅游场景对高效且轻量模型的需求,结合知识蒸馏理论,选择DI-BERT-WWM-EXT作为教师模型指导双向长短期记忆网络进行蒸馏,完成KD-DKF的构建。实验结果表明,KD-DKF准确率可达85.79%,高于其他8个同类别轻量模型;总训练时间为152.43 s,参数量为9.62×106,在保持较高准确率的同时提高了分类效率。
To address the limitations of pre-trained models in real-world tourism scenarios due to their large number of parameters and long inference time,this paper proposes a lightweight sentiment classification model that integrates knowledge distillation and domain knowledge(KD-DKF).A tourism domain lexicon is constructed,and an enhanced BERT-WWM-EXT model is developed by incorporating improved part-of-speech factor vectors and positional similarity matrices,resulting in the domain information BERT-WWM-EXT model(DI-BERT-WWM-EXT).Considering the need for efficient and lightweight models in tourism applications,the DI-BERT-WWM-EXT model is used as a teacher to guide a bidirectional long short-term memory(BiLSTM)network through knowledge distillation,thereby constructing the KD-DKF model.Experimental results show that KD-DKF achieves an accuracy of 85.79%,outperforming eight other lightweight models in the same category.It has a total training time of 152.43 seconds and a parameter count of 9.62×106,offering improved classification efficiency while maintaining high accuracy.
作者
李锦辉
刘继
闵兰
LI Jinhui;LIU Ji;MIN Lan(Department of Basic Teaching and Research,Xinjiang College of Science&Technology,Korla 841000,P.R.China;Institute of Statistics&Data Science,Xinjiang University of Finance&Economics,Urumqi 830012,P.R.China;Research Center for Xinjiang Social and Economic Statistics and Big Data Application,Xinjiang University of Finance&Economics,Urumqi 830012,P.R.China)
出处
《重庆邮电大学学报(自然科学版)》
北大核心
2025年第4期617-626,共10页
Journal of Chongqing University of Posts and Telecommunications(Natural Science Edition)
基金
国家自然科学基金项目(72164034)
新疆社会科学基金项目(2024BTJ057)。
