摘要
深度学习算法在虚假新闻检测关键特征提取方面具有优势,然而,现有的基于深度学习的多模态虚假新闻检测方法仍存在不足之处,例如,从输入的图像与文本中提取特征并进行特征融合时存在融合不充分的问题。针对这一问题,该文提出了一种基于多模态上下文融合及语义增强的虚假新闻检测模型MCEFSE(Multimodal Context based Early Fusion and Semantic Enhancement)。首先,该文利用预训练语言模型BERT对句子进行编码。同时,以Swin Transformer模型作为主要框架,在早期视觉特征编码时引入文本特征,增强语义交互。此外,我们还使用InceptionNetV3作为图像模式分析器。最后,对文本语义、视觉语义和图像模式特征进行细化和融合,得到最终的多模态特征表示。结果显示,MCEFSE模型在微博数据集和微博-21数据集上的准确率分别为0.921和0.932,验证了该方法的有效性。
Deep learning is the mainstream solution in multimodal fake news detection.To better capture information from input image and text,this paper proposes a fake news detection model named MCEFSE(Multimodal Context-based Early Fusion and Semantic Enhancement).Pre-trained language model BERT is employed for sentence encoding.Swin Transformer serves as the primary framework to introduce text features during visual feature coding.InceptionNetV3 is adopted as the image pattern analyzer.Features including text semantics,visual semantics,and image patterns are further refined and fused to form the final comprehensive multimodal representation.Experimental results on Weibo and Weibo-21 demonstrate the proposed model achieves the accuracy of 0.921 and 0.932,respectively.
作者
郝秀兰
徐稳静
魏少华
刘权
HAO Xiulan;XU Wenjing;WEI Shaohua;LIU Quan(School of Information Engineering,Huzhou Normal University,Huzhou,Zhejiang 313000,China;Zhejiang Province Key Laboratory of Smart Management and Application of Modern Agricultural Resources,Huzhou Normal University,Huzhou,Zhejiang 313000,China;Zhejiang Key Laboratory of Intelligent Education Fechnology and Application,Zhejiang Normal University,Jinhua,Zhejiang 321004,China)
出处
《中文信息学报》
北大核心
2025年第5期140-149,共10页
Journal of Chinese Information Processing
基金
湖州师范学院研究生科研创新项目课题(2023KYCX42)
浙江省现代农业资源智慧管理与应用研究重点实验室基金(2020E10017)。
关键词
虚假新闻检测
多模态上下文
特征融合
语义增强
fake news detection
multimodal context
features fusion
semantic enhancement
