摘要
针对当前多模态谣言检测模型存在的模态间信息融合不足和过于依赖各模态信息完整度的问题,提出一种基于多级融合的多模态谣言检测模型。分别利用TextCNN和Resnet18网络对文本和图片编码并进行特征级融合,对纯文本模型、纯图片模型和特征级融合模型进行决策级融合并对决策级融合进行改进。多级融合框架加深各模态间的信息融合程度,改进后的决策级融合有效缓解了传统模型对各模态信息完整度要求过高的问题。实验结果表明,该模型在微博数据集上的F1值和准确率均高于传统的多模态谣言检测模型,进一步提升了谣言检测效果。
Aiming at the problems of insufficient information fusion among modes and over-dependence on the integrity of information of each mode in current multimodal rumor detection models,a multimodal rumor detection model based on multilevel fusion was proposed.TextCNN and Resnet18 networks were used to encode text and image respectively,and feature level fusion was carried out.Decision level fusion was carried out on the pure text model,pure image model and feature level fusion model,and the decision level fusion was improved.The multi-level fusion framework deepened the degree of information fusion among various modes,and the improved decision-level fusion effectively alleviated the problem that the traditional model requires too much information integrity of each mode.Experimental results show that the F1 value and accuracy of the proposed model are higher than that of the traditional multimodal rumor detection model on the microblog dataset,which further improves the rumor detection effect.
作者
王壮
隋杰
WANG Zhuang;SUI Jie(School of Engineering Sciences,University of Chinese Academy of Sciences,Beijing 100049,China)
出处
《计算机工程与设计》
北大核心
2022年第6期1756-1761,共6页
Computer Engineering and Design
基金
国家重点研发计划基金项目(2017YFB0803001)
国家自然科学基金面上基金项目(61572459)。
