摘要
谣言会对社会生活造成不利影响,同时具有多种模态的网络谣言比纯文字谣言更容易误导用户和传播,这使得对多模态的谣言检测不可忽视。目前关于多模态谣言检测方法没有关注词与图片区域对象之间的特征融合,因此提出了一种基于注意力机制的多模态融合网络AMFNN应用于谣言检测,该方法在词-视觉对象层面进行高级信息交互,利用注意力机制捕捉与关键词语相关的视觉特征;提出了基于自注意力机制的自适应注意力机制Adapive-SA,通过增加辅助条件来约束内部的信息流动,使得模态内的关系建模更有目标性和多样性。在两个多模态谣言检测数据集上进行了对比实验,结果表明,与目前相关的多模态谣言检测方法相比,AMFNN能够合理地处理多模态信息,从而提高了谣言检测的准确性。
Rumors have an adverse impact on social life,and online rumors with multiple modalities are more likely to mislead users and spread better than pure text rumors,which makes the detection of multimodal rumors not negligible.At present,the multimodal rumor detection method does not pay attention to the feature fusion between the word and the image area object.Therefore,this paper proposes an attention-based multimodal fusion neural network(AMFNN)for rumor detection.The method carries out high-level information interaction between word and visual object,using the attention mechanism to capture visual features related to the key feature of words.An adaptive attention mechanism(Adaptive-SA)based on the self-attention mechanism is proposed,which restricts the internal information flow by adding auxiliary conditions to make the relationship modeling within the modality is more targeted and diverse.The paper conducts comparative experiments on two datasets about multimodal rumor detection.Experimental results show that,compared with the current related multimodal rumor detection methods,AMFNN can reasonably process multimodal information,thereby improving the accuracy of rumor detection.
作者
戚力鑫
万书振
唐斌
徐义春
QI Lixin;WAN Shuzhen;TANG Bin;XU Yichun(College of Computer and Information,Three Gorges University,Yichang,Hubei 443000,China)
出处
《计算机工程与应用》
CSCD
北大核心
2022年第19期209-217,共9页
Computer Engineering and Applications
基金
NSFC-新疆联合基金重点项目(U1703261)
水电工程智能视觉监测湖北省重点实验室(三峡大学)开放基金(2017SDSJ06)。
关键词
深度学习
注意力机制
多模态融合
谣言检测
deep learning
attention mechanism
multimodal fusion
rumor detection
