摘要
虚假新闻如果得不到及时处理,可能会造成严重后果。当前的多模态虚假新闻检测方法主要使用各种注意力机制对单模态特征进行融合,未考虑到不同模态的特征间可能存在语义差距,也未充分利用多模态预训练模型的潜力。本文提出一个新的多模态虚假新闻检测模型,对特征进行多阶段融合。该模型利用多模态预训练模型提取已对齐的特征,然后借助注意力机制使特征互相增强,拼接经过增强的特征以实现早期融合,再通过多尺度融合模块捕捉不同模态特征之间的交互信息,并学习融合权重以实现特征的后期融合。实验结果显示,本文提出的模型取得比同类模型更好的效果,验证了注意力机制与多尺度融合模块的有效性。
Fake news can have serious consequences if not dealt with in a timely manner.Currently,various attention mechanisms are mainly employed in multimodal fake news detection methods to fuse unimodal features.The semantic gaps that may exist between different modal features are not taken into account,nor is the potential of multimodal pre-training models fully exploited.To this end,a new multimodal fake news detection model that performs multistage fusion of features is proposed in this paper.The pretrained multimodal model is utilized by the proposed model to extract the aligned features.Then the features are enhanced by each other through the attention mechanism,and the enhanced features are spliced to achieve early fusion.Finally,the interaction information between different modal features is captured by the multiscale fusion module,and the fusion weights are learned to realize the late fusion of features.It is shown by the experimental results that the model proposed in this paper achieves better results than similar models,and the effectiveness of the attention mechanism and the multiscale fusion module is also verified by the experimental results.
作者
施子豪
蒙祖强
谈超洪
SHI Zihao;MENG Zuqiang;TAN Chaohong(College of Computer,Electronics and Information,Guangxi University,Nanning Guangxi 530004,China;Guangxi Key Laboratory of Digital Infrastructure(Guangxi Zhuang Autonomous Region Information Center),Nanning Guangxi 530000,China)
出处
《广西师范大学学报(自然科学版)》
北大核心
2026年第1期68-79,共12页
Journal of Guangxi Normal University:Natural Science Edition
基金
国家自然科学基金(62266004)
广西数字基础设施重点实验室开放基金(GXDINBC202401)。
关键词
多模态
虚假新闻检测
注意力机制
多尺度融合
多阶段融合
multimodality
fake news detection
attention mechanism
multiscale fusion
multistage fusion
