摘要
面对日益增加的社交平台数据,多模态方面级情感分析对于理解用户的潜在情感至关重要。现有研究工作集中于通过跨模态融合图像和文本来完成情感分析任务,无法有效地捕获图像和文本中的隐含情感。此外,传统方法受限于模型具有的黑箱性质而缺乏可解释性。为应对上述问题,提出了基于置信度引导的提示学习(CPL)的多模态方面级情感分类模型。该模型由多模态特征处理模块(MF)、基于置信度的门控模块(CG)、提示构建模块(PC)和多模态分类模块(MC)组成。多模态特征提取模块用以提取多模态数据的特征;基于置信度的门控模块旨在通过自注意力网络的置信度评估样本的分类难度,对不同难易程度的样本进行自适应性处理;提示构建模块根据难易样本,采取不同的适应性模板提示,以引导T5大语言模型生成辅助情感线索;多模态分类模块用以预测结果。在公开数据集Twitter-2015和Twitter-2017的实验结果表明,与现有基线方法相比,所提出的多模态方面级情感分类模型具有显著性能优势,准确率分别提高了0.48%和1.06%。
With the increasing volume of data from social media platforms,multimodal aspect-level sentiment analysis is crucial for understanding the underlying emotions of users.Existing research primarily focuses on sentiment analysis tasks by fusing image and text modalities,but these methods fail to effectively capture the implicit emotions in both image and text.Furthermore,traditional approaches are often constrained by the black-box nature of the models,which lack interpretability.To address these issues,this paper proposes a confidence-guided prompt learning(CPL)based multimodal aspect-level sentiment analysis model,which consists of four key components:a multimodal feature processing module(MF),a confidence-based gating module(CG),a prompt construction module(PC),and a multimodal classification module(MC).The multimodal feature processing module is responsible for extracting features from multimodal data.The confidence-guided gating module evaluates the classification difficulty of samples using confidence assessment through a self-attention network and adaptively processes samples based on their difficulty.The prompt construction module generates adaptive prompt templates for different difficulty levels of samples to guide the T5 large language model in generating auxiliary sentiment cues.And the multimodal classification module is used for final sentiment prediction.Experimental results on the public datasets Twitter-2015 and Twitter-2017 show that,compared to existing baseline methods,the proposed multimodal aspect-level sentiment classification model achieves significant performance improvements,with accuracy increases of 0.48%and 1.06%,respectively.
作者
李懋林
林嘉杰
杨振国
LI Maolin;LIN Jiajie;YANG Zhenguo(School of Computer Science and Technology,Guangdong University of Technology,Guangzhou 510006,China)
出处
《计算机科学》
北大核心
2025年第7期241-247,共7页
Computer Science
基金
广东省基础与应用基础研究基金(2024A1515010237)。
关键词
多模态数据
大语言模型
情感分类
提示学习
分类置信度
Multimodal data
Large language models
Sentiment classification
Prompt learning
Classification confidence
