摘要
多模态情绪识别技术在心理健康检测与机器情感分析中应用广泛,但现有方法多依赖全局或局部特征,忽略了二者的联合建模,限制了情绪识别性能。为此,提出了一种基于Transformer的双级门控分段式多模态情绪识别模型(dual-stage gated segmented multimodal emotion recognition method,DGM)。DGM采用分段式融合架构,包括交互阶段与双级门控阶段。交互阶段采用OAGL融合策略建模全局-局部跨模态交互,优化特征融合效率;双级门控阶段整合局部与全局特征,充分利用情绪信息。此外,针对模态间局部时序特征不对齐问题,设计了基于缩放点积的序列对齐方法以提升融合精度。在CMU-MOSI、CMU-MOSEI和CH-SIMS 3个基准数据集上的实验表明,DGM在多数据集上的识别效果优于现有算法,验证了其捕捉情绪细节的能力与泛化性能。
Multimodal emotion recognition has broad applications in mental health detection and affective computing.However,most existing methods rely on either global or local features,neglecting the joint modeling of both,which limits emotion recognition performance.To address this,a Transformer-based dual-stage gated segmented multimodal emotion recognition method(DGM).DGM adopts a segmented fusion architecture was proposed,consisting of an interaction stage and a dual-stage gating stage.In the interaction stage,the OAGL fusion strategy was employed to model globallocal cross-modal interactions,improving the efficiency of feature fusion.The dual-stage gating stage integrates local and global features was designed to fully utilize emotional information.Additionally,to resolve the misalignment of local temporal features across modalities,a scaled dot-product-based sequence alignment method was developed to enhance fusion accuracy.Experimental were conducted on three benchmark datasets(CMU-MOSI,CMU-MOSEI,and CH-SIMS),and the results demonstrate that DGM outperforms representative algorithms on multiple datasets,validating its ability to capture emotional details and its strong generalization capability.
作者
马飞
李树志
杨飞霞
徐光宪
MA Fei;LI Shuzhi;YANG Feixia;XU Guangxian(School of Electronic and Information Engineering,Liaoning Technical University,Huludao 125105,China;School of Electrical and Control Engineering,Liaoning Technical University,Huludao 125105,China)
出处
《智能科学与技术学报》
2025年第2期257-267,共11页
Chinese Journal of Intelligent Science and Technology
基金
辽宁省教育科学“十四五”规划课题(No.JG24DB219)
辽宁省科技厅自然科学基金计划面上项目(No.2023-MS-314)
辽宁省教育厅高校科研业务经费项目(No.LJ242410147006)
辽宁工程技术大学GPU资源支持项目。
