摘要
针对现有步态识别研究中步态信息挖掘不足和跨模态特征对齐不充分导致真实场景中识别性能受限的问题,提出基于蒙皮多人线性(SMPL)模态分解与嵌入融合的多模态步态识别方法.通过将SMPL模型分解为形状分支和姿势分支,全面提取人体静态形状特征和动态运动特征;构建自适应帧关节注意力模块,自适应聚焦关键帧与重要关节,增强姿势特征表达能力;设计模态嵌入融合模块,将不同模态特征投影至统一语义空间,并构建模态一致性损失函数,优化跨模态特征对齐,提升融合效果.在Gait3D数据集上的实验结果表明,与6种基于轮廓的方法、2种基于骨骼的方法以及5种基于轮廓和骨骼或SMPL模型的多模态方法比较,所提方法 Rank-1准确率达到70.4%,在复杂真实场景中表现出更高鲁棒性,验证了所提方法在模态特征提取和跨模态特征对齐方面的有效性.
A multimodal gait recognition method based on skinned multi-person linear(SMPL)modal decomposition and embedding fusion was proposed,to address the limitations in current gait recognition research,including insufficient gait information mining and inadequate cross-modal feature alignment that restrict recognition performance in real-world scenarios.The SMPL model was decomposed into a shape branch and a pose branch to comprehensively extract static body shape features and dynamic motion characteristics.An adaptive frame-joint attention module was constructed to focus on key frames and significant joints adaptively,thereby enhancing pose feature representation.A modality embedding fusion module was designed to project different modal features into a unified semantic space,and a modality consistency loss function was built to optimize cross-modal feature alignment and improve fusion effectiveness.Experimental results on the Gait3D dataset demonstrated that the proposed method achieved a Rank-1 accuracy of 70.4%,outperforming six silhouette-based methods,two skeleton-based methods,and five multimodal approaches combining silhouettes with skeletons or SMPL models.The method exhibits superior robustness in complex real-world scenarios,validating its effectiveness in modal feature extraction and cross-modal feature alignment.
作者
吴越
梁铮
高巍
杨茂达
赵培森
邓红霞
常媛媛
WU Yue;LIANG Zheng;GAO Wei;YANG Maoda;ZHAO Peisen;DENG Hongxia;CHANG Yuanyuan(College of Computer Science and Technology(College of Data Science),Taiyuan University of Technology,Taiyuan 030024,China;School of Physical Education and Health Engineering,Taiyuan University of Technology,Taiyuan 030024,China)
出处
《浙江大学学报(工学版)》
北大核心
2026年第1期52-60,共9页
Journal of Zhejiang University(Engineering Science)
基金
山西省中央引导地方科技发展资金项目(YDZJSX2022A016)
山西省重点研发计划资助项目(2022ZDYF128)
山西省科技战略项目(202404030401080)。
