We introduce an innovative approach to address a significant challenge in interaction recognition,specificallythe capture of correlation features between different interaction body parts.These features are oftenoverlo...We introduce an innovative approach to address a significant challenge in interaction recognition,specificallythe capture of correlation features between different interaction body parts.These features are oftenoverlooked by traditional graph convolution networks commonly used in interaction recognition tasks.Oursolution,the Merge-and-Split Graph Convolutional Network,takes a unique perspective,treating interactionrecognition as a global problem.It leverages a Merge-and-Split Graph structure to effectively capturedependencies between interaction body parts.To extract the essential interaction features,we introducethe Merge-and-Split Graph Convolution module,which seamlessly combines the Merge-and-Split Graphwith Graph Convolutional Networks.This fusion enables the extraction of rich semantic information betweenadjacent joint points.In addition,we introduce a Short-term Dependence module designed to extract jointand motion characteristics specific to each type of interaction.Furthermore,to extract correlation featuresbetween different hierarchical sets,we present the Hierarchical Guided Attention Module.This module playsa crucial role in highlighting the relevant hierarchical sets that contain essential interaction information.The effectiveness of our proposed model is demonstrated by achieving state-of-the-art performance on 2widely recognized datasets,namely,the NTU60 and NTU120 interaction datasets.Our model’s efficacy isrigorously validated through extensive experiments,and we have made the code available for the researchcommunity at https://github.com/wanghq05/MS-GCN/.展开更多
基金funding from the NationalNatural Science Foundation of China under Grant.No.62073004support from the Shenzhen Fundamental ResearchProgram under Grants.No.GXWD20201231165807007-20200807164903001 and JCYJ20200109140410340.
文摘We introduce an innovative approach to address a significant challenge in interaction recognition,specificallythe capture of correlation features between different interaction body parts.These features are oftenoverlooked by traditional graph convolution networks commonly used in interaction recognition tasks.Oursolution,the Merge-and-Split Graph Convolutional Network,takes a unique perspective,treating interactionrecognition as a global problem.It leverages a Merge-and-Split Graph structure to effectively capturedependencies between interaction body parts.To extract the essential interaction features,we introducethe Merge-and-Split Graph Convolution module,which seamlessly combines the Merge-and-Split Graphwith Graph Convolutional Networks.This fusion enables the extraction of rich semantic information betweenadjacent joint points.In addition,we introduce a Short-term Dependence module designed to extract jointand motion characteristics specific to each type of interaction.Furthermore,to extract correlation featuresbetween different hierarchical sets,we present the Hierarchical Guided Attention Module.This module playsa crucial role in highlighting the relevant hierarchical sets that contain essential interaction information.The effectiveness of our proposed model is demonstrated by achieving state-of-the-art performance on 2widely recognized datasets,namely,the NTU60 and NTU120 interaction datasets.Our model’s efficacy isrigorously validated through extensive experiments,and we have made the code available for the researchcommunity at https://github.com/wanghq05/MS-GCN/.
