In view of the weak ability of the convolutional neural networks to explicitly learn spatial invariance and the probabilistic loss of discriminative features caused by occlusion and background interference in pedestri...In view of the weak ability of the convolutional neural networks to explicitly learn spatial invariance and the probabilistic loss of discriminative features caused by occlusion and background interference in pedestrian re-identification tasks,a person re-identification method combining spatial feature learning and multi-granularity feature fusion was proposed.First,an attention spatial transformation network(A-STN)is proposed to learn spatial features and solve the problem of misalignment of pedestrian spatial features.Then the network was divided into a global branch,a local coarse-grained fusion branch,and a local fine-grained fusion branch to extract pedestrian global features,coarse-grained fusion features,and fine-grained fusion features,respectively.Among them,the global branch enriches the global features by fusing different pooling features.The local coarse-grained fusion branch uses an overlay pooling to enhance each local feature while learning the correlation relationship between multi-granularity features.The local fine-grained fusion branch uses a differential pooling to obtain the differential features that were fused with global features to learn the relationship between pedestrian local features and pedestrian global features.Finally,the proposed method was compared on three public datasets:Market1501,DukeMTMC-ReID and CUHK03.The experimental results were better than those of the comparative methods,which verifies the effectiveness of the proposed method.展开更多
Predicting human motion based on historical motion sequences is a fundamental problem in computer vision,which is at the core of many applications.Existing approaches primarily focus on encoding spatial dependencies a...Predicting human motion based on historical motion sequences is a fundamental problem in computer vision,which is at the core of many applications.Existing approaches primarily focus on encoding spatial dependencies among human joints while ignoring the temporal cues and the complex relationships across non-consecutive frames.These limitations hinder the model’s ability to generate accurate predictions over longer time horizons and in scenarios with complex motion patterns.To address the above problems,we proposed a novel multi-level spatial and temporal learning model,which consists of a Cross Spatial Dependencies Encoding Module(CSM)and a Dynamic Temporal Connection Encoding Module(DTM).Specifically,the CSM is designed to capture complementary local and global spatial dependent information at both the joint level and the joint pair level.We further present DTM to encode diverse temporal evolution contexts and compress motion features to a deep level,enabling the model to capture both short-term and long-term dependencies efficiently.Extensive experiments conducted on the Human 3.6M and CMU Mocap datasets demonstrate that our model achieves state-of-the-art performance in both short-term and long-term predictions,outperforming existing methods by up to 20.3% in accuracy.Furthermore,ablation studies confirm the significant contributions of the CSM and DTM in enhancing prediction accuracy.展开更多
Microphone array-based sound source localization(SSL)is a challenging task in adverse acoustic scenarios.To address this,a novel SSL algorithm based on deep neural network(DNN)using steered response power-phase transf...Microphone array-based sound source localization(SSL)is a challenging task in adverse acoustic scenarios.To address this,a novel SSL algorithm based on deep neural network(DNN)using steered response power-phase transform(SRP-PHAT)spatial spectrum as input feature is presented in this paper.Since the SRP-PHAT spatial power spectrum contains spatial location information,it is adopted as the input feature for sound source localization.DNN is exploited to extract the efficient location information from SRP-PHAT spatial power spectrum due to its advantage on extracting high-level features.SRP-PHAT at each steering position within a frame is arranged into a vector,which is treated as DNN input.A DNN model which can map the SRP-PHAT spatial spectrum to the azimuth of sound source is learned from the training signals.The azimuth of sound source is estimated through trained DNN model from the testing signals.Experiment results demonstrate that the proposed algorithm significantly improves localization performance whether the training and testing condition setup are the same or not,and is more robust to noise and reverberation.展开更多
基金the Foshan Science and technology Innovation Team Project(No.FS0AA-KJ919-4402-0060)the National Natural Science Foundation of China(No.62263018)。
文摘In view of the weak ability of the convolutional neural networks to explicitly learn spatial invariance and the probabilistic loss of discriminative features caused by occlusion and background interference in pedestrian re-identification tasks,a person re-identification method combining spatial feature learning and multi-granularity feature fusion was proposed.First,an attention spatial transformation network(A-STN)is proposed to learn spatial features and solve the problem of misalignment of pedestrian spatial features.Then the network was divided into a global branch,a local coarse-grained fusion branch,and a local fine-grained fusion branch to extract pedestrian global features,coarse-grained fusion features,and fine-grained fusion features,respectively.Among them,the global branch enriches the global features by fusing different pooling features.The local coarse-grained fusion branch uses an overlay pooling to enhance each local feature while learning the correlation relationship between multi-granularity features.The local fine-grained fusion branch uses a differential pooling to obtain the differential features that were fused with global features to learn the relationship between pedestrian local features and pedestrian global features.Finally,the proposed method was compared on three public datasets:Market1501,DukeMTMC-ReID and CUHK03.The experimental results were better than those of the comparative methods,which verifies the effectiveness of the proposed method.
基金supported by the Urgent Need for Overseas Talent Project of Jiangxi Province(Grant No.20223BCJ25040)the Thousand Talents Plan of Jiangxi Province(Grant No.jxsg2023101085)+3 种基金the National Natural Science Foundation of China(Grant No.62106093)the Natural Science Foundation of Jiangxi(Grant Nos.20224BAB212011,20232BAB212008,20242BAB25078,and 20232BAB202051)The Youth Talent Cultivation Innovation Fund Project of Nanchang University(Grant No.XX202506030015)funded by Princess Nourah bint Abdulrahman University Researchers Supporting Project number(PNURSP2025R759),Princess Nourah bint Abdulrahman University,Riyadh,Saudi Arabia.
文摘Predicting human motion based on historical motion sequences is a fundamental problem in computer vision,which is at the core of many applications.Existing approaches primarily focus on encoding spatial dependencies among human joints while ignoring the temporal cues and the complex relationships across non-consecutive frames.These limitations hinder the model’s ability to generate accurate predictions over longer time horizons and in scenarios with complex motion patterns.To address the above problems,we proposed a novel multi-level spatial and temporal learning model,which consists of a Cross Spatial Dependencies Encoding Module(CSM)and a Dynamic Temporal Connection Encoding Module(DTM).Specifically,the CSM is designed to capture complementary local and global spatial dependent information at both the joint level and the joint pair level.We further present DTM to encode diverse temporal evolution contexts and compress motion features to a deep level,enabling the model to capture both short-term and long-term dependencies efficiently.Extensive experiments conducted on the Human 3.6M and CMU Mocap datasets demonstrate that our model achieves state-of-the-art performance in both short-term and long-term predictions,outperforming existing methods by up to 20.3% in accuracy.Furthermore,ablation studies confirm the significant contributions of the CSM and DTM in enhancing prediction accuracy.
基金This work is supported by the National Nature Science Foundation of China(NSFC)under Grant No.61571106Jiangsu Natural Science Foundation under Grant No.BK20170757the Natural Science Foundation of the Jiangsu Higher Education Institutions of China under grant No.17KJD510002.
文摘Microphone array-based sound source localization(SSL)is a challenging task in adverse acoustic scenarios.To address this,a novel SSL algorithm based on deep neural network(DNN)using steered response power-phase transform(SRP-PHAT)spatial spectrum as input feature is presented in this paper.Since the SRP-PHAT spatial power spectrum contains spatial location information,it is adopted as the input feature for sound source localization.DNN is exploited to extract the efficient location information from SRP-PHAT spatial power spectrum due to its advantage on extracting high-level features.SRP-PHAT at each steering position within a frame is arranged into a vector,which is treated as DNN input.A DNN model which can map the SRP-PHAT spatial spectrum to the azimuth of sound source is learned from the training signals.The azimuth of sound source is estimated through trained DNN model from the testing signals.Experiment results demonstrate that the proposed algorithm significantly improves localization performance whether the training and testing condition setup are the same or not,and is more robust to noise and reverberation.
