Real-time Human action classification in complex scenes has applications in various domains such as visual surveillance, video retrieval and human robot interaction. While, the task is challenging due to computation e...Real-time Human action classification in complex scenes has applications in various domains such as visual surveillance, video retrieval and human robot interaction. While, the task is challenging due to computation efficiency, cluttered backgrounds and intro-variability among same type of actions. Spatio-temporal interest point （STIP） based methods have shown promising results to tackle human action classification in complex scenes efficiently. However, the state-of-the-art works typically utilize bag-of-visual words （BoVW） model which only focuses on the word distribution of STIPs and ignore the distinctive character of word structure. In this paper, the distribution of STIPs is organized into a salient directed graph, which reflects salient motions and can be divided into a time salient directed graph and a space salient directed graph, aiming at adding spatio-temporal discriminant to BoVW. Generally speaking, both salient directed graphs are constructed by labeled STIPs in pairs. In detail, the ＂directional co-occurrence＂ property of different labeled pairwise STIPs in same frame is utilized to represent the time saliency, and the space saliency is reflected by the ＂geometric relationships＂ between same labeled pairwise STIPs across different frames. Then, new statistical features namely the Time Salient Pairwise feature （TSP） and the Space Salient Pairwise feature （SSP） are designed to describe two salient directed graphs, respectively. Experiments are carried out with a homogeneous kernel SVM classifier, on four challenging datasets KTH, ADL and UT-Interaction. Final results confirm the complementary of TSP and SSP, and our multi-cue representation TSP ＋ SSP ＋ BoVW can properly describe human actions with large intro-variability in real-time.展开更多
For vision-based mobile robot navigation, images of the same scene may undergo a general affine transformation in the case of significant viewpoint changes. So, a novel method for detecting affine invariant interest p...For vision-based mobile robot navigation, images of the same scene may undergo a general affine transformation in the case of significant viewpoint changes. So, a novel method for detecting affine invariant interest points is proposed to obtain the invariant local features, which is coined polynomial local orientation tensor(PLOT). The new detector is based on image local orientation tensor that is constructed from the polynomial expansion of image signal. Firstly, the properties of local orientation tensor of PLOT are analyzed, and a suitable tuning parameter of local orientation tensor is chosen so as to extract invariant features. The initial interest points are detected by local maxima search for the smaller eigenvalues of the orientation tensor. Then, an iterative procedure is used to allow the initial interest points to converge to affine invariant interest points and regions. The performances of this detector are evaluated on the repeatability criteria and recall versus 1-precision graphs, and then are compared with other existing approaches. Experimental results for PLOT show strong performance under affine transformation in the real-world conditions.展开更多
Convolutional Neural Networks(CNNs)have been successfully adopted by state-of-the-art feature point detection and description networks for the past number of years.The focus of these systems has been predominately on ...Convolutional Neural Networks(CNNs)have been successfully adopted by state-of-the-art feature point detection and description networks for the past number of years.The focus of these systems has been predominately on the accuracy of the system,rather than on its efficiency or ability to be implemented in real-time on embedded robotic devices.This paper demonstrates how techniques,developed for other CNN use cases,can be integrated into interest point detection and description systems to compress their network size and reduce the computational complexity;this reduces the barrier to their uptake in computationally challenged environments.This paper documents the integration of these techniques into the popular Reliable Detector and Descriptor(R2D2)network.Along with the integration details,a comprehensive Key Performance Indicator(KPI)framework is developed to test all aspects of the networks.As a result,this paper presents a lightweight variant of the R2D2 network that significantly reduces parameters and computational complexity while crucially maintaining an acceptable level of accuracy.Consequently,this new compressed network is more appropriate for use in real world systems and advances the efforts to implement such CNN based system for mobile devices.展开更多
By mining of the requirements of lots of electric vehicle users for charging piles, this paper proposes the charging pile siting algorithm via the fusion of Points of Interest and vehicle trajectories. The proposed al...By mining of the requirements of lots of electric vehicle users for charging piles, this paper proposes the charging pile siting algorithm via the fusion of Points of Interest and vehicle trajectories. The proposed algorithm computes appropriate charging pile locations by: 1) mining user Points of Interest from social network; 2) mining parking sites of vehicle form GPS trajectories and 3) fusing the Points of Interest and parking sites together then clustering the fusions with our improved DBSCAN algorithm, whose clustering results indicates the final appropriate charging pile locations. Experimental results show that our proposed methods are more efficient than existing methods.展开更多
传统兴趣点(point of interest,POI)推荐方法对用户和POI的关联关系挖掘不充分,无法全面捕捉用户偏好;基于图增强的推荐方法虽能挖掘关联关系,却易引入噪声,降低推荐性能。针对这些问题,本文提出了结合通用轨迹图和多偏好的POI推荐方法...传统兴趣点(point of interest,POI)推荐方法对用户和POI的关联关系挖掘不充分,无法全面捕捉用户偏好;基于图增强的推荐方法虽能挖掘关联关系,却易引入噪声,降低推荐性能。针对这些问题,本文提出了结合通用轨迹图和多偏好的POI推荐方法。首先构建了用户与POI的带权二部图,利用图卷积网络捕捉用户和POI的交互关系,学习用户兴趣偏好;利用兴趣偏好完成用户聚类,进而构建同类型用户通用轨迹图,减少噪声信息影响;利用图卷积网络捕捉同类型用户的群体特征,丰富特征表示。其次,将群体特征与用户当前轨迹中时间类别感知信息、时空上下文信息相结合,利用Transformer挖掘用户的深层行为偏好。再次,构造非线性加性函数并将兴趣偏好和行为偏好动态组合,全面捕捉用户偏好,完成POI推荐。最后,在真实数据集上验证了本文方法的有效性。展开更多
基金This work is supported by the National Natural Science Foundation of China （NSFC, nos. 61340046）, the National High Technology Research and Development Programme of China （863 Programme, no. 2006AA04Z247）, the Scientific and Technical Innovation Commission of Shenzhen Munici-pality （nos. JCYJ20130331144631730）, and the Specialized Research Fund for the Doctoral Programme of Higher Edu- cation （SRFDP, no. 20130001110011）.
文摘Real-time Human action classification in complex scenes has applications in various domains such as visual surveillance, video retrieval and human robot interaction. While, the task is challenging due to computation efficiency, cluttered backgrounds and intro-variability among same type of actions. Spatio-temporal interest point （STIP） based methods have shown promising results to tackle human action classification in complex scenes efficiently. However, the state-of-the-art works typically utilize bag-of-visual words （BoVW） model which only focuses on the word distribution of STIPs and ignore the distinctive character of word structure. In this paper, the distribution of STIPs is organized into a salient directed graph, which reflects salient motions and can be divided into a time salient directed graph and a space salient directed graph, aiming at adding spatio-temporal discriminant to BoVW. Generally speaking, both salient directed graphs are constructed by labeled STIPs in pairs. In detail, the ＂directional co-occurrence＂ property of different labeled pairwise STIPs in same frame is utilized to represent the time saliency, and the space saliency is reflected by the ＂geometric relationships＂ between same labeled pairwise STIPs across different frames. Then, new statistical features namely the Time Salient Pairwise feature （TSP） and the Space Salient Pairwise feature （SSP） are designed to describe two salient directed graphs, respectively. Experiments are carried out with a homogeneous kernel SVM classifier, on four challenging datasets KTH, ADL and UT-Interaction. Final results confirm the complementary of TSP and SSP, and our multi-cue representation TSP ＋ SSP ＋ BoVW can properly describe human actions with large intro-variability in real-time.
基金Projects(61203332,61203208) supported by the National Natural Science Foundation of China
文摘For vision-based mobile robot navigation, images of the same scene may undergo a general affine transformation in the case of significant viewpoint changes. So, a novel method for detecting affine invariant interest points is proposed to obtain the invariant local features, which is coined polynomial local orientation tensor(PLOT). The new detector is based on image local orientation tensor that is constructed from the polynomial expansion of image signal. Firstly, the properties of local orientation tensor of PLOT are analyzed, and a suitable tuning parameter of local orientation tensor is chosen so as to extract invariant features. The initial interest points are detected by local maxima search for the smaller eigenvalues of the orientation tensor. Then, an iterative procedure is used to allow the initial interest points to converge to affine invariant interest points and regions. The performances of this detector are evaluated on the repeatability criteria and recall versus 1-precision graphs, and then are compared with other existing approaches. Experimental results for PLOT show strong performance under affine transformation in the real-world conditions.
基金Supported by National Natural Science Foundation of China（61105089） State Key Laboratory of Robotics and System（SKLRS-2013-ZD-03） Open Foundation of the State Key Laboratory of Fluid Power Transmission and Control（GZKF-201212）
文摘Convolutional Neural Networks(CNNs)have been successfully adopted by state-of-the-art feature point detection and description networks for the past number of years.The focus of these systems has been predominately on the accuracy of the system,rather than on its efficiency or ability to be implemented in real-time on embedded robotic devices.This paper demonstrates how techniques,developed for other CNN use cases,can be integrated into interest point detection and description systems to compress their network size and reduce the computational complexity;this reduces the barrier to their uptake in computationally challenged environments.This paper documents the integration of these techniques into the popular Reliable Detector and Descriptor(R2D2)network.Along with the integration details,a comprehensive Key Performance Indicator(KPI)framework is developed to test all aspects of the networks.As a result,this paper presents a lightweight variant of the R2D2 network that significantly reduces parameters and computational complexity while crucially maintaining an acceptable level of accuracy.Consequently,this new compressed network is more appropriate for use in real world systems and advances the efforts to implement such CNN based system for mobile devices.
文摘By mining of the requirements of lots of electric vehicle users for charging piles, this paper proposes the charging pile siting algorithm via the fusion of Points of Interest and vehicle trajectories. The proposed algorithm computes appropriate charging pile locations by: 1) mining user Points of Interest from social network; 2) mining parking sites of vehicle form GPS trajectories and 3) fusing the Points of Interest and parking sites together then clustering the fusions with our improved DBSCAN algorithm, whose clustering results indicates the final appropriate charging pile locations. Experimental results show that our proposed methods are more efficient than existing methods.
文摘传统兴趣点(point of interest,POI)推荐方法对用户和POI的关联关系挖掘不充分,无法全面捕捉用户偏好;基于图增强的推荐方法虽能挖掘关联关系,却易引入噪声,降低推荐性能。针对这些问题,本文提出了结合通用轨迹图和多偏好的POI推荐方法。首先构建了用户与POI的带权二部图,利用图卷积网络捕捉用户和POI的交互关系,学习用户兴趣偏好;利用兴趣偏好完成用户聚类,进而构建同类型用户通用轨迹图,减少噪声信息影响;利用图卷积网络捕捉同类型用户的群体特征,丰富特征表示。其次,将群体特征与用户当前轨迹中时间类别感知信息、时空上下文信息相结合,利用Transformer挖掘用户的深层行为偏好。再次,构造非线性加性函数并将兴趣偏好和行为偏好动态组合,全面捕捉用户偏好,完成POI推荐。最后,在真实数据集上验证了本文方法的有效性。
