Learning from demonstration is widely regarded as a promising paradigm for robots to acquire diverse skills.Other than the artificial learning from observation-action pairs for machines,humans can learn to imitate in ...Learning from demonstration is widely regarded as a promising paradigm for robots to acquire diverse skills.Other than the artificial learning from observation-action pairs for machines,humans can learn to imitate in a more versatile and effective manner:acquiring skills through mere“observation”.Video to Command task is widely perceived as a promising approach for task-based learning,which yet faces two key challenges:(1)High redundancy and low frame rate of fine-grained action sequences make it difficult to manipulate objects robustly and accurately.(2)Video to Command models often prioritize accuracy and richness of output commands over physical capabilities,leading to impractical or unsafe instructions for robots.This article presents a novel Video to Command framework that employs multiple data associations and physical constraints.First,we introduce an object-level appearancecontrasting multiple data association strategy to effectively associate manipulated objects in visually complex environments,capturing dynamic changes in video content.Then,we propose a multi-task Video to Command model that utilizes object-level video content changes to compile expert demonstrations into manipulation commands.Finally,a multi-task hybrid loss function is proposed to train a Video to Command model that adheres to the constraints of the physical world and manipulation tasks.Our method achieved over 10%on BLEU_N,METEOR,ROUGE_L,and CIDEr compared to the up-to-date methods.The dual-arm robot prototype was established to demonstrate the whole process of learning from an expert demonstration of multiple skills and then executing the tasks by a robot.展开更多
Indoor multi-tracking is more challenging compared with outdoor tasks due to frequent occlusion, view-truncation, severe scale change and pose variation, which may bring considerable unreliability and ambiguity to tar...Indoor multi-tracking is more challenging compared with outdoor tasks due to frequent occlusion, view-truncation, severe scale change and pose variation, which may bring considerable unreliability and ambiguity to target representation and data association. So discriminative and reliable target representation is vital for accurate data association in multi-tracking. Pervious works always combine bunch of features to increase the discriminative power, but this is prone to error accumulation and unnecessary computational cost, which may increase ambiguity on the contrary. Moreover, reliability of a same feature in different scenes may vary a lot, especially for currently widespread network cameras, which are settled in various and complex indoor scenes, previous fixed feature selection schemes cannot meet general requirements. To properly handle these problems, first, we propose a scene-adaptive hierarchical data association scheme, which adaptively selects features with higher reliability on target representation in the applied scene, and gradually combines features to the minimum requirement of discriminating ambiguous targets; second, a novel depth-invariant part-based appearance model using RGB-D data is proposed which makes the appearance model robust to scale change, partial occlusion and view-truncation. The introduce of RGB-D data increases the diversity of features, which provides more types of features for feature selection in data association and enhances the final multi-tracking performance. We validate our method from several aspects including scene-adaptive feature selection scheme, hierarchical data association scheme and RGB-D based appearance modeling scheme in various indoor scenes, which demonstrates its effectiveness and efficiency on improving multi-tracking performances in various indoor scenes.展开更多
The purpose of this research is to improve the robustness of the autonomous system in order to improve the position and velocity estimation of an Unmanned Aerial Vehicle(UAV).Therefore, new integrated SINS/GPS navigat...The purpose of this research is to improve the robustness of the autonomous system in order to improve the position and velocity estimation of an Unmanned Aerial Vehicle(UAV).Therefore, new integrated SINS/GPS navigation scheme based on Interacting Multiple Nonlinear Fuzzy Adaptive H_∞ Models(IMM-NFAH_∞) filtering technique for UAV is presented. The proposed IMM-NFAH_∞ strategy switches between two different Nonlinear Fuzzy Adaptive H_∞(NFAH_∞) filters and each NFAH_∞ filter is based on different fuzzy logic inference systems. The newly proposed technique takes into consideration the high order Taylor series terms and adapts the nonlinear H_∞ filter based on different fuzzy inference systems via adaptive filter bounds(di),along with disturbance attenuation parameter c. Simulation analysis validates the performance of the proposed algorithm, and the comparison with nonlinear H_∞(NH_∞) filter and that with different NFAH_∞ filters demonstrate the effectiveness of UAV localization utilizing IMM-NFAH_∞ filter.展开更多
To solve the problem of strong nonlinear and motion model switching of maneuvering target tracking system in clutter environment, a novel maneuvering multi-target tracking algorithm based on multiple model particle fi...To solve the problem of strong nonlinear and motion model switching of maneuvering target tracking system in clutter environment, a novel maneuvering multi-target tracking algorithm based on multiple model particle filter is presented in this paper. The algorithm realizes dynamic combination of multiple model particle filter and joint probabilistic data association algorithm. The rapid expan- sion of computational complexity, caused by the simple combination of the interacting multiple model algorithm and particle filter is solved by introducing model information into the sampling process of particle state, and the effective validation and utilization of echo is accomplished by the joint proba- bilistic data association algorithm. The concrete steps of the algorithm are given, and the theory analysis and simulation results show the validity of the method.展开更多
In the technique of video multi-target tracking,the common particle filter can not deal well with uncertain relations among multiple targets.To solve this problem,many researchers use data association method to reduce...In the technique of video multi-target tracking,the common particle filter can not deal well with uncertain relations among multiple targets.To solve this problem,many researchers use data association method to reduce the multi-target uncertainty.However,the traditional data association method is difficult to track accurately when the target is occluded.To remove the occlusion in the video,combined with the theory of data association,this paper adopts the probabilistic graphical model for multi-target modeling and analysis of the targets relationship in the particle filter framework.Ex-perimental results show that the proposed algorithm can solve the occlusion problem better compared with the traditional algorithm.展开更多
The state estimation of a maneuvering target,of which the trajectory shape is independent on dynamic characteristics,is studied.The conventional motion models in Cartesian coordinates imply that the trajectory of a ta...The state estimation of a maneuvering target,of which the trajectory shape is independent on dynamic characteristics,is studied.The conventional motion models in Cartesian coordinates imply that the trajectory of a target is completely determined by its dynamic characteristics.However,this is not true in the applications of road-target,sea-route-target or flight route-target tracking,where target trajectory shape is uncoupled with target velocity properties.In this paper,a new estimation algorithm based on separate modeling of target trajectory shape and dynamic characteristics is proposed.The trajectory of a target over a sliding window is described by a linear function of the arc length.To determine the unknown target trajectory,an augmented system is derived by denoting the unknown coefficients of the function as states in mileage coordinates.At every estimation cycle except the first one,the interaction(mixing)stage of the proposed algorithm starts from the latest estimated base state and a recalculated parameter vector,which is determined by the least squares(LS).Numerical experiments are conducted to assess the performance of the proposed algorithm.Simulation results show that the proposed algorithm can achieve better performance than the conventional coupled model-based algorithms in the presence of target maneuvers.展开更多
Recently,lots of smoothing techniques have been presented for maneuvering target tracking.Interacting multiple model-probabilistic data association(IMM-PDA)fixed-lag smoothing algorithm provides an efficient solution ...Recently,lots of smoothing techniques have been presented for maneuvering target tracking.Interacting multiple model-probabilistic data association(IMM-PDA)fixed-lag smoothing algorithm provides an efficient solution to track a maneuvering target in a cluttered environment.Whereas,the smoothing lag of each model in a model set is a fixed constant in traditional algorithms.A new approach is developed in this paper.Although this method is still based on IMM-PDA approach to a state augmented system,it adopts different smoothing lag according to diverse degrees of complexity of each model.As a result,the application is more flexible and the computational load is reduced greatly.Some simulations were conducted to track a highly maneuvering target in a cluttered environment using two sensors.The results illustrate the superiority of the proposed algorithm over comparative schemes,both in accuracy of track estimation and the computational load.展开更多
基金Supported by Zhejiang Provincial Key Research and Development Program(Grant No.2021C04015)。
文摘Learning from demonstration is widely regarded as a promising paradigm for robots to acquire diverse skills.Other than the artificial learning from observation-action pairs for machines,humans can learn to imitate in a more versatile and effective manner:acquiring skills through mere“observation”.Video to Command task is widely perceived as a promising approach for task-based learning,which yet faces two key challenges:(1)High redundancy and low frame rate of fine-grained action sequences make it difficult to manipulate objects robustly and accurately.(2)Video to Command models often prioritize accuracy and richness of output commands over physical capabilities,leading to impractical or unsafe instructions for robots.This article presents a novel Video to Command framework that employs multiple data associations and physical constraints.First,we introduce an object-level appearancecontrasting multiple data association strategy to effectively associate manipulated objects in visually complex environments,capturing dynamic changes in video content.Then,we propose a multi-task Video to Command model that utilizes object-level video content changes to compile expert demonstrations into manipulation commands.Finally,a multi-task hybrid loss function is proposed to train a Video to Command model that adheres to the constraints of the physical world and manipulation tasks.Our method achieved over 10%on BLEU_N,METEOR,ROUGE_L,and CIDEr compared to the up-to-date methods.The dual-arm robot prototype was established to demonstrate the whole process of learning from an expert demonstration of multiple skills and then executing the tasks by a robot.
基金This work is supported by National Natural Science Foundation of China （NSFC, No. 61340046）, National High Technology Research and Development Program of China （863 Program, No. 2006AA04Z247）, Scientific and Technical Innovation Commission of Shenzhen Municipality （JCYJ20130331144631730, JCYJ20130331144716089）, Specialized Research Fund for the Doctoral Program of Higher Education （No. 20130001110011）.
文摘Indoor multi-tracking is more challenging compared with outdoor tasks due to frequent occlusion, view-truncation, severe scale change and pose variation, which may bring considerable unreliability and ambiguity to target representation and data association. So discriminative and reliable target representation is vital for accurate data association in multi-tracking. Pervious works always combine bunch of features to increase the discriminative power, but this is prone to error accumulation and unnecessary computational cost, which may increase ambiguity on the contrary. Moreover, reliability of a same feature in different scenes may vary a lot, especially for currently widespread network cameras, which are settled in various and complex indoor scenes, previous fixed feature selection schemes cannot meet general requirements. To properly handle these problems, first, we propose a scene-adaptive hierarchical data association scheme, which adaptively selects features with higher reliability on target representation in the applied scene, and gradually combines features to the minimum requirement of discriminating ambiguous targets; second, a novel depth-invariant part-based appearance model using RGB-D data is proposed which makes the appearance model robust to scale change, partial occlusion and view-truncation. The introduce of RGB-D data increases the diversity of features, which provides more types of features for feature selection in data association and enhances the final multi-tracking performance. We validate our method from several aspects including scene-adaptive feature selection scheme, hierarchical data association scheme and RGB-D based appearance modeling scheme in various indoor scenes, which demonstrates its effectiveness and efficiency on improving multi-tracking performances in various indoor scenes.
基金supported by a grant from the National Natural Science Foundation of China(No.61375082)
文摘The purpose of this research is to improve the robustness of the autonomous system in order to improve the position and velocity estimation of an Unmanned Aerial Vehicle(UAV).Therefore, new integrated SINS/GPS navigation scheme based on Interacting Multiple Nonlinear Fuzzy Adaptive H_∞ Models(IMM-NFAH_∞) filtering technique for UAV is presented. The proposed IMM-NFAH_∞ strategy switches between two different Nonlinear Fuzzy Adaptive H_∞(NFAH_∞) filters and each NFAH_∞ filter is based on different fuzzy logic inference systems. The newly proposed technique takes into consideration the high order Taylor series terms and adapts the nonlinear H_∞ filter based on different fuzzy inference systems via adaptive filter bounds(di),along with disturbance attenuation parameter c. Simulation analysis validates the performance of the proposed algorithm, and the comparison with nonlinear H_∞(NH_∞) filter and that with different NFAH_∞ filters demonstrate the effectiveness of UAV localization utilizing IMM-NFAH_∞ filter.
基金Supported by the National Natural Science Foundation of China （60634030）, the National Natural Science Foundation of China （60702066, 6097219） and the Natural Science Foundation of Henan Province （092300410158）.
文摘To solve the problem of strong nonlinear and motion model switching of maneuvering target tracking system in clutter environment, a novel maneuvering multi-target tracking algorithm based on multiple model particle filter is presented in this paper. The algorithm realizes dynamic combination of multiple model particle filter and joint probabilistic data association algorithm. The rapid expan- sion of computational complexity, caused by the simple combination of the interacting multiple model algorithm and particle filter is solved by introducing model information into the sampling process of particle state, and the effective validation and utilization of echo is accomplished by the joint proba- bilistic data association algorithm. The concrete steps of the algorithm are given, and the theory analysis and simulation results show the validity of the method.
基金Supported by the National High Technology Research and Development Program of China(No.2007AA11Z227)the Natural Science Foundation of Jiangsu Province of China(No.BK2009352)the Fundamental Research Funds for the Central Universities of China(No.2010B16414)
文摘In the technique of video multi-target tracking,the common particle filter can not deal well with uncertain relations among multiple targets.To solve this problem,many researchers use data association method to reduce the multi-target uncertainty.However,the traditional data association method is difficult to track accurately when the target is occluded.To remove the occlusion in the video,combined with the theory of data association,this paper adopts the probabilistic graphical model for multi-target modeling and analysis of the targets relationship in the particle filter framework.Ex-perimental results show that the proposed algorithm can solve the occlusion problem better compared with the traditional algorithm.
基金supported by the National Natural Science Foundation of China(61671181).
文摘The state estimation of a maneuvering target,of which the trajectory shape is independent on dynamic characteristics,is studied.The conventional motion models in Cartesian coordinates imply that the trajectory of a target is completely determined by its dynamic characteristics.However,this is not true in the applications of road-target,sea-route-target or flight route-target tracking,where target trajectory shape is uncoupled with target velocity properties.In this paper,a new estimation algorithm based on separate modeling of target trajectory shape and dynamic characteristics is proposed.The trajectory of a target over a sliding window is described by a linear function of the arc length.To determine the unknown target trajectory,an augmented system is derived by denoting the unknown coefficients of the function as states in mileage coordinates.At every estimation cycle except the first one,the interaction(mixing)stage of the proposed algorithm starts from the latest estimated base state and a recalculated parameter vector,which is determined by the least squares(LS).Numerical experiments are conducted to assess the performance of the proposed algorithm.Simulation results show that the proposed algorithm can achieve better performance than the conventional coupled model-based algorithms in the presence of target maneuvers.
基金This work is supported by the Projects of the State Key Fundamental Research(No.2001CB309403)
文摘Recently,lots of smoothing techniques have been presented for maneuvering target tracking.Interacting multiple model-probabilistic data association(IMM-PDA)fixed-lag smoothing algorithm provides an efficient solution to track a maneuvering target in a cluttered environment.Whereas,the smoothing lag of each model in a model set is a fixed constant in traditional algorithms.A new approach is developed in this paper.Although this method is still based on IMM-PDA approach to a state augmented system,it adopts different smoothing lag according to diverse degrees of complexity of each model.As a result,the application is more flexible and the computational load is reduced greatly.Some simulations were conducted to track a highly maneuvering target in a cluttered environment using two sensors.The results illustrate the superiority of the proposed algorithm over comparative schemes,both in accuracy of track estimation and the computational load.
