To improve the safety of construction workers and help workers remotely control humanoid robots in construc-tion,this study designs and implements a computer vision based virtual construction simulation system.For thi...To improve the safety of construction workers and help workers remotely control humanoid robots in construc-tion,this study designs and implements a computer vision based virtual construction simulation system.For this pur-pose,human skeleton motion data are collected using a Ki-nect depth camera,and the obtained data are optimized via abnormal data elimination,smoothing,and normalization.MediaPipe extracts three-dimensional hand motion coordi-nates for accurate human posture tracking.Blender is used to build a virtual worker and site model,and the virtual worker motion is controlled based on the quaternion inverse kinematics algorithm while limiting the joint angle to en-hance the authenticity of motion simulation.Experimental results show that the system frame rate is stable at 60 frame/s,end-to-end delay is less than 20 ms,and virtual task comple-tion time is close to the real scene,verifying its engineering applicability.The proposed system can drive virtual work-ers to perform tasks and provide technical support for con-struction safety training.展开更多
Visual object tracking(VOT),aiming to track a target object in a continuous video,is a fundamental and critical task in computer vision.However,the reliance on third-party resources(e.g.,dataset)for training poses con...Visual object tracking(VOT),aiming to track a target object in a continuous video,is a fundamental and critical task in computer vision.However,the reliance on third-party resources(e.g.,dataset)for training poses concealed threats to the security of VOT models.In this paper,we reveal that VOT models are vulnerable to a poison-only and targeted backdoor attack,where the adversary can achieve arbitrary tracking predictions by manipulating only part of the training data.Specifically,we first define and formulate three different variants of the targeted attacks:size-manipulation,trajectory-manipulation,and hybrid attacks.To implement these,we introduce Random Video Poisoning(RVP),a novel poison-only strategy that exploits temporal correlations within video data by poisoning entire video sequences.Extensive experiments demonstrate that RVP effectively injects controllable backdoors,enabling precise manipulation of tracking behavior upon trigger activation,while maintaining high performance on benign data,thus ensuring stealth.Our findings not only expose significant vulnerabilities but also highlight that the underlying principles could be adapted for beneficial uses,such as dataset watermarking for copyright protection.展开更多
This paper proposes the analysis model of sports human body based on computer vision tracking technology. Visual target tracking is an important research field of the computer vision, motion trajectory and it can prov...This paper proposes the analysis model of sports human body based on computer vision tracking technology. Visual target tracking is an important research field of the computer vision, motion trajectory and it can provide not only the goal, to provide the initial data movement analysis, scene understanding, behavior or the event detection in intelligent surveillance, human-computer interaction, robot visual navigation and motion recognition based on field has a broad application prospect. For this reason, it is possible to consider the use of a large number of unlabeled samples to assist the training classifier to improve its performance. This type of machine learning method using both labeled and that unlabeled samples is called the semi-supervised learning. This paper proposes the novel idea of the related research topics to propose the new perspective of the model that will be later give us the novel idea of making it efficient for further development of sport science.展开更多
To improve small object detection and trajectory estimation from an aerial moving perspective,we propose the Aerial View Attention-PRB(AVA-PRB)model.AVA-PRB integrates two attention mechanisms—Coordinate Attention(CA...To improve small object detection and trajectory estimation from an aerial moving perspective,we propose the Aerial View Attention-PRB(AVA-PRB)model.AVA-PRB integrates two attention mechanisms—Coordinate Attention(CA)and the Convolutional Block Attention Module(CBAM)—to enhance detection accuracy.Additionally,Shape-IoU is employed as the loss function to refine localization precision.Our model further incorporates an adaptive feature fusion mechanism,which optimizes multi-scale object representation,ensuring robust tracking in complex aerial environments.We evaluate the performance of AVA-PRB on two benchmark datasets:Aerial Person Detection and VisDrone2019-Det.The model achieves 60.9%mAP@0.5 on the Aerial Person Detection dataset,and 51.2%mAP@0.5 on VisDrone2019-Det,demonstrating its effectiveness in aerial object detection.Beyond detection,we propose a novel trajectory estimation method that improves movement path prediction under aerial motion.Experimental results indicate that our approach reduces path deviation by up to 64%,effectively mitigating errors caused by rapid camera movements and background variations.By optimizing feature extraction and enhancing spatialtemporal coherence,our method significantly improves object tracking under aerial moving perspectives.This research addresses the limitations of fixed-camera tracking,enhancing flexibility and accuracy in aerial tracking applications.The proposed approach has broad potential for real-world applications,including surveillance,traffic monitoring,and environmental observation.展开更多
Advancements in animal behavior quantification methods have driven the development of computational ethology,enabling fully automated behavior analysis.Existing multianimal pose estimation workflows rely on tracking-b...Advancements in animal behavior quantification methods have driven the development of computational ethology,enabling fully automated behavior analysis.Existing multianimal pose estimation workflows rely on tracking-bydetection frameworks for either bottom-up or top-down approaches,requiring retraining to accommodate diverse animal appearances.This study introduces InteBOMB,an integrated workflow that enhances top-down approaches by incorporating generic object tracking,eliminating the need for prior knowledge of target animals while maintaining broad generalizability.InteBOMB includes two key strategies for tracking and segmentation in laboratory environments and two techniques for pose estimation in natural settings.The“background enhancement”strategy optimizesforeground-backgroundcontrastiveloss,generating more discriminative correlation maps.The“online proofreading”strategy stores human-in-the-loop long-term memory and dynamic short-term memory,enabling adaptive updates to object visual features.The“automated labeling suggestion”technique reuses the visual features saved during tracking to identify representative frames for training set labeling.Additionally,the“joint behavior analysis”technique integrates these features with multimodal data,expanding the latent space for behavior classification and clustering.To evaluate the framework,six datasets of mice and six datasets of nonhuman primates were compiled,covering laboratory and natural scenes.Benchmarking results demonstrated a24%improvement in zero-shot generic tracking and a 21%enhancement in joint latent space performance across datasets,highlighting the effectiveness of this approach in robust,generalizable behavior analysis.展开更多
An improved estimation of motion vectors of feature points is proposed for tracking moving objects of dynamic image sequence. Feature points are firstly extracted by the improved minimum intensity change （MIC） algor...An improved estimation of motion vectors of feature points is proposed for tracking moving objects of dynamic image sequence. Feature points are firstly extracted by the improved minimum intensity change （MIC） algorithm. The matching points of these feature points are then determined by adaptive rood pattern searching. Based on the random sample consensus （RANSAC） method, the background motion is finally compensated by the parameters of an affine transform of the background motion. With reasonable morphological filtering, the moving objects are completely extracted from the background, and then tracked accurately. Experimental results show that the improved method is successful on the motion background compensation and offers great promise in tracking moving objects of the dynamic image sequence.展开更多
Improved picture quality is critical to the effectiveness of object recog-nition and tracking.The consistency of those photos is impacted by night-video systems because the contrast between high-profile items and diffe...Improved picture quality is critical to the effectiveness of object recog-nition and tracking.The consistency of those photos is impacted by night-video systems because the contrast between high-profile items and different atmospheric conditions,such as mist,fog,dust etc.The pictures then shift in intensity,colour,polarity and consistency.A general challenge for computer vision analyses lies in the horrid appearance of night images in arbitrary illumination and ambient envir-onments.In recent years,target recognition techniques focused on deep learning and machine learning have become standard algorithms for object detection with the exponential growth of computer performance capabilities.However,the iden-tification of objects in the night world also poses further problems because of the distorted backdrop and dim light.The Correlation aware LSTM based YOLO(You Look Only Once)classifier method for exact object recognition and deter-mining its properties under night vision was a major inspiration for this work.In order to create virtual target sets similar to daily environments,we employ night images as inputs;and to obtain high enhanced image using histogram based enhancement and iterative wienerfilter for removing the noise in the image.The process of the feature extraction and feature selection was done for electing the potential features using the Adaptive internal linear embedding(AILE)and uplift linear discriminant analysis(ULDA).The region of interest mask can be segmen-ted using the Recurrent-Phase Level set Segmentation.Finally,we use deep con-volution feature fusion and region of interest pooling to integrate the presently extremely sophisticated quicker Long short term memory based(LSTM)with YOLO method for object tracking system.A range of experimentalfindings demonstrate that our technique achieves high average accuracy with a precision of 99.7%for object detection of SSAN datasets that is considerably more than that of the other standard object detection mechanism.Our approach may therefore satisfy the true demands of night scene target detection applications.We very much believe that our method will help future research.展开更多
There are two main trends in the development of unmanned aerial vehicle(UAV)technologies:miniaturization and intellectualization,in which realizing object tracking capabilities for a nano-scale UAV is one of the most ...There are two main trends in the development of unmanned aerial vehicle(UAV)technologies:miniaturization and intellectualization,in which realizing object tracking capabilities for a nano-scale UAV is one of the most challenging problems.In this paper,we present a visual object tracking and servoing control system utilizing a tailor-made 38 g nano-scale quadrotor.A lightweight visual module is integrated to enable object tracking capabilities,and a micro positioning deck is mounted to provide accurate pose estimation.In order to be robust against object appearance variations,a novel object tracking algorithm,denoted by RMCTer,is proposed,which integrates a powerful short-term tracking module and an efficient long-term processing module.In particular,the long-term processing module can provide additional object information and modify the short-term tracking model in a timely manner.Furthermore,a positionbased visual servoing control method is proposed for the quadrotor,where an adaptive tracking controller is designed by leveraging backstepping and adaptive techniques.Stable and accurate object tracking is achieved even under disturbances.Experimental results are presented to demonstrate the high accuracy and stability of the whole tracking system.展开更多
On accomplishing an efficacious object tracking,the activity of an object concerned becomes notified in a forthright manner.An accurate form of object tracking task necessitates a robust object tracking procedures irr...On accomplishing an efficacious object tracking,the activity of an object concerned becomes notified in a forthright manner.An accurate form of object tracking task necessitates a robust object tracking procedures irrespective of hardware assistance.Such approaches inferred a vast computational complexity to track an object with high accuracy in a stipulated amount of processing time.On the other hand,the tracking gets affected owing to the existence of varied quality diminishing factors such as occlusion,illumination changes,shadows etc.,In order to rectify all these inadequacies in tracking an object,a novel background normalization procedure articulated on the basis of a textural pattern is proposed in this paper.After preprocessing an acquired image,employment of an Environmental Succession Prediction algorithm for discriminating disparate background environment by background clustering approach have been accomplished.Afterward,abstract textural characterizations through utilization of a Probability based Gradient Pattern(PGP)approach for recognizing the similarity between patterns obtained so far.Comparison between standardized frame obtained in prior and those processed patterns detects the motion exposed by an object and the object concerned gets identified within a blob.Hence,the system is resistant towards illumination variations.These illumination variation was interpreted in object tracking residing within a dynamic background.Devised approach certainly outperforms other object tracking methodologies like Group Target Tracking(GTT),Vi PER-GT,grabcut,snakes in terms of accuracy and average time.Proposed PGP-based pattern texture analysis is compared with Gamifying Video Object(GVO)approach and hence,it evidently outperforms in terms of precision,recall and F1 measure.展开更多
Significant advancements have beenwitnessed in visual tracking applications leveragingViT in recent years,mainly due to the formidablemodeling capabilities of Vision Transformer(ViT).However,the strong performance of ...Significant advancements have beenwitnessed in visual tracking applications leveragingViT in recent years,mainly due to the formidablemodeling capabilities of Vision Transformer(ViT).However,the strong performance of such trackers heavily relies on ViT models pretrained for long periods,limitingmore flexible model designs for tracking tasks.To address this issue,we propose an efficient unsupervised ViT pretraining method for the tracking task based on masked autoencoders,called TrackMAE.During pretraining,we employ two shared-parameter ViTs,serving as the appearance encoder and motion encoder,respectively.The appearance encoder encodes randomly masked image data,while the motion encoder encodes randomly masked pairs of video frames.Subsequently,an appearance decoder and a motion decoder separately reconstruct the original image data and video frame data at the pixel level.In this way,ViT learns to understand both the appearance of images and the motion between video frames simultaneously.Experimental results demonstrate that ViT-Base and ViT-Large models,pretrained with TrackMAE and combined with a simple tracking head,achieve state-of-the-art(SOTA)performance without additional design.Moreover,compared to the currently popular MAE pretraining methods,TrackMAE consumes only 1/5 of the training time,which will facilitate the customization of diverse models for tracking.For instance,we additionally customize a lightweight ViT-XS,which achieves SOTA efficient tracking performance.展开更多
This paper discusses about the new approach of multiple object track-ing relative to background information.The concept of multiple object tracking through background learning is based upon the theory of relativity,th...This paper discusses about the new approach of multiple object track-ing relative to background information.The concept of multiple object tracking through background learning is based upon the theory of relativity,that involves a frame of reference in spatial domain to localize and/or track any object.Thefield of multiple object tracking has seen a lot of research,but researchers have considered the background as redundant.However,in object tracking,the back-ground plays a vital role and leads to definite improvement in the overall process of tracking.In the present work an algorithm is proposed for the multiple object tracking through background learning.The learning framework is based on graph embedding approach for localizing multiple objects.The graph utilizes the inher-ent capabilities of depth modelling that assist in prior to track occlusion avoidance among multiple objects.The proposed algorithm has been compared with the recent work available in literature on numerous performance evaluation measures.It is observed that our proposed algorithm gives better performance.展开更多
Visual object tracking plays a crucial role in computer vision.In recent years,researchers have proposed various methods to achieve high-performance object tracking.Among these,methods based on Transformers have becom...Visual object tracking plays a crucial role in computer vision.In recent years,researchers have proposed various methods to achieve high-performance object tracking.Among these,methods based on Transformers have become a research hotspot due to their ability to globally model and contextualize information.However,current Transformer-based object tracking methods still face challenges such as low tracking accuracy and the presence of redundant feature information.In this paper,we introduce self-calibration multi-head self-attention Transformer(SMSTracker)as a solution to these challenges.It employs a hybrid tensor decomposition self-organizing multihead self-attention transformermechanism,which not only compresses and accelerates Transformer operations but also significantly reduces redundant data,thereby enhancing the accuracy and efficiency of tracking.Additionally,we introduce a self-calibration attention fusion block to resolve common issues of attention ambiguities and inconsistencies found in traditional trackingmethods,ensuring the stability and reliability of tracking performance across various scenarios.By integrating a hybrid tensor decomposition approach with a self-organizingmulti-head self-attentive transformer mechanism,SMSTracker enhances the efficiency and accuracy of the tracking process.Experimental results show that SMSTracker achieves competitive performance in visual object tracking,promising more robust and efficient tracking systems,demonstrating its potential to providemore robust and efficient tracking solutions in real-world applications.展开更多
The performance and accuracy of computer vision systems are affected by noise in different forms.Although numerous solutions and algorithms have been presented for dealing with every type of noise,a comprehensive tech...The performance and accuracy of computer vision systems are affected by noise in different forms.Although numerous solutions and algorithms have been presented for dealing with every type of noise,a comprehensive technique that can cover all the diverse noises and mitigate their damaging effects on the performance and precision of various systems is still missing.In this paper,we have focused on the stability and robustness of one computer vision branch(i.e.,visual object tracking).We have demonstrated that,without imposing a heavy computational load on a model or changing its algorithms,the drop in the performance and accuracy of a system when it is exposed to an unseen noise-laden test dataset can be prevented by simply applying the style transfer technique on the train dataset and training the model with a combination of these and the original untrained data.To verify our proposed approach,it is applied on a generic object tracker by using regression networks.This method’s validity is confirmed by testing it on an exclusive benchmark comprising 50 image sequences,with each sequence containing 15 types of noise at five different intensity levels.The OPE curves obtained show a 40%increase in the robustness of the proposed object tracker against noise,compared to the other trackers considered.展开更多
Visual object tracking is an important issue that has received long-term attention in computer vision.The ability to effectively handle occlusion,especially severe occlusion,is an important aspect of evaluating the pe...Visual object tracking is an important issue that has received long-term attention in computer vision.The ability to effectively handle occlusion,especially severe occlusion,is an important aspect of evaluating the performance of object tracking algorithms in long-term tracking,and is of great significance to improving the robustness of object tracking algorithms.However,most object tracking algorithms lack a processing mechanism specifically for occlusion.In the case of occlusion,due to the lack of target information,it is necessary to predict the target position based on the motion trajectory.Kalman filtering and particle filtering can effectively predict the target motion state based on the historical motion information.A single object tracking method,called probabilistic discriminative model prediction(PrDiMP),is based on the spatial attention mechanism in complex scenes and occlusions.In order to improve the performance of PrDiMP,Kalman filtering,particle filtering and linear filtering are introduced.First,for the occlusion situation,Kalman filtering and particle filtering are respectively introduced to predict the object position,thereby replacing the detection result of the original tracking algorithm and stopping recursion of target model.Second,for detection-jump problem of similar objects in complex scenes,a linear filtering window is added.The evaluation results on the three datasets,including GOT-10k,UAV123 and LaSOT,and the visualization results on several videos,show that our algorithms have improved tracking performance under occlusion and the detection-jump is effectively suppressed.展开更多
A snake algorithm has been known that it has a strong point in extracting the exact contour of an object. But it is apt to be influenced by scattered edges around the control points. Since the shape of a moving object...A snake algorithm has been known that it has a strong point in extracting the exact contour of an object. But it is apt to be influenced by scattered edges around the control points. Since the shape of a moving object in 2D image changes a lot due to its rotation and translation in the 3D space, the conventional algorithm that takes into account slowly moving objects cannot provide an appropriate solution. To utilize the advantages of the snake algorithm while minimizing the drawbacks, this paper proposes the area variation based color snake algorithm for moving object tracking. The proposed algorithm includes a new energy term which is used for preserving the shape of an object between two consecutive images. The proposed one can also segment precisely interesting objects on complex image since it is based on color information. Experiment results show that the proposed algorithm is very effective in various environments.展开更多
Visual object-tracking is a fundamental task applied in many applications of computer vision. Particle filter is one of the techniques which has been widely used in object tracking. Due to the virtue of extendability ...Visual object-tracking is a fundamental task applied in many applications of computer vision. Particle filter is one of the techniques which has been widely used in object tracking. Due to the virtue of extendability and flexibility on both linear and non-linear environments, various particle filter-based trackers have been proposed in the literature. However, the conventional approach cannot handle very large videos efficiently in the current data intensive information age. In this work, a parallelized particle filter is provided in a distributed framework provided by the Hadoop/Map-Reduce infrastructure to tackle object-tracking tasks. The experiments indicate that the proposed algorithm has a better convergence and accuracy as compared to the traditional particle filter. The computational power and the scalability of the proposed particle filter in single object tracking have been enhanced as well.展开更多
This paper addresses the problem of real-time object tracking for unmanned aerial vehicles. We consider the task of object tracking as a classification problem. Training a good classifier always needs a huge number of...This paper addresses the problem of real-time object tracking for unmanned aerial vehicles. We consider the task of object tracking as a classification problem. Training a good classifier always needs a huge number of samples, which is always time-consuming and not suitable for realtime applications. In this paper, we transform the large-scale least-squares problem in the spatial domain to a series of small-scale least-squares problems with constraints in the Fourier domain using the correlation filter technique. Then, this problem is efficiently solved by two stages. In the first stage, a fast method based on recursive least squares is used to solve the correlation filter problem without constraints in the Fourier domain. In the second stage, a weight matrix is constructed to prune the solution attained in the first stage to approach the constraints in the spatial domain. Then, the pruned classifier is used for tracking. To evaluate proposed tracker’s performance, comprehensive experiments are conducted on challenging aerial sequences in the UAV123 dataset. Experimental results demonstrate that proposed approach achieves a state-ofthe-art tracking performance in aerial sequences and operates at a mean speed of beyond 40 frames/s. For further analysis of proposed tracker’s robustness, extensive experiments are also performed on recent benchmarks OTB50, OTB100, and VOT2016.展开更多
Collaborative Robotics is one of the high-interest research topics in the area of academia and industry.It has been progressively utilized in numerous applications,particularly in intelligent surveillance systems.It a...Collaborative Robotics is one of the high-interest research topics in the area of academia and industry.It has been progressively utilized in numerous applications,particularly in intelligent surveillance systems.It allows the deployment of smart cameras or optical sensors with computer vision techniques,which may serve in several object detection and tracking tasks.These tasks have been considered challenging and high-level perceptual problems,frequently dominated by relative information about the environment,where main concerns such as occlusion,illumination,background,object deformation,and object class variations are commonplace.In order to show the importance of top view surveillance,a collaborative robotics framework has been presented.It can assist in the detection and tracking of multiple objects in top view surveillance.The framework consists of a smart robotic camera embedded with the visual processing unit.The existing pre-trained deep learning models named SSD and YOLO has been adopted for object detection and localization.The detection models are further combined with different tracking algorithms,including GOTURN,MEDIANFLOW,TLD,KCF,MIL,and BOOSTING.These algorithms,along with detection models,help to track and predict the trajectories of detected objects.The pre-trained models are employed;therefore,the generalization performance is also investigated through testing the models on various sequences of top view data set.The detection models achieved maximum True Detection Rate 93%to 90%with a maximum 0.6%False Detection Rate.The tracking results of different algorithms are nearly identical,with tracking accuracy ranging from 90%to 94%.Furthermore,a discussion has been carried out on output results along with future guidelines.展开更多
Inspired by human behaviors,a robot object tracking model is proposed on the basis of visual attention mechanism,which is fit for the theory of topological perception.The model integrates the image-driven,bottom-up at...Inspired by human behaviors,a robot object tracking model is proposed on the basis of visual attention mechanism,which is fit for the theory of topological perception.The model integrates the image-driven,bottom-up attention and the object-driven,top-down attention,whereas the previous attention model has mostly focused on either the bottom-up or top-down attention.By the bottom-up component,the whole scene is segmented into the ground region and the salient regions.Guided by top-down strategy which is achieved by a topological graph,the object regions are separated from the salient regions.The salient regions except the object regions are the barrier regions.In order to estimate the model,a mobile robot platform is developed,on which some experiments are implemented.The experimental results indicate that processing an image with a resolution of 752×480 pixels takes less than 200 ms and the object regions are unabridged.The analysis obtained by comparing the proposed model with the existing model demonstrates that the proposed model has some advantages in robot object tracking in terms of speed and efficiency.展开更多
In YCbCr colorspace,a method is proposed to reconstruct the background and extract moving objects based on the Gaussian model of chroma components.Background model is updated according to changes of chroma components....In YCbCr colorspace,a method is proposed to reconstruct the background and extract moving objects based on the Gaussian model of chroma components.Background model is updated according to changes of chroma components.In order to eliminate the disturbance of shadow,a shadow detecting principle is proposed in YCbCr colorspace.A Kalman filter is introduced to estimate objects'positions in the image and then the pedestrian is tracked according to its information of shape.Experiments show that the background reconstruction and updating are successful,object extraction and shadow suppression are satisfactory,and real-time and reliable tracking is realized.展开更多
基金The Eighth National “Ten Thousand Talents Plan for Top Young Talents” of Chinathe National Natural Science Foundation of China (No. 52478117, 52378120)。
文摘To improve the safety of construction workers and help workers remotely control humanoid robots in construc-tion,this study designs and implements a computer vision based virtual construction simulation system.For this pur-pose,human skeleton motion data are collected using a Ki-nect depth camera,and the obtained data are optimized via abnormal data elimination,smoothing,and normalization.MediaPipe extracts three-dimensional hand motion coordi-nates for accurate human posture tracking.Blender is used to build a virtual worker and site model,and the virtual worker motion is controlled based on the quaternion inverse kinematics algorithm while limiting the joint angle to en-hance the authenticity of motion simulation.Experimental results show that the system frame rate is stable at 60 frame/s,end-to-end delay is less than 20 ms,and virtual task comple-tion time is close to the real scene,verifying its engineering applicability.The proposed system can drive virtual work-ers to perform tasks and provide technical support for con-struction safety training.
基金supported in part by the"Pioneer"and"Leading Goose"R&D Program of Zhejiang under Grant No. 2024C01169the National Natural Science Foundation of China under Grant Nos. 62441238 and U2441240。
文摘Visual object tracking(VOT),aiming to track a target object in a continuous video,is a fundamental and critical task in computer vision.However,the reliance on third-party resources(e.g.,dataset)for training poses concealed threats to the security of VOT models.In this paper,we reveal that VOT models are vulnerable to a poison-only and targeted backdoor attack,where the adversary can achieve arbitrary tracking predictions by manipulating only part of the training data.Specifically,we first define and formulate three different variants of the targeted attacks:size-manipulation,trajectory-manipulation,and hybrid attacks.To implement these,we introduce Random Video Poisoning(RVP),a novel poison-only strategy that exploits temporal correlations within video data by poisoning entire video sequences.Extensive experiments demonstrate that RVP effectively injects controllable backdoors,enabling precise manipulation of tracking behavior upon trigger activation,while maintaining high performance on benign data,thus ensuring stealth.Our findings not only expose significant vulnerabilities but also highlight that the underlying principles could be adapted for beneficial uses,such as dataset watermarking for copyright protection.
文摘This paper proposes the analysis model of sports human body based on computer vision tracking technology. Visual target tracking is an important research field of the computer vision, motion trajectory and it can provide not only the goal, to provide the initial data movement analysis, scene understanding, behavior or the event detection in intelligent surveillance, human-computer interaction, robot visual navigation and motion recognition based on field has a broad application prospect. For this reason, it is possible to consider the use of a large number of unlabeled samples to assist the training classifier to improve its performance. This type of machine learning method using both labeled and that unlabeled samples is called the semi-supervised learning. This paper proposes the novel idea of the related research topics to propose the new perspective of the model that will be later give us the novel idea of making it efficient for further development of sport science.
基金funded by theNational Science and TechnologyCouncil(NSTC),Taiwan,under grant numbers NSTC 113-2634-F-A49-007 and NSTC 112-2634-F-A49-007.
文摘To improve small object detection and trajectory estimation from an aerial moving perspective,we propose the Aerial View Attention-PRB(AVA-PRB)model.AVA-PRB integrates two attention mechanisms—Coordinate Attention(CA)and the Convolutional Block Attention Module(CBAM)—to enhance detection accuracy.Additionally,Shape-IoU is employed as the loss function to refine localization precision.Our model further incorporates an adaptive feature fusion mechanism,which optimizes multi-scale object representation,ensuring robust tracking in complex aerial environments.We evaluate the performance of AVA-PRB on two benchmark datasets:Aerial Person Detection and VisDrone2019-Det.The model achieves 60.9%mAP@0.5 on the Aerial Person Detection dataset,and 51.2%mAP@0.5 on VisDrone2019-Det,demonstrating its effectiveness in aerial object detection.Beyond detection,we propose a novel trajectory estimation method that improves movement path prediction under aerial motion.Experimental results indicate that our approach reduces path deviation by up to 64%,effectively mitigating errors caused by rapid camera movements and background variations.By optimizing feature extraction and enhancing spatialtemporal coherence,our method significantly improves object tracking under aerial moving perspectives.This research addresses the limitations of fixed-camera tracking,enhancing flexibility and accuracy in aerial tracking applications.The proposed approach has broad potential for real-world applications,including surveillance,traffic monitoring,and environmental observation.
基金supported by the STI 2030-Major Projects(2022ZD0211900,2022ZD0211902)STI 2030-Major Projects(2021ZD0204500,2021ZD0204503)+1 种基金National Natural Science Foundation of China(32171461)National Key Research and Development Program of China(2023YFC3208303)。
文摘Advancements in animal behavior quantification methods have driven the development of computational ethology,enabling fully automated behavior analysis.Existing multianimal pose estimation workflows rely on tracking-bydetection frameworks for either bottom-up or top-down approaches,requiring retraining to accommodate diverse animal appearances.This study introduces InteBOMB,an integrated workflow that enhances top-down approaches by incorporating generic object tracking,eliminating the need for prior knowledge of target animals while maintaining broad generalizability.InteBOMB includes two key strategies for tracking and segmentation in laboratory environments and two techniques for pose estimation in natural settings.The“background enhancement”strategy optimizesforeground-backgroundcontrastiveloss,generating more discriminative correlation maps.The“online proofreading”strategy stores human-in-the-loop long-term memory and dynamic short-term memory,enabling adaptive updates to object visual features.The“automated labeling suggestion”technique reuses the visual features saved during tracking to identify representative frames for training set labeling.Additionally,the“joint behavior analysis”technique integrates these features with multimodal data,expanding the latent space for behavior classification and clustering.To evaluate the framework,six datasets of mice and six datasets of nonhuman primates were compiled,covering laboratory and natural scenes.Benchmarking results demonstrated a24%improvement in zero-shot generic tracking and a 21%enhancement in joint latent space performance across datasets,highlighting the effectiveness of this approach in robust,generalizable behavior analysis.
文摘An improved estimation of motion vectors of feature points is proposed for tracking moving objects of dynamic image sequence. Feature points are firstly extracted by the improved minimum intensity change （MIC） algorithm. The matching points of these feature points are then determined by adaptive rood pattern searching. Based on the random sample consensus （RANSAC） method, the background motion is finally compensated by the parameters of an affine transform of the background motion. With reasonable morphological filtering, the moving objects are completely extracted from the background, and then tracked accurately. Experimental results show that the improved method is successful on the motion background compensation and offers great promise in tracking moving objects of the dynamic image sequence.
文摘Improved picture quality is critical to the effectiveness of object recog-nition and tracking.The consistency of those photos is impacted by night-video systems because the contrast between high-profile items and different atmospheric conditions,such as mist,fog,dust etc.The pictures then shift in intensity,colour,polarity and consistency.A general challenge for computer vision analyses lies in the horrid appearance of night images in arbitrary illumination and ambient envir-onments.In recent years,target recognition techniques focused on deep learning and machine learning have become standard algorithms for object detection with the exponential growth of computer performance capabilities.However,the iden-tification of objects in the night world also poses further problems because of the distorted backdrop and dim light.The Correlation aware LSTM based YOLO(You Look Only Once)classifier method for exact object recognition and deter-mining its properties under night vision was a major inspiration for this work.In order to create virtual target sets similar to daily environments,we employ night images as inputs;and to obtain high enhanced image using histogram based enhancement and iterative wienerfilter for removing the noise in the image.The process of the feature extraction and feature selection was done for electing the potential features using the Adaptive internal linear embedding(AILE)and uplift linear discriminant analysis(ULDA).The region of interest mask can be segmen-ted using the Recurrent-Phase Level set Segmentation.Finally,we use deep con-volution feature fusion and region of interest pooling to integrate the presently extremely sophisticated quicker Long short term memory based(LSTM)with YOLO method for object tracking system.A range of experimentalfindings demonstrate that our technique achieves high average accuracy with a precision of 99.7%for object detection of SSAN datasets that is considerably more than that of the other standard object detection mechanism.Our approach may therefore satisfy the true demands of night scene target detection applications.We very much believe that our method will help future research.
基金supported in part by the Institute for Guo Qiang of Tsinghua University(2019GQG1023)in part by Graduate Education and Teaching Reform Project of Tsinghua University(202007J007)+1 种基金in part by National Natural Science Foundation of China(U19B2029,62073028,61803222)in part by the Independent Research Program of Tsinghua University(2018Z05JDX002)。
文摘There are two main trends in the development of unmanned aerial vehicle(UAV)technologies:miniaturization and intellectualization,in which realizing object tracking capabilities for a nano-scale UAV is one of the most challenging problems.In this paper,we present a visual object tracking and servoing control system utilizing a tailor-made 38 g nano-scale quadrotor.A lightweight visual module is integrated to enable object tracking capabilities,and a micro positioning deck is mounted to provide accurate pose estimation.In order to be robust against object appearance variations,a novel object tracking algorithm,denoted by RMCTer,is proposed,which integrates a powerful short-term tracking module and an efficient long-term processing module.In particular,the long-term processing module can provide additional object information and modify the short-term tracking model in a timely manner.Furthermore,a positionbased visual servoing control method is proposed for the quadrotor,where an adaptive tracking controller is designed by leveraging backstepping and adaptive techniques.Stable and accurate object tracking is achieved even under disturbances.Experimental results are presented to demonstrate the high accuracy and stability of the whole tracking system.
文摘On accomplishing an efficacious object tracking,the activity of an object concerned becomes notified in a forthright manner.An accurate form of object tracking task necessitates a robust object tracking procedures irrespective of hardware assistance.Such approaches inferred a vast computational complexity to track an object with high accuracy in a stipulated amount of processing time.On the other hand,the tracking gets affected owing to the existence of varied quality diminishing factors such as occlusion,illumination changes,shadows etc.,In order to rectify all these inadequacies in tracking an object,a novel background normalization procedure articulated on the basis of a textural pattern is proposed in this paper.After preprocessing an acquired image,employment of an Environmental Succession Prediction algorithm for discriminating disparate background environment by background clustering approach have been accomplished.Afterward,abstract textural characterizations through utilization of a Probability based Gradient Pattern(PGP)approach for recognizing the similarity between patterns obtained so far.Comparison between standardized frame obtained in prior and those processed patterns detects the motion exposed by an object and the object concerned gets identified within a blob.Hence,the system is resistant towards illumination variations.These illumination variation was interpreted in object tracking residing within a dynamic background.Devised approach certainly outperforms other object tracking methodologies like Group Target Tracking(GTT),Vi PER-GT,grabcut,snakes in terms of accuracy and average time.Proposed PGP-based pattern texture analysis is compared with Gamifying Video Object(GVO)approach and hence,it evidently outperforms in terms of precision,recall and F1 measure.
基金supported in part by National Natural Science Foundation of China(No.62176041)in part by Excellent Science and Technique Talent Foundation of Dalian(No.2022RY21).
文摘Significant advancements have beenwitnessed in visual tracking applications leveragingViT in recent years,mainly due to the formidablemodeling capabilities of Vision Transformer(ViT).However,the strong performance of such trackers heavily relies on ViT models pretrained for long periods,limitingmore flexible model designs for tracking tasks.To address this issue,we propose an efficient unsupervised ViT pretraining method for the tracking task based on masked autoencoders,called TrackMAE.During pretraining,we employ two shared-parameter ViTs,serving as the appearance encoder and motion encoder,respectively.The appearance encoder encodes randomly masked image data,while the motion encoder encodes randomly masked pairs of video frames.Subsequently,an appearance decoder and a motion decoder separately reconstruct the original image data and video frame data at the pixel level.In this way,ViT learns to understand both the appearance of images and the motion between video frames simultaneously.Experimental results demonstrate that ViT-Base and ViT-Large models,pretrained with TrackMAE and combined with a simple tracking head,achieve state-of-the-art(SOTA)performance without additional design.Moreover,compared to the currently popular MAE pretraining methods,TrackMAE consumes only 1/5 of the training time,which will facilitate the customization of diverse models for tracking.For instance,we additionally customize a lightweight ViT-XS,which achieves SOTA efficient tracking performance.
文摘This paper discusses about the new approach of multiple object track-ing relative to background information.The concept of multiple object tracking through background learning is based upon the theory of relativity,that involves a frame of reference in spatial domain to localize and/or track any object.Thefield of multiple object tracking has seen a lot of research,but researchers have considered the background as redundant.However,in object tracking,the back-ground plays a vital role and leads to definite improvement in the overall process of tracking.In the present work an algorithm is proposed for the multiple object tracking through background learning.The learning framework is based on graph embedding approach for localizing multiple objects.The graph utilizes the inher-ent capabilities of depth modelling that assist in prior to track occlusion avoidance among multiple objects.The proposed algorithm has been compared with the recent work available in literature on numerous performance evaluation measures.It is observed that our proposed algorithm gives better performance.
基金supported by the National Natural Science Foundation of China under Grant 62177029the Postgraduate Research&Practice Innovation Program of Jiangsu Province(KYCX21_0740),China.
文摘Visual object tracking plays a crucial role in computer vision.In recent years,researchers have proposed various methods to achieve high-performance object tracking.Among these,methods based on Transformers have become a research hotspot due to their ability to globally model and contextualize information.However,current Transformer-based object tracking methods still face challenges such as low tracking accuracy and the presence of redundant feature information.In this paper,we introduce self-calibration multi-head self-attention Transformer(SMSTracker)as a solution to these challenges.It employs a hybrid tensor decomposition self-organizing multihead self-attention transformermechanism,which not only compresses and accelerates Transformer operations but also significantly reduces redundant data,thereby enhancing the accuracy and efficiency of tracking.Additionally,we introduce a self-calibration attention fusion block to resolve common issues of attention ambiguities and inconsistencies found in traditional trackingmethods,ensuring the stability and reliability of tracking performance across various scenarios.By integrating a hybrid tensor decomposition approach with a self-organizingmulti-head self-attentive transformer mechanism,SMSTracker enhances the efficiency and accuracy of the tracking process.Experimental results show that SMSTracker achieves competitive performance in visual object tracking,promising more robust and efficient tracking systems,demonstrating its potential to providemore robust and efficient tracking solutions in real-world applications.
文摘The performance and accuracy of computer vision systems are affected by noise in different forms.Although numerous solutions and algorithms have been presented for dealing with every type of noise,a comprehensive technique that can cover all the diverse noises and mitigate their damaging effects on the performance and precision of various systems is still missing.In this paper,we have focused on the stability and robustness of one computer vision branch(i.e.,visual object tracking).We have demonstrated that,without imposing a heavy computational load on a model or changing its algorithms,the drop in the performance and accuracy of a system when it is exposed to an unseen noise-laden test dataset can be prevented by simply applying the style transfer technique on the train dataset and training the model with a combination of these and the original untrained data.To verify our proposed approach,it is applied on a generic object tracker by using regression networks.This method’s validity is confirmed by testing it on an exclusive benchmark comprising 50 image sequences,with each sequence containing 15 types of noise at five different intensity levels.The OPE curves obtained show a 40%increase in the robustness of the proposed object tracker against noise,compared to the other trackers considered.
基金the National Natural Science Foundation of China (No.61673269)。
文摘Visual object tracking is an important issue that has received long-term attention in computer vision.The ability to effectively handle occlusion,especially severe occlusion,is an important aspect of evaluating the performance of object tracking algorithms in long-term tracking,and is of great significance to improving the robustness of object tracking algorithms.However,most object tracking algorithms lack a processing mechanism specifically for occlusion.In the case of occlusion,due to the lack of target information,it is necessary to predict the target position based on the motion trajectory.Kalman filtering and particle filtering can effectively predict the target motion state based on the historical motion information.A single object tracking method,called probabilistic discriminative model prediction(PrDiMP),is based on the spatial attention mechanism in complex scenes and occlusions.In order to improve the performance of PrDiMP,Kalman filtering,particle filtering and linear filtering are introduced.First,for the occlusion situation,Kalman filtering and particle filtering are respectively introduced to predict the object position,thereby replacing the detection result of the original tracking algorithm and stopping recursion of target model.Second,for detection-jump problem of similar objects in complex scenes,a linear filtering window is added.The evaluation results on the three datasets,including GOT-10k,UAV123 and LaSOT,and the visualization results on several videos,show that our algorithms have improved tracking performance under occlusion and the detection-jump is effectively suppressed.
文摘A snake algorithm has been known that it has a strong point in extracting the exact contour of an object. But it is apt to be influenced by scattered edges around the control points. Since the shape of a moving object in 2D image changes a lot due to its rotation and translation in the 3D space, the conventional algorithm that takes into account slowly moving objects cannot provide an appropriate solution. To utilize the advantages of the snake algorithm while minimizing the drawbacks, this paper proposes the area variation based color snake algorithm for moving object tracking. The proposed algorithm includes a new energy term which is used for preserving the shape of an object between two consecutive images. The proposed one can also segment precisely interesting objects on complex image since it is based on color information. Experiment results show that the proposed algorithm is very effective in various environments.
文摘Visual object-tracking is a fundamental task applied in many applications of computer vision. Particle filter is one of the techniques which has been widely used in object tracking. Due to the virtue of extendability and flexibility on both linear and non-linear environments, various particle filter-based trackers have been proposed in the literature. However, the conventional approach cannot handle very large videos efficiently in the current data intensive information age. In this work, a parallelized particle filter is provided in a distributed framework provided by the Hadoop/Map-Reduce infrastructure to tackle object-tracking tasks. The experiments indicate that the proposed algorithm has a better convergence and accuracy as compared to the traditional particle filter. The computational power and the scalability of the proposed particle filter in single object tracking have been enhanced as well.
基金supported by the National Natural Science Foundation of China(No.61671002)
文摘This paper addresses the problem of real-time object tracking for unmanned aerial vehicles. We consider the task of object tracking as a classification problem. Training a good classifier always needs a huge number of samples, which is always time-consuming and not suitable for realtime applications. In this paper, we transform the large-scale least-squares problem in the spatial domain to a series of small-scale least-squares problems with constraints in the Fourier domain using the correlation filter technique. Then, this problem is efficiently solved by two stages. In the first stage, a fast method based on recursive least squares is used to solve the correlation filter problem without constraints in the Fourier domain. In the second stage, a weight matrix is constructed to prune the solution attained in the first stage to approach the constraints in the spatial domain. Then, the pruned classifier is used for tracking. To evaluate proposed tracker’s performance, comprehensive experiments are conducted on challenging aerial sequences in the UAV123 dataset. Experimental results demonstrate that proposed approach achieves a state-ofthe-art tracking performance in aerial sequences and operates at a mean speed of beyond 40 frames/s. For further analysis of proposed tracker’s robustness, extensive experiments are also performed on recent benchmarks OTB50, OTB100, and VOT2016.
基金the Framework of International Cooperation Program managed by the National Research Foundation of Korea(2019K1A3A1A8011295711).
文摘Collaborative Robotics is one of the high-interest research topics in the area of academia and industry.It has been progressively utilized in numerous applications,particularly in intelligent surveillance systems.It allows the deployment of smart cameras or optical sensors with computer vision techniques,which may serve in several object detection and tracking tasks.These tasks have been considered challenging and high-level perceptual problems,frequently dominated by relative information about the environment,where main concerns such as occlusion,illumination,background,object deformation,and object class variations are commonplace.In order to show the importance of top view surveillance,a collaborative robotics framework has been presented.It can assist in the detection and tracking of multiple objects in top view surveillance.The framework consists of a smart robotic camera embedded with the visual processing unit.The existing pre-trained deep learning models named SSD and YOLO has been adopted for object detection and localization.The detection models are further combined with different tracking algorithms,including GOTURN,MEDIANFLOW,TLD,KCF,MIL,and BOOSTING.These algorithms,along with detection models,help to track and predict the trajectories of detected objects.The pre-trained models are employed;therefore,the generalization performance is also investigated through testing the models on various sequences of top view data set.The detection models achieved maximum True Detection Rate 93%to 90%with a maximum 0.6%False Detection Rate.The tracking results of different algorithms are nearly identical,with tracking accuracy ranging from 90%to 94%.Furthermore,a discussion has been carried out on output results along with future guidelines.
基金supported by National Basic Research Program of China(973 Program)(No.2006CB300407)National Natural Science Foundation of China(No.50775017)
文摘Inspired by human behaviors,a robot object tracking model is proposed on the basis of visual attention mechanism,which is fit for the theory of topological perception.The model integrates the image-driven,bottom-up attention and the object-driven,top-down attention,whereas the previous attention model has mostly focused on either the bottom-up or top-down attention.By the bottom-up component,the whole scene is segmented into the ground region and the salient regions.Guided by top-down strategy which is achieved by a topological graph,the object regions are separated from the salient regions.The salient regions except the object regions are the barrier regions.In order to estimate the model,a mobile robot platform is developed,on which some experiments are implemented.The experimental results indicate that processing an image with a resolution of 752×480 pixels takes less than 200 ms and the object regions are unabridged.The analysis obtained by comparing the proposed model with the existing model demonstrates that the proposed model has some advantages in robot object tracking in terms of speed and efficiency.
基金This project is supported by Program for New Century Excellent Talents in University,China(No.NCET-04-0545)National Defense Science Foundation of China(No.51416070104JW0404)
文摘In YCbCr colorspace,a method is proposed to reconstruct the background and extract moving objects based on the Gaussian model of chroma components.Background model is updated according to changes of chroma components.In order to eliminate the disturbance of shadow,a shadow detecting principle is proposed in YCbCr colorspace.A Kalman filter is introduced to estimate objects'positions in the image and then the pedestrian is tracked according to its information of shape.Experiments show that the background reconstruction and updating are successful,object extraction and shadow suppression are satisfactory,and real-time and reliable tracking is realized.
