In daily life,human need various senses to obtain information about their surroundings,and touch is one of the five major human sensing signals.Similarly,it is extremely important for robots to be endowed with tactile...In daily life,human need various senses to obtain information about their surroundings,and touch is one of the five major human sensing signals.Similarly,it is extremely important for robots to be endowed with tactile sensing ability.In recent years,vision-based tactile sensing technology has been the research hotspot and frontier in the field of tactile perception.Compared to conventional tactile sensing technologies,vision-based tactile sensing technologies are capable of obtaining highquality and high-resolution tactile information at a lower cost,while not being limited by the size and shape of sensors.Several previous articles have reviewed the sensing mechanism and electrical components of vision-based sensors,greatly promoting the innovation of tactile sensing.Different from existing reviews,this article concentrates on the underlying tracking method which converts real-time images into deformation information,including contact,sliding and friction.We will show the history and development of both model-based and model-free tracking methods,among which model-based approaches rely on schematic mechanical theories,and model-free approaches mainly involve machine learning algorithms.Comparing the efficiency and accuracy of existing deformation tracking methods,future research directions of vision-based tactile sensors for smart manipulations and robots are also discussed.展开更多
Vision-based air-to-air tracking of multi-UAV swarms is crucial for effective swarm perception.UAVs in a swarm typically share similar appearance features and exhibit nonlinear motion,as they are usually of the same t...Vision-based air-to-air tracking of multi-UAV swarms is crucial for effective swarm perception.UAVs in a swarm typically share similar appearance features and exhibit nonlinear motion,as they are usually of the same type.This homogeneity poses challenges for the existing multi-object tracking(MOT)algorithms,which often suffer performance degradation due to the difficulties in capturing instance-specific appearance and motion cues.In this paper,we propose a novel multiframe pose-attention-based appearance feature extraction component that captures instance-level pose features of UAVs across consecutive frames.Additionally,we introduce a motion difference accumulation strategy to extract spatial and motion cues from multiple adjacent frames.By combining these techniques,we design a multi-frame association framework that effectively distinguishes between similar UAVs in a swarm by leveraging object features over consecutive frames.To address the lack of relevant datasets,we create the AIRMOT dataset,specifically tailored for air-to-air tracking of homogeneous UAV swarms.Our method is evaluated on the AIRMOT dataset as well as the publicly available MOT-FLY and UAVSwarm datasets.The experimental results demonstrate that our approach outperforms other state-of-the-art(SOTA)methods,delivering superior tracking performance.展开更多
To address the issues of frequent identity switches(IDs)and degraded identification accuracy in multi object tracking(MOT)under complex occlusion scenarios,this study proposes an occlusion-robust tracking framework ba...To address the issues of frequent identity switches(IDs)and degraded identification accuracy in multi object tracking(MOT)under complex occlusion scenarios,this study proposes an occlusion-robust tracking framework based on face-pedestrian joint feature modeling.By constructing a joint tracking model centered on“intra-class independent tracking+cross-category dynamic binding”,designing a multi-modal matching metric with spatio-temporal and appearance constraints,and innovatively introducing a cross-category feature mutual verification mechanism and a dual matching strategy,this work effectively resolves performance degradation in traditional single-category tracking methods caused by short-term occlusion,cross-camera tracking,and crowded environments.Experiments on the Chokepoint_Face_Pedestrian_Track test set demonstrate that in complex scenes,the proposed method improves Face-Pedestrian Matching F1 area under the curve(F1 AUC)by approximately 4 to 43 percentage points compared to several traditional methods.The joint tracking model achieves overall performance metrics of IDF1:85.1825%and MOTA:86.5956%,representing improvements of 0.91 and 0.06 percentage points,respectively,over the baseline model.Ablation studies confirm the effectiveness of key modules such as the Intersection over Area(IoA)/Intersection over Union(IoU)joint metric and dynamic threshold adjustment,validating the significant role of the cross-category identity matching mechanism in enhancing tracking stability.Our_model shows a 16.7%frame per second(FPS)drop vs.fairness of detection and re-identification in multiple object tracking(FairMOT),with its cross-category binding module adding aboute 10%overhead,yet maintains near-real-time performance for essential face-pedestrian tracking at small resolutions.展开更多
This paper synchronizes control theory with computer vision by formalizing object tracking as a sequential decision-making process.A reinforcement learning(RL)agent successfully tracks an interface between two liquids...This paper synchronizes control theory with computer vision by formalizing object tracking as a sequential decision-making process.A reinforcement learning(RL)agent successfully tracks an interface between two liquids,which is often a critical variable to track in many chemical,petrochemical,metallurgical,and oil industries.This method utilizes less than 100 images for creating an environment,from which the agent generates its own data without the need for expert knowledge.Unlike supervised learning(SL)methods that rely on a huge number of parameters,this approach requires far fewer parameters,which naturally reduces its maintenance cost.Besides its frugal nature,the agent is robust to environmental uncertainties such as occlusion,intensity changes,and excessive noise.From a closed-loop control context,an interface location-based deviation is chosen as the optimization goal during training.The methodology showcases RL for real-time object-tracking applications in the oil sands industry.Along with a presentation of the interface tracking problem,this paper provides a detailed review of one of the most effective RL methodologies:actor–critic policy.展开更多
An important and challenging aspect of developing an intelligent transportation system is the identification of nighttime vehicles. Most accidents occur at night owing to the absence of night lighting conditions. Vehi...An important and challenging aspect of developing an intelligent transportation system is the identification of nighttime vehicles. Most accidents occur at night owing to the absence of night lighting conditions. Vehicle detection has become a vital subject for research to ensure safety and avoid accidents. New vision-based on-road nighttime vehicle detection and tracking system are suggested in this survey paper using taillight and headlight features. Using computer vision and some image processing techniques, the proposed system can identify vehicles based on taillight and headlight features. For vehicle tracking, a centroid tracking algorithm has been used. Euclidean Distance method has been used for measuring the distances between two neighboring objects and tracks the nearest neighbor. In the proposed system two flexible fixed Region of Interest (ROI) have been used, one is the Headlight ROI, and another is the Taillight ROI that could adapt to different resolutions of the images and videos. The achievement of this research work is that the proposed two ROIs can work simultaneously in a frame to identify oncoming and preceding vehicles at night. The segmentation techniques and double thresholding method have been used to extract the red and white components from the scene to identify the vehicle headlights and taillights. To evaluate the capability of the proposed process, two types of datasets have been used. Experimental findings indicate that the performance of the proposed technique is reliable and effective in distinct nighttime environments for detection and tracking of vehicles. The proposed method has been able to detect and track double lights as well as single light such as motorcycle light and achieved average accuracy and average processing time of vehicle detection about 97.22% and 0.01 s per frame respectively.展开更多
Controlling multiple multi-joint fish-like robots has long captivated the attention of engineers and biologists,for which a fundamental but challenging topic is to robustly track the postures of the individuals in rea...Controlling multiple multi-joint fish-like robots has long captivated the attention of engineers and biologists,for which a fundamental but challenging topic is to robustly track the postures of the individuals in real time.This requires detecting multiple robots,estimating multi-joint postures,and tracking identities,as well as processing fast in real time.To the best of our knowledge,this challenge has not been tackled in the previous studies.In this paper,to precisely track the planar postures of multiple swimming multi-joint fish-like robots in real time,we propose a novel deep neural network-based method,named TAB-IOL.Its TAB part fuses the top-down and bottom-up approaches for vision-based pose estimation,while the IOL part with long short-term memory considers the motion constraints among joints for precise pose tracking.The satisfying performance of our TAB-IOL is verified by testing on a group of freely swimming fish-like robots in various scenarios with strong disturbances and by a deed comparison of accuracy,speed,and robustness with most state-of-the-art algorithms.Further,based on the precise pose estimation and tracking realized by our TAB-IOL,several formation control experiments are conducted for the group of fish-like robots.The results clearly demonstrate that our TAB-IOL lays a solid foundation for the coordination control of multiple fish-like robots in a real working environment.We believe our proposed method will facilitate the growth and development of related fields.展开更多
Atom tracking technology enhanced with innovative algorithms has been implemented in this study,utilizing a comprehensive suite of controllers and software independently developed domestically.Leveraging an on-board f...Atom tracking technology enhanced with innovative algorithms has been implemented in this study,utilizing a comprehensive suite of controllers and software independently developed domestically.Leveraging an on-board field-programmable gate array(FPGA)with a core frequency of 100 MHz,our system facilitates reading and writing operations across 16 channels,performing discrete incremental proportional-integral-derivative(PID)calculations within 3.4 microseconds.Building upon this foundation,gradient and extremum algorithms are further integrated,incorporating circular and spiral scanning modes with a horizontal movement accuracy of 0.38 pm.This integration enhances the real-time performance and significantly increases the accuracy of atom tracking.Atom tracking achieves an equivalent precision of at least 142 pm on a highly oriented pyrolytic graphite(HOPG)surface under room temperature atmospheric conditions.Through applying computer vision and image processing algorithms,atom tracking can be used when scanning a large area.The techniques primarily consist of two algorithms:the region of interest(ROI)-based feature matching algorithm,which achieves 97.92%accuracy,and the feature description-based matching algorithm,with an impressive 99.99%accuracy.Both implementation approaches have been tested for scanner drift measurements,and these technologies are scalable and applicable in various domains of scanning probe microscopy with broad application prospects in the field of nanoengineering.展开更多
Unmanned aerial vehicles(UAVs)have become crucial tools in moving target tracking due to their agility and ability to operate in complex,dynamic environments.UAVs must meet several requirements to achieve stable track...Unmanned aerial vehicles(UAVs)have become crucial tools in moving target tracking due to their agility and ability to operate in complex,dynamic environments.UAVs must meet several requirements to achieve stable tracking,including maintaining continuous target visibility amidst occlusions,ensuring flight safety,and achieving smooth trajectory planning.This paper reviews the latest advancements in UAV-based target tracking,highlighting information prediction,tracking strategies,and swarm cooperation.To address challenges including target visibility and occlusion,real-time prediction and tracking in dynamic environments,flight safety and coordination,resource management and energy efficiency,the paper identifies future research directions aimed at improving the performance,reliability,and scalability of UAV tracking system.展开更多
This paper studies the tracking control problem for stratospheric airships with userspecified performance.Dealing with the infinite gain phenomenon in the prescribed-time stability,a new stability criterion with bound...This paper studies the tracking control problem for stratospheric airships with userspecified performance.Dealing with the infinite gain phenomenon in the prescribed-time stability,a new stability criterion with bounded gain is proposed by using a new time-varying scaling function.Moreover,a same-side performance function and a novel barrier Lyapunov function are incorporated into the control algorithm,which can compress the feasible domain of tracking error to minimize the overshoot and solve the difficult in tracking error not converging to zero simultaneously.The proposed scheme guarantees the airship capable of operating autonomously with satisfactory transient performance and tracking accuracy,where the performance parameters can be designed artificially and link to the physical process directly.Finally,the effectiveness of the proposed control scheme is verified by theoretical analysis and numerical simulation.展开更多
This work proposes the application of an iterative learning model predictive control(ILMPC)approach based on an adaptive fault observer(FOBILMPC)for fault-tolerant control and trajectory tracking in air-breathing hype...This work proposes the application of an iterative learning model predictive control(ILMPC)approach based on an adaptive fault observer(FOBILMPC)for fault-tolerant control and trajectory tracking in air-breathing hypersonic vehicles.In order to increase the control amount,this online control legislation makes use of model predictive control(MPC)that is based on the concept of iterative learning control(ILC).By using offline data to decrease the linearized model’s faults,the strategy may effectively increase the robustness of the control system and guarantee that disturbances can be suppressed.An adaptive fault observer is created based on the suggested ILMPC approach in order to enhance overall fault tolerance by estimating and compensating for actuator disturbance and fault degree.During the derivation process,a linearized model of longitudinal dynamics is established.The suggested ILMPC approach is likely to be used in the design of hypersonic vehicle control systems since numerical simulations have demonstrated that it can decrease tracking error and speed up convergence when compared to the offline controller.展开更多
This paper investigates the challenges associated with Unmanned Aerial Vehicle (UAV) collaborative search and target tracking in dynamic and unknown environments characterized by limited field of view. The primary obj...This paper investigates the challenges associated with Unmanned Aerial Vehicle (UAV) collaborative search and target tracking in dynamic and unknown environments characterized by limited field of view. The primary objective is to explore the unknown environments to locate and track targets effectively. To address this problem, we propose a novel Multi-Agent Reinforcement Learning (MARL) method based on Graph Neural Network (GNN). Firstly, a method is introduced for encoding continuous-space multi-UAV problem data into spatial graphs which establish essential relationships among agents, obstacles, and targets. Secondly, a Graph AttenTion network (GAT) model is presented, which focuses exclusively on adjacent nodes, learns attention weights adaptively and allows agents to better process information in dynamic environments. Reward functions are specifically designed to tackle exploration challenges in environments with sparse rewards. By introducing a framework that integrates centralized training and distributed execution, the advancement of models is facilitated. Simulation results show that the proposed method outperforms the existing MARL method in search rate and tracking performance with less collisions. The experiments show that the proposed method can be extended to applications with a larger number of agents, which provides a potential solution to the challenging problem of multi-UAV autonomous tracking in dynamic unknown environments.展开更多
To realize effective co-phasing adjustment in large-aperture sparse-aperture telescopes,a multichannel stripe tracking approach is employed,allowing simultaneous interferometric measurements of multiple optical paths ...To realize effective co-phasing adjustment in large-aperture sparse-aperture telescopes,a multichannel stripe tracking approach is employed,allowing simultaneous interferometric measurements of multiple optical paths and circumventing the need for pairwise measurements along the mirror boundaries in traditional interferometric methods.This approach enhances detection efficiency and reduces system complexity.Here,the principles of the multibeam interference process and construction of a co-phasing detection module based on direct optical fiber connections were analyzed using wavefront optics theory.Error analysis was conducted on the system surface obtained through multipath interference.Potential applications of the interferometric method were explored.Finally,the principle was verified by experiment,an interferometric fringe contrast better than 0.4 is achieved through flat field calibration and incoherent digital synthesis.The dynamic range of the measurement exceeds 10 times of the center wavelength of the working band(1550 nm).Moreover,a resolution better than one-tenth of the working center wavelength(1550 nm)was achieved.Simultaneous three-beam interference can be achieved,leading to a 50%improvement in detection efficiency.This method can effectively enhance the efficiency of sparse aperture telescope co-phasing,meeting the requirements for observations of 8-10 m telescopes.This study provides a technological foundation for observing distant and faint celestial objects.展开更多
Embodied visual exploration is critical for building intelligent visual agents. This paper presents the neural exploration with feature-based visual odometry and tracking-failure-reduction policy(Ne OR), a framework f...Embodied visual exploration is critical for building intelligent visual agents. This paper presents the neural exploration with feature-based visual odometry and tracking-failure-reduction policy(Ne OR), a framework for embodied visual exploration that possesses the efficient exploration capabilities of deep reinforcement learning(DRL)-based exploration policies and leverages feature-based visual odometry(VO) for more accurate mapping and positioning results. An improved local policy is also proposed to reduce tracking failures of feature-based VO in weakly textured scenes through a refined multi-discrete action space, keyframe fusion, and an auxiliary task. The experimental results demonstrate that Ne OR has better mapping and positioning accuracy compared to other entirely learning-based exploration frameworks and improves the robustness of feature-based VO by significantly reducing tracking failures in weakly textured scenes.展开更多
This paper presents the design and ground verification for vision-based relative navigation systems of microsatellites,which offers a comprehensive hardware design solution and a robust experimental verification metho...This paper presents the design and ground verification for vision-based relative navigation systems of microsatellites,which offers a comprehensive hardware design solution and a robust experimental verification methodology for practical implementation of vision-based navigation technology on the microsatellite platform.Firstly,a low power consumption,light weight,and high performance vision-based relative navigation optical sensor is designed.Subsequently,a set of ground verification system is designed for the hardware-in-the-loop testing of the vision-based relative navigation systems.Finally,the designed vision-based relative navigation optical sensor and the proposed angles-only navigation algorithms are tested on the ground verification system.The results verify that the optical simulator after geometrical calibration can meet the requirements of the hardware-in-the-loop testing of vision-based relative navigation systems.Based on experimental results,the relative position accuracy of the angles-only navigation filter at terminal time is increased by 25.5%,and the relative speed accuracy is increased by 31.3% compared with those of optical simulator before geometrical calibration.展开更多
The forthcoming sixth generation(6G)of mobile communication networks is envisioned to be AInative,supporting intelligent services and pervasive computing at unprecedented scale.Among the key paradigms enabling this vi...The forthcoming sixth generation(6G)of mobile communication networks is envisioned to be AInative,supporting intelligent services and pervasive computing at unprecedented scale.Among the key paradigms enabling this vision,Federated Learning(FL)has gained prominence as a distributed machine learning framework that allows multiple devices to collaboratively train models without sharing raw data,thereby preserving privacy and reducing the need for centralized storage.This capability is particularly attractive for vision-based applications,where image and video data are both sensitive and bandwidth-intensive.However,the integration of FL with 6G networks presents unique challenges,including communication bottlenecks,device heterogeneity,and trade-offs between model accuracy,latency,and energy consumption.In this paper,we developed a simulation-based framework to investigate the performance of FL in representative vision tasks under 6G-like environments.We formalize the system model,incorporating both the federated averaging(FedAvg)training process and a simplified communication costmodel that captures bandwidth constraints,packet loss,and variable latency across edge devices.Using standard image datasets(e.g.,MNIST,CIFAR-10)as benchmarks,we analyze how factors such as the number of participating clients,degree of data heterogeneity,and communication frequency influence convergence speed and model accuracy.Additionally,we evaluate the effectiveness of lightweight communication-efficient strategies,including local update tuning and gradient compression,in mitigating network overhead.The experimental results reveal several key insights:(i)communication limitations can significantly degrade FL convergence in vision tasks if not properly addressed;(ii)judicious tuning of local training epochs and client participation levels enables notable improvements in both efficiency and accuracy;and(iii)communication-efficient FL strategies provide a promising pathway to balance performance with the stringent latency and reliability requirements expected in 6G.These findings highlight the synergistic role of AI and nextgeneration networks in enabling privacy-preserving,real-time vision applications,and they provide concrete design guidelines for researchers and practitioners working at the intersection of FL and 6G.展开更多
Unlike ensemble-averaging measurements,single-molecule tracking provides quantitative information on the kinetics of individual molecules within living cells in real time and may provide insight into the respective mo...Unlike ensemble-averaging measurements,single-molecule tracking provides quantitative information on the kinetics of individual molecules within living cells in real time and may provide insight into the respective molecular interactions behind that.The advancement of single-molecule tracking has been signi-cantly boosted by the development of high-resolution microscopy techniques.In this review,we will discuss this aspect with a particular focus on their recent advance in MINFLUX nanoscopy with feedback approaches where tracking is performed in real time.MINFLUX localization requires fewer than 100 photons from a-1 nm-sized°uorophore,enabling precise tracking.This approach,which demands over an order of magnitude fewer photons than other localization-based techniques(such as STORM,PLAM),allows molecular tracking with single-digit nanometer accuracy in less than 1 ms—an achievement previously unattainable.展开更多
Background:Early identification of concussion-related vision disorders(CRVDs)may improve outcomes by enabling earlier management,referral,and treatment.Objective eye tracking may provide additional data to support the...Background:Early identification of concussion-related vision disorders(CRVDs)may improve outcomes by enabling earlier management,referral,and treatment.Objective eye tracking may provide additional data to support the diagnose of CRVDs.The purpose of this study was to determine the utility of objective infrared eye tracking in identifying CRVDs among adolescents experiencing persisting post-concussive symptoms(PPCS)more than 28 days after injury.Methods:This was a prospective study of adolescents with PPCS evaluated with visio-vestibular examination(VVE),comprehensive vision examination,and an eye tracking device.Results:Of the 108 adolescents enrolled,67(62%)were diagnosed with a CRVD by comprehensive vision examination.On VVE,the near point of convergence break(5.5±3.2 cm vs.3.9±1.7 cm(mean±SD),p<0.001)and recovery(8.1±3.3 cm vs.6.8±2.3 cm,p=0.02)distinguished between those with and without CRVD.Concussion symptom provocation on VVE with horizontal saccades(35(52%)vs.12(29%),p=0.02)and horizontal vestibulo-ocular reflex testing(37(55%)vs.14(34%),p=0.03),and sway on tandem gait under the forward eyes closed condition(25(37%)vs.6(15%),p=0.01)also identified those with CRVD.From the eye tracking device,the BOX score(8.1±5.8 vs.5.2±4.1,p=0.007)and a metric of the left eye tracking along the bottom of the visual target(0.094±0.500 vs.-0.124±0.410,p=0.02)identified those with CRVD,with a multivariable receiver operating characteristic curve analysis,including the BOX score,achieving an area under the receiver operating characteristic curve of 0.7637.Conclusion:CRVDs are common in those with PPCS,with impact on recovery after concussion.Novel eye-tracking metrics can serve as an aid in the identification of those with CRVDs who would benefit from referral for comprehensive diagnosis and treatment.展开更多
The publisher regrets that the Appendix A.Supplementary data was not updated as per author and editor’s request.The publisher would like to apologise for any inconvenience caused.
Multi-axle Swerve-drive Autonomous Mobile Robots(MS-AMRs)equipped with independently steerable wheels are commonly used for high-payload transportation.In this work,we present a novel Model Predictive Control(MPC)meth...Multi-axle Swerve-drive Autonomous Mobile Robots(MS-AMRs)equipped with independently steerable wheels are commonly used for high-payload transportation.In this work,we present a novel Model Predictive Control(MPC)method for MS-AGV trajectory tracking that takes tire wear minimization consideration in the objective function.To speed up the problem-solving process,we propose a hierarchical controller design and simplify the dynamic model by integrating the magic formula tire model and simplified tire wear model.In the experiment,the proposed method can be solved by simulated annealing in real-time on a normal personal computer and by incorporating tire wear into the objective function,tire wear is reduced by 19.19%while maintaining the tracking accuracy in curve-tracking experiments.In the more challenging scene:the desired trajectory is offset by 60 degrees from the vehicle's heading,the reduction in tire wear increased to 65.20%compared to the kinematic model without considering the tire wear optimization.展开更多
Syntax and semantics are two important factors that influence sentence processing.Studies have found different aging effects in syntactic and semantic processing during sentence comprehension.While there is consensus ...Syntax and semantics are two important factors that influence sentence processing.Studies have found different aging effects in syntactic and semantic processing during sentence comprehension.While there is consensus on the aging effects in syntactic processing,the presence of aging in semantic processing remains debated.The present study aimed to explore whether there were aging effects in lexical-semantic information processing in complex sentence.79 participants were recruited to take part in this study,including 40 younger adults(mean age of 21.1±1.19 years)and 39 older adults(mean age of 66.24±3.02 years).Using eye-movement tracking technology and manipulating the animacy of head nouns in Chinese subject relative clauses(SRCs)and object relative clauses(ORCs),we investigated the abilities of young and old adults in relative clauses(RCs)processing.The results of comprehension accuracy revealed a significant effect of aging in RCs processing,with older participants exhibiting poor performance compared with younger counterparts across all four clause conditions.Furthermore,younger participants demonstrated a clear animacy effect in RCs processing,but this effect was not found in older participants.Reading times indicated a prominent aging effect in clause processing,with older participants showing significantly longer reading times across all four types of RCs compared to younger participants.It was observed that processing ORCs in Chinese was relatively easier than processing SRCs.Additionally,a noticeable aging effect in semantic processing was found,specifically,the difficulties of processing SRCs and ORCs vary with the animacy configuration of the head nouns for younger participants but were not observed in older participants.In summary,aging in cognition would also inhinder semantic processing in complex sentence comprehension.展开更多
基金supported by the National Key Research and Development of China(Grant No.2022YFB3805700)the National Natural Science Foundation of China(Grant Nos.12122202 and 12372162)the Fundamental Research Funds for the Central Universities(Grant No.2024CX06021).
文摘In daily life,human need various senses to obtain information about their surroundings,and touch is one of the five major human sensing signals.Similarly,it is extremely important for robots to be endowed with tactile sensing ability.In recent years,vision-based tactile sensing technology has been the research hotspot and frontier in the field of tactile perception.Compared to conventional tactile sensing technologies,vision-based tactile sensing technologies are capable of obtaining highquality and high-resolution tactile information at a lower cost,while not being limited by the size and shape of sensors.Several previous articles have reviewed the sensing mechanism and electrical components of vision-based sensors,greatly promoting the innovation of tactile sensing.Different from existing reviews,this article concentrates on the underlying tracking method which converts real-time images into deformation information,including contact,sliding and friction.We will show the history and development of both model-based and model-free tracking methods,among which model-based approaches rely on schematic mechanical theories,and model-free approaches mainly involve machine learning algorithms.Comparing the efficiency and accuracy of existing deformation tracking methods,future research directions of vision-based tactile sensors for smart manipulations and robots are also discussed.
基金co-supported by the National Key Research and Development Program of China(No.2021YFF0601304)the National Natural Science Foundation of China(No.62206020)。
文摘Vision-based air-to-air tracking of multi-UAV swarms is crucial for effective swarm perception.UAVs in a swarm typically share similar appearance features and exhibit nonlinear motion,as they are usually of the same type.This homogeneity poses challenges for the existing multi-object tracking(MOT)algorithms,which often suffer performance degradation due to the difficulties in capturing instance-specific appearance and motion cues.In this paper,we propose a novel multiframe pose-attention-based appearance feature extraction component that captures instance-level pose features of UAVs across consecutive frames.Additionally,we introduce a motion difference accumulation strategy to extract spatial and motion cues from multiple adjacent frames.By combining these techniques,we design a multi-frame association framework that effectively distinguishes between similar UAVs in a swarm by leveraging object features over consecutive frames.To address the lack of relevant datasets,we create the AIRMOT dataset,specifically tailored for air-to-air tracking of homogeneous UAV swarms.Our method is evaluated on the AIRMOT dataset as well as the publicly available MOT-FLY and UAVSwarm datasets.The experimental results demonstrate that our approach outperforms other state-of-the-art(SOTA)methods,delivering superior tracking performance.
基金supported by the confidential research grant No.a8317。
文摘To address the issues of frequent identity switches(IDs)and degraded identification accuracy in multi object tracking(MOT)under complex occlusion scenarios,this study proposes an occlusion-robust tracking framework based on face-pedestrian joint feature modeling.By constructing a joint tracking model centered on“intra-class independent tracking+cross-category dynamic binding”,designing a multi-modal matching metric with spatio-temporal and appearance constraints,and innovatively introducing a cross-category feature mutual verification mechanism and a dual matching strategy,this work effectively resolves performance degradation in traditional single-category tracking methods caused by short-term occlusion,cross-camera tracking,and crowded environments.Experiments on the Chokepoint_Face_Pedestrian_Track test set demonstrate that in complex scenes,the proposed method improves Face-Pedestrian Matching F1 area under the curve(F1 AUC)by approximately 4 to 43 percentage points compared to several traditional methods.The joint tracking model achieves overall performance metrics of IDF1:85.1825%and MOTA:86.5956%,representing improvements of 0.91 and 0.06 percentage points,respectively,over the baseline model.Ablation studies confirm the effectiveness of key modules such as the Intersection over Area(IoA)/Intersection over Union(IoU)joint metric and dynamic threshold adjustment,validating the significant role of the cross-category identity matching mechanism in enhancing tracking stability.Our_model shows a 16.7%frame per second(FPS)drop vs.fairness of detection and re-identification in multiple object tracking(FairMOT),with its cross-category binding module adding aboute 10%overhead,yet maintains near-real-time performance for essential face-pedestrian tracking at small resolutions.
文摘This paper synchronizes control theory with computer vision by formalizing object tracking as a sequential decision-making process.A reinforcement learning(RL)agent successfully tracks an interface between two liquids,which is often a critical variable to track in many chemical,petrochemical,metallurgical,and oil industries.This method utilizes less than 100 images for creating an environment,from which the agent generates its own data without the need for expert knowledge.Unlike supervised learning(SL)methods that rely on a huge number of parameters,this approach requires far fewer parameters,which naturally reduces its maintenance cost.Besides its frugal nature,the agent is robust to environmental uncertainties such as occlusion,intensity changes,and excessive noise.From a closed-loop control context,an interface location-based deviation is chosen as the optimization goal during training.The methodology showcases RL for real-time object-tracking applications in the oil sands industry.Along with a presentation of the interface tracking problem,this paper provides a detailed review of one of the most effective RL methodologies:actor–critic policy.
文摘An important and challenging aspect of developing an intelligent transportation system is the identification of nighttime vehicles. Most accidents occur at night owing to the absence of night lighting conditions. Vehicle detection has become a vital subject for research to ensure safety and avoid accidents. New vision-based on-road nighttime vehicle detection and tracking system are suggested in this survey paper using taillight and headlight features. Using computer vision and some image processing techniques, the proposed system can identify vehicles based on taillight and headlight features. For vehicle tracking, a centroid tracking algorithm has been used. Euclidean Distance method has been used for measuring the distances between two neighboring objects and tracks the nearest neighbor. In the proposed system two flexible fixed Region of Interest (ROI) have been used, one is the Headlight ROI, and another is the Taillight ROI that could adapt to different resolutions of the images and videos. The achievement of this research work is that the proposed two ROIs can work simultaneously in a frame to identify oncoming and preceding vehicles at night. The segmentation techniques and double thresholding method have been used to extract the red and white components from the scene to identify the vehicle headlights and taillights. To evaluate the capability of the proposed process, two types of datasets have been used. Experimental findings indicate that the performance of the proposed technique is reliable and effective in distinct nighttime environments for detection and tracking of vehicles. The proposed method has been able to detect and track double lights as well as single light such as motorcycle light and achieved average accuracy and average processing time of vehicle detection about 97.22% and 0.01 s per frame respectively.
基金This work was supported in part by the National Natural Science Foundation of China(61973007,61633002).
文摘Controlling multiple multi-joint fish-like robots has long captivated the attention of engineers and biologists,for which a fundamental but challenging topic is to robustly track the postures of the individuals in real time.This requires detecting multiple robots,estimating multi-joint postures,and tracking identities,as well as processing fast in real time.To the best of our knowledge,this challenge has not been tackled in the previous studies.In this paper,to precisely track the planar postures of multiple swimming multi-joint fish-like robots in real time,we propose a novel deep neural network-based method,named TAB-IOL.Its TAB part fuses the top-down and bottom-up approaches for vision-based pose estimation,while the IOL part with long short-term memory considers the motion constraints among joints for precise pose tracking.The satisfying performance of our TAB-IOL is verified by testing on a group of freely swimming fish-like robots in various scenarios with strong disturbances and by a deed comparison of accuracy,speed,and robustness with most state-of-the-art algorithms.Further,based on the precise pose estimation and tracking realized by our TAB-IOL,several formation control experiments are conducted for the group of fish-like robots.The results clearly demonstrate that our TAB-IOL lays a solid foundation for the coordination control of multiple fish-like robots in a real working environment.We believe our proposed method will facilitate the growth and development of related fields.
基金Project supported by the National Science Fund for Distinguished Young Scholars(Grant No.T2125014)the Special Fund for Research on National Major Research Instruments of the National Natural Science Foundation of China(Grant No.11927808)the CAS Key Technology Research and Development Team Project(Grant No.GJJSTD20200005)。
文摘Atom tracking technology enhanced with innovative algorithms has been implemented in this study,utilizing a comprehensive suite of controllers and software independently developed domestically.Leveraging an on-board field-programmable gate array(FPGA)with a core frequency of 100 MHz,our system facilitates reading and writing operations across 16 channels,performing discrete incremental proportional-integral-derivative(PID)calculations within 3.4 microseconds.Building upon this foundation,gradient and extremum algorithms are further integrated,incorporating circular and spiral scanning modes with a horizontal movement accuracy of 0.38 pm.This integration enhances the real-time performance and significantly increases the accuracy of atom tracking.Atom tracking achieves an equivalent precision of at least 142 pm on a highly oriented pyrolytic graphite(HOPG)surface under room temperature atmospheric conditions.Through applying computer vision and image processing algorithms,atom tracking can be used when scanning a large area.The techniques primarily consist of two algorithms:the region of interest(ROI)-based feature matching algorithm,which achieves 97.92%accuracy,and the feature description-based matching algorithm,with an impressive 99.99%accuracy.Both implementation approaches have been tested for scanner drift measurements,and these technologies are scalable and applicable in various domains of scanning probe microscopy with broad application prospects in the field of nanoengineering.
基金financial support provided by the Natural Science Foundation of Hunan Province of China(Grant No.2021JJ10045)the Open Research Subject of State Key Laboratory of Intelligent Game(Grant No.ZBKF-24-01)+1 种基金the Postdoctoral Fellowship Program of CPSF(Grant No.GZB20240989)the China Postdoctoral Science Foundation(Grant No.2024M754304)。
文摘Unmanned aerial vehicles(UAVs)have become crucial tools in moving target tracking due to their agility and ability to operate in complex,dynamic environments.UAVs must meet several requirements to achieve stable tracking,including maintaining continuous target visibility amidst occlusions,ensuring flight safety,and achieving smooth trajectory planning.This paper reviews the latest advancements in UAV-based target tracking,highlighting information prediction,tracking strategies,and swarm cooperation.To address challenges including target visibility and occlusion,real-time prediction and tracking in dynamic environments,flight safety and coordination,resource management and energy efficiency,the paper identifies future research directions aimed at improving the performance,reliability,and scalability of UAV tracking system.
基金supported by the National Natural Science Foundation of China(Nos.51775021,52302511)the Fundamental Research Funds for the Central Universities,China(Nos.501JCGG2024129003,501JCGG2024129005,501JCGG2024129006),the Fundamental Research Funds for the Central Universities,China(No.YWF-24-JC-09)the National Key Research and Development Program of China(No.2018YFC1506401)。
文摘This paper studies the tracking control problem for stratospheric airships with userspecified performance.Dealing with the infinite gain phenomenon in the prescribed-time stability,a new stability criterion with bounded gain is proposed by using a new time-varying scaling function.Moreover,a same-side performance function and a novel barrier Lyapunov function are incorporated into the control algorithm,which can compress the feasible domain of tracking error to minimize the overshoot and solve the difficult in tracking error not converging to zero simultaneously.The proposed scheme guarantees the airship capable of operating autonomously with satisfactory transient performance and tracking accuracy,where the performance parameters can be designed artificially and link to the physical process directly.Finally,the effectiveness of the proposed control scheme is verified by theoretical analysis and numerical simulation.
基金supported by the National Natural Science Foundation of China(12072090).
文摘This work proposes the application of an iterative learning model predictive control(ILMPC)approach based on an adaptive fault observer(FOBILMPC)for fault-tolerant control and trajectory tracking in air-breathing hypersonic vehicles.In order to increase the control amount,this online control legislation makes use of model predictive control(MPC)that is based on the concept of iterative learning control(ILC).By using offline data to decrease the linearized model’s faults,the strategy may effectively increase the robustness of the control system and guarantee that disturbances can be suppressed.An adaptive fault observer is created based on the suggested ILMPC approach in order to enhance overall fault tolerance by estimating and compensating for actuator disturbance and fault degree.During the derivation process,a linearized model of longitudinal dynamics is established.The suggested ILMPC approach is likely to be used in the design of hypersonic vehicle control systems since numerical simulations have demonstrated that it can decrease tracking error and speed up convergence when compared to the offline controller.
基金supported by the National Natural Science Foundation of China(Nos.12272104,U22B2013).
文摘This paper investigates the challenges associated with Unmanned Aerial Vehicle (UAV) collaborative search and target tracking in dynamic and unknown environments characterized by limited field of view. The primary objective is to explore the unknown environments to locate and track targets effectively. To address this problem, we propose a novel Multi-Agent Reinforcement Learning (MARL) method based on Graph Neural Network (GNN). Firstly, a method is introduced for encoding continuous-space multi-UAV problem data into spatial graphs which establish essential relationships among agents, obstacles, and targets. Secondly, a Graph AttenTion network (GAT) model is presented, which focuses exclusively on adjacent nodes, learns attention weights adaptively and allows agents to better process information in dynamic environments. Reward functions are specifically designed to tackle exploration challenges in environments with sparse rewards. By introducing a framework that integrates centralized training and distributed execution, the advancement of models is facilitated. Simulation results show that the proposed method outperforms the existing MARL method in search rate and tracking performance with less collisions. The experiments show that the proposed method can be extended to applications with a larger number of agents, which provides a potential solution to the challenging problem of multi-UAV autonomous tracking in dynamic unknown environments.
文摘To realize effective co-phasing adjustment in large-aperture sparse-aperture telescopes,a multichannel stripe tracking approach is employed,allowing simultaneous interferometric measurements of multiple optical paths and circumventing the need for pairwise measurements along the mirror boundaries in traditional interferometric methods.This approach enhances detection efficiency and reduces system complexity.Here,the principles of the multibeam interference process and construction of a co-phasing detection module based on direct optical fiber connections were analyzed using wavefront optics theory.Error analysis was conducted on the system surface obtained through multipath interference.Potential applications of the interferometric method were explored.Finally,the principle was verified by experiment,an interferometric fringe contrast better than 0.4 is achieved through flat field calibration and incoherent digital synthesis.The dynamic range of the measurement exceeds 10 times of the center wavelength of the working band(1550 nm).Moreover,a resolution better than one-tenth of the working center wavelength(1550 nm)was achieved.Simultaneous three-beam interference can be achieved,leading to a 50%improvement in detection efficiency.This method can effectively enhance the efficiency of sparse aperture telescope co-phasing,meeting the requirements for observations of 8-10 m telescopes.This study provides a technological foundation for observing distant and faint celestial objects.
基金supported by the National Natural Science Foundation of China (No.62202137)the China Postdoctoral Science Foundation (No.2023M730599)the Zhejiang Provincial Natural Science Foundation of China (No.LMS25F020009)。
文摘Embodied visual exploration is critical for building intelligent visual agents. This paper presents the neural exploration with feature-based visual odometry and tracking-failure-reduction policy(Ne OR), a framework for embodied visual exploration that possesses the efficient exploration capabilities of deep reinforcement learning(DRL)-based exploration policies and leverages feature-based visual odometry(VO) for more accurate mapping and positioning results. An improved local policy is also proposed to reduce tracking failures of feature-based VO in weakly textured scenes through a refined multi-discrete action space, keyframe fusion, and an auxiliary task. The experimental results demonstrate that Ne OR has better mapping and positioning accuracy compared to other entirely learning-based exploration frameworks and improves the robustness of feature-based VO by significantly reducing tracking failures in weakly textured scenes.
基金supported in part by the Doctoral Initiation Fund of Nanchang Hangkong University(No.EA202403107)Jiangxi Province Early Career Youth Science and Technology Talent Training Project(No.CK202403509).
文摘This paper presents the design and ground verification for vision-based relative navigation systems of microsatellites,which offers a comprehensive hardware design solution and a robust experimental verification methodology for practical implementation of vision-based navigation technology on the microsatellite platform.Firstly,a low power consumption,light weight,and high performance vision-based relative navigation optical sensor is designed.Subsequently,a set of ground verification system is designed for the hardware-in-the-loop testing of the vision-based relative navigation systems.Finally,the designed vision-based relative navigation optical sensor and the proposed angles-only navigation algorithms are tested on the ground verification system.The results verify that the optical simulator after geometrical calibration can meet the requirements of the hardware-in-the-loop testing of vision-based relative navigation systems.Based on experimental results,the relative position accuracy of the angles-only navigation filter at terminal time is increased by 25.5%,and the relative speed accuracy is increased by 31.3% compared with those of optical simulator before geometrical calibration.
文摘The forthcoming sixth generation(6G)of mobile communication networks is envisioned to be AInative,supporting intelligent services and pervasive computing at unprecedented scale.Among the key paradigms enabling this vision,Federated Learning(FL)has gained prominence as a distributed machine learning framework that allows multiple devices to collaboratively train models without sharing raw data,thereby preserving privacy and reducing the need for centralized storage.This capability is particularly attractive for vision-based applications,where image and video data are both sensitive and bandwidth-intensive.However,the integration of FL with 6G networks presents unique challenges,including communication bottlenecks,device heterogeneity,and trade-offs between model accuracy,latency,and energy consumption.In this paper,we developed a simulation-based framework to investigate the performance of FL in representative vision tasks under 6G-like environments.We formalize the system model,incorporating both the federated averaging(FedAvg)training process and a simplified communication costmodel that captures bandwidth constraints,packet loss,and variable latency across edge devices.Using standard image datasets(e.g.,MNIST,CIFAR-10)as benchmarks,we analyze how factors such as the number of participating clients,degree of data heterogeneity,and communication frequency influence convergence speed and model accuracy.Additionally,we evaluate the effectiveness of lightweight communication-efficient strategies,including local update tuning and gradient compression,in mitigating network overhead.The experimental results reveal several key insights:(i)communication limitations can significantly degrade FL convergence in vision tasks if not properly addressed;(ii)judicious tuning of local training epochs and client participation levels enables notable improvements in both efficiency and accuracy;and(iii)communication-efficient FL strategies provide a promising pathway to balance performance with the stringent latency and reliability requirements expected in 6G.These findings highlight the synergistic role of AI and nextgeneration networks in enabling privacy-preserving,real-time vision applications,and they provide concrete design guidelines for researchers and practitioners working at the intersection of FL and 6G.
基金supported by the Science and Technology Commission of Shanghai Municipality(21DZ1100500)the Shanghai Municipal Science and Technology Major Project+2 种基金the Shanghai Frontiers Science Center Program(2021–2025 No.20)The National Natural Science Foundation of China(32471545)the Natural Science Foundation of Shanghai(24ZR1454300).
文摘Unlike ensemble-averaging measurements,single-molecule tracking provides quantitative information on the kinetics of individual molecules within living cells in real time and may provide insight into the respective molecular interactions behind that.The advancement of single-molecule tracking has been signi-cantly boosted by the development of high-resolution microscopy techniques.In this review,we will discuss this aspect with a particular focus on their recent advance in MINFLUX nanoscopy with feedback approaches where tracking is performed in real time.MINFLUX localization requires fewer than 100 photons from a-1 nm-sized°uorophore,enabling precise tracking.This approach,which demands over an order of magnitude fewer photons than other localization-based techniques(such as STORM,PLAM),allows molecular tracking with single-digit nanometer accuracy in less than 1 ms—an achievement previously unattainable.
基金supported by funding from the National Institution of Neurological Disorders and Stroke(1R41NS103698-01A1 to CLM)。
文摘Background:Early identification of concussion-related vision disorders(CRVDs)may improve outcomes by enabling earlier management,referral,and treatment.Objective eye tracking may provide additional data to support the diagnose of CRVDs.The purpose of this study was to determine the utility of objective infrared eye tracking in identifying CRVDs among adolescents experiencing persisting post-concussive symptoms(PPCS)more than 28 days after injury.Methods:This was a prospective study of adolescents with PPCS evaluated with visio-vestibular examination(VVE),comprehensive vision examination,and an eye tracking device.Results:Of the 108 adolescents enrolled,67(62%)were diagnosed with a CRVD by comprehensive vision examination.On VVE,the near point of convergence break(5.5±3.2 cm vs.3.9±1.7 cm(mean±SD),p<0.001)and recovery(8.1±3.3 cm vs.6.8±2.3 cm,p=0.02)distinguished between those with and without CRVD.Concussion symptom provocation on VVE with horizontal saccades(35(52%)vs.12(29%),p=0.02)and horizontal vestibulo-ocular reflex testing(37(55%)vs.14(34%),p=0.03),and sway on tandem gait under the forward eyes closed condition(25(37%)vs.6(15%),p=0.01)also identified those with CRVD.From the eye tracking device,the BOX score(8.1±5.8 vs.5.2±4.1,p=0.007)and a metric of the left eye tracking along the bottom of the visual target(0.094±0.500 vs.-0.124±0.410,p=0.02)identified those with CRVD,with a multivariable receiver operating characteristic curve analysis,including the BOX score,achieving an area under the receiver operating characteristic curve of 0.7637.Conclusion:CRVDs are common in those with PPCS,with impact on recovery after concussion.Novel eye-tracking metrics can serve as an aid in the identification of those with CRVDs who would benefit from referral for comprehensive diagnosis and treatment.
文摘The publisher regrets that the Appendix A.Supplementary data was not updated as per author and editor’s request.The publisher would like to apologise for any inconvenience caused.
基金supported by the National Research Foundation,Singapore,under its Medium-Sized Center for Advanced Robotics Technology Innovation(CARTIN)。
文摘Multi-axle Swerve-drive Autonomous Mobile Robots(MS-AMRs)equipped with independently steerable wheels are commonly used for high-payload transportation.In this work,we present a novel Model Predictive Control(MPC)method for MS-AGV trajectory tracking that takes tire wear minimization consideration in the objective function.To speed up the problem-solving process,we propose a hierarchical controller design and simplify the dynamic model by integrating the magic formula tire model and simplified tire wear model.In the experiment,the proposed method can be solved by simulated annealing in real-time on a normal personal computer and by incorporating tire wear into the objective function,tire wear is reduced by 19.19%while maintaining the tracking accuracy in curve-tracking experiments.In the more challenging scene:the desired trajectory is offset by 60 degrees from the vehicle's heading,the reduction in tire wear increased to 65.20%compared to the kinematic model without considering the tire wear optimization.
基金supported by the National Social Science Foundation of China(Grant No.24BYY117).
文摘Syntax and semantics are two important factors that influence sentence processing.Studies have found different aging effects in syntactic and semantic processing during sentence comprehension.While there is consensus on the aging effects in syntactic processing,the presence of aging in semantic processing remains debated.The present study aimed to explore whether there were aging effects in lexical-semantic information processing in complex sentence.79 participants were recruited to take part in this study,including 40 younger adults(mean age of 21.1±1.19 years)and 39 older adults(mean age of 66.24±3.02 years).Using eye-movement tracking technology and manipulating the animacy of head nouns in Chinese subject relative clauses(SRCs)and object relative clauses(ORCs),we investigated the abilities of young and old adults in relative clauses(RCs)processing.The results of comprehension accuracy revealed a significant effect of aging in RCs processing,with older participants exhibiting poor performance compared with younger counterparts across all four clause conditions.Furthermore,younger participants demonstrated a clear animacy effect in RCs processing,but this effect was not found in older participants.Reading times indicated a prominent aging effect in clause processing,with older participants showing significantly longer reading times across all four types of RCs compared to younger participants.It was observed that processing ORCs in Chinese was relatively easier than processing SRCs.Additionally,a noticeable aging effect in semantic processing was found,specifically,the difficulties of processing SRCs and ORCs vary with the animacy configuration of the head nouns for younger participants but were not observed in older participants.In summary,aging in cognition would also inhinder semantic processing in complex sentence comprehension.
