The impact of location services on people’s lives has grown significantly in the era of widespread smart device usage.Due to global navigation satellite system(GNSS)signal rejection,weak signal strength in indoor env...The impact of location services on people’s lives has grown significantly in the era of widespread smart device usage.Due to global navigation satellite system(GNSS)signal rejection,weak signal strength in indoor environments and radio signal interference caused by multiwall environments,which collectively lead to significant positioning errors,vision-based positioning has emerged as a crucial method in indoor positioning research.This paper introduces a scale hierarchical matching model to tackle challenges associated with large visual databases and high scene similarity,both of which will compromise matching accuracy and lead to prolonged positioning delays.The proposed model establishes an image feature database using GIST features and speeded up robust feature(SURF)in the offline stage.In the online stage,a positioning navigating algorithm is constructed based on Dijkstra’s path planning.Additionally,a corresponding Android application has been developed to facilitate visual positioning and navigation in indoor environments.Experimental results obtained in real indoor environments demonstrate that the proposed method significantly enhances positioning accuracy compared with similar algorithms,while effectively reducing time overhead.This improvement caters to the requirements for indoor positioning and navigation,thereby meeting user needs.展开更多
Lunar Laser Ranging has extremely high requirements for the pointing accuracy of the telescopes used.To improve its pointing accuracy and solve the problem of insufficiently accurate telescope pointing correction achi...Lunar Laser Ranging has extremely high requirements for the pointing accuracy of the telescopes used.To improve its pointing accuracy and solve the problem of insufficiently accurate telescope pointing correction achieved by tracking stars in the all-sky region,we propose a processing scheme to select larger-sized lunar craters near the Lunar Corner Cube Retroreflector as reference features for telescope pointing bias computation.Accurately determining the position of the craters in the images is crucial for calculating the pointing bias;therefore,we propose a method for accurately calculating the crater position based on lunar surface feature matching.This method uses matched feature points obtained from image feature matching,using a deep learning method to solve the image transformation matrix.The known position of a crater in a reference image is mapped using this matrix to calculate the crater position in the target image.We validate this method using craters near the Lunar Corner Cube Retroreflectors of Apollo 15 and Luna 17 and find that the calculated position of a crater on the target image falls on the center of the crater,even for image features with large distortion near the lunar limb.The maximum image matching error is approximately 1″,and the minimum is only 0.47″,which meets the pointing requirements of Lunar Laser Ranging.This method provides a new technical means for the high-precision pointing bias calculation of the Lunar Laser Ranging system.展开更多
Accurate and robust navigation in complex surgical environments is crucial for bronchoscopic surgeries.This study purposes a bronchoscopic lumen feature matching network(BLFM-Net)based on deep learning to address the ...Accurate and robust navigation in complex surgical environments is crucial for bronchoscopic surgeries.This study purposes a bronchoscopic lumen feature matching network(BLFM-Net)based on deep learning to address the challenges of image noise,anatomical complexity,and the stringent real-time requirements.The BLFM-Net enhances bronchoscopic image processing by integrating several functional modules.The FFA-Net preprocessing module mitigates image fogging and improves visual clarity for subsequent processing.The feature extraction module derives multi-dimensional features,such as centroids,area,and shape descriptors,from dehazed images.The Faster RCNN Object detection module detects bronchial regions of interest and generates bounding boxes to localize key areas.The feature matching module accelerates the process by combining detection boxes,extracted features,and a KD-Tree(K-Dimensional Tree)-based algorithm,ensuring efficient and accurate regional feature associations.The BLFM-Net was evaluated on 5212 bronchoscopic images,demonstrating superior performance compared to traditional and other deep learning-based image matching methods.It achieved real-time matching with an average frame time of 6 ms,with a matching accuracy of over 96%.The method remained robust under challenging conditions including frame dropping(0,5,10,20),shadowed regions,and variable lighting,maintaining accuracy of above 94%even with the frame dropping of 20.This study presents BLFM-Net,a deep learning-based matching network designed to enhance and match bronchial features in bronchoscopic images.The BLFM-Net shows improved accuracy,real-time performance,and reliability,making a valuable tool for bronchoscopic surgeries.展开更多
Feature matching plays a key role in computer vision. However, due to the limitations of the descriptors, the putative matches are inevitably contaminated by massive outliers.This paper attempts to tackle the outlier ...Feature matching plays a key role in computer vision. However, due to the limitations of the descriptors, the putative matches are inevitably contaminated by massive outliers.This paper attempts to tackle the outlier filtering problem from two aspects. First, a robust and efficient graph interaction model,is proposed, with the assumption that matches are correlated with each other rather than independently distributed. To this end, we construct a graph based on the local relationships of matches and formulate the outlier filtering task as a binary labeling energy minimization problem, where the pairwise term encodes the interaction between matches. We further show that this formulation can be solved globally by graph cut algorithm. Our new formulation always improves the performance of previous localitybased method without noticeable deterioration in processing time,adding a few milliseconds. Second, to construct a better graph structure, a robust and geometrically meaningful topology-aware relationship is developed to capture the topology relationship between matches. The two components in sum lead to topology interaction matching(TIM), an effective and efficient method for outlier filtering. Extensive experiments on several large and diverse datasets for multiple vision tasks including general feature matching, as well as relative pose estimation, homography and fundamental matrix estimation, loop-closure detection, and multi-modal image matching, demonstrate that our TIM is more competitive than current state-of-the-art methods, in terms of generality, efficiency, and effectiveness. The source code is publicly available at http://github.com/YifanLu2000/TIM.展开更多
Three-dimensional(3D)reconstruction based on aerial images has broad prospects,and feature matching is an important step of it.However,for high-resolution aerial images,there are usually problems such as long time,mis...Three-dimensional(3D)reconstruction based on aerial images has broad prospects,and feature matching is an important step of it.However,for high-resolution aerial images,there are usually problems such as long time,mismatching and sparse feature pairs using traditional algorithms.Therefore,an algorithm is proposed to realize fast,accurate and dense feature matching.The algorithm consists of four steps.Firstly,we achieve a balance between the feature matching time and the number of matching pairs by appropriately reducing the image resolution.Secondly,to realize further screening of the mismatches,a feature screening algorithm based on similarity judgment or local optimization is proposed.Thirdly,to make the algorithm more widely applicable,we combine the results of different algorithms to get dense results.Finally,all matching feature pairs in the low-resolution images are restored to the original images.Comparisons between the original algorithms and our algorithm show that the proposed algorithm can effectively reduce the matching time,screen out the mismatches,and improve the number of matches.展开更多
Domain adaptation and adversarial networks are two main approaches for transfer learning.Domain adaptation methods match the mean values of source and target domains,which requires a very large batch size during train...Domain adaptation and adversarial networks are two main approaches for transfer learning.Domain adaptation methods match the mean values of source and target domains,which requires a very large batch size during training.However,adversarial networks are usually unstable when training.In this paper,we propose a joint method of feature matching and adversarial networks to reduce domain discrepancy and mine domaininvariant features from the local and global aspects.At the same time,our method improves the stability of training.Moreover,the method is embedded into a unified convolutional neural network that can be easily optimized by gradient descent.Experimental results show that our joint method can yield the state-of-the-art results on three common public datasets.展开更多
Feature matching is of significance in the field of computer vision.In this paper,a trifocal tensor based feature matching algorithm is proposed for three views,including a trinocular vision system.Initial matching po...Feature matching is of significance in the field of computer vision.In this paper,a trifocal tensor based feature matching algorithm is proposed for three views,including a trinocular vision system.Initial matching point-pairs can be determined according to generic matching algorithms,on which an initial trifocal tensor of three views can be confirmed.Then the initial matching point-pairs should be re-selected.Meanwhile,the trifocal tensor will be recomputed.Iteratively,the optimized trifocal tensor can be obtained.Compatible fundamental matrix of every two views can be determined.Furthermore,in the trinocular vision sensor,the trifocal tensor can be calculated based on the intrinsic parameter matrix of each camera.With the strict constraint provided by the trifocal tensor,feature matching results will be optimized.Experiments show that our proposed algorithm has the characteristics of feasibility and precision.展开更多
In this paper an automatic visual method of seam recognizing and seam tracking based on textural feature matching was proposed, in order to recognize the weld of multi-layer or multi-pass welding in which the weld is ...In this paper an automatic visual method of seam recognizing and seam tracking based on textural feature matching was proposed, in order to recognize the weld of multi-layer or multi-pass welding in which the weld is difficult to be recognized by conventional visual methods. This method focuses on the obvious difference of image textural feature between the weld region and the base metal region, as well as the similarity of the textural features along the welding direction. The method consists of the following steps ： setting image template and choosing the edge region as ROI （ region of interest ）, extracting the image textural feature of the template and the edge region, feature matching, and recognition of weld region. Experiment showed that the method proposed was effective for weld seam recognition in multi-layer welding.展开更多
The optimal planning and operation of multi-type flexible resources(FRs)are critical prerequisites for maintaining power and energy balance in regional power grids with a high proportion of clean energy.However,insuff...The optimal planning and operation of multi-type flexible resources(FRs)are critical prerequisites for maintaining power and energy balance in regional power grids with a high proportion of clean energy.However,insufficient consideration of the multi-dimensional and heterogeneous features of FRs,such as the regulation characteristics of diversified battery energy storage systems(BESSs),poses a challenge in economically relieving imbalance power and adequately sharing feature information between power supply and demand.In view of this disadvantage,an optimal planning and operation method based on differentiated feature matching through response capability characterization and difference quantification of FRs is proposed in this paper.In the planning stage,a model for the optimal planning of diversified energy storages(ESs)including Lithium-ion battery(Li-B),supercapacitor energy storage(SCES),compressed air energy storage(CAES),and pumped hydroelectric storage(PHS)is established.Subsequently,in the operating stage,the potential,direction,and cost of FR response behaviors are refined to match with the power and energy balance demand(PEBD)of power grid operation.An optimal operating algorithm is then employed to quantify the feature differences and output response sequences of multi-type FRs.The performance and effectiveness of the proposed method are demonstrated through comparative studies conducted on an actual regional power grid in northwest China.Analysis and simulation results illustrate that the proposed method can effectively highlight the advantages of BESSs compared with other ESs,and economically reduce imbalance power of the regional power grid under practical operating conditions.展开更多
The traditional Chinese-English translation model tends to translate some source words repeatedly,while mistakenly ignoring some words.Therefore,we propose a novel English-Chinese neural machine translation based on s...The traditional Chinese-English translation model tends to translate some source words repeatedly,while mistakenly ignoring some words.Therefore,we propose a novel English-Chinese neural machine translation based on self-organizing mapping neural network and deep feature matching.In this model,word vector,two-way LSTM,2D neural network and other deep learning models are used to extract the semantic matching features of question-answer pairs.Self-organizing mapping(SOM)is used to classify and identify the sentence feature.The attention mechanism-based neural machine translation model is taken as the baseline system.The experimental results show that this framework significantly improves the adequacy of English-Chinese machine translation and achieves better results than the traditional attention mechanism-based English-Chinese machine translation model.展开更多
In minimally invasive surgery,endoscopes or laparoscopes equipped with miniature cameras and tools are used to enter the human body for therapeutic purposes through small incisions or natural cavities.However,in clini...In minimally invasive surgery,endoscopes or laparoscopes equipped with miniature cameras and tools are used to enter the human body for therapeutic purposes through small incisions or natural cavities.However,in clinical operating environments,endoscopic images often suffer from challenges such as low texture,uneven illumination,and non-rigid structures,which affect feature observation and extraction.This can severely impact surgical navigation or clinical diagnosis due to missing feature points in endoscopic images,leading to treatment and postoperative recovery issues for patients.To address these challenges,this paper introduces,for the first time,a Cross-Channel Multi-Modal Adaptive Spatial Feature Fusion(ASFF)module based on the lightweight architecture of EfficientViT.Additionally,a novel lightweight feature extraction and matching network based on attention mechanism is proposed.This network dynamically adjusts attention weights for cross-modal information from grayscale images and optical flow images through a dual-branch Siamese network.It extracts static and dynamic information features ranging from low-level to high-level,and from local to global,ensuring robust feature extraction across different widths,noise levels,and blur scenarios.Global and local matching are performed through a multi-level cascaded attention mechanism,with cross-channel attention introduced to simultaneously extract low-level and high-level features.Extensive ablation experiments and comparative studies are conducted on the HyperKvasir,EAD,M2caiSeg,CVC-ClinicDB,and UCL synthetic datasets.Experimental results demonstrate that the proposed network improves upon the baseline EfficientViT-B3 model by 75.4%in accuracy(Acc),while also enhancing runtime performance and storage efficiency.When compared with the complex DenseDescriptor feature extraction network,the difference in Acc is less than 7.22%,and IoU calculation results on specific datasets outperform complex dense models.Furthermore,this method increases the F1 score by 33.2%and accelerates runtime by 70.2%.It is noteworthy that the speed of CMMCAN surpasses that of comparative lightweight models,with feature extraction and matching performance comparable to existing complex models but with faster speed and higher cost-effectiveness.展开更多
Feature detection and matching play important roles in many fields of computer vision, such as image understanding, feature recognition, 3D-reconstruction, video analysis, etc. Extracting features is usually the first...Feature detection and matching play important roles in many fields of computer vision, such as image understanding, feature recognition, 3D-reconstruction, video analysis, etc. Extracting features is usually the first step for feature detection or matching, and the gradient feature is one of the most used selections. In this paper, a new image feature-absence importance （AI） feature, which can directly characterize the local structure information, is proposed. Greatly different from the most existing features, the proposed absence importance feature is mainly based on the consideration that the absence of the important pixel will have a great effect on the local structure. Two absence importance features, mean absence importance （MAI） and standard deviation absence importance （SDAI）, are defined and used subsequently to construct new algorithms for feature detection and matching. Experiments demonstrate that the proposed absence importance features can be used as an important complement of the gradient feature and applied successfully to the fields of feature detection and matching.展开更多
Feature recognition and surface reconstruction from point clouds are difficulties in reverse engineering. A new surface reconstruction algorithm for slicing point cloud was presented. The contours of slice were extrac...Feature recognition and surface reconstruction from point clouds are difficulties in reverse engineering. A new surface reconstruction algorithm for slicing point cloud was presented. The contours of slice were extracted. Then, the intersection of two adjacent curve segments in the contour was obtained and curves feature was extracted. Finally, adjacent section contours were matched directly with Fourier-Mellin curve matching method for feature extraction. An example of 3-D model reconstruction shows the reliability and application of the algorithm.展开更多
A robust and eficient feature matching method is necessary for visual navigation in asteroid-landing missions.Based on the visual navigation framework and motion characteristics of asteroids,a robust and efficient tem...A robust and eficient feature matching method is necessary for visual navigation in asteroid-landing missions.Based on the visual navigation framework and motion characteristics of asteroids,a robust and efficient template feature matching method is proposed to adapt to feature distortion and scale change cases for visual navigation of asteroids.The proposed method is primarily based on a motion-constrained discriminative correlation filter(DCF).The prior information provided by the motion constraints between sequence images is used to provide a predicted search region for template feature matching.Additionally,some specific template feature samples are generated using the motion constraints for correlation filter learning,which is beneficial for training a scale and feature distortion adaptive correlation filter for accurate feature matching.Moreover,average peak-to-correlation energy(APCE)and jointly consistent measurements(JCMs)were used to eliminate false matching.Images captured by the Touch And Go Camera System(TAGCAMS)of the Bennu asteroid were used to evaluate the performance of the proposed method.In particular,both the robustness and accuracy of region matching and template center matching are evaluated.The qualitative and quantitative results illustrate the advancement of the proposed method in adapting to feature distortions and large-scale changes during spacecraft landing.展开更多
We present GeoGlue,a novel method using high-resolution UAV imagery for accurate feature matching,which is normally challenging due to the complicated scenes.Current feature detection methods are performed without gui...We present GeoGlue,a novel method using high-resolution UAV imagery for accurate feature matching,which is normally challenging due to the complicated scenes.Current feature detection methods are performed without guidance of geometric priors(e.g.,geometric lines),lacking enough attention given to salient geometric features which are indispensable for accurate matching due to their stable existence across views.In this work,geometric lines arefirstly detected by a CNN-based geometry detector(GD)which is pre-trained in a self-supervised manner through automatically generated images.Then,geometric lines are naturally vectorized based on GD and thus non-significant features can be disregarded as judged by their disordered geometric morphology.A graph attention network(GAT)is utilized forfinal feature matching,spanning across the image pair with geometric priors informed by GD.Comprehensive experiments show that GeoGlue outperforms other state-of-the-art methods in feature-matching accuracy and performance stability,achieving pose estimation with maximum rotation and translation errors under 1%in challenging scenes from benchmark datasets,Tanks&Temples and ETH3D.This study also proposes thefirst self-supervised deep-learning model for curved line detection,generating geometric priors for matching so that more attention is put on prominent features and improving the visual effect of 3D reconstruction.展开更多
This paper puts forward a method for abdomen panorama reconstruction based on a stereo vision system. For the purpose of recovering the abdomen completely and accurately under the condition of actual photographing wit...This paper puts forward a method for abdomen panorama reconstruction based on a stereo vision system. For the purpose of recovering the abdomen completely and accurately under the condition of actual photographing with illumination variance and blur noise, some innovative combined feature descriptors are presented on the basis of Hu-moment invariants. Furthermore, considering the study on the abdomen surface reconstruction, a circle template which is divided into 6 sectors is designed. It is noted that a descriptor merely using gray intensity is not able to provide sufficient information for feature description. Consequently, the sector entropy which denotes the structure characteristics is drawn into the feature descriptor. By means of the combined effect of the gray intensity and the sector entropy, the similarity measurement is conducted for the final abdomen reconstruction. The experimental results reveal that the proposed method can acquire a high precision of abdomen reconstruction similar to the 3D scanner. This stereo vision system has wide practicability in the field of clothing.展开更多
Individual identification of dairy cows is the prerequisite for automatic analysis and intelligent perception of dairy cows'behavior.At present,individual identification of dairy cows based on deep convolutional n...Individual identification of dairy cows is the prerequisite for automatic analysis and intelligent perception of dairy cows'behavior.At present,individual identification of dairy cows based on deep convolutional neural network had the disadvantages in prolonged training at the additions of new cows samples.Therefore,a cow individual identification framework was proposed based on deep feature extraction and matching,and the individual identification of dairy cows based on this framework could avoid repeated training.Firstly,the trained convolutional neural network model was used as the feature extractor;secondly,the feature extraction was used to extract features and stored the features into the template feature library to complete the enrollment;finally,the identifies of dairy cows were identified.Based on this framework,when new cows joined the herd,enrollment could be completed quickly.In order to evaluate the application performance of this method in closed-set and open-set individual identification of dairy cows,back images of 524 cows were collected,among which the back images of 150 cows were selected as the training data to train feature extractor.The data of the remaining 374 cows were used to generate the template data set and the data to be identified.The experiment results showed that in the closed-set individual identification of dairy cows,the highest identification accuracy of top-1 was 99.73%,the highest identification accuracy from top-2 to top-5 was 100%,and the identification time of a single cow was 0.601 s,this method was verified to be effective.In the open-set individual identification of dairy cows,the recall was 90.38%,and the accuracy was 89.46%.When false accept rate(FAR)=0.05,true accept rate(TAR)=84.07%,this method was verified that the application had certain research value in open-set individual identification of dairy cows,which provided a certain idea for the application of individual identification in the field of intelligent animal husbandry.展开更多
Augmented solar images were used to research the adaptability of four representative image extraction and matching algorithms in space weather domain.These include the scale-invariant feature transform algorithm,speed...Augmented solar images were used to research the adaptability of four representative image extraction and matching algorithms in space weather domain.These include the scale-invariant feature transform algorithm,speeded-up robust features algorithm,binary robust invariant scalable keypoints algorithm,and oriented fast and rotated brief algorithm.The performance of these algorithms was estimated in terms of matching accuracy,feature point richness,and running time.The experiment result showed that no algorithm achieved high accuracy while keeping low running time,and all algorithms are not suitable for image feature extraction and matching of augmented solar images.To solve this problem,an improved method was proposed by using two-frame matching to utilize the accuracy advantage of the scale-invariant feature transform algorithm and the speed advantage of the oriented fast and rotated brief algorithm.Furthermore,our method and the four representative algorithms were applied to augmented solar images.Our application experiments proved that our method achieved a similar high recognition rate to the scale-invariant feature transform algorithm which is significantly higher than other algorithms.Our method also obtained a similar low running time to the oriented fast and rotated brief algorithm,which is significantly lower than other algorithms.展开更多
Aming at the problem of the low accuracy of low dynamic vehicle velocity under the environment of uneven distribution of light intensity,an improved adaptive Kalman filter method for the velocity error estimate by the...Aming at the problem of the low accuracy of low dynamic vehicle velocity under the environment of uneven distribution of light intensity,an improved adaptive Kalman filter method for the velocity error estimate by the fusion of optical flow tracking and scale mvaiant feature transform(SIFT)is proposed.The algorithm introduces anonlinear fuzzy membership function and the filter residual for the noise covariance matrix in the adaptive adjustment process.In the process of calculating the velocity of the vehicle,the tracking and matching of the inter-frame displacement a d the vehicle velocity calculation a e carried out by using the optical fow tracing and the SIF'T methods,respectively.Meanwhile,the velocity difference between theoutputs of thesetwo methods is used as the observation of the improved adaptive Kalman filter.Finally,the velocity calculated by the optical fow method is corrected by using the velocity error estimate of the output of the modified adaptive Kalman filter.The results of semi-physical experiments show that the maximum velocityeror of the fusion algorithm is decreased by29%than that of the optical fow method,and the computation time is reduced by80%compared with the SIFT method.展开更多
The ORB-SLAM2 based on the constant velocity model is difficult to determine the search window of the reprojection of map points when the objects are in variable velocity motion,which leads to a false matching,with an...The ORB-SLAM2 based on the constant velocity model is difficult to determine the search window of the reprojection of map points when the objects are in variable velocity motion,which leads to a false matching,with an inaccurate pose estimation or failed tracking.To address the challenge above,a new method of feature point matching is proposed in this paper,which combines the variable velocity model with the reverse optical flow method.First,the constant velocity model is extended to a new variable velocity model,and the expanded variable velocity model is used to provide the initial pixel shifting for the reverse optical flow method.Then the search range of feature points is accurately determined according to the results of the reverse optical flow method,thereby improving the accuracy and reliability of feature matching,with strengthened interframe tracking effects.Finally,we tested on TUM data set based on the RGB-D camera.Experimental results show that this method can reduce the probability of tracking failure and improve localization accuracy on SLAM(Simultaneous Localization and Mapping)systems.Compared with the traditional ORB-SLAM2,the test error of this method on each sequence in the TUM data set is significantly reduced,and the root mean square error is only 63.8%of the original system under the optimal condition.展开更多
基金Supported by the National Natural Science Foundation of China(No.61971162,61771186)the Natural Science Foundation of Heilongjiang Province(No.PL2024F025)+1 种基金the Open Research Fund of National Mobile Communications Research Laboratory in Southeast University(No.2023D07)the Fundamental Scientific Research Funds of Heilongjiang Province(No.2022-KYYWF-1050).
文摘The impact of location services on people’s lives has grown significantly in the era of widespread smart device usage.Due to global navigation satellite system(GNSS)signal rejection,weak signal strength in indoor environments and radio signal interference caused by multiwall environments,which collectively lead to significant positioning errors,vision-based positioning has emerged as a crucial method in indoor positioning research.This paper introduces a scale hierarchical matching model to tackle challenges associated with large visual databases and high scene similarity,both of which will compromise matching accuracy and lead to prolonged positioning delays.The proposed model establishes an image feature database using GIST features and speeded up robust feature(SURF)in the offline stage.In the online stage,a positioning navigating algorithm is constructed based on Dijkstra’s path planning.Additionally,a corresponding Android application has been developed to facilitate visual positioning and navigation in indoor environments.Experimental results obtained in real indoor environments demonstrate that the proposed method significantly enhances positioning accuracy compared with similar algorithms,while effectively reducing time overhead.This improvement caters to the requirements for indoor positioning and navigation,thereby meeting user needs.
基金funded by Natural Science Foundation of Jilin Province(20220101125JC)the National Natural Science Foundation of China(12273079).
文摘Lunar Laser Ranging has extremely high requirements for the pointing accuracy of the telescopes used.To improve its pointing accuracy and solve the problem of insufficiently accurate telescope pointing correction achieved by tracking stars in the all-sky region,we propose a processing scheme to select larger-sized lunar craters near the Lunar Corner Cube Retroreflector as reference features for telescope pointing bias computation.Accurately determining the position of the craters in the images is crucial for calculating the pointing bias;therefore,we propose a method for accurately calculating the crater position based on lunar surface feature matching.This method uses matched feature points obtained from image feature matching,using a deep learning method to solve the image transformation matrix.The known position of a crater in a reference image is mapped using this matrix to calculate the crater position in the target image.We validate this method using craters near the Lunar Corner Cube Retroreflectors of Apollo 15 and Luna 17 and find that the calculated position of a crater on the target image falls on the center of the crater,even for image features with large distortion near the lunar limb.The maximum image matching error is approximately 1″,and the minimum is only 0.47″,which meets the pointing requirements of Lunar Laser Ranging.This method provides a new technical means for the high-precision pointing bias calculation of the Lunar Laser Ranging system.
基金funded by the National Natural Science Foundation of China(Grant No.52175028).
文摘Accurate and robust navigation in complex surgical environments is crucial for bronchoscopic surgeries.This study purposes a bronchoscopic lumen feature matching network(BLFM-Net)based on deep learning to address the challenges of image noise,anatomical complexity,and the stringent real-time requirements.The BLFM-Net enhances bronchoscopic image processing by integrating several functional modules.The FFA-Net preprocessing module mitigates image fogging and improves visual clarity for subsequent processing.The feature extraction module derives multi-dimensional features,such as centroids,area,and shape descriptors,from dehazed images.The Faster RCNN Object detection module detects bronchial regions of interest and generates bounding boxes to localize key areas.The feature matching module accelerates the process by combining detection boxes,extracted features,and a KD-Tree(K-Dimensional Tree)-based algorithm,ensuring efficient and accurate regional feature associations.The BLFM-Net was evaluated on 5212 bronchoscopic images,demonstrating superior performance compared to traditional and other deep learning-based image matching methods.It achieved real-time matching with an average frame time of 6 ms,with a matching accuracy of over 96%.The method remained robust under challenging conditions including frame dropping(0,5,10,20),shadowed regions,and variable lighting,maintaining accuracy of above 94%even with the frame dropping of 20.This study presents BLFM-Net,a deep learning-based matching network designed to enhance and match bronchial features in bronchoscopic images.The BLFM-Net shows improved accuracy,real-time performance,and reliability,making a valuable tool for bronchoscopic surgeries.
基金supported by the National Natural Science Foundation of China (62276192)。
文摘Feature matching plays a key role in computer vision. However, due to the limitations of the descriptors, the putative matches are inevitably contaminated by massive outliers.This paper attempts to tackle the outlier filtering problem from two aspects. First, a robust and efficient graph interaction model,is proposed, with the assumption that matches are correlated with each other rather than independently distributed. To this end, we construct a graph based on the local relationships of matches and formulate the outlier filtering task as a binary labeling energy minimization problem, where the pairwise term encodes the interaction between matches. We further show that this formulation can be solved globally by graph cut algorithm. Our new formulation always improves the performance of previous localitybased method without noticeable deterioration in processing time,adding a few milliseconds. Second, to construct a better graph structure, a robust and geometrically meaningful topology-aware relationship is developed to capture the topology relationship between matches. The two components in sum lead to topology interaction matching(TIM), an effective and efficient method for outlier filtering. Extensive experiments on several large and diverse datasets for multiple vision tasks including general feature matching, as well as relative pose estimation, homography and fundamental matrix estimation, loop-closure detection, and multi-modal image matching, demonstrate that our TIM is more competitive than current state-of-the-art methods, in terms of generality, efficiency, and effectiveness. The source code is publicly available at http://github.com/YifanLu2000/TIM.
基金This work was supported by the Equipment Pre-Research Foundation of China(6140001020310).
文摘Three-dimensional(3D)reconstruction based on aerial images has broad prospects,and feature matching is an important step of it.However,for high-resolution aerial images,there are usually problems such as long time,mismatching and sparse feature pairs using traditional algorithms.Therefore,an algorithm is proposed to realize fast,accurate and dense feature matching.The algorithm consists of four steps.Firstly,we achieve a balance between the feature matching time and the number of matching pairs by appropriately reducing the image resolution.Secondly,to realize further screening of the mismatches,a feature screening algorithm based on similarity judgment or local optimization is proposed.Thirdly,to make the algorithm more widely applicable,we combine the results of different algorithms to get dense results.Finally,all matching feature pairs in the low-resolution images are restored to the original images.Comparisons between the original algorithms and our algorithm show that the proposed algorithm can effectively reduce the matching time,screen out the mismatches,and improve the number of matches.
基金the Aerospace Science and Technology Foundation(No.20115557007)the National Natural Science Foundation of China(No.61673262)the Military Science and Technology Foundation of China(No.18-H863-03-ZT-001-006-06)
文摘Domain adaptation and adversarial networks are two main approaches for transfer learning.Domain adaptation methods match the mean values of source and target domains,which requires a very large batch size during training.However,adversarial networks are usually unstable when training.In this paper,we propose a joint method of feature matching and adversarial networks to reduce domain discrepancy and mine domaininvariant features from the local and global aspects.At the same time,our method improves the stability of training.Moreover,the method is embedded into a unified convolutional neural network that can be easily optimized by gradient descent.Experimental results show that our joint method can yield the state-of-the-art results on three common public datasets.
基金Supported by the Key Research and Development Programs of Shandong Province(2018GGX101040)Applied Basic Research Programs of Qingdao(18-2-2-62-jch)。
文摘Feature matching is of significance in the field of computer vision.In this paper,a trifocal tensor based feature matching algorithm is proposed for three views,including a trinocular vision system.Initial matching point-pairs can be determined according to generic matching algorithms,on which an initial trifocal tensor of three views can be confirmed.Then the initial matching point-pairs should be re-selected.Meanwhile,the trifocal tensor will be recomputed.Iteratively,the optimized trifocal tensor can be obtained.Compatible fundamental matrix of every two views can be determined.Furthermore,in the trinocular vision sensor,the trifocal tensor can be calculated based on the intrinsic parameter matrix of each camera.With the strict constraint provided by the trifocal tensor,feature matching results will be optimized.Experiments show that our proposed algorithm has the characteristics of feasibility and precision.
基金This research is supported by the National Natural Science Foundation of China （No. 50875145） and the National High Technology Research and Development Program （＂863＂ Program） of China （Contract No. 2007AAO4Z258）.
文摘In this paper an automatic visual method of seam recognizing and seam tracking based on textural feature matching was proposed, in order to recognize the weld of multi-layer or multi-pass welding in which the weld is difficult to be recognized by conventional visual methods. This method focuses on the obvious difference of image textural feature between the weld region and the base metal region, as well as the similarity of the textural features along the welding direction. The method consists of the following steps ： setting image template and choosing the edge region as ROI （ region of interest ）, extracting the image textural feature of the template and the edge region, feature matching, and recognition of weld region. Experiment showed that the method proposed was effective for weld seam recognition in multi-layer welding.
基金This work was supported by the Science and Technology Major Project of Tibetan Autonomous Region of China(No.XZ202201ZD0003G).
文摘The optimal planning and operation of multi-type flexible resources(FRs)are critical prerequisites for maintaining power and energy balance in regional power grids with a high proportion of clean energy.However,insufficient consideration of the multi-dimensional and heterogeneous features of FRs,such as the regulation characteristics of diversified battery energy storage systems(BESSs),poses a challenge in economically relieving imbalance power and adequately sharing feature information between power supply and demand.In view of this disadvantage,an optimal planning and operation method based on differentiated feature matching through response capability characterization and difference quantification of FRs is proposed in this paper.In the planning stage,a model for the optimal planning of diversified energy storages(ESs)including Lithium-ion battery(Li-B),supercapacitor energy storage(SCES),compressed air energy storage(CAES),and pumped hydroelectric storage(PHS)is established.Subsequently,in the operating stage,the potential,direction,and cost of FR response behaviors are refined to match with the power and energy balance demand(PEBD)of power grid operation.An optimal operating algorithm is then employed to quantify the feature differences and output response sequences of multi-type FRs.The performance and effectiveness of the proposed method are demonstrated through comparative studies conducted on an actual regional power grid in northwest China.Analysis and simulation results illustrate that the proposed method can effectively highlight the advantages of BESSs compared with other ESs,and economically reduce imbalance power of the regional power grid under practical operating conditions.
文摘The traditional Chinese-English translation model tends to translate some source words repeatedly,while mistakenly ignoring some words.Therefore,we propose a novel English-Chinese neural machine translation based on self-organizing mapping neural network and deep feature matching.In this model,word vector,two-way LSTM,2D neural network and other deep learning models are used to extract the semantic matching features of question-answer pairs.Self-organizing mapping(SOM)is used to classify and identify the sentence feature.The attention mechanism-based neural machine translation model is taken as the baseline system.The experimental results show that this framework significantly improves the adequacy of English-Chinese machine translation and achieves better results than the traditional attention mechanism-based English-Chinese machine translation model.
基金This work was supported by Science and Technology Cooperation Special Project of Shijiazhuang(SJZZXA23005).
文摘In minimally invasive surgery,endoscopes or laparoscopes equipped with miniature cameras and tools are used to enter the human body for therapeutic purposes through small incisions or natural cavities.However,in clinical operating environments,endoscopic images often suffer from challenges such as low texture,uneven illumination,and non-rigid structures,which affect feature observation and extraction.This can severely impact surgical navigation or clinical diagnosis due to missing feature points in endoscopic images,leading to treatment and postoperative recovery issues for patients.To address these challenges,this paper introduces,for the first time,a Cross-Channel Multi-Modal Adaptive Spatial Feature Fusion(ASFF)module based on the lightweight architecture of EfficientViT.Additionally,a novel lightweight feature extraction and matching network based on attention mechanism is proposed.This network dynamically adjusts attention weights for cross-modal information from grayscale images and optical flow images through a dual-branch Siamese network.It extracts static and dynamic information features ranging from low-level to high-level,and from local to global,ensuring robust feature extraction across different widths,noise levels,and blur scenarios.Global and local matching are performed through a multi-level cascaded attention mechanism,with cross-channel attention introduced to simultaneously extract low-level and high-level features.Extensive ablation experiments and comparative studies are conducted on the HyperKvasir,EAD,M2caiSeg,CVC-ClinicDB,and UCL synthetic datasets.Experimental results demonstrate that the proposed network improves upon the baseline EfficientViT-B3 model by 75.4%in accuracy(Acc),while also enhancing runtime performance and storage efficiency.When compared with the complex DenseDescriptor feature extraction network,the difference in Acc is less than 7.22%,and IoU calculation results on specific datasets outperform complex dense models.Furthermore,this method increases the F1 score by 33.2%and accelerates runtime by 70.2%.It is noteworthy that the speed of CMMCAN surpasses that of comparative lightweight models,with feature extraction and matching performance comparable to existing complex models but with faster speed and higher cost-effectiveness.
基金supported by National Natural Science Foundation of China(Nos.61201395,61272394,61472119 and 61472373)the program for Science & Technology Innovation Talents in Universities of Henan Province(No.13HASTIT039)the Program for Young Backbone Teachers in Universities of Henan Province(Nos.2012GGJS-057 and 2013GGJS-052)
文摘Feature detection and matching play important roles in many fields of computer vision, such as image understanding, feature recognition, 3D-reconstruction, video analysis, etc. Extracting features is usually the first step for feature detection or matching, and the gradient feature is one of the most used selections. In this paper, a new image feature-absence importance （AI） feature, which can directly characterize the local structure information, is proposed. Greatly different from the most existing features, the proposed absence importance feature is mainly based on the consideration that the absence of the important pixel will have a great effect on the local structure. Two absence importance features, mean absence importance （MAI） and standard deviation absence importance （SDAI）, are defined and used subsequently to construct new algorithms for feature detection and matching. Experiments demonstrate that the proposed absence importance features can be used as an important complement of the gradient feature and applied successfully to the fields of feature detection and matching.
基金Supported by the Foundation of Department of Science and Technology of Jiangxi Province and the Multidiscipline Foundation of Nanchang University
文摘Feature recognition and surface reconstruction from point clouds are difficulties in reverse engineering. A new surface reconstruction algorithm for slicing point cloud was presented. The contours of slice were extracted. Then, the intersection of two adjacent curve segments in the contour was obtained and curves feature was extracted. Finally, adjacent section contours were matched directly with Fourier-Mellin curve matching method for feature extraction. An example of 3-D model reconstruction shows the reliability and application of the algorithm.
基金funded by the National Natural Science Foundation of China under Grant Nos.41822106 and 42101447the Dawn Scholar of Shanghai Program under Grant No.18SG22+2 种基金the Science and Technology on Aerospace Flight Dynamics Laboratory,China,under Grant No.KGJ6142210110305State Key Laboratory of Disaster Reduction in Civil Engineering under Grant No.SLDRCE19-B-35Fundamental Research Funds for the Central Universities of China.
文摘A robust and eficient feature matching method is necessary for visual navigation in asteroid-landing missions.Based on the visual navigation framework and motion characteristics of asteroids,a robust and efficient template feature matching method is proposed to adapt to feature distortion and scale change cases for visual navigation of asteroids.The proposed method is primarily based on a motion-constrained discriminative correlation filter(DCF).The prior information provided by the motion constraints between sequence images is used to provide a predicted search region for template feature matching.Additionally,some specific template feature samples are generated using the motion constraints for correlation filter learning,which is beneficial for training a scale and feature distortion adaptive correlation filter for accurate feature matching.Moreover,average peak-to-correlation energy(APCE)and jointly consistent measurements(JCMs)were used to eliminate false matching.Images captured by the Touch And Go Camera System(TAGCAMS)of the Bennu asteroid were used to evaluate the performance of the proposed method.In particular,both the robustness and accuracy of region matching and template center matching are evaluated.The qualitative and quantitative results illustrate the advancement of the proposed method in adapting to feature distortions and large-scale changes during spacecraft landing.
基金supported by the Strategic Priority Research Program of the Chinese Academy of Sciences[Grant No.XDA19080101]the Director Fund of the International Research Center of Big Data for Sus-tainable Development Goals[Grant No.CBAS2022DF015]+1 种基金the National Natural Science Foundation of China[Grant No.41901328 and 41974108]the National Key Research and Development Program of China[Grant No.2022YFC3800700].
文摘We present GeoGlue,a novel method using high-resolution UAV imagery for accurate feature matching,which is normally challenging due to the complicated scenes.Current feature detection methods are performed without guidance of geometric priors(e.g.,geometric lines),lacking enough attention given to salient geometric features which are indispensable for accurate matching due to their stable existence across views.In this work,geometric lines arefirstly detected by a CNN-based geometry detector(GD)which is pre-trained in a self-supervised manner through automatically generated images.Then,geometric lines are naturally vectorized based on GD and thus non-significant features can be disregarded as judged by their disordered geometric morphology.A graph attention network(GAT)is utilized forfinal feature matching,spanning across the image pair with geometric priors informed by GD.Comprehensive experiments show that GeoGlue outperforms other state-of-the-art methods in feature-matching accuracy and performance stability,achieving pose estimation with maximum rotation and translation errors under 1%in challenging scenes from benchmark datasets,Tanks&Temples and ETH3D.This study also proposes thefirst self-supervised deep-learning model for curved line detection,generating geometric priors for matching so that more attention is put on prominent features and improving the visual effect of 3D reconstruction.
基金supported by National Natural Science Foundation of China(No.61462046)Jiangxi Province Education Department of Science and Technology(Nos.GJJ13539,GJJ12465,GJJ13553,GJJ14558 and GJJ14559)+1 种基金Jiangxi Province Science and Technology(No.20123BBE50076)Jinggangshan University Doctoral Scientific Research Foundation(No.20111101)
文摘This paper puts forward a method for abdomen panorama reconstruction based on a stereo vision system. For the purpose of recovering the abdomen completely and accurately under the condition of actual photographing with illumination variance and blur noise, some innovative combined feature descriptors are presented on the basis of Hu-moment invariants. Furthermore, considering the study on the abdomen surface reconstruction, a circle template which is divided into 6 sectors is designed. It is noted that a descriptor merely using gray intensity is not able to provide sufficient information for feature description. Consequently, the sector entropy which denotes the structure characteristics is drawn into the feature descriptor. By means of the combined effect of the gray intensity and the sector entropy, the similarity measurement is conducted for the final abdomen reconstruction. The experimental results reveal that the proposed method can acquire a high precision of abdomen reconstruction similar to the 3D scanner. This stereo vision system has wide practicability in the field of clothing.
基金Supported by the National Key Research and Development Program of China(2019YFE0125600)China Agriculture Research System(CARS-36)。
文摘Individual identification of dairy cows is the prerequisite for automatic analysis and intelligent perception of dairy cows'behavior.At present,individual identification of dairy cows based on deep convolutional neural network had the disadvantages in prolonged training at the additions of new cows samples.Therefore,a cow individual identification framework was proposed based on deep feature extraction and matching,and the individual identification of dairy cows based on this framework could avoid repeated training.Firstly,the trained convolutional neural network model was used as the feature extractor;secondly,the feature extraction was used to extract features and stored the features into the template feature library to complete the enrollment;finally,the identifies of dairy cows were identified.Based on this framework,when new cows joined the herd,enrollment could be completed quickly.In order to evaluate the application performance of this method in closed-set and open-set individual identification of dairy cows,back images of 524 cows were collected,among which the back images of 150 cows were selected as the training data to train feature extractor.The data of the remaining 374 cows were used to generate the template data set and the data to be identified.The experiment results showed that in the closed-set individual identification of dairy cows,the highest identification accuracy of top-1 was 99.73%,the highest identification accuracy from top-2 to top-5 was 100%,and the identification time of a single cow was 0.601 s,this method was verified to be effective.In the open-set individual identification of dairy cows,the recall was 90.38%,and the accuracy was 89.46%.When false accept rate(FAR)=0.05,true accept rate(TAR)=84.07%,this method was verified that the application had certain research value in open-set individual identification of dairy cows,which provided a certain idea for the application of individual identification in the field of intelligent animal husbandry.
基金Supported by the Key Research Program of the Chinese Academy of Sciences(ZDRE-KT-2021-3)。
文摘Augmented solar images were used to research the adaptability of four representative image extraction and matching algorithms in space weather domain.These include the scale-invariant feature transform algorithm,speeded-up robust features algorithm,binary robust invariant scalable keypoints algorithm,and oriented fast and rotated brief algorithm.The performance of these algorithms was estimated in terms of matching accuracy,feature point richness,and running time.The experiment result showed that no algorithm achieved high accuracy while keeping low running time,and all algorithms are not suitable for image feature extraction and matching of augmented solar images.To solve this problem,an improved method was proposed by using two-frame matching to utilize the accuracy advantage of the scale-invariant feature transform algorithm and the speed advantage of the oriented fast and rotated brief algorithm.Furthermore,our method and the four representative algorithms were applied to augmented solar images.Our application experiments proved that our method achieved a similar high recognition rate to the scale-invariant feature transform algorithm which is significantly higher than other algorithms.Our method also obtained a similar low running time to the oriented fast and rotated brief algorithm,which is significantly lower than other algorithms.
基金The National Natural Science Foundation of China(No.51375087,51405203)the Transformation Program of Science and Technology Achievements of Jiangsu Province(No.BA2016139)
文摘Aming at the problem of the low accuracy of low dynamic vehicle velocity under the environment of uneven distribution of light intensity,an improved adaptive Kalman filter method for the velocity error estimate by the fusion of optical flow tracking and scale mvaiant feature transform(SIFT)is proposed.The algorithm introduces anonlinear fuzzy membership function and the filter residual for the noise covariance matrix in the adaptive adjustment process.In the process of calculating the velocity of the vehicle,the tracking and matching of the inter-frame displacement a d the vehicle velocity calculation a e carried out by using the optical fow tracing and the SIF'T methods,respectively.Meanwhile,the velocity difference between theoutputs of thesetwo methods is used as the observation of the improved adaptive Kalman filter.Finally,the velocity calculated by the optical fow method is corrected by using the velocity error estimate of the output of the modified adaptive Kalman filter.The results of semi-physical experiments show that the maximum velocityeror of the fusion algorithm is decreased by29%than that of the optical fow method,and the computation time is reduced by80%compared with the SIFT method.
基金This work was supported by The National Natural Science Foundation of China under Grant No.61304205 and NO.61502240The Natural Science Foundation of Jiangsu Province under Grant No.BK20191401 and No.BK20201136Postgraduate Research&Practice Innovation Program of Jiangsu Province under Grant No.SJCX21_0364 and No.SJCX21_0363.
文摘The ORB-SLAM2 based on the constant velocity model is difficult to determine the search window of the reprojection of map points when the objects are in variable velocity motion,which leads to a false matching,with an inaccurate pose estimation or failed tracking.To address the challenge above,a new method of feature point matching is proposed in this paper,which combines the variable velocity model with the reverse optical flow method.First,the constant velocity model is extended to a new variable velocity model,and the expanded variable velocity model is used to provide the initial pixel shifting for the reverse optical flow method.Then the search range of feature points is accurately determined according to the results of the reverse optical flow method,thereby improving the accuracy and reliability of feature matching,with strengthened interframe tracking effects.Finally,we tested on TUM data set based on the RGB-D camera.Experimental results show that this method can reduce the probability of tracking failure and improve localization accuracy on SLAM(Simultaneous Localization and Mapping)systems.Compared with the traditional ORB-SLAM2,the test error of this method on each sequence in the TUM data set is significantly reduced,and the root mean square error is only 63.8%of the original system under the optimal condition.
