The geolocation of ground targets by airborne image sensors is an important task for unmanned aerial vehicles or surveillance aircraft.This paper proposes an Iterative Geolocation based on Cross-view Image Registratio...The geolocation of ground targets by airborne image sensors is an important task for unmanned aerial vehicles or surveillance aircraft.This paper proposes an Iterative Geolocation based on Cross-view Image Registration(IGCIR)that can provide real-time target location results with high precision.The proposed method has two key features.First,a cross-view image registration process is introduced,including a projective transformation and a two-stage multi-sensor registration.This process utilizes both gradient information and phase information of cross-view images.This allows the registration process to reach a good balance between matching precision and computational efficiency.By matching the airborne camera view to the preloaded digital map,the geolocation accuracy can reach the accuracy level of the digital map for any ground target appearing in the airborne camera view.Second,the proposed method uses the registration results to perform an iteration process,which compensates for the bias of the strap-down initial navigation module online.Although it is challenging to provide cross-view registration results with high frequency,such an iteration process allows the method to generate real-time,highly accurate location results.The effectiveness of the proposed IGCIR method is verified by a series of flying-test experiments.The results show that the location accuracy of the method can reach 4.18 m(at 10 km standoff distance).展开更多
Medical image registration is vital for disease diagnosis and treatment with its ability to merge diverse informa-tion of images,which may be captured under different times,angles,or modalities.Although several survey...Medical image registration is vital for disease diagnosis and treatment with its ability to merge diverse informa-tion of images,which may be captured under different times,angles,or modalities.Although several surveys have reviewed the development of medical image registration,they have not systematically summarized the existing med-ical image registration methods.To this end,a comprehensive review of these methods is provided from traditional and deep-learning-based perspectives,aiming to help audiences quickly understand the development of medical image registration.In particular,we review recent advances in retinal image registration,which has not attracted much attention.In addition,current challenges in retinal image registration are discussed and insights and prospects for future research provided.展开更多
Multi-modal histological image registration tasks pose significant challenges due to tissue staining operations causing partial loss and folding of tissue.Convolutional neural network(CNN)and generative adversarial ne...Multi-modal histological image registration tasks pose significant challenges due to tissue staining operations causing partial loss and folding of tissue.Convolutional neural network(CNN)and generative adversarial network(GAN)are pivotal inmedical image registration.However,existing methods often struggle with severe interference and deformation,as seen in histological images of conditions like Cushing’s disease.We argue that the failure of current approaches lies in underutilizing the feature extraction capability of the discriminator inGAN.In this study,we propose a novel multi-modal registration approach GAN-DIRNet based on GAN for deformable histological image registration.To begin with,the discriminators of two GANs are embedded as a new dual parallel feature extraction module into the unsupervised registration networks,characterized by implicitly extracting feature descriptors of specific modalities.Additionally,modal feature description layers and registration layers collaborate in unsupervised optimization,facilitating faster convergence and more precise results.Lastly,experiments and evaluations were conducted on the registration of the Mixed National Institute of Standards and Technology database(MNIST),eight publicly available datasets of histological sections and the Clustering-Registration-Classification-Segmentation(CRCS)dataset on the Cushing’s disease.Experimental results demonstrate that our proposed GAN-DIRNet method surpasses existing approaches like DIRNet in terms of both registration accuracy and time efficiency,while also exhibiting robustness across different image types.展开更多
Mutual information is widely used in medical image registration, because it does not require preprocessing the image. However, the local maximum problem in the registration is insurmountable. We combine mutual informa...Mutual information is widely used in medical image registration, because it does not require preprocessing the image. However, the local maximum problem in the registration is insurmountable. We combine mutual information and gradient information to solve this problem and apply it to the non-rigid deformation image registration. To improve the accuracy, we provide some implemental issues, for example, the Powell searching algorithm, gray interpolation and consideration of outlier points. The experimental results show the accuracy of the method and the feasibility in non-rigid medical image registration.展开更多
Image fusion aims to integrate complementary information in source images to synthesize a fused image comprehensively characterizing the imaging scene. However, existing image fusion algorithms are only applicable to ...Image fusion aims to integrate complementary information in source images to synthesize a fused image comprehensively characterizing the imaging scene. However, existing image fusion algorithms are only applicable to strictly aligned source images and cause severe artifacts in the fusion results when input images have slight shifts or deformations. In addition,the fusion results typically only have good visual effect, but neglect the semantic requirements of high-level vision tasks.This study incorporates image registration, image fusion, and semantic requirements of high-level vision tasks into a single framework and proposes a novel image registration and fusion method, named Super Fusion. Specifically, we design a registration network to estimate bidirectional deformation fields to rectify geometric distortions of input images under the supervision of both photometric and end-point constraints. The registration and fusion are combined in a symmetric scheme, in which while mutual promotion can be achieved by optimizing the naive fusion loss, it is further enhanced by the mono-modal consistent constraint on symmetric fusion outputs. In addition, the image fusion network is equipped with the global spatial attention mechanism to achieve adaptive feature integration. Moreover, the semantic constraint based on the pre-trained segmentation model and Lovasz-Softmax loss is deployed to guide the fusion network to focus more on the semantic requirements of high-level vision tasks. Extensive experiments on image registration, image fusion,and semantic segmentation tasks demonstrate the superiority of our Super Fusion compared to the state-of-the-art alternatives.The source code and pre-trained model are publicly available at https://github.com/Linfeng-Tang/Super Fusion.展开更多
Point features, as the basis of lines, surfaces, and bodies, are commonly used in medical image registration. To obtain an elegant spatial transformation of extracted feature points, many point set matching algorithms...Point features, as the basis of lines, surfaces, and bodies, are commonly used in medical image registration. To obtain an elegant spatial transformation of extracted feature points, many point set matching algorithms(PMs) have been developed to match two point sets by optimizing multifarious distance functions. There are ample reviews related to medical image registration and PMs which summarize their basic principles and main algorithms separately. However,to data, detailed summary of PMs used in medical image registration in different clinical environments has not been published. In this paper, we provide a comprehensive review of the existing key techniques of the PMs applied to medical image registration according to the basic principles and clinical applications. As the core technique of the PMs, geometric transformation models are elaborated in this paper, demonstrating the mechanism of point set registration. We also focus on the clinical applications of the PMs and propose a practical classification method according to their applications in different clinical surgeries. The aim of this paper is to provide a summary of pointfeaturebased methods used in medical image registration and to guide doctors or researchers interested in this field to choose appropriate techniques in their research.展开更多
Remote sensing image registration is still a challenging task owing to the significant influence of nonlinear differences between remote sensing images.To solve this problem,this paper proposes a novel approach with r...Remote sensing image registration is still a challenging task owing to the significant influence of nonlinear differences between remote sensing images.To solve this problem,this paper proposes a novel approach with regard to feature-based remote sensing image registration.There are two key contributions:1)we bring forward an improved strategy of composite nonlinear diffusion filtering according to the scale factors in multi-scale space and 2)we design a gradually decreasing resolution of multi-scale pyramid space.And a binary code string is served as feature descriptors to improve matching efficiency.Extensive experiments of different categories of remote image datasets on feature extraction and feature registration are performed.The experimental results demonstrate the superiority of our proposed scheme compared with other classical algorithms in terms of correct matching ratio,accuracy and computation efficiency.展开更多
This paper presents a novel medical image registration algorithm named total variation constrained graphregularization for non-negative matrix factorization(TV-GNMF).The method utilizes non-negative matrix factorizati...This paper presents a novel medical image registration algorithm named total variation constrained graphregularization for non-negative matrix factorization(TV-GNMF).The method utilizes non-negative matrix factorization by total variation constraint and graph regularization.The main contributions of our work are the following.First,total variation is incorporated into NMF to control the diffusion speed.The purpose is to denoise in smooth regions and preserve features or details of the data in edge regions by using a diffusion coefficient based on gradient information.Second,we add graph regularization into NMF to reveal intrinsic geometry and structure information of features to enhance the discrimination power.Third,the multiplicative update rules and proof of convergence of the TV-GNMF algorithm are given.Experiments conducted on datasets show that the proposed TV-GNMF method outperforms other state-of-the-art algorithms.展开更多
The proposed algorithm relies on a group of new formulas for calculating tangent slope so as to address angle feature of edge curves of image. It can utilize tangent angle features to estimate automatically and fully ...The proposed algorithm relies on a group of new formulas for calculating tangent slope so as to address angle feature of edge curves of image. It can utilize tangent angle features to estimate automatically and fully the rotation parameters of geometric transform and enable rough matching of images with huge rotation difference. After angle compensation, it can search for matching point sets by correlation criterion, then calculate parameters of affine transform, enable higher-precision emendation of rotation and transferring. Finally, it fulfills precise matching for images with relax-tense iteration method. Compared with the registration approach based on wavelet direction-angle features, the matching algorithm with tangent feature of image edge is more robust and realizes precise registration of various images. Furthermore, it is also helpful in graphics matching.展开更多
Multi-sensor image registration has been widely used in remote sensing and medical image field, but registration performance is degenerated when heterogeneous images are involved. An image registration method based on...Multi-sensor image registration has been widely used in remote sensing and medical image field, but registration performance is degenerated when heterogeneous images are involved. An image registration method based on multi-resolution shape analysis is proposed in this paper, to deal with the problem that the shape of similar objects is always invariant. The contours of shapes are first detected as visual features using an extended contour search algorithm in order to reduce effects of noise, and the multi-resolution shape descriptor is constructed through Fourier curvature representation of the contour’s chain code. Then a minimum distance function is used to judge the similarity between two contours. To avoid the effect of different resolution and intensity distribution, suitable resolution of each image is selected by maximizing the consistency of its pyramid shapes. Finally, the transformation parameters are estimated based on the matched control-point pairs which are the centers of gravity of the closed contours. Multi-sensor Landsat TM imagery and infrared imagery have been used as experimental data for comparison with the classical contour-based registration. Our results have been shown to be superior to the classical ones.展开更多
In recent years,computational intelligence has been widely used in many fields and achieved remarkable performance.Evolutionary computing and deep learning are important branches of computational intelligence.Many met...In recent years,computational intelligence has been widely used in many fields and achieved remarkable performance.Evolutionary computing and deep learning are important branches of computational intelligence.Many methods based on evolutionary computation and deep learning have achieved good performance in remote sensing image registration.This paper introduces the application of computational intelligence in remote sensing image registration from the two directions of evolutionary computing and deep learning.In the part of remote sensing image registration based on evolutionary calculation,the principles of evolutionary algorithms and swarm intelligence algorithms are elaborated and their application in remote sensing image registration is discussed.The application of deep learning in remote sensing image registration is also discussed.At the same time,the development status and future of remote sensing image registration are summarized and their prospects are examined.展开更多
Homologous feature point extraction is a key problem in the optical and synthetic aperture radar （SAR） image registration for islands. A new feature point extraction method using a threshold shrink operator and non-...Homologous feature point extraction is a key problem in the optical and synthetic aperture radar （SAR） image registration for islands. A new feature point extraction method using a threshold shrink operator and non-subsampled wavelet transform （TSO-NSWT） for optical and SAR image registration was proposed. Moreover, the matching for this proposed feature was different from the traditional feature matching strategies and was performed using a similarity measure computed from neighborhood circles in low-frequency bands. Then, a number of reliably matched couples with even distributions were obtained, which assured the accuracy of the registration. Application of the proposed algorithm to SPOT-5 （multi-spectral） and YG-1 （SAR） images showed that a large number of accurately matched couples could be identified. Additionally, both of the root mean square error （RMSE） patterns of the registration parameters computed based on the TSO-NSWT algorithm and traditional NSWT algorithm were analyzed and compared, which further demonstrated the effectiveness of the proposed algorithm. The algorithm can supply the crucial step for island image registration and island recognition.展开更多
In this paper, we proposed a registration method by combining the morphological component analysis(MCA) and scale-invariant feature transform(SIFT) algorithm. This method uses the perception dictionaries,and combines ...In this paper, we proposed a registration method by combining the morphological component analysis(MCA) and scale-invariant feature transform(SIFT) algorithm. This method uses the perception dictionaries,and combines the Basis-Pursuit algorithm and the Total-Variation regularization scheme to extract the cartoon part containing basic geometrical information from the original image, and is stable and unsusceptible to noise interference. Then a smaller number of the distinctive key points will be obtained by using the SIFT algorithm based on the cartoon part of the original image. Matching the key points by the constrained Euclidean distance,we will obtain a more correct and robust matching result. The experimental results show that the geometrical transform parameters inferred by the matched key points based on MCA+SIFT registration method are more exact than the ones based on the direct SIFT algorithm.展开更多
A new coarse-to-fine strategy was proposed for nonrigid registration of computed tomography(CT) and magnetic resonance(MR) images of a liver.This hierarchical framework consisted of an affine transformation and a B-sp...A new coarse-to-fine strategy was proposed for nonrigid registration of computed tomography(CT) and magnetic resonance(MR) images of a liver.This hierarchical framework consisted of an affine transformation and a B-splines free-form deformation(FFD).The affine transformation performed a rough registration targeting the mismatch between the CT and MR images.The B-splines FFD transformation performed a finer registration by correcting local motion deformation.In the registration algorithm,the normalized mutual information(NMI) was used as similarity measure,and the limited memory Broyden-Fletcher- Goldfarb-Shannon(L-BFGS) optimization method was applied for optimization process.The algorithm was applied to the fully automated registration of liver CT and MR images in three subjects.The results demonstrate that the proposed method not only significantly improves the registration accuracy but also reduces the running time,which is effective and efficient for nonrigid registration.展开更多
Presence of higher breast density(BD)and persistence over time are risk factors for breast cancer.A quantitatively accurate and highly reproducible BD measure that relies on precise and reproducible whole-breast segme...Presence of higher breast density(BD)and persistence over time are risk factors for breast cancer.A quantitatively accurate and highly reproducible BD measure that relies on precise and reproducible whole-breast segmentation is desirable.In this study,we aimed to develop a highly reproducible and accurate whole-breast segmentation algorithm for the generation of reproducible BD measures.Three datasets of volunteers from two clinical trials were included.Breast MR images were acquired on 3T Siemens Biograph mMR,Prisma,and Skyra using 3D Cartesian six-echo GRE sequences with a fat-water separation technique.Two whole-breast segmentation strategies,utiliz-ing image registration and 3D U-Net,were developed.Manual segmentation was performed.A task-based analysis was performed:a previously developed MR-based BD measure,MagDensity,was calculated and assessed using automated and manual segmentation.The mean squared error(MSE)and intraclass correlation coefficient(ICC)between MagDensity were evaluated using the manual segmentation as a reference.The test-retest reproducibility of MagDensity derived from different breast segmentation methods was assessed using the difference between the test and retest measures(Δ_(2-1)),MSE,and ICC.The results showed that MagDensity derived by the registration and deep learning segmentation methods exhibited high concordance with manual segmentation,with ICCs of 0.986(95%CI:0.974-0.993)and 0.983(95%CI:0.961-0.992),respectively.For test-retest analysis,MagDensity derived using the regis-tration algorithm achieved the smallest MSE of 0.370 and highest ICC of 0.993(95%CI:0.982-0.997)when compared to other segmentation methods.In conclusion,the proposed registration and deep learning whole-breast segmentation methods are accurate and reliable for estimating BD.Both methods outperformed a previously developed algorithm and manual segmentation in the test-retest assessment,with the registration exhibiting superior performance for highly reproducible BD measurements.展开更多
In order to improve the registration accuracy of brain magnetic resonance images(MRI),some deep learning registration methods use segmentation images for training model.How-ever,the segmentation values are constant fo...In order to improve the registration accuracy of brain magnetic resonance images(MRI),some deep learning registration methods use segmentation images for training model.How-ever,the segmentation values are constant for each label,which leads to the gradient variation con-centrating on the boundary.Thus,the dense deformation field(DDF)is gathered on the boundary and there even appears folding phenomenon.In order to fully leverage the label information,the morphological opening and closing information maps are introduced to enlarge the non-zero gradi-ent regions and improve the accuracy of DDF estimation.The opening information maps supervise the registration model to focus on smaller,narrow brain regions.The closing information maps supervise the registration model to pay more attention to the complex boundary region.Then,opening and closing morphology networks(OC_Net)are designed to automatically generate open-ing and closing information maps to realize the end-to-end training process.Finally,a new registra-tion architecture,VM_(seg+oc),is proposed by combining OC_Net and VoxelMorph.Experimental results show that the registration accuracy of VM_(seg+oc) is significantly improved on LPBA40 and OASIS1 datasets.Especially,VM_(seg+oc) can well improve registration accuracy in smaller brain regions and narrow regions.展开更多
Image registration is an important research topic in the field of computer vision,in which the registration and mosaic of side-scan sonar images is the keypoints of underwater navigation.However,the image registration...Image registration is an important research topic in the field of computer vision,in which the registration and mosaic of side-scan sonar images is the keypoints of underwater navigation.However,the image registration method of keypoints is not suitable for sonar images which do not have obvious feature points.Therefore,a method of sonar-image registration and mosaic based on line segment extraction and triangle matching is proposed in this paper.Firstly,in order to extract features from sonar image,the LSD method is introduced to detect line feature from images,and line segments are filtered by the principle of attention;after that,triangles are formed from line segments,an image transformation matrix can be calculated through the heuristic greedy algorithm from these triangles;finally,images are merged based on the transformation information.On the basis of practical tests,it is found that,the feature extraction method used in this paper can better describe the outline of underwater terrain,and there is no obvious stitching gap between the result of sonar images stitched.Experimental results show that the proposed method is effective than the keypoints method of the registration and mosaic of sonar images.展开更多
An improved image registration method is proposed based on mutual infor- mation with hybrid optimizer. Firstly, mutual information measure is combined with morphological gradient information. The essence of the gradie...An improved image registration method is proposed based on mutual infor- mation with hybrid optimizer. Firstly, mutual information measure is combined with morphological gradient information. The essence of the gradient information is that locations a large gradient magnitude should be aligned, but also the orientation of the gradients at those locations should be similar. Secondly, a hybrid optimizer combined PSO with Powell algorithm is proposed to restrain local maxima of mutual information function and improve the registration accuracy to sub-pixel level. Lastly, muhlresolution data structure based on Mallat decomposition can not only improve the behavior of registration function, but also improve the speed of the algorithm. Experimental results demonstrate that the new method can yield good registration result, superior to traditional optimizer with respect to smoothness and attraction basin as well as convergence speed.展开更多
In this paper, a new approach of muhi-modality image registration is represented with not only image intensity, but also features describing image structure. There are two novelties in the proposed method. Firstly, in...In this paper, a new approach of muhi-modality image registration is represented with not only image intensity, but also features describing image structure. There are two novelties in the proposed method. Firstly, instead of standard mutual information （ MI ） based on joint intensity histogram, regional mutual information （ RMI ） is employed, which allows neighborhood information to be taken into account. Secondly, a new feature images obtained by means of phase congruency are invariants to brightness or contrast changes. By incorporating these features and intensity into RMI, we can combine the aspects of both structural and neighborhood information together, which offers a more robust and a high level of registration accuracy.展开更多
In this paper, the authors propose a refined Branch-and-Bound algorithm for affine-transformation based image registration. Given two feature point-sets in two images respectively, the authors first extract a sequence...In this paper, the authors propose a refined Branch-and-Bound algorithm for affine-transformation based image registration. Given two feature point-sets in two images respectively, the authors first extract a sequence of high-probability matched point-pairs by considering well-defined features. Each resultant point-pair can be regarded as a constraint in the search space of Branch-and-Bound algorithm guiding the search process. The authors carry out Branch-and-Bound search with the constraint of a pair-point selected by using Monte Carlo sampling according to the match measures of point-pairs. If such one cannot lead to correct result, additional candidate is chosen to start another search. High-probability matched point-pairs usually results in fewer loops and the search process is accelerated greatly. Experimental results verify the high efficiency and robustness of the author’s approach.展开更多
基金supported by the National Level Project of China(No.52-L0D01-0613-20/22)。
文摘The geolocation of ground targets by airborne image sensors is an important task for unmanned aerial vehicles or surveillance aircraft.This paper proposes an Iterative Geolocation based on Cross-view Image Registration(IGCIR)that can provide real-time target location results with high precision.The proposed method has two key features.First,a cross-view image registration process is introduced,including a projective transformation and a two-stage multi-sensor registration.This process utilizes both gradient information and phase information of cross-view images.This allows the registration process to reach a good balance between matching precision and computational efficiency.By matching the airborne camera view to the preloaded digital map,the geolocation accuracy can reach the accuracy level of the digital map for any ground target appearing in the airborne camera view.Second,the proposed method uses the registration results to perform an iteration process,which compensates for the bias of the strap-down initial navigation module online.Although it is challenging to provide cross-view registration results with high frequency,such an iteration process allows the method to generate real-time,highly accurate location results.The effectiveness of the proposed IGCIR method is verified by a series of flying-test experiments.The results show that the location accuracy of the method can reach 4.18 m(at 10 km standoff distance).
基金supported in part by General Program of National Natural Science Foundation of China,Nos.82102189 and 82272086Guangdong Provincial Department of Education,No.SJZLGC202202.
文摘Medical image registration is vital for disease diagnosis and treatment with its ability to merge diverse informa-tion of images,which may be captured under different times,angles,or modalities.Although several surveys have reviewed the development of medical image registration,they have not systematically summarized the existing med-ical image registration methods.To this end,a comprehensive review of these methods is provided from traditional and deep-learning-based perspectives,aiming to help audiences quickly understand the development of medical image registration.In particular,we review recent advances in retinal image registration,which has not attracted much attention.In addition,current challenges in retinal image registration are discussed and insights and prospects for future research provided.
文摘Multi-modal histological image registration tasks pose significant challenges due to tissue staining operations causing partial loss and folding of tissue.Convolutional neural network(CNN)and generative adversarial network(GAN)are pivotal inmedical image registration.However,existing methods often struggle with severe interference and deformation,as seen in histological images of conditions like Cushing’s disease.We argue that the failure of current approaches lies in underutilizing the feature extraction capability of the discriminator inGAN.In this study,we propose a novel multi-modal registration approach GAN-DIRNet based on GAN for deformable histological image registration.To begin with,the discriminators of two GANs are embedded as a new dual parallel feature extraction module into the unsupervised registration networks,characterized by implicitly extracting feature descriptors of specific modalities.Additionally,modal feature description layers and registration layers collaborate in unsupervised optimization,facilitating faster convergence and more precise results.Lastly,experiments and evaluations were conducted on the registration of the Mixed National Institute of Standards and Technology database(MNIST),eight publicly available datasets of histological sections and the Clustering-Registration-Classification-Segmentation(CRCS)dataset on the Cushing’s disease.Experimental results demonstrate that our proposed GAN-DIRNet method surpasses existing approaches like DIRNet in terms of both registration accuracy and time efficiency,while also exhibiting robustness across different image types.
文摘Mutual information is widely used in medical image registration, because it does not require preprocessing the image. However, the local maximum problem in the registration is insurmountable. We combine mutual information and gradient information to solve this problem and apply it to the non-rigid deformation image registration. To improve the accuracy, we provide some implemental issues, for example, the Powell searching algorithm, gray interpolation and consideration of outlier points. The experimental results show the accuracy of the method and the feasibility in non-rigid medical image registration.
基金supported by the National Natural Science Foundation of China(62276192,62075169,62061160370)the Key Research and Development Program of Hubei Province(2020BAB113)。
文摘Image fusion aims to integrate complementary information in source images to synthesize a fused image comprehensively characterizing the imaging scene. However, existing image fusion algorithms are only applicable to strictly aligned source images and cause severe artifacts in the fusion results when input images have slight shifts or deformations. In addition,the fusion results typically only have good visual effect, but neglect the semantic requirements of high-level vision tasks.This study incorporates image registration, image fusion, and semantic requirements of high-level vision tasks into a single framework and proposes a novel image registration and fusion method, named Super Fusion. Specifically, we design a registration network to estimate bidirectional deformation fields to rectify geometric distortions of input images under the supervision of both photometric and end-point constraints. The registration and fusion are combined in a symmetric scheme, in which while mutual promotion can be achieved by optimizing the naive fusion loss, it is further enhanced by the mono-modal consistent constraint on symmetric fusion outputs. In addition, the image fusion network is equipped with the global spatial attention mechanism to achieve adaptive feature integration. Moreover, the semantic constraint based on the pre-trained segmentation model and Lovasz-Softmax loss is deployed to guide the fusion network to focus more on the semantic requirements of high-level vision tasks. Extensive experiments on image registration, image fusion,and semantic segmentation tasks demonstrate the superiority of our Super Fusion compared to the state-of-the-art alternatives.The source code and pre-trained model are publicly available at https://github.com/Linfeng-Tang/Super Fusion.
基金Supported by the National Natural Science Foundation of China(Grant No.61533016)
文摘Point features, as the basis of lines, surfaces, and bodies, are commonly used in medical image registration. To obtain an elegant spatial transformation of extracted feature points, many point set matching algorithms(PMs) have been developed to match two point sets by optimizing multifarious distance functions. There are ample reviews related to medical image registration and PMs which summarize their basic principles and main algorithms separately. However,to data, detailed summary of PMs used in medical image registration in different clinical environments has not been published. In this paper, we provide a comprehensive review of the existing key techniques of the PMs applied to medical image registration according to the basic principles and clinical applications. As the core technique of the PMs, geometric transformation models are elaborated in this paper, demonstrating the mechanism of point set registration. We also focus on the clinical applications of the PMs and propose a practical classification method according to their applications in different clinical surgeries. The aim of this paper is to provide a summary of pointfeaturebased methods used in medical image registration and to guide doctors or researchers interested in this field to choose appropriate techniques in their research.
基金supported by National Nature Science Foundation of China(Nos.61640412 and 61762052)the Natural Science Foundation of Jiangxi Province(No.20192BAB207021)the Science and Technology Research Projects of Jiangxi Province Education Department(Nos.GJJ170633 and GJJ170632).
文摘Remote sensing image registration is still a challenging task owing to the significant influence of nonlinear differences between remote sensing images.To solve this problem,this paper proposes a novel approach with regard to feature-based remote sensing image registration.There are two key contributions:1)we bring forward an improved strategy of composite nonlinear diffusion filtering according to the scale factors in multi-scale space and 2)we design a gradually decreasing resolution of multi-scale pyramid space.And a binary code string is served as feature descriptors to improve matching efficiency.Extensive experiments of different categories of remote image datasets on feature extraction and feature registration are performed.The experimental results demonstrate the superiority of our proposed scheme compared with other classical algorithms in terms of correct matching ratio,accuracy and computation efficiency.
基金supported by the National Natural Science Foundation of China(61702251,41971424,61701191,U1605254)the Natural Science Basic Research Plan in Shaanxi Province of China(2018JM6030)+4 种基金the Key Technical Project of Fujian Province(2017H6015)the Science and Technology Project of Xiamen(3502Z20183032)the Doctor Scientific Research Starting Foundation of Northwest University(338050050)Youth Academic Talent Support Program of Northwest University(360051900151)the Natural Sciences and Engineering Research Council of Canada,Canada。
文摘This paper presents a novel medical image registration algorithm named total variation constrained graphregularization for non-negative matrix factorization(TV-GNMF).The method utilizes non-negative matrix factorization by total variation constraint and graph regularization.The main contributions of our work are the following.First,total variation is incorporated into NMF to control the diffusion speed.The purpose is to denoise in smooth regions and preserve features or details of the data in edge regions by using a diffusion coefficient based on gradient information.Second,we add graph regularization into NMF to reveal intrinsic geometry and structure information of features to enhance the discrimination power.Third,the multiplicative update rules and proof of convergence of the TV-GNMF algorithm are given.Experiments conducted on datasets show that the proposed TV-GNMF method outperforms other state-of-the-art algorithms.
基金Supported by the National Natural Science Foundation of China (No.60141002) the Aviation Basic Science Foundation (02I53073)
文摘The proposed algorithm relies on a group of new formulas for calculating tangent slope so as to address angle feature of edge curves of image. It can utilize tangent angle features to estimate automatically and fully the rotation parameters of geometric transform and enable rough matching of images with huge rotation difference. After angle compensation, it can search for matching point sets by correlation criterion, then calculate parameters of affine transform, enable higher-precision emendation of rotation and transferring. Finally, it fulfills precise matching for images with relax-tense iteration method. Compared with the registration approach based on wavelet direction-angle features, the matching algorithm with tangent feature of image edge is more robust and realizes precise registration of various images. Furthermore, it is also helpful in graphics matching.
基金Project supported by the National Natural Science Foundation of China (No. 60272031), the Hi-Tech Research and Development Program (863) of China (No. 2003AA131032-2), and the Natural Science Foundation of Zhejiang Province (No. M603202), China
文摘Multi-sensor image registration has been widely used in remote sensing and medical image field, but registration performance is degenerated when heterogeneous images are involved. An image registration method based on multi-resolution shape analysis is proposed in this paper, to deal with the problem that the shape of similar objects is always invariant. The contours of shapes are first detected as visual features using an extended contour search algorithm in order to reduce effects of noise, and the multi-resolution shape descriptor is constructed through Fourier curvature representation of the contour’s chain code. Then a minimum distance function is used to judge the similarity between two contours. To avoid the effect of different resolution and intensity distribution, suitable resolution of each image is selected by maximizing the consistency of its pyramid shapes. Finally, the transformation parameters are estimated based on the matched control-point pairs which are the centers of gravity of the closed contours. Multi-sensor Landsat TM imagery and infrared imagery have been used as experimental data for comparison with the classical contour-based registration. Our results have been shown to be superior to the classical ones.
基金National Natural Science Foundation of China(Nos.61702392 and 61772393)Key Research and Development Program of Shaanxi Province(Nos.2018ZDXM-GY-045 and 2019JQ-189).
文摘In recent years,computational intelligence has been widely used in many fields and achieved remarkable performance.Evolutionary computing and deep learning are important branches of computational intelligence.Many methods based on evolutionary computation and deep learning have achieved good performance in remote sensing image registration.This paper introduces the application of computational intelligence in remote sensing image registration from the two directions of evolutionary computing and deep learning.In the part of remote sensing image registration based on evolutionary calculation,the principles of evolutionary algorithms and swarm intelligence algorithms are elaborated and their application in remote sensing image registration is discussed.The application of deep learning in remote sensing image registration is also discussed.At the same time,the development status and future of remote sensing image registration are summarized and their prospects are examined.
基金The National Natural Science Foundation of China under contract No.41271409the National Key Technology Research and Development Program under contract No.2011BAH23B00the National High Technology Research and Development Program(863 Program)of China under contract No.2012AA12A406
文摘Homologous feature point extraction is a key problem in the optical and synthetic aperture radar （SAR） image registration for islands. A new feature point extraction method using a threshold shrink operator and non-subsampled wavelet transform （TSO-NSWT） for optical and SAR image registration was proposed. Moreover, the matching for this proposed feature was different from the traditional feature matching strategies and was performed using a similarity measure computed from neighborhood circles in low-frequency bands. Then, a number of reliably matched couples with even distributions were obtained, which assured the accuracy of the registration. Application of the proposed algorithm to SPOT-5 （multi-spectral） and YG-1 （SAR） images showed that a large number of accurately matched couples could be identified. Additionally, both of the root mean square error （RMSE） patterns of the registration parameters computed based on the TSO-NSWT algorithm and traditional NSWT algorithm were analyzed and compared, which further demonstrated the effectiveness of the proposed algorithm. The algorithm can supply the crucial step for island image registration and island recognition.
基金the National Science Foundation of China(No.61471185)the Natural Science Foundation of Shandong Province(No.ZR2016FM21)+1 种基金Shandong Province Science and Technology Plan Project(No.2015GSF116001)Yantai City Key Research and Development Plan Project(Nos.2014ZH157 and2016ZH057)
文摘In this paper, we proposed a registration method by combining the morphological component analysis(MCA) and scale-invariant feature transform(SIFT) algorithm. This method uses the perception dictionaries,and combines the Basis-Pursuit algorithm and the Total-Variation regularization scheme to extract the cartoon part containing basic geometrical information from the original image, and is stable and unsusceptible to noise interference. Then a smaller number of the distinctive key points will be obtained by using the SIFT algorithm based on the cartoon part of the original image. Matching the key points by the constrained Euclidean distance,we will obtain a more correct and robust matching result. The experimental results show that the geometrical transform parameters inferred by the matched key points based on MCA+SIFT registration method are more exact than the ones based on the direct SIFT algorithm.
基金Project(61240010)supported by the National Natural Science Foundation of ChinaProject(20070007070)supported by Specialized Research Fund for the Doctoral Program of Higher Education of China
文摘A new coarse-to-fine strategy was proposed for nonrigid registration of computed tomography(CT) and magnetic resonance(MR) images of a liver.This hierarchical framework consisted of an affine transformation and a B-splines free-form deformation(FFD).The affine transformation performed a rough registration targeting the mismatch between the CT and MR images.The B-splines FFD transformation performed a finer registration by correcting local motion deformation.In the registration algorithm,the normalized mutual information(NMI) was used as similarity measure,and the limited memory Broyden-Fletcher- Goldfarb-Shannon(L-BFGS) optimization method was applied for optimization process.The algorithm was applied to the fully automated registration of liver CT and MR images in three subjects.The results demonstrate that the proposed method not only significantly improves the registration accuracy but also reduces the running time,which is effective and efficient for nonrigid registration.
基金This work is partially supported by the National Institute of Health R03CA223052The sulindac trial was supported by R01CA161534The metformin trial was supported by R01CA172444 and P30CA023074。
文摘Presence of higher breast density(BD)and persistence over time are risk factors for breast cancer.A quantitatively accurate and highly reproducible BD measure that relies on precise and reproducible whole-breast segmentation is desirable.In this study,we aimed to develop a highly reproducible and accurate whole-breast segmentation algorithm for the generation of reproducible BD measures.Three datasets of volunteers from two clinical trials were included.Breast MR images were acquired on 3T Siemens Biograph mMR,Prisma,and Skyra using 3D Cartesian six-echo GRE sequences with a fat-water separation technique.Two whole-breast segmentation strategies,utiliz-ing image registration and 3D U-Net,were developed.Manual segmentation was performed.A task-based analysis was performed:a previously developed MR-based BD measure,MagDensity,was calculated and assessed using automated and manual segmentation.The mean squared error(MSE)and intraclass correlation coefficient(ICC)between MagDensity were evaluated using the manual segmentation as a reference.The test-retest reproducibility of MagDensity derived from different breast segmentation methods was assessed using the difference between the test and retest measures(Δ_(2-1)),MSE,and ICC.The results showed that MagDensity derived by the registration and deep learning segmentation methods exhibited high concordance with manual segmentation,with ICCs of 0.986(95%CI:0.974-0.993)and 0.983(95%CI:0.961-0.992),respectively.For test-retest analysis,MagDensity derived using the regis-tration algorithm achieved the smallest MSE of 0.370 and highest ICC of 0.993(95%CI:0.982-0.997)when compared to other segmentation methods.In conclusion,the proposed registration and deep learning whole-breast segmentation methods are accurate and reliable for estimating BD.Both methods outperformed a previously developed algorithm and manual segmentation in the test-retest assessment,with the registration exhibiting superior performance for highly reproducible BD measurements.
基金supported by Shandong Provincial Natural Science Foundation(No.ZR2023MF062)the National Natural Science Foundation of China(No.61771230).
文摘In order to improve the registration accuracy of brain magnetic resonance images(MRI),some deep learning registration methods use segmentation images for training model.How-ever,the segmentation values are constant for each label,which leads to the gradient variation con-centrating on the boundary.Thus,the dense deformation field(DDF)is gathered on the boundary and there even appears folding phenomenon.In order to fully leverage the label information,the morphological opening and closing information maps are introduced to enlarge the non-zero gradi-ent regions and improve the accuracy of DDF estimation.The opening information maps supervise the registration model to focus on smaller,narrow brain regions.The closing information maps supervise the registration model to pay more attention to the complex boundary region.Then,opening and closing morphology networks(OC_Net)are designed to automatically generate open-ing and closing information maps to realize the end-to-end training process.Finally,a new registra-tion architecture,VM_(seg+oc),is proposed by combining OC_Net and VoxelMorph.Experimental results show that the registration accuracy of VM_(seg+oc) is significantly improved on LPBA40 and OASIS1 datasets.Especially,VM_(seg+oc) can well improve registration accuracy in smaller brain regions and narrow regions.
文摘Image registration is an important research topic in the field of computer vision,in which the registration and mosaic of side-scan sonar images is the keypoints of underwater navigation.However,the image registration method of keypoints is not suitable for sonar images which do not have obvious feature points.Therefore,a method of sonar-image registration and mosaic based on line segment extraction and triangle matching is proposed in this paper.Firstly,in order to extract features from sonar image,the LSD method is introduced to detect line feature from images,and line segments are filtered by the principle of attention;after that,triangles are formed from line segments,an image transformation matrix can be calculated through the heuristic greedy algorithm from these triangles;finally,images are merged based on the transformation information.On the basis of practical tests,it is found that,the feature extraction method used in this paper can better describe the outline of underwater terrain,and there is no obvious stitching gap between the result of sonar images stitched.Experimental results show that the proposed method is effective than the keypoints method of the registration and mosaic of sonar images.
文摘An improved image registration method is proposed based on mutual infor- mation with hybrid optimizer. Firstly, mutual information measure is combined with morphological gradient information. The essence of the gradient information is that locations a large gradient magnitude should be aligned, but also the orientation of the gradients at those locations should be similar. Secondly, a hybrid optimizer combined PSO with Powell algorithm is proposed to restrain local maxima of mutual information function and improve the registration accuracy to sub-pixel level. Lastly, muhlresolution data structure based on Mallat decomposition can not only improve the behavior of registration function, but also improve the speed of the algorithm. Experimental results demonstrate that the new method can yield good registration result, superior to traditional optimizer with respect to smoothness and attraction basin as well as convergence speed.
文摘In this paper, a new approach of muhi-modality image registration is represented with not only image intensity, but also features describing image structure. There are two novelties in the proposed method. Firstly, instead of standard mutual information （ MI ） based on joint intensity histogram, regional mutual information （ RMI ） is employed, which allows neighborhood information to be taken into account. Secondly, a new feature images obtained by means of phase congruency are invariants to brightness or contrast changes. By incorporating these features and intensity into RMI, we can combine the aspects of both structural and neighborhood information together, which offers a more robust and a high level of registration accuracy.
基金Project supported by the National Basic Research Program (973) of China (No. 2002CB312101), the National Natural Science Founda-tion of China (Nos. 60475013 and 60273053) and Defense Science and Technology Key Lab. Foundation of China (No. 51476070101JW0409)
文摘In this paper, the authors propose a refined Branch-and-Bound algorithm for affine-transformation based image registration. Given two feature point-sets in two images respectively, the authors first extract a sequence of high-probability matched point-pairs by considering well-defined features. Each resultant point-pair can be regarded as a constraint in the search space of Branch-and-Bound algorithm guiding the search process. The authors carry out Branch-and-Bound search with the constraint of a pair-point selected by using Monte Carlo sampling according to the match measures of point-pairs. If such one cannot lead to correct result, additional candidate is chosen to start another search. High-probability matched point-pairs usually results in fewer loops and the search process is accelerated greatly. Experimental results verify the high efficiency and robustness of the author’s approach.
