Pan-sharpening aims to seek high-resolution multispectral(HRMS) images from paired multispectral images of low resolution(LRMS) and panchromatic(PAN) images, the key to which is how to maximally integrate spatial and ...Pan-sharpening aims to seek high-resolution multispectral(HRMS) images from paired multispectral images of low resolution(LRMS) and panchromatic(PAN) images, the key to which is how to maximally integrate spatial and spectral information from PAN and LRMS images. Following the principle of gradual advance, this paper designs a novel network that contains two main logical functions, i.e., detail enhancement and progressive fusion, to solve the problem. More specifically, the detail enhancement module attempts to produce enhanced MS results with the same spatial sizes as corresponding PAN images, which are of higher quality than directly up-sampling LRMS images.Having a better MS base(enhanced MS) and its PAN, we progressively extract information from the PAN and enhanced MS images, expecting to capture pivotal and complementary information of the two modalities for the purpose of constructing the desired HRMS. Extensive experiments together with ablation studies on widely-used datasets are provided to verify the efficacy of our design, and demonstrate its superiority over other state-of-the-art methods both quantitatively and qualitatively. Our code has been released at https://github.com/JiaYN1/PAPS.展开更多
The merging of a panchromatic (PAN) image with a multispectral satellite image (MSI) to increase the spatial resolution of the MSI, while simultaneously preserving its spectral information is classically referred as P...The merging of a panchromatic (PAN) image with a multispectral satellite image (MSI) to increase the spatial resolution of the MSI, while simultaneously preserving its spectral information is classically referred as PAN-sharpening. We employed a recent dataset derived from very high resolution of WorldView-2 satellite (PAN and MSI) for two test sites (one over an urban area and the other over Antarctica), to comprehensively evaluate the performance of six existing PAN-sharpening algorithms. The algorithms under consideration were the Gram-Schmidt (GS), Ehlers fusion (EF), modified hue-intensity-saturation (Mod-HIS), high pass filtering (HPF), the Brovey transform (BT), and wavelet-based principal component analysis (W-PC). Quality assessment of the sharpened images was carried out by using 20 quality indices. We also analyzed the performance of nearest neighbour (NN), bilinear interpolation (BI), and cubic convolution (CC) resampling methods to test their practicability in the PAN-sharpening process. Our results indicate that the comprehensive performance of PAN-sharpening methods decreased in the following order: GS > W-PC > EF > HPF > Mod-HIS > BT, while resampling methods followed the order: NN > BI > CC.展开更多
Intensity-hue-saturation (IHS) transform is the most commonly used method for image fusion purpose. Usually, the intensity image is replaced by Panchromatic (PAN) image, or the difference between PAN and intensity ima...Intensity-hue-saturation (IHS) transform is the most commonly used method for image fusion purpose. Usually, the intensity image is replaced by Panchromatic (PAN) image, or the difference between PAN and intensity image is added to each bands of RGB images. Spatial structure information in the PAN image can be effectively injected into the fused multi-spectral (MS) images using IHS method. However, spectral distortion has become the typical factor deteriorating the quality of fused results. A hybrid image fusion method which integrates IHS and minimum mean-square-error (MMSE) was proposed to mitigate the spectral distortion phenomenon in this study. Firstly, IHS transform was used to derive the intensity image;secondly, the MMSE algorithm was used to fuse the histogram matched PAN image and intensity image;thirdly, optimization calculation was employed to derive the combination coefficients, and the new intensity image could be expressed as the combination of intensity image and PAN image. Fused MS images with high spatial resolution can be generated by inverse IHS transform. In numerical experiments, QuickBird images were used to evaluate the performance of the proposed algorithm. It was found that the spatial resolution was increased significantly;meanwhile, spectral distortion phenomenon was abated in the fusion results.展开更多
Pan-sharpening is a process of obtaining a high spatial and spectral multispectral image(HMS)by combining a low-resolution multispectral image(LMS)with a high-resolution panchromatic image(PAN).In this paper,a pan-sha...Pan-sharpening is a process of obtaining a high spatial and spectral multispectral image(HMS)by combining a low-resolution multispectral image(LMS)with a high-resolution panchromatic image(PAN).In this paper,a pan-sharpening method called PAIHS is proposed,which is based on adaptive intensity-hue-saturation(AIHS)transformation,variational pan-sharpening framework and the two fidelity hypotheses.The suitable objective function is established and optimized by adopting particle swarm optimization(PSO)to obtain the optimal control parameters and minimum value.This value corresponds to the best pan-sharpening quality.The experimental results show that the proposed method has high efficiency and reliability,and the obtained performance index is superior to the four mainstream pan-sharpening methods.展开更多
针对高分辨率影像全色(Panchromatic,Pan)波段和多光谱(Multispectral,MS)波段的pan-sharpening融合后图像光谱失真的问题,基于调制传递函数(Modulation Transfer Function,MTF)的全色多光谱图像融合模型考虑到了多光谱图像的MTF值对融...针对高分辨率影像全色(Panchromatic,Pan)波段和多光谱(Multispectral,MS)波段的pan-sharpening融合后图像光谱失真的问题,基于调制传递函数(Modulation Transfer Function,MTF)的全色多光谱图像融合模型考虑到了多光谱图像的MTF值对融合图像质量的影响,采用了与多光谱图像相同的MTF值所构建的低通滤波器,得到较好的融合结果,但如何选择一个合适的MTF值还没有很好地解决。该文针对不同MTF值对模型融合结果的影响做了详细的分析与实验,并通过线性搜索的方式找出最优的MTF值。实验结果证明了该最优MTF能够同时提高模型融合结果的光谱细节和空间细节。展开更多
This paper introduces some of the image processing techniques developed in the Canada Research Chair in Advanced Geomatics Image Processing Laboratory (CRC-AGIP Lab) and in the Department of Geodesy and Geomatics Engi...This paper introduces some of the image processing techniques developed in the Canada Research Chair in Advanced Geomatics Image Processing Laboratory (CRC-AGIP Lab) and in the Department of Geodesy and Geomatics Engineering (GGE) at the University of New Brunswick (UNB), Canada. The techniques were developed by innovatively/“smartly” utilizing the characteristics of the available very high resolution optical remote sensing images to solve important problems or create new applications in photogrammetry and remote sensing. The techniques to be introduced are: automated image fusion (UNB-PanSharp), satellite image online mapping, street view technology, moving vehicle detection using single set satellite imagery, supervised image segmentation, image matching in smooth areas, and change detection using images from different viewing angles. Because of their broad application potential, some of the techniques have made a global impact, and some have demonstrated the potential for a global impact.展开更多
A pan-sharpen technique artificially produces a high-resolution image by image fusion techniques using high-resolution panchromatic and low-resolution multispectral images. Thus, the appearance of the color image can ...A pan-sharpen technique artificially produces a high-resolution image by image fusion techniques using high-resolution panchromatic and low-resolution multispectral images. Thus, the appearance of the color image can improve. In this paper, the effectiveness of three pan-sharpening methods based on the HSI transform approach is investigated. Three models are the hexcone, double hexcones, and Haydn’s approach. Furthermore, the effect of smoothing the low-resolution multispectral image is also investigated. The smoothing techniques are the Gaussian filter and the bilateral filter. The experimental results show that Haydn’s model is superior to others. The effectiveness of smoothing the low-resolution multispectral image is also shown.展开更多
Pan-sharpening is a method of integrating low-resolution multispectral images with corresponding high-resolution panchromatic images to obtain multispectral images with high spectral and spatial resolution. A novel va...Pan-sharpening is a method of integrating low-resolution multispectral images with corresponding high-resolution panchromatic images to obtain multispectral images with high spectral and spatial resolution. A novel variational model for pan-sharpening is proposed in this paper. The model is mainly based on three hypotheses: 1) the pan-shaipened image can be linearly represented by the corresponding panchromatic image;2) the low-resolution multispectral image is down-sampled from the high-resolution multispectral image through the down-sampling operator;and 3) the satellite image has the low-rank property. Three energy components corresponding to these assumptions are integrated into a variational framework to obtain a total energy function. We adopt the alternating direction method of multipliers (ADMM) to optimize the total energy function. The experimental results show that the proposed method performs better than other mainstream methods in spectral and spatial information preserving aspect.展开更多
基金partially supported by the National Natural Science Foundation of China (62372251)。
文摘Pan-sharpening aims to seek high-resolution multispectral(HRMS) images from paired multispectral images of low resolution(LRMS) and panchromatic(PAN) images, the key to which is how to maximally integrate spatial and spectral information from PAN and LRMS images. Following the principle of gradual advance, this paper designs a novel network that contains two main logical functions, i.e., detail enhancement and progressive fusion, to solve the problem. More specifically, the detail enhancement module attempts to produce enhanced MS results with the same spatial sizes as corresponding PAN images, which are of higher quality than directly up-sampling LRMS images.Having a better MS base(enhanced MS) and its PAN, we progressively extract information from the PAN and enhanced MS images, expecting to capture pivotal and complementary information of the two modalities for the purpose of constructing the desired HRMS. Extensive experiments together with ablation studies on widely-used datasets are provided to verify the efficacy of our design, and demonstrate its superiority over other state-of-the-art methods both quantitatively and qualitatively. Our code has been released at https://github.com/JiaYN1/PAPS.
文摘The merging of a panchromatic (PAN) image with a multispectral satellite image (MSI) to increase the spatial resolution of the MSI, while simultaneously preserving its spectral information is classically referred as PAN-sharpening. We employed a recent dataset derived from very high resolution of WorldView-2 satellite (PAN and MSI) for two test sites (one over an urban area and the other over Antarctica), to comprehensively evaluate the performance of six existing PAN-sharpening algorithms. The algorithms under consideration were the Gram-Schmidt (GS), Ehlers fusion (EF), modified hue-intensity-saturation (Mod-HIS), high pass filtering (HPF), the Brovey transform (BT), and wavelet-based principal component analysis (W-PC). Quality assessment of the sharpened images was carried out by using 20 quality indices. We also analyzed the performance of nearest neighbour (NN), bilinear interpolation (BI), and cubic convolution (CC) resampling methods to test their practicability in the PAN-sharpening process. Our results indicate that the comprehensive performance of PAN-sharpening methods decreased in the following order: GS > W-PC > EF > HPF > Mod-HIS > BT, while resampling methods followed the order: NN > BI > CC.
文摘Intensity-hue-saturation (IHS) transform is the most commonly used method for image fusion purpose. Usually, the intensity image is replaced by Panchromatic (PAN) image, or the difference between PAN and intensity image is added to each bands of RGB images. Spatial structure information in the PAN image can be effectively injected into the fused multi-spectral (MS) images using IHS method. However, spectral distortion has become the typical factor deteriorating the quality of fused results. A hybrid image fusion method which integrates IHS and minimum mean-square-error (MMSE) was proposed to mitigate the spectral distortion phenomenon in this study. Firstly, IHS transform was used to derive the intensity image;secondly, the MMSE algorithm was used to fuse the histogram matched PAN image and intensity image;thirdly, optimization calculation was employed to derive the combination coefficients, and the new intensity image could be expressed as the combination of intensity image and PAN image. Fused MS images with high spatial resolution can be generated by inverse IHS transform. In numerical experiments, QuickBird images were used to evaluate the performance of the proposed algorithm. It was found that the spatial resolution was increased significantly;meanwhile, spectral distortion phenomenon was abated in the fusion results.
基金National Natural Science Foundation of China(No.61703278)。
文摘Pan-sharpening is a process of obtaining a high spatial and spectral multispectral image(HMS)by combining a low-resolution multispectral image(LMS)with a high-resolution panchromatic image(PAN).In this paper,a pan-sharpening method called PAIHS is proposed,which is based on adaptive intensity-hue-saturation(AIHS)transformation,variational pan-sharpening framework and the two fidelity hypotheses.The suitable objective function is established and optimized by adopting particle swarm optimization(PSO)to obtain the optimal control parameters and minimum value.This value corresponds to the best pan-sharpening quality.The experimental results show that the proposed method has high efficiency and reliability,and the obtained performance index is superior to the four mainstream pan-sharpening methods.
文摘针对高分辨率影像全色(Panchromatic,Pan)波段和多光谱(Multispectral,MS)波段的pan-sharpening融合后图像光谱失真的问题,基于调制传递函数(Modulation Transfer Function,MTF)的全色多光谱图像融合模型考虑到了多光谱图像的MTF值对融合图像质量的影响,采用了与多光谱图像相同的MTF值所构建的低通滤波器,得到较好的融合结果,但如何选择一个合适的MTF值还没有很好地解决。该文针对不同MTF值对模型融合结果的影响做了详细的分析与实验,并通过线性搜索的方式找出最优的MTF值。实验结果证明了该最优MTF能够同时提高模型融合结果的光谱细节和空间细节。
文摘This paper introduces some of the image processing techniques developed in the Canada Research Chair in Advanced Geomatics Image Processing Laboratory (CRC-AGIP Lab) and in the Department of Geodesy and Geomatics Engineering (GGE) at the University of New Brunswick (UNB), Canada. The techniques were developed by innovatively/“smartly” utilizing the characteristics of the available very high resolution optical remote sensing images to solve important problems or create new applications in photogrammetry and remote sensing. The techniques to be introduced are: automated image fusion (UNB-PanSharp), satellite image online mapping, street view technology, moving vehicle detection using single set satellite imagery, supervised image segmentation, image matching in smooth areas, and change detection using images from different viewing angles. Because of their broad application potential, some of the techniques have made a global impact, and some have demonstrated the potential for a global impact.
文摘A pan-sharpen technique artificially produces a high-resolution image by image fusion techniques using high-resolution panchromatic and low-resolution multispectral images. Thus, the appearance of the color image can improve. In this paper, the effectiveness of three pan-sharpening methods based on the HSI transform approach is investigated. Three models are the hexcone, double hexcones, and Haydn’s approach. Furthermore, the effect of smoothing the low-resolution multispectral image is also investigated. The smoothing techniques are the Gaussian filter and the bilateral filter. The experimental results show that Haydn’s model is superior to others. The effectiveness of smoothing the low-resolution multispectral image is also shown.
基金the editor and reviewers for their insightful comments and constructive suggestions on the article, and thank Dr. Thomas James Godfrey for helping us to revise the grammar. This work was supported in part by “Chenguang Program” supported by Shanghai Education Development Foundation and Shanghai Municipal Education Commission (Grant No. 17CG25)in part by the Key Project of the National Natural Science Foundation of China (Grant No. 61731009)in part by the National Natural Science Foundation of China (Grant No. 61871185).
文摘Pan-sharpening is a method of integrating low-resolution multispectral images with corresponding high-resolution panchromatic images to obtain multispectral images with high spectral and spatial resolution. A novel variational model for pan-sharpening is proposed in this paper. The model is mainly based on three hypotheses: 1) the pan-shaipened image can be linearly represented by the corresponding panchromatic image;2) the low-resolution multispectral image is down-sampled from the high-resolution multispectral image through the down-sampling operator;and 3) the satellite image has the low-rank property. Three energy components corresponding to these assumptions are integrated into a variational framework to obtain a total energy function. We adopt the alternating direction method of multipliers (ADMM) to optimize the total energy function. The experimental results show that the proposed method performs better than other mainstream methods in spectral and spatial information preserving aspect.
