摘要
多光谱图像全色锐化是遥感影像处理与解译领域的热点问题。相较于传统全色锐化方法,基于深度学习的全色锐化方法聚焦于图像深层次特征的提取,大幅提升了融合图像的质量。本文提出一种基于混合双分支卷积神经网络和图卷积神经网络的全色锐化方法,旨在同时挖掘图像的光谱、空间与非几何结构信息,以提升融合图像空间分辨率和光谱分辨率。本方法建立在多分辨率分析融合框架的基础上,利用深度神经网络构建了特征提取、特征融合和图像重构模块。混合双分支网络模块是由2D和3D卷积神经网络构建,其中,2D卷积神经网络负责挖掘多光谱图像与全色图像的空间特征,3D卷积神经网络负责挖掘图像的光谱特征。引入了图卷积神经网络以捕捉图像图结构中节点的空间关系,从而整合非局部信息。将多光谱图像与全色图像的空间、光谱和非几何特征通过特征融合模块进行融合。将融合特征输入图像重构网络重建高质量多光谱图像。本文算法在GeoEye-1和IKONOS遥感数据上进行了实验验证,实验结果表明:与其他方法相比,本文算法在主观视觉和客观评价指标上均表现出优秀性能。
The pansharpening of multispectral images represents a trending research topic in remote sensing image processing and interpretation.Moreover,compared with traditional pansharpening methods,deep learning-based pansharpening methods mainly extract deep features,thereby greatly improving the quality of fused images.Here,a method based on hybrid dual-branch convolutional neural network(CNN)and graph convolutional neural network(GCNN)is proposed to simultaneously extract spectral information,spatial information,and non-geometric structural information and improve the spatial and spectral resolutions of fused images.This hybrid method comprises the construction of a multi-resolution analysis fusion framework,followed by the construction of a feature extraction module,a feature fusion module,and an image reconstruction module based on deep neural networks.First,the hybrid dual-branch network module was constructed using 2D and 3D CNNs that focus on extracting spatial and spectral features,respectively.Second,GCNN was introduced to capture the spatial relationships of the nodes in the graph structure of the image and integrate non-local information.Afterward,the spatial,spectral,and non-geometric features extracted from multispectral and panchromatic images were fused by the feature fusion module.Finally,the fused features were input into the image reconstruction network to reconstruct the high-quality multispectral images.The proposed method was experimentally validated using GeoEye-1 and IKONOS remote sensing data.Compared with other methods,the experimental results obtained by the proposed method reveal its excellent performance in subjective and objective vision evaluations.
作者
王文卿
张小乔
何霁
刘涵
刘丁
WANG Wenqing;ZHANG Xiaoqiao;HE Ji;LIU Han;LIU Ding(School of Automation and Information Engineering,Xi’an University of Technology,Xi’an 710048,China;Shaanxi Key Laboratory of Complex System Control and Intelligent Information Processing,Xi’an University of Technology,Xi’an 710048,China)
出处
《智能系统学报》
北大核心
2025年第3期649-657,共9页
CAAI Transactions on Intelligent Systems
基金
国家自然科学基金项目(62376214,92270117)
陕西省自然科学基础研究计划项目(2023-JC-YB-533)。
关键词
图像融合
遥感
图像处理
深度学习
卷积神经网络
机器学习
特征提取
图像重构
image fusion
remote sensing
image processing
deep learning
convolutional neural network
machine learning
feature extraction
image reconstruction
