摘要
目的针对在超分辨率任务中,Transformer模型存在特征提取模式单一、重建图像高频细节丢失和结构失真的问题,提出了一种融合通道注意力的跨尺度Transformer图像超分辨率重建模型。方法模型由4个模块组成:浅层特征提取、跨尺度深层特征提取、多级特征融合以及高质量重建模块。浅层特征提取利用卷积处理早期图像,获得更稳定的输出;跨尺度深层特征提取利用跨尺度Transformer和强化通道注意力机制,扩大感受野并通过加权筛选提取不同尺度特征以便融合;多级特征融合模块利用强化通道注意力机制,实现对不同尺度特征通道权重的动态调整,促进模型对丰富上下文信息的学习,增强模型在图像超分辨率重建任务中的能力。结果在Set5、Set14、BSD100(Berkeley segmentation dataset 100)、Urban100(urban scene 100)和Manga109标准数据集上的模型评估结果表明,相较于SwinIR超分辨率模型,所提模型在峰值信噪比上提高了0.06~0.25 dB,且重建图像视觉效果更好。结论提出的融合通道注意力的跨尺度Transformer图像超分辨率重建模型,通过融合卷积特征与Transformer特征,并利用强化通道注意力机制减少图像中噪声和冗余信息,降低模型产生图像模糊失真的可能性,图像超分辨率性能有效提升,在多个公共实验数据集的测试结果验证了所提模型的有效性。
Objective The image super-resolution reconstruction technique refers to a method for converting low-resolution(LR)images to high-resolution(HR)images within the same scene.In recent years,this technique has been widely used in computer vision,image processing,and other fields due to its wide practical application value and far-reaching theoretical importance.Although the model based on convolutional neural networks has made remarkable progress,most superresolution network structures remain in a single-layer level end-to-end format to improve the reconstruction performance.This approach often overlooks the multilayer level feature information during the network reconstruction process,limiting the reconstruction performance of the model.With the advancement of deep learning technology,Transformer-based network architectures have been introduced into the field of computer vision,yielding substantial results.Researchers have applied Transform models to underlying vision tasks,including image super-resolution reconstruction.However,in this context,the Transformer model suffers from a single feature extraction pattern,loss of high-frequency details in the reconstructed image,and structural distortion.A cross-scale Transformer image super-resolution reconstruction model with fusion channel attention is proposed to address these problems.Method The model comprises the following four modules:shallow feature extraction,cross-scale deep feature extraction,multilevel feature fusion,and a high-quality reconstruction module.Shallow feature extraction uses convolution to process early images to obtain highly stable outputs,and the convolutional layer can provide stable optimization and extraction results during early visual feature processing.The cross-scale deep feature extraction module uses the cross-scale Transformer and the enhanced channel attention mechanism to acquire features at different scales.The core of the cross-scale Transformer lies in the cross-scale self-attention mechanism and the gated convolutional feedforward network,which down samples the feature maps to different scales by scale factors and learns contextual information using image self-similarity,and the gated convolutional network encodes spatial neighboring pixel position information and helps learn the local image structure,replacing the feedforward network in the traditional Transformer.A reinforced channel attention mechanism is used after the cross-scale Transformer to expand the sensory field and extract different scale features to replace the original features via weighted filtering for backward propagation.Increasing the depth of the network will lead to saturation.Thus,the number of residual cross-scale Transformer blocks is set to 3 to maintain a balance between model complexity and super-resolution reconstruction performance.After stacking different scale features in the multilevel feature fusion module,the enhanced channel attention mechanism is used to dynamically adjust the channel weights of different scale features and learn rich contextual information,thereby enhancing the network reconstruction capability.In the high-quality reconstruction module,convolutional layers and pixel blending methods are used to up-sample features to the corresponding dimensions of high-resolution images.In the training phase,the model is trained using 900 HR images from the DIV2K dataset,and the corresponding LR images are generated from the HR images using double-triple downsampling(with downsampling multiples of×2,×3 and×4).The network is optimized using Adam’s algorithm with L1loss as the loss function.Result Tests on five standard datasets,namely,Set5,Set14,BSD100,Urban100,and Manga109,are performed,and the performance of the proposed model is compared with 10 state-of-the-art models.These models include the following:enhanced deep residual networks for single image superresolution(EDSR),residual channel attention networks(RCAN),second-order attention network(SAN),cross-scale non-local attention(CSNLA),the cross-scale internal graph neural network(IGNN),holistic attention network(HAN),non-local sparse attention(NLSA),image restoration using Swin Transformer(SwinIR),efficient long-range attention network(ELAN),and permuted self-attention(SRFormer).Peak signal-to-noise ratio(PSNR)and structural similarity(SSIM)are used as metrics to measure the performance of these methods.Humans are very sensitive to the brightness of an image;therefore,these metrics are measured in the Y-channel of the image.Experimental results show that the proposed model obtains high PSNR and SSIM values and recovers additional detailed information and highly accurate textures at magnification factors of×2,×3,and×4.The proposed method improves 0.13~0.25 dB over SwinIR and 0.07~0.21 dB over ELAN on the Urban100 dataset and 0.07~0.21 dB over SwinIR and 0.06~0.19 dB over ELAN on the Manga109 dataset.The localized attribution map(LAM)is used to further explore the model performance.The experimental results revealed that the proposed model can utilize a wider range of pixel information,and the proposed model exhibits a higher diffusion index(DI)compared to SwinIR,proving the effectiveness of the proposed model from the interpretability viewpoint.Conclusion The proposed cross-scale Transformer image super-resolution reconstruction model with multilevel fusion channel attention reduces noise and redundant information in the image by fusing convolutional features with Transformer features.This model also uses a strengthened channel attention mechanism to reduce the likelihood of image blurring and distortion in the model,and the image super-resolution performance is effectively improved.The test results verify the effectiveness of the multi-tip model in numerous public experimental datasets.The model visually obtains a reconstructed image that is sharper and closer to the real image with fewer artefacts.
作者
李焱
董仕豪
张家伟
赵茹
郑钰辉
Li Yan;Dong Shihao;Zhang Jiawei;Zhao Ru;Zheng Yuhui(School of Computer Science,Nanjing University of Information Science and Technology,Nanjing 210044,China;School of Journalism and Communication,Northwest University,Xi’an 710127,China)
出处
《中国图象图形学报》
北大核心
2025年第3期784-797,共14页
Journal of Image and Graphics
基金
国家自然科学基金项目(U20B2065)。
