摘要
目的针对Transformer在图像去模糊过程中难以精确恢复图像细节的问题,提出一种结合Mamba模型与蛇形卷积技术的图像去模糊网络MSNet(Mamba snake convolution network)。方法首先,结合Mamba框架与蛇形卷积,提出蛇形状态空间模块(snake state-space module,SSSM)。SSSM通过调整卷积核的形状和路径,动态适应图像局部特征并调整卷积方向,以对齐不同的模糊条纹模式;其次,使用多方向扫描模块(direction scan module,DSM)进行多个方向的扫描,捕捉图像中的长期依赖。再利用离散状态空间方程合并多方向的结构信息,增强模型对全局结构的捕捉能力;最后,引入蛇形通道注意力(snake channel attention,SCA),利用门控设计筛选和调整模糊信息的权重,确保在去除模糊的同时保留关键细节。结果实验在GoPro和HIDE数据集上,与主流的卷积神经网络(convolutional neural network,CNN)和Transformer去模糊方法相比,MSNet的峰值信噪比(peak signal to noise ratio,PSNR)分别提升1.2%和1.9%,结构相似性(structural similarity,SSIM)分别提升0.6%和0.7%。结论本文方法可以有效去除复杂场景下产生的图像模糊,并复原细节。
Objective Traditional image deblurring methods,such as those based on convolutional neural networks(CNNs)and Transformers,have achieved substantial advancements in improving deblurring performance.Despite these achievements,these methods are still constrained by high computational demands and limitations in restoring intricate image details.In complex conditions involving motion blur or high-frequency details,existing approaches often rely on fixed convolution kernels or global self-attention mechanisms.Such static designs lack the adaptability to handle diverse types of blur effectively,which leads to suboptimal detail recovery and inadequate reconstruction of global image structures.Moreover,Transformer-based deblurring methods frequently require extensive computational resources,which significantly diminishes their feasibility for deployment on mobile devices or embedded systems.These resource constraints not only restrict their applicability in practical scenarios but also impede their broader adoption in real-world applications.To address these challenges,this study proposes a novel image deblurring method,which is termed MSNet.By integrating the efficient state space modeling capabilities of the Mamba framework with snake convolution techniques,MSNet leverages the complementary strengths of these innovations.This approach aims to reduce computational overhead while achieving high-fidelity recovery of fine image details and structural information.With its enhanced adaptability and efficiency,MSNet is better suited for practical applications.It offers robust performance in tackling complex deblurring tasks across diverse scenarios.Method To achieve the objective,the MSNet network integrates three key modules:the snake state space module(SSSM),the directional scanning module(DSM),and the snake channel attention module(SCA).Each module is designed for a specific purpose,and their combination effectively tackles local detail recovery and global structure restoration.The SSSM combines the Mamba framework with snake convolution technology,with the aim of enhancing the capability of the model to capture subtle blur features.Unlike traditional CNN-based methods relying on fixed convolution kernels,SSSM dynamically adjusts the shape and path of the convolutional kernels.This way allows them to adapt to local image features and blur stripe patterns.Snake convolution alters the convolution path to effectively capture local blur features.Moreover,the Mamba framework takes advantage of state space models through processing long-range dependencies with linear computational complexity.In contrast to the high computational complexity of Transformer-based models relying on self-attention,Mamba can more efficiently capture long-term dependencies in the image,which avoids the excessive computational burden associated with Transformer models.Simultaneously,snake convolution enhances the precision with which the network adapts to local image features.Thus,it offers notable advantages in capturing complex motion blur and fine detail blur.The DSM module transforms image features into a one-dimensional sequence and scans these features in multiple directions(diagonal,horizontal,and vertical)to capture long-range dependencies.This module effectively improves global structure restoration,particularly in scenes with objects moving simultaneously in multiple directions,which allows for better reconstruction of the overall image structure.The SCA module uses a gating mechanism to filter and adjust the weights of the blurred information.Through combining snake convolution with a channel attention mechanism,this module allows the model to dynamically adjust the weights of different features,which prioritizes key image details while removing irrelevant blur information.Through this selective focus,the SCA module significantly enhances detail recovery and optimizes the overall deblurring performance.Result To validate the effectiveness of MSNet,we conducted comparative and ablation experiments on two widely used image deblurring benchmark datasets:GoPro and HIDE.During the experiments,MSNet was compared against several commonly used deblurring methods.The results show that MSNet exhibited outstanding performance in addressing image blur artifacts and restoring fine details.On the GoPro dataset,MSNet achieved significant improvements in PSNR and SSIM compared with Transformer-and CNN-based methods.MSNet demonstrated superior accuracy in restoring blurred regions,which effectively addressed the limitations of existing methods in handling complex scenes.This performance highlights capability of MSNet to process images with intricate details and challenging blur conditions more effectively than its counterparts.On the HIDE dataset,MSNet also outperformed Transformer-and CNN-based methods through achieving higher PSNR and SSIM scores.It showed remarkable accuracy in deblurring fine textual and facial details in blurred images.By leveraging its adaptive convolution design and multidirectional scanning approach,MSNet exhibited strong robustness and generalization capabilities.Thus,it is well suited for complex and dynamic scenarios.Moreover,MSNet demonstrated exceptional computational efficiency.It achieved a computational complexity of 63.7 GFLOPs on the GoPro dataset,which was significantly lower than those of MIMO-UNet and other comparative methods.This balance of high deblurring performance and low computational cost makes MSNet an ideal solution for real-time deblurring tasks in resource-constrained environments.Ablation studies further validated the contributions of the key modules of MSNet.The removal of the SSSM or the SCA module led to a significant drop in PSNR,with the greatest decrease occurring when both modules were removed.These findings highlight the critical role of these modules in improving deblurring accuracy and restoring fine image details.In addition,network depth analysis revealed that MSNet-28(28 layers)achieved the best performance,with a PSNR of 33.51 dB and an SSIM of 0.97.This result confirms the importance of optimizing network depth and module design to enhance overall performance.Conclusion MSNet demonstrates outstanding performance across multiple datasets.It not only showcases its exceptional deblurring accuracy and detail recovery capabilities but also achieves a good balance in computational efficiency.By incorporating the state space model of the Mamba framework and the flexibility of serpentine convolution,MSNet efficiently handles long-range dependencies,particularly exhibiting stronger adaptability in complex blur scenarios.The ablation experiments validate the importance of each module,with the SSSM and SCA modules playing key roles in detail recovery and global structure reconstruction.Overall,MSNet excels in deblurring tasks with its strong generalization capabilities,efficient computation,and superior performance in detail recovery.
作者
邱云飞
刘则延
王茂华
Qiu Yunfei;Liu Zeyan;Wang Maohua(School of Software,Liaoning Technical University,Huludao 125105,China;College of Information Engineering,Liaoning Institute of Science and Engineering,Jinzhou 121000,China;School of Computer and Cyberspace Security,Fujian Normal University,Fuzhou 350117,China)
出处
《中国图象图形学报》
北大核心
2025年第10期3187-3198,共12页
Journal of Image and Graphics
基金
国家自然科学基金项目(62173171)
辽宁省自然科学基金项目(2015020095)。
