Traditional cameras inevitably suffer from motion blur when facing high-speed moving objects.Event cameras,as high temporal resolution bionic cameras,record intensity changes in an asynchronous manner,and their record...Traditional cameras inevitably suffer from motion blur when facing high-speed moving objects.Event cameras,as high temporal resolution bionic cameras,record intensity changes in an asynchronous manner,and their recorded high temporal resolution information can effectively solve the problem of time information loss in motion blur.Existing event-based deblurring methods still face challenges when facing high-speed moving objects.We conducted an in-depth study of the imaging principle of event cameras.We found that the event stream contains excessive noise.The valid information is sparse.Invalid event features hinder the expression of valid features due to the uncertainty of the global threshold.To address this problem,a denoising-based long and short-term memory module(DTM)is designed in this paper.The DTM suppressed the original event information by noise reduction process.Invalid features in the event stream and solves the problem of sparse valid information in the event stream,and it also combines with the long short-term memory module(LSTM),which further enhances the event feature information in the time scale.In addition,through the in-depth understanding of the unique characteristics of event features,it is found that the high-frequency information recorded by event features does not effectively guide the fusion feature deblurring process in the spatial-domain-based feature processing,and for this reason,we introduce the residual fast fourier transform module(RES-FFT)to further enhance the high-frequency characteristics of the fusion features by performing the feature extraction of the fusion features from the perspective of the frequency domain.Ultimately,our proposed event image fusion network based on event denoising and frequency domain feature enhancement(DNEFNET)achieved Peak Signal-to-Noise Ratio(PSNR)/Structural Similarity Index Measure(SSIM)scores of 35.55/0.972 on the GoPro dataset and 38.27/0.975 on the REBlur dataset,achieving the state of the art(SOTA)effect.展开更多
文摘Traditional cameras inevitably suffer from motion blur when facing high-speed moving objects.Event cameras,as high temporal resolution bionic cameras,record intensity changes in an asynchronous manner,and their recorded high temporal resolution information can effectively solve the problem of time information loss in motion blur.Existing event-based deblurring methods still face challenges when facing high-speed moving objects.We conducted an in-depth study of the imaging principle of event cameras.We found that the event stream contains excessive noise.The valid information is sparse.Invalid event features hinder the expression of valid features due to the uncertainty of the global threshold.To address this problem,a denoising-based long and short-term memory module(DTM)is designed in this paper.The DTM suppressed the original event information by noise reduction process.Invalid features in the event stream and solves the problem of sparse valid information in the event stream,and it also combines with the long short-term memory module(LSTM),which further enhances the event feature information in the time scale.In addition,through the in-depth understanding of the unique characteristics of event features,it is found that the high-frequency information recorded by event features does not effectively guide the fusion feature deblurring process in the spatial-domain-based feature processing,and for this reason,we introduce the residual fast fourier transform module(RES-FFT)to further enhance the high-frequency characteristics of the fusion features by performing the feature extraction of the fusion features from the perspective of the frequency domain.Ultimately,our proposed event image fusion network based on event denoising and frequency domain feature enhancement(DNEFNET)achieved Peak Signal-to-Noise Ratio(PSNR)/Structural Similarity Index Measure(SSIM)scores of 35.55/0.972 on the GoPro dataset and 38.27/0.975 on the REBlur dataset,achieving the state of the art(SOTA)effect.
