Single-photon sensors are novel devices with extremely high single-photon sensitivity and temporal resolution.However,these advantages also make them highly susceptible to noise.Moreover,single-photon cameras face sev...Single-photon sensors are novel devices with extremely high single-photon sensitivity and temporal resolution.However,these advantages also make them highly susceptible to noise.Moreover,single-photon cameras face severe quantization as low as 1 bit/frame.These factors make it a daunting task to recover high-quality scene information from noisy single-photon data.Most current image reconstruction methods for single-photon data are mathematical approaches,which limits information utilization and algorithm performance.In this work,we propose a hybrid information enhancement model which can significantly enhance the efficiency of information utilization by leveraging attention mechanisms from both spatial and channel branches.Furthermore,we introduce a structural feature enhance module for the FFN of the transformer,which explicitly improves the model's ability to extract and enhance high-frequency structural information through two symmetric convolution branches.Additionally,we propose a single-photon data simulation pipeline based on RAW images to address the challenge of the lack of single-photon datasets.Experimental results show that the proposed method outperforms state-of-the-art methods in various noise levels and exhibits a more efficient capability for recovering high-frequency structures and extracting information.展开更多
文摘Single-photon sensors are novel devices with extremely high single-photon sensitivity and temporal resolution.However,these advantages also make them highly susceptible to noise.Moreover,single-photon cameras face severe quantization as low as 1 bit/frame.These factors make it a daunting task to recover high-quality scene information from noisy single-photon data.Most current image reconstruction methods for single-photon data are mathematical approaches,which limits information utilization and algorithm performance.In this work,we propose a hybrid information enhancement model which can significantly enhance the efficiency of information utilization by leveraging attention mechanisms from both spatial and channel branches.Furthermore,we introduce a structural feature enhance module for the FFN of the transformer,which explicitly improves the model's ability to extract and enhance high-frequency structural information through two symmetric convolution branches.Additionally,we propose a single-photon data simulation pipeline based on RAW images to address the challenge of the lack of single-photon datasets.Experimental results show that the proposed method outperforms state-of-the-art methods in various noise levels and exhibits a more efficient capability for recovering high-frequency structures and extracting information.
基金supported by the National Natural Science Foundation of China(No.21973086,No.22203083)the Ministry of Science and Technology of China(No.2021YFA1200103)the Fundamental Research Funds for the Central Universities(No.WK2060000018)。
