High-speed imaging is crucial for understanding the transient dynamics of the world,but conventional frame-by-frame video acquisition is limited by specialized hardware and substantial data storage requirements.We int...High-speed imaging is crucial for understanding the transient dynamics of the world,but conventional frame-by-frame video acquisition is limited by specialized hardware and substantial data storage requirements.We introduce“SpeedShot,”a computational imaging framework for efficient high-speed video imaging.SpeedShot features a low-speed dual-camera setup,which simultaneously captures two temporally coded snapshots.Cross-referencing these two snapshots extracts a multiplexed temporal gradient image,producing a compact and multiframe motion representation for video reconstruction.Recognizing the unique temporal-only modulation model,we propose an explicable motion-guided scale-recurrent transformer for video decoding.It exploits cross-scale error maps to bolster the cycle consistency between predicted and observed data.Evaluations on both simulated datasets and real imaging setups demonstrate SpeedShot’s effectiveness in video-rate up-conversion,with pronounced improvement over video frame interpolation and deblurring methods.The proposed framework is compatible with commercial low-speed cameras,offering a versatile low-bandwidth alternative for video-related applications,such as video surveillance and sports analysis.展开更多
基金supported by the National Natural Science Foundation of China(Grant No.62305184)the Basic and Applied Basic Research Foundation of Guangdong Province(Grant No.2023A1515012932)+7 种基金the Science,Technology,and Innovation Commission of Shenzhen Municipality(Grant No.JCYJ20241202123919027)the Major Key Project of Pengcheng Laboratory(Grant No.PCL2024A1)the Science Fund for Distinguished Young Scholars of Zhejiang Province(Grant No.LR23F010001)the Research Center for Industries of the Future(RCIF)at Westlake University and and the Key Project of Westlake Institute for Optoelectronics(Grant No.2023GD007)the Zhejiang“Pioneer”and“Leading Goose”R&D Program(Grant Nos.2024SDXHDX0006 and 2024C03182)the Ningbo Science and Technology Bureau“Science and Technology Yongjiang 2035”Key Technology Breakthrough Program(Grant No.2024Z126)the Research Grants Council of the Hong Kong Special Administrative Region,China(Grant Nos.C5031-22G,CityU11310522,and CityU11300123)the City University of Hong Kong(Grant No.9610628).
文摘High-speed imaging is crucial for understanding the transient dynamics of the world,but conventional frame-by-frame video acquisition is limited by specialized hardware and substantial data storage requirements.We introduce“SpeedShot,”a computational imaging framework for efficient high-speed video imaging.SpeedShot features a low-speed dual-camera setup,which simultaneously captures two temporally coded snapshots.Cross-referencing these two snapshots extracts a multiplexed temporal gradient image,producing a compact and multiframe motion representation for video reconstruction.Recognizing the unique temporal-only modulation model,we propose an explicable motion-guided scale-recurrent transformer for video decoding.It exploits cross-scale error maps to bolster the cycle consistency between predicted and observed data.Evaluations on both simulated datasets and real imaging setups demonstrate SpeedShot’s effectiveness in video-rate up-conversion,with pronounced improvement over video frame interpolation and deblurring methods.The proposed framework is compatible with commercial low-speed cameras,offering a versatile low-bandwidth alternative for video-related applications,such as video surveillance and sports analysis.
