Video snapshot compressive imaging(Video SCI) modulates scenes using various encoding masks and captures compressed measurements with a low-speed camera during a single exposure. Subsequently, reconstruction algorithm...Video snapshot compressive imaging(Video SCI) modulates scenes using various encoding masks and captures compressed measurements with a low-speed camera during a single exposure. Subsequently, reconstruction algorithms restore image sequences of dynamic scenes, offering advantages such as reduced bandwidth and storage space requirements. The temporal correlation in video data is crucial for Video SCI, as it leverages the temporal relationships among frames to enhance the efficiency and quality of reconstruction algorithms, particularly for fast-moving objects.This paper discretizes video frames to create image datasets with the same data volume but differing temporal correlations. We utilized the state-of-the-art(SOTA) reconstruction framework, EfficientSCI++, to train various compressed reconstruction models with these differing temporal correlations. Evaluating the reconstruction results from these models, our simulation experiments confirm that a reduction in temporal correlation leads to decreased reconstruction accuracy. Additionally, we simulated the reconstruction outcomes of datasets devoid of temporal correlation, illustrating that models trained on non-temporal data affect the temporal feature extraction capabilities of transformers, resulting in negligible impacts on the evaluation of reconstruction results for non-temporal correlation test datasets.展开更多
Ultrahigh-speed imaging is an essential tool for capturing fast dynamic scenes across various fields.Despite the development of numerous technical strategies,achieving ultrahigh-speed imaging with high spatiotemporal ...Ultrahigh-speed imaging is an essential tool for capturing fast dynamic scenes across various fields.Despite the development of numerous technical strategies,achieving ultrahigh-speed imaging with high spatiotemporal resolution and substantial sequence depth remains a significant challenge.To address this issue,we present a compressive ultrahigh-speed imaging technique based on acousto-optic frequency sweeping,termed AOFSCUSI.AOFS-CUSI employs light with rapidly time-varying spectra generated by acousto-optic modulation to illuminate dynamic scenes,records spatio-spectral information using snapshot compressive imaging,and ultimately reconstructs spatiotemporal information through time-spectrum mapping.This technique achieves a temporal resolution of 1.55 million frames per second,a spatial resolution of 228 lp/mm,and a sequence depth of 31 in a single shot.We experimentally validate the superior performance of AOFS-CUSI by capturing the rotation of an optical chopper,the movement of microspheres in a microchannel,and the femtosecondlaser-induced cavitation bubble dynamics.By eliminating the requirement for ultrafast laser sources and simultaneously extending the temporal window,AOFS-CUSI offers an excellent solution for recording and analyzing various fast dynamics,presenting significant potential for applications in both fundamental and applied research.展开更多
High-throughput imaging is highly desirable in intelligent analysis of computer vision tasks.In conventional design,throughput is limited by the separation between physical image capture and digital post processing.Co...High-throughput imaging is highly desirable in intelligent analysis of computer vision tasks.In conventional design,throughput is limited by the separation between physical image capture and digital post processing.Computational imaging increases throughput by mixing analog and digital processing through the image capture pipeline.Yet,recent advances of computational imaging focus on the“compressive sampling”,this precludes the wide applications in practical tasks.This paper presents a systematic analysis of the next step for computational imaging built on snapshot compressive imaging(SCI)and semantic computer vision(SCV)tasks,which have independently emerged over the past decade as basic computational imaging platforms.SCI is a physical layer process that maximizes information capacity per sample while minimizing system size,power and cost.SCV is an abstraction layer process that analyzes image data as objects and features,rather than simple pixel maps.In current practice,SCI and SCV are independent and sequential.This concatenated pipeline results in the following problems:i)a large amount of resources are spent on task-irrelevant computation and transmission,ii)the sampling and design efficiency of SCI is attenuated,and iii)the final performance of SCV is limited by the reconstruction errors of SCI.Bearing these concerns in mind,this paper takes one step further aiming to bridge the gap between SCI and SCV to take full advantage of both approaches.After reviewing the current status of SCI,we propose a novel joint framework by conducting SCV on raw measurements captured by SCI to select the region of interest,and then perform reconstruction on these regions to speed up processing time.We use our recently built SCI prototype to verify the framework.Preliminary results are presented and the prospects for a joint SCI and SCV regime are discussed.By conducting computer vision tasks in the compressed domain,we envision that a new era of snapshot compressive imaging with limited end-to-end bandwidth is coming.展开更多
Over the last few decades,ultrafast laser processing has become a widely used tool for manufacturing microstructures and nanostructures.The real-time monitoring of laser material processing provides opportunities to i...Over the last few decades,ultrafast laser processing has become a widely used tool for manufacturing microstructures and nanostructures.The real-time monitoring of laser material processing provides opportunities to inspect processes and provide feedback.To date,in-situ and real-time monitoring of laser material processing has rarely been performed.To this end,we propose dual-path snapshot compressive microscopy(DP-SCM)for high-speed,large field-of-view,and high-resolution imaging for in-situ and real-time ultrafast laser processing.In the evaluation of DP-SCM,the field of view,lateral resolution,and imaging speed were measured to be 2 mm,775 nm,and 500 fps,respectively.In ultrafast laser processing,the laser scanning process is observed using a DP-SCM system when translating the sample stage and scanning the focused femtosecond laser.Finally,we monitored the development of a self-organized nanograting structure to validate the potential of our system for unveiling new material mechanisms.The proposed method serves as an add-up(plug-and-play)module for any imaging setup and has vast potential for opening new avenues for high-throughput imaging in laser material processing.展开更多
基金supported in part by the National Natural Science Foundation of China (No. U23B2011)。
文摘Video snapshot compressive imaging(Video SCI) modulates scenes using various encoding masks and captures compressed measurements with a low-speed camera during a single exposure. Subsequently, reconstruction algorithms restore image sequences of dynamic scenes, offering advantages such as reduced bandwidth and storage space requirements. The temporal correlation in video data is crucial for Video SCI, as it leverages the temporal relationships among frames to enhance the efficiency and quality of reconstruction algorithms, particularly for fast-moving objects.This paper discretizes video frames to create image datasets with the same data volume but differing temporal correlations. We utilized the state-of-the-art(SOTA) reconstruction framework, EfficientSCI++, to train various compressed reconstruction models with these differing temporal correlations. Evaluating the reconstruction results from these models, our simulation experiments confirm that a reduction in temporal correlation leads to decreased reconstruction accuracy. Additionally, we simulated the reconstruction outcomes of datasets devoid of temporal correlation, illustrating that models trained on non-temporal data affect the temporal feature extraction capabilities of transformers, resulting in negligible impacts on the evaluation of reconstruction results for non-temporal correlation test datasets.
基金National Natural Science Foundation of China(12325408,12274129,12374274,12274139,62175066,92150102,62475070,12474404,12471368)Shanghai Municipal Education Commission(2024AI01007)+1 种基金Open Fund of Guangdong Provincial Key Laboratory of Nanophotonic Manipulation(202504)Guangdong ST Program(2023B1212010008).
文摘Ultrahigh-speed imaging is an essential tool for capturing fast dynamic scenes across various fields.Despite the development of numerous technical strategies,achieving ultrahigh-speed imaging with high spatiotemporal resolution and substantial sequence depth remains a significant challenge.To address this issue,we present a compressive ultrahigh-speed imaging technique based on acousto-optic frequency sweeping,termed AOFSCUSI.AOFS-CUSI employs light with rapidly time-varying spectra generated by acousto-optic modulation to illuminate dynamic scenes,records spatio-spectral information using snapshot compressive imaging,and ultimately reconstructs spatiotemporal information through time-spectrum mapping.This technique achieves a temporal resolution of 1.55 million frames per second,a spatial resolution of 228 lp/mm,and a sequence depth of 31 in a single shot.We experimentally validate the superior performance of AOFS-CUSI by capturing the rotation of an optical chopper,the movement of microspheres in a microchannel,and the femtosecondlaser-induced cavitation bubble dynamics.By eliminating the requirement for ultrafast laser sources and simultaneously extending the temporal window,AOFS-CUSI offers an excellent solution for recording and analyzing various fast dynamics,presenting significant potential for applications in both fundamental and applied research.
基金supported by the Ministry of Science and Technology of the People’s Republic of China[grant number 2020AAA0108202]the National Natural Science Foundation of China[grant numbers 61931012,62088102].
文摘High-throughput imaging is highly desirable in intelligent analysis of computer vision tasks.In conventional design,throughput is limited by the separation between physical image capture and digital post processing.Computational imaging increases throughput by mixing analog and digital processing through the image capture pipeline.Yet,recent advances of computational imaging focus on the“compressive sampling”,this precludes the wide applications in practical tasks.This paper presents a systematic analysis of the next step for computational imaging built on snapshot compressive imaging(SCI)and semantic computer vision(SCV)tasks,which have independently emerged over the past decade as basic computational imaging platforms.SCI is a physical layer process that maximizes information capacity per sample while minimizing system size,power and cost.SCV is an abstraction layer process that analyzes image data as objects and features,rather than simple pixel maps.In current practice,SCI and SCV are independent and sequential.This concatenated pipeline results in the following problems:i)a large amount of resources are spent on task-irrelevant computation and transmission,ii)the sampling and design efficiency of SCI is attenuated,and iii)the final performance of SCV is limited by the reconstruction errors of SCI.Bearing these concerns in mind,this paper takes one step further aiming to bridge the gap between SCI and SCV to take full advantage of both approaches.After reviewing the current status of SCI,we propose a novel joint framework by conducting SCV on raw measurements captured by SCI to select the region of interest,and then perform reconstruction on these regions to speed up processing time.We use our recently built SCI prototype to verify the framework.Preliminary results are presented and the prospects for a joint SCI and SCV regime are discussed.By conducting computer vision tasks in the compressed domain,we envision that a new era of snapshot compressive imaging with limited end-to-end bandwidth is coming.
基金supported by the National Natural Science Foundation of China(62271414)Science Fund for Distinguished Young Scholars of Zhejiang Province(LR23F010001)Research Center for Industries of the Future(RCIF)at Westlake University.and Key Project of the Westlake Institute for Optoelectronics(Grant No.2023GD007).
文摘Over the last few decades,ultrafast laser processing has become a widely used tool for manufacturing microstructures and nanostructures.The real-time monitoring of laser material processing provides opportunities to inspect processes and provide feedback.To date,in-situ and real-time monitoring of laser material processing has rarely been performed.To this end,we propose dual-path snapshot compressive microscopy(DP-SCM)for high-speed,large field-of-view,and high-resolution imaging for in-situ and real-time ultrafast laser processing.In the evaluation of DP-SCM,the field of view,lateral resolution,and imaging speed were measured to be 2 mm,775 nm,and 500 fps,respectively.In ultrafast laser processing,the laser scanning process is observed using a DP-SCM system when translating the sample stage and scanning the focused femtosecond laser.Finally,we monitored the development of a self-organized nanograting structure to validate the potential of our system for unveiling new material mechanisms.The proposed method serves as an add-up(plug-and-play)module for any imaging setup and has vast potential for opening new avenues for high-throughput imaging in laser material processing.
