Structured illumination microscopy(SIM)has been widely used in live-cell superresolution(SR)imaging.However,conventional physical model-based SIM SR reconstruction algorithms are prone to artifacts in handling raw ima...Structured illumination microscopy(SIM)has been widely used in live-cell superresolution(SR)imaging.However,conventional physical model-based SIM SR reconstruction algorithms are prone to artifacts in handling raw images with low signal-to-noise ratios(SNRs).Deep-learning(DL)-based methods can address this challenge but may lead to degradation and hallucinations.By combining the physical inversion model with a total deep variation(TDV)regularization,we propose a hybrid restoration method(TDV-SIM)that outperforms conventional or DL methods in suppressing artifacts and hallucinations while maintaining resolutions.We demonstrate the performance superiority of TDV-SIM in restoring actin filaments,endoplasmic reticulum,and mitochondrial cristae from extremely low SNR raw images.Thus TDV-SIM represents the ideal method for prolonged live-cell SR imaging with minimal exposure and photodamage.Overall,TDV-SIM proves the power of integrating model-based reconstruction methods with DL ones,possibly leading to the rapid exploration of similar strategies in high-fidelity reconstructions of other microscopy methods.展开更多
Single-image super-resolution(SISR)typically focuses on restoring various degraded low-resolution(LR)images to a single high-resolution(HR)image.However,during SISR tasks,it is often challenging for models to simultan...Single-image super-resolution(SISR)typically focuses on restoring various degraded low-resolution(LR)images to a single high-resolution(HR)image.However,during SISR tasks,it is often challenging for models to simultaneously maintain high quality and rapid sampling while preserving diversity in details and texture features.This challenge can lead to issues such as model collapse,lack of rich details and texture features in the reconstructed HR images,and excessive time consumption for model sampling.To address these problems,this paper proposes a Latent Feature-oriented Diffusion Probability Model(LDDPM).First,we designed a conditional encoder capable of effectively encoding LR images,reducing the solution space for model image reconstruction and thereby improving the quality of the reconstructed images.We then employed a normalized flow and multimodal adversarial training,learning from complex multimodal distributions,to model the denoising distribution.Doing so boosts the generative modeling capabilities within a minimal number of sampling steps.Experimental comparisons of our proposed model with existing SISR methods on mainstream datasets demonstrate that our model reconstructs more realistic HR images and achieves better performance on multiple evaluation metrics,providing a fresh perspective for tackling SISR tasks.展开更多
基金support by grants from the National Science and Technology Major Project Program(Grant Nos.2021YFA1100201,2022YFF0712500,and 2022YFC3400600)the National Natural Science Foundation of China(Grant Nos.92054301,81925022,92150301,32170691,62103071,and 31901061)+5 种基金the Beijing Natural Science Foundation(Grant No.Z20J00059)the Lingang Laboratory(Grant No.LG-QS-202206-06)Clinical Medicine Plus X-Young Scholars Project,Peking University,the Fundamental Research Funds for the Central Universities,the Natural Science Foundation of Chongqing(Grant No.cstc2021jcyj-msxmX0526)the Science and Technology Research Program of Chongqing Municipal Education Commission(Grant No.KJQN202100630)the Strategic Priority Research Program of Chinese Academy of Sciences(Grant No.XDA16021200)the High-Performance Computing Platform of Peking University.
文摘Structured illumination microscopy(SIM)has been widely used in live-cell superresolution(SR)imaging.However,conventional physical model-based SIM SR reconstruction algorithms are prone to artifacts in handling raw images with low signal-to-noise ratios(SNRs).Deep-learning(DL)-based methods can address this challenge but may lead to degradation and hallucinations.By combining the physical inversion model with a total deep variation(TDV)regularization,we propose a hybrid restoration method(TDV-SIM)that outperforms conventional or DL methods in suppressing artifacts and hallucinations while maintaining resolutions.We demonstrate the performance superiority of TDV-SIM in restoring actin filaments,endoplasmic reticulum,and mitochondrial cristae from extremely low SNR raw images.Thus TDV-SIM represents the ideal method for prolonged live-cell SR imaging with minimal exposure and photodamage.Overall,TDV-SIM proves the power of integrating model-based reconstruction methods with DL ones,possibly leading to the rapid exploration of similar strategies in high-fidelity reconstructions of other microscopy methods.
基金supported by General Project of Guangxi Science and Technology Major Project(AA19254016)Beihai City Science and Technology Planning Project(202082033)+1 种基金Beihai City Science and Technology Planning Project(202082023)Guangxi Graduate Student Innovation Project(YCSW2021174)。
文摘Single-image super-resolution(SISR)typically focuses on restoring various degraded low-resolution(LR)images to a single high-resolution(HR)image.However,during SISR tasks,it is often challenging for models to simultaneously maintain high quality and rapid sampling while preserving diversity in details and texture features.This challenge can lead to issues such as model collapse,lack of rich details and texture features in the reconstructed HR images,and excessive time consumption for model sampling.To address these problems,this paper proposes a Latent Feature-oriented Diffusion Probability Model(LDDPM).First,we designed a conditional encoder capable of effectively encoding LR images,reducing the solution space for model image reconstruction and thereby improving the quality of the reconstructed images.We then employed a normalized flow and multimodal adversarial training,learning from complex multimodal distributions,to model the denoising distribution.Doing so boosts the generative modeling capabilities within a minimal number of sampling steps.Experimental comparisons of our proposed model with existing SISR methods on mainstream datasets demonstrate that our model reconstructs more realistic HR images and achieves better performance on multiple evaluation metrics,providing a fresh perspective for tackling SISR tasks.
