Light-field imaging has wide applications in various domains,including microscale life science imaging,mesoscale neuroimaging,and macroscale fluid dynamics imaging.The development of deep learning-based reconstruction...Light-field imaging has wide applications in various domains,including microscale life science imaging,mesoscale neuroimaging,and macroscale fluid dynamics imaging.The development of deep learning-based reconstruction methods has greatly facilitated high-resolution light-field image processing,however,current deep learning-based light-field reconstruction methods have predominantly concentrated on the microscale.Considering the multiscale imaging capacity of light-field technique,a network that can work over variant scales of light-field image reconstruction will significantly benefit the development of volumetric imaging.Unfortunately,to our knowledge,no one has reported a universal high-resolution light-field image reconstruction algorithm that is compatible with microscale,mesoscale,and macroscale.To fill this gap,we present a real-time and universal network(RTU-Net)to reconstruct high-resolution light-field images at any scale.RTU-Net,as the first network that works over multiscale light-field image reconstruction,employs an adaptive loss function based on generative adversarial theory and consequently exhibits strong generalization capability.We comprehensively assessed the performance of RTU-Net through the reconstruction of multiscale light-field images,including microscale tubulin and mitochondrion dataset,mesoscale synthetic mouse neuro dataset,and macroscale light-field particle imaging velocimetry dataset.The results indicated that RTU-Net has achieved real-time and high-resolution light-field image reconstruction for volume sizes ranging from 300μm×300μm×12μm to 25 mm×25 mm×25 mm,and demonstrated higher resolution when compared with recently reported light-field reconstruction networks.The high-resolution,strong robustness,high efficiency,and especially the general applicability of RTU-Net will significantly deepen our insight into high-resolution and volumetric imaging.展开更多
基金supported by National Natural Science Foundation of China(12402336,82201637,U20A2070,and 12025202)National High-Level Talent Project(YQR23069)+6 种基金Natural Science Foundation of Jiangsu Province(BK20230876)the Young Elite Scientist Sponsorship Program by CAST(YESS20210238)Forwardlooking layout projects(1002-ILB24009)Zhejang Provincial Medical and Health Technology Project(Grant No.2024KY246,2025KY180)Scientific Research Foundation of Hangzhou City University(No.J-202402)Open Research Fund of the State Key Laboratory of Brain-Machine Intelligence,Zhejiang University(Grant No.BMI2400025)Hangzhou Science and Technology Bureau.
文摘Light-field imaging has wide applications in various domains,including microscale life science imaging,mesoscale neuroimaging,and macroscale fluid dynamics imaging.The development of deep learning-based reconstruction methods has greatly facilitated high-resolution light-field image processing,however,current deep learning-based light-field reconstruction methods have predominantly concentrated on the microscale.Considering the multiscale imaging capacity of light-field technique,a network that can work over variant scales of light-field image reconstruction will significantly benefit the development of volumetric imaging.Unfortunately,to our knowledge,no one has reported a universal high-resolution light-field image reconstruction algorithm that is compatible with microscale,mesoscale,and macroscale.To fill this gap,we present a real-time and universal network(RTU-Net)to reconstruct high-resolution light-field images at any scale.RTU-Net,as the first network that works over multiscale light-field image reconstruction,employs an adaptive loss function based on generative adversarial theory and consequently exhibits strong generalization capability.We comprehensively assessed the performance of RTU-Net through the reconstruction of multiscale light-field images,including microscale tubulin and mitochondrion dataset,mesoscale synthetic mouse neuro dataset,and macroscale light-field particle imaging velocimetry dataset.The results indicated that RTU-Net has achieved real-time and high-resolution light-field image reconstruction for volume sizes ranging from 300μm×300μm×12μm to 25 mm×25 mm×25 mm,and demonstrated higher resolution when compared with recently reported light-field reconstruction networks.The high-resolution,strong robustness,high efficiency,and especially the general applicability of RTU-Net will significantly deepen our insight into high-resolution and volumetric imaging.
