The current state of traditional optoelectronic imaging technology is constrained by the inherent limitations of its hardware.These limitations pose significant challenges in acquiring higher-dimensional information a...The current state of traditional optoelectronic imaging technology is constrained by the inherent limitations of its hardware.These limitations pose significant challenges in acquiring higher-dimensional information and reconstructing accurate images,particularly in applications such as scattering imaging,superresolution,and complex scene reconstruction.However,the rapid development and widespread adoption of deep learning are reshaping the field of optical imaging through computational imaging technology.Datadriven computational imaging has ushered in a paradigm shift by leveraging the nonlinear expression and feature learning capabilities of neural networks.This approach transcends the limitations of conventional physical models,enabling the adaptive extraction of critical features directly from data.As a result,computational imaging overcomes the traditional“what you see is what you get”paradigm,paving the way for more compact optical system designs,broader information acquisition,and improved image reconstruction accuracy.These advancements have significantly enhanced the interpretation of highdimensional light-field information and the processing of complex images.This review presents a comprehensive analysis of the integration of deep learning and computational imaging,emphasizing its transformative potential in three core areas:computational optical system design,high-dimensional information interpretation,and image enhancement and processing.Additionally,this review addresses the challenges and future directions of this cutting-edge technology,providing novel insights into interdisciplinary imaging research.展开更多
We present a new label-free three-dimensional(3D)microscopy technique,termed transport of intensity diffraction tomography with non-interferometric synthetic aperture(TIDT-NSA).Without resorting to interferometric det...We present a new label-free three-dimensional(3D)microscopy technique,termed transport of intensity diffraction tomography with non-interferometric synthetic aperture(TIDT-NSA).Without resorting to interferometric detection,TIDT-NSA retrieves the 3D refractive index(RI)distribution of biological specimens from 3D intensity-only measurements at various illumination angles,allowing incoherent-diffraction-limited quantitative 3D phase-contrast imaging.The unique combination of z-scanning the sample with illumination angle diversity in TIDT-NSA provides strong defocus phase contrast and better optical sectioning capabilities suitable for high-resolution tomography of thick biological samples.Based on an off-the-shelf bright-field microscope with a programmable light-emitting-diode(LED)illumination source,TIDT-NSA achieves an imaging resolution of 206 nm laterally and 520 nm axially with a high-NA oil immersion objective.We validate the 3D RI tomographic imaging performance on various unlabeled fixed and live samples,including human breast cancer cell lines MCF-7,human hepatocyte carcinoma cell lines HepG2,mouse macrophage cell lines RAW 264.7,Caenorhabditis elegans(C.elegans),and live Henrietta Lacks(HeLa)cells.These results establish TIDT-NSA as a new non-interferometric approach to optical diffraction tomography and 3D label-free microscopy,permitting quantitative characterization of cell morphology and time-dependent subcellular changes for widespread biological and medical applications.展开更多
Computational microscopy,as a subfield of computational imaging,combines optical manipulation and image algorithmic reconstruction to recover multi-dimensional microscopic images or information of micro-objects.In rec...Computational microscopy,as a subfield of computational imaging,combines optical manipulation and image algorithmic reconstruction to recover multi-dimensional microscopic images or information of micro-objects.In recent years,the revolution in light-emitting diodes(LEDs),low-cost consumer image sensors,modern digital computers,and smartphones provide fertile opportunities for the rapid development of computational microscopy.Consequently,diverse forms of computational microscopy have been invented,including digital holographic microscopy(DHM),transport of intensity equation(TIE),differential phase contrast(DPC)microscopy,lens-free on-chip holography,and Fourier ptychographic microscopy(FPM).These computational microscopy techniques not only provide high-resolution,label-free,quantitative phase imaging capability but also decipher new and advanced biomedical research and industrial applications.Nevertheless,most computational microscopy techniques are still at an early stage of“proof of concept”or“proof of prototype”(based on commercially available microscope platforms).Translating those concepts to stand-alone optical instruments for practical use is an essential step for the promotion and adoption of computational microscopy by the wider bio-medicine,industry,and education community.In this paper,we present four smart computational light microscopes(SCLMs)developed by our laboratory,i.e.,smart computational imaging laboratory(SCILab)of Nanjing University of Science and Technology(NJUST),China.These microscopes are empowered by advanced computational microscopy techniques,including digital holography,TIE,DPC,lensless holography,and FPM,which not only enables multi-modal contrast-enhanced observations for unstained specimens,but also can recover their three-dimensional profiles quantitatively.We introduce their basic principles,hardware configurations,reconstruction algorithms,and software design,quantify their imaging performance,and illustrate their typical applications for cell analysis,medical diagnosis,and microlens characterization.展开更多
Transport of intensity equation(TIE)is a well-established non-interferometric phase retrieval approach that enables quantitative phase imaging(QPI)by simply measuring intensity images at multiple axially displaced pla...Transport of intensity equation(TIE)is a well-established non-interferometric phase retrieval approach that enables quantitative phase imaging(QPI)by simply measuring intensity images at multiple axially displaced planes.The advantage of a TIE-based QPI system is its compatibility with partially coherent illumination,which provides speckle-free imaging with resolution beyond the coherent diffraction limit.However,TIE is generally implemented with a brightfield(BF)configuration,and the maximum achievable imaging resolution is still limited to the incoherent diffraction limit(twice the coherent diffraction limit).It is desirable that TIE-related approaches can surpass this limit and achieve high-throughput[high-resolution and wide field of view(FOV)]QPI.We propose a hybrid BF and darkfield transport of intensity(HBDTI)approach for highthroughput quantitative phase microscopy.Two through-focus intensity stacks corresponding to BF and darkfield illuminations are acquired through a low-numerical-aperture(NA)objective lens.The high-resolution and large-FOV complex amplitude(both quantitative absorption and phase distributions)can then be synthesized based on an iterative phase retrieval algorithm taking the coherence model decomposition into account.The effectiveness of the proposed method is experimentally verified by the retrieval of the USAF resolution target and different types of biological cells.The experimental results demonstrate that the half-width imaging resolution can be improved from 1230 nm to 488 nm with 2.5×expansion across a 4×FOV of 7.19 mm2,corresponding to a 6.25×increase in space-bandwidth product from∼5 to∼30.2 megapixels.In contrast to conventional TIE-based QPI methods where only BF illumination is used,the synthetic aperture process of HBDTI further incorporates darkfield illuminations to expand the accessible object frequency,thereby significantly extending the maximum available resolution from 2NA to∼5NA with a∼5×promotion of the coherent diffraction limit.Given its capability for high-throughput QPI,the proposed HBDTI approach is expected to be adopted in biomedical fields,such as personalized genomics and cancer diagnostics.展开更多
The transport-of-intensity equation(TIE) enables quantitative phase imaging(QPI) under partially coherent illumination by measuring the through-focus intensities combined with a linearized inverse reconstruction algor...The transport-of-intensity equation(TIE) enables quantitative phase imaging(QPI) under partially coherent illumination by measuring the through-focus intensities combined with a linearized inverse reconstruction algorithm. However, overcoming its sensitivity to imaging settings remains a challenging problem because of the difficulty in tuning the optical parameters of the imaging system accurately and because of the instability to long-time measurements. To address these limitations, we propose and experimentally validate a solution called neural-field-assisted transport-of-intensity phase microscopy(NFTPM) by introducing a tunable defocus parameter into neural field. Without weak object approximation, NFTPM incorporates the physical prior of partially coherent image formation to constrain the neural field and learns the continuous representation of phase object without the need for training. Simulation and experimental results of He La cells demonstrate that NFTPM can achieve accurate, partially coherent QPI under unknown defocus distances, providing new possibilities for extending applications in live cell biology.展开更多
Lens-free on-chip microscopy is a powerful and promising high-throughput computational microscopy technique due to its unique advantage of creating high-resolution images across the full field-of-view(FOV)of the imagi...Lens-free on-chip microscopy is a powerful and promising high-throughput computational microscopy technique due to its unique advantage of creating high-resolution images across the full field-of-view(FOV)of the imaging sensor.Nevertheless,most current lens-free microscopy methods have been designed for imaging only two-dimensional thin samples.Lens-free on-chip tomography(LFOCT)with a uniform resolution across the entire FOV and at a subpixel level remains a critical challenge.In this paper,we demonstrated a new LFOCT technique and associated imaging platform based on wavelength scanning Fourier ptychographic diffraction tomography(wsFPDT).Instead of using angularlyvariable illuminations,in wsFPDT,the sample is illuminated by on-axis wavelength-variable illuminations,ranging from 430 to 1200 nm.The corresponding under-sampled diffraction patterns are recorded,and then an iterative ptychographic reconstruction procedure is applied to fill the spectrum of the three-dimensional(3D)scattering potential to recover the sample’s 3D refractive index(RI)distribution.The wavelength-scanning scheme not only eliminates the need for mechanical motion during image acquisition and precise registration of the raw images but secures a quasi-uniform,pixel-super-resolved imaging resolution across the entire imaging FOV.With wsFPDT,we demonstrate the high-throughput,billion-voxel 3D tomographic imaging results with a half-pitch lateral resolution of 775 nm and an axial resolution of 5.43μm across a large FOV of 29.85mm2 and an imaging depth of>200μm.The effectiveness of the proposed method was demonstrated by imaging various types of samples,including micropolystyrene beads,diatoms,and mouse mononuclear macrophage cells.The unique capability to reveal quantitative morphological properties,such as area,volume,and sphericity index of single cell over large cell populations makes wsFPDT a powerful quantitative and label-free tool for high-throughput biological applications.展开更多
The transport of intensity equation(TIE)is a well-established phase retrieval technique that enables incoherent diffraction limit-resolution imaging and is compatible with widely available brightfield microscopy hardw...The transport of intensity equation(TIE)is a well-established phase retrieval technique that enables incoherent diffraction limit-resolution imaging and is compatible with widely available brightfield microscopy hardware.However,existing TIE methods encounter difficulties in decoupling the independent contributions of phase and aberrations to the measurements in the case of unknown pupil function.Additionally,spatially nonuniform and temporally varied aberrations dramatically degrade the imaging performance for long-term research.Hence,it remains a critical challenge to realize the high-throughput quantitative phase imaging(QPI)with aberration correction under partially coherent illumination.To address these issues,we propose a novel method for highthroughput microscopy with annular illumination,termed as transport-of-intensity QPI with aberration correction(TI-AC).By combining aberration correction and pixel super-resolution technique,TI-AC is made compatible with large pixel-size sensors to enable a broader field of view.Furthermore,it surpasses the theoretical Nyquist-Shannon sampling resolution limit,resulting in an improvement of more than two times.Experimental results demonstrate that the half-width imaging resolution can be improved to~345 nm across a 10×field of view of 1.77 mm^(2)(0.4 NA).Given its high-throughput capability for QPI,TI-AC is expected to be adopted in biomedical fields,such as drug discovery and cancer diagnostics.展开更多
基金supported by the National Natural Science Foundation of China(Nos.62405231,62205259,62075175,62105254,and 62375212)the National Key Laboratory of Infrared Detection Technologies(No.IRDT-23-06)+1 种基金the Fundamental Research Funds for the Central Universities(Nos.XJSJ24028 and XJS222202)the Open Research Fund of Beijing Key Laboratory of Advanced Optical Remote Sensing Technology(No.AORS202405).
文摘The current state of traditional optoelectronic imaging technology is constrained by the inherent limitations of its hardware.These limitations pose significant challenges in acquiring higher-dimensional information and reconstructing accurate images,particularly in applications such as scattering imaging,superresolution,and complex scene reconstruction.However,the rapid development and widespread adoption of deep learning are reshaping the field of optical imaging through computational imaging technology.Datadriven computational imaging has ushered in a paradigm shift by leveraging the nonlinear expression and feature learning capabilities of neural networks.This approach transcends the limitations of conventional physical models,enabling the adaptive extraction of critical features directly from data.As a result,computational imaging overcomes the traditional“what you see is what you get”paradigm,paving the way for more compact optical system designs,broader information acquisition,and improved image reconstruction accuracy.These advancements have significantly enhanced the interpretation of highdimensional light-field information and the processing of complex images.This review presents a comprehensive analysis of the integration of deep learning and computational imaging,emphasizing its transformative potential in three core areas:computational optical system design,high-dimensional information interpretation,and image enhancement and processing.Additionally,this review addresses the challenges and future directions of this cutting-edge technology,providing novel insights into interdisciplinary imaging research.
基金This work was supported by the National Natural Science Foundationof China(61905115,62105151,U21B2033)Leading Technology of Jiangsu Basic Research Plan(BK20192003)+2 种基金Youth Foundationof Jiangsu Province(BK20190445,BK20210338)Fundamental ResearchFundsfortheCentral Universities(30920032101)Open Research Fund of Jjiangsu Key Laboratory of Spectral Imaging&Intelligent Sense(USGP202105).
文摘We present a new label-free three-dimensional(3D)microscopy technique,termed transport of intensity diffraction tomography with non-interferometric synthetic aperture(TIDT-NSA).Without resorting to interferometric detection,TIDT-NSA retrieves the 3D refractive index(RI)distribution of biological specimens from 3D intensity-only measurements at various illumination angles,allowing incoherent-diffraction-limited quantitative 3D phase-contrast imaging.The unique combination of z-scanning the sample with illumination angle diversity in TIDT-NSA provides strong defocus phase contrast and better optical sectioning capabilities suitable for high-resolution tomography of thick biological samples.Based on an off-the-shelf bright-field microscope with a programmable light-emitting-diode(LED)illumination source,TIDT-NSA achieves an imaging resolution of 206 nm laterally and 520 nm axially with a high-NA oil immersion objective.We validate the 3D RI tomographic imaging performance on various unlabeled fixed and live samples,including human breast cancer cell lines MCF-7,human hepatocyte carcinoma cell lines HepG2,mouse macrophage cell lines RAW 264.7,Caenorhabditis elegans(C.elegans),and live Henrietta Lacks(HeLa)cells.These results establish TIDT-NSA as a new non-interferometric approach to optical diffraction tomography and 3D label-free microscopy,permitting quantitative characterization of cell morphology and time-dependent subcellular changes for widespread biological and medical applications.
基金supported by the National Natural Science Foundation of China(61905115)Leading Technology of Jiangsu Basic Research Plan(BK20192003)+3 种基金National Defense Science and Technology Foundation of China(2019-JCJQ-JJ-381)Youth Foundation of Jiangsu Province(BK20190445)Fundamental Research Funds for the Central Universities(30920032101)Open Research Fund of Jiangsu Key Laboratory of Spectral Imaging&Intelligent Sense(3091801410411).
文摘Computational microscopy,as a subfield of computational imaging,combines optical manipulation and image algorithmic reconstruction to recover multi-dimensional microscopic images or information of micro-objects.In recent years,the revolution in light-emitting diodes(LEDs),low-cost consumer image sensors,modern digital computers,and smartphones provide fertile opportunities for the rapid development of computational microscopy.Consequently,diverse forms of computational microscopy have been invented,including digital holographic microscopy(DHM),transport of intensity equation(TIE),differential phase contrast(DPC)microscopy,lens-free on-chip holography,and Fourier ptychographic microscopy(FPM).These computational microscopy techniques not only provide high-resolution,label-free,quantitative phase imaging capability but also decipher new and advanced biomedical research and industrial applications.Nevertheless,most computational microscopy techniques are still at an early stage of“proof of concept”or“proof of prototype”(based on commercially available microscope platforms).Translating those concepts to stand-alone optical instruments for practical use is an essential step for the promotion and adoption of computational microscopy by the wider bio-medicine,industry,and education community.In this paper,we present four smart computational light microscopes(SCLMs)developed by our laboratory,i.e.,smart computational imaging laboratory(SCILab)of Nanjing University of Science and Technology(NJUST),China.These microscopes are empowered by advanced computational microscopy techniques,including digital holography,TIE,DPC,lensless holography,and FPM,which not only enables multi-modal contrast-enhanced observations for unstained specimens,but also can recover their three-dimensional profiles quantitatively.We introduce their basic principles,hardware configurations,reconstruction algorithms,and software design,quantify their imaging performance,and illustrate their typical applications for cell analysis,medical diagnosis,and microlens characterization.
基金the National Natural Science Foundation of China(61905115,62105151,62175109,and U21B2033)Leading Technology of Jiangsu Basic Research Plan(BK20192003)+2 种基金Youth Foundation of Jiangsu Province(BK20190445,BK20210338)Fundamental Research Funds for the Central Universities(30920032101)Open Research Fund of Jiangsu Key Laboratory of Spectral Imaging and Intelligent Sense(JSGP202105).
文摘Transport of intensity equation(TIE)is a well-established non-interferometric phase retrieval approach that enables quantitative phase imaging(QPI)by simply measuring intensity images at multiple axially displaced planes.The advantage of a TIE-based QPI system is its compatibility with partially coherent illumination,which provides speckle-free imaging with resolution beyond the coherent diffraction limit.However,TIE is generally implemented with a brightfield(BF)configuration,and the maximum achievable imaging resolution is still limited to the incoherent diffraction limit(twice the coherent diffraction limit).It is desirable that TIE-related approaches can surpass this limit and achieve high-throughput[high-resolution and wide field of view(FOV)]QPI.We propose a hybrid BF and darkfield transport of intensity(HBDTI)approach for highthroughput quantitative phase microscopy.Two through-focus intensity stacks corresponding to BF and darkfield illuminations are acquired through a low-numerical-aperture(NA)objective lens.The high-resolution and large-FOV complex amplitude(both quantitative absorption and phase distributions)can then be synthesized based on an iterative phase retrieval algorithm taking the coherence model decomposition into account.The effectiveness of the proposed method is experimentally verified by the retrieval of the USAF resolution target and different types of biological cells.The experimental results demonstrate that the half-width imaging resolution can be improved from 1230 nm to 488 nm with 2.5×expansion across a 4×FOV of 7.19 mm2,corresponding to a 6.25×increase in space-bandwidth product from∼5 to∼30.2 megapixels.In contrast to conventional TIE-based QPI methods where only BF illumination is used,the synthetic aperture process of HBDTI further incorporates darkfield illuminations to expand the accessible object frequency,thereby significantly extending the maximum available resolution from 2NA to∼5NA with a∼5×promotion of the coherent diffraction limit.Given its capability for high-throughput QPI,the proposed HBDTI approach is expected to be adopted in biomedical fields,such as personalized genomics and cancer diagnostics.
基金National Natural Science Foundation of China(62227818, 62105151, 62175109, U21B2033)National Key Research and Development Program of China(2022YFA1205002)+6 种基金Leading Technology of Jiangsu Basic Research Plan (BK20192003)Youth Foundation of Jiangsu Province (BK20210338)Biomedical Competition Foundation of Jiangsu Province (BE2022847)Key National Industrial Technology Cooperation Foundation of Jiangsu Province (BZ2022039)Fundamental Research Funds for the Central Universities (30920032101, 30923010206)Fundamental Scientific Research Business Fee Funds for the Central Universities (2023102001)Open Research Fund of Jiangsu Key Laboratory of Spectral Imaging Intelligent Sense(JSGP202105, JSGP202201)。
文摘The transport-of-intensity equation(TIE) enables quantitative phase imaging(QPI) under partially coherent illumination by measuring the through-focus intensities combined with a linearized inverse reconstruction algorithm. However, overcoming its sensitivity to imaging settings remains a challenging problem because of the difficulty in tuning the optical parameters of the imaging system accurately and because of the instability to long-time measurements. To address these limitations, we propose and experimentally validate a solution called neural-field-assisted transport-of-intensity phase microscopy(NFTPM) by introducing a tunable defocus parameter into neural field. Without weak object approximation, NFTPM incorporates the physical prior of partially coherent image formation to constrain the neural field and learns the continuous representation of phase object without the need for training. Simulation and experimental results of He La cells demonstrate that NFTPM can achieve accurate, partially coherent QPI under unknown defocus distances, providing new possibilities for extending applications in live cell biology.
基金supported by the National Key Research and Development Program of China(2022YFA1205002,2024YFE0101300)National Natural Science Foundation of China(62105151,62175109,U21B2033,62227818,62361136588)+6 种基金Leading Technology of Jiangsu Basic Research Plan(BK20192003)Youth Foundation of Jiangsu Province(BK20210338)Biomedical Competition Foundation of Jiangsu Province(BE2022847)Key National Industrial Technology Cooperation Foundation of Jiangsu Province(BZ2022039)Fundamental Research Funds for the Central Universities(30920032101,30923010206)Fundamental Scientific Research Business Fee Funds for the Central Universities(2023102001)Open Research Fund of Jiangsu Key Laboratory of Spectral Imaging&Intelligent Sense(JSGP202105,JSGP202201).
文摘Lens-free on-chip microscopy is a powerful and promising high-throughput computational microscopy technique due to its unique advantage of creating high-resolution images across the full field-of-view(FOV)of the imaging sensor.Nevertheless,most current lens-free microscopy methods have been designed for imaging only two-dimensional thin samples.Lens-free on-chip tomography(LFOCT)with a uniform resolution across the entire FOV and at a subpixel level remains a critical challenge.In this paper,we demonstrated a new LFOCT technique and associated imaging platform based on wavelength scanning Fourier ptychographic diffraction tomography(wsFPDT).Instead of using angularlyvariable illuminations,in wsFPDT,the sample is illuminated by on-axis wavelength-variable illuminations,ranging from 430 to 1200 nm.The corresponding under-sampled diffraction patterns are recorded,and then an iterative ptychographic reconstruction procedure is applied to fill the spectrum of the three-dimensional(3D)scattering potential to recover the sample’s 3D refractive index(RI)distribution.The wavelength-scanning scheme not only eliminates the need for mechanical motion during image acquisition and precise registration of the raw images but secures a quasi-uniform,pixel-super-resolved imaging resolution across the entire imaging FOV.With wsFPDT,we demonstrate the high-throughput,billion-voxel 3D tomographic imaging results with a half-pitch lateral resolution of 775 nm and an axial resolution of 5.43μm across a large FOV of 29.85mm2 and an imaging depth of>200μm.The effectiveness of the proposed method was demonstrated by imaging various types of samples,including micropolystyrene beads,diatoms,and mouse mononuclear macrophage cells.The unique capability to reveal quantitative morphological properties,such as area,volume,and sphericity index of single cell over large cell populations makes wsFPDT a powerful quantitative and label-free tool for high-throughput biological applications.
基金supported by the National Natural Science Foundation of China(62227818,62105151,62175109,U21B2033,62105156,62361136588)National Key Research and Development Program of China(2022YFA1205002)+7 种基金Leading Technology of Jiangsu Basic Research Plan(BK20192003)Youth Foundation of Jiangsu Province(BK20210338)Biomedical Competition Foundation of Jia-ngsu Province(BE2022847)Key National Industrial Technology Cooperation Foundation of Jiangsu Province(BZ2022039)Fundamental Research Funds for the Central Universities(30920032101,30923010206)Fundamental Scientific Research Business Fee Funds for the Central Universities(2023102001)Open Research Fund of Jiangsu Key Laboratory of Spectral Imaging&Intelligent Sense(JSGP202105,JSGP202201)National Science Center,Poland(2020/37/B/ST7/03629).
文摘The transport of intensity equation(TIE)is a well-established phase retrieval technique that enables incoherent diffraction limit-resolution imaging and is compatible with widely available brightfield microscopy hardware.However,existing TIE methods encounter difficulties in decoupling the independent contributions of phase and aberrations to the measurements in the case of unknown pupil function.Additionally,spatially nonuniform and temporally varied aberrations dramatically degrade the imaging performance for long-term research.Hence,it remains a critical challenge to realize the high-throughput quantitative phase imaging(QPI)with aberration correction under partially coherent illumination.To address these issues,we propose a novel method for highthroughput microscopy with annular illumination,termed as transport-of-intensity QPI with aberration correction(TI-AC).By combining aberration correction and pixel super-resolution technique,TI-AC is made compatible with large pixel-size sensors to enable a broader field of view.Furthermore,it surpasses the theoretical Nyquist-Shannon sampling resolution limit,resulting in an improvement of more than two times.Experimental results demonstrate that the half-width imaging resolution can be improved to~345 nm across a 10×field of view of 1.77 mm^(2)(0.4 NA).Given its high-throughput capability for QPI,TI-AC is expected to be adopted in biomedical fields,such as drug discovery and cancer diagnostics.
