Single-pixel imaging(SPI)enables an invisible target to be imaged onto a photosensitive surface without a lens,emerging as a promising way for indirect optical encryption.However,due to its linear and broadcast imagin...Single-pixel imaging(SPI)enables an invisible target to be imaged onto a photosensitive surface without a lens,emerging as a promising way for indirect optical encryption.However,due to its linear and broadcast imaging principles,SPI encryption has been confined to a single-user framework for the long term.We propose a multi-image SPI encryption method and combine it with orthogonal frequency division multiplexing-assisted key management,to achieve a multiuser SPI encryption and authentication framework.Multiple images are first encrypted as a composite intensity sequence containing the plaintexts and authentication information,simultaneously generating different sets of keys for users.Then,the SPI keys for encryption and authentication are asymmetrically isolated into independent frequency carriers and encapsulated into a Malus metasurface,so as to establish an individually private and content-independent channel for each user.Users can receive different plaintexts privately and verify the authenticity,eliminating the broadcast transparency of SPI encryption.The improved linear security is also verified by simulating attacks.By the combination of direct key management and indirect image encryption,our work achieves the encryption and authentication functionality under a multiuser computational imaging framework,facilitating its application in optical communication,imaging,and security.展开更多
Coherent optical control within or through scattering media via wavefront shaping has seen broad applications since its invention around 2007.Wavefront shaping is aimed at overcoming the strong scattering,featured by ...Coherent optical control within or through scattering media via wavefront shaping has seen broad applications since its invention around 2007.Wavefront shaping is aimed at overcoming the strong scattering,featured by random interference,namely speckle patterns.This randomness occurs due to the refractive index inhomogeneity in complex media like biological tissue or the modal dispersion in multimode fiber,yet this randomness is actually deterministic and potentially can be time reversal or precompensated.Various wavefront shaping approaches,such as optical phase conjugation,iterative optimization,and transmission matrix measurement,have been developed to generate tight and intense optical delivery or high-resolution image of an optical object behind or within a scattering medium.The performance of these modula-tions,however,is far from satisfaction.Most recently,artifcial intelligence has brought new inspirations to this field,providing exciting hopes to tackle the challenges by mapping the input and output optical patterns and building a neuron network that inherently links them.In this paper,we survey the developments to date on this topic and briefly discuss our views on how to harness machine learning(deep learning in particular)for further advancements in the field.展开更多
The current state of traditional optoelectronic imaging technology is constrained by the inherent limitations of its hardware.These limitations pose significant challenges in acquiring higher-dimensional information a...The current state of traditional optoelectronic imaging technology is constrained by the inherent limitations of its hardware.These limitations pose significant challenges in acquiring higher-dimensional information and reconstructing accurate images,particularly in applications such as scattering imaging,superresolution,and complex scene reconstruction.However,the rapid development and widespread adoption of deep learning are reshaping the field of optical imaging through computational imaging technology.Datadriven computational imaging has ushered in a paradigm shift by leveraging the nonlinear expression and feature learning capabilities of neural networks.This approach transcends the limitations of conventional physical models,enabling the adaptive extraction of critical features directly from data.As a result,computational imaging overcomes the traditional“what you see is what you get”paradigm,paving the way for more compact optical system designs,broader information acquisition,and improved image reconstruction accuracy.These advancements have significantly enhanced the interpretation of highdimensional light-field information and the processing of complex images.This review presents a comprehensive analysis of the integration of deep learning and computational imaging,emphasizing its transformative potential in three core areas:computational optical system design,high-dimensional information interpretation,and image enhancement and processing.Additionally,this review addresses the challenges and future directions of this cutting-edge technology,providing novel insights into interdisciplinary imaging research.展开更多
Metasurface-based imaging has attracted considerable attention owing to its compactness,multifunctionality,and subwavelength coding capability.With the integration of computational imaging techniques,researchers have ...Metasurface-based imaging has attracted considerable attention owing to its compactness,multifunctionality,and subwavelength coding capability.With the integration of computational imaging techniques,researchers have actively explored the extended capabilities of metasurfaces,enabling a wide range of imaging methods.We present an overview of the recent progress in metasurface-based imaging techniques,focusing on the perspective of computational imaging.Specifically,we categorize and review existing metasurface-based imaging into three main groups,including(i)conventional metasurface design employing canonical methods,(ii)computation introduced independently in either the imaging process or postprocessing,and(iii)an end-to-end computation-optimized imaging system based upon metasurfaces.We highlight the advantages and challenges associated with each computational metasurface-based imaging technique and discuss the potential and future prospects of the computational boosted metaimager.展开更多
BACKGROUND The diagnostic accuracy for detecting metastatic lymph nodes in colorectal cancer(CRC)remains suboptimal.To address this limitation,our study investigates the potential of gemstone spectral computed tomogra...BACKGROUND The diagnostic accuracy for detecting metastatic lymph nodes in colorectal cancer(CRC)remains suboptimal.To address this limitation,our study investigates the potential of gemstone spectral computed tomography imaging(GSI)to improve diagnostic accuracy in lymph node metastasis(LNM)assessment.AIM To extensively investigate the clinical utility of GSI in the preoperative assessment of CRC.METHODS The subject population included 200 patients with CRC who were admitted to Zibo Central Hospital from January 2022 to December 2023.All patients underwent dual-phase contrast-enhanced scans in the arterial and venous phases using GSI before surgical intervention.During the research,meticulous quantification was conducted regarding the number of patients with CRC with LNM as well as the exact count of metastatic lymph nodes.Moreover,for both metastatic and non-metastatic lymph nodes,the short diameter at the maximum crosssectional area(covering the axial,sagittal,and coronal planes),morphological features(including manifestations such as margin blurring,aggregation,and enhancement),and spectral parameters in the arterial and venous phases[specifically iodine concentration(IC),normalized IC(NIC),and the slope of the spectral curve(λHU)]were measured and recorded,and a comparative analysis was conducted.The diagnostic efficacy of each index with differences was systematically assessed using the receiver operating characteristic(ROC)curve.Concurrently,receiver operating characteristic curves were constructed for LNM screening based on the short diameter at the maximum cross-sectional area of lymph nodes and each spectral parameter in the arterial and venous phases.RESULTS The area under the curve of GSI for diagnosing LNM in patients with CRC can reach 0.897,with sensitivity,specificity,and accuracy of 92.59%,85.87%,and 89.50%,respectively.A total of 265 lymph nodes were analyzed from the 200 participants with CRC,with metastatic lymph nodes accounting for 56.60%.Compared with nonmetastatic lymph nodes,the short diameters of metastatic lymph nodes in the axial,sagittal,and coronal planes were significantly increased,whereas the IC values in the arterial and venous phases,the NIC value in the arterial phase,and theλHU values in the arterial and venous phases were significantly decreased.The short axial,sagittal,and coronal diameters,arterial-phase IC,venous-phase IC,arterial-phase NIC,arterial-phaseλHU,and venousphaseλHU for diagnosing metastatic lymph nodes demonstrated area under the curve values of 0.631,0.681,0.659,0.862,0.808,0.831,0.801,and 0.706,respectively.CONCLUSION GSI exhibits substantial clinical significance in the preoperative assessment of CRC.Among the parameters assessed,the arterial-phase IC demonstrates the most outstanding diagnostic performance,effectively improving the diagnostic efficacy for preoperative LNM in CRC.展开更多
Imaging through diffusers presents a challenging problem with various digital image reconstruction solutions demonstrated to date using computers.Here,we present a computer-free,all-optical image reconstruction method...Imaging through diffusers presents a challenging problem with various digital image reconstruction solutions demonstrated to date using computers.Here,we present a computer-free,all-optical image reconstruction method to see through random diffusers at the speed of light.Using deep learning,a set of transmissive diffractive surfaces are trained to all-optically reconstruct images of arbitrary objects that are completely covered by unknown,random phase diffusers.After the training stage,which is a one-time effort,the resulting diffractive surfaces are fabricated and form a passive optical network that is physically positioned between the unknown object and the image plane to all-optically reconstruct the object pattern through an unknown,new phase diffuser.We experimentally demonstrated this concept using coherent THz illumination and all-optically reconstructed objects distorted by unknown,random diffusers,never used during training.Unlike digital methods,all-optical diffractive reconstructions do not require power except for the illumination light.This diffractive solution to see through diffusers can be extended to other wavelengths,and might fuel various applications in biomedical imaging,astronomy,atmospheric sciences,oceanography,security,robotics,autonomous vehicles,among many others.展开更多
Computational imaging describes the whole imaging process from the perspective of light transport and information transmission, features traditional optical computing capabilities, and assists in breaking through the ...Computational imaging describes the whole imaging process from the perspective of light transport and information transmission, features traditional optical computing capabilities, and assists in breaking through the limitations of visual information recording. Progress in computational imaging promotes the development of diverse basic and applied disciplines. In this review, we provide an overview of the fundamental principles and methods in computational imaging, the history of this field, and the important roles that it plays in the development of science. We review the most recent and promising advances in computational imaging, from the perspective of different dimensions of visual signals, including spatial dimension, temporal dimension, angular dimension, spectral dimension, and phase. We also discuss some topics worth studying for future developments in computational imaging.展开更多
Two mainstream approaches for solving inverse sample reconstruction problems in programmable illumination computational microscopy rely on either deep models or physical models.Solutions based on physical models posse...Two mainstream approaches for solving inverse sample reconstruction problems in programmable illumination computational microscopy rely on either deep models or physical models.Solutions based on physical models possess strong generalization capabilities while struggling with global optimization of inverse problems due to a lack of sufficient physical constraints.In contrast,deep-learning methods have strong problem-solving abilities,but their generalization ability is often questioned because of the unclear physical principles.In addition,conventional deep models are difficult to apply to some specific scenes because of the difficulty in acquiring high-quality training data and their limited capacity to generalize across different scenarios.To combine the advantages of deep models and physical models together,we propose a hybrid framework consisting of three subneural networks(two deep-learning networks and one physics-based network).We first obtain a result with rich semantic information through a light deeplearning neural network and then use it as the initial value of the physical network to make its output comply with physical process constraints.These two results are then used as the input of a fusion deeplearning neural work that utilizes the paired features between the reconstruction results of two different models to further enhance imaging quality.The proposed hybrid framework integrates the advantages of both deep models and physical models and can quickly solve the computational reconstruction inverse problem in programmable illumination computational microscopy and achieve better results.We verified the feasibility and effectiveness of the proposed hybrid framework with theoretical analysis and actual experiments on resolution targets and biological samples.展开更多
Owing to the constraints on the fabrication ofγ-ray coding plates with many pixels,few studies have been carried out onγ-ray computational ghost imaging.Thus,the development of coding plates with fewer pixels is ess...Owing to the constraints on the fabrication ofγ-ray coding plates with many pixels,few studies have been carried out onγ-ray computational ghost imaging.Thus,the development of coding plates with fewer pixels is essential to achieveγ-ray computational ghost imaging.Based on the regional similarity between Hadamard subcoding plates,this study presents an optimization method to reduce the number of pixels of Hadamard coding plates.First,a moving distance matrix was obtained to describe the regional similarity quantitatively.Second,based on the matrix,we used two ant colony optimization arrangement algorithms to maximize the reuse of pixels in the regional similarity area and obtain new compressed coding plates.With full sampling,these two algorithms improved the pixel utilization of the coding plate,and the compression ratio values were 54.2%and 58.9%,respectively.In addition,three undersampled sequences(the Harr,Russian dolls,and cake-cutting sequences)with different sampling rates were tested and discussed.With different sampling rates,our method reduced the number of pixels of all three sequences,especially for the Russian dolls and cake-cutting sequences.Therefore,our method can reduce the number of pixels,manufacturing cost,and difficulty of the coding plate,which is beneficial for the implementation and application ofγ-ray computational ghost imaging.展开更多
Computational ghost imaging(CGI)provides an elegant framework for indirect imaging,but its application has been restricted by low imaging performance.Herein,we propose a novel approach that significantly improves the ...Computational ghost imaging(CGI)provides an elegant framework for indirect imaging,but its application has been restricted by low imaging performance.Herein,we propose a novel approach that significantly improves the imaging performance of CGI.In this scheme,we optimize the conventional CGI data processing algorithm by using a novel compressed sensing(CS)algorithm based on a deep convolution generative adversarial network(DCGAN).CS is used to process the data output by a conventional CGI device.The processed data are trained by a DCGAN to reconstruct the image.Qualitative and quantitative results show that this method significantly improves the quality of reconstructed images by jointly training a generator and the optimization process for reconstruction via meta-learning.Moreover,the background noise can be eliminated well by this method.展开更多
Spectral imaging is an important tool for a wide variety of applications. We present a technique for spectral imaging using computational imaging pattern based on compressive sensing （CS）. The spectral and spatial i...Spectral imaging is an important tool for a wide variety of applications. We present a technique for spectral imaging using computational imaging pattern based on compressive sensing （CS）. The spectral and spatial infor- mation is simultaneously obtained using a fiber spectrometer and the spatial light modulation without mechanical scanning. The method allows high-speed, stable, and sub sampling acquisition of spectral data from specimens. The relationship between sampling rate and image quality is discussed and two CS algorithms are compared.展开更多
Imaging through fluctuating scattering media such as fog is of challenge since it seriously degrades the image quality.We investigate how the image quality of computational ghost imaging is reduced by fluctuating fog ...Imaging through fluctuating scattering media such as fog is of challenge since it seriously degrades the image quality.We investigate how the image quality of computational ghost imaging is reduced by fluctuating fog and how to obtain a high-quality defogging ghost image. We show theoretically and experimentally that the photon number fluctuations introduced by fluctuating fog is the reason for ghost image degradation. An algorithm is proposed to process the signals collected by the computational ghost imaging device to eliminate photon number fluctuations of different measurement events. Thus, a high-quality defogging ghost image is reconstructed even though fog is evenly distributed on the optical path. A nearly 100% defogging ghost image is obtained by further using a cycle generative adversarial network to process the reconstructed defogging image.展开更多
Computational optical imaging is an interdisciplinary subject integrating optics, mathematics, and information technology. It introduces information processing into optical imaging and combines it with intelligent com...Computational optical imaging is an interdisciplinary subject integrating optics, mathematics, and information technology. It introduces information processing into optical imaging and combines it with intelligent computing, subverting the imaging mechanism of traditional optical imaging which only relies on orderly information transmission. To meet the high-precision requirements of traditional optical imaging for optical processing and adjustment, as well as to solve its problems of being sensitive to gravity and temperature in use, we establish an optical imaging system model from the perspective of computational optical imaging and studies how to design and solve the imaging consistency problem of optical system under the influence of gravity, thermal effect, stress, and other external environment to build a high robustness optical system. The results show that the high robustness interval of the optical system exists and can effectively reduce the sensitivity of the optical system to the disturbance of each link, thus realizing the high robustness of optical imaging.展开更多
In a single-pixel fast imaging setup,the data collected by the single-pixel detector needs to be processed by a computer,but the speed of the latter will affect the image reconstruction time.Here we propose two kinds ...In a single-pixel fast imaging setup,the data collected by the single-pixel detector needs to be processed by a computer,but the speed of the latter will affect the image reconstruction time.Here we propose two kinds of setups which are able to transform non-visible into visible light imaging,wherein their computing process is replaced by a camera integration mode.The image captured by the camera has a low contrast,so here we present an algorithm that can realize a high quality image in near-infrared to visible cross-waveband imaging.The scheme is verified both by simulation and in actual experiments.The setups demonstrate the great potential for single-pixel imaging and high-speed cross-waveband imaging for future practical applications.展开更多
An extreme ultraviolet solar corona multispectral imager can allow direct observation of high temperature coronal plasma,which is related to solar flares,coronal mass ejections and other significant coronal activities...An extreme ultraviolet solar corona multispectral imager can allow direct observation of high temperature coronal plasma,which is related to solar flares,coronal mass ejections and other significant coronal activities.This manuscript proposes a novel end-to-end computational design method for an extreme ultraviolet(EUV)solar corona multispectral imager operating at wavelengths near 100 nm,including a stray light suppression design and computational image recovery.To suppress the strong stray light from the solar disk,an outer opto-mechanical structure is designed to protect the imaging component of the system.Considering the low reflectivity(less than 70%)and strong-scattering(roughness)of existing extreme ultraviolet optical elements,the imaging component comprises only a primary mirror and a curved grating.A Lyot aperture is used to further suppress any residual stray light.Finally,a deep learning computational imaging method is used to correct the individual multi-wavelength images from the original recorded multi-slit data.In results and data,this can achieve a far-field angular resolution below 7",and spectral resolution below 0.05 nm.The field of view is±3 R_(☉)along the multi-slit moving direction,where R☉represents the radius of the solar disk.The ratio of the corona's stray light intensity to the solar center's irradiation intensity is less than 10-6 at the circle of 1.3 R_(☉).展开更多
Fourier ptychographic microscopy(FPM)is an innovative computational microscopy approach that enables high-throughput imaging with high resolution,wide field of view,and quantitative phase imaging(QPI)by simultaneously...Fourier ptychographic microscopy(FPM)is an innovative computational microscopy approach that enables high-throughput imaging with high resolution,wide field of view,and quantitative phase imaging(QPI)by simultaneously capturing bright-field and dark-field images.However,effectively utilizing dark-field intensity images,including both normally exposed and overexposed data,which contain valuable high-angle illumination information,remains a complex challenge.Successfully extracting and applying this information could significantly enhance phase reconstruction,benefiting processes such as virtual staining and QPI imaging.To address this,we introduce a multi-exposure image fusion(MEIF)framework that optimizes dark-field information by incorporating it into the FPM preprocessing workflow.MEIF increases the data available for reconstruction without requiring changes to the optical setup.We evaluate the framework using both feature-domain and traditional FPM,demonstrating that it achieves substantial improvements in intensity resolution and phase information for biological samples that exceed the performance of conventional high dynamic range(HDR)methods.This image preprocessing-based information-maximization strategy fully leverages existing datasets and offers promising potential to drive advancements in fields such as microscopy,remote sensing,and crystallography.展开更多
It has been over a decade since the first coded aperture video compressive sensing(CS)system was reported.The underlying principle of this technology is to employ a high-frequency modulator in the optical path to modu...It has been over a decade since the first coded aperture video compressive sensing(CS)system was reported.The underlying principle of this technology is to employ a high-frequency modulator in the optical path to modulate a recorded high-speed scene within one integration time.The superimposed image captured in this manner is modulated and compressed,since multiple modulation patterns are imposed.Following this,reconstruction algorithms are utilized to recover the desired high-speed scene.One leading advantage of video CS is that a single captured measurement can be used to reconstruct a multi-frame video,thereby enabling a low-speed camera to capture high-speed scenes.Inspired by this,a number of variants of video CS systems have been built,mainly using different modulation devices.Meanwhile,in order to obtain high-quality reconstruction videos,many algorithms have been developed,from optimization-based iterative algorithms to deep-learning-based ones.Recently,emerging deep learning methods have been dominant due to their high-speed inference and high-quality reconstruction,highlighting the possibility of deploying video CS in practical applications.Toward this end,this paper reviews the progress that has been achieved in video CS during the past decade.We further analyze the efforts that need to be made—in terms of both hardware and algorithms—to enable real applications.Research gaps are put forward and future directions are summarized to help researchers and engineers working on this topic.展开更多
Fourier Ptychographic Microscopy(FPM)is a high-throughput computational optical imaging technology reported in 2013.It effectively breaks through the trade-off between high-resolution imaging and wide-field imaging.In...Fourier Ptychographic Microscopy(FPM)is a high-throughput computational optical imaging technology reported in 2013.It effectively breaks through the trade-off between high-resolution imaging and wide-field imaging.In recent years,it has been found that FPM is not only a tool to break through the trade-off between field of view and spatial resolution,but also a paradigm to break through those trade-off problems,thus attracting extensive attention.Compared with previous reviews,this review does not introduce its concept,basic principles,optical system and series of applications once again,but focuses on elaborating the three major difficulties faced by FPM technology in the process from“looking good”in the laboratory to“working well”in practical applications:mismatch between numerical model and physical reality,long reconstruction time and high computing power demand,and lack of multi-modal expansion.It introduces how to achieve key technological innovations in FPM through the dual drive of Artificial Intelligence(AI)and physics,including intelligent reconstruction algorithms introducing machine learning concepts,optical-algorithm co-design,fusion of frequency domain extrapolation methods and generative adversarial networks,multi-modal imaging schemes and data fusion enhancement,etc.,gradually solving the difficulties of FPM technology.Conversely,this review deeply considers the unique value of FPM technology in potentially feeding back to the development of“AI+optics”,such as providing AI benchmark tests under physical constraints,inspirations for the balance of computing power and bandwidth in miniaturized intelligent microscopes,and photoelectric hybrid architectures.Finally,it introduces the industrialization path and frontier directions of FPM technology,pointing out that with the promotion of the dual drive of AI and physics,it will generate a large number of industrial application case,and looks forward to the possibilities of future application scenarios and expansions,for instance,body fluid biopsy and point-of-care testing at the grassroots level represent the expansion of the growth market.展开更多
Return signal processing and reconstruction plays a pivotal role in coherent field imaging, having a significant in- fluence on the quality of the reconstructed image. To reduce the required samples and accelerate the...Return signal processing and reconstruction plays a pivotal role in coherent field imaging, having a significant in- fluence on the quality of the reconstructed image. To reduce the required samples and accelerate the sampling process, we propose a genuine sparse reconstruction scheme based on compressed sensing theory. By analyzing the sparsity of the received signal in the Fourier spectrum domain, we accomplish an effective random projection and then reconstruct the return signal from as little as 10% of traditional samples, finally acquiring the target image precisely. The results of the numerical simulations and practical experiments verify the correctness of the proposed method, providing an efficient processing approach for imaging fast-moving targets in the future.展开更多
Computational biomedical imaging lies at the intersection of physics,computer science,and biomedicine,aiming to produce visual representations of biological or physiological phenomena that may be otherwise imperceptib...Computational biomedical imaging lies at the intersection of physics,computer science,and biomedicine,aiming to produce visual representations of biological or physiological phenomena that may be otherwise imperceptible to measuring instruments.Over the last few decades,breakthroughs in imaging physics-as evidenced by modalities like magnetic resonance imaging(MRI),computed tomography(CT),ultrasound,optical microscopy,and endoscopy-have profoundly impacted the way clinicians visualize and understand living systems.展开更多
基金supported by the National Key R&D Program of China(Grant No.2021YFB3900300)National Natural Science Foundation of China(Grant Nos.61860206007,62275177,and 62371321)+4 种基金Ministry of Education Science and Technology Chunhui Project(Grant No.HZKY20220559)International S and T Cooperation Program of Sichuan Province(Grant No.2023YFH0030)Sichuan Science and Technology Innovation Seeding Project(Grant No.23-YCG034)Sichuan Science and Technology Program(Grant No.2023YFG0334)Chengdu Science and Technology Program(Grant No.2022-GH02-00001-HZ).
文摘Single-pixel imaging(SPI)enables an invisible target to be imaged onto a photosensitive surface without a lens,emerging as a promising way for indirect optical encryption.However,due to its linear and broadcast imaging principles,SPI encryption has been confined to a single-user framework for the long term.We propose a multi-image SPI encryption method and combine it with orthogonal frequency division multiplexing-assisted key management,to achieve a multiuser SPI encryption and authentication framework.Multiple images are first encrypted as a composite intensity sequence containing the plaintexts and authentication information,simultaneously generating different sets of keys for users.Then,the SPI keys for encryption and authentication are asymmetrically isolated into independent frequency carriers and encapsulated into a Malus metasurface,so as to establish an individually private and content-independent channel for each user.Users can receive different plaintexts privately and verify the authenticity,eliminating the broadcast transparency of SPI encryption.The improved linear security is also verified by simulating attacks.By the combination of direct key management and indirect image encryption,our work achieves the encryption and authentication functionality under a multiuser computational imaging framework,facilitating its application in optical communication,imaging,and security.
基金supported by the National Natural Science Foundation of China(Nos.81671726 and 81627805)the Hong Kong Research Grant Council(No.25204416)+1 种基金the Shenzhen Science and Technology Innovation Commission(No.JCYJ20170818104421564)the Hong Kong Innovation and Technology Commission(No.ITS/022/18).
文摘Coherent optical control within or through scattering media via wavefront shaping has seen broad applications since its invention around 2007.Wavefront shaping is aimed at overcoming the strong scattering,featured by random interference,namely speckle patterns.This randomness occurs due to the refractive index inhomogeneity in complex media like biological tissue or the modal dispersion in multimode fiber,yet this randomness is actually deterministic and potentially can be time reversal or precompensated.Various wavefront shaping approaches,such as optical phase conjugation,iterative optimization,and transmission matrix measurement,have been developed to generate tight and intense optical delivery or high-resolution image of an optical object behind or within a scattering medium.The performance of these modula-tions,however,is far from satisfaction.Most recently,artifcial intelligence has brought new inspirations to this field,providing exciting hopes to tackle the challenges by mapping the input and output optical patterns and building a neuron network that inherently links them.In this paper,we survey the developments to date on this topic and briefly discuss our views on how to harness machine learning(deep learning in particular)for further advancements in the field.
基金supported by the National Natural Science Foundation of China(Nos.62405231,62205259,62075175,62105254,and 62375212)the National Key Laboratory of Infrared Detection Technologies(No.IRDT-23-06)+1 种基金the Fundamental Research Funds for the Central Universities(Nos.XJSJ24028 and XJS222202)the Open Research Fund of Beijing Key Laboratory of Advanced Optical Remote Sensing Technology(No.AORS202405).
文摘The current state of traditional optoelectronic imaging technology is constrained by the inherent limitations of its hardware.These limitations pose significant challenges in acquiring higher-dimensional information and reconstructing accurate images,particularly in applications such as scattering imaging,superresolution,and complex scene reconstruction.However,the rapid development and widespread adoption of deep learning are reshaping the field of optical imaging through computational imaging technology.Datadriven computational imaging has ushered in a paradigm shift by leveraging the nonlinear expression and feature learning capabilities of neural networks.This approach transcends the limitations of conventional physical models,enabling the adaptive extraction of critical features directly from data.As a result,computational imaging overcomes the traditional“what you see is what you get”paradigm,paving the way for more compact optical system designs,broader information acquisition,and improved image reconstruction accuracy.These advancements have significantly enhanced the interpretation of highdimensional light-field information and the processing of complex images.This review presents a comprehensive analysis of the integration of deep learning and computational imaging,emphasizing its transformative potential in three core areas:computational optical system design,high-dimensional information interpretation,and image enhancement and processing.Additionally,this review addresses the challenges and future directions of this cutting-edge technology,providing novel insights into interdisciplinary imaging research.
基金supported by the National Key Research and Development Program of China(Grant Nos.2022YFA1205000 and 2022YFA1207200)the National Natural Science Foundation of China(Grant Nos.12274217,61971465,and 12104225)the Fundamental Research Funds for the Central Universities,China(Grant No.0210-14380184)
文摘Metasurface-based imaging has attracted considerable attention owing to its compactness,multifunctionality,and subwavelength coding capability.With the integration of computational imaging techniques,researchers have actively explored the extended capabilities of metasurfaces,enabling a wide range of imaging methods.We present an overview of the recent progress in metasurface-based imaging techniques,focusing on the perspective of computational imaging.Specifically,we categorize and review existing metasurface-based imaging into three main groups,including(i)conventional metasurface design employing canonical methods,(ii)computation introduced independently in either the imaging process or postprocessing,and(iii)an end-to-end computation-optimized imaging system based upon metasurfaces.We highlight the advantages and challenges associated with each computational metasurface-based imaging technique and discuss the potential and future prospects of the computational boosted metaimager.
文摘BACKGROUND The diagnostic accuracy for detecting metastatic lymph nodes in colorectal cancer(CRC)remains suboptimal.To address this limitation,our study investigates the potential of gemstone spectral computed tomography imaging(GSI)to improve diagnostic accuracy in lymph node metastasis(LNM)assessment.AIM To extensively investigate the clinical utility of GSI in the preoperative assessment of CRC.METHODS The subject population included 200 patients with CRC who were admitted to Zibo Central Hospital from January 2022 to December 2023.All patients underwent dual-phase contrast-enhanced scans in the arterial and venous phases using GSI before surgical intervention.During the research,meticulous quantification was conducted regarding the number of patients with CRC with LNM as well as the exact count of metastatic lymph nodes.Moreover,for both metastatic and non-metastatic lymph nodes,the short diameter at the maximum crosssectional area(covering the axial,sagittal,and coronal planes),morphological features(including manifestations such as margin blurring,aggregation,and enhancement),and spectral parameters in the arterial and venous phases[specifically iodine concentration(IC),normalized IC(NIC),and the slope of the spectral curve(λHU)]were measured and recorded,and a comparative analysis was conducted.The diagnostic efficacy of each index with differences was systematically assessed using the receiver operating characteristic(ROC)curve.Concurrently,receiver operating characteristic curves were constructed for LNM screening based on the short diameter at the maximum cross-sectional area of lymph nodes and each spectral parameter in the arterial and venous phases.RESULTS The area under the curve of GSI for diagnosing LNM in patients with CRC can reach 0.897,with sensitivity,specificity,and accuracy of 92.59%,85.87%,and 89.50%,respectively.A total of 265 lymph nodes were analyzed from the 200 participants with CRC,with metastatic lymph nodes accounting for 56.60%.Compared with nonmetastatic lymph nodes,the short diameters of metastatic lymph nodes in the axial,sagittal,and coronal planes were significantly increased,whereas the IC values in the arterial and venous phases,the NIC value in the arterial phase,and theλHU values in the arterial and venous phases were significantly decreased.The short axial,sagittal,and coronal diameters,arterial-phase IC,venous-phase IC,arterial-phase NIC,arterial-phaseλHU,and venousphaseλHU for diagnosing metastatic lymph nodes demonstrated area under the curve values of 0.631,0.681,0.659,0.862,0.808,0.831,0.801,and 0.706,respectively.CONCLUSION GSI exhibits substantial clinical significance in the preoperative assessment of CRC.Among the parameters assessed,the arterial-phase IC demonstrates the most outstanding diagnostic performance,effectively improving the diagnostic efficacy for preoperative LNM in CRC.
基金The authors acknowledge the U.S.National Science Foundation and Fujikura.
文摘Imaging through diffusers presents a challenging problem with various digital image reconstruction solutions demonstrated to date using computers.Here,we present a computer-free,all-optical image reconstruction method to see through random diffusers at the speed of light.Using deep learning,a set of transmissive diffractive surfaces are trained to all-optically reconstruct images of arbitrary objects that are completely covered by unknown,random phase diffusers.After the training stage,which is a one-time effort,the resulting diffractive surfaces are fabricated and form a passive optical network that is physically positioned between the unknown object and the image plane to all-optically reconstruct the object pattern through an unknown,new phase diffuser.We experimentally demonstrated this concept using coherent THz illumination and all-optically reconstructed objects distorted by unknown,random diffusers,never used during training.Unlike digital methods,all-optical diffractive reconstructions do not require power except for the illumination light.This diffractive solution to see through diffusers can be extended to other wavelengths,and might fuel various applications in biomedical imaging,astronomy,atmospheric sciences,oceanography,security,robotics,autonomous vehicles,among many others.
基金Project supported by the National Natural Science Foundation of China （Nos. 61327902 and 61631009）
文摘Computational imaging describes the whole imaging process from the perspective of light transport and information transmission, features traditional optical computing capabilities, and assists in breaking through the limitations of visual information recording. Progress in computational imaging promotes the development of diverse basic and applied disciplines. In this review, we provide an overview of the fundamental principles and methods in computational imaging, the history of this field, and the important roles that it plays in the development of science. We review the most recent and promising advances in computational imaging, from the perspective of different dimensions of visual signals, including spatial dimension, temporal dimension, angular dimension, spectral dimension, and phase. We also discuss some topics worth studying for future developments in computational imaging.
基金supported by the National Natural Science Foundation of China(Grant No.62275020).
文摘Two mainstream approaches for solving inverse sample reconstruction problems in programmable illumination computational microscopy rely on either deep models or physical models.Solutions based on physical models possess strong generalization capabilities while struggling with global optimization of inverse problems due to a lack of sufficient physical constraints.In contrast,deep-learning methods have strong problem-solving abilities,but their generalization ability is often questioned because of the unclear physical principles.In addition,conventional deep models are difficult to apply to some specific scenes because of the difficulty in acquiring high-quality training data and their limited capacity to generalize across different scenarios.To combine the advantages of deep models and physical models together,we propose a hybrid framework consisting of three subneural networks(two deep-learning networks and one physics-based network).We first obtain a result with rich semantic information through a light deeplearning neural network and then use it as the initial value of the physical network to make its output comply with physical process constraints.These two results are then used as the input of a fusion deeplearning neural work that utilizes the paired features between the reconstruction results of two different models to further enhance imaging quality.The proposed hybrid framework integrates the advantages of both deep models and physical models and can quickly solve the computational reconstruction inverse problem in programmable illumination computational microscopy and achieve better results.We verified the feasibility and effectiveness of the proposed hybrid framework with theoretical analysis and actual experiments on resolution targets and biological samples.
基金supported by the Youth Science Foundation of Sichuan Province(Nos.22NSFSC3816 and 2022NSFSC1231)the General Project of the National Natural Science Foundation of China(Nos.12075039 and 41874121)the Key Project of the National Natural Science Foundation of China(No.U19A2086).
文摘Owing to the constraints on the fabrication ofγ-ray coding plates with many pixels,few studies have been carried out onγ-ray computational ghost imaging.Thus,the development of coding plates with fewer pixels is essential to achieveγ-ray computational ghost imaging.Based on the regional similarity between Hadamard subcoding plates,this study presents an optimization method to reduce the number of pixels of Hadamard coding plates.First,a moving distance matrix was obtained to describe the regional similarity quantitatively.Second,based on the matrix,we used two ant colony optimization arrangement algorithms to maximize the reuse of pixels in the regional similarity area and obtain new compressed coding plates.With full sampling,these two algorithms improved the pixel utilization of the coding plate,and the compression ratio values were 54.2%and 58.9%,respectively.In addition,three undersampled sequences(the Harr,Russian dolls,and cake-cutting sequences)with different sampling rates were tested and discussed.With different sampling rates,our method reduced the number of pixels of all three sequences,especially for the Russian dolls and cake-cutting sequences.Therefore,our method can reduce the number of pixels,manufacturing cost,and difficulty of the coding plate,which is beneficial for the implementation and application ofγ-ray computational ghost imaging.
基金Project supported by the National Natural Science Foundation of China(Grant Nos.11704221,11574178,and 61675115)the Taishan Scholar Project of Shandong Province,China(Grant No.tsqn201812059)。
文摘Computational ghost imaging(CGI)provides an elegant framework for indirect imaging,but its application has been restricted by low imaging performance.Herein,we propose a novel approach that significantly improves the imaging performance of CGI.In this scheme,we optimize the conventional CGI data processing algorithm by using a novel compressed sensing(CS)algorithm based on a deep convolution generative adversarial network(DCGAN).CS is used to process the data output by a conventional CGI device.The processed data are trained by a DCGAN to reconstruct the image.Qualitative and quantitative results show that this method significantly improves the quality of reconstructed images by jointly training a generator and the optimization process for reconstruction via meta-learning.Moreover,the background noise can be eliminated well by this method.
基金Supported by the National Major Scientific Instruments Development Project of China under Grant No 2013YQ030595the National Natural Science Foundation of China under Grant Nos 11675014,61601442,61605218,61474123 and 61575207+2 种基金the Science and Technology Innovation Foundation of Chinese Academy of Sciences under Grant No CXJJ-16S047,the National Defense Science and Technology Innovation Foundation of Chinese Academy of Sciencesthe Program of International S&T Cooperation under Grant No 2016YFE0131500the Advance Research Project under Grant No 30102070101
文摘Spectral imaging is an important tool for a wide variety of applications. We present a technique for spectral imaging using computational imaging pattern based on compressive sensing （CS）. The spectral and spatial infor- mation is simultaneously obtained using a fiber spectrometer and the spatial light modulation without mechanical scanning. The method allows high-speed, stable, and sub sampling acquisition of spectral data from specimens. The relationship between sampling rate and image quality is discussed and two CS algorithms are compared.
基金supported by the Natural Science Foundation of Shandong Province, China (Grant No. ZR2022MF249)。
文摘Imaging through fluctuating scattering media such as fog is of challenge since it seriously degrades the image quality.We investigate how the image quality of computational ghost imaging is reduced by fluctuating fog and how to obtain a high-quality defogging ghost image. We show theoretically and experimentally that the photon number fluctuations introduced by fluctuating fog is the reason for ghost image degradation. An algorithm is proposed to process the signals collected by the computational ghost imaging device to eliminate photon number fluctuations of different measurement events. Thus, a high-quality defogging ghost image is reconstructed even though fog is evenly distributed on the optical path. A nearly 100% defogging ghost image is obtained by further using a cycle generative adversarial network to process the reconstructed defogging image.
文摘Computational optical imaging is an interdisciplinary subject integrating optics, mathematics, and information technology. It introduces information processing into optical imaging and combines it with intelligent computing, subverting the imaging mechanism of traditional optical imaging which only relies on orderly information transmission. To meet the high-precision requirements of traditional optical imaging for optical processing and adjustment, as well as to solve its problems of being sensitive to gravity and temperature in use, we establish an optical imaging system model from the perspective of computational optical imaging and studies how to design and solve the imaging consistency problem of optical system under the influence of gravity, thermal effect, stress, and other external environment to build a high robustness optical system. The results show that the high robustness interval of the optical system exists and can effectively reduce the sensitivity of the optical system to the disturbance of each link, thus realizing the high robustness of optical imaging.
文摘In a single-pixel fast imaging setup,the data collected by the single-pixel detector needs to be processed by a computer,but the speed of the latter will affect the image reconstruction time.Here we propose two kinds of setups which are able to transform non-visible into visible light imaging,wherein their computing process is replaced by a camera integration mode.The image captured by the camera has a low contrast,so here we present an algorithm that can realize a high quality image in near-infrared to visible cross-waveband imaging.The scheme is verified both by simulation and in actual experiments.The setups demonstrate the great potential for single-pixel imaging and high-speed cross-waveband imaging for future practical applications.
基金This study is partially supported by the National Natural Science Foundation of China(NSFC)(62005120,62125504).
文摘An extreme ultraviolet solar corona multispectral imager can allow direct observation of high temperature coronal plasma,which is related to solar flares,coronal mass ejections and other significant coronal activities.This manuscript proposes a novel end-to-end computational design method for an extreme ultraviolet(EUV)solar corona multispectral imager operating at wavelengths near 100 nm,including a stray light suppression design and computational image recovery.To suppress the strong stray light from the solar disk,an outer opto-mechanical structure is designed to protect the imaging component of the system.Considering the low reflectivity(less than 70%)and strong-scattering(roughness)of existing extreme ultraviolet optical elements,the imaging component comprises only a primary mirror and a curved grating.A Lyot aperture is used to further suppress any residual stray light.Finally,a deep learning computational imaging method is used to correct the individual multi-wavelength images from the original recorded multi-slit data.In results and data,this can achieve a far-field angular resolution below 7",and spectral resolution below 0.05 nm.The field of view is±3 R_(☉)along the multi-slit moving direction,where R☉represents the radius of the solar disk.The ratio of the corona's stray light intensity to the solar center's irradiation intensity is less than 10-6 at the circle of 1.3 R_(☉).
基金supported by the National Natural Science Foundation of China(Grant No.12104500)the Key Research and Development Projects of Shaanxi Province of China(Grant No.2023-YBSF-263).
文摘Fourier ptychographic microscopy(FPM)is an innovative computational microscopy approach that enables high-throughput imaging with high resolution,wide field of view,and quantitative phase imaging(QPI)by simultaneously capturing bright-field and dark-field images.However,effectively utilizing dark-field intensity images,including both normally exposed and overexposed data,which contain valuable high-angle illumination information,remains a complex challenge.Successfully extracting and applying this information could significantly enhance phase reconstruction,benefiting processes such as virtual staining and QPI imaging.To address this,we introduce a multi-exposure image fusion(MEIF)framework that optimizes dark-field information by incorporating it into the FPM preprocessing workflow.MEIF increases the data available for reconstruction without requiring changes to the optical setup.We evaluate the framework using both feature-domain and traditional FPM,demonstrating that it achieves substantial improvements in intensity resolution and phase information for biological samples that exceed the performance of conventional high dynamic range(HDR)methods.This image preprocessing-based information-maximization strategy fully leverages existing datasets and offers promising potential to drive advancements in fields such as microscopy,remote sensing,and crystallography.
基金supported by the National Natural Science Foundation of China(61931012,62171258,62088102,and 62271414)the Zhejiang Provincial Outstanding Youth Science Foundation(LR23F010001)the Key Project of Westlake Institute for Optoelectronics(2023GD007).
文摘It has been over a decade since the first coded aperture video compressive sensing(CS)system was reported.The underlying principle of this technology is to employ a high-frequency modulator in the optical path to modulate a recorded high-speed scene within one integration time.The superimposed image captured in this manner is modulated and compressed,since multiple modulation patterns are imposed.Following this,reconstruction algorithms are utilized to recover the desired high-speed scene.One leading advantage of video CS is that a single captured measurement can be used to reconstruct a multi-frame video,thereby enabling a low-speed camera to capture high-speed scenes.Inspired by this,a number of variants of video CS systems have been built,mainly using different modulation devices.Meanwhile,in order to obtain high-quality reconstruction videos,many algorithms have been developed,from optimization-based iterative algorithms to deep-learning-based ones.Recently,emerging deep learning methods have been dominant due to their high-speed inference and high-quality reconstruction,highlighting the possibility of deploying video CS in practical applications.Toward this end,this paper reviews the progress that has been achieved in video CS during the past decade.We further analyze the efforts that need to be made—in terms of both hardware and algorithms—to enable real applications.Research gaps are put forward and future directions are summarized to help researchers and engineers working on this topic.
基金National Natural Science Foundation of China(No.12574332)the Space Optoelectronic Measurement and Perception Lab.,Beijing Institute of Control Engineering(No.LabSOMP-2023-10)Major Science and Technology Innovation Program of Xianyang City(No.L2024-ZDKJ-ZDCGZH-0021)。
文摘Fourier Ptychographic Microscopy(FPM)is a high-throughput computational optical imaging technology reported in 2013.It effectively breaks through the trade-off between high-resolution imaging and wide-field imaging.In recent years,it has been found that FPM is not only a tool to break through the trade-off between field of view and spatial resolution,but also a paradigm to break through those trade-off problems,thus attracting extensive attention.Compared with previous reviews,this review does not introduce its concept,basic principles,optical system and series of applications once again,but focuses on elaborating the three major difficulties faced by FPM technology in the process from“looking good”in the laboratory to“working well”in practical applications:mismatch between numerical model and physical reality,long reconstruction time and high computing power demand,and lack of multi-modal expansion.It introduces how to achieve key technological innovations in FPM through the dual drive of Artificial Intelligence(AI)and physics,including intelligent reconstruction algorithms introducing machine learning concepts,optical-algorithm co-design,fusion of frequency domain extrapolation methods and generative adversarial networks,multi-modal imaging schemes and data fusion enhancement,etc.,gradually solving the difficulties of FPM technology.Conversely,this review deeply considers the unique value of FPM technology in potentially feeding back to the development of“AI+optics”,such as providing AI benchmark tests under physical constraints,inspirations for the balance of computing power and bandwidth in miniaturized intelligent microscopes,and photoelectric hybrid architectures.Finally,it introduces the industrialization path and frontier directions of FPM technology,pointing out that with the promotion of the dual drive of AI and physics,it will generate a large number of industrial application case,and looks forward to the possibilities of future application scenarios and expansions,for instance,body fluid biopsy and point-of-care testing at the grassroots level represent the expansion of the growth market.
基金supported by the National Natural Science Foundation of China(Grant No.61505248)the Fund from Chinese Academy of Sciences,the Light of"Western"Talent Cultivation Plan"Dr.Western Fund Project"(Grant No.Y429621213)
文摘Return signal processing and reconstruction plays a pivotal role in coherent field imaging, having a significant in- fluence on the quality of the reconstructed image. To reduce the required samples and accelerate the sampling process, we propose a genuine sparse reconstruction scheme based on compressed sensing theory. By analyzing the sparsity of the received signal in the Fourier spectrum domain, we accomplish an effective random projection and then reconstruct the return signal from as little as 10% of traditional samples, finally acquiring the target image precisely. The results of the numerical simulations and practical experiments verify the correctness of the proposed method, providing an efficient processing approach for imaging fast-moving targets in the future.
基金supported by the National Key Research and Development Program of China(2022YFC3401100,2024YFC3406402,and 2024YFF0507400)the National Natural Science Foundation of China(62371007 and 6220071694)the Beijing Natural Science Foundation(Z240010).
文摘Computational biomedical imaging lies at the intersection of physics,computer science,and biomedicine,aiming to produce visual representations of biological or physiological phenomena that may be otherwise imperceptible to measuring instruments.Over the last few decades,breakthroughs in imaging physics-as evidenced by modalities like magnetic resonance imaging(MRI),computed tomography(CT),ultrasound,optical microscopy,and endoscopy-have profoundly impacted the way clinicians visualize and understand living systems.
