Manufacturing-robust imaging systems leveraging computational optics hold immense potential for easing manufacturing constraints and enabling the development of cost-effective,high-quality imaging solutions.However,co...Manufacturing-robust imaging systems leveraging computational optics hold immense potential for easing manufacturing constraints and enabling the development of cost-effective,high-quality imaging solutions.However,conventional approaches,which typically rely on data-driven neural networks to correct optical aberrations caused by manufacturing errors,are constrained by the lack of effective tolerance analysis methods for quantitatively evaluating manufacturing error boundaries.This limitation is crucial for further relaxing manufacturing constraints and providing practical guidance for fabrication.We propose a physics-informed design paradigm for manufacturing-robust imaging systems with computational optics,integrating a physics-informed tolerance analysis methodology for evaluating manufacturing error boundaries and a physics-informed neural network for image reconstruction.With this approach,we achieve a manufacturing-robust imaging system based on an off-axis three-mirror freeform all-aluminum design,delivering a modulation transfer function exceeding 0.34 at the Nyquist frequency(72 lp/mm)in simulation.Notably,this system requires a manufacturing precision of only 0.5λin root mean square(RMS),representing a remarkable 25-fold relaxation compared with the conventional requirement of 0.02λin RMS.Experimental validation further confirmed that the manufacturing-robust imaging system maintains excellent performance in diverse indoor and outdoor environments.Our proposed method paves the way for achieving high-quality imaging without the necessity of high manufacturing precision,enabling practical solutions that are more cost-effective and time-efficient.展开更多
Computational optics introduces computation into optics and consequently helps overcome traditional optical limitations such as low sensing dimension,low light throughput,low resolution,and so on.The combination of op...Computational optics introduces computation into optics and consequently helps overcome traditional optical limitations such as low sensing dimension,low light throughput,low resolution,and so on.The combination of optical encoding and computational decoding offers enhanced imaging and sensing capabilities with diverse applications in biomedicine,astronomy,agriculture,etc.With the great advance of artificial intelligence in the last decade,deep learning has further boosted computational optics with higher precision and efficiency.Recently,there developed an end-to-end joint optimization technique that digitally twins optical encoding to neural network layers,and then facilitates simultaneous optimization with the decoding process.This framework offers effective performance enhancement over conventional techniques.However,the reverse physical twinning from optimized encoding parameters to practical modulation elements faces a serious challenge,due to the discrepant gap in such as bit depth,numerical range,and stability.In this regard,this review explores various optical modulation elements across spatial,phase,and spectral dimensions in the digital twin model for joint encoding-decoding optimization.Our analysis offers constructive guidance for finding the most appropriate modulation element in diverse imaging and sensing tasks concerning various requirements of precision,speed,and robustness.The review may help tackle the above twinning challenge and pave the way for next-generation computational optics.展开更多
Fourier Ptychographic Microscopy(FPM)is a high-throughput computational optical imaging technology reported in 2013.It effectively breaks through the trade-off between high-resolution imaging and wide-field imaging.In...Fourier Ptychographic Microscopy(FPM)is a high-throughput computational optical imaging technology reported in 2013.It effectively breaks through the trade-off between high-resolution imaging and wide-field imaging.In recent years,it has been found that FPM is not only a tool to break through the trade-off between field of view and spatial resolution,but also a paradigm to break through those trade-off problems,thus attracting extensive attention.Compared with previous reviews,this review does not introduce its concept,basic principles,optical system and series of applications once again,but focuses on elaborating the three major difficulties faced by FPM technology in the process from“looking good”in the laboratory to“working well”in practical applications:mismatch between numerical model and physical reality,long reconstruction time and high computing power demand,and lack of multi-modal expansion.It introduces how to achieve key technological innovations in FPM through the dual drive of Artificial Intelligence(AI)and physics,including intelligent reconstruction algorithms introducing machine learning concepts,optical-algorithm co-design,fusion of frequency domain extrapolation methods and generative adversarial networks,multi-modal imaging schemes and data fusion enhancement,etc.,gradually solving the difficulties of FPM technology.Conversely,this review deeply considers the unique value of FPM technology in potentially feeding back to the development of“AI+optics”,such as providing AI benchmark tests under physical constraints,inspirations for the balance of computing power and bandwidth in miniaturized intelligent microscopes,and photoelectric hybrid architectures.Finally,it introduces the industrialization path and frontier directions of FPM technology,pointing out that with the promotion of the dual drive of AI and physics,it will generate a large number of industrial application case,and looks forward to the possibilities of future application scenarios and expansions,for instance,body fluid biopsy and point-of-care testing at the grassroots level represent the expansion of the growth market.展开更多
Computational optical imaging is an interdisciplinary subject integrating optics, mathematics, and information technology. It introduces information processing into optical imaging and combines it with intelligent com...Computational optical imaging is an interdisciplinary subject integrating optics, mathematics, and information technology. It introduces information processing into optical imaging and combines it with intelligent computing, subverting the imaging mechanism of traditional optical imaging which only relies on orderly information transmission. To meet the high-precision requirements of traditional optical imaging for optical processing and adjustment, as well as to solve its problems of being sensitive to gravity and temperature in use, we establish an optical imaging system model from the perspective of computational optical imaging and studies how to design and solve the imaging consistency problem of optical system under the influence of gravity, thermal effect, stress, and other external environment to build a high robustness optical system. The results show that the high robustness interval of the optical system exists and can effectively reduce the sensitivity of the optical system to the disturbance of each link, thus realizing the high robustness of optical imaging.展开更多
Optical and hybrid convolutional neural networks(CNNs)recently have become of increasing interest to achieve low-latency,low-power image classification,and computer-vision tasks.However,implementing optical nonlineari...Optical and hybrid convolutional neural networks(CNNs)recently have become of increasing interest to achieve low-latency,low-power image classification,and computer-vision tasks.However,implementing optical nonlinearity is challenging,and omitting the nonlinear layers in a standard CNN comes with a significant reduction in accuracy.We use knowledge distillation to compress modified AlexNet to a single linear convolutional layer and an electronic backend(two fully connected layers).We obtain comparable performance with a purely electronic CNN with five convolutional layers and three fully connected layers.We implement the convolution optically via engineering the point spread function of an inverse-designed meta-optic.Using this hybrid approach,we estimate a reduction in multiply-accumulate operations from 17M in a conventional electronic modified AlexNet to only 86 K in the hybrid compressed network enabled by the optical front end.This constitutes over 2 orders of magnitude of reduction in latency and power consumption.Furthermore,we experimentally demonstrate that the classification accuracy of the system exceeds 93%on the MNIST dataset of handwritten digits.展开更多
Photonic hardware implementation of spiking neural networks,regarded as a viable potential paradigm for ultra-high speed and energy efficiency computing,leverages spatiotemporal spike encoding and event-driven dynamic...Photonic hardware implementation of spiking neural networks,regarded as a viable potential paradigm for ultra-high speed and energy efficiency computing,leverages spatiotemporal spike encoding and event-driven dynamics to simulate brain-like parallel information processing.Silicon-based microring resonators(MRRs)offer a power efficiency and ultrahigh flexibility scheme to mimic biological neuron,however,their substantial potential for integrated neuromorphic systems remains limited by insufficient exploration of MRR-based spiking digital and analog computation.Here,an all-optical neural dynamics framework,encompassing both excitatory and inhibitory behaviors based on multi-wavelength auxiliary and competition mechanism in an MRR,is proposed numerically.Leveraging multi-wavelength resonance characteristics and wavelength division multiplexing(WDM)technology,a single MRR implements the five fundamental optical digital logic gates:AND,OR,NOT,XNOR and XOR.Besides,the cascading capabilities of MRR-based spiking neurons are demonstrated through multi-level digital logic gates including NAND,NOR,4-input AND,8-input AND,and a full adder,emphasizing their promise for large-scale digital logic networks.Furthermore,an exemplary binary convolution has been achieved by utilizing the proposed MRR-based digital logic operation,illustrating the potential of all-optical binary convolution to compute image gradient magnitudes for edge detection.Such passive photonic neurons and networks promise access to the high transmission speed and low power consumption inherent to optical systems,thus enabling direct hardware-algorithm co-computation and accelerating artificial intelligence.展开更多
Diffractive optical neural networks(DONNs)have exhibited the advantages of parallelization,high speed,and low consumption.However,the existing DONNs based on free-space diffractive optical elements are bulky and unste...Diffractive optical neural networks(DONNs)have exhibited the advantages of parallelization,high speed,and low consumption.However,the existing DONNs based on free-space diffractive optical elements are bulky and unsteady.In this study,we propose a planar-waveguide integrated diffractive neural network chip architecture.The three diffractive layers are engraved on the same side of a quartz wafer.The three-layer chip is designed with 32-mm3 processing space and enables a computing speed of 3.1×109 Tera operations per second.The results show that the proposed chip achieves 73.4%experimental accuracy for the Modified National Institute of Standards and Technology database while showing the system’s robustness in a cycle test.The consistency of experiments is 88.6%,and the arithmetic mean standard deviation of the results is~4.7%.The proposed chip architecture can potentially revolutionize high-resolution optical processing tasks with high robustness.展开更多
With the advancement of artificial intelligence,optic in-sensing reservoir computing based on emerging semiconductor devices is high desirable for real-time analog signal processing.Here,we disclose a flexible optomem...With the advancement of artificial intelligence,optic in-sensing reservoir computing based on emerging semiconductor devices is high desirable for real-time analog signal processing.Here,we disclose a flexible optomemristor based on C_(27)H_(30)O_(15)/FeOx heterostructure that presents a highly sensitive to the light stimuli and artificial optic synaptic features such as short-and long-term plasticity(STP and LTP),enabling the developed optomemristor to implement complex analogy signal processing through building a real-physical dynamic-based in-sensing reservoir computing algorithm and yielding an accuracy of 94.88%for speech recognition.The charge trapping and detrapping mediated by the optic active layer of C_(27)H_(30)O_(15) that is extracted from the lotus flower is response for the positive photoconductance memory in the prepared optomemristor.This work provides a feasible organic−inorganic heterostructure as well as an optic in-sensing vision computing for an advanced optic computing system in future complex signal processing.展开更多
Ising problems are critical for a wide range of applications.Solving these problems on a photonic platform takes advantage of the unique properties of photons,such as high speed,low power consumption,and large bandwid...Ising problems are critical for a wide range of applications.Solving these problems on a photonic platform takes advantage of the unique properties of photons,such as high speed,low power consumption,and large bandwidth.Recently,there has been growing interest in using photonic platforms to accelerate the optimization of Ising models,paving the way for the development of ultrafast hardware in machine learning.However,these proposed systems face challenges in simultaneously achieving high spin scalability,encoding flexibility,and low system complexity.We propose a wavelength-domain optical Ising machine that utilizes optical signals at different wavelengths to represent distinct Ising spins for Ising simulation.We design and experimentally validate a chip-scale Ising machine capable of solving classical non-deterministic polynomial-time problems.The proposed Ising machine supports 32 spins and features 2 distinct coupling encoding schemes.Furthermore,we demonstrate the feasibility of scaling the system to 256 spins.This approach verifies the viability of performing Ising simulations in the wavelength dimension,offering substantial advantages in scalability.These advancements lay the groundwork for future large-scale expansion and practical applications in cloud computing.展开更多
Feature extraction in the optical domain offers a promising low-latency,high-throughput solution.Optical diffraction-based feature extraction operating under a coherent light source can further achieve parallel output...Feature extraction in the optical domain offers a promising low-latency,high-throughput solution.Optical diffraction-based feature extraction operating under a coherent light source can further achieve parallel outputs with low energy consumption.However,it presents significant challenges for maintaining the coherent input,scaling the operation rates beyond 10 GHz,and ensuring the effective extraction of functional configuration simultaneously.We propose an optical feature extraction engine(OFE^(2)),which is composed of a diffraction operator and a data preparation module,powering high-speed feature extraction for both image and temporal series tasks.This OFE^(2)can achieve a core latency of less than 250.5 ps;in addition,it can reach a throughput of 250 GOPS and an efficiency of 2.06 TOPS/W.Supported by the OFE^(2),a novel feature extraction paradigm is emerging,enabling high-speed,low-latency service access for applications in scene recognition,medical assistance,and digital finance.展开更多
In recent years,there has been a significant transformation in the field of incoherent imaging with new possibilities of compressing three-dimensional(3D)information into a two-dimensional intensity distribution witho...In recent years,there has been a significant transformation in the field of incoherent imaging with new possibilities of compressing three-dimensional(3D)information into a two-dimensional intensity distribution without two-beam interference(TBI).Most of the incoherent 3D imagers without TBI are based on scattering by a random phase mask exhibiting sharp autocorrelation and low cross-correlation along the depth.Consequently,during reconstruction,high lateral and axial resolutions are obtained.Imaging based on scattering requires an astronomical photon budget and is therefore precluded in many power-sensitive applications.In this study,a proof-of-concept 3D imaging method without TBI using deterministic fields has been demonstrated.A new reconstruction method called the Lucy-Richardson-Rosen algorithm has been developed for this imaging concept.We believe that the proposed approach will cause a paradigm-shift in the current state-of-the-art incoherent imaging,fluorescence microscopy,mid-infrared fingerprinting,astronomical imaging,and fast object recognition applications.展开更多
Optical deep learning based on diffractive optical elements offers unique advantages for parallel processing,computational speed,and power efficiency.One landmark method is the diffractive deep neural network(D^(2) NN...Optical deep learning based on diffractive optical elements offers unique advantages for parallel processing,computational speed,and power efficiency.One landmark method is the diffractive deep neural network(D^(2) NN)based on three-dimensional printing technology operated in the terahertz spectral range.Since the terahertz bandwidth involves limited interparticle coupling and material losses,this paper extends D^(2) NN to visible wavelengths.A general theory including a revised formula is proposed to solve any contradictions between wavelength,neuron size,and fabrication limitations.A novel visible light D^(2) NN classifier is used to recognize unchanged targets(handwritten digits ranging from 0 to 9)and targets that have been changed(i.e.,targets that have been covered or altered)at a visible wavelength of 632.8 nm.The obtained experimental classification accuracy(84%)and numerical classification accuracy(91.57%)quantify the match between the theoretical design and fabricated system performance.The presented framework can be used to apply a D^(2) NN to various practical applications and design other new applications.展开更多
The rapid development of artificial intelligence(AI)facilitates various applications from all areas but also poses great challenges in its hardware implementation in terms of speed and energy because of the explosive ...The rapid development of artificial intelligence(AI)facilitates various applications from all areas but also poses great challenges in its hardware implementation in terms of speed and energy because of the explosive growth of data.Optical computing provides a distinctive perspective to address this bottleneck by harnessing the unique properties of photons including broad bandwidth,low latency,and high energy efficiency.In this review,we introduce the latest developments of optical computing for different AI models,including feedforward neural networks,reservoir computing,and spiking neural networks(SNNs).Recent progress in integrated photonic devices,combined with the rise of AI,provides a great opportunity for the renaissance of optical computing in practical applications.This effort requires multidisciplinary efforts from a broad community.This review provides an overview of the state-of-the-art accomplishments in recent years,discusses the availability of current technologies,and points out various remaining challenges in different aspects to push the frontier.We anticipate that the era of large-scale integrated photonics processors will soon arrive for practical AI applications in the form of hybrid optoelectronic frameworks.展开更多
A single-lens computational imaging system combines a single lens with post-processing algorithms to achieve a lightweight design while maintaining imaging quality.However,the computational inefficiency of existing re...A single-lens computational imaging system combines a single lens with post-processing algorithms to achieve a lightweight design while maintaining imaging quality.However,the computational inefficiency of existing reconstruction methods often limits the achievable frame rate on edge devices,falling short of the practical requirement of 30-60 frames per second(fps).Here,we adopt a physics-informed neural network that integrates an improved Wiener deconvolution(IWD)with a compact Res-Unet variant.The simple yet effective Wiener deconvolution step reduces image blur and spatially variant degradation,thereby alleviating the workload of the subsequent network and enabling high-quality,real-time reconstruction.Simulation and experimental results demonstrate that this framework can further reduce the algorithmic complexity for a single-lens system,achieving real-time reconstruction at 40 fps for 640×480 resolution on an RK3588 system-on-chip(SoC),while maintaining a system modulation transfer function(MTF)above 0.39 at Nyquist frequency(42 lp/mm).展开更多
The explosive growth of data and information has motivated various emerging non-von Neumann computational approaches in the More-than-Moore era.Photonics neuromorphic computing has attracted lots of attention due to t...The explosive growth of data and information has motivated various emerging non-von Neumann computational approaches in the More-than-Moore era.Photonics neuromorphic computing has attracted lots of attention due to the fascinating advantages such as high speed,wide bandwidth,and massive parallelism.Here,we offer a review on the optical neural computing in our research groups at the device and system levels.The photonics neuron and photonics synapse plasticity are presented.In addition,we introduce several optical neural computing architectures and algorithms including photonic spiking neural network,photonic convolutional neural network,photonic matrix computation,photonic reservoir computing,and photonic reinforcement learning.Finally,we summarize the major challenges faced by photonic neuromorphic computing,and propose promising solutions and perspectives.展开更多
Object identification and three-dimensional reconstruction techniques are always attractive research interests in machine vision,virtual reality,augmented reality,and biomedical engineering.Optical computing metasurfa...Object identification and three-dimensional reconstruction techniques are always attractive research interests in machine vision,virtual reality,augmented reality,and biomedical engineering.Optical computing metasurface,as a two-dimensional artificial design component,has displayed the supernormal character of controlling phase,amplitude,polarization,and frequency distributions of the light beam,capable of performing mathematical operations on the input light field.Here,we propose and demonstrate an all-optical object identification technique based on optical computing metasurface,and apply it to 3D reconstruction.Unlike traditional mechanisms,this scheme reduces memory consumption in the processing of the contour surface extraction.The identification and reconstruction of experimental results from high-contrast and low-contrast objects agree well with the real objects.The exploration of the all-optical object identification and 3D reconstruction techniques provides potential applications of high efficiencies,low consumption,and compact systems.展开更多
A new implementation of high-dimensional quantum key distribution （QKD） protocol is discussed. Using three mutual unbiased bases, we present a d？level six-state QKD protocol that exploits the orbital angular moment...A new implementation of high-dimensional quantum key distribution （QKD） protocol is discussed. Using three mutual unbiased bases, we present a d？level six-state QKD protocol that exploits the orbital angular momentum with the spatial mode of the light beam. The protocol shows that the feature of a high capacity since keys are encoded using photon modes in d-level Hilbert space. The devices for state preparation and measurement are also discussed. This protocol has high security and the alignment of shared reference frames is not needed between sender and receiver.展开更多
Spiking neural networks(SNNs)utilize brain-like spatiotemporal spike encoding for simulating brain functions.Photonic SNN offers an ultrahigh speed and power efficiency platform for implementing high-performance neuro...Spiking neural networks(SNNs)utilize brain-like spatiotemporal spike encoding for simulating brain functions.Photonic SNN offers an ultrahigh speed and power efficiency platform for implementing high-performance neuromorphic computing.Here,we proposed a multi-synaptic photonic SNN,combining the modified remote supervised learning with delayweight co-training to achieve pattern classification.The impact of multi-synaptic connections and the robustness of the network were investigated through numerical simulations.In addition,the collaborative computing of algorithm and hardware was demonstrated based on a fabricated integrated distributed feedback laser with a saturable absorber(DFB-SA),where 10 different noisy digital patterns were successfully classified.A functional photonic SNN that far exceeds the scale limit of hardware integration was achieved based on time-division multiplexing,demonstrating the capability of hardware-algorithm co-computation.展开更多
The generation of various entangled states is an essential task in quantum information processing. Recently, a scheme （PRA 79, 022304） has been suggested for generating Greenberger-Horne-Zeilinger state and cluster ...The generation of various entangled states is an essential task in quantum information processing. Recently, a scheme （PRA 79, 022304） has been suggested for generating Greenberger-Horne-Zeilinger state and cluster state with atomic ensembles based on the Rydberg blockade. Using similar resources as the earlier scheme, here we propose an experimentally feasible scheme of preparing arbitrary four-qubit W class of maximally and non- maximally entangled states with atomic ensembles in a single step. Moreover, we carefully analyze the realistic noises and predict that four-qubit W states can be produced with high fidelity （F - 0.994） via our scheme.展开更多
Optical computing and optical neural network have gained increasing attention in recent years because of their potential advantages of parallel processing at the speed of light and low power consumption by comparison ...Optical computing and optical neural network have gained increasing attention in recent years because of their potential advantages of parallel processing at the speed of light and low power consumption by comparison with electronic computing.The optical implementation of the fundamental building blocks of a digital computer,i.e.logic gates,has been investigated extensively in the past few decades.Optical logic gate computing is an alternative approach to various analogue optical computing architectures.In this paper,the latest development of optical logic gate computing with different kinds of implementations is reviewed.Firstly,the basic concepts of analogue and digital computing with logic gates in the electronic and optical domains are introduced.And then a comprehensive summary of various optical logic gate schemes including spatial encoding of light field,semiconductor optical amplifiers(SOA),highly nonlinear fiber(HNLF),microscale and nanoscale waveguides,and photonic crystal structures is presented.To conclude,the formidable challenges in developing practical all-optical logic gates are analyzed and the prospects of the future are discussed.展开更多
基金supported by the National Natural Science Foundation of China(Grant Nos.62192774,62105243,61925504,6201101335,62020106009,62192770,62192772,62105244,62305250,and 62322217)the Science and Technology Commission of Shanghai Municipality(Grant Nos.17JC1400800,20JC1414600,and 21JC1406100)+1 种基金the Shanghai Municipal Science and Technology Major Project(Grant No.2021SHZDZX0100)the Fundamental Research Funds for the Central Universities.
文摘Manufacturing-robust imaging systems leveraging computational optics hold immense potential for easing manufacturing constraints and enabling the development of cost-effective,high-quality imaging solutions.However,conventional approaches,which typically rely on data-driven neural networks to correct optical aberrations caused by manufacturing errors,are constrained by the lack of effective tolerance analysis methods for quantitatively evaluating manufacturing error boundaries.This limitation is crucial for further relaxing manufacturing constraints and providing practical guidance for fabrication.We propose a physics-informed design paradigm for manufacturing-robust imaging systems with computational optics,integrating a physics-informed tolerance analysis methodology for evaluating manufacturing error boundaries and a physics-informed neural network for image reconstruction.With this approach,we achieve a manufacturing-robust imaging system based on an off-axis three-mirror freeform all-aluminum design,delivering a modulation transfer function exceeding 0.34 at the Nyquist frequency(72 lp/mm)in simulation.Notably,this system requires a manufacturing precision of only 0.5λin root mean square(RMS),representing a remarkable 25-fold relaxation compared with the conventional requirement of 0.02λin RMS.Experimental validation further confirmed that the manufacturing-robust imaging system maintains excellent performance in diverse indoor and outdoor environments.Our proposed method paves the way for achieving high-quality imaging without the necessity of high manufacturing precision,enabling practical solutions that are more cost-effective and time-efficient.
基金supported by the National Natural Science Foundation of China(Nos.62131003,62322502,62088101)the Guangdong Province Key Laboratory of Intelligent Detection in Complex Environment of Aerospace,Land and Sea(No.2022KSYS016).
文摘Computational optics introduces computation into optics and consequently helps overcome traditional optical limitations such as low sensing dimension,low light throughput,low resolution,and so on.The combination of optical encoding and computational decoding offers enhanced imaging and sensing capabilities with diverse applications in biomedicine,astronomy,agriculture,etc.With the great advance of artificial intelligence in the last decade,deep learning has further boosted computational optics with higher precision and efficiency.Recently,there developed an end-to-end joint optimization technique that digitally twins optical encoding to neural network layers,and then facilitates simultaneous optimization with the decoding process.This framework offers effective performance enhancement over conventional techniques.However,the reverse physical twinning from optimized encoding parameters to practical modulation elements faces a serious challenge,due to the discrepant gap in such as bit depth,numerical range,and stability.In this regard,this review explores various optical modulation elements across spatial,phase,and spectral dimensions in the digital twin model for joint encoding-decoding optimization.Our analysis offers constructive guidance for finding the most appropriate modulation element in diverse imaging and sensing tasks concerning various requirements of precision,speed,and robustness.The review may help tackle the above twinning challenge and pave the way for next-generation computational optics.
基金National Natural Science Foundation of China(No.12574332)the Space Optoelectronic Measurement and Perception Lab.,Beijing Institute of Control Engineering(No.LabSOMP-2023-10)Major Science and Technology Innovation Program of Xianyang City(No.L2024-ZDKJ-ZDCGZH-0021)。
文摘Fourier Ptychographic Microscopy(FPM)is a high-throughput computational optical imaging technology reported in 2013.It effectively breaks through the trade-off between high-resolution imaging and wide-field imaging.In recent years,it has been found that FPM is not only a tool to break through the trade-off between field of view and spatial resolution,but also a paradigm to break through those trade-off problems,thus attracting extensive attention.Compared with previous reviews,this review does not introduce its concept,basic principles,optical system and series of applications once again,but focuses on elaborating the three major difficulties faced by FPM technology in the process from“looking good”in the laboratory to“working well”in practical applications:mismatch between numerical model and physical reality,long reconstruction time and high computing power demand,and lack of multi-modal expansion.It introduces how to achieve key technological innovations in FPM through the dual drive of Artificial Intelligence(AI)and physics,including intelligent reconstruction algorithms introducing machine learning concepts,optical-algorithm co-design,fusion of frequency domain extrapolation methods and generative adversarial networks,multi-modal imaging schemes and data fusion enhancement,etc.,gradually solving the difficulties of FPM technology.Conversely,this review deeply considers the unique value of FPM technology in potentially feeding back to the development of“AI+optics”,such as providing AI benchmark tests under physical constraints,inspirations for the balance of computing power and bandwidth in miniaturized intelligent microscopes,and photoelectric hybrid architectures.Finally,it introduces the industrialization path and frontier directions of FPM technology,pointing out that with the promotion of the dual drive of AI and physics,it will generate a large number of industrial application case,and looks forward to the possibilities of future application scenarios and expansions,for instance,body fluid biopsy and point-of-care testing at the grassroots level represent the expansion of the growth market.
文摘Computational optical imaging is an interdisciplinary subject integrating optics, mathematics, and information technology. It introduces information processing into optical imaging and combines it with intelligent computing, subverting the imaging mechanism of traditional optical imaging which only relies on orderly information transmission. To meet the high-precision requirements of traditional optical imaging for optical processing and adjustment, as well as to solve its problems of being sensitive to gravity and temperature in use, we establish an optical imaging system model from the perspective of computational optical imaging and studies how to design and solve the imaging consistency problem of optical system under the influence of gravity, thermal effect, stress, and other external environment to build a high robustness optical system. The results show that the high robustness interval of the optical system exists and can effectively reduce the sensitivity of the optical system to the disturbance of each link, thus realizing the high robustness of optical imaging.
基金supported by the National Science Foundation(Grant Nos.NSF-ECCS-2127235 and EFRI-BRAID-2223495)Part of this work was conducted at the Washington Nanofabrication Facility/Molecular Analysis Facility,a National Nanotechnology Coordinated Infrastructure(NNCI)site at the University of Washington with partial support from the National Science Foundation(Grant Nos.NNCI-1542101 and NNCI-2025489).
文摘Optical and hybrid convolutional neural networks(CNNs)recently have become of increasing interest to achieve low-latency,low-power image classification,and computer-vision tasks.However,implementing optical nonlinearity is challenging,and omitting the nonlinear layers in a standard CNN comes with a significant reduction in accuracy.We use knowledge distillation to compress modified AlexNet to a single linear convolutional layer and an electronic backend(two fully connected layers).We obtain comparable performance with a purely electronic CNN with five convolutional layers and three fully connected layers.We implement the convolution optically via engineering the point spread function of an inverse-designed meta-optic.Using this hybrid approach,we estimate a reduction in multiply-accumulate operations from 17M in a conventional electronic modified AlexNet to only 86 K in the hybrid compressed network enabled by the optical front end.This constitutes over 2 orders of magnitude of reduction in latency and power consumption.Furthermore,we experimentally demonstrate that the classification accuracy of the system exceeds 93%on the MNIST dataset of handwritten digits.
基金supports from National Natural Science Foundation of China(62171087,62475036).
文摘Photonic hardware implementation of spiking neural networks,regarded as a viable potential paradigm for ultra-high speed and energy efficiency computing,leverages spatiotemporal spike encoding and event-driven dynamics to simulate brain-like parallel information processing.Silicon-based microring resonators(MRRs)offer a power efficiency and ultrahigh flexibility scheme to mimic biological neuron,however,their substantial potential for integrated neuromorphic systems remains limited by insufficient exploration of MRR-based spiking digital and analog computation.Here,an all-optical neural dynamics framework,encompassing both excitatory and inhibitory behaviors based on multi-wavelength auxiliary and competition mechanism in an MRR,is proposed numerically.Leveraging multi-wavelength resonance characteristics and wavelength division multiplexing(WDM)technology,a single MRR implements the five fundamental optical digital logic gates:AND,OR,NOT,XNOR and XOR.Besides,the cascading capabilities of MRR-based spiking neurons are demonstrated through multi-level digital logic gates including NAND,NOR,4-input AND,8-input AND,and a full adder,emphasizing their promise for large-scale digital logic networks.Furthermore,an exemplary binary convolution has been achieved by utilizing the proposed MRR-based digital logic operation,illustrating the potential of all-optical binary convolution to compute image gradient magnitudes for edge detection.Such passive photonic neurons and networks promise access to the high transmission speed and low power consumption inherent to optical systems,thus enabling direct hardware-algorithm co-computation and accelerating artificial intelligence.
基金supported by the National Natural Science Foundation of China(Grant Nos.62175050 and U2341245)the Fundamental Research Funds for the Central Universities(Grant No.HIT.OCEF.2024054).
文摘Diffractive optical neural networks(DONNs)have exhibited the advantages of parallelization,high speed,and low consumption.However,the existing DONNs based on free-space diffractive optical elements are bulky and unsteady.In this study,we propose a planar-waveguide integrated diffractive neural network chip architecture.The three diffractive layers are engraved on the same side of a quartz wafer.The three-layer chip is designed with 32-mm3 processing space and enables a computing speed of 3.1×109 Tera operations per second.The results show that the proposed chip achieves 73.4%experimental accuracy for the Modified National Institute of Standards and Technology database while showing the system’s robustness in a cycle test.The consistency of experiments is 88.6%,and the arithmetic mean standard deviation of the results is~4.7%.The proposed chip architecture can potentially revolutionize high-resolution optical processing tasks with high robustness.
基金supported by the Key Project of Chongqing Natural Science Foundation Joint Fund[CSTB2023NSCQ-LZX0103,(G.Z.)]Chongqing Natural Science Foundation[CSTB2024NSCQ-MSX0012,(C.L.)]+1 种基金Fundamental Research Funds for the Central Universities[SWUZLPY03,(G.Z.)]Fundamental Research Funds for the Central Universities[Swu020019,(G.Z.):SWU-XDJH202319,(G.Z.)1].
文摘With the advancement of artificial intelligence,optic in-sensing reservoir computing based on emerging semiconductor devices is high desirable for real-time analog signal processing.Here,we disclose a flexible optomemristor based on C_(27)H_(30)O_(15)/FeOx heterostructure that presents a highly sensitive to the light stimuli and artificial optic synaptic features such as short-and long-term plasticity(STP and LTP),enabling the developed optomemristor to implement complex analogy signal processing through building a real-physical dynamic-based in-sensing reservoir computing algorithm and yielding an accuracy of 94.88%for speech recognition.The charge trapping and detrapping mediated by the optic active layer of C_(27)H_(30)O_(15) that is extracted from the lotus flower is response for the positive photoconductance memory in the prepared optomemristor.This work provides a feasible organic−inorganic heterostructure as well as an optic in-sensing vision computing for an advanced optic computing system in future complex signal processing.
基金supported by the National Key Research and Development Program of China(Grant No.2022YFB2804203)the National Natural Science Foundation of China(Grant No.U21A20511)the Knowledge Innovation Program of Wuhan-Basic Research(Grant No.2023010201010049).
文摘Ising problems are critical for a wide range of applications.Solving these problems on a photonic platform takes advantage of the unique properties of photons,such as high speed,low power consumption,and large bandwidth.Recently,there has been growing interest in using photonic platforms to accelerate the optimization of Ising models,paving the way for the development of ultrafast hardware in machine learning.However,these proposed systems face challenges in simultaneously achieving high spin scalability,encoding flexibility,and low system complexity.We propose a wavelength-domain optical Ising machine that utilizes optical signals at different wavelengths to represent distinct Ising spins for Ising simulation.We design and experimentally validate a chip-scale Ising machine capable of solving classical non-deterministic polynomial-time problems.The proposed Ising machine supports 32 spins and features 2 distinct coupling encoding schemes.Furthermore,we demonstrate the feasibility of scaling the system to 256 spins.This approach verifies the viability of performing Ising simulations in the wavelength dimension,offering substantial advantages in scalability.These advancements lay the groundwork for future large-scale expansion and practical applications in cloud computing.
基金supported by the National Key Research and Development Program of China(Grant No.2024YFE0203600)the National Natural Science Foundation of China(Grant No.62135009).
文摘Feature extraction in the optical domain offers a promising low-latency,high-throughput solution.Optical diffraction-based feature extraction operating under a coherent light source can further achieve parallel outputs with low energy consumption.However,it presents significant challenges for maintaining the coherent input,scaling the operation rates beyond 10 GHz,and ensuring the effective extraction of functional configuration simultaneously.We propose an optical feature extraction engine(OFE^(2)),which is composed of a diffraction operator and a data preparation module,powering high-speed feature extraction for both image and temporal series tasks.This OFE^(2)can achieve a core latency of less than 250.5 ps;in addition,it can reach a throughput of 250 GOPS and an efficiency of 2.06 TOPS/W.Supported by the OFE^(2),a novel feature extraction paradigm is emerging,enabling high-speed,low-latency service access for applications in scene recognition,medical assistance,and digital finance.
基金European Union’s Horizon 2020 research and innovation programme under grant agreement No.857627(CIPHR).
文摘In recent years,there has been a significant transformation in the field of incoherent imaging with new possibilities of compressing three-dimensional(3D)information into a two-dimensional intensity distribution without two-beam interference(TBI).Most of the incoherent 3D imagers without TBI are based on scattering by a random phase mask exhibiting sharp autocorrelation and low cross-correlation along the depth.Consequently,during reconstruction,high lateral and axial resolutions are obtained.Imaging based on scattering requires an astronomical photon budget and is therefore precluded in many power-sensitive applications.In this study,a proof-of-concept 3D imaging method without TBI using deterministic fields has been demonstrated.A new reconstruction method called the Lucy-Richardson-Rosen algorithm has been developed for this imaging concept.We believe that the proposed approach will cause a paradigm-shift in the current state-of-the-art incoherent imaging,fluorescence microscopy,mid-infrared fingerprinting,astronomical imaging,and fast object recognition applications.
基金This research was supported in part by National Natural Science Foundation of China(61675056 and 61875048).
文摘Optical deep learning based on diffractive optical elements offers unique advantages for parallel processing,computational speed,and power efficiency.One landmark method is the diffractive deep neural network(D^(2) NN)based on three-dimensional printing technology operated in the terahertz spectral range.Since the terahertz bandwidth involves limited interparticle coupling and material losses,this paper extends D^(2) NN to visible wavelengths.A general theory including a revised formula is proposed to solve any contradictions between wavelength,neuron size,and fabrication limitations.A novel visible light D^(2) NN classifier is used to recognize unchanged targets(handwritten digits ranging from 0 to 9)and targets that have been changed(i.e.,targets that have been covered or altered)at a visible wavelength of 632.8 nm.The obtained experimental classification accuracy(84%)and numerical classification accuracy(91.57%)quantify the match between the theoretical design and fabricated system performance.The presented framework can be used to apply a D^(2) NN to various practical applications and design other new applications.
基金supported by the National Natural Science Foundation of China(61927802,61722209,and 61805145)the Beijing Municipal Science and Technology Commission(Z181100003118014)+3 种基金the National Key Research and Development Program of China(2020AAA0130000)the support from the National Postdoctoral Program for Innovative TalentShuimu Tsinghua Scholar Programthe support from the Hong Kong Research Grants Council(16306220)。
文摘The rapid development of artificial intelligence(AI)facilitates various applications from all areas but also poses great challenges in its hardware implementation in terms of speed and energy because of the explosive growth of data.Optical computing provides a distinctive perspective to address this bottleneck by harnessing the unique properties of photons including broad bandwidth,low latency,and high energy efficiency.In this review,we introduce the latest developments of optical computing for different AI models,including feedforward neural networks,reservoir computing,and spiking neural networks(SNNs).Recent progress in integrated photonic devices,combined with the rise of AI,provides a great opportunity for the renaissance of optical computing in practical applications.This effort requires multidisciplinary efforts from a broad community.This review provides an overview of the state-of-the-art accomplishments in recent years,discusses the availability of current technologies,and points out various remaining challenges in different aspects to push the frontier.We anticipate that the era of large-scale integrated photonics processors will soon arrive for practical AI applications in the form of hybrid optoelectronic frameworks.
基金supported by the National Natural Science Foundation of China(Nos.62305250,61925504,61621001,62105243)the Major Projects of Science and Technology Commission of Shanghai(No.17JC1400800)the Fundamental Research Funds for the Central Universities,Shanghai Municipal Science and Technology Major Project(No.2021SHZDZX0100)。
文摘A single-lens computational imaging system combines a single lens with post-processing algorithms to achieve a lightweight design while maintaining imaging quality.However,the computational inefficiency of existing reconstruction methods often limits the achievable frame rate on edge devices,falling short of the practical requirement of 30-60 frames per second(fps).Here,we adopt a physics-informed neural network that integrates an improved Wiener deconvolution(IWD)with a compact Res-Unet variant.The simple yet effective Wiener deconvolution step reduces image blur and spatially variant degradation,thereby alleviating the workload of the subsequent network and enabling high-quality,real-time reconstruction.Simulation and experimental results demonstrate that this framework can further reduce the algorithmic complexity for a single-lens system,achieving real-time reconstruction at 40 fps for 640×480 resolution on an RK3588 system-on-chip(SoC),while maintaining a system modulation transfer function(MTF)above 0.39 at Nyquist frequency(42 lp/mm).
基金This work was supported in part by the National Outstanding Youth Science Fund Project of National Natural Science Foundation of China(62022062)the National Natural Science Foundation of China(61974177,61674119)the Fundamental Research Funds for the Central Universities.
文摘The explosive growth of data and information has motivated various emerging non-von Neumann computational approaches in the More-than-Moore era.Photonics neuromorphic computing has attracted lots of attention due to the fascinating advantages such as high speed,wide bandwidth,and massive parallelism.Here,we offer a review on the optical neural computing in our research groups at the device and system levels.The photonics neuron and photonics synapse plasticity are presented.In addition,we introduce several optical neural computing architectures and algorithms including photonic spiking neural network,photonic convolutional neural network,photonic matrix computation,photonic reservoir computing,and photonic reinforcement learning.Finally,we summarize the major challenges faced by photonic neuromorphic computing,and propose promising solutions and perspectives.
基金support from the National Natural Science Foundation of China(Grant Nos.12174097 and 12304321)the Natural Science Foundation of Hunan Province(Grant Nos.2021JJ10008 and 2023JJ40202)the Research Foundation of Education Bureau of Hunan Province(Grant No.22B0871).
文摘Object identification and three-dimensional reconstruction techniques are always attractive research interests in machine vision,virtual reality,augmented reality,and biomedical engineering.Optical computing metasurface,as a two-dimensional artificial design component,has displayed the supernormal character of controlling phase,amplitude,polarization,and frequency distributions of the light beam,capable of performing mathematical operations on the input light field.Here,we propose and demonstrate an all-optical object identification technique based on optical computing metasurface,and apply it to 3D reconstruction.Unlike traditional mechanisms,this scheme reduces memory consumption in the processing of the contour surface extraction.The identification and reconstruction of experimental results from high-contrast and low-contrast objects agree well with the real objects.The exploration of the all-optical object identification and 3D reconstruction techniques provides potential applications of high efficiencies,low consumption,and compact systems.
基金Supported by the National Basic Research Program of China under Grant Nos 2006CB921106 and 2010CB923202, the Fundamental Research Funds for the Central Universities No BUPT2009RC0710, the Specialized Research Fund for the Doctoral Program of Higher Education of China under Grant No 20090005120008, and the National Natural Science Foundation of China under Grant No 10947151.
文摘A new implementation of high-dimensional quantum key distribution （QKD） protocol is discussed. Using three mutual unbiased bases, we present a d？level six-state QKD protocol that exploits the orbital angular momentum with the spatial mode of the light beam. The protocol shows that the feature of a high capacity since keys are encoded using photon modes in d-level Hilbert space. The devices for state preparation and measurement are also discussed. This protocol has high security and the alignment of shared reference frames is not needed between sender and receiver.
基金supports from the National Key Research and Development Program of China (Nos.2021YFB2801900,2021YFB2801901,2021YFB2801902,2021YFB2801903,2021YFB2801904)the National Outstanding Youth Science Fund Project of National Natural Science Foundation of China (No.62022062)+1 种基金the National Natural Science Foundation of China (No.61974177)the Fundamental Research Funds for the Central Universities (No.QTZX23041).
文摘Spiking neural networks(SNNs)utilize brain-like spatiotemporal spike encoding for simulating brain functions.Photonic SNN offers an ultrahigh speed and power efficiency platform for implementing high-performance neuromorphic computing.Here,we proposed a multi-synaptic photonic SNN,combining the modified remote supervised learning with delayweight co-training to achieve pattern classification.The impact of multi-synaptic connections and the robustness of the network were investigated through numerical simulations.In addition,the collaborative computing of algorithm and hardware was demonstrated based on a fabricated integrated distributed feedback laser with a saturable absorber(DFB-SA),where 10 different noisy digital patterns were successfully classified.A functional photonic SNN that far exceeds the scale limit of hardware integration was achieved based on time-division multiplexing,demonstrating the capability of hardware-algorithm co-computation.
基金Supported by the National Natural Science Foundation of China under Grant No 10774192, the Fund of Innovation of Graduate School of National University of Defense Technology under Grant No 080201.
文摘The generation of various entangled states is an essential task in quantum information processing. Recently, a scheme （PRA 79, 022304） has been suggested for generating Greenberger-Horne-Zeilinger state and cluster state with atomic ensembles based on the Rydberg blockade. Using similar resources as the earlier scheme, here we propose an experimentally feasible scheme of preparing arbitrary four-qubit W class of maximally and non- maximally entangled states with atomic ensembles in a single step. Moreover, we carefully analyze the realistic noises and predict that four-qubit W states can be produced with high fidelity （F - 0.994） via our scheme.
基金supported by the National Key Research and Development Program of China(Grants No.2021YFA1401500)the National Natural Science Foundation of China(12022416)+3 种基金the Department of Natural Resources of Guangdong Province(No.GDNRC[2022]22)Department of Science and Technology of Guangdong Province(No.2021A0505080002)Intelligent Laser Basic Research Laboratory(No.PCL2021A14-B1)the Hong Kong Research Grants Council(16306220).
文摘Optical computing and optical neural network have gained increasing attention in recent years because of their potential advantages of parallel processing at the speed of light and low power consumption by comparison with electronic computing.The optical implementation of the fundamental building blocks of a digital computer,i.e.logic gates,has been investigated extensively in the past few decades.Optical logic gate computing is an alternative approach to various analogue optical computing architectures.In this paper,the latest development of optical logic gate computing with different kinds of implementations is reviewed.Firstly,the basic concepts of analogue and digital computing with logic gates in the electronic and optical domains are introduced.And then a comprehensive summary of various optical logic gate schemes including spatial encoding of light field,semiconductor optical amplifiers(SOA),highly nonlinear fiber(HNLF),microscale and nanoscale waveguides,and photonic crystal structures is presented.To conclude,the formidable challenges in developing practical all-optical logic gates are analyzed and the prospects of the future are discussed.
