Manufacturing-robust imaging systems leveraging computational optics hold immense potential for easing manufacturing constraints and enabling the development of cost-effective,high-quality imaging solutions.However,co...Manufacturing-robust imaging systems leveraging computational optics hold immense potential for easing manufacturing constraints and enabling the development of cost-effective,high-quality imaging solutions.However,conventional approaches,which typically rely on data-driven neural networks to correct optical aberrations caused by manufacturing errors,are constrained by the lack of effective tolerance analysis methods for quantitatively evaluating manufacturing error boundaries.This limitation is crucial for further relaxing manufacturing constraints and providing practical guidance for fabrication.We propose a physics-informed design paradigm for manufacturing-robust imaging systems with computational optics,integrating a physics-informed tolerance analysis methodology for evaluating manufacturing error boundaries and a physics-informed neural network for image reconstruction.With this approach,we achieve a manufacturing-robust imaging system based on an off-axis three-mirror freeform all-aluminum design,delivering a modulation transfer function exceeding 0.34 at the Nyquist frequency(72 lp/mm)in simulation.Notably,this system requires a manufacturing precision of only 0.5λin root mean square(RMS),representing a remarkable 25-fold relaxation compared with the conventional requirement of 0.02λin RMS.Experimental validation further confirmed that the manufacturing-robust imaging system maintains excellent performance in diverse indoor and outdoor environments.Our proposed method paves the way for achieving high-quality imaging without the necessity of high manufacturing precision,enabling practical solutions that are more cost-effective and time-efficient.展开更多
Fourier Ptychographic Microscopy(FPM)is a high-throughput computational optical imaging technology reported in 2013.It effectively breaks through the trade-off between high-resolution imaging and wide-field imaging.In...Fourier Ptychographic Microscopy(FPM)is a high-throughput computational optical imaging technology reported in 2013.It effectively breaks through the trade-off between high-resolution imaging and wide-field imaging.In recent years,it has been found that FPM is not only a tool to break through the trade-off between field of view and spatial resolution,but also a paradigm to break through those trade-off problems,thus attracting extensive attention.Compared with previous reviews,this review does not introduce its concept,basic principles,optical system and series of applications once again,but focuses on elaborating the three major difficulties faced by FPM technology in the process from“looking good”in the laboratory to“working well”in practical applications:mismatch between numerical model and physical reality,long reconstruction time and high computing power demand,and lack of multi-modal expansion.It introduces how to achieve key technological innovations in FPM through the dual drive of Artificial Intelligence(AI)and physics,including intelligent reconstruction algorithms introducing machine learning concepts,optical-algorithm co-design,fusion of frequency domain extrapolation methods and generative adversarial networks,multi-modal imaging schemes and data fusion enhancement,etc.,gradually solving the difficulties of FPM technology.Conversely,this review deeply considers the unique value of FPM technology in potentially feeding back to the development of“AI+optics”,such as providing AI benchmark tests under physical constraints,inspirations for the balance of computing power and bandwidth in miniaturized intelligent microscopes,and photoelectric hybrid architectures.Finally,it introduces the industrialization path and frontier directions of FPM technology,pointing out that with the promotion of the dual drive of AI and physics,it will generate a large number of industrial application case,and looks forward to the possibilities of future application scenarios and expansions,for instance,body fluid biopsy and point-of-care testing at the grassroots level represent the expansion of the growth market.展开更多
Optical and hybrid convolutional neural networks(CNNs)recently have become of increasing interest to achieve low-latency,low-power image classification,and computer-vision tasks.However,implementing optical nonlineari...Optical and hybrid convolutional neural networks(CNNs)recently have become of increasing interest to achieve low-latency,low-power image classification,and computer-vision tasks.However,implementing optical nonlinearity is challenging,and omitting the nonlinear layers in a standard CNN comes with a significant reduction in accuracy.We use knowledge distillation to compress modified AlexNet to a single linear convolutional layer and an electronic backend(two fully connected layers).We obtain comparable performance with a purely electronic CNN with five convolutional layers and three fully connected layers.We implement the convolution optically via engineering the point spread function of an inverse-designed meta-optic.Using this hybrid approach,we estimate a reduction in multiply-accumulate operations from 17M in a conventional electronic modified AlexNet to only 86 K in the hybrid compressed network enabled by the optical front end.This constitutes over 2 orders of magnitude of reduction in latency and power consumption.Furthermore,we experimentally demonstrate that the classification accuracy of the system exceeds 93%on the MNIST dataset of handwritten digits.展开更多
Diffractive optical neural networks(DONNs)have exhibited the advantages of parallelization,high speed,and low consumption.However,the existing DONNs based on free-space diffractive optical elements are bulky and unste...Diffractive optical neural networks(DONNs)have exhibited the advantages of parallelization,high speed,and low consumption.However,the existing DONNs based on free-space diffractive optical elements are bulky and unsteady.In this study,we propose a planar-waveguide integrated diffractive neural network chip architecture.The three diffractive layers are engraved on the same side of a quartz wafer.The three-layer chip is designed with 32-mm3 processing space and enables a computing speed of 3.1×109 Tera operations per second.The results show that the proposed chip achieves 73.4%experimental accuracy for the Modified National Institute of Standards and Technology database while showing the system’s robustness in a cycle test.The consistency of experiments is 88.6%,and the arithmetic mean standard deviation of the results is~4.7%.The proposed chip architecture can potentially revolutionize high-resolution optical processing tasks with high robustness.展开更多
Computational optical imaging is an interdisciplinary subject integrating optics, mathematics, and information technology. It introduces information processing into optical imaging and combines it with intelligent com...Computational optical imaging is an interdisciplinary subject integrating optics, mathematics, and information technology. It introduces information processing into optical imaging and combines it with intelligent computing, subverting the imaging mechanism of traditional optical imaging which only relies on orderly information transmission. To meet the high-precision requirements of traditional optical imaging for optical processing and adjustment, as well as to solve its problems of being sensitive to gravity and temperature in use, we establish an optical imaging system model from the perspective of computational optical imaging and studies how to design and solve the imaging consistency problem of optical system under the influence of gravity, thermal effect, stress, and other external environment to build a high robustness optical system. The results show that the high robustness interval of the optical system exists and can effectively reduce the sensitivity of the optical system to the disturbance of each link, thus realizing the high robustness of optical imaging.展开更多
On-chip diffractive optical neural networks(DONNs)bring the advantages of parallel processing and low energy consumption.However,an accurate representation of the optical field’s evolution in the structure cannot be ...On-chip diffractive optical neural networks(DONNs)bring the advantages of parallel processing and low energy consumption.However,an accurate representation of the optical field’s evolution in the structure cannot be provided using the previous diffraction-based analysis method.Moreover,the loss caused by the open boundaries poses challenges to applications.A multimode DONN architecture based on a more precise eigenmode analysis method is proposed.We have constructed a universal library of input,output,and metaline structures utilizing this method,and realized a multimode DONN composed of the structures from the library.On the designed multimode DONNs with only one layer of the metaline,the classification task of an Iris plants dataset is verified with an accuracy of 90%on the blind test dataset,and the performance of the one-bit binary adder task is also validated.Compared to the previous architectures,the multimode DONN exhibits a more compact design and higher energy efficiency.展开更多
In a single-pixel fast imaging setup,the data collected by the single-pixel detector needs to be processed by a computer,but the speed of the latter will affect the image reconstruction time.Here we propose two kinds ...In a single-pixel fast imaging setup,the data collected by the single-pixel detector needs to be processed by a computer,but the speed of the latter will affect the image reconstruction time.Here we propose two kinds of setups which are able to transform non-visible into visible light imaging,wherein their computing process is replaced by a camera integration mode.The image captured by the camera has a low contrast,so here we present an algorithm that can realize a high quality image in near-infrared to visible cross-waveband imaging.The scheme is verified both by simulation and in actual experiments.The setups demonstrate the great potential for single-pixel imaging and high-speed cross-waveband imaging for future practical applications.展开更多
The ultimate goal of artificial intelligence(AI)is to mimic the human brain to perform decision-making and control directly from high-dimensional sensory input.Diffractive optical networks(DONs)provide a promising sol...The ultimate goal of artificial intelligence(AI)is to mimic the human brain to perform decision-making and control directly from high-dimensional sensory input.Diffractive optical networks(DONs)provide a promising solution for implementing AI with high speed and low power-consumption.Most reported DONs focus on tasks that do not involve environmental interaction,such as object recognition and image classification.By contrast,the networks capable of decision-making and control have not been developed.Here,we propose using deep reinforcement learning to implement DONs that imitate human-level decisionmaking and control capability.Such networks,which take advantage of a residual architecture,allow finding optimal control policies through interaction with the environment and can be readily implemented with existing optical devices.The superior performance is verified using three types of classic games:tic-tac-toe,Super Mario Bros.,and Car Racing.Finally,we present an experimental demonstration of playing tic-tac-toe using the network based on a spatial light modulator.Our work represents a solid step forward in advancing DONs,which promises a fundamental shift from simple recognition or classification tasks to the high-level sensory capability of AI.It may find exciting applications in autonomous driving,intelligent robots,and intelligent manufacturing.展开更多
With the advancement of artificial intelligence,optic in-sensing reservoir computing based on emerging semiconductor devices is high desirable for real-time analog signal processing.Here,we disclose a flexible optomem...With the advancement of artificial intelligence,optic in-sensing reservoir computing based on emerging semiconductor devices is high desirable for real-time analog signal processing.Here,we disclose a flexible optomemristor based on C_(27)H_(30)O_(15)/FeOx heterostructure that presents a highly sensitive to the light stimuli and artificial optic synaptic features such as short-and long-term plasticity(STP and LTP),enabling the developed optomemristor to implement complex analogy signal processing through building a real-physical dynamic-based in-sensing reservoir computing algorithm and yielding an accuracy of 94.88%for speech recognition.The charge trapping and detrapping mediated by the optic active layer of C_(27)H_(30)O_(15) that is extracted from the lotus flower is response for the positive photoconductance memory in the prepared optomemristor.This work provides a feasible organic−inorganic heterostructure as well as an optic in-sensing vision computing for an advanced optic computing system in future complex signal processing.展开更多
Computational optics introduces computation into optics and consequently helps overcome traditional optical limitations such as low sensing dimension,low light throughput,low resolution,and so on.The combination of op...Computational optics introduces computation into optics and consequently helps overcome traditional optical limitations such as low sensing dimension,low light throughput,low resolution,and so on.The combination of optical encoding and computational decoding offers enhanced imaging and sensing capabilities with diverse applications in biomedicine,astronomy,agriculture,etc.With the great advance of artificial intelligence in the last decade,deep learning has further boosted computational optics with higher precision and efficiency.Recently,there developed an end-to-end joint optimization technique that digitally twins optical encoding to neural network layers,and then facilitates simultaneous optimization with the decoding process.This framework offers effective performance enhancement over conventional techniques.However,the reverse physical twinning from optimized encoding parameters to practical modulation elements faces a serious challenge,due to the discrepant gap in such as bit depth,numerical range,and stability.In this regard,this review explores various optical modulation elements across spatial,phase,and spectral dimensions in the digital twin model for joint encoding-decoding optimization.Our analysis offers constructive guidance for finding the most appropriate modulation element in diverse imaging and sensing tasks concerning various requirements of precision,speed,and robustness.The review may help tackle the above twinning challenge and pave the way for next-generation computational optics.展开更多
The rapid development of artificial intelligence(AI)facilitates various applications from all areas but also poses great challenges in its hardware implementation in terms of speed and energy because of the explosive ...The rapid development of artificial intelligence(AI)facilitates various applications from all areas but also poses great challenges in its hardware implementation in terms of speed and energy because of the explosive growth of data.Optical computing provides a distinctive perspective to address this bottleneck by harnessing the unique properties of photons including broad bandwidth,low latency,and high energy efficiency.In this review,we introduce the latest developments of optical computing for different AI models,including feedforward neural networks,reservoir computing,and spiking neural networks(SNNs).Recent progress in integrated photonic devices,combined with the rise of AI,provides a great opportunity for the renaissance of optical computing in practical applications.This effort requires multidisciplinary efforts from a broad community.This review provides an overview of the state-of-the-art accomplishments in recent years,discusses the availability of current technologies,and points out various remaining challenges in different aspects to push the frontier.We anticipate that the era of large-scale integrated photonics processors will soon arrive for practical AI applications in the form of hybrid optoelectronic frameworks.展开更多
The explosive growth of data and information has motivated various emerging non-von Neumann computational approaches in the More-than-Moore era.Photonics neuromorphic computing has attracted lots of attention due to t...The explosive growth of data and information has motivated various emerging non-von Neumann computational approaches in the More-than-Moore era.Photonics neuromorphic computing has attracted lots of attention due to the fascinating advantages such as high speed,wide bandwidth,and massive parallelism.Here,we offer a review on the optical neural computing in our research groups at the device and system levels.The photonics neuron and photonics synapse plasticity are presented.In addition,we introduce several optical neural computing architectures and algorithms including photonic spiking neural network,photonic convolutional neural network,photonic matrix computation,photonic reservoir computing,and photonic reinforcement learning.Finally,we summarize the major challenges faced by photonic neuromorphic computing,and propose promising solutions and perspectives.展开更多
Object identification and three-dimensional reconstruction techniques are always attractive research interests in machine vision,virtual reality,augmented reality,and biomedical engineering.Optical computing metasurfa...Object identification and three-dimensional reconstruction techniques are always attractive research interests in machine vision,virtual reality,augmented reality,and biomedical engineering.Optical computing metasurface,as a two-dimensional artificial design component,has displayed the supernormal character of controlling phase,amplitude,polarization,and frequency distributions of the light beam,capable of performing mathematical operations on the input light field.Here,we propose and demonstrate an all-optical object identification technique based on optical computing metasurface,and apply it to 3D reconstruction.Unlike traditional mechanisms,this scheme reduces memory consumption in the processing of the contour surface extraction.The identification and reconstruction of experimental results from high-contrast and low-contrast objects agree well with the real objects.The exploration of the all-optical object identification and 3D reconstruction techniques provides potential applications of high efficiencies,low consumption,and compact systems.展开更多
Optical computing and optical neural network have gained increasing attention in recent years because of their potential advantages of parallel processing at the speed of light and low power consumption by comparison ...Optical computing and optical neural network have gained increasing attention in recent years because of their potential advantages of parallel processing at the speed of light and low power consumption by comparison with electronic computing.The optical implementation of the fundamental building blocks of a digital computer,i.e.logic gates,has been investigated extensively in the past few decades.Optical logic gate computing is an alternative approach to various analogue optical computing architectures.In this paper,the latest development of optical logic gate computing with different kinds of implementations is reviewed.Firstly,the basic concepts of analogue and digital computing with logic gates in the electronic and optical domains are introduced.And then a comprehensive summary of various optical logic gate schemes including spatial encoding of light field,semiconductor optical amplifiers(SOA),highly nonlinear fiber(HNLF),microscale and nanoscale waveguides,and photonic crystal structures is presented.To conclude,the formidable challenges in developing practical all-optical logic gates are analyzed and the prospects of the future are discussed.展开更多
Orbital angular momentum(OAM),emerging as an inherently high-dimensional property of photons,has boosted information capacity in optical communications.However,the potential of OAM in optical computing remains almost ...Orbital angular momentum(OAM),emerging as an inherently high-dimensional property of photons,has boosted information capacity in optical communications.However,the potential of OAM in optical computing remains almost unexplored.Here,we present a highly efficient optical computing protocol for complex vector convolution with the superposition of high-dimensional OAM eigenmodes.We used two cascaded spatial light modulators to prepare suitable OAM superpositions to encode two complex vectors.Then,a deep-learning strategy is devised to decode the complex OAM spectrum,thus accomplishing the optical convolution task.In our experiment,we succeed in demonstrating 7-,9-,and 11-dimensional complex vector convolutions,in which an average proximity better than 95%and a mean relative error<6%are achieved.Our present scheme can be extended to incorporate other degrees of freedom for a more versatile optical computing in the high-dimensional Hilbert space.展开更多
Computed tomography（CT） is considered the most sensitive method for the detection of intraocular foreign bodies（IOFBs）. The purpose of this study was to evaluate a new method of 3-dimentional（3D） localization of...Computed tomography（CT） is considered the most sensitive method for the detection of intraocular foreign bodies（IOFBs）. The purpose of this study was to evaluate a new method of 3-dimentional（3D） localization of IOFBs that takes advantage of the anatomical structure of the optic nerve and to assess the clinical outcomes using this new method. Twenty-two trauma patients with IOFBs or suspected IOFBs admitted to our hospital were scanned with multislice CT（MSCT） between July and December 2003. All scanning was performed with a 16-row spiral CT in axial plane using a sequential scanning protocol. During the scanning, the eyeball of the patient was kept stable and was not allowed to rotate internally or externally. Section collimation was set at 16 mm × 0.75 mm. Table feed was 12 mm. Reconstruction index was 0.75 mm. After scanning, the reconstructed images were loaded into a workstation to create the multiplanar reconstruction images with the aid of the 3D software. We compared the localization results with the operative findings. Multiplanar reconstruction images showed IOFBs in all 22 patients. IOFBs occurred in the eyeball of 14 patients, in the wall of the eyeball of 5 patients and in the posterior orbits of 3 patients. Different surgical procedures were designed according to the localization by this new method and all IOFBs were successfully removed. All of these foreign bodies were metallic and the localization of IOFB using MSCT was consistent with that found by operative findings. It was suggested that MSCT is a simple and effective imaging modality for the localization of IOFBs. In our study, we localized the IOFBs more quickly and accurately by taking advantage of the fixed position of the intraocular segment of the optic nerve, and determined the necessary surgical parameters.展开更多
The next-generation optical network is a service oriented network,which could be delivered by utilizing the generalized multiprotocol label switching(GMPLS) based control plane to realize lots of intelligent features ...The next-generation optical network is a service oriented network,which could be delivered by utilizing the generalized multiprotocol label switching(GMPLS) based control plane to realize lots of intelligent features such as rapid provisioning,automated protection and restoration(P&R),efficient resource allocation,and support for different quality of service(QoS) requirements.In this paper,we propose a novel stateful PCE-cloud(SPC)based architecture of GMPLS optical networks for cloud services.The cloud computing technologies(e.g.virtualization and parallel computing) are applied to the construction of SPC for improving the reliability and maximizing resource utilization.The functions of SPC and GMPLS based control plane are expanded according to the features of cloud services for different QoS requirements.The architecture and detailed description of the components of SPC are provided.Different potential cooperation relationships between public stateful PCE cloud(PSPC) and region stateful PCE cloud(RSPC) are investigated.Moreover,we present the policy-enabled and constraint-based routing scheme base on the cooperation of PSPC and RSPC.Simulation results for verifying the performance of routing and control plane reliability are analyzed.展开更多
We utilize three parallel reservoir computers using semiconductor lasers with optical feedback and light injection to model radar probe signals with delays.Three radar probe signals are generated by driving lasers con...We utilize three parallel reservoir computers using semiconductor lasers with optical feedback and light injection to model radar probe signals with delays.Three radar probe signals are generated by driving lasers constructed by a threeelement laser array with self-feedback.The response lasers are implemented also by a three-element lase array with both delay-time feedback and optical injection,which are utilized as nonlinear nodes to realize the reservoirs.We show that each delayed radar probe signal can be predicted well and to synchronize with its corresponding trained reservoir,even when parameter mismatches exist between the response laser array and the driving laser array.Based on this,the three synchronous probe signals are utilized for ranging to three targets,respectively,using Hilbert transform.It is demonstrated that the relative errors for ranging can be very small and less than 0.6%.Our findings show that optical reservoir computing provides an effective way for applications of target ranging.展开更多
As an optical processor,a diffractive deep neural network(D2NN)utilizes engineered diffractive surfaces designed through machine learning to perform all-optical information processing,completing its tasks at the speed...As an optical processor,a diffractive deep neural network(D2NN)utilizes engineered diffractive surfaces designed through machine learning to perform all-optical information processing,completing its tasks at the speed of light propagation through thin optical layers.With sufficient degrees of freedom,D2NNs can perform arbitrary complex-valued linear transformations using spatially coherent light.Similarly,D2NNs can also perform arbitrary linear intensity transformations with spatially incoherent illumination;however,under spatially incoherent light,these transformations are nonnegative,acting on diffraction-limited optical intensity patterns at the input field of view.Here,we expand the use of spatially incoherent D2NNs to complex-valued information processing for executing arbitrary complex-valued linear transformations using spatially incoherent light.Through simulations,we show that as the number of optimized diffractive features increases beyond a threshold dictated by the multiplication of the input and output space-bandwidth products,a spatially incoherent diffractive visual processor can approximate any complex-valued linear transformation and be used for all-optical image encryption using incoherent illumination.The findings are important for the all-optical processing of information under natural light using various forms of diffractive surface-based optical processors.展开更多
基金supported by the National Natural Science Foundation of China(Grant Nos.62192774,62105243,61925504,6201101335,62020106009,62192770,62192772,62105244,62305250,and 62322217)the Science and Technology Commission of Shanghai Municipality(Grant Nos.17JC1400800,20JC1414600,and 21JC1406100)+1 种基金the Shanghai Municipal Science and Technology Major Project(Grant No.2021SHZDZX0100)the Fundamental Research Funds for the Central Universities.
文摘Manufacturing-robust imaging systems leveraging computational optics hold immense potential for easing manufacturing constraints and enabling the development of cost-effective,high-quality imaging solutions.However,conventional approaches,which typically rely on data-driven neural networks to correct optical aberrations caused by manufacturing errors,are constrained by the lack of effective tolerance analysis methods for quantitatively evaluating manufacturing error boundaries.This limitation is crucial for further relaxing manufacturing constraints and providing practical guidance for fabrication.We propose a physics-informed design paradigm for manufacturing-robust imaging systems with computational optics,integrating a physics-informed tolerance analysis methodology for evaluating manufacturing error boundaries and a physics-informed neural network for image reconstruction.With this approach,we achieve a manufacturing-robust imaging system based on an off-axis three-mirror freeform all-aluminum design,delivering a modulation transfer function exceeding 0.34 at the Nyquist frequency(72 lp/mm)in simulation.Notably,this system requires a manufacturing precision of only 0.5λin root mean square(RMS),representing a remarkable 25-fold relaxation compared with the conventional requirement of 0.02λin RMS.Experimental validation further confirmed that the manufacturing-robust imaging system maintains excellent performance in diverse indoor and outdoor environments.Our proposed method paves the way for achieving high-quality imaging without the necessity of high manufacturing precision,enabling practical solutions that are more cost-effective and time-efficient.
基金National Natural Science Foundation of China(No.12574332)the Space Optoelectronic Measurement and Perception Lab.,Beijing Institute of Control Engineering(No.LabSOMP-2023-10)Major Science and Technology Innovation Program of Xianyang City(No.L2024-ZDKJ-ZDCGZH-0021)。
文摘Fourier Ptychographic Microscopy(FPM)is a high-throughput computational optical imaging technology reported in 2013.It effectively breaks through the trade-off between high-resolution imaging and wide-field imaging.In recent years,it has been found that FPM is not only a tool to break through the trade-off between field of view and spatial resolution,but also a paradigm to break through those trade-off problems,thus attracting extensive attention.Compared with previous reviews,this review does not introduce its concept,basic principles,optical system and series of applications once again,but focuses on elaborating the three major difficulties faced by FPM technology in the process from“looking good”in the laboratory to“working well”in practical applications:mismatch between numerical model and physical reality,long reconstruction time and high computing power demand,and lack of multi-modal expansion.It introduces how to achieve key technological innovations in FPM through the dual drive of Artificial Intelligence(AI)and physics,including intelligent reconstruction algorithms introducing machine learning concepts,optical-algorithm co-design,fusion of frequency domain extrapolation methods and generative adversarial networks,multi-modal imaging schemes and data fusion enhancement,etc.,gradually solving the difficulties of FPM technology.Conversely,this review deeply considers the unique value of FPM technology in potentially feeding back to the development of“AI+optics”,such as providing AI benchmark tests under physical constraints,inspirations for the balance of computing power and bandwidth in miniaturized intelligent microscopes,and photoelectric hybrid architectures.Finally,it introduces the industrialization path and frontier directions of FPM technology,pointing out that with the promotion of the dual drive of AI and physics,it will generate a large number of industrial application case,and looks forward to the possibilities of future application scenarios and expansions,for instance,body fluid biopsy and point-of-care testing at the grassroots level represent the expansion of the growth market.
基金supported by the National Science Foundation(Grant Nos.NSF-ECCS-2127235 and EFRI-BRAID-2223495)Part of this work was conducted at the Washington Nanofabrication Facility/Molecular Analysis Facility,a National Nanotechnology Coordinated Infrastructure(NNCI)site at the University of Washington with partial support from the National Science Foundation(Grant Nos.NNCI-1542101 and NNCI-2025489).
文摘Optical and hybrid convolutional neural networks(CNNs)recently have become of increasing interest to achieve low-latency,low-power image classification,and computer-vision tasks.However,implementing optical nonlinearity is challenging,and omitting the nonlinear layers in a standard CNN comes with a significant reduction in accuracy.We use knowledge distillation to compress modified AlexNet to a single linear convolutional layer and an electronic backend(two fully connected layers).We obtain comparable performance with a purely electronic CNN with five convolutional layers and three fully connected layers.We implement the convolution optically via engineering the point spread function of an inverse-designed meta-optic.Using this hybrid approach,we estimate a reduction in multiply-accumulate operations from 17M in a conventional electronic modified AlexNet to only 86 K in the hybrid compressed network enabled by the optical front end.This constitutes over 2 orders of magnitude of reduction in latency and power consumption.Furthermore,we experimentally demonstrate that the classification accuracy of the system exceeds 93%on the MNIST dataset of handwritten digits.
基金supported by the National Natural Science Foundation of China(Grant Nos.62175050 and U2341245)the Fundamental Research Funds for the Central Universities(Grant No.HIT.OCEF.2024054).
文摘Diffractive optical neural networks(DONNs)have exhibited the advantages of parallelization,high speed,and low consumption.However,the existing DONNs based on free-space diffractive optical elements are bulky and unsteady.In this study,we propose a planar-waveguide integrated diffractive neural network chip architecture.The three diffractive layers are engraved on the same side of a quartz wafer.The three-layer chip is designed with 32-mm3 processing space and enables a computing speed of 3.1×109 Tera operations per second.The results show that the proposed chip achieves 73.4%experimental accuracy for the Modified National Institute of Standards and Technology database while showing the system’s robustness in a cycle test.The consistency of experiments is 88.6%,and the arithmetic mean standard deviation of the results is~4.7%.The proposed chip architecture can potentially revolutionize high-resolution optical processing tasks with high robustness.
文摘Computational optical imaging is an interdisciplinary subject integrating optics, mathematics, and information technology. It introduces information processing into optical imaging and combines it with intelligent computing, subverting the imaging mechanism of traditional optical imaging which only relies on orderly information transmission. To meet the high-precision requirements of traditional optical imaging for optical processing and adjustment, as well as to solve its problems of being sensitive to gravity and temperature in use, we establish an optical imaging system model from the perspective of computational optical imaging and studies how to design and solve the imaging consistency problem of optical system under the influence of gravity, thermal effect, stress, and other external environment to build a high robustness optical system. The results show that the high robustness interval of the optical system exists and can effectively reduce the sensitivity of the optical system to the disturbance of each link, thus realizing the high robustness of optical imaging.
基金supported by the National Natural Science Foundation of China (Grant No.62135009)the Beijing Municipal Science and Technology Commission,Administrative Commission of Zhongguancun Science Park (Grant No.Z221100005322010).
文摘On-chip diffractive optical neural networks(DONNs)bring the advantages of parallel processing and low energy consumption.However,an accurate representation of the optical field’s evolution in the structure cannot be provided using the previous diffraction-based analysis method.Moreover,the loss caused by the open boundaries poses challenges to applications.A multimode DONN architecture based on a more precise eigenmode analysis method is proposed.We have constructed a universal library of input,output,and metaline structures utilizing this method,and realized a multimode DONN composed of the structures from the library.On the designed multimode DONNs with only one layer of the metaline,the classification task of an Iris plants dataset is verified with an accuracy of 90%on the blind test dataset,and the performance of the one-bit binary adder task is also validated.Compared to the previous architectures,the multimode DONN exhibits a more compact design and higher energy efficiency.
文摘In a single-pixel fast imaging setup,the data collected by the single-pixel detector needs to be processed by a computer,but the speed of the latter will affect the image reconstruction time.Here we propose two kinds of setups which are able to transform non-visible into visible light imaging,wherein their computing process is replaced by a camera integration mode.The image captured by the camera has a low contrast,so here we present an algorithm that can realize a high quality image in near-infrared to visible cross-waveband imaging.The scheme is verified both by simulation and in actual experiments.The setups demonstrate the great potential for single-pixel imaging and high-speed cross-waveband imaging for future practical applications.
基金supported by the National Natural Science Foundation of China(Grant Nos.12064025,12264028,12364045,and 12304420)the Natural Science Foundation of Jiangxi Province(Grant Nos.20212ACB202006,20232BAB201040,and 20232BAB211025)+3 种基金the Shanghai Pujiang Program(Grant No.22PJ1402900)the Australian Research Council Discovery Project(Grant No.DP200101353)the Interdisciplinary Innovation Fund of Nanchang University(Grant No.2019-9166-27060003)the China Scholarship Council(Grant No.202008420045).
文摘The ultimate goal of artificial intelligence(AI)is to mimic the human brain to perform decision-making and control directly from high-dimensional sensory input.Diffractive optical networks(DONs)provide a promising solution for implementing AI with high speed and low power-consumption.Most reported DONs focus on tasks that do not involve environmental interaction,such as object recognition and image classification.By contrast,the networks capable of decision-making and control have not been developed.Here,we propose using deep reinforcement learning to implement DONs that imitate human-level decisionmaking and control capability.Such networks,which take advantage of a residual architecture,allow finding optimal control policies through interaction with the environment and can be readily implemented with existing optical devices.The superior performance is verified using three types of classic games:tic-tac-toe,Super Mario Bros.,and Car Racing.Finally,we present an experimental demonstration of playing tic-tac-toe using the network based on a spatial light modulator.Our work represents a solid step forward in advancing DONs,which promises a fundamental shift from simple recognition or classification tasks to the high-level sensory capability of AI.It may find exciting applications in autonomous driving,intelligent robots,and intelligent manufacturing.
基金supported by the Key Project of Chongqing Natural Science Foundation Joint Fund[CSTB2023NSCQ-LZX0103,(G.Z.)]Chongqing Natural Science Foundation[CSTB2024NSCQ-MSX0012,(C.L.)]+1 种基金Fundamental Research Funds for the Central Universities[SWUZLPY03,(G.Z.)]Fundamental Research Funds for the Central Universities[Swu020019,(G.Z.):SWU-XDJH202319,(G.Z.)1].
文摘With the advancement of artificial intelligence,optic in-sensing reservoir computing based on emerging semiconductor devices is high desirable for real-time analog signal processing.Here,we disclose a flexible optomemristor based on C_(27)H_(30)O_(15)/FeOx heterostructure that presents a highly sensitive to the light stimuli and artificial optic synaptic features such as short-and long-term plasticity(STP and LTP),enabling the developed optomemristor to implement complex analogy signal processing through building a real-physical dynamic-based in-sensing reservoir computing algorithm and yielding an accuracy of 94.88%for speech recognition.The charge trapping and detrapping mediated by the optic active layer of C_(27)H_(30)O_(15) that is extracted from the lotus flower is response for the positive photoconductance memory in the prepared optomemristor.This work provides a feasible organic−inorganic heterostructure as well as an optic in-sensing vision computing for an advanced optic computing system in future complex signal processing.
基金supported by the National Natural Science Foundation of China(Nos.62131003,62322502,62088101)the Guangdong Province Key Laboratory of Intelligent Detection in Complex Environment of Aerospace,Land and Sea(No.2022KSYS016).
文摘Computational optics introduces computation into optics and consequently helps overcome traditional optical limitations such as low sensing dimension,low light throughput,low resolution,and so on.The combination of optical encoding and computational decoding offers enhanced imaging and sensing capabilities with diverse applications in biomedicine,astronomy,agriculture,etc.With the great advance of artificial intelligence in the last decade,deep learning has further boosted computational optics with higher precision and efficiency.Recently,there developed an end-to-end joint optimization technique that digitally twins optical encoding to neural network layers,and then facilitates simultaneous optimization with the decoding process.This framework offers effective performance enhancement over conventional techniques.However,the reverse physical twinning from optimized encoding parameters to practical modulation elements faces a serious challenge,due to the discrepant gap in such as bit depth,numerical range,and stability.In this regard,this review explores various optical modulation elements across spatial,phase,and spectral dimensions in the digital twin model for joint encoding-decoding optimization.Our analysis offers constructive guidance for finding the most appropriate modulation element in diverse imaging and sensing tasks concerning various requirements of precision,speed,and robustness.The review may help tackle the above twinning challenge and pave the way for next-generation computational optics.
基金supported by the National Natural Science Foundation of China(61927802,61722209,and 61805145)the Beijing Municipal Science and Technology Commission(Z181100003118014)+3 种基金the National Key Research and Development Program of China(2020AAA0130000)the support from the National Postdoctoral Program for Innovative TalentShuimu Tsinghua Scholar Programthe support from the Hong Kong Research Grants Council(16306220)。
文摘The rapid development of artificial intelligence(AI)facilitates various applications from all areas but also poses great challenges in its hardware implementation in terms of speed and energy because of the explosive growth of data.Optical computing provides a distinctive perspective to address this bottleneck by harnessing the unique properties of photons including broad bandwidth,low latency,and high energy efficiency.In this review,we introduce the latest developments of optical computing for different AI models,including feedforward neural networks,reservoir computing,and spiking neural networks(SNNs).Recent progress in integrated photonic devices,combined with the rise of AI,provides a great opportunity for the renaissance of optical computing in practical applications.This effort requires multidisciplinary efforts from a broad community.This review provides an overview of the state-of-the-art accomplishments in recent years,discusses the availability of current technologies,and points out various remaining challenges in different aspects to push the frontier.We anticipate that the era of large-scale integrated photonics processors will soon arrive for practical AI applications in the form of hybrid optoelectronic frameworks.
基金This work was supported in part by the National Outstanding Youth Science Fund Project of National Natural Science Foundation of China(62022062)the National Natural Science Foundation of China(61974177,61674119)the Fundamental Research Funds for the Central Universities.
文摘The explosive growth of data and information has motivated various emerging non-von Neumann computational approaches in the More-than-Moore era.Photonics neuromorphic computing has attracted lots of attention due to the fascinating advantages such as high speed,wide bandwidth,and massive parallelism.Here,we offer a review on the optical neural computing in our research groups at the device and system levels.The photonics neuron and photonics synapse plasticity are presented.In addition,we introduce several optical neural computing architectures and algorithms including photonic spiking neural network,photonic convolutional neural network,photonic matrix computation,photonic reservoir computing,and photonic reinforcement learning.Finally,we summarize the major challenges faced by photonic neuromorphic computing,and propose promising solutions and perspectives.
基金support from the National Natural Science Foundation of China(Grant Nos.12174097 and 12304321)the Natural Science Foundation of Hunan Province(Grant Nos.2021JJ10008 and 2023JJ40202)the Research Foundation of Education Bureau of Hunan Province(Grant No.22B0871).
文摘Object identification and three-dimensional reconstruction techniques are always attractive research interests in machine vision,virtual reality,augmented reality,and biomedical engineering.Optical computing metasurface,as a two-dimensional artificial design component,has displayed the supernormal character of controlling phase,amplitude,polarization,and frequency distributions of the light beam,capable of performing mathematical operations on the input light field.Here,we propose and demonstrate an all-optical object identification technique based on optical computing metasurface,and apply it to 3D reconstruction.Unlike traditional mechanisms,this scheme reduces memory consumption in the processing of the contour surface extraction.The identification and reconstruction of experimental results from high-contrast and low-contrast objects agree well with the real objects.The exploration of the all-optical object identification and 3D reconstruction techniques provides potential applications of high efficiencies,low consumption,and compact systems.
基金supported by the National Key Research and Development Program of China(Grants No.2021YFA1401500)the National Natural Science Foundation of China(12022416)+3 种基金the Department of Natural Resources of Guangdong Province(No.GDNRC[2022]22)Department of Science and Technology of Guangdong Province(No.2021A0505080002)Intelligent Laser Basic Research Laboratory(No.PCL2021A14-B1)the Hong Kong Research Grants Council(16306220).
文摘Optical computing and optical neural network have gained increasing attention in recent years because of their potential advantages of parallel processing at the speed of light and low power consumption by comparison with electronic computing.The optical implementation of the fundamental building blocks of a digital computer,i.e.logic gates,has been investigated extensively in the past few decades.Optical logic gate computing is an alternative approach to various analogue optical computing architectures.In this paper,the latest development of optical logic gate computing with different kinds of implementations is reviewed.Firstly,the basic concepts of analogue and digital computing with logic gates in the electronic and optical domains are introduced.And then a comprehensive summary of various optical logic gate schemes including spatial encoding of light field,semiconductor optical amplifiers(SOA),highly nonlinear fiber(HNLF),microscale and nanoscale waveguides,and photonic crystal structures is presented.To conclude,the formidable challenges in developing practical all-optical logic gates are analyzed and the prospects of the future are discussed.
基金supported by the National Natural Science Foundation of China(Grant Nos.12034016,61975169,and 11904303)the Youth Innovation Fund of Xiamen(Grant No.3502Z20206045)+2 种基金the Fundamental Research Funds for the Central Universities at Xiamen University(Grant Nos.20720200074 and 20720220030)the Natural Science Foundation of Fujian Province of China(Grant No.2021J02002)and for Distinguished Young Scientists(Grant No.2015J06002)the Program for New Century Excellent Talents in University of China(Grant No.NCET-13-0495).
文摘Orbital angular momentum(OAM),emerging as an inherently high-dimensional property of photons,has boosted information capacity in optical communications.However,the potential of OAM in optical computing remains almost unexplored.Here,we present a highly efficient optical computing protocol for complex vector convolution with the superposition of high-dimensional OAM eigenmodes.We used two cascaded spatial light modulators to prepare suitable OAM superpositions to encode two complex vectors.Then,a deep-learning strategy is devised to decode the complex OAM spectrum,thus accomplishing the optical convolution task.In our experiment,we succeed in demonstrating 7-,9-,and 11-dimensional complex vector convolutions,in which an average proximity better than 95%and a mean relative error<6%are achieved.Our present scheme can be extended to incorporate other degrees of freedom for a more versatile optical computing in the high-dimensional Hilbert space.
文摘Computed tomography（CT） is considered the most sensitive method for the detection of intraocular foreign bodies（IOFBs）. The purpose of this study was to evaluate a new method of 3-dimentional（3D） localization of IOFBs that takes advantage of the anatomical structure of the optic nerve and to assess the clinical outcomes using this new method. Twenty-two trauma patients with IOFBs or suspected IOFBs admitted to our hospital were scanned with multislice CT（MSCT） between July and December 2003. All scanning was performed with a 16-row spiral CT in axial plane using a sequential scanning protocol. During the scanning, the eyeball of the patient was kept stable and was not allowed to rotate internally or externally. Section collimation was set at 16 mm × 0.75 mm. Table feed was 12 mm. Reconstruction index was 0.75 mm. After scanning, the reconstructed images were loaded into a workstation to create the multiplanar reconstruction images with the aid of the 3D software. We compared the localization results with the operative findings. Multiplanar reconstruction images showed IOFBs in all 22 patients. IOFBs occurred in the eyeball of 14 patients, in the wall of the eyeball of 5 patients and in the posterior orbits of 3 patients. Different surgical procedures were designed according to the localization by this new method and all IOFBs were successfully removed. All of these foreign bodies were metallic and the localization of IOFB using MSCT was consistent with that found by operative findings. It was suggested that MSCT is a simple and effective imaging modality for the localization of IOFBs. In our study, we localized the IOFBs more quickly and accurately by taking advantage of the fixed position of the intraocular segment of the optic nerve, and determined the necessary surgical parameters.
基金supported by National Natural Science Foundation of China(No.61571061)Innovative Research Fund of Beijing University of Posts and Telecommunications (2015RC16)
文摘The next-generation optical network is a service oriented network,which could be delivered by utilizing the generalized multiprotocol label switching(GMPLS) based control plane to realize lots of intelligent features such as rapid provisioning,automated protection and restoration(P&R),efficient resource allocation,and support for different quality of service(QoS) requirements.In this paper,we propose a novel stateful PCE-cloud(SPC)based architecture of GMPLS optical networks for cloud services.The cloud computing technologies(e.g.virtualization and parallel computing) are applied to the construction of SPC for improving the reliability and maximizing resource utilization.The functions of SPC and GMPLS based control plane are expanded according to the features of cloud services for different QoS requirements.The architecture and detailed description of the components of SPC are provided.Different potential cooperation relationships between public stateful PCE cloud(PSPC) and region stateful PCE cloud(RSPC) are investigated.Moreover,we present the policy-enabled and constraint-based routing scheme base on the cooperation of PSPC and RSPC.Simulation results for verifying the performance of routing and control plane reliability are analyzed.
基金the National Natural Science Foundation of China(Grant No.62075168)Guang Dong Basic and Applied Basic Research Foundation(Grant No.2020A1515011088)Special Project in Key Fields of Guangdong Provincial Department of Education of China(Grant No.2020ZDZX3052 and 2019KZDZX1025)。
文摘We utilize three parallel reservoir computers using semiconductor lasers with optical feedback and light injection to model radar probe signals with delays.Three radar probe signals are generated by driving lasers constructed by a threeelement laser array with self-feedback.The response lasers are implemented also by a three-element lase array with both delay-time feedback and optical injection,which are utilized as nonlinear nodes to realize the reservoirs.We show that each delayed radar probe signal can be predicted well and to synchronize with its corresponding trained reservoir,even when parameter mismatches exist between the response laser array and the driving laser array.Based on this,the three synchronous probe signals are utilized for ranging to three targets,respectively,using Hilbert transform.It is demonstrated that the relative errors for ranging can be very small and less than 0.6%.Our findings show that optical reservoir computing provides an effective way for applications of target ranging.
基金support of the U.S.Department of Energy (DOE),Office of Basic Energy Sciences,Division of Materials Sciences and Engineering under Award#DE-SC0023088.
文摘As an optical processor,a diffractive deep neural network(D2NN)utilizes engineered diffractive surfaces designed through machine learning to perform all-optical information processing,completing its tasks at the speed of light propagation through thin optical layers.With sufficient degrees of freedom,D2NNs can perform arbitrary complex-valued linear transformations using spatially coherent light.Similarly,D2NNs can also perform arbitrary linear intensity transformations with spatially incoherent illumination;however,under spatially incoherent light,these transformations are nonnegative,acting on diffraction-limited optical intensity patterns at the input field of view.Here,we expand the use of spatially incoherent D2NNs to complex-valued information processing for executing arbitrary complex-valued linear transformations using spatially incoherent light.Through simulations,we show that as the number of optimized diffractive features increases beyond a threshold dictated by the multiplication of the input and output space-bandwidth products,a spatially incoherent diffractive visual processor can approximate any complex-valued linear transformation and be used for all-optical image encryption using incoherent illumination.The findings are important for the all-optical processing of information under natural light using various forms of diffractive surface-based optical processors.
