Diffractive optical neural networks(DONNs)have exhibited the advantages of parallelization,high speed,and low consumption.However,the existing DONNs based on free-space diffractive optical elements are bulky and unste...Diffractive optical neural networks(DONNs)have exhibited the advantages of parallelization,high speed,and low consumption.However,the existing DONNs based on free-space diffractive optical elements are bulky and unsteady.In this study,we propose a planar-waveguide integrated diffractive neural network chip architecture.The three diffractive layers are engraved on the same side of a quartz wafer.The three-layer chip is designed with 32-mm3 processing space and enables a computing speed of 3.1×109 Tera operations per second.The results show that the proposed chip achieves 73.4%experimental accuracy for the Modified National Institute of Standards and Technology database while showing the system’s robustness in a cycle test.The consistency of experiments is 88.6%,and the arithmetic mean standard deviation of the results is~4.7%.The proposed chip architecture can potentially revolutionize high-resolution optical processing tasks with high robustness.展开更多
The vector vortex beam(VVB)has attracted significant attention due to its intrinsic diversity of information and has found great applications in both classical and quantum communications.However,a VVB is unavoidably a...The vector vortex beam(VVB)has attracted significant attention due to its intrinsic diversity of information and has found great applications in both classical and quantum communications.However,a VVB is unavoidably affected by atmospheric turbulence(AT)when it propagates through the free-space optical communication environment,which results in detection errors at the receiver.In this paper,we propose a VVB classification scheme to detect VVBs with continuously changing polarization states under AT,where a diffractive deep neural network(DDNN)is designed and trained to classify the intensity distribution of the input distorted VVBs,and the horizontal direction of polarization of the input distorted beam is adopted as the feature for the classification through the DDNN.The numerical simulations and experimental results demonstrate that the proposed scheme has high accuracy in classification tasks.The energy distribution percentage remains above 95%from weak to medium AT,and the classification accuracy can remain above 95%for various strengths of turbulence.It has a faster convergence and better accuracy than that based on a convolutional neural network.展开更多
On-chip diffractive optical neural networks(DONNs)bring the advantages of parallel processing and low energy consumption.However,an accurate representation of the optical field’s evolution in the structure cannot be ...On-chip diffractive optical neural networks(DONNs)bring the advantages of parallel processing and low energy consumption.However,an accurate representation of the optical field’s evolution in the structure cannot be provided using the previous diffraction-based analysis method.Moreover,the loss caused by the open boundaries poses challenges to applications.A multimode DONN architecture based on a more precise eigenmode analysis method is proposed.We have constructed a universal library of input,output,and metaline structures utilizing this method,and realized a multimode DONN composed of the structures from the library.On the designed multimode DONNs with only one layer of the metaline,the classification task of an Iris plants dataset is verified with an accuracy of 90%on the blind test dataset,and the performance of the one-bit binary adder task is also validated.Compared to the previous architectures,the multimode DONN exhibits a more compact design and higher energy efficiency.展开更多
Orbital angular momentum(OAM)has the characteristics of mutual orthogonality between modes,and has been applied to underwater wireless optical communication(UWOC)systems to increase the channel capacity.In this work,w...Orbital angular momentum(OAM)has the characteristics of mutual orthogonality between modes,and has been applied to underwater wireless optical communication(UWOC)systems to increase the channel capacity.In this work,we propose a diffractive deep neural network(DDNN)based OAM mode recognition scheme,where the DDNN is trained to capture the features of the intensity distribution of the OAM modes and output the corresponding azimuthal indices and radial indices.The results show that the proposed scheme can recognize the azimuthal indices and radial indices of the OAM modes accurately and quickly.In addition,the proposed scheme can resist weak oceanic turbulence(OT),and exhibit excellent ability to recognize OAM modes in a strong OT environment.The DDNN-based OAM mode recognition scheme has potential applications in UWOC systems.展开更多
Efficiently tracking and imaging interested moving targets is crucial across various applications,from autonomous systems to surveillance.However,persistent challenges remain in various fields,including environmental ...Efficiently tracking and imaging interested moving targets is crucial across various applications,from autonomous systems to surveillance.However,persistent challenges remain in various fields,including environmental intricacies,limitations in perceptual technologies,and privacy considerations.We present a teacher-student learning model,the generative adversarial network(GAN)-guided diffractive neural network(DNN),which performs visual tracking and imaging of the interested moving target.The GAN,as a teacher model,empowers efficient acquisition of the skill to differentiate the specific target of interest in the domains of visual tracking and imaging.The DNN-based student model learns to master the skill to differentiate the interested target from the GAN.The process of obtaining a GAN-guided DNN starts with capturing moving objects effectively using an event camera with high temporal resolution and low latency.Then,the generative power of GAN is utilized to generate data with position-tracking capability for the interested moving target,subsequently serving as labels to the training of the DNN.The DNN learns to image the target during training while retaining the target’s positional information.Our experimental demonstration highlights the efficacy of the GAN-guided DNN in visual tracking and imaging of the interested moving target.We expect the GAN-guided DNN can significantly enhance autonomous systems and surveillance.展开更多
Optical deep learning based on diffractive optical elements offers unique advantages for parallel processing,computational speed,and power efficiency.One landmark method is the diffractive deep neural network(D^(2) NN...Optical deep learning based on diffractive optical elements offers unique advantages for parallel processing,computational speed,and power efficiency.One landmark method is the diffractive deep neural network(D^(2) NN)based on three-dimensional printing technology operated in the terahertz spectral range.Since the terahertz bandwidth involves limited interparticle coupling and material losses,this paper extends D^(2) NN to visible wavelengths.A general theory including a revised formula is proposed to solve any contradictions between wavelength,neuron size,and fabrication limitations.A novel visible light D^(2) NN classifier is used to recognize unchanged targets(handwritten digits ranging from 0 to 9)and targets that have been changed(i.e.,targets that have been covered or altered)at a visible wavelength of 632.8 nm.The obtained experimental classification accuracy(84%)and numerical classification accuracy(91.57%)quantify the match between the theoretical design and fabricated system performance.The presented framework can be used to apply a D^(2) NN to various practical applications and design other new applications.展开更多
As an optical processor,a diffractive deep neural network(D2NN)utilizes engineered diffractive surfaces designed through machine learning to perform all-optical information processing,completing its tasks at the speed...As an optical processor,a diffractive deep neural network(D2NN)utilizes engineered diffractive surfaces designed through machine learning to perform all-optical information processing,completing its tasks at the speed of light propagation through thin optical layers.With sufficient degrees of freedom,D2NNs can perform arbitrary complex-valued linear transformations using spatially coherent light.Similarly,D2NNs can also perform arbitrary linear intensity transformations with spatially incoherent illumination;however,under spatially incoherent light,these transformations are nonnegative,acting on diffraction-limited optical intensity patterns at the input field of view.Here,we expand the use of spatially incoherent D2NNs to complex-valued information processing for executing arbitrary complex-valued linear transformations using spatially incoherent light.Through simulations,we show that as the number of optimized diffractive features increases beyond a threshold dictated by the multiplication of the input and output space-bandwidth products,a spatially incoherent diffractive visual processor can approximate any complex-valued linear transformation and be used for all-optical image encryption using incoherent illumination.The findings are important for the all-optical processing of information under natural light using various forms of diffractive surface-based optical processors.展开更多
Free-space diffractive neural networks(DNNs)have been an intense research topic in machine learning for image recognition and encryption due to their high speed,lower power consumption,and high neuron density.Recent a...Free-space diffractive neural networks(DNNs)have been an intense research topic in machine learning for image recognition and encryption due to their high speed,lower power consumption,and high neuron density.Recent advances in DNNs have highlighted the need for smaller device footprints and the shift toward visible wavelengths.However,DNNs fabricated by electron beam lithography,are not suitable for microscopic imaging applications due to their large sizes,and DNNs fabricated by two-photon nanolithography with cylindrical neurons are not optimal for visible wavelengths,as the highorder diffraction could induce low diffraction efficiency.In this paper,we demonstrate that cubical diffraction neurons are more efficient diffraction elements for DNNs compared with cylindrical neurons.Based on the theoretical analysis of the relationship between the detector area sizes and classification accuracy,we reduced the size of DNNs operating at the wavelength of 532 nm for handwritten digit classification to micrometer scale by two-photon nanolithography.The DNNs with cubical neurons demonstrated an experimental classification accuracy(89.3%)for single-layer DNN,and 83.3%for two-layer DNN with device sizes similar to that of biological cells(about 100μm×100μm).Our results paved the pathway to integrate 3D micrometer-scale DNNs with microscopic imaging systems for biological imaging and cell recognition.展开更多
We propose a design method for a diffractive neural network(DNN)for imaging through scattering media,offering robustness against the spatial coherence of illumination,scattering strength,and scattering dynamics.Most t...We propose a design method for a diffractive neural network(DNN)for imaging through scattering media,offering robustness against the spatial coherence of illumination,scattering strength,and scattering dynamics.Most techniques for imaging through scattering media are time-consuming and/or tailored to specific optical conditions.The DNN,composed of layers of diffractive optical elements(DOEs),optically reproduces the intensity distributions of objects behind scattering media without any computational processing.Datasets with randomized optical parameters are provided during the training process to achieve this robustness.We demonstrate the proposed method through numerical calculations and show its promising capability for DOE design.Our study paves the way for unifying and generalizing techniques for imaging through scattering media,which are currently fragmented by specific scenarios,enabling highly flexible imaging independent of optical conditions.展开更多
Optical neural networks have emerged as feasible alternatives to their electronic counterparts,offering significant benefits such as low power consumption,low latency,and high parallelism.However,the realization of ul...Optical neural networks have emerged as feasible alternatives to their electronic counterparts,offering significant benefits such as low power consumption,low latency,and high parallelism.However,the realization of ultra-compact nonlinear deep neural networks and multi-thread processing remain crucial challenges for optical computing.We present a monolithically integrated all-optical nonlinear diffractive deep neural network(AON-D^(2) NN)chip for the first time.The all-optical nonlinear activation function is implemented using germanium microstructures,which provide low loss and are compatible with the standard silicon photonics fabrication process.Assisted by the germanium activation function,the classification accuracy is improved by 9.1%for four-classification tasks.In addition,the chip's reconfigurability enables multi-task learning in situ via an innovative cross-training algorithm,yielding two task-specific inference results with accuracies of 95%and 96%,respectively.Furthermore,leveraging the wavelength-dependent response of the chip,the multi-thread nonlinear optical neural network is implemented for the first time,capable of handling two different tasks in parallel.The proposed AON-D^(2)NN contains three hidden layers with a footprint of only 0.73 mm^(2).It can achieve ultra-low latency(172 ps),paving the path for realizing high-performance optical neural networks.展开更多
Orbital angular momentum(OAM)modes provide an additional orthogonal physical dimension,offering transformative potential for enhancing optical communication capacity.Despite significant progress in mode multiplexing,t...Orbital angular momentum(OAM)modes provide an additional orthogonal physical dimension,offering transformative potential for enhancing optical communication capacity.Despite significant progress in mode multiplexing,the development of robust communication networks faces persistent challenges,particularly in effectively routing and controlling these multiplexed channels among network nodes.To tackle these dilemmas,we propose a rotatable diffractive neural network(R-DNN)strategy and demonstrate its capability for port-controllable OAM mode routing.By leveraging the correlation between the orthogonal evolution of OAM modes in free space and phase modulations during propagation,the R-DNN precisely shapes the spatial evolution of mode fields through multiple rotatable phase layers,enabling efficient routing to specific output ports.This approach exploits the interaction of secondary wavelets with the relative states of the rotatable layers,allowing on-demand control of mode evolution paths and enhancing routing flexibility.As a proof of concept,we developed a tri-functional router that successfully directs three OAM modes to individually controllable output ports.This router achieves an average intermode crosstalk of less than−16.4 dB across three functional states,one-dimensional,two-dimensional,and cross-connected switching,while supporting the routing of 5.85 Tbit/s quadrature phase-shift keying signals.These results highlight the R-DNN’s effectiveness in achieving precise and controllable OAM mode manipulation,paving the way for advanced applications in mode-multiplexed communication networks and beyond.展开更多
As a successful case of combining deep learning with photonics,the research on optical machine learning has recently undergone rapid development.Among various optical classification frameworks,diffractive networks hav...As a successful case of combining deep learning with photonics,the research on optical machine learning has recently undergone rapid development.Among various optical classification frameworks,diffractive networks have been shown to have unique advantages in all-optical reasoning.As an important property of light,the orbital angular momentum(OAM)of light shows orthogonality and mode-infinity,which can enhance the ability of parallel classification in information processing.However,there have been few all-optical diffractive networks under the OAM mode encoding.Here,we report a strategy of OAM-encoded diffractive deep neural network(OAM-encoded D2NN)that encodes the spatial information of objects into the OAM spectrum of the diffracted light to perform all-optical object classification.We demonstrated three different OAM-encoded D2NNs to realize(1)single detector OAM-encoded D2NN for single task classification,(2)single detector OAM-encoded D2NN for multitask classification,and(3)multidetector OAM-encoded D2NN for repeatable multitask classification.We provide a feasible way to improve the performance of all-optical object classification and open up promising research directions for D2NN by proposing OAMencoded D2NN.展开更多
Optical neural networks have significant advantages in terms of power consumption,parallelism,and high computing speed,which has intrigued extensive attention in both academic and engineering communities.It has been c...Optical neural networks have significant advantages in terms of power consumption,parallelism,and high computing speed,which has intrigued extensive attention in both academic and engineering communities.It has been considered as one of the powerful tools in promoting the fields of imaging processing and object recognition.However,the existing optical system architecture cannot be reconstructed to the realization of multi-functional artificial intelligence systems simultaneously.To push the development of this issue,we propose the pluggable diffractive neural networks(P-DNN),a general paradigm resorting to the cascaded metasurfaces,which can be applied to recognize various tasks by switching internal plug-ins.As the proof-of-principle,the recognition functions of six types of handwritten digits and six types of fashions are numerical simulated and experimental demonstrated at near-infrared regimes.Encouragingly,the proposed paradigm not only improves the flexibility of the optical neural networks but paves the new route for achieving high-speed,low-power and versatile artificial intelligence systems.展开更多
Research in the ocean places high demands on chips'robustness,speed,and energy consumption.Diffractive neural networks(DNNs)enable direct optical image processing at light speed,with great potential for underwater...Research in the ocean places high demands on chips'robustness,speed,and energy consumption.Diffractive neural networks(DNNs)enable direct optical image processing at light speed,with great potential for underwater applications.Here,we experimentally demonstrate a compact DNN chip capable of operating directly in both water and air by multiobjective training and initial training value optimization.The two layers of DNNs are integrated on the two surfaces of a quartz plate,respectively.The chip achieved high accuracies above 90%in recognition tasks for handwritten digits and fashion products.The architecture and material ensure the chip's high stability for long-term underwater use.展开更多
Unidirectional imagers form images of input objects only in one direction,e.g.,from field-of-view(FOV)A to FOV B,while blocking the image formation in the reverse direction,from FOV B to FOV A.Here,we report unidirect...Unidirectional imagers form images of input objects only in one direction,e.g.,from field-of-view(FOV)A to FOV B,while blocking the image formation in the reverse direction,from FOV B to FOV A.Here,we report unidirectional imaging under spatially partially coherent light and demonstrate high-quality imaging only in the forward direction(A→B)with high power efficiency while distorting the image formation in the backward direction(B→A)along with low power efficiency.Our reciprocal design features a set of spatially engineered linear diffractive layers that are statistically optimized for partially coherent illumination with a given phase correlation length.Our analyses reveal that when illuminated by a partially coherent beam with a correlation length of≥∼1.5λ,whereλis the wavelength of light,diffractive unidirectional imagers achieve robust performance,exhibiting asymmetric imaging performance between the forward and backward directions—as desired.A partially coherent unidirectional imager designed with a smaller correlation length of<1.5λstill supports unidirectional image transmission but with a reduced figure of merit.These partially coherent diffractive unidirectional imagers are compact(axially spanning<75λ),polarization-independent,and compatible with various types of illumination sources,making them well-suited for applications in asymmetric visual information processing and communication.展开更多
Vector structured beams(VSBs)offer infinite eigenstates and open up new possibilities for highcapacity optical and quantum communications by the multiplexing of the states.Therefore,the sorting and measuring of VSBs a...Vector structured beams(VSBs)offer infinite eigenstates and open up new possibilities for highcapacity optical and quantum communications by the multiplexing of the states.Therefore,the sorting and measuring of VSBs are extremely important.However,the efficient manipulations of a large number of VSBs have simultaneously remained challenging up to now,especially in integrated optical systems.Here,we propose a compact spin-multiplexed diffractive metasurface capable of continuously sorting and detecting arbitrary VSBs through spatial intensity separation.By introducing a diffractive optical neural network with cascaded metasurface systems,we demonstrate arbitrary VSBs sorters that can simultaneously identify Laguerre–Gaussian modes(l=−4 to 4,p=1 to 4),Hermitian–Gaussian modes(m=1 to 4,n=1 to 3),and Bessel–Gaussian modes(l=1 to 12).Such a sorter for arbitrary VSBs could revolutionize applications in integrated and high-dimensional optical communication systems.展开更多
Optical computing provides unique opportunities in terms of parallelization,scalability,power efficiency,and computational speed and has attracted major interest for machine learning.Diffractive deep neural networks h...Optical computing provides unique opportunities in terms of parallelization,scalability,power efficiency,and computational speed and has attracted major interest for machine learning.Diffractive deep neural networks have been introduced earlier as an optical machine learning framework that uses task-specific diffractive surfaces designed by deep learning to all-optically perform inference,achieving promising performance for object classification and imaging.We demonstrate systematic improvements in diffractive optical neural networks,based on a differential measurement technique that mitigates the strict nonnegativity constraint of light intensity.In this differential detection scheme,each class is assigned to a separate pair of detectors,behind a diffractive optical network,and the class inference is made by maximizing the normalized signal difference between the photodetector pairs.Using this differential detection scheme,involving 10 photodetector pairs behind 5 diffractive layers with a total of 0.2 million neurons,we numerically achieved blind testing accuracies of 98.54%,90.54%,and 48.51%for MNIST,Fashion-MNIST,and grayscale CIFAR-10 datasets,respectively.Moreover,by utilizing the inherent parallelization capability of optical systems,we reduced the cross-talk and optical signal coupling between the positive and negative detectors of each class by dividing the optical path into two jointly trained diffractive neural networks that work in parallel.We further made use of this parallelization approach and divided individual classes in a target dataset among multiple jointly trained diffractive neural networks.Using this class-specific differential detection in jointly optimized diffractive neural networks that operate in parallel,our simulations achieved blind testing accuracies of 98.52%,91.48%,and 50.82%for MNIST,Fashion-MNIST,and grayscale CIFAR-10 datasets,respectively,coming close to the performance of some of the earlier generations of all-electronic deep neural networks,e.g.,LeNet,which achieves classification accuracies of 98.77%,90.27%,and 55.21%corresponding to the same datasets,respectively.In addition to these jointly optimized diffractive neural networks,we also independently optimized multiple diffractive networks and utilized them in a way that is similar to ensemble methods practiced in machine learning;using 3 independently optimized differential diffractive neural networks that optically project their light onto a common output/detector plane,we numerically achieved blind testing accuracies of 98.59%,91.06%,and 51.44%for MNIST,Fashion-MNIST,and grayscale CIFAR-10 datasets,respectively.Through these systematic advances in designing diffractive neural networks,the reported classification accuracies set the state of the art for all-optical neural network design.The presented framework might be useful to bring optical neural network-based low power solutions for various machine learning applications and help us design new computational cameras that are task-specific.展开更多
Integrated diffractive optical neural networks(DONNs)have significant potential for complex machine learning tasks with high speed and ultralow energy consumption.However,the on-chip implementation of a high-performan...Integrated diffractive optical neural networks(DONNs)have significant potential for complex machine learning tasks with high speed and ultralow energy consumption.However,the on-chip implementation of a high-performance optical neural network is limited by input dimensions.In contrast to existing photonic neural networks,a space-time interleaving technology based on arrayed waveguides is designed to realize an on-chip DONN with high-speed,high-dimensional,and all-optical input signal modulation.To demonstrate the performance of the on-chip DONN with high-speed space-time interleaving modulation,an on-chip DONN with a designed footprint of 0.0945 mm~2is proposed to resolve the vowel recognition task,reaching a computation speed of about 1.4×10^(13)operations per second and yielding an accuracy of 98.3%in numerical calculation.In addition,the function of the specially designed arrayed waveguides for realizing parallel signal inputs using space-time conversion has been verified experimentally.This method can realize the on-chip DONN with higher input dimension and lower energy consumption.展开更多
Optical computing is considered a promising solution for the growing demand for parallel computing in various cutting-edge fields that require high integration and high-speed computational capacity.We propose an optic...Optical computing is considered a promising solution for the growing demand for parallel computing in various cutting-edge fields that require high integration and high-speed computational capacity.We propose an optical computation architecture called diffraction casting(DC)for flexible and scalable parallel logic operations.In DC,a diffractive neural network is designed for single instruction,multiple data(SIMD)operations.This approach allows for the alteration of logic operations simply by changing the illumination patterns.Furthermore,it eliminates the need for encoding and decoding of the input and output,respectively,by introducing a buffer around the input area,facilitating end-to-end all-optical computing.We numerically demonstrate DC by performing all 16 logic operations on two arbitrary 256-bit parallel binary inputs.Additionally,we showcase several distinctive attributes inherent in DC,such as the benefit of cohesively designing the diffractive elements for SIMD logic operations that assure high scalability and high integration capability.Our study offers a design architecture for optical computers and paves the way for a next-generation optical computing paradigm.展开更多
Quantitative phase imaging(QPI)is a label-free computational imaging technique used in various fields,including biology and medical research.Modern QPI systems typically rely on digital processing using iterative algo...Quantitative phase imaging(QPI)is a label-free computational imaging technique used in various fields,including biology and medical research.Modern QPI systems typically rely on digital processing using iterative algorithms for phase retrieval and image reconstruction.Here,we report a diffractive optical network trained to convert the phase information of input objects positioned behind random diffusers into intensity variations at the output plane,all-optically performing phase recovery and quantitative imaging of phase objects completely hidden by unknown,random phase diffusers.This QPI diffractive network is composed of successive diffractive layers,axially spanning in total~70λ,where is the illumination wavelength;unlike existing digital image reconstruction and phase retrieval methods,it forms an all-optical processor that does not require external power beyond the illumination beam to complete its QPI reconstruction at the speed of light propagation.This all-optical diffractive processor can provide a low-power,high frame rate and compact alternative for quantitative imaging of phase objects through random,unknown diffusers and can operate at different parts of the electromagnetic spectrum for various applications in biomedical imaging and sensing.The presented QPI diffractive designs can be integrated onto the active area of standard CCD/CMOS-based image sensors to convert an existing optical microscope into a diffractive QPI microscope,performing phase recovery and image reconstruction on a chip through light diffraction within passive structured layers.展开更多
基金supported by the National Natural Science Foundation of China(Grant Nos.62175050 and U2341245)the Fundamental Research Funds for the Central Universities(Grant No.HIT.OCEF.2024054).
文摘Diffractive optical neural networks(DONNs)have exhibited the advantages of parallelization,high speed,and low consumption.However,the existing DONNs based on free-space diffractive optical elements are bulky and unsteady.In this study,we propose a planar-waveguide integrated diffractive neural network chip architecture.The three diffractive layers are engraved on the same side of a quartz wafer.The three-layer chip is designed with 32-mm3 processing space and enables a computing speed of 3.1×109 Tera operations per second.The results show that the proposed chip achieves 73.4%experimental accuracy for the Modified National Institute of Standards and Technology database while showing the system’s robustness in a cycle test.The consistency of experiments is 88.6%,and the arithmetic mean standard deviation of the results is~4.7%.The proposed chip architecture can potentially revolutionize high-resolution optical processing tasks with high robustness.
基金Project supported by the National Natural Science Foundation of China(Grant Nos.62375140 and 62001249)the Open Research Fund of National Laboratory of Solid State Microstructures(Grant No.M36055).
文摘The vector vortex beam(VVB)has attracted significant attention due to its intrinsic diversity of information and has found great applications in both classical and quantum communications.However,a VVB is unavoidably affected by atmospheric turbulence(AT)when it propagates through the free-space optical communication environment,which results in detection errors at the receiver.In this paper,we propose a VVB classification scheme to detect VVBs with continuously changing polarization states under AT,where a diffractive deep neural network(DDNN)is designed and trained to classify the intensity distribution of the input distorted VVBs,and the horizontal direction of polarization of the input distorted beam is adopted as the feature for the classification through the DDNN.The numerical simulations and experimental results demonstrate that the proposed scheme has high accuracy in classification tasks.The energy distribution percentage remains above 95%from weak to medium AT,and the classification accuracy can remain above 95%for various strengths of turbulence.It has a faster convergence and better accuracy than that based on a convolutional neural network.
基金supported by the National Natural Science Foundation of China (Grant No.62135009)the Beijing Municipal Science and Technology Commission,Administrative Commission of Zhongguancun Science Park (Grant No.Z221100005322010).
文摘On-chip diffractive optical neural networks(DONNs)bring the advantages of parallel processing and low energy consumption.However,an accurate representation of the optical field’s evolution in the structure cannot be provided using the previous diffraction-based analysis method.Moreover,the loss caused by the open boundaries poses challenges to applications.A multimode DONN architecture based on a more precise eigenmode analysis method is proposed.We have constructed a universal library of input,output,and metaline structures utilizing this method,and realized a multimode DONN composed of the structures from the library.On the designed multimode DONNs with only one layer of the metaline,the classification task of an Iris plants dataset is verified with an accuracy of 90%on the blind test dataset,and the performance of the one-bit binary adder task is also validated.Compared to the previous architectures,the multimode DONN exhibits a more compact design and higher energy efficiency.
基金Project supported by the National Natural Science Foundation of China(Grant Nos.61871234 and 62001249)the Postgraduate Research and Practice Innovation Program of Jiangsu Province,China(Grant No.KYCX200718)。
文摘Orbital angular momentum(OAM)has the characteristics of mutual orthogonality between modes,and has been applied to underwater wireless optical communication(UWOC)systems to increase the channel capacity.In this work,we propose a diffractive deep neural network(DDNN)based OAM mode recognition scheme,where the DDNN is trained to capture the features of the intensity distribution of the OAM modes and output the corresponding azimuthal indices and radial indices.The results show that the proposed scheme can recognize the azimuthal indices and radial indices of the OAM modes accurately and quickly.In addition,the proposed scheme can resist weak oceanic turbulence(OT),and exhibit excellent ability to recognize OAM modes in a strong OT environment.The DDNN-based OAM mode recognition scheme has potential applications in UWOC systems.
基金supported by the National Natural Science Foundation of China(Grant Nos.62422509 and 62405188)the Shanghai Natural Science Foundation(Grant No.23ZR1443700)+3 种基金the Shuguang Program of Shanghai Education Development Foundation and Shanghai Municipal Education Commission(Grant No.23SG41)the Young Elite Scientist Sponsorship Program by CAST(Grant No.20220042)the Science and Technology Commission of Shanghai Municipality(Grant No.21DZ1100500)the Shanghai Municipal Science and Technology Major Project,and the Shanghai Frontiers Science Center Program(2021-2025 No.20).
文摘Efficiently tracking and imaging interested moving targets is crucial across various applications,from autonomous systems to surveillance.However,persistent challenges remain in various fields,including environmental intricacies,limitations in perceptual technologies,and privacy considerations.We present a teacher-student learning model,the generative adversarial network(GAN)-guided diffractive neural network(DNN),which performs visual tracking and imaging of the interested moving target.The GAN,as a teacher model,empowers efficient acquisition of the skill to differentiate the specific target of interest in the domains of visual tracking and imaging.The DNN-based student model learns to master the skill to differentiate the interested target from the GAN.The process of obtaining a GAN-guided DNN starts with capturing moving objects effectively using an event camera with high temporal resolution and low latency.Then,the generative power of GAN is utilized to generate data with position-tracking capability for the interested moving target,subsequently serving as labels to the training of the DNN.The DNN learns to image the target during training while retaining the target’s positional information.Our experimental demonstration highlights the efficacy of the GAN-guided DNN in visual tracking and imaging of the interested moving target.We expect the GAN-guided DNN can significantly enhance autonomous systems and surveillance.
基金This research was supported in part by National Natural Science Foundation of China(61675056 and 61875048).
文摘Optical deep learning based on diffractive optical elements offers unique advantages for parallel processing,computational speed,and power efficiency.One landmark method is the diffractive deep neural network(D^(2) NN)based on three-dimensional printing technology operated in the terahertz spectral range.Since the terahertz bandwidth involves limited interparticle coupling and material losses,this paper extends D^(2) NN to visible wavelengths.A general theory including a revised formula is proposed to solve any contradictions between wavelength,neuron size,and fabrication limitations.A novel visible light D^(2) NN classifier is used to recognize unchanged targets(handwritten digits ranging from 0 to 9)and targets that have been changed(i.e.,targets that have been covered or altered)at a visible wavelength of 632.8 nm.The obtained experimental classification accuracy(84%)and numerical classification accuracy(91.57%)quantify the match between the theoretical design and fabricated system performance.The presented framework can be used to apply a D^(2) NN to various practical applications and design other new applications.
基金support of the U.S.Department of Energy (DOE),Office of Basic Energy Sciences,Division of Materials Sciences and Engineering under Award#DE-SC0023088.
文摘As an optical processor,a diffractive deep neural network(D2NN)utilizes engineered diffractive surfaces designed through machine learning to perform all-optical information processing,completing its tasks at the speed of light propagation through thin optical layers.With sufficient degrees of freedom,D2NNs can perform arbitrary complex-valued linear transformations using spatially coherent light.Similarly,D2NNs can also perform arbitrary linear intensity transformations with spatially incoherent illumination;however,under spatially incoherent light,these transformations are nonnegative,acting on diffraction-limited optical intensity patterns at the input field of view.Here,we expand the use of spatially incoherent D2NNs to complex-valued information processing for executing arbitrary complex-valued linear transformations using spatially incoherent light.Through simulations,we show that as the number of optimized diffractive features increases beyond a threshold dictated by the multiplication of the input and output space-bandwidth products,a spatially incoherent diffractive visual processor can approximate any complex-valued linear transformation and be used for all-optical image encryption using incoherent illumination.The findings are important for the all-optical processing of information under natural light using various forms of diffractive surface-based optical processors.
基金supported by the National Key Research and Development Program of China(Nos.2021YFB2802000 and 2022YFB2804301)the Science and Technology Commission of Shanghai Municipality(No.21DZ1100500)+4 种基金the Shanghai Municipal Science and Technology Major Project,the Shanghai Frontiers Science Center Program(2021–2025 No.20)the National Natural Science Foundation of China(Nos.61975123,62305219,and 62205208)the Shanghai Natural Science Foundation(No.23ZR1443200)the China Postdoctoral Science Foundation(Nos.2022M712138 and 2021M702192)the Shanghai Super Postdoctoral Incentive Scheme(Nos.5B22904002 and 5B22904006)。
文摘Free-space diffractive neural networks(DNNs)have been an intense research topic in machine learning for image recognition and encryption due to their high speed,lower power consumption,and high neuron density.Recent advances in DNNs have highlighted the need for smaller device footprints and the shift toward visible wavelengths.However,DNNs fabricated by electron beam lithography,are not suitable for microscopic imaging applications due to their large sizes,and DNNs fabricated by two-photon nanolithography with cylindrical neurons are not optimal for visible wavelengths,as the highorder diffraction could induce low diffraction efficiency.In this paper,we demonstrate that cubical diffraction neurons are more efficient diffraction elements for DNNs compared with cylindrical neurons.Based on the theoretical analysis of the relationship between the detector area sizes and classification accuracy,we reduced the size of DNNs operating at the wavelength of 532 nm for handwritten digit classification to micrometer scale by two-photon nanolithography.The DNNs with cubical neurons demonstrated an experimental classification accuracy(89.3%)for single-layer DNN,and 83.3%for two-layer DNN with device sizes similar to that of biological cells(about 100μm×100μm).Our results paved the pathway to integrate 3D micrometer-scale DNNs with microscopic imaging systems for biological imaging and cell recognition.
基金Japan Society for the Promotion of Science(JP20H05890,JP22H05197,JP23H05444,JP23K26567)。
文摘We propose a design method for a diffractive neural network(DNN)for imaging through scattering media,offering robustness against the spatial coherence of illumination,scattering strength,and scattering dynamics.Most techniques for imaging through scattering media are time-consuming and/or tailored to specific optical conditions.The DNN,composed of layers of diffractive optical elements(DOEs),optically reproduces the intensity distributions of objects behind scattering media without any computational processing.Datasets with randomized optical parameters are provided during the training process to achieve this robustness.We demonstrate the proposed method through numerical calculations and show its promising capability for DOE design.Our study paves the way for unifying and generalizing techniques for imaging through scattering media,which are currently fragmented by specific scenarios,enabling highly flexible imaging independent of optical conditions.
基金supported by the National Key Research and Development Program of China(Grant No.2023YFB2806502)the National Natural Science Foundation of China(Grant No.62425504)the Knowledge Innovation Program of Wuhan—Basic Research(Grant No.2023010201010049)。
文摘Optical neural networks have emerged as feasible alternatives to their electronic counterparts,offering significant benefits such as low power consumption,low latency,and high parallelism.However,the realization of ultra-compact nonlinear deep neural networks and multi-thread processing remain crucial challenges for optical computing.We present a monolithically integrated all-optical nonlinear diffractive deep neural network(AON-D^(2) NN)chip for the first time.The all-optical nonlinear activation function is implemented using germanium microstructures,which provide low loss and are compatible with the standard silicon photonics fabrication process.Assisted by the germanium activation function,the classification accuracy is improved by 9.1%for four-classification tasks.In addition,the chip's reconfigurability enables multi-task learning in situ via an innovative cross-training algorithm,yielding two task-specific inference results with accuracies of 95%and 96%,respectively.Furthermore,leveraging the wavelength-dependent response of the chip,the multi-thread nonlinear optical neural network is implemented for the first time,capable of handling two different tasks in parallel.The proposed AON-D^(2)NN contains three hidden layers with a footprint of only 0.73 mm^(2).It can achieve ultra-low latency(172 ps),paving the path for realizing high-performance optical neural networks.
基金supported by the National Natural Science Foundation of China(Grant Nos.62271322,62331004,and 62222501)the Guangdong Basic and Applied Basic Research Foundation(Grant No.2023A1515030152)+1 种基金the Science and Technology Project of Shenzhen(Grant No.ZDSYS201707271014468)the Natural Science Foundation of Top Talent of SZTU(Grant No.GDRC202204)。
文摘Orbital angular momentum(OAM)modes provide an additional orthogonal physical dimension,offering transformative potential for enhancing optical communication capacity.Despite significant progress in mode multiplexing,the development of robust communication networks faces persistent challenges,particularly in effectively routing and controlling these multiplexed channels among network nodes.To tackle these dilemmas,we propose a rotatable diffractive neural network(R-DNN)strategy and demonstrate its capability for port-controllable OAM mode routing.By leveraging the correlation between the orthogonal evolution of OAM modes in free space and phase modulations during propagation,the R-DNN precisely shapes the spatial evolution of mode fields through multiple rotatable phase layers,enabling efficient routing to specific output ports.This approach exploits the interaction of secondary wavelets with the relative states of the rotatable layers,allowing on-demand control of mode evolution paths and enhancing routing flexibility.As a proof of concept,we developed a tri-functional router that successfully directs three OAM modes to individually controllable output ports.This router achieves an average intermode crosstalk of less than−16.4 dB across three functional states,one-dimensional,two-dimensional,and cross-connected switching,while supporting the routing of 5.85 Tbit/s quadrature phase-shift keying signals.These results highlight the R-DNN’s effectiveness in achieving precise and controllable OAM mode manipulation,paving the way for advanced applications in mode-multiplexed communication networks and beyond.
基金supported by the National Key Research and Development Program of China(Grant Nos.2021YFB2800604,2021YFB2800302,and 2018YFB2200403)the National Natural Science Foundation of China(Grant Nos.12274478,91950204,and 92150302)the Graduate Research and Practice Projects of Minzu University of China.
文摘As a successful case of combining deep learning with photonics,the research on optical machine learning has recently undergone rapid development.Among various optical classification frameworks,diffractive networks have been shown to have unique advantages in all-optical reasoning.As an important property of light,the orbital angular momentum(OAM)of light shows orthogonality and mode-infinity,which can enhance the ability of parallel classification in information processing.However,there have been few all-optical diffractive networks under the OAM mode encoding.Here,we report a strategy of OAM-encoded diffractive deep neural network(OAM-encoded D2NN)that encodes the spatial information of objects into the OAM spectrum of the diffracted light to perform all-optical object classification.We demonstrated three different OAM-encoded D2NNs to realize(1)single detector OAM-encoded D2NN for single task classification,(2)single detector OAM-encoded D2NN for multitask classification,and(3)multidetector OAM-encoded D2NN for repeatable multitask classification.We provide a feasible way to improve the performance of all-optical object classification and open up promising research directions for D2NN by proposing OAMencoded D2NN.
基金The authors acknowledge the funding provided by the National Key R&D Program of China(2021YFA1401200)Beijing Outstanding Young Scientist Program(BJJWZYJH01201910007022)+2 种基金National Natural Science Foundation of China(No.U21A20140,No.92050117,No.62005017)programBeijing Municipal Science&Technology Commission,Administrative Commission of Zhongguancun Science Park(No.Z211100004821009)This work was supported by the Synergetic Extreme Condition User Facility(SECUF).
文摘Optical neural networks have significant advantages in terms of power consumption,parallelism,and high computing speed,which has intrigued extensive attention in both academic and engineering communities.It has been considered as one of the powerful tools in promoting the fields of imaging processing and object recognition.However,the existing optical system architecture cannot be reconstructed to the realization of multi-functional artificial intelligence systems simultaneously.To push the development of this issue,we propose the pluggable diffractive neural networks(P-DNN),a general paradigm resorting to the cascaded metasurfaces,which can be applied to recognize various tasks by switching internal plug-ins.As the proof-of-principle,the recognition functions of six types of handwritten digits and six types of fashions are numerical simulated and experimental demonstrated at near-infrared regimes.Encouragingly,the proposed paradigm not only improves the flexibility of the optical neural networks but paves the new route for achieving high-speed,low-power and versatile artificial intelligence systems.
基金upported by the National Key Research and Development Program of China(Nos.2022YFB2804301 and 2021YFB2802000)the Science and Technology Commission of Shanghai Municipality(No.21DZ1100500)+2 种基金the Shanghai Municipal Science and Technology Major Projectthe Shanghai Frontiers Science Center Program(2021–2025 No.20)the Shanghai Sailing Program(No.23YF1429500).
文摘Research in the ocean places high demands on chips'robustness,speed,and energy consumption.Diffractive neural networks(DNNs)enable direct optical image processing at light speed,with great potential for underwater applications.Here,we experimentally demonstrate a compact DNN chip capable of operating directly in both water and air by multiobjective training and initial training value optimization.The two layers of DNNs are integrated on the two surfaces of a quartz plate,respectively.The chip achieved high accuracies above 90%in recognition tasks for handwritten digits and fashion products.The architecture and material ensure the chip's high stability for long-term underwater use.
文摘Unidirectional imagers form images of input objects only in one direction,e.g.,from field-of-view(FOV)A to FOV B,while blocking the image formation in the reverse direction,from FOV B to FOV A.Here,we report unidirectional imaging under spatially partially coherent light and demonstrate high-quality imaging only in the forward direction(A→B)with high power efficiency while distorting the image formation in the backward direction(B→A)along with low power efficiency.Our reciprocal design features a set of spatially engineered linear diffractive layers that are statistically optimized for partially coherent illumination with a given phase correlation length.Our analyses reveal that when illuminated by a partially coherent beam with a correlation length of≥∼1.5λ,whereλis the wavelength of light,diffractive unidirectional imagers achieve robust performance,exhibiting asymmetric imaging performance between the forward and backward directions—as desired.A partially coherent unidirectional imager designed with a smaller correlation length of<1.5λstill supports unidirectional image transmission but with a reduced figure of merit.These partially coherent diffractive unidirectional imagers are compact(axially spanning<75λ),polarization-independent,and compatible with various types of illumination sources,making them well-suited for applications in asymmetric visual information processing and communication.
基金supported by the National Natural Science Foundation of China(Grant No.12274105)the Heilongjiang Natural Science Funds for Distinguished Young Scholars(Grant No.JQ2022A001)+1 种基金the Fundamental Research Funds for the Central Universities(Grant No.HIT.OCEF.2021020)the Joint Guidance Project of the Natural Science Foundation of Heilongjiang Province(Grant No.LH2023A006).
文摘Vector structured beams(VSBs)offer infinite eigenstates and open up new possibilities for highcapacity optical and quantum communications by the multiplexing of the states.Therefore,the sorting and measuring of VSBs are extremely important.However,the efficient manipulations of a large number of VSBs have simultaneously remained challenging up to now,especially in integrated optical systems.Here,we propose a compact spin-multiplexed diffractive metasurface capable of continuously sorting and detecting arbitrary VSBs through spatial intensity separation.By introducing a diffractive optical neural network with cascaded metasurface systems,we demonstrate arbitrary VSBs sorters that can simultaneously identify Laguerre–Gaussian modes(l=−4 to 4,p=1 to 4),Hermitian–Gaussian modes(m=1 to 4,n=1 to 3),and Bessel–Gaussian modes(l=1 to 12).Such a sorter for arbitrary VSBs could revolutionize applications in integrated and high-dimensional optical communication systems.
文摘Optical computing provides unique opportunities in terms of parallelization,scalability,power efficiency,and computational speed and has attracted major interest for machine learning.Diffractive deep neural networks have been introduced earlier as an optical machine learning framework that uses task-specific diffractive surfaces designed by deep learning to all-optically perform inference,achieving promising performance for object classification and imaging.We demonstrate systematic improvements in diffractive optical neural networks,based on a differential measurement technique that mitigates the strict nonnegativity constraint of light intensity.In this differential detection scheme,each class is assigned to a separate pair of detectors,behind a diffractive optical network,and the class inference is made by maximizing the normalized signal difference between the photodetector pairs.Using this differential detection scheme,involving 10 photodetector pairs behind 5 diffractive layers with a total of 0.2 million neurons,we numerically achieved blind testing accuracies of 98.54%,90.54%,and 48.51%for MNIST,Fashion-MNIST,and grayscale CIFAR-10 datasets,respectively.Moreover,by utilizing the inherent parallelization capability of optical systems,we reduced the cross-talk and optical signal coupling between the positive and negative detectors of each class by dividing the optical path into two jointly trained diffractive neural networks that work in parallel.We further made use of this parallelization approach and divided individual classes in a target dataset among multiple jointly trained diffractive neural networks.Using this class-specific differential detection in jointly optimized diffractive neural networks that operate in parallel,our simulations achieved blind testing accuracies of 98.52%,91.48%,and 50.82%for MNIST,Fashion-MNIST,and grayscale CIFAR-10 datasets,respectively,coming close to the performance of some of the earlier generations of all-electronic deep neural networks,e.g.,LeNet,which achieves classification accuracies of 98.77%,90.27%,and 55.21%corresponding to the same datasets,respectively.In addition to these jointly optimized diffractive neural networks,we also independently optimized multiple diffractive networks and utilized them in a way that is similar to ensemble methods practiced in machine learning;using 3 independently optimized differential diffractive neural networks that optically project their light onto a common output/detector plane,we numerically achieved blind testing accuracies of 98.59%,91.06%,and 51.44%for MNIST,Fashion-MNIST,and grayscale CIFAR-10 datasets,respectively.Through these systematic advances in designing diffractive neural networks,the reported classification accuracies set the state of the art for all-optical neural network design.The presented framework might be useful to bring optical neural network-based low power solutions for various machine learning applications and help us design new computational cameras that are task-specific.
基金supported by the National Natural Science Foundation of China(NSFC)(No.62135009)the Beijing Municipal Science&Technology Commission,Administrative Commission of Zhongguancun Science Park(No.Z221100005322010)。
文摘Integrated diffractive optical neural networks(DONNs)have significant potential for complex machine learning tasks with high speed and ultralow energy consumption.However,the on-chip implementation of a high-performance optical neural network is limited by input dimensions.In contrast to existing photonic neural networks,a space-time interleaving technology based on arrayed waveguides is designed to realize an on-chip DONN with high-speed,high-dimensional,and all-optical input signal modulation.To demonstrate the performance of the on-chip DONN with high-speed space-time interleaving modulation,an on-chip DONN with a designed footprint of 0.0945 mm~2is proposed to resolve the vowel recognition task,reaching a computation speed of about 1.4×10^(13)operations per second and yielding an accuracy of 98.3%in numerical calculation.In addition,the function of the specially designed arrayed waveguides for realizing parallel signal inputs using space-time conversion has been verified experimentally.This method can realize the on-chip DONN with higher input dimension and lower energy consumption.
基金supported by Japan Society for the Promotion of Science(Grant Nos.JP20K05361,JP22H05197,and JP23K26567).
文摘Optical computing is considered a promising solution for the growing demand for parallel computing in various cutting-edge fields that require high integration and high-speed computational capacity.We propose an optical computation architecture called diffraction casting(DC)for flexible and scalable parallel logic operations.In DC,a diffractive neural network is designed for single instruction,multiple data(SIMD)operations.This approach allows for the alteration of logic operations simply by changing the illumination patterns.Furthermore,it eliminates the need for encoding and decoding of the input and output,respectively,by introducing a buffer around the input area,facilitating end-to-end all-optical computing.We numerically demonstrate DC by performing all 16 logic operations on two arbitrary 256-bit parallel binary inputs.Additionally,we showcase several distinctive attributes inherent in DC,such as the benefit of cohesively designing the diffractive elements for SIMD logic operations that assure high scalability and high integration capability.Our study offers a design architecture for optical computers and paves the way for a next-generation optical computing paradigm.
文摘Quantitative phase imaging(QPI)is a label-free computational imaging technique used in various fields,including biology and medical research.Modern QPI systems typically rely on digital processing using iterative algorithms for phase retrieval and image reconstruction.Here,we report a diffractive optical network trained to convert the phase information of input objects positioned behind random diffusers into intensity variations at the output plane,all-optically performing phase recovery and quantitative imaging of phase objects completely hidden by unknown,random phase diffusers.This QPI diffractive network is composed of successive diffractive layers,axially spanning in total~70λ,where is the illumination wavelength;unlike existing digital image reconstruction and phase retrieval methods,it forms an all-optical processor that does not require external power beyond the illumination beam to complete its QPI reconstruction at the speed of light propagation.This all-optical diffractive processor can provide a low-power,high frame rate and compact alternative for quantitative imaging of phase objects through random,unknown diffusers and can operate at different parts of the electromagnetic spectrum for various applications in biomedical imaging and sensing.The presented QPI diffractive designs can be integrated onto the active area of standard CCD/CMOS-based image sensors to convert an existing optical microscope into a diffractive QPI microscope,performing phase recovery and image reconstruction on a chip through light diffraction within passive structured layers.
