Speckle-based optical cryptosystems are promising technologies for information security.However,existing techniques mostly rely on digital decryption,resulting in computational delay and undermining the high-speed adv...Speckle-based optical cryptosystems are promising technologies for information security.However,existing techniques mostly rely on digital decryption,resulting in computational delay and undermining the high-speed advantage of optical encryption.Moreover,conventional neural networks are typically effective only on images from the same distribution as the training datasets,limiting their general applicability.In this paper,we propose an all-optical high-speed decryption scheme for real-time recovery of speckle-encoded ciphertexts.By constructing a physics-informed diffractive neural network that approximates the inverse transmission matrix of the scattering medium,secret images can be directly reconstructed from speckle fields without optoelectronic conversion or post-processing.The network is trained with only 2048 samples from the MNIST dataset.Its transfer learning capability is validated across three out-of-distribution datasets,with decrypted images achieving a Pearson correlation coefficient of 0.82 and a structural similarity index measure of 0.75,demonstrating excellent transfer learning capability.For the first time,to our knowledge,this scheme simultaneously overcomes the bottlenecks of decryption delay and limited network generalizability in conventional speckle-based cryptosystems,achieving real-time image decryption with strong transferability.It provides a new pathway for developing low-power,real-time,and broadly applicable optical encryption systems,demonstrating significant potential for applications in high-speed security optical communications.展开更多
The asymmetric imaging device is a crucial and highly desired component in optical and electromagnetic systems.However,most existing asymmetric imaging devices are based on active or nonlinear materials and are limite...The asymmetric imaging device is a crucial and highly desired component in optical and electromagnetic systems.However,most existing asymmetric imaging devices are based on active or nonlinear materials and are limited to one-directional applications.This paper reports a method to realize asymmetric image transmission and transformation in two opposite directions,respectively,based on diffractive deep neural networks(D^(2)NNs),named Janus meta-imager.It is a passive device composed of several diffractive layers that are well-trained using deep-learning-based algorithms.We first experimentally fabricate and validate this Janus meta-imager in the near-infrared(NIR)band,which agrees well with simulation results,thus verifying the asymmetric imaging function.This scheme has the merits of high-speed all-optical processing,low energy consumption,and small size,offering potential applications in all-optical encryption and information storage.展开更多
Efficiently tracking and imaging interested moving targets is crucial across various applications,from autonomous systems to surveillance.However,persistent challenges remain in various fields,including environmental ...Efficiently tracking and imaging interested moving targets is crucial across various applications,from autonomous systems to surveillance.However,persistent challenges remain in various fields,including environmental intricacies,limitations in perceptual technologies,and privacy considerations.We present a teacher-student learning model,the generative adversarial network(GAN)-guided diffractive neural network(DNN),which performs visual tracking and imaging of the interested moving target.The GAN,as a teacher model,empowers efficient acquisition of the skill to differentiate the specific target of interest in the domains of visual tracking and imaging.The DNN-based student model learns to master the skill to differentiate the interested target from the GAN.The process of obtaining a GAN-guided DNN starts with capturing moving objects effectively using an event camera with high temporal resolution and low latency.Then,the generative power of GAN is utilized to generate data with position-tracking capability for the interested moving target,subsequently serving as labels to the training of the DNN.The DNN learns to image the target during training while retaining the target’s positional information.Our experimental demonstration highlights the efficacy of the GAN-guided DNN in visual tracking and imaging of the interested moving target.We expect the GAN-guided DNN can significantly enhance autonomous systems and surveillance.展开更多
We propose a design method for a diffractive neural network(DNN)for imaging through scattering media,offering robustness against the spatial coherence of illumination,scattering strength,and scattering dynamics.Most t...We propose a design method for a diffractive neural network(DNN)for imaging through scattering media,offering robustness against the spatial coherence of illumination,scattering strength,and scattering dynamics.Most techniques for imaging through scattering media are time-consuming and/or tailored to specific optical conditions.The DNN,composed of layers of diffractive optical elements(DOEs),optically reproduces the intensity distributions of objects behind scattering media without any computational processing.Datasets with randomized optical parameters are provided during the training process to achieve this robustness.We demonstrate the proposed method through numerical calculations and show its promising capability for DOE design.Our study paves the way for unifying and generalizing techniques for imaging through scattering media,which are currently fragmented by specific scenarios,enabling highly flexible imaging independent of optical conditions.展开更多
Diffractive optical neural networks(DONNs)have exhibited the advantages of parallelization,high speed,and low consumption.However,the existing DONNs based on free-space diffractive optical elements are bulky and unste...Diffractive optical neural networks(DONNs)have exhibited the advantages of parallelization,high speed,and low consumption.However,the existing DONNs based on free-space diffractive optical elements are bulky and unsteady.In this study,we propose a planar-waveguide integrated diffractive neural network chip architecture.The three diffractive layers are engraved on the same side of a quartz wafer.The three-layer chip is designed with 32-mm3 processing space and enables a computing speed of 3.1×109 Tera operations per second.The results show that the proposed chip achieves 73.4%experimental accuracy for the Modified National Institute of Standards and Technology database while showing the system’s robustness in a cycle test.The consistency of experiments is 88.6%,and the arithmetic mean standard deviation of the results is~4.7%.The proposed chip architecture can potentially revolutionize high-resolution optical processing tasks with high robustness.展开更多
Terahertz(THz)communication has emerged as one of the key technologies for sixthgeneration(6G)wireless networks.Nevertheless,the transition to higher operational frequencies poses various challenges including high-spe...Terahertz(THz)communication has emerged as one of the key technologies for sixthgeneration(6G)wireless networks.Nevertheless,the transition to higher operational frequencies poses various challenges including high-speed digital-to-analog conversion(DACs)and analog-to-digital conversion(ADCs),heterogeneous integration of optoelectronic devices,resulting in an urgent need for solutions.In this paper,we demonstrate a groundbreaking THz analog differential operator driven by diffractive neural networks(DNN),implementing ultra-fast and high-throughput analog domain differential operations.The designed multilayer all-optical DNN composed of compact dielectric metasurfaces is trained with trigonometric functions to perform analog differential computing of complex input signals by approximating the differentiation of finite decompositions of time-domain function based on the Fourier transform theory,significantly improving integration,throughput,and processing speed.Our design has been experimentally validated to successfully implement single-direction differential operation on one-(1D)and two-dimensional(2D)signals with superior structural similarity index measure(SSIM)and peak signal-to-noise ratio(PSNR),providing a promising path for the development of integrated and ultrafast THz communication systems.展开更多
Orbital angular momentum(OAM)modes provide an additional orthogonal physical dimension,offering transformative potential for enhancing optical communication capacity.Despite significant progress in mode multiplexing,t...Orbital angular momentum(OAM)modes provide an additional orthogonal physical dimension,offering transformative potential for enhancing optical communication capacity.Despite significant progress in mode multiplexing,the development of robust communication networks faces persistent challenges,particularly in effectively routing and controlling these multiplexed channels among network nodes.To tackle these dilemmas,we propose a rotatable diffractive neural network(R-DNN)strategy and demonstrate its capability for port-controllable OAM mode routing.By leveraging the correlation between the orthogonal evolution of OAM modes in free space and phase modulations during propagation,the R-DNN precisely shapes the spatial evolution of mode fields through multiple rotatable phase layers,enabling efficient routing to specific output ports.This approach exploits the interaction of secondary wavelets with the relative states of the rotatable layers,allowing on-demand control of mode evolution paths and enhancing routing flexibility.As a proof of concept,we developed a tri-functional router that successfully directs three OAM modes to individually controllable output ports.This router achieves an average intermode crosstalk of less than−16.4 dB across three functional states,one-dimensional,two-dimensional,and cross-connected switching,while supporting the routing of 5.85 Tbit/s quadrature phase-shift keying signals.These results highlight the R-DNN’s effectiveness in achieving precise and controllable OAM mode manipulation,paving the way for advanced applications in mode-multiplexed communication networks and beyond.展开更多
Free-space diffractive neural networks(DNNs)have been an intense research topic in machine learning for image recognition and encryption due to their high speed,lower power consumption,and high neuron density.Recent a...Free-space diffractive neural networks(DNNs)have been an intense research topic in machine learning for image recognition and encryption due to their high speed,lower power consumption,and high neuron density.Recent advances in DNNs have highlighted the need for smaller device footprints and the shift toward visible wavelengths.However,DNNs fabricated by electron beam lithography,are not suitable for microscopic imaging applications due to their large sizes,and DNNs fabricated by two-photon nanolithography with cylindrical neurons are not optimal for visible wavelengths,as the highorder diffraction could induce low diffraction efficiency.In this paper,we demonstrate that cubical diffraction neurons are more efficient diffraction elements for DNNs compared with cylindrical neurons.Based on the theoretical analysis of the relationship between the detector area sizes and classification accuracy,we reduced the size of DNNs operating at the wavelength of 532 nm for handwritten digit classification to micrometer scale by two-photon nanolithography.The DNNs with cubical neurons demonstrated an experimental classification accuracy(89.3%)for single-layer DNN,and 83.3%for two-layer DNN with device sizes similar to that of biological cells(about 100μm×100μm).Our results paved the pathway to integrate 3D micrometer-scale DNNs with microscopic imaging systems for biological imaging and cell recognition.展开更多
Research in the ocean places high demands on chips'robustness,speed,and energy consumption.Diffractive neural networks(DNNs)enable direct optical image processing at light speed,with great potential for underwater...Research in the ocean places high demands on chips'robustness,speed,and energy consumption.Diffractive neural networks(DNNs)enable direct optical image processing at light speed,with great potential for underwater applications.Here,we experimentally demonstrate a compact DNN chip capable of operating directly in both water and air by multiobjective training and initial training value optimization.The two layers of DNNs are integrated on the two surfaces of a quartz plate,respectively.The chip achieved high accuracies above 90%in recognition tasks for handwritten digits and fashion products.The architecture and material ensure the chip's high stability for long-term underwater use.展开更多
Optical deep learning based on diffractive optical elements offers unique advantages for parallel processing,computational speed,and power efficiency.One landmark method is the diffractive deep neural network(D^(2) NN...Optical deep learning based on diffractive optical elements offers unique advantages for parallel processing,computational speed,and power efficiency.One landmark method is the diffractive deep neural network(D^(2) NN)based on three-dimensional printing technology operated in the terahertz spectral range.Since the terahertz bandwidth involves limited interparticle coupling and material losses,this paper extends D^(2) NN to visible wavelengths.A general theory including a revised formula is proposed to solve any contradictions between wavelength,neuron size,and fabrication limitations.A novel visible light D^(2) NN classifier is used to recognize unchanged targets(handwritten digits ranging from 0 to 9)and targets that have been changed(i.e.,targets that have been covered or altered)at a visible wavelength of 632.8 nm.The obtained experimental classification accuracy(84%)and numerical classification accuracy(91.57%)quantify the match between the theoretical design and fabricated system performance.The presented framework can be used to apply a D^(2) NN to various practical applications and design other new applications.展开更多
Optical neural networks have significant advantages in terms of power consumption,parallelism,and high computing speed,which has intrigued extensive attention in both academic and engineering communities.It has been c...Optical neural networks have significant advantages in terms of power consumption,parallelism,and high computing speed,which has intrigued extensive attention in both academic and engineering communities.It has been considered as one of the powerful tools in promoting the fields of imaging processing and object recognition.However,the existing optical system architecture cannot be reconstructed to the realization of multi-functional artificial intelligence systems simultaneously.To push the development of this issue,we propose the pluggable diffractive neural networks(P-DNN),a general paradigm resorting to the cascaded metasurfaces,which can be applied to recognize various tasks by switching internal plug-ins.As the proof-of-principle,the recognition functions of six types of handwritten digits and six types of fashions are numerical simulated and experimental demonstrated at near-infrared regimes.Encouragingly,the proposed paradigm not only improves the flexibility of the optical neural networks but paves the new route for achieving high-speed,low-power and versatile artificial intelligence systems.展开更多
On-chip diffractive optical neural networks(DONNs)bring the advantages of parallel processing and low energy consumption.However,an accurate representation of the optical field’s evolution in the structure cannot be ...On-chip diffractive optical neural networks(DONNs)bring the advantages of parallel processing and low energy consumption.However,an accurate representation of the optical field’s evolution in the structure cannot be provided using the previous diffraction-based analysis method.Moreover,the loss caused by the open boundaries poses challenges to applications.A multimode DONN architecture based on a more precise eigenmode analysis method is proposed.We have constructed a universal library of input,output,and metaline structures utilizing this method,and realized a multimode DONN composed of the structures from the library.On the designed multimode DONNs with only one layer of the metaline,the classification task of an Iris plants dataset is verified with an accuracy of 90%on the blind test dataset,and the performance of the one-bit binary adder task is also validated.Compared to the previous architectures,the multimode DONN exhibits a more compact design and higher energy efficiency.展开更多
As an optical processor,a diffractive deep neural network(D2NN)utilizes engineered diffractive surfaces designed through machine learning to perform all-optical information processing,completing its tasks at the speed...As an optical processor,a diffractive deep neural network(D2NN)utilizes engineered diffractive surfaces designed through machine learning to perform all-optical information processing,completing its tasks at the speed of light propagation through thin optical layers.With sufficient degrees of freedom,D2NNs can perform arbitrary complex-valued linear transformations using spatially coherent light.Similarly,D2NNs can also perform arbitrary linear intensity transformations with spatially incoherent illumination;however,under spatially incoherent light,these transformations are nonnegative,acting on diffraction-limited optical intensity patterns at the input field of view.Here,we expand the use of spatially incoherent D2NNs to complex-valued information processing for executing arbitrary complex-valued linear transformations using spatially incoherent light.Through simulations,we show that as the number of optimized diffractive features increases beyond a threshold dictated by the multiplication of the input and output space-bandwidth products,a spatially incoherent diffractive visual processor can approximate any complex-valued linear transformation and be used for all-optical image encryption using incoherent illumination.The findings are important for the all-optical processing of information under natural light using various forms of diffractive surface-based optical processors.展开更多
Orbital angular momentum(OAM)has the characteristics of mutual orthogonality between modes,and has been applied to underwater wireless optical communication(UWOC)systems to increase the channel capacity.In this work,w...Orbital angular momentum(OAM)has the characteristics of mutual orthogonality between modes,and has been applied to underwater wireless optical communication(UWOC)systems to increase the channel capacity.In this work,we propose a diffractive deep neural network(DDNN)based OAM mode recognition scheme,where the DDNN is trained to capture the features of the intensity distribution of the OAM modes and output the corresponding azimuthal indices and radial indices.The results show that the proposed scheme can recognize the azimuthal indices and radial indices of the OAM modes accurately and quickly.In addition,the proposed scheme can resist weak oceanic turbulence(OT),and exhibit excellent ability to recognize OAM modes in a strong OT environment.The DDNN-based OAM mode recognition scheme has potential applications in UWOC systems.展开更多
As a successful case of combining deep learning with photonics,the research on optical machine learning has recently undergone rapid development.Among various optical classification frameworks,diffractive networks hav...As a successful case of combining deep learning with photonics,the research on optical machine learning has recently undergone rapid development.Among various optical classification frameworks,diffractive networks have been shown to have unique advantages in all-optical reasoning.As an important property of light,the orbital angular momentum(OAM)of light shows orthogonality and mode-infinity,which can enhance the ability of parallel classification in information processing.However,there have been few all-optical diffractive networks under the OAM mode encoding.Here,we report a strategy of OAM-encoded diffractive deep neural network(OAM-encoded D2NN)that encodes the spatial information of objects into the OAM spectrum of the diffracted light to perform all-optical object classification.We demonstrated three different OAM-encoded D2NNs to realize(1)single detector OAM-encoded D2NN for single task classification,(2)single detector OAM-encoded D2NN for multitask classification,and(3)multidetector OAM-encoded D2NN for repeatable multitask classification.We provide a feasible way to improve the performance of all-optical object classification and open up promising research directions for D2NN by proposing OAMencoded D2NN.展开更多
The vector vortex beam(VVB)has attracted significant attention due to its intrinsic diversity of information and has found great applications in both classical and quantum communications.However,a VVB is unavoidably a...The vector vortex beam(VVB)has attracted significant attention due to its intrinsic diversity of information and has found great applications in both classical and quantum communications.However,a VVB is unavoidably affected by atmospheric turbulence(AT)when it propagates through the free-space optical communication environment,which results in detection errors at the receiver.In this paper,we propose a VVB classification scheme to detect VVBs with continuously changing polarization states under AT,where a diffractive deep neural network(DDNN)is designed and trained to classify the intensity distribution of the input distorted VVBs,and the horizontal direction of polarization of the input distorted beam is adopted as the feature for the classification through the DDNN.The numerical simulations and experimental results demonstrate that the proposed scheme has high accuracy in classification tasks.The energy distribution percentage remains above 95%from weak to medium AT,and the classification accuracy can remain above 95%for various strengths of turbulence.It has a faster convergence and better accuracy than that based on a convolutional neural network.展开更多
Vector structured beams(VSBs)offer infinite eigenstates and open up new possibilities for highcapacity optical and quantum communications by the multiplexing of the states.Therefore,the sorting and measuring of VSBs a...Vector structured beams(VSBs)offer infinite eigenstates and open up new possibilities for highcapacity optical and quantum communications by the multiplexing of the states.Therefore,the sorting and measuring of VSBs are extremely important.However,the efficient manipulations of a large number of VSBs have simultaneously remained challenging up to now,especially in integrated optical systems.Here,we propose a compact spin-multiplexed diffractive metasurface capable of continuously sorting and detecting arbitrary VSBs through spatial intensity separation.By introducing a diffractive optical neural network with cascaded metasurface systems,we demonstrate arbitrary VSBs sorters that can simultaneously identify Laguerre–Gaussian modes(l=−4 to 4,p=1 to 4),Hermitian–Gaussian modes(m=1 to 4,n=1 to 3),and Bessel–Gaussian modes(l=1 to 12).Such a sorter for arbitrary VSBs could revolutionize applications in integrated and high-dimensional optical communication systems.展开更多
Optical neural networks have emerged as feasible alternatives to their electronic counterparts,offering significant benefits such as low power consumption,low latency,and high parallelism.However,the realization of ul...Optical neural networks have emerged as feasible alternatives to their electronic counterparts,offering significant benefits such as low power consumption,low latency,and high parallelism.However,the realization of ultra-compact nonlinear deep neural networks and multi-thread processing remain crucial challenges for optical computing.We present a monolithically integrated all-optical nonlinear diffractive deep neural network(AON-D^(2) NN)chip for the first time.The all-optical nonlinear activation function is implemented using germanium microstructures,which provide low loss and are compatible with the standard silicon photonics fabrication process.Assisted by the germanium activation function,the classification accuracy is improved by 9.1%for four-classification tasks.In addition,the chip's reconfigurability enables multi-task learning in situ via an innovative cross-training algorithm,yielding two task-specific inference results with accuracies of 95%and 96%,respectively.Furthermore,leveraging the wavelength-dependent response of the chip,the multi-thread nonlinear optical neural network is implemented for the first time,capable of handling two different tasks in parallel.The proposed AON-D^(2)NN contains three hidden layers with a footprint of only 0.73 mm^(2).It can achieve ultra-low latency(172 ps),paving the path for realizing high-performance optical neural networks.展开更多
Optical computing provides unique opportunities in terms of parallelization,scalability,power efficiency,and computational speed and has attracted major interest for machine learning.Diffractive deep neural networks h...Optical computing provides unique opportunities in terms of parallelization,scalability,power efficiency,and computational speed and has attracted major interest for machine learning.Diffractive deep neural networks have been introduced earlier as an optical machine learning framework that uses task-specific diffractive surfaces designed by deep learning to all-optically perform inference,achieving promising performance for object classification and imaging.We demonstrate systematic improvements in diffractive optical neural networks,based on a differential measurement technique that mitigates the strict nonnegativity constraint of light intensity.In this differential detection scheme,each class is assigned to a separate pair of detectors,behind a diffractive optical network,and the class inference is made by maximizing the normalized signal difference between the photodetector pairs.Using this differential detection scheme,involving 10 photodetector pairs behind 5 diffractive layers with a total of 0.2 million neurons,we numerically achieved blind testing accuracies of 98.54%,90.54%,and 48.51%for MNIST,Fashion-MNIST,and grayscale CIFAR-10 datasets,respectively.Moreover,by utilizing the inherent parallelization capability of optical systems,we reduced the cross-talk and optical signal coupling between the positive and negative detectors of each class by dividing the optical path into two jointly trained diffractive neural networks that work in parallel.We further made use of this parallelization approach and divided individual classes in a target dataset among multiple jointly trained diffractive neural networks.Using this class-specific differential detection in jointly optimized diffractive neural networks that operate in parallel,our simulations achieved blind testing accuracies of 98.52%,91.48%,and 50.82%for MNIST,Fashion-MNIST,and grayscale CIFAR-10 datasets,respectively,coming close to the performance of some of the earlier generations of all-electronic deep neural networks,e.g.,LeNet,which achieves classification accuracies of 98.77%,90.27%,and 55.21%corresponding to the same datasets,respectively.In addition to these jointly optimized diffractive neural networks,we also independently optimized multiple diffractive networks and utilized them in a way that is similar to ensemble methods practiced in machine learning;using 3 independently optimized differential diffractive neural networks that optically project their light onto a common output/detector plane,we numerically achieved blind testing accuracies of 98.59%,91.06%,and 51.44%for MNIST,Fashion-MNIST,and grayscale CIFAR-10 datasets,respectively.Through these systematic advances in designing diffractive neural networks,the reported classification accuracies set the state of the art for all-optical neural network design.The presented framework might be useful to bring optical neural network-based low power solutions for various machine learning applications and help us design new computational cameras that are task-specific.展开更多
Integrated diffractive optical neural networks(DONNs)have significant potential for complex machine learning tasks with high speed and ultralow energy consumption.However,the on-chip implementation of a high-performan...Integrated diffractive optical neural networks(DONNs)have significant potential for complex machine learning tasks with high speed and ultralow energy consumption.However,the on-chip implementation of a high-performance optical neural network is limited by input dimensions.In contrast to existing photonic neural networks,a space-time interleaving technology based on arrayed waveguides is designed to realize an on-chip DONN with high-speed,high-dimensional,and all-optical input signal modulation.To demonstrate the performance of the on-chip DONN with high-speed space-time interleaving modulation,an on-chip DONN with a designed footprint of 0.0945 mm~2is proposed to resolve the vowel recognition task,reaching a computation speed of about 1.4×10^(13)operations per second and yielding an accuracy of 98.3%in numerical calculation.In addition,the function of the specially designed arrayed waveguides for realizing parallel signal inputs using space-time conversion has been verified experimentally.This method can realize the on-chip DONN with higher input dimension and lower energy consumption.展开更多
基金supported by the Guangdong Major Project of Basic Research(Grant No.2020B0301030009)the National Natural Science Foundation of China(Grant Nos.12174204,12174203,12074203,62335012,and 62435010)+5 种基金the Natural Science Foundation of Guangdong Province(Grant No.2023A1515012888)the Science and Technology Innovation Commission of Shenzhen(Grant Nos.JCYJ20220818101417039 and JCYJ20241202124428038)the Medical-Engineering Interdisciplinary Research Foundation of Shenzhen University(Grant No.86901/00000311)the Scientific Instrument Developing Project of Shenzhen University(Grant No.2023YQ001)the Shenzhen University 2035 Initiative(Grant No.2023B004)the Key R&D Program of Zhejiang(Grant No.30003AA240100)。
文摘Speckle-based optical cryptosystems are promising technologies for information security.However,existing techniques mostly rely on digital decryption,resulting in computational delay and undermining the high-speed advantage of optical encryption.Moreover,conventional neural networks are typically effective only on images from the same distribution as the training datasets,limiting their general applicability.In this paper,we propose an all-optical high-speed decryption scheme for real-time recovery of speckle-encoded ciphertexts.By constructing a physics-informed diffractive neural network that approximates the inverse transmission matrix of the scattering medium,secret images can be directly reconstructed from speckle fields without optoelectronic conversion or post-processing.The network is trained with only 2048 samples from the MNIST dataset.Its transfer learning capability is validated across three out-of-distribution datasets,with decrypted images achieving a Pearson correlation coefficient of 0.82 and a structural similarity index measure of 0.75,demonstrating excellent transfer learning capability.For the first time,to our knowledge,this scheme simultaneously overcomes the bottlenecks of decryption delay and limited network generalizability in conventional speckle-based cryptosystems,achieving real-time image decryption with strong transferability.It provides a new pathway for developing low-power,real-time,and broadly applicable optical encryption systems,demonstrating significant potential for applications in high-speed security optical communications.
基金funded by the National Key R&D Program of China(No.2021YFA1401200)National Natural Science Foundation of China(Nos.62231001 and U21A20140)Beijing Natural Science Foundation(No.JQ24028).
文摘The asymmetric imaging device is a crucial and highly desired component in optical and electromagnetic systems.However,most existing asymmetric imaging devices are based on active or nonlinear materials and are limited to one-directional applications.This paper reports a method to realize asymmetric image transmission and transformation in two opposite directions,respectively,based on diffractive deep neural networks(D^(2)NNs),named Janus meta-imager.It is a passive device composed of several diffractive layers that are well-trained using deep-learning-based algorithms.We first experimentally fabricate and validate this Janus meta-imager in the near-infrared(NIR)band,which agrees well with simulation results,thus verifying the asymmetric imaging function.This scheme has the merits of high-speed all-optical processing,low energy consumption,and small size,offering potential applications in all-optical encryption and information storage.
基金supported by the National Natural Science Foundation of China(Grant Nos.62422509 and 62405188)the Shanghai Natural Science Foundation(Grant No.23ZR1443700)+3 种基金the Shuguang Program of Shanghai Education Development Foundation and Shanghai Municipal Education Commission(Grant No.23SG41)the Young Elite Scientist Sponsorship Program by CAST(Grant No.20220042)the Science and Technology Commission of Shanghai Municipality(Grant No.21DZ1100500)the Shanghai Municipal Science and Technology Major Project,and the Shanghai Frontiers Science Center Program(2021-2025 No.20).
文摘Efficiently tracking and imaging interested moving targets is crucial across various applications,from autonomous systems to surveillance.However,persistent challenges remain in various fields,including environmental intricacies,limitations in perceptual technologies,and privacy considerations.We present a teacher-student learning model,the generative adversarial network(GAN)-guided diffractive neural network(DNN),which performs visual tracking and imaging of the interested moving target.The GAN,as a teacher model,empowers efficient acquisition of the skill to differentiate the specific target of interest in the domains of visual tracking and imaging.The DNN-based student model learns to master the skill to differentiate the interested target from the GAN.The process of obtaining a GAN-guided DNN starts with capturing moving objects effectively using an event camera with high temporal resolution and low latency.Then,the generative power of GAN is utilized to generate data with position-tracking capability for the interested moving target,subsequently serving as labels to the training of the DNN.The DNN learns to image the target during training while retaining the target’s positional information.Our experimental demonstration highlights the efficacy of the GAN-guided DNN in visual tracking and imaging of the interested moving target.We expect the GAN-guided DNN can significantly enhance autonomous systems and surveillance.
基金Japan Society for the Promotion of Science(JP20H05890,JP22H05197,JP23H05444,JP23K26567)。
文摘We propose a design method for a diffractive neural network(DNN)for imaging through scattering media,offering robustness against the spatial coherence of illumination,scattering strength,and scattering dynamics.Most techniques for imaging through scattering media are time-consuming and/or tailored to specific optical conditions.The DNN,composed of layers of diffractive optical elements(DOEs),optically reproduces the intensity distributions of objects behind scattering media without any computational processing.Datasets with randomized optical parameters are provided during the training process to achieve this robustness.We demonstrate the proposed method through numerical calculations and show its promising capability for DOE design.Our study paves the way for unifying and generalizing techniques for imaging through scattering media,which are currently fragmented by specific scenarios,enabling highly flexible imaging independent of optical conditions.
基金supported by the National Natural Science Foundation of China(Grant Nos.62175050 and U2341245)the Fundamental Research Funds for the Central Universities(Grant No.HIT.OCEF.2024054).
文摘Diffractive optical neural networks(DONNs)have exhibited the advantages of parallelization,high speed,and low consumption.However,the existing DONNs based on free-space diffractive optical elements are bulky and unsteady.In this study,we propose a planar-waveguide integrated diffractive neural network chip architecture.The three diffractive layers are engraved on the same side of a quartz wafer.The three-layer chip is designed with 32-mm3 processing space and enables a computing speed of 3.1×109 Tera operations per second.The results show that the proposed chip achieves 73.4%experimental accuracy for the Modified National Institute of Standards and Technology database while showing the system’s robustness in a cycle test.The consistency of experiments is 88.6%,and the arithmetic mean standard deviation of the results is~4.7%.The proposed chip architecture can potentially revolutionize high-resolution optical processing tasks with high robustness.
基金supported by Fundamental Research Funds for the Central Universities(HIT.OCEF.2023035).
文摘Terahertz(THz)communication has emerged as one of the key technologies for sixthgeneration(6G)wireless networks.Nevertheless,the transition to higher operational frequencies poses various challenges including high-speed digital-to-analog conversion(DACs)and analog-to-digital conversion(ADCs),heterogeneous integration of optoelectronic devices,resulting in an urgent need for solutions.In this paper,we demonstrate a groundbreaking THz analog differential operator driven by diffractive neural networks(DNN),implementing ultra-fast and high-throughput analog domain differential operations.The designed multilayer all-optical DNN composed of compact dielectric metasurfaces is trained with trigonometric functions to perform analog differential computing of complex input signals by approximating the differentiation of finite decompositions of time-domain function based on the Fourier transform theory,significantly improving integration,throughput,and processing speed.Our design has been experimentally validated to successfully implement single-direction differential operation on one-(1D)and two-dimensional(2D)signals with superior structural similarity index measure(SSIM)and peak signal-to-noise ratio(PSNR),providing a promising path for the development of integrated and ultrafast THz communication systems.
基金supported by the National Natural Science Foundation of China(Grant Nos.62271322,62331004,and 62222501)the Guangdong Basic and Applied Basic Research Foundation(Grant No.2023A1515030152)+1 种基金the Science and Technology Project of Shenzhen(Grant No.ZDSYS201707271014468)the Natural Science Foundation of Top Talent of SZTU(Grant No.GDRC202204)。
文摘Orbital angular momentum(OAM)modes provide an additional orthogonal physical dimension,offering transformative potential for enhancing optical communication capacity.Despite significant progress in mode multiplexing,the development of robust communication networks faces persistent challenges,particularly in effectively routing and controlling these multiplexed channels among network nodes.To tackle these dilemmas,we propose a rotatable diffractive neural network(R-DNN)strategy and demonstrate its capability for port-controllable OAM mode routing.By leveraging the correlation between the orthogonal evolution of OAM modes in free space and phase modulations during propagation,the R-DNN precisely shapes the spatial evolution of mode fields through multiple rotatable phase layers,enabling efficient routing to specific output ports.This approach exploits the interaction of secondary wavelets with the relative states of the rotatable layers,allowing on-demand control of mode evolution paths and enhancing routing flexibility.As a proof of concept,we developed a tri-functional router that successfully directs three OAM modes to individually controllable output ports.This router achieves an average intermode crosstalk of less than−16.4 dB across three functional states,one-dimensional,two-dimensional,and cross-connected switching,while supporting the routing of 5.85 Tbit/s quadrature phase-shift keying signals.These results highlight the R-DNN’s effectiveness in achieving precise and controllable OAM mode manipulation,paving the way for advanced applications in mode-multiplexed communication networks and beyond.
基金supported by the National Key Research and Development Program of China(Nos.2021YFB2802000 and 2022YFB2804301)the Science and Technology Commission of Shanghai Municipality(No.21DZ1100500)+4 种基金the Shanghai Municipal Science and Technology Major Project,the Shanghai Frontiers Science Center Program(2021–2025 No.20)the National Natural Science Foundation of China(Nos.61975123,62305219,and 62205208)the Shanghai Natural Science Foundation(No.23ZR1443200)the China Postdoctoral Science Foundation(Nos.2022M712138 and 2021M702192)the Shanghai Super Postdoctoral Incentive Scheme(Nos.5B22904002 and 5B22904006)。
文摘Free-space diffractive neural networks(DNNs)have been an intense research topic in machine learning for image recognition and encryption due to their high speed,lower power consumption,and high neuron density.Recent advances in DNNs have highlighted the need for smaller device footprints and the shift toward visible wavelengths.However,DNNs fabricated by electron beam lithography,are not suitable for microscopic imaging applications due to their large sizes,and DNNs fabricated by two-photon nanolithography with cylindrical neurons are not optimal for visible wavelengths,as the highorder diffraction could induce low diffraction efficiency.In this paper,we demonstrate that cubical diffraction neurons are more efficient diffraction elements for DNNs compared with cylindrical neurons.Based on the theoretical analysis of the relationship between the detector area sizes and classification accuracy,we reduced the size of DNNs operating at the wavelength of 532 nm for handwritten digit classification to micrometer scale by two-photon nanolithography.The DNNs with cubical neurons demonstrated an experimental classification accuracy(89.3%)for single-layer DNN,and 83.3%for two-layer DNN with device sizes similar to that of biological cells(about 100μm×100μm).Our results paved the pathway to integrate 3D micrometer-scale DNNs with microscopic imaging systems for biological imaging and cell recognition.
基金upported by the National Key Research and Development Program of China(Nos.2022YFB2804301 and 2021YFB2802000)the Science and Technology Commission of Shanghai Municipality(No.21DZ1100500)+2 种基金the Shanghai Municipal Science and Technology Major Projectthe Shanghai Frontiers Science Center Program(2021–2025 No.20)the Shanghai Sailing Program(No.23YF1429500).
文摘Research in the ocean places high demands on chips'robustness,speed,and energy consumption.Diffractive neural networks(DNNs)enable direct optical image processing at light speed,with great potential for underwater applications.Here,we experimentally demonstrate a compact DNN chip capable of operating directly in both water and air by multiobjective training and initial training value optimization.The two layers of DNNs are integrated on the two surfaces of a quartz plate,respectively.The chip achieved high accuracies above 90%in recognition tasks for handwritten digits and fashion products.The architecture and material ensure the chip's high stability for long-term underwater use.
基金This research was supported in part by National Natural Science Foundation of China(61675056 and 61875048).
文摘Optical deep learning based on diffractive optical elements offers unique advantages for parallel processing,computational speed,and power efficiency.One landmark method is the diffractive deep neural network(D^(2) NN)based on three-dimensional printing technology operated in the terahertz spectral range.Since the terahertz bandwidth involves limited interparticle coupling and material losses,this paper extends D^(2) NN to visible wavelengths.A general theory including a revised formula is proposed to solve any contradictions between wavelength,neuron size,and fabrication limitations.A novel visible light D^(2) NN classifier is used to recognize unchanged targets(handwritten digits ranging from 0 to 9)and targets that have been changed(i.e.,targets that have been covered or altered)at a visible wavelength of 632.8 nm.The obtained experimental classification accuracy(84%)and numerical classification accuracy(91.57%)quantify the match between the theoretical design and fabricated system performance.The presented framework can be used to apply a D^(2) NN to various practical applications and design other new applications.
基金The authors acknowledge the funding provided by the National Key R&D Program of China(2021YFA1401200)Beijing Outstanding Young Scientist Program(BJJWZYJH01201910007022)+2 种基金National Natural Science Foundation of China(No.U21A20140,No.92050117,No.62005017)programBeijing Municipal Science&Technology Commission,Administrative Commission of Zhongguancun Science Park(No.Z211100004821009)This work was supported by the Synergetic Extreme Condition User Facility(SECUF).
文摘Optical neural networks have significant advantages in terms of power consumption,parallelism,and high computing speed,which has intrigued extensive attention in both academic and engineering communities.It has been considered as one of the powerful tools in promoting the fields of imaging processing and object recognition.However,the existing optical system architecture cannot be reconstructed to the realization of multi-functional artificial intelligence systems simultaneously.To push the development of this issue,we propose the pluggable diffractive neural networks(P-DNN),a general paradigm resorting to the cascaded metasurfaces,which can be applied to recognize various tasks by switching internal plug-ins.As the proof-of-principle,the recognition functions of six types of handwritten digits and six types of fashions are numerical simulated and experimental demonstrated at near-infrared regimes.Encouragingly,the proposed paradigm not only improves the flexibility of the optical neural networks but paves the new route for achieving high-speed,low-power and versatile artificial intelligence systems.
基金supported by the National Natural Science Foundation of China (Grant No.62135009)the Beijing Municipal Science and Technology Commission,Administrative Commission of Zhongguancun Science Park (Grant No.Z221100005322010).
文摘On-chip diffractive optical neural networks(DONNs)bring the advantages of parallel processing and low energy consumption.However,an accurate representation of the optical field’s evolution in the structure cannot be provided using the previous diffraction-based analysis method.Moreover,the loss caused by the open boundaries poses challenges to applications.A multimode DONN architecture based on a more precise eigenmode analysis method is proposed.We have constructed a universal library of input,output,and metaline structures utilizing this method,and realized a multimode DONN composed of the structures from the library.On the designed multimode DONNs with only one layer of the metaline,the classification task of an Iris plants dataset is verified with an accuracy of 90%on the blind test dataset,and the performance of the one-bit binary adder task is also validated.Compared to the previous architectures,the multimode DONN exhibits a more compact design and higher energy efficiency.
基金support of the U.S.Department of Energy (DOE),Office of Basic Energy Sciences,Division of Materials Sciences and Engineering under Award#DE-SC0023088.
文摘As an optical processor,a diffractive deep neural network(D2NN)utilizes engineered diffractive surfaces designed through machine learning to perform all-optical information processing,completing its tasks at the speed of light propagation through thin optical layers.With sufficient degrees of freedom,D2NNs can perform arbitrary complex-valued linear transformations using spatially coherent light.Similarly,D2NNs can also perform arbitrary linear intensity transformations with spatially incoherent illumination;however,under spatially incoherent light,these transformations are nonnegative,acting on diffraction-limited optical intensity patterns at the input field of view.Here,we expand the use of spatially incoherent D2NNs to complex-valued information processing for executing arbitrary complex-valued linear transformations using spatially incoherent light.Through simulations,we show that as the number of optimized diffractive features increases beyond a threshold dictated by the multiplication of the input and output space-bandwidth products,a spatially incoherent diffractive visual processor can approximate any complex-valued linear transformation and be used for all-optical image encryption using incoherent illumination.The findings are important for the all-optical processing of information under natural light using various forms of diffractive surface-based optical processors.
基金Project supported by the National Natural Science Foundation of China(Grant Nos.61871234 and 62001249)the Postgraduate Research and Practice Innovation Program of Jiangsu Province,China(Grant No.KYCX200718)。
文摘Orbital angular momentum(OAM)has the characteristics of mutual orthogonality between modes,and has been applied to underwater wireless optical communication(UWOC)systems to increase the channel capacity.In this work,we propose a diffractive deep neural network(DDNN)based OAM mode recognition scheme,where the DDNN is trained to capture the features of the intensity distribution of the OAM modes and output the corresponding azimuthal indices and radial indices.The results show that the proposed scheme can recognize the azimuthal indices and radial indices of the OAM modes accurately and quickly.In addition,the proposed scheme can resist weak oceanic turbulence(OT),and exhibit excellent ability to recognize OAM modes in a strong OT environment.The DDNN-based OAM mode recognition scheme has potential applications in UWOC systems.
基金supported by the National Key Research and Development Program of China(Grant Nos.2021YFB2800604,2021YFB2800302,and 2018YFB2200403)the National Natural Science Foundation of China(Grant Nos.12274478,91950204,and 92150302)the Graduate Research and Practice Projects of Minzu University of China.
文摘As a successful case of combining deep learning with photonics,the research on optical machine learning has recently undergone rapid development.Among various optical classification frameworks,diffractive networks have been shown to have unique advantages in all-optical reasoning.As an important property of light,the orbital angular momentum(OAM)of light shows orthogonality and mode-infinity,which can enhance the ability of parallel classification in information processing.However,there have been few all-optical diffractive networks under the OAM mode encoding.Here,we report a strategy of OAM-encoded diffractive deep neural network(OAM-encoded D2NN)that encodes the spatial information of objects into the OAM spectrum of the diffracted light to perform all-optical object classification.We demonstrated three different OAM-encoded D2NNs to realize(1)single detector OAM-encoded D2NN for single task classification,(2)single detector OAM-encoded D2NN for multitask classification,and(3)multidetector OAM-encoded D2NN for repeatable multitask classification.We provide a feasible way to improve the performance of all-optical object classification and open up promising research directions for D2NN by proposing OAMencoded D2NN.
基金Project supported by the National Natural Science Foundation of China(Grant Nos.62375140 and 62001249)the Open Research Fund of National Laboratory of Solid State Microstructures(Grant No.M36055).
文摘The vector vortex beam(VVB)has attracted significant attention due to its intrinsic diversity of information and has found great applications in both classical and quantum communications.However,a VVB is unavoidably affected by atmospheric turbulence(AT)when it propagates through the free-space optical communication environment,which results in detection errors at the receiver.In this paper,we propose a VVB classification scheme to detect VVBs with continuously changing polarization states under AT,where a diffractive deep neural network(DDNN)is designed and trained to classify the intensity distribution of the input distorted VVBs,and the horizontal direction of polarization of the input distorted beam is adopted as the feature for the classification through the DDNN.The numerical simulations and experimental results demonstrate that the proposed scheme has high accuracy in classification tasks.The energy distribution percentage remains above 95%from weak to medium AT,and the classification accuracy can remain above 95%for various strengths of turbulence.It has a faster convergence and better accuracy than that based on a convolutional neural network.
基金supported by the National Natural Science Foundation of China(Grant No.12274105)the Heilongjiang Natural Science Funds for Distinguished Young Scholars(Grant No.JQ2022A001)+1 种基金the Fundamental Research Funds for the Central Universities(Grant No.HIT.OCEF.2021020)the Joint Guidance Project of the Natural Science Foundation of Heilongjiang Province(Grant No.LH2023A006).
文摘Vector structured beams(VSBs)offer infinite eigenstates and open up new possibilities for highcapacity optical and quantum communications by the multiplexing of the states.Therefore,the sorting and measuring of VSBs are extremely important.However,the efficient manipulations of a large number of VSBs have simultaneously remained challenging up to now,especially in integrated optical systems.Here,we propose a compact spin-multiplexed diffractive metasurface capable of continuously sorting and detecting arbitrary VSBs through spatial intensity separation.By introducing a diffractive optical neural network with cascaded metasurface systems,we demonstrate arbitrary VSBs sorters that can simultaneously identify Laguerre–Gaussian modes(l=−4 to 4,p=1 to 4),Hermitian–Gaussian modes(m=1 to 4,n=1 to 3),and Bessel–Gaussian modes(l=1 to 12).Such a sorter for arbitrary VSBs could revolutionize applications in integrated and high-dimensional optical communication systems.
基金supported by the National Key Research and Development Program of China(Grant No.2023YFB2806502)the National Natural Science Foundation of China(Grant No.62425504)the Knowledge Innovation Program of Wuhan—Basic Research(Grant No.2023010201010049)。
文摘Optical neural networks have emerged as feasible alternatives to their electronic counterparts,offering significant benefits such as low power consumption,low latency,and high parallelism.However,the realization of ultra-compact nonlinear deep neural networks and multi-thread processing remain crucial challenges for optical computing.We present a monolithically integrated all-optical nonlinear diffractive deep neural network(AON-D^(2) NN)chip for the first time.The all-optical nonlinear activation function is implemented using germanium microstructures,which provide low loss and are compatible with the standard silicon photonics fabrication process.Assisted by the germanium activation function,the classification accuracy is improved by 9.1%for four-classification tasks.In addition,the chip's reconfigurability enables multi-task learning in situ via an innovative cross-training algorithm,yielding two task-specific inference results with accuracies of 95%and 96%,respectively.Furthermore,leveraging the wavelength-dependent response of the chip,the multi-thread nonlinear optical neural network is implemented for the first time,capable of handling two different tasks in parallel.The proposed AON-D^(2)NN contains three hidden layers with a footprint of only 0.73 mm^(2).It can achieve ultra-low latency(172 ps),paving the path for realizing high-performance optical neural networks.
文摘Optical computing provides unique opportunities in terms of parallelization,scalability,power efficiency,and computational speed and has attracted major interest for machine learning.Diffractive deep neural networks have been introduced earlier as an optical machine learning framework that uses task-specific diffractive surfaces designed by deep learning to all-optically perform inference,achieving promising performance for object classification and imaging.We demonstrate systematic improvements in diffractive optical neural networks,based on a differential measurement technique that mitigates the strict nonnegativity constraint of light intensity.In this differential detection scheme,each class is assigned to a separate pair of detectors,behind a diffractive optical network,and the class inference is made by maximizing the normalized signal difference between the photodetector pairs.Using this differential detection scheme,involving 10 photodetector pairs behind 5 diffractive layers with a total of 0.2 million neurons,we numerically achieved blind testing accuracies of 98.54%,90.54%,and 48.51%for MNIST,Fashion-MNIST,and grayscale CIFAR-10 datasets,respectively.Moreover,by utilizing the inherent parallelization capability of optical systems,we reduced the cross-talk and optical signal coupling between the positive and negative detectors of each class by dividing the optical path into two jointly trained diffractive neural networks that work in parallel.We further made use of this parallelization approach and divided individual classes in a target dataset among multiple jointly trained diffractive neural networks.Using this class-specific differential detection in jointly optimized diffractive neural networks that operate in parallel,our simulations achieved blind testing accuracies of 98.52%,91.48%,and 50.82%for MNIST,Fashion-MNIST,and grayscale CIFAR-10 datasets,respectively,coming close to the performance of some of the earlier generations of all-electronic deep neural networks,e.g.,LeNet,which achieves classification accuracies of 98.77%,90.27%,and 55.21%corresponding to the same datasets,respectively.In addition to these jointly optimized diffractive neural networks,we also independently optimized multiple diffractive networks and utilized them in a way that is similar to ensemble methods practiced in machine learning;using 3 independently optimized differential diffractive neural networks that optically project their light onto a common output/detector plane,we numerically achieved blind testing accuracies of 98.59%,91.06%,and 51.44%for MNIST,Fashion-MNIST,and grayscale CIFAR-10 datasets,respectively.Through these systematic advances in designing diffractive neural networks,the reported classification accuracies set the state of the art for all-optical neural network design.The presented framework might be useful to bring optical neural network-based low power solutions for various machine learning applications and help us design new computational cameras that are task-specific.
基金supported by the National Natural Science Foundation of China(NSFC)(No.62135009)the Beijing Municipal Science&Technology Commission,Administrative Commission of Zhongguancun Science Park(No.Z221100005322010)。
文摘Integrated diffractive optical neural networks(DONNs)have significant potential for complex machine learning tasks with high speed and ultralow energy consumption.However,the on-chip implementation of a high-performance optical neural network is limited by input dimensions.In contrast to existing photonic neural networks,a space-time interleaving technology based on arrayed waveguides is designed to realize an on-chip DONN with high-speed,high-dimensional,and all-optical input signal modulation.To demonstrate the performance of the on-chip DONN with high-speed space-time interleaving modulation,an on-chip DONN with a designed footprint of 0.0945 mm~2is proposed to resolve the vowel recognition task,reaching a computation speed of about 1.4×10^(13)operations per second and yielding an accuracy of 98.3%in numerical calculation.In addition,the function of the specially designed arrayed waveguides for realizing parallel signal inputs using space-time conversion has been verified experimentally.This method can realize the on-chip DONN with higher input dimension and lower energy consumption.
