Retrieving high-fidelity images from optical speckles remains challenging,especially when the information in speckles is severely insufficient.To address classification through scattering media under such constraints,...Retrieving high-fidelity images from optical speckles remains challenging,especially when the information in speckles is severely insufficient.To address classification through scattering media under such constraints,we propose Speckle Transformer,a vision-transformer-based model that directly classifies objects using raw speckle patterns without intermediate image retrieval.By leveraging inherent features within speckles to extract discriminative features,our approach achieves nearly 90%accuracy for classifying speckles encoded with different images,outperforming traditional retrieval-classification pipelines by up to five times,even with extreme information sparsity(i.e.,1/1024 speckle regions of interest).In addition,we quantify speckle information capacity via information entropy analysis,demonstrating that classification accuracy correlates strongly with the information entropy of speckle autocorrelation.We not only overcome limitations of conventional methods but also establish a paradigm for real-time classification in scattering environments with constrained data.展开更多
Coherent optical control within or through scattering media via wavefront shaping has seen broad applications since its invention around 2007.Wavefront shaping is aimed at overcoming the strong scattering,featured by ...Coherent optical control within or through scattering media via wavefront shaping has seen broad applications since its invention around 2007.Wavefront shaping is aimed at overcoming the strong scattering,featured by random interference,namely speckle patterns.This randomness occurs due to the refractive index inhomogeneity in complex media like biological tissue or the modal dispersion in multimode fiber,yet this randomness is actually deterministic and potentially can be time reversal or precompensated.Various wavefront shaping approaches,such as optical phase conjugation,iterative optimization,and transmission matrix measurement,have been developed to generate tight and intense optical delivery or high-resolution image of an optical object behind or within a scattering medium.The performance of these modula-tions,however,is far from satisfaction.Most recently,artifcial intelligence has brought new inspirations to this field,providing exciting hopes to tackle the challenges by mapping the input and output optical patterns and building a neuron network that inherently links them.In this paper,we survey the developments to date on this topic and briefly discuss our views on how to harness machine learning(deep learning in particular)for further advancements in the field.展开更多
Transmission matrix(TM)allows light control through complex media,such as multimode fibers(MMFs),gaining great attention in areas,such as biophotonics,over the past decade.Efforts have been taken to retrieve a complex...Transmission matrix(TM)allows light control through complex media,such as multimode fibers(MMFs),gaining great attention in areas,such as biophotonics,over the past decade.Efforts have been taken to retrieve a complex-valued TM directly from intensity measurements with several representative phase-retrieval algorithms,which still see limitations of slow or suboptimum recovery,especially under noisy environments.Here,we propose a modified nonconvex optimization approach.Through numerical evaluations,it shows that the optimum focusing efficiency is approached with less running time or sampling ratio.The comparative tests under different signal-to-noise levels further indicate its improved robustness.Experimentally,the superior focusing performance of our algorithm is collectively validated by single-and multispot focusing;especially with a sampling ratio of 8,it achieves a 93.6%efficiency of the gold-standard holography method.Based on the recovered TM,image transmission through an MMF is realized with high fidelity.Due to parallel operation and GPU acceleration,our nonconvex approach retrieves a 8685×1024 TM(sampling ratio is 8)with 42.3 s on average on a regular computer.The proposed method provides optimum efficiency and fast execution for TM retrieval that avoids the need for an external reference beam,which will facilitate applications of deep-tissue optical imaging,manipulation,and treatment.展开更多
Optical techniques offer a wide variety of applications as light-matter interactions provide extremely sensitive mechanisms to probe or treat target media.Most of these implementations rely on the usage of ballistic o...Optical techniques offer a wide variety of applications as light-matter interactions provide extremely sensitive mechanisms to probe or treat target media.Most of these implementations rely on the usage of ballistic or quasi-ballistic photons to achieve high spatial resolution.However,the inherent scattering nature of light in biological tissues or tissue-like scattering media constitutes a critical obstacle that has restricted the penetration depth of non-scattered photons and hence limited the implementation of most optical techniques for wider applications.In addition,the components of an optical system are usually designed and manufactured for a fixed function or performance.Recent advances in wavefront shaping have demonstrated that scattering-or component-induced phase distortions can be compensated by optimizing the wavefront of the input light pattern through iteration or by conjugating the transmission matrix of the scattering medium.展开更多
Information retrieval from visually random optical speckle patterns is desired in many scenarios yet considered challenging.It requires accurate understanding or mapping of the multiple scattering process,or reliable ...Information retrieval from visually random optical speckle patterns is desired in many scenarios yet considered challenging.It requires accurate understanding or mapping of the multiple scattering process,or reliable capability to reverse or compensate for the scattering-induced phase distortions.In whatever situation,effective resolving and digitization of speckle patterns are necessary.Nevertheless,on some occasions,to increase the acquisition speed and/or signal-to-noise ratio(SNR),speckles captured by cameras are inevitably sampled in the sub-Nyquist domain via pixel binning(one camera pixel contains multiple speckle grains)due to finite size or limited bandwidth of photosensors.Such a down-sampling process is irreversible;it undermines the fine structures of speckle grains and hence the encoded information,preventing successful information extraction.To retrace the lost information,super-resolution interpolation for such sub-Nyquist sampled speckles is needed.In this work,a deep neural network,namely SpkSRNet,is proposed to effectively up sample speckles that are sampled below 1/10 of the Nyquist criterion to well-resolved ones that not only resemble the comprehensive morphology of original speckles(decompose multiple speckle grains from one camera pixel)but also recover the lost complex information(human face in this study)with high fidelity under normal-and low-light conditions,which is impossible with classic interpolation methods.These successful speckle super-resolution interpolation demonstrations are essentially enabled by the strong implicit correlation among speckle grains,which is non-quantifiable but could be discovered by the well-trained network.With further engineering,the proposed learning platform may benefit many scenarios that are physically inaccessible,enabling fast acquisition of speckles with sufficient SNR and opening up new avenues for seeing big and seeing clearly simultaneously in complex scenarios.展开更多
Imaging through scattering media is valuable for many areas,such as biomedicine and communication.Recent progress enabled by deep learning(DL)has shown superiority especially in the model generalization.However,there ...Imaging through scattering media is valuable for many areas,such as biomedicine and communication.Recent progress enabled by deep learning(DL)has shown superiority especially in the model generalization.However,there is a lack of research to physically reveal the origin or define the boundary for such model scalability,which is important for utilizing DL approaches for scalable imaging despite scattering with high confidence.In this paper,we find the amount of the ballistic light component in the output field is the prerequisite for endowing a DL model with generalization capability by using a“one-to-all”training strategy,which offers a physical meaning invariance among the multisource data.The findings are supported by both experimental and simulated tests in which the roles of scattered and ballistic components are revealed in contributing to the origin and physical boundary of the model scalability.Experimentally,the generalization performance of the network is enhanced by increasing the portion of ballistic photons in detection.The mechanism understanding and practical guidance by our research are beneficial for developing DL methods for descattering with high adaptivity.展开更多
Multimode fibers(MMFs)are a promising solution for high-throughput signal transmission in the time domain.However,crosstalk among different optical modes within the MMF scrambles input information and creates seemingl...Multimode fibers(MMFs)are a promising solution for high-throughput signal transmission in the time domain.However,crosstalk among different optical modes within the MMF scrambles input information and creates seemingly random speckle patterns at the output.To characterize this process,a transmission matrix(TM)can be used to relate input and output fields.Recent innovations use TMs to manipulate the output field by shaping the input wavefront for exciting advances in deep-brain imaging,neuron stimulation,quantum networks,and analog operators.However,these approaches consider input/output segments as independent,limiting their use for separate signal processing,such as logic operations.Our proposed method,which makes input/output segments as interdependent,adjusts the phase of corresponding output fields using phase bias maps superimposed on input segments.Coherent superposition enables signal logic operations through a 15-m-long MMF.In experiments,a single optical logic gate containing three basic logic functions and cascading multiple logic gates to handle binary operands is demonstrated.Bitwise operations are performed for multi-bit logic operations,and multiple optical logic gates are reconstructed simultaneously in a single logic gate with polarization multiplexing.The proposed method may open new avenues for long-range logic signal processing and transmission via MMFs.展开更多
Multiple scattering can significantly scramble the amplitude and phase profile of an optical field.It obscures subtle observations but only speckle patterns can be seen,unlike the ballistic regime where the informatio...Multiple scattering can significantly scramble the amplitude and phase profile of an optical field.It obscures subtle observations but only speckle patterns can be seen,unlike the ballistic regime where the information or the optical field can be identified with limited distortions.Efficient optical manipulation including information transmission and precise focusing is therefore obstructed as light travels deep into turbidmedia such as fog,turbid fluids,and biological tissues.展开更多
基金supported by the National Natural Science Foundation of China(Grant Nos.81930048 and 82330061)the Hong Kong Research Grant Council(Grant Nos.15217721,C7074-21GF,and 15125724)+4 种基金the Hong Kong Innovation and Technology Commission(Grant Nos.GHP/043/19SZ and GHP/044/19GD)the Guangdong Science and Technology Commission(Grant No.2019BT02X105)the Shenzhen Science and Technology Innovation Commission(Grant No.JCYJ20220818100202005)the Hong Kong Polytechnic University(Grant Nos.P0038180,P0039517,P0043485,and P0045762)the Fundamental Research Funds for the Central Universities(Grant No.QTZX25121).
文摘Retrieving high-fidelity images from optical speckles remains challenging,especially when the information in speckles is severely insufficient.To address classification through scattering media under such constraints,we propose Speckle Transformer,a vision-transformer-based model that directly classifies objects using raw speckle patterns without intermediate image retrieval.By leveraging inherent features within speckles to extract discriminative features,our approach achieves nearly 90%accuracy for classifying speckles encoded with different images,outperforming traditional retrieval-classification pipelines by up to five times,even with extreme information sparsity(i.e.,1/1024 speckle regions of interest).In addition,we quantify speckle information capacity via information entropy analysis,demonstrating that classification accuracy correlates strongly with the information entropy of speckle autocorrelation.We not only overcome limitations of conventional methods but also establish a paradigm for real-time classification in scattering environments with constrained data.
基金supported by the National Natural Science Foundation of China(Nos.81671726 and 81627805)the Hong Kong Research Grant Council(No.25204416)+1 种基金the Shenzhen Science and Technology Innovation Commission(No.JCYJ20170818104421564)the Hong Kong Innovation and Technology Commission(No.ITS/022/18).
文摘Coherent optical control within or through scattering media via wavefront shaping has seen broad applications since its invention around 2007.Wavefront shaping is aimed at overcoming the strong scattering,featured by random interference,namely speckle patterns.This randomness occurs due to the refractive index inhomogeneity in complex media like biological tissue or the modal dispersion in multimode fiber,yet this randomness is actually deterministic and potentially can be time reversal or precompensated.Various wavefront shaping approaches,such as optical phase conjugation,iterative optimization,and transmission matrix measurement,have been developed to generate tight and intense optical delivery or high-resolution image of an optical object behind or within a scattering medium.The performance of these modula-tions,however,is far from satisfaction.Most recently,artifcial intelligence has brought new inspirations to this field,providing exciting hopes to tackle the challenges by mapping the input and output optical patterns and building a neuron network that inherently links them.In this paper,we survey the developments to date on this topic and briefly discuss our views on how to harness machine learning(deep learning in particular)for further advancements in the field.
基金supported by the National Natural Science Foundation of China(NSFC)(Grant No.81930048)the Hong Kong Innovation and Technology Commission(Grant Nos.GHP/043/19SZ and GHP/044/19GD)+3 种基金the Hong Kong Research Grant Council(Grant Nos.15217721,R5029-19 and C7074-21GF)the Guangdong Science and Technology Commission(Grant No.2019BT02X105)the Shenzhen Science and Technology Innovation Commission(Grant No.JCYJ20220818100202005)the Hong Kong Polytechnic University(Grant Nos.P0038180,P0039517,P0043485 and P0045762).
文摘Transmission matrix(TM)allows light control through complex media,such as multimode fibers(MMFs),gaining great attention in areas,such as biophotonics,over the past decade.Efforts have been taken to retrieve a complex-valued TM directly from intensity measurements with several representative phase-retrieval algorithms,which still see limitations of slow or suboptimum recovery,especially under noisy environments.Here,we propose a modified nonconvex optimization approach.Through numerical evaluations,it shows that the optimum focusing efficiency is approached with less running time or sampling ratio.The comparative tests under different signal-to-noise levels further indicate its improved robustness.Experimentally,the superior focusing performance of our algorithm is collectively validated by single-and multispot focusing;especially with a sampling ratio of 8,it achieves a 93.6%efficiency of the gold-standard holography method.Based on the recovered TM,image transmission through an MMF is realized with high fidelity.Due to parallel operation and GPU acceleration,our nonconvex approach retrieves a 8685×1024 TM(sampling ratio is 8)with 42.3 s on average on a regular computer.The proposed method provides optimum efficiency and fast execution for TM retrieval that avoids the need for an external reference beam,which will facilitate applications of deep-tissue optical imaging,manipulation,and treatment.
基金supported by National Natural Science Foundation of China(NSFC)(81930048,81627805)Hong Kong Research Grant Council(15217721,R5029-19,C7074-21GF)+3 种基金Hong Kong Innovation and Technology Commission(GHP/043/19SZ,GHP/044/19GD)Guangdong Science and Technology Commission(2019A1515011374,2019BT02X105)National Research Foundation of Korea(2015R1A3A2066550,2021R1A2C3012903)Institute of Information&Communications Technology Planning&Evaluation(IITP,2021-0-00745)grant funded by the Korea government(MSIT).
文摘Optical techniques offer a wide variety of applications as light-matter interactions provide extremely sensitive mechanisms to probe or treat target media.Most of these implementations rely on the usage of ballistic or quasi-ballistic photons to achieve high spatial resolution.However,the inherent scattering nature of light in biological tissues or tissue-like scattering media constitutes a critical obstacle that has restricted the penetration depth of non-scattered photons and hence limited the implementation of most optical techniques for wider applications.In addition,the components of an optical system are usually designed and manufactured for a fixed function or performance.Recent advances in wavefront shaping have demonstrated that scattering-or component-induced phase distortions can be compensated by optimizing the wavefront of the input light pattern through iteration or by conjugating the transmission matrix of the scattering medium.
基金Agency for Science,Technology and Research(A18A7b0058)Innovation and Technology Commission(GHP/043/19SZ,GHP/044/19GD)+2 种基金Hong Kong Research Grant Council(15217721,C5078-21EF,R5029-19)Guangdong Science and Technology Department(2019A1515011374,2019BT02X105)National Natural Science Foundation of China(81627805,81930048)。
文摘Information retrieval from visually random optical speckle patterns is desired in many scenarios yet considered challenging.It requires accurate understanding or mapping of the multiple scattering process,or reliable capability to reverse or compensate for the scattering-induced phase distortions.In whatever situation,effective resolving and digitization of speckle patterns are necessary.Nevertheless,on some occasions,to increase the acquisition speed and/or signal-to-noise ratio(SNR),speckles captured by cameras are inevitably sampled in the sub-Nyquist domain via pixel binning(one camera pixel contains multiple speckle grains)due to finite size or limited bandwidth of photosensors.Such a down-sampling process is irreversible;it undermines the fine structures of speckle grains and hence the encoded information,preventing successful information extraction.To retrace the lost information,super-resolution interpolation for such sub-Nyquist sampled speckles is needed.In this work,a deep neural network,namely SpkSRNet,is proposed to effectively up sample speckles that are sampled below 1/10 of the Nyquist criterion to well-resolved ones that not only resemble the comprehensive morphology of original speckles(decompose multiple speckle grains from one camera pixel)but also recover the lost complex information(human face in this study)with high fidelity under normal-and low-light conditions,which is impossible with classic interpolation methods.These successful speckle super-resolution interpolation demonstrations are essentially enabled by the strong implicit correlation among speckle grains,which is non-quantifiable but could be discovered by the well-trained network.With further engineering,the proposed learning platform may benefit many scenarios that are physically inaccessible,enabling fast acquisition of speckles with sufficient SNR and opening up new avenues for seeing big and seeing clearly simultaneously in complex scenarios.
基金National Natural Science Foundation of China(81930048)Guangdong Science and Technology Department(2019BT02X105)+2 种基金Research Grants Council,University Grants Committee(15217721,C7074-21GF,R5029-19)Innovation and Technology Commission(GHP/043/19SZ,GHP/044/19GD)Hong Kong Polytechnic University(P0038180,P0039517,P0043485)。
文摘Imaging through scattering media is valuable for many areas,such as biomedicine and communication.Recent progress enabled by deep learning(DL)has shown superiority especially in the model generalization.However,there is a lack of research to physically reveal the origin or define the boundary for such model scalability,which is important for utilizing DL approaches for scalable imaging despite scattering with high confidence.In this paper,we find the amount of the ballistic light component in the output field is the prerequisite for endowing a DL model with generalization capability by using a“one-to-all”training strategy,which offers a physical meaning invariance among the multisource data.The findings are supported by both experimental and simulated tests in which the roles of scattered and ballistic components are revealed in contributing to the origin and physical boundary of the model scalability.Experimentally,the generalization performance of the network is enhanced by increasing the portion of ballistic photons in detection.The mechanism understanding and practical guidance by our research are beneficial for developing DL methods for descattering with high adaptivity.
基金The Hong Kong Polytechnic University(P0038180,P0039517,P0043485,P0045762)Shenzhen Science and Technology Innovation Program(JCYJ20220818100202005)+3 种基金Guangdong Science and Technology Department(2019BT02X105)Hong Kong Research Grant Council(15217721,C7074-21GF,R5029-19)Innovation and Technology Commission(GHP/043/19SZ,GHP/044/19GD)National Natural Science Foundation of China(81930048)。
文摘Multimode fibers(MMFs)are a promising solution for high-throughput signal transmission in the time domain.However,crosstalk among different optical modes within the MMF scrambles input information and creates seemingly random speckle patterns at the output.To characterize this process,a transmission matrix(TM)can be used to relate input and output fields.Recent innovations use TMs to manipulate the output field by shaping the input wavefront for exciting advances in deep-brain imaging,neuron stimulation,quantum networks,and analog operators.However,these approaches consider input/output segments as independent,limiting their use for separate signal processing,such as logic operations.Our proposed method,which makes input/output segments as interdependent,adjusts the phase of corresponding output fields using phase bias maps superimposed on input segments.Coherent superposition enables signal logic operations through a 15-m-long MMF.In experiments,a single optical logic gate containing three basic logic functions and cascading multiple logic gates to handle binary operands is demonstrated.Bitwise operations are performed for multi-bit logic operations,and multiple optical logic gates are reconstructed simultaneously in a single logic gate with polarization multiplexing.The proposed method may open new avenues for long-range logic signal processing and transmission via MMFs.
文摘Multiple scattering can significantly scramble the amplitude and phase profile of an optical field.It obscures subtle observations but only speckle patterns can be seen,unlike the ballistic regime where the information or the optical field can be identified with limited distortions.Efficient optical manipulation including information transmission and precise focusing is therefore obstructed as light travels deep into turbidmedia such as fog,turbid fluids,and biological tissues.
