Recent advancements in artificial intelligence have transformed three-dimensional(3D)optical imaging and metrology,enabling high-resolution and high-precision 3D surface geometry measurements from one single fringe pa...Recent advancements in artificial intelligence have transformed three-dimensional(3D)optical imaging and metrology,enabling high-resolution and high-precision 3D surface geometry measurements from one single fringe pattern projection.However,the imaging speed of conventional fringe projection profilometry(FPP)remains limited by the native sensor refresh rates due to the inherent"one-to-one"synchronization mechanism between pattern projection and image acquisition in standard structured light techniques.Here,we present dual-frequency angular-multiplexed fringe projection profilometry(DFAMFPP),a deep learning-enabled 3D imaging technique that achieves high-speed,high-precision,and large-depth-range absolute 3D surface measurements at speeds 16 times faster than the sensor's native frame rate.By encoding multi-timeframe 3D information into a single multiplexed image using multiple pairs of dual-frequency fringes,high-accuracy absolute phase maps are reconstructed using specially trained two-stage number-theoretical-based deep neural networks.We validate the effectiveness of DFAMFPP through dynamic scene measurements,achieving 10,000 Hz 3D imaging of a running turbofan engine prototype with only a 625 Hz camera.By overcoming the sensor hardware bottleneck,DFAMFPP significantly advances high-speed and ultra-high-speed 3D imaging,opening new avenues for exploring dynamic processes across diverse scientific disciplines.展开更多
Quantitative phase imaging(QPI)enables non-invasive cellular analysis by utilizing cell thickness and refractive index as intrinsic probes,revolutionizing label-free microscopy in cellular research.Differential phase ...Quantitative phase imaging(QPI)enables non-invasive cellular analysis by utilizing cell thickness and refractive index as intrinsic probes,revolutionizing label-free microscopy in cellular research.Differential phase contrast(DPC),a non-interferometric QPI technique,requires only four intensity images under asymmetric illumination to recover the phase of a sample,offering the advantages of being label-free,non-coherent and highly robust.Its phase reconstruction result relies on precise modeling of the phase transfer function(PTF).However,in real optical systems,the PTF will deviate from its theoretical ideal due to the unknown wavefront aberrations,which will lead to significant artifacts and distortions in the reconstructed phase.We propose an aberration-corrected DPC(ACDPC)method that utilizes three intensity images under annular illumination to jointly retrieve the aberration and the phase,achieving high-quality QPI with minimal raw data.By employing three annular illuminations precisely matched to the numerical aperture of the objective lens,the object information is transmitted into the acquired intensity with a high signal-to-noise ratio.Phase retrieval is achieved by an iterative deconvolution algorithm that uses simulated annealing to estimate the aberration and further employs regularized deconvolution to reconstruct the phase,ultimately obtaining a refined complex pupil function and an aberration-corrected quantitative phase.We demonstrate that ACDPC is robust to multi-order aberrations without any priori knowledge,and can effectively retrieve and correct system aberrations to obtain high-quality quantitative phase.Experimental results show that ACDPC can clearly reproduce subcellular structures such as vesicles and lipid droplets with higher resolution than conventional DPC,which opens up new possibilities for more accurate subcellular structure analysis in cell biology.展开更多
Oil spill monitoring in remote sensing field has become a very popular technology to detect the spatial distribution of polluted regions.However,previous studies mainly focus on the supervised detection technologies,w...Oil spill monitoring in remote sensing field has become a very popular technology to detect the spatial distribution of polluted regions.However,previous studies mainly focus on the supervised detection technologies,which requires a large number of high-quality training set.To solve this problem,we propose a self-supervised learning method to learn the deep neural network from unlabelled hyperspectral data for oil spill detection,which consists of three parts:data augmentation,unsupervised deep feature learning,and oil spill detection network.First,the original image is augmented with spectral and spatial transformation to improve robustness of the self-supervised model.Then,the deep neural networks are trained on the augmented data without label information to produce the high-level semantic features.Finally,the pre-trained parameters are transferred to establish a neural network classifier to obtain the detection result,where a contrastive loss is developed to fine-tune the learned parameters so as to improve the generalization ability of the proposed method.Experiments performed on ten oil spill datasets reveal that the proposed method obtains promising detection performance with respect to other state-of-the-art hyperspectral detection approaches.展开更多
Infrared(IR)vibrational spectroscopy provides rich molecular fingerprint information for label-free detection of chemical bonds and conformational structures,with broad applications in biochemistry,pharmacology,and ma...Infrared(IR)vibrational spectroscopy provides rich molecular fingerprint information for label-free detection of chemical bonds and conformational structures,with broad applications in biochemistry,pharmacology,and materials analysis.1-3 However,its practical utility in complex,especially aqueous,environments remains constrained by two fundamental limitations:the intrinsically weak vibrational absorption cross-sections of typical biomolecules and the poor spatial resolution dictated by mid-infrared(mid-IR)wavelengths,exacerbated by strong water absorption.展开更多
In the current work,we explored a new knowledge amalgama-tion problem,termed Federated Selective Aggregation for on-device knowledge amalgamation(FedSA).FedSA aims to train an on-device student model for a new task wi...In the current work,we explored a new knowledge amalgama-tion problem,termed Federated Selective Aggregation for on-device knowledge amalgamation(FedSA).FedSA aims to train an on-device student model for a new task with the help of several decentralized teachers whose pre-training tasks and data are different and agnos-tic.The motivation to investigate such a problem setup stems from a recent dilemma of model sharing.Due to privacy,security or in-tellectual property issues,the pre-trained models are,however,not able to be shared,and the resources of devices are usually limited.The proposed FedSA offers a solution to this dilemma and makes it one step further,again,the method can be employed on low-power and resource-limited devices.To this end,a dedicated strategy was proposed to handle the knowledge amalgamation.Specifically,the student-training process in the current work was driven by a novel saliency-based approach which adaptively selects teachers as the par-ticipants and integrated their representative capabilities into the stu-dent.To evaluate the effectiveness of FedSA,experiments on both single-task and multi-task settings were conducted.The experimental results demonstrate that FedSA could effectively amalgamate knowl-edge from decentralized models and achieve competitive performance to centralized baselines.展开更多
基金supported by National Key Research and Development Program of China(2022YFB2804603,2022YFB2804605)National Natural Science Foundation of China(U21B2033)+4 种基金Fundamental Research Funds forthe Central Universities(2023102001,2024202002)National Key Laborato-ry of Shock Wave and Detonation Physics(JCKYS2024212111)China Post-doctoral Science Fund(2023T160318)Open Research Fund of JiangsuKey Laboratory of Spectral Imaging&Intelligent Sense(JSGP202105,JSGP202201)Postgraduate Research&Practice Innovation Program of Jiangsu Province(KYCX25_0695,SJCX25_0188)。
文摘Recent advancements in artificial intelligence have transformed three-dimensional(3D)optical imaging and metrology,enabling high-resolution and high-precision 3D surface geometry measurements from one single fringe pattern projection.However,the imaging speed of conventional fringe projection profilometry(FPP)remains limited by the native sensor refresh rates due to the inherent"one-to-one"synchronization mechanism between pattern projection and image acquisition in standard structured light techniques.Here,we present dual-frequency angular-multiplexed fringe projection profilometry(DFAMFPP),a deep learning-enabled 3D imaging technique that achieves high-speed,high-precision,and large-depth-range absolute 3D surface measurements at speeds 16 times faster than the sensor's native frame rate.By encoding multi-timeframe 3D information into a single multiplexed image using multiple pairs of dual-frequency fringes,high-accuracy absolute phase maps are reconstructed using specially trained two-stage number-theoretical-based deep neural networks.We validate the effectiveness of DFAMFPP through dynamic scene measurements,achieving 10,000 Hz 3D imaging of a running turbofan engine prototype with only a 625 Hz camera.By overcoming the sensor hardware bottleneck,DFAMFPP significantly advances high-speed and ultra-high-speed 3D imaging,opening new avenues for exploring dynamic processes across diverse scientific disciplines.
基金supported by the National Natural Science Foundation of China(62305162,62227818,62361136588)China Postdoctoral Science Foundation(2023TQ0160,2023M731683)+5 种基金Nanjing University of Science and Technology independent research project(30923010305)National Key Research and Development Program of China(2024YFE0101300)Biomedical Competition Foundation of Jiangsu Province(BE2022847)Key National Industrial Technology Cooperation Foundation of Jiangsu Province(BZ2022039)Fundamental Research Funds for the Central Universities(2023102001)Open Research Fund of Jiangsu Key Laboratory of Spectral Imaging&Intelligent Sense(JSGP202105,JSGP202201,JSGPCXZNGZ202401)。
文摘Quantitative phase imaging(QPI)enables non-invasive cellular analysis by utilizing cell thickness and refractive index as intrinsic probes,revolutionizing label-free microscopy in cellular research.Differential phase contrast(DPC),a non-interferometric QPI technique,requires only four intensity images under asymmetric illumination to recover the phase of a sample,offering the advantages of being label-free,non-coherent and highly robust.Its phase reconstruction result relies on precise modeling of the phase transfer function(PTF).However,in real optical systems,the PTF will deviate from its theoretical ideal due to the unknown wavefront aberrations,which will lead to significant artifacts and distortions in the reconstructed phase.We propose an aberration-corrected DPC(ACDPC)method that utilizes three intensity images under annular illumination to jointly retrieve the aberration and the phase,achieving high-quality QPI with minimal raw data.By employing three annular illuminations precisely matched to the numerical aperture of the objective lens,the object information is transmitted into the acquired intensity with a high signal-to-noise ratio.Phase retrieval is achieved by an iterative deconvolution algorithm that uses simulated annealing to estimate the aberration and further employs regularized deconvolution to reconstruct the phase,ultimately obtaining a refined complex pupil function and an aberration-corrected quantitative phase.We demonstrate that ACDPC is robust to multi-order aberrations without any priori knowledge,and can effectively retrieve and correct system aberrations to obtain high-quality quantitative phase.Experimental results show that ACDPC can clearly reproduce subcellular structures such as vesicles and lipid droplets with higher resolution than conventional DPC,which opens up new possibilities for more accurate subcellular structure analysis in cell biology.
基金supported by the National Natural Science Foundation of China (Grant No. 61890962 and 61871179)the Scientific Research Project of Hunan Education Department (Grant No. 19B105)+3 种基金the Natural Science Foundation of Hunan Province (Grant Nos. 2019JJ50036 and 2020GK2038)the National Key Research and Development Project (Grant No. 2021YFA0715203)the Hunan Provincial Natural Science Foundation for Distinguished Young Scholars (Grant No. 2021JJ022)the Huxiang Young Talents Science and Technology Innovation Program (Grant No. 2020RC3013)
文摘Oil spill monitoring in remote sensing field has become a very popular technology to detect the spatial distribution of polluted regions.However,previous studies mainly focus on the supervised detection technologies,which requires a large number of high-quality training set.To solve this problem,we propose a self-supervised learning method to learn the deep neural network from unlabelled hyperspectral data for oil spill detection,which consists of three parts:data augmentation,unsupervised deep feature learning,and oil spill detection network.First,the original image is augmented with spectral and spatial transformation to improve robustness of the self-supervised model.Then,the deep neural networks are trained on the augmented data without label information to produce the high-level semantic features.Finally,the pre-trained parameters are transferred to establish a neural network classifier to obtain the detection result,where a contrastive loss is developed to fine-tune the learned parameters so as to improve the generalization ability of the proposed method.Experiments performed on ten oil spill datasets reveal that the proposed method obtains promising detection performance with respect to other state-of-the-art hyperspectral detection approaches.
文摘Infrared(IR)vibrational spectroscopy provides rich molecular fingerprint information for label-free detection of chemical bonds and conformational structures,with broad applications in biochemistry,pharmacology,and materials analysis.1-3 However,its practical utility in complex,especially aqueous,environments remains constrained by two fundamental limitations:the intrinsically weak vibrational absorption cross-sections of typical biomolecules and the poor spatial resolution dictated by mid-infrared(mid-IR)wavelengths,exacerbated by strong water absorption.
基金supported by National Natural Science Foundation of China (61976186,U20B2066)the Fundamental Research Funds for the Central Universities (2021FZZX001-23,226-2023-00048).
文摘In the current work,we explored a new knowledge amalgama-tion problem,termed Federated Selective Aggregation for on-device knowledge amalgamation(FedSA).FedSA aims to train an on-device student model for a new task with the help of several decentralized teachers whose pre-training tasks and data are different and agnos-tic.The motivation to investigate such a problem setup stems from a recent dilemma of model sharing.Due to privacy,security or in-tellectual property issues,the pre-trained models are,however,not able to be shared,and the resources of devices are usually limited.The proposed FedSA offers a solution to this dilemma and makes it one step further,again,the method can be employed on low-power and resource-limited devices.To this end,a dedicated strategy was proposed to handle the knowledge amalgamation.Specifically,the student-training process in the current work was driven by a novel saliency-based approach which adaptively selects teachers as the par-ticipants and integrated their representative capabilities into the stu-dent.To evaluate the effectiveness of FedSA,experiments on both single-task and multi-task settings were conducted.The experimental results demonstrate that FedSA could effectively amalgamate knowl-edge from decentralized models and achieve competitive performance to centralized baselines.
