Importance:Precisely decoding brain dysfunction from high-dimensional functional recordings is crucial for advancing our understanding of brain dysfunction in brain disorders.Self-supervised learning(SSL)models offer ...Importance:Precisely decoding brain dysfunction from high-dimensional functional recordings is crucial for advancing our understanding of brain dysfunction in brain disorders.Self-supervised learning(SSL)models offer a transformative approach for mapping dependencies in functional neuroimaging data.Leveraging the intrinsic organization of brain signals for comprehensive feature extraction,these models enable the analysis of critical neurofunctional features within a clinically relevant framework,overcoming challenges related to data heterogeneity and the scarcity of labeled data.Highlight:This paper provides a comprehensive overview of SSL techniques applied to functional neuroimaging data,such as functional magnetic resonance imaging and electroencephalography,with a specific focus on their applications in various neuropsychiatric disorders.We discuss 3 main categories of SSL methods:contrastive learning,generative learning,and generative-contrastive methods,outlining their basic principles and representative methods.Critically,we highlight the potential of SSL in addressing data scarcity,multimodal integration,and dynamic network modeling for disease detection and prediction.We showcase successful applications of these techniques in understanding and classifying conditions such as Alzheimer’s disease,Parkinson’s disease,and epilepsy,demonstrating their potential in downstream neuropsychological applications.Conclusion:SSL models provide a scalable and effective methodology for individual detection and prediction in brain disorders.Despite current limitations in interpretability and data heterogeneity,the potential of SSL for future clinical applications,particularly in the areas of transdiagnostic psychosis subtyping and decoding task-based brain functional recordings,is substantial.展开更多
Contrastive self‐supervised representation learning on attributed graph networks with Graph Neural Networks has attracted considerable research interest recently.However,there are still two challenges.First,most of t...Contrastive self‐supervised representation learning on attributed graph networks with Graph Neural Networks has attracted considerable research interest recently.However,there are still two challenges.First,most of the real‐word system are multiple relations,where entities are linked by different types of relations,and each relation is a view of the graph network.Second,the rich multi‐scale information(structure‐level and feature‐level)of the graph network can be seen as self‐supervised signals,which are not fully exploited.A novel contrastive self‐supervised representation learning framework on attributed multiplex graph networks with multi‐scale(named CoLM^(2)S)information is presented in this study.It mainly contains two components:intra‐relation contrast learning and interrelation contrastive learning.Specifically,the contrastive self‐supervised representation learning framework on attributed single‐layer graph networks with multi‐scale information(CoLMS)framework with the graph convolutional network as encoder to capture the intra‐relation information with multi‐scale structure‐level and feature‐level selfsupervised signals is introduced first.The structure‐level information includes the edge structure and sub‐graph structure,and the feature‐level information represents the output of different graph convolutional layer.Second,according to the consensus assumption among inter‐relations,the CoLM^(2)S framework is proposed to jointly learn various graph relations in attributed multiplex graph network to achieve global consensus node embedding.The proposed method can fully distil the graph information.Extensive experiments on unsupervised node clustering and graph visualisation tasks demonstrate the effectiveness of our methods,and it outperforms existing competitive baselines.展开更多
Fluorescence microscopy has become one of the most widely employed in vivo imaging modalities,enabling the discovery of new biopathological mechanisms.However,the application of fluorescence imaging is often hindered ...Fluorescence microscopy has become one of the most widely employed in vivo imaging modalities,enabling the discovery of new biopathological mechanisms.However,the application of fluorescence imaging is often hindered by signal-to-noise ratio issues owing to inherent noise arising from various systemic and biophysical characteristics.These limitations pose a growing challenge,especially with the desire to elucidate dynamic biomechanisms at previously unreachable rapid speeds.Here,we propose a temporal gradient(TG)-based self-supervised denoising network(TeD)that could enable an unprecedented advance in spatially dynamic fluorescence imaging.Our strategy is predicated on the insight that judicious utilization of spatiotemporal information is more advantageous for denoising predictions.Adopting the TG,which intrinsically embodies spatial dynamic features,enables TeD to prudently focus on spatiotemporal information.We showed that TeD can provide new interpretative opportunities for understanding dynamic fluorescence signals in in vivo imaging of mice,representing cellular flow.Furthermore,we demonstrated that TeD is robust even when fluorescence signals exhibit temporal kinetics without spatial dynamics,as seen in neuronal population imaging.We believe that TeD’s superior performance even with spatially dynamic samples,including the complex behavior of cells or organisms,could make a substantial contribution to various biological studies.展开更多
Scanning Electron Microscopes(SEMs)are widely used in experimental science laboratories,often requiring cumbersome and repetitive user analysis.Automating SEM image analysis processes is highly desirable to address th...Scanning Electron Microscopes(SEMs)are widely used in experimental science laboratories,often requiring cumbersome and repetitive user analysis.Automating SEM image analysis processes is highly desirable to address this challenge.In particle sample analysis,Machine Learning(ML)has emerged as the most effective approach for particle segmentation.However,the time-intensive process of manually annotating thousands of SEM images limits the applicability of supervised learning approaches.Self-Supervised Learning(SSL)offers a promising alternative by enabling knowledge extraction from raw,unlabeled data.This study presents a framework for evaluating SSL techniques in SEM image analysis,focusing on novel methods leveraging the ConvNeXtV2 architecture for particle detection.A dataset comprising 25,000 SEM images is curated to benchmark these proposed SSL methods.The results demonstrate that ConvNeXtV2 models,with varying parameter counts,consistently outperform other techniques in particle detection across different length scales,achieving up to a34%reduction in relative error compared to established SSL methods.Furthermore,an ablation study explores the relationship between dataset size and SSL performance,providing actionable insights for practitioners regarding model selection and resource efficiency.This research advances the integration of SSL into autonomous analysis pipelines and supports its application in accelerating materials science discovery.展开更多
Volumetric imaging is increasingly in demand for its precision in statistically visualizing and analyzing the intricacies of biological phenomena.To visualize the intricate details of these minute structures and facil...Volumetric imaging is increasingly in demand for its precision in statistically visualizing and analyzing the intricacies of biological phenomena.To visualize the intricate details of these minute structures and facilitate the analysis in biomedical research,high-signal-to-noise ratio(SNR)images are indispensable.However,the inevitable noise presents a significant barrier to imaging qualities.Here,we propose SelfMirror,a self-supervised deep-learning denoising method for volumetric image reconstruction.SelfMirror is developed based on the insight that the variation of biological structure is continuous and smooth;when the sampling interval in volumetric imaging is sufficiently small,the similarity of neighboring slices in terms of the spatial structure becomes apparent.Such similarity can be used to train our proposed network to revive the signals and suppress the noise accurately.The denoising performance of SelfMirror exhibits remarkable robustness and fidelity even in extremely low-SNR conditions.We demonstrate the broad applicability of SelfMirror on multiple imaging modalities,including two-photon microscopy,confocal microscopy,expansion microscopy,computed tomography,and 3D electron microscopy.This versatility extends from single neuron cells to tissues and organs,highlighting SelfMirror's potential for integration into diverse imaging and analysis pipelines.展开更多
Malignant and premalignant ocular surface tumors(OSTs)can be sight-threatening or even life-threatening if not diagnosed and treated promptly.Artificial intelligence holds great promise for the early detection of thes...Malignant and premalignant ocular surface tumors(OSTs)can be sight-threatening or even life-threatening if not diagnosed and treated promptly.Artificial intelligence holds great promise for the early detection of these diseases.However,training traditional convolutional neural networks(CNNs)for this task presents challenges due to the lack of large,well-annotated datasets containing OST images labeled according to histopathological results.Here,we introduce the ocular surface pretrained model(OSPM),a domain-specific pretrained model designed to address the scarcity of labeled data.OSPM is constructed utilizing self-supervised learning on approximately 0.76 million unlabeled ocular surface images from 10 clinical centers across China and can be readily adapted to the OST classification task.We then develop and evaluate an OSPM-enhanced classification model(OECM)using 1,455 OST images labeled with histopathological diagnoses to differentiate between malignant,premalignant,and benign OSTs.OECM achieves excellent performance with AUROCs ranging from 0.891 to 0.993 on internal,external,and prospective test datasets,significantly outperforming the traditional CNN models.OECM demonstrated performance comparable to that of senior ophthalmologists and increased the diagnostic accuracy of junior ophthalmologists.Greater label efficiency was observed in OECM compared to CNN models.Our proposed model has high potential to enhance the early detection and treatment of malignant and premalignant OSTs,thereby reducing cancer-related mortality and optimizing functional outcomes.展开更多
基金supported by grants from the National Natural Science Foundation of P.R.China(62276081 and 62106113)Guangdong Basic and Applied Basic Research Foundation(2023A1515010792 and 2023B1515120065)Shenzhen Science and Technology Program(GXWD20231129121139001 and JCYJ20240813110522029).
文摘Importance:Precisely decoding brain dysfunction from high-dimensional functional recordings is crucial for advancing our understanding of brain dysfunction in brain disorders.Self-supervised learning(SSL)models offer a transformative approach for mapping dependencies in functional neuroimaging data.Leveraging the intrinsic organization of brain signals for comprehensive feature extraction,these models enable the analysis of critical neurofunctional features within a clinically relevant framework,overcoming challenges related to data heterogeneity and the scarcity of labeled data.Highlight:This paper provides a comprehensive overview of SSL techniques applied to functional neuroimaging data,such as functional magnetic resonance imaging and electroencephalography,with a specific focus on their applications in various neuropsychiatric disorders.We discuss 3 main categories of SSL methods:contrastive learning,generative learning,and generative-contrastive methods,outlining their basic principles and representative methods.Critically,we highlight the potential of SSL in addressing data scarcity,multimodal integration,and dynamic network modeling for disease detection and prediction.We showcase successful applications of these techniques in understanding and classifying conditions such as Alzheimer’s disease,Parkinson’s disease,and epilepsy,demonstrating their potential in downstream neuropsychological applications.Conclusion:SSL models provide a scalable and effective methodology for individual detection and prediction in brain disorders.Despite current limitations in interpretability and data heterogeneity,the potential of SSL for future clinical applications,particularly in the areas of transdiagnostic psychosis subtyping and decoding task-based brain functional recordings,is substantial.
基金support by the National Natural Science Foundation of China(NSFC)under grant number 61873274.
文摘Contrastive self‐supervised representation learning on attributed graph networks with Graph Neural Networks has attracted considerable research interest recently.However,there are still two challenges.First,most of the real‐word system are multiple relations,where entities are linked by different types of relations,and each relation is a view of the graph network.Second,the rich multi‐scale information(structure‐level and feature‐level)of the graph network can be seen as self‐supervised signals,which are not fully exploited.A novel contrastive self‐supervised representation learning framework on attributed multiplex graph networks with multi‐scale(named CoLM^(2)S)information is presented in this study.It mainly contains two components:intra‐relation contrast learning and interrelation contrastive learning.Specifically,the contrastive self‐supervised representation learning framework on attributed single‐layer graph networks with multi‐scale information(CoLMS)framework with the graph convolutional network as encoder to capture the intra‐relation information with multi‐scale structure‐level and feature‐level selfsupervised signals is introduced first.The structure‐level information includes the edge structure and sub‐graph structure,and the feature‐level information represents the output of different graph convolutional layer.Second,according to the consensus assumption among inter‐relations,the CoLM^(2)S framework is proposed to jointly learn various graph relations in attributed multiplex graph network to achieve global consensus node embedding.The proposed method can fully distil the graph information.Extensive experiments on unsupervised node clustering and graph visualisation tasks demonstrate the effectiveness of our methods,and it outperforms existing competitive baselines.
基金supported by the National Research Foundation of Korea(NRF)funded by the Ministry of Education,Science and Technology(NRF-RS-2023-00208888 and NRF-RS-2024-00401786)by a Korea Medical Device Development Fund grant funded by the Korean government(Ministry of Science and Information and Communication Technologies,Ministry of Trade,Industry and Energy,Ministry of Health and Welfare,Ministry of Food and Drug Safety)(RS-2023-00254566).
文摘Fluorescence microscopy has become one of the most widely employed in vivo imaging modalities,enabling the discovery of new biopathological mechanisms.However,the application of fluorescence imaging is often hindered by signal-to-noise ratio issues owing to inherent noise arising from various systemic and biophysical characteristics.These limitations pose a growing challenge,especially with the desire to elucidate dynamic biomechanisms at previously unreachable rapid speeds.Here,we propose a temporal gradient(TG)-based self-supervised denoising network(TeD)that could enable an unprecedented advance in spatially dynamic fluorescence imaging.Our strategy is predicated on the insight that judicious utilization of spatiotemporal information is more advantageous for denoising predictions.Adopting the TG,which intrinsically embodies spatial dynamic features,enables TeD to prudently focus on spatiotemporal information.We showed that TeD can provide new interpretative opportunities for understanding dynamic fluorescence signals in in vivo imaging of mice,representing cellular flow.Furthermore,we demonstrated that TeD is robust even when fluorescence signals exhibit temporal kinetics without spatial dynamics,as seen in neuronal population imaging.We believe that TeD’s superior performance even with spatially dynamic samples,including the complex behavior of cells or organisms,could make a substantial contribution to various biological studies.
基金funded by the German Research Foundation(DFG)under Project ID 390874152(POLiS Cluster of Excellence)N.J.S.,A.G.,G.C.,O.D.,A.J.were supported by the D2S2 program within the U.S.Department of Energy,Office of Basic Energy Sciences,Materials Sciences and Engineering Division under Contract No.DE-AC02-05-CH11231(D2S2 program,KCD2S2)A.G.acknowledges support from the Swiss National Science Foundation(SNSF,project#P500PN_222166).
文摘Scanning Electron Microscopes(SEMs)are widely used in experimental science laboratories,often requiring cumbersome and repetitive user analysis.Automating SEM image analysis processes is highly desirable to address this challenge.In particle sample analysis,Machine Learning(ML)has emerged as the most effective approach for particle segmentation.However,the time-intensive process of manually annotating thousands of SEM images limits the applicability of supervised learning approaches.Self-Supervised Learning(SSL)offers a promising alternative by enabling knowledge extraction from raw,unlabeled data.This study presents a framework for evaluating SSL techniques in SEM image analysis,focusing on novel methods leveraging the ConvNeXtV2 architecture for particle detection.A dataset comprising 25,000 SEM images is curated to benchmark these proposed SSL methods.The results demonstrate that ConvNeXtV2 models,with varying parameter counts,consistently outperform other techniques in particle detection across different length scales,achieving up to a34%reduction in relative error compared to established SSL methods.Furthermore,an ablation study explores the relationship between dataset size and SSL performance,providing actionable insights for practitioners regarding model selection and resource efficiency.This research advances the integration of SSL into autonomous analysis pipelines and supports its application in accelerating materials science discovery.
基金National Natural Science Foundation of China(62027812,62333012)。
文摘Volumetric imaging is increasingly in demand for its precision in statistically visualizing and analyzing the intricacies of biological phenomena.To visualize the intricate details of these minute structures and facilitate the analysis in biomedical research,high-signal-to-noise ratio(SNR)images are indispensable.However,the inevitable noise presents a significant barrier to imaging qualities.Here,we propose SelfMirror,a self-supervised deep-learning denoising method for volumetric image reconstruction.SelfMirror is developed based on the insight that the variation of biological structure is continuous and smooth;when the sampling interval in volumetric imaging is sufficiently small,the similarity of neighboring slices in terms of the spatial structure becomes apparent.Such similarity can be used to train our proposed network to revive the signals and suppress the noise accurately.The denoising performance of SelfMirror exhibits remarkable robustness and fidelity even in extremely low-SNR conditions.We demonstrate the broad applicability of SelfMirror on multiple imaging modalities,including two-photon microscopy,confocal microscopy,expansion microscopy,computed tomography,and 3D electron microscopy.This versatility extends from single neuron cells to tissues and organs,highlighting SelfMirror's potential for integration into diverse imaging and analysis pipelines.
基金funding from the National Natural Science Foundation of China(grant nos.82201148 and 62276210)the Natural Science Foundation of Zhejiang Province(grant no.LQ22H120002)+2 种基金the Medical Health Science and Technology Project of Zhejiang Province(grant no.2023KY1140)the Natural Science Foundation of Ningbo(grant no.2023J390)the Ningbo Top Medical and Health Research Program(grant no.2023030716).
文摘Malignant and premalignant ocular surface tumors(OSTs)can be sight-threatening or even life-threatening if not diagnosed and treated promptly.Artificial intelligence holds great promise for the early detection of these diseases.However,training traditional convolutional neural networks(CNNs)for this task presents challenges due to the lack of large,well-annotated datasets containing OST images labeled according to histopathological results.Here,we introduce the ocular surface pretrained model(OSPM),a domain-specific pretrained model designed to address the scarcity of labeled data.OSPM is constructed utilizing self-supervised learning on approximately 0.76 million unlabeled ocular surface images from 10 clinical centers across China and can be readily adapted to the OST classification task.We then develop and evaluate an OSPM-enhanced classification model(OECM)using 1,455 OST images labeled with histopathological diagnoses to differentiate between malignant,premalignant,and benign OSTs.OECM achieves excellent performance with AUROCs ranging from 0.891 to 0.993 on internal,external,and prospective test datasets,significantly outperforming the traditional CNN models.OECM demonstrated performance comparable to that of senior ophthalmologists and increased the diagnostic accuracy of junior ophthalmologists.Greater label efficiency was observed in OECM compared to CNN models.Our proposed model has high potential to enhance the early detection and treatment of malignant and premalignant OSTs,thereby reducing cancer-related mortality and optimizing functional outcomes.
