The authors regret that there were errors in the affiliations and the funding declaration in the original published version.The affiliations a and b of the original manuscript are"School of Information Engineerin...The authors regret that there were errors in the affiliations and the funding declaration in the original published version.The affiliations a and b of the original manuscript are"School of Information Engineering,Jiangxi Provincial Key Laboratory of Advanced Signal Processing and Intelligent Communications,Nanchang University,Nanchang 330031,China",and"School of Internet of Things Engineering,Jiangnan University,Wuxi 214122,China",respectively.The order of the two affiliations are not correct.展开更多
Importance:Precisely decoding brain dysfunction from high-dimensional functional recordings is crucial for advancing our understanding of brain dysfunction in brain disorders.Self-supervised learning(SSL)models offer ...Importance:Precisely decoding brain dysfunction from high-dimensional functional recordings is crucial for advancing our understanding of brain dysfunction in brain disorders.Self-supervised learning(SSL)models offer a transformative approach for mapping dependencies in functional neuroimaging data.Leveraging the intrinsic organization of brain signals for comprehensive feature extraction,these models enable the analysis of critical neurofunctional features within a clinically relevant framework,overcoming challenges related to data heterogeneity and the scarcity of labeled data.Highlight:This paper provides a comprehensive overview of SSL techniques applied to functional neuroimaging data,such as functional magnetic resonance imaging and electroencephalography,with a specific focus on their applications in various neuropsychiatric disorders.We discuss 3 main categories of SSL methods:contrastive learning,generative learning,and generative-contrastive methods,outlining their basic principles and representative methods.Critically,we highlight the potential of SSL in addressing data scarcity,multimodal integration,and dynamic network modeling for disease detection and prediction.We showcase successful applications of these techniques in understanding and classifying conditions such as Alzheimer’s disease,Parkinson’s disease,and epilepsy,demonstrating their potential in downstream neuropsychological applications.Conclusion:SSL models provide a scalable and effective methodology for individual detection and prediction in brain disorders.Despite current limitations in interpretability and data heterogeneity,the potential of SSL for future clinical applications,particularly in the areas of transdiagnostic psychosis subtyping and decoding task-based brain functional recordings,is substantial.展开更多
Contrastive self‐supervised representation learning on attributed graph networks with Graph Neural Networks has attracted considerable research interest recently.However,there are still two challenges.First,most of t...Contrastive self‐supervised representation learning on attributed graph networks with Graph Neural Networks has attracted considerable research interest recently.However,there are still two challenges.First,most of the real‐word system are multiple relations,where entities are linked by different types of relations,and each relation is a view of the graph network.Second,the rich multi‐scale information(structure‐level and feature‐level)of the graph network can be seen as self‐supervised signals,which are not fully exploited.A novel contrastive self‐supervised representation learning framework on attributed multiplex graph networks with multi‐scale(named CoLM^(2)S)information is presented in this study.It mainly contains two components:intra‐relation contrast learning and interrelation contrastive learning.Specifically,the contrastive self‐supervised representation learning framework on attributed single‐layer graph networks with multi‐scale information(CoLMS)framework with the graph convolutional network as encoder to capture the intra‐relation information with multi‐scale structure‐level and feature‐level selfsupervised signals is introduced first.The structure‐level information includes the edge structure and sub‐graph structure,and the feature‐level information represents the output of different graph convolutional layer.Second,according to the consensus assumption among inter‐relations,the CoLM^(2)S framework is proposed to jointly learn various graph relations in attributed multiplex graph network to achieve global consensus node embedding.The proposed method can fully distil the graph information.Extensive experiments on unsupervised node clustering and graph visualisation tasks demonstrate the effectiveness of our methods,and it outperforms existing competitive baselines.展开更多
Scanning Electron Microscopes(SEMs)are widely used in experimental science laboratories,often requiring cumbersome and repetitive user analysis.Automating SEM image analysis processes is highly desirable to address th...Scanning Electron Microscopes(SEMs)are widely used in experimental science laboratories,often requiring cumbersome and repetitive user analysis.Automating SEM image analysis processes is highly desirable to address this challenge.In particle sample analysis,Machine Learning(ML)has emerged as the most effective approach for particle segmentation.However,the time-intensive process of manually annotating thousands of SEM images limits the applicability of supervised learning approaches.Self-Supervised Learning(SSL)offers a promising alternative by enabling knowledge extraction from raw,unlabeled data.This study presents a framework for evaluating SSL techniques in SEM image analysis,focusing on novel methods leveraging the ConvNeXtV2 architecture for particle detection.A dataset comprising 25,000 SEM images is curated to benchmark these proposed SSL methods.The results demonstrate that ConvNeXtV2 models,with varying parameter counts,consistently outperform other techniques in particle detection across different length scales,achieving up to a34%reduction in relative error compared to established SSL methods.Furthermore,an ablation study explores the relationship between dataset size and SSL performance,providing actionable insights for practitioners regarding model selection and resource efficiency.This research advances the integration of SSL into autonomous analysis pipelines and supports its application in accelerating materials science discovery.展开更多
Deep learning in electron microscopy(EM)data analysis is predominantly supervised,relying on manually labeled data.This dependence limits scalability and slows the development of highthroughput EM characterization of ...Deep learning in electron microscopy(EM)data analysis is predominantly supervised,relying on manually labeled data.This dependence limits scalability and slows the development of highthroughput EM characterization of materials.While simulation-based approaches provide an alternative,they often struggle with morphological heterogeneity,contrast complexity,and experimental artifacts,reducing their real-world effectiveness.Weintroduce EMcopilot,a closed-loop generative learning framework that enables label-free EM segmentation.EMcopilot leverages the general vision model to extract morphological priors and employs a conditional generative adversarial network to generate contrast-aware images.An EM-specificdomain adapter further enhances realism by modeling key microscope-specific perturbations.Benchmark results show that EMcopilot-trained models not only achieve segmentation accuracy comparable to human-annotated models but also outperform them in detecting nanoparticles in poor-contrast regions and spatially clustered configurations,overcoming inherent human biases in annotation.By illustrating how generative models distill and transform complex EM features into a robust training resource in a self-supervised manner,EMcopilot provides a scalable solution for automated microscopy analysis.展开更多
Fluorescence microscopy has become one of the most widely employed in vivo imaging modalities,enabling the discovery of new biopathological mechanisms.However,the application of fluorescence imaging is often hindered ...Fluorescence microscopy has become one of the most widely employed in vivo imaging modalities,enabling the discovery of new biopathological mechanisms.However,the application of fluorescence imaging is often hindered by signal-to-noise ratio issues owing to inherent noise arising from various systemic and biophysical characteristics.These limitations pose a growing challenge,especially with the desire to elucidate dynamic biomechanisms at previously unreachable rapid speeds.Here,we propose a temporal gradient(TG)-based self-supervised denoising network(TeD)that could enable an unprecedented advance in spatially dynamic fluorescence imaging.Our strategy is predicated on the insight that judicious utilization of spatiotemporal information is more advantageous for denoising predictions.Adopting the TG,which intrinsically embodies spatial dynamic features,enables TeD to prudently focus on spatiotemporal information.We showed that TeD can provide new interpretative opportunities for understanding dynamic fluorescence signals in in vivo imaging of mice,representing cellular flow.Furthermore,we demonstrated that TeD is robust even when fluorescence signals exhibit temporal kinetics without spatial dynamics,as seen in neuronal population imaging.We believe that TeD’s superior performance even with spatially dynamic samples,including the complex behavior of cells or organisms,could make a substantial contribution to various biological studies.展开更多
小样本语义分割旨在完成标注稀缺条件下的像素级分类任务。为进一步提升基于原型网络的小样本语义分割对不可见类的泛化能力,针对支持样本与查询图像之间存在外观差异、原型质量不佳的问题,提出了一种基于自相关强化和原型监督的小样本...小样本语义分割旨在完成标注稀缺条件下的像素级分类任务。为进一步提升基于原型网络的小样本语义分割对不可见类的泛化能力,针对支持样本与查询图像之间存在外观差异、原型质量不佳的问题,提出了一种基于自相关强化和原型监督的小样本语义分割方法。首先,设计自相关强化模块,利用查询图像内部像素间的自相关性驱使初始辅助先验向查询数据迁移,生成高层次类原型以得到具有高指点性的强化先验信息;其次,引入多重渐进式监督损失,以原型复原支持掩码的程度为原型质量监督指标,对原型进行自正则化更新,对辅助先验进行自匹配更新,有效提高了原型对支持信息的概括能力,鼓励辅助先验更多地保留与查询特征相关联的细节。所提出的方法在小样本基准数据集PASCAL-5i上进行验证,结果表明,1-shot设定下mIoU(mean intersection over union)值达到64.4%,FB-IoU(foreground-background intersection over union)值达到73.5%,该方法具备一定的先进性和有效性。展开更多
文摘The authors regret that there were errors in the affiliations and the funding declaration in the original published version.The affiliations a and b of the original manuscript are"School of Information Engineering,Jiangxi Provincial Key Laboratory of Advanced Signal Processing and Intelligent Communications,Nanchang University,Nanchang 330031,China",and"School of Internet of Things Engineering,Jiangnan University,Wuxi 214122,China",respectively.The order of the two affiliations are not correct.
基金supported by grants from the National Natural Science Foundation of P.R.China(62276081 and 62106113)Guangdong Basic and Applied Basic Research Foundation(2023A1515010792 and 2023B1515120065)Shenzhen Science and Technology Program(GXWD20231129121139001 and JCYJ20240813110522029).
文摘Importance:Precisely decoding brain dysfunction from high-dimensional functional recordings is crucial for advancing our understanding of brain dysfunction in brain disorders.Self-supervised learning(SSL)models offer a transformative approach for mapping dependencies in functional neuroimaging data.Leveraging the intrinsic organization of brain signals for comprehensive feature extraction,these models enable the analysis of critical neurofunctional features within a clinically relevant framework,overcoming challenges related to data heterogeneity and the scarcity of labeled data.Highlight:This paper provides a comprehensive overview of SSL techniques applied to functional neuroimaging data,such as functional magnetic resonance imaging and electroencephalography,with a specific focus on their applications in various neuropsychiatric disorders.We discuss 3 main categories of SSL methods:contrastive learning,generative learning,and generative-contrastive methods,outlining their basic principles and representative methods.Critically,we highlight the potential of SSL in addressing data scarcity,multimodal integration,and dynamic network modeling for disease detection and prediction.We showcase successful applications of these techniques in understanding and classifying conditions such as Alzheimer’s disease,Parkinson’s disease,and epilepsy,demonstrating their potential in downstream neuropsychological applications.Conclusion:SSL models provide a scalable and effective methodology for individual detection and prediction in brain disorders.Despite current limitations in interpretability and data heterogeneity,the potential of SSL for future clinical applications,particularly in the areas of transdiagnostic psychosis subtyping and decoding task-based brain functional recordings,is substantial.
基金support by the National Natural Science Foundation of China(NSFC)under grant number 61873274.
文摘Contrastive self‐supervised representation learning on attributed graph networks with Graph Neural Networks has attracted considerable research interest recently.However,there are still two challenges.First,most of the real‐word system are multiple relations,where entities are linked by different types of relations,and each relation is a view of the graph network.Second,the rich multi‐scale information(structure‐level and feature‐level)of the graph network can be seen as self‐supervised signals,which are not fully exploited.A novel contrastive self‐supervised representation learning framework on attributed multiplex graph networks with multi‐scale(named CoLM^(2)S)information is presented in this study.It mainly contains two components:intra‐relation contrast learning and interrelation contrastive learning.Specifically,the contrastive self‐supervised representation learning framework on attributed single‐layer graph networks with multi‐scale information(CoLMS)framework with the graph convolutional network as encoder to capture the intra‐relation information with multi‐scale structure‐level and feature‐level selfsupervised signals is introduced first.The structure‐level information includes the edge structure and sub‐graph structure,and the feature‐level information represents the output of different graph convolutional layer.Second,according to the consensus assumption among inter‐relations,the CoLM^(2)S framework is proposed to jointly learn various graph relations in attributed multiplex graph network to achieve global consensus node embedding.The proposed method can fully distil the graph information.Extensive experiments on unsupervised node clustering and graph visualisation tasks demonstrate the effectiveness of our methods,and it outperforms existing competitive baselines.
基金funded by the German Research Foundation(DFG)under Project ID 390874152(POLiS Cluster of Excellence)N.J.S.,A.G.,G.C.,O.D.,A.J.were supported by the D2S2 program within the U.S.Department of Energy,Office of Basic Energy Sciences,Materials Sciences and Engineering Division under Contract No.DE-AC02-05-CH11231(D2S2 program,KCD2S2)A.G.acknowledges support from the Swiss National Science Foundation(SNSF,project#P500PN_222166).
文摘Scanning Electron Microscopes(SEMs)are widely used in experimental science laboratories,often requiring cumbersome and repetitive user analysis.Automating SEM image analysis processes is highly desirable to address this challenge.In particle sample analysis,Machine Learning(ML)has emerged as the most effective approach for particle segmentation.However,the time-intensive process of manually annotating thousands of SEM images limits the applicability of supervised learning approaches.Self-Supervised Learning(SSL)offers a promising alternative by enabling knowledge extraction from raw,unlabeled data.This study presents a framework for evaluating SSL techniques in SEM image analysis,focusing on novel methods leveraging the ConvNeXtV2 architecture for particle detection.A dataset comprising 25,000 SEM images is curated to benchmark these proposed SSL methods.The results demonstrate that ConvNeXtV2 models,with varying parameter counts,consistently outperform other techniques in particle detection across different length scales,achieving up to a34%reduction in relative error compared to established SSL methods.Furthermore,an ablation study explores the relationship between dataset size and SSL performance,providing actionable insights for practitioners regarding model selection and resource efficiency.This research advances the integration of SSL into autonomous analysis pipelines and supports its application in accelerating materials science discovery.
基金support from the National Research Foundation(NRF)Singapore,under its NRF Fellowship(NRF-NRFF11-2019-0002)Singapore Low-Carbon Energy Research Program Funding Initiative hosted under A*STAR(Grant No.U2305d4003).We thank Dr.Jen-It Wong,Senior Application Engineer of JEOL Asia Pte Ltd,for his help in configuring computer connections for the in-situ experiments.
文摘Deep learning in electron microscopy(EM)data analysis is predominantly supervised,relying on manually labeled data.This dependence limits scalability and slows the development of highthroughput EM characterization of materials.While simulation-based approaches provide an alternative,they often struggle with morphological heterogeneity,contrast complexity,and experimental artifacts,reducing their real-world effectiveness.Weintroduce EMcopilot,a closed-loop generative learning framework that enables label-free EM segmentation.EMcopilot leverages the general vision model to extract morphological priors and employs a conditional generative adversarial network to generate contrast-aware images.An EM-specificdomain adapter further enhances realism by modeling key microscope-specific perturbations.Benchmark results show that EMcopilot-trained models not only achieve segmentation accuracy comparable to human-annotated models but also outperform them in detecting nanoparticles in poor-contrast regions and spatially clustered configurations,overcoming inherent human biases in annotation.By illustrating how generative models distill and transform complex EM features into a robust training resource in a self-supervised manner,EMcopilot provides a scalable solution for automated microscopy analysis.
基金supported by the National Research Foundation of Korea(NRF)funded by the Ministry of Education,Science and Technology(NRF-RS-2023-00208888 and NRF-RS-2024-00401786)by a Korea Medical Device Development Fund grant funded by the Korean government(Ministry of Science and Information and Communication Technologies,Ministry of Trade,Industry and Energy,Ministry of Health and Welfare,Ministry of Food and Drug Safety)(RS-2023-00254566).
文摘Fluorescence microscopy has become one of the most widely employed in vivo imaging modalities,enabling the discovery of new biopathological mechanisms.However,the application of fluorescence imaging is often hindered by signal-to-noise ratio issues owing to inherent noise arising from various systemic and biophysical characteristics.These limitations pose a growing challenge,especially with the desire to elucidate dynamic biomechanisms at previously unreachable rapid speeds.Here,we propose a temporal gradient(TG)-based self-supervised denoising network(TeD)that could enable an unprecedented advance in spatially dynamic fluorescence imaging.Our strategy is predicated on the insight that judicious utilization of spatiotemporal information is more advantageous for denoising predictions.Adopting the TG,which intrinsically embodies spatial dynamic features,enables TeD to prudently focus on spatiotemporal information.We showed that TeD can provide new interpretative opportunities for understanding dynamic fluorescence signals in in vivo imaging of mice,representing cellular flow.Furthermore,we demonstrated that TeD is robust even when fluorescence signals exhibit temporal kinetics without spatial dynamics,as seen in neuronal population imaging.We believe that TeD’s superior performance even with spatially dynamic samples,including the complex behavior of cells or organisms,could make a substantial contribution to various biological studies.
文摘小样本语义分割旨在完成标注稀缺条件下的像素级分类任务。为进一步提升基于原型网络的小样本语义分割对不可见类的泛化能力,针对支持样本与查询图像之间存在外观差异、原型质量不佳的问题,提出了一种基于自相关强化和原型监督的小样本语义分割方法。首先,设计自相关强化模块,利用查询图像内部像素间的自相关性驱使初始辅助先验向查询数据迁移,生成高层次类原型以得到具有高指点性的强化先验信息;其次,引入多重渐进式监督损失,以原型复原支持掩码的程度为原型质量监督指标,对原型进行自正则化更新,对辅助先验进行自匹配更新,有效提高了原型对支持信息的概括能力,鼓励辅助先验更多地保留与查询特征相关联的细节。所提出的方法在小样本基准数据集PASCAL-5i上进行验证,结果表明,1-shot设定下mIoU(mean intersection over union)值达到64.4%,FB-IoU(foreground-background intersection over union)值达到73.5%,该方法具备一定的先进性和有效性。
