Two-dimensional endoscopic images are susceptible to interferences such as specular reflections and monotonous texture illumination,hindering accurate three-dimensional lesion reconstruction by surgical robots.This st...Two-dimensional endoscopic images are susceptible to interferences such as specular reflections and monotonous texture illumination,hindering accurate three-dimensional lesion reconstruction by surgical robots.This study proposes a novel end-to-end disparity estimation model to address these challenges.Our approach combines a Pseudo-Siamese neural network architecture with pyramid dilated convolutions,integrating multi-scale image information to enhance robustness against lighting interferences.This study introduces a Pseudo-Siamese structure-based disparity regression model that simplifies left-right image comparison,improving accuracy and efficiency.The model was evaluated using a dataset of stereo endoscopic videos captured by the Da Vinci surgical robot,comprising simulated silicone heart sequences and real heart video data.Experimental results demonstrate significant improvement in the network’s resistance to lighting interference without substantially increasing parameters.Moreover,the model exhibited faster convergence during training,contributing to overall performance enhancement.This study advances endoscopic image processing accuracy and has potential implications for surgical robot applications in complex environments.展开更多
Feature fusion is an important technique in medical image classification that can improve diagnostic accuracy by integrating complementary information from multiple sources.Recently,Deep Learning(DL)has been widely us...Feature fusion is an important technique in medical image classification that can improve diagnostic accuracy by integrating complementary information from multiple sources.Recently,Deep Learning(DL)has been widely used in pulmonary disease diagnosis,such as pneumonia and tuberculosis.However,traditional feature fusion methods often suffer from feature disparity,information loss,redundancy,and increased complexity,hindering the further extension of DL algorithms.To solve this problem,we propose a Graph-Convolution Fusion Network with Self-Supervised Feature Alignment(Self-FAGCFN)to address the limitations of traditional feature fusion methods in deep learning-based medical image classification for respiratory diseases such as pneumonia and tuberculosis.The network integrates Convolutional Neural Networks(CNNs)for robust feature extraction from two-dimensional grid structures and Graph Convolutional Networks(GCNs)within a Graph Neural Network branch to capture features based on graph structure,focusing on significant node representations.Additionally,an Attention-Embedding Ensemble Block is included to capture critical features from GCN outputs.To ensure effective feature alignment between pre-and post-fusion stages,we introduce a feature alignment loss that minimizes disparities.Moreover,to address the limitations of proposed methods,such as inappropriate centroid discrepancies during feature alignment and class imbalance in the dataset,we develop a Feature-Centroid Fusion(FCF)strategy and a Multi-Level Feature-Centroid Update(MLFCU)algorithm,respectively.Extensive experiments on public datasets LungVision and Chest-Xray demonstrate that the Self-FAGCFN model significantly outperforms existing methods in diagnosing pneumonia and tuberculosis,highlighting its potential for practical medical applications.展开更多
Few-shot learning has emerged as a crucial technique for coral species classification,addressing the challenge of limited labeled data in underwater environments.This study introduces an optimized few-shot learning mo...Few-shot learning has emerged as a crucial technique for coral species classification,addressing the challenge of limited labeled data in underwater environments.This study introduces an optimized few-shot learning model that enhances classification accuracy while minimizing reliance on extensive data collection.The proposed model integrates a hybrid similarity measure combining Euclidean distance and cosine similarity,effectively capturing both feature magnitude and directional relationships.This approach achieves a notable accuracy of 71.8%under a 5-way 5-shot evaluation,outperforming state-of-the-art models such as Prototypical Networks,FEAT,and ESPT by up to 10%.Notably,the model demonstrates high precision in classifying Siderastreidae(87.52%)and Fungiidae(88.95%),underscoring its effectiveness in distinguishing subtle morphological differences.To further enhance performance,we incorporate a self-supervised learning mechanism based on contrastive learning,enabling the model to extract robust representations by leveraging local structural patterns in corals.This enhancement significantly improves classification accuracy,particularly for species with high intra-class variation,leading to an overall accuracy of 76.52%under a 5-way 10-shot evaluation.Additionally,the model exploits the repetitive structures inherent in corals,introducing a local feature aggregation strategy that refines classification through spatial information integration.Beyond its technical contributions,this study presents a scalable and efficient approach for automated coral reef monitoring,reducing annotation costs while maintaining high classification accuracy.By improving few-shot learning performance in underwater environments,our model enhances monitoring accuracy by up to 15%compared to traditional methods,offering a practical solution for large-scale coral conservation efforts.展开更多
Computed Tomography(CT)reconstruction is essential inmedical imaging and other engineering fields.However,blurring of the projection during CT imaging can lead to artifacts in the reconstructed images.Projection blur ...Computed Tomography(CT)reconstruction is essential inmedical imaging and other engineering fields.However,blurring of the projection during CT imaging can lead to artifacts in the reconstructed images.Projection blur combines factors such as larger ray sources,scattering and imaging system vibration.To address the problem,we propose DeblurTomo,a novel self-supervised learning-based deblurring and reconstruction algorithm that efficiently reconstructs sharp CT images from blurry input without needing external data and blur measurement.Specifically,we constructed a coordinate-based implicit neural representation reconstruction network,which can map the coordinates to the attenuation coefficient in the reconstructed space formore convenient ray representation.Then,wemodel the blur as aweighted sumof offset rays and design the RayCorrectionNetwork(RCN)andWeight ProposalNetwork(WPN)to fit these rays and their weights bymulti-view consistency and geometric information,thereby extending 2D deblurring to 3D space.In the training phase,we use the blurry input as the supervision signal to optimize the reconstruction network,the RCN,and the WPN simultaneously.Extensive experiments on the widely used synthetic dataset show that DeblurTomo performs superiorly on the limited-angle and sparse-view in the simulated blurred scenarios.Further experiments on real datasets demonstrate the superiority of our method in practical scenarios.展开更多
Self-supervised monocular depth estimation has emerged as a major research focus in recent years,primarily due to the elimination of ground-truth depth dependence.However,the prevailing architectures in this domain su...Self-supervised monocular depth estimation has emerged as a major research focus in recent years,primarily due to the elimination of ground-truth depth dependence.However,the prevailing architectures in this domain suffer from inherent limitations:existing pose network branches infer camera ego-motion exclusively under static-scene and Lambertian-surface assumptions.These assumptions are often violated in real-world scenarios due to dynamic objects,non-Lambertian reflectance,and unstructured background elements,leading to pervasive artifacts such as depth discontinuities(“holes”),structural collapse,and ambiguous reconstruction.To address these challenges,we propose a novel framework that integrates scene dynamic pose estimation into the conventional self-supervised depth network,enhancing its ability to model complex scene dynamics.Our contributions are threefold:(1)a pixel-wise dynamic pose estimation module that jointly resolves the pose transformations of moving objects and localized scene perturbations;(2)a physically-informed loss function that couples dynamic pose and depth predictions,designed to mitigate depth errors arising from high-speed distant objects and geometrically inconsistent motion profiles;(3)an efficient SE(3)transformation parameterization that streamlines network complexity and temporal pre-processing.Extensive experiments on the KITTI and NYU-V2 benchmarks show that our framework achieves state-of-the-art performance in both quantitative metrics and qualitative visual fidelity,significantly improving the robustness and generalization of monocular depth estimation under dynamic conditions.展开更多
Blended acquisition offers efficiency improvements over conventional seismic data acquisition, at the cost of introducing blending noise effects. Besides, seismic data often suffers from irregularly missing shots caus...Blended acquisition offers efficiency improvements over conventional seismic data acquisition, at the cost of introducing blending noise effects. Besides, seismic data often suffers from irregularly missing shots caused by artificial or natural effects during blended acquisition. Therefore, blending noise attenuation and missing shots reconstruction are essential for providing high-quality seismic data for further seismic processing and interpretation. The iterative shrinkage thresholding algorithm can help obtain deblended data based on sparsity assumptions of complete unblended data, and it characterizes seismic data linearly. Supervised learning algorithms can effectively capture the nonlinear relationship between incomplete pseudo-deblended data and complete unblended data. However, the dependence on complete unblended labels limits their practicality in field applications. Consequently, a self-supervised algorithm is presented for simultaneous deblending and interpolation of incomplete blended data, which minimizes the difference between simulated and observed incomplete pseudo-deblended data. The used blind-trace U-Net (BTU-Net) prevents identity mapping during complete unblended data estimation. Furthermore, a multistep process with blending noise simulation-subtraction and missing traces reconstruction-insertion is used in each step to improve the deblending and interpolation performance. Experiments with synthetic and field incomplete blended data demonstrate the effectiveness of the multistep self-supervised BTU-Net algorithm.展开更多
Seismic data denoising is a critical process usually applied at various stages of the seismic processing workflow,as our ability to mitigate noise in seismic data affects the quality of our subsequent analyses.However...Seismic data denoising is a critical process usually applied at various stages of the seismic processing workflow,as our ability to mitigate noise in seismic data affects the quality of our subsequent analyses.However,finding an optimal balance between preserving seismic signals and effectively reducing seismic noise presents a substantial challenge.In this study,we introduce a multi-stage deep learning model,trained in a self-supervised manner,designed specifically to suppress seismic noise while minimizing signal leakage.This model operates as a patch-based approach,extracting overlapping patches from the noisy data and converting them into 1D vectors for input.It consists of two identical sub-networks,each configured differently.Inspired by the transformer architecture,each sub-network features an embedded block that comprises two fully connected layers,which are utilized for feature extraction from the input patches.After reshaping,a multi-head attention module enhances the model’s focus on significant features by assigning higher attention weights to them.The key difference between the two sub-networks lies in the number of neurons within their fully connected layers.The first sub-network serves as a strong denoiser with a small number of neurons,effectively attenuating seismic noise;in contrast,the second sub-network functions as a signal-add-back model,using a larger number of neurons to retrieve some of the signal that was not preserved in the output of the first sub-network.The proposed model produces two outputs,each corresponding to one of the sub-networks,and both sub-networks are optimized simultaneously using the noisy data as the label for both outputs.Evaluations conducted on both synthetic and field data demonstrate the model’s effectiveness in suppressing seismic noise with minimal signal leakage,outperforming some benchmark methods.展开更多
Accurate aging diagnosis is crucial for the health and safety management of lithium-ion batteries in electric vehicles.Despite significant advancements achieved by data-driven methods,diagnosis accuracy remains constr...Accurate aging diagnosis is crucial for the health and safety management of lithium-ion batteries in electric vehicles.Despite significant advancements achieved by data-driven methods,diagnosis accuracy remains constrained by the high costs of check-up tests and the scarcity of labeled data.This paper presents a framework utilizing self-supervised machine learning to harness the potential of unlabeled data for diagnosing battery aging in electric vehicles during field operations.We validate our method using battery degradation datasets collected over more than two years from twenty real-world electric vehicles.Our analysis comprehensively addresses cell inconsistencies,physical interpretations,and charging uncertainties in real-world applications.This is achieved through self-supervised feature extraction using random short charging sequences in the main peak of incremental capacity curves.By leveraging inexpensive unlabeled data in a self-supervised approach,our method demonstrates improvements in average root mean square errors of 74.54%and 60.50%in the best and worst cases,respectively,compared to the supervised benchmark.This work underscores the potential of employing low-cost unlabeled data with self-supervised machine learning for effective battery health and safety management in realworld scenarios.展开更多
Low-light images suffer from low quality due to poor lighting conditions,noise pollution,and improper settings of cameras.To enhance low-light images,most existing methods rely on normal-light images for guidance but ...Low-light images suffer from low quality due to poor lighting conditions,noise pollution,and improper settings of cameras.To enhance low-light images,most existing methods rely on normal-light images for guidance but the collection of suitable normal-light images is difficult.In contrast,a self-supervised method breaks free from the reliance on normal-light data,resulting in more convenience and better generalization.Existing self-supervised methods primarily focus on illumination adjustment and design pixel-based adjustment methods,resulting in remnants of other degradations,uneven brightness and artifacts.In response,this paper proposes a self-supervised enhancement method,termed as SLIE.It can handle multiple degradations including illumination attenuation,noise pollution,and color shift,all in a self-supervised manner.Illumination attenuation is estimated based on physical principles and local neighborhood information.The removal and correction of noise and color shift removal are solely realized with noisy images and images with color shifts.Finally,the comprehensive and fully self-supervised approach can achieve better adaptability and generalization.It is applicable to various low light conditions,and can reproduce the original color of scenes in natural light.Extensive experiments conducted on four public datasets demonstrate the superiority of SLIE to thirteen state-of-the-art methods.Our code is available at https://github.com/hanna-xu/SLIE.展开更多
The encoding aperture snapshot spectral imaging system,based on the compressive sensing theory,can be regarded as an encoder,which can efficiently obtain compressed two-dimensional spectral data and then decode it int...The encoding aperture snapshot spectral imaging system,based on the compressive sensing theory,can be regarded as an encoder,which can efficiently obtain compressed two-dimensional spectral data and then decode it into three-dimensional spectral data through deep neural networks.However,training the deep neural net⁃works requires a large amount of clean data that is difficult to obtain.To address the problem of insufficient training data for deep neural networks,a self-supervised hyperspectral denoising neural network based on neighbor⁃hood sampling is proposed.This network is integrated into a deep plug-and-play framework to achieve self-supervised spectral reconstruction.The study also examines the impact of different noise degradation models on the fi⁃nal reconstruction quality.Experimental results demonstrate that the self-supervised learning method enhances the average peak signal-to-noise ratio by 1.18 dB and improves the structural similarity by 0.009 compared with the supervised learning method.Additionally,it achieves better visual reconstruction results.展开更多
Spectroscopy,especially for plasma spectroscopy,provides a powerful platform for biological and material analysis with its elemental and molecular fingerprinting capability.Artificial intelligence(AI)has the tremendou...Spectroscopy,especially for plasma spectroscopy,provides a powerful platform for biological and material analysis with its elemental and molecular fingerprinting capability.Artificial intelligence(AI)has the tremendous potential to build a universal quantitative framework covering all branches of plasma spectroscopy based on its unmatched representation and generalization ability.Herein,we introduce an AI-based unified method called self-supervised image-spectrum twin information fusion detection(SISTIFD)to collect twin co-occurrence signals of the plasma and to intelligently predict the physical parameters for improving the performances of all plasma spectroscopic techniques.It can fuse the spectra and plasma images in synchronization,derive the plasma parameters(total number density,plasma temperature,electron density,and other implicit factors),and provide accurate results.The experimental data demonstrate their excellent utility and capacity,with a reduction of 98%in evaluation indices(root mean square error,relative standard deviation,etc.)and an analysis frequency of 143 Hz(much faster than the mainstream detection frame rate of 1 Hz).In addition,as a completely end-to-end and self-supervised framework,the SISTIFD enables automatic detection without manual preprocessing or intervention.With these advantages,it has remarkably enhanced various plasma spectroscopic techniques with state-of-the-art performance and unsealed their possibility in industry,especially in the regions that require both capability and efficiency.This scheme brings new inspiration to the whole field of plasma spectroscopy and enables in situ analysis with a real-world scenario of high throughput,cross-interference,various analyte complexity,and diverse applications.展开更多
The federated self-supervised framework is a distributed machine learning method that combines federated learning and self-supervised learning, which can effectively solve the problem of traditional federated learning...The federated self-supervised framework is a distributed machine learning method that combines federated learning and self-supervised learning, which can effectively solve the problem of traditional federated learning being difficult to process large-scale unlabeled data. The existing federated self-supervision framework has problems with low communication efficiency and high communication delay between clients and central servers. Therefore, we added edge servers to the federated self-supervision framework to reduce the pressure on the central server caused by frequent communication between both ends. A communication compression scheme using gradient quantization and sparsification was proposed to optimize the communication of the entire framework, and the algorithm of the sparse communication compression module was improved. Experiments have proved that the learning rate changes of the improved sparse communication compression module are smoother and more stable. Our communication compression scheme effectively reduced the overall communication overhead.展开更多
Learning discriminative representations with deep neural networks often relies on massive labeled data, which is expensive and difficult to obtain in many real scenarios. As an alternative, self-supervised learning th...Learning discriminative representations with deep neural networks often relies on massive labeled data, which is expensive and difficult to obtain in many real scenarios. As an alternative, self-supervised learning that leverages input itself as supervision is strongly preferred for its soaring performance on visual representation learning. This paper introduces a contrastive self-supervised framework for learning generalizable representations on the synthetic data that can be obtained easily with complete controllability.Specifically, we propose to optimize a contrastive learning task and a physical property prediction task simultaneously. Given the synthetic scene, the first task aims to maximize agreement between a pair of synthetic images generated by our proposed view sampling module, while the second task aims to predict three physical property maps, i.e., depth, instance contour maps, and surface normal maps. In addition, a feature-level domain adaptation technique with adversarial training is applied to reduce the domain difference between the realistic and the synthetic data. Experiments demonstrate that our proposed method achieves state-of-the-art performance on several visual recognition datasets.展开更多
State of health(SoH) estimation plays a key role in smart battery health prognostic and management.However,poor generalization,lack of labeled data,and unused measurements during aging are still major challenges to ac...State of health(SoH) estimation plays a key role in smart battery health prognostic and management.However,poor generalization,lack of labeled data,and unused measurements during aging are still major challenges to accurate SoH estimation.Toward this end,this paper proposes a self-supervised learning framework to boost the performance of battery SoH estimation.Different from traditional data-driven methods which rely on a considerable training dataset obtained from numerous battery cells,the proposed method achieves accurate and robust estimations using limited labeled data.A filter-based data preprocessing technique,which enables the extraction of partial capacity-voltage curves under dynamic charging profiles,is applied at first.Unsupervised learning is then used to learn the aging characteristics from the unlabeled data through an auto-encoder-decoder.The learned network parameters are transferred to the downstream SoH estimation task and are fine-tuned with very few sparsely labeled data,which boosts the performance of the estimation framework.The proposed method has been validated under different battery chemistries,formats,operating conditions,and ambient.The estimation accuracy can be guaranteed by using only three labeled data from the initial 20% life cycles,with overall errors less than 1.14% and error distribution of all testing scenarios maintaining less than 4%,and robustness increases with aging.Comparisons with other pure supervised machine learning methods demonstrate the superiority of the proposed method.This simple and data-efficient estimation framework is promising in real-world applications under a variety of scenarios.展开更多
Time series data has attached extensive attention as multi-domain data, but it is difficult to analyze due to its high dimension and few labels. Self-supervised representation learning provides an effective way for pr...Time series data has attached extensive attention as multi-domain data, but it is difficult to analyze due to its high dimension and few labels. Self-supervised representation learning provides an effective way for processing such data. Considering the frequency domain features of the time series data itself and the contextual feature in the classification task, this paper proposes an unsupervised Long Short-Term Memory(LSTM) and contrastive transformer-based time series representation model using contrastive learning. Firstly, transforming data with frequency domainbased augmentation increases the ability to represent features in the frequency domain. Secondly, the encoder module with three layers of LSTM and convolution maps the augmented data to the latent space and calculates the temporal loss with a contrastive transformer module and contextual loss. Finally, after selfsupervised training, the representation vector of the original data can be got from the pre-trained encoder. Our model achieves satisfied performances on Human Activity Recognition(HAR) and sleepEDF real-life datasets.展开更多
Noise suppression is an essential step in many seismic processing workflows.A portion of this noise,particularly in land datasets,presents itself as random noise.In recent years,neural networks have been successfully ...Noise suppression is an essential step in many seismic processing workflows.A portion of this noise,particularly in land datasets,presents itself as random noise.In recent years,neural networks have been successfully used to denoise seismic data in a supervised fashion.However,supervised learning always comes with the often unachievable requirement of having noisy-clean data pairs for training.Using blind-spot networks,we redefine the denoising task as a self-supervised procedure where the network uses the surrounding noisy samples to estimate the noise-free value of a central sample.Based on the assumption that noise is statistically independent between samples,the network struggles to predict the noise component of the sample due to its randomicity,whilst the signal component is accurately predicted due to its spatio-temporal coherency.Illustrated on synthetic examples,the blind-spot network is shown to be an efficient denoiser of seismic data contaminated by random noise with minimal damage to the signal;therefore,providing improvements in both the image domain and down-the-line tasks,such as post-stack inversion.To conclude our study,the suggested approach is applied to field data and the results are compared with two commonly used random denoising techniques:FX-deconvolution and sparsity-promoting inversion by Curvelet transform.By demonstrating that blind-spot networks are an efficient suppressor of random noise,we believe this is just the beginning of utilising self-supervised learning in seismic applications.展开更多
In recent years,self-supervised learning which does not require a large number of manual labels generate supervised signals through the data itself to attain the characterization learning of samples.Self-supervised le...In recent years,self-supervised learning which does not require a large number of manual labels generate supervised signals through the data itself to attain the characterization learning of samples.Self-supervised learning solves the problem of learning semantic features from unlabeled data,and realizes pre-training of models in large data sets.Its significant advantages have been extensively studied by scholars in recent years.There are usually three types of self-supervised learning:"Generative,Contrastive,and GeneTative-Contrastive."The model of the comparative learning method is relatively simple,and the performance of the current downstream task is comparable to that of the supervised learning method.Therefore,we propose a conceptual analysis framework:data augmentation pipeline,architectures,pretext tasks,comparison methods,semisupervised fine-tuning.Based on this conceptual framework,we qualitatively analyze the existing comparative self-supervised learning methods for computer vision,and then further analyze its performance at different stages,and finally summarize the research status of sei supervised comparative learning methods in other fields.展开更多
In this paper,a cross-sensor generative self-supervised learning network is proposed for fault detection of multi-sensor.By modeling the sensor signals in multiple dimensions to achieve correlation information mining ...In this paper,a cross-sensor generative self-supervised learning network is proposed for fault detection of multi-sensor.By modeling the sensor signals in multiple dimensions to achieve correlation information mining between channels to deal with the pretext task,the shared features between multi-sensor data can be captured,and the gap between channel data features will be reduced.Meanwhile,in order to model fault features in the downstream task,the salience module is developed to optimize cross-sensor data features based on a small amount of labeled data to make warning feature information prominent for improving the separator accuracy.Finally,experimental results on the public datasets FEMTO-ST dataset and the private datasets SMT shock absorber dataset(SMT-SA dataset)show that the proposed method performs favorably against other STATE-of-the-art methods.展开更多
Intelligent robotic manufacturing systems are revolutionizing the production industry.These next-generation systems employ robots as actuators,multi-source sensors for perception,and artificial intelligence for decisi...Intelligent robotic manufacturing systems are revolutionizing the production industry.These next-generation systems employ robots as actuators,multi-source sensors for perception,and artificial intelligence for decision-making,aiming to execute routine manufacturing tasks with greater autonomy and flexibility.In footwear manufacturing,sole deburring presents a specific challenge in detecting defects and elaborating deburring paths,which skilled workers traditionally handle.The present research goes beyond solving such problems traditionally with computer vision and hard robot programming.Instead,it focuses on developing a learning structure mimicking human motion planning capability from vision inputs.Like humans who mentally visualize and predict a path before refining it in real-time,we want to give the robot the ability to predetermine the trajectory needed for a finishing task,exploiting only vision data.The system is designed to learn how to identify defects and directly correlate this information with motions by utilizing a latent space representation,transitioning from simple programmed responses to more adaptive and intelligent behaviors.We call it a self-supervised vision-proprioception model,an AI framework that autonomously learns to correlate visual observations to proprioceptive data(end effector trajectories)for effective task execution.This is achieved by integrating a vision-based latent space learning phase(learn to see),followed by a reinforcement learning stage,where the agent learns to associate the latent space with deburring actions in a simulated environment(learn to act).Recognizing the common performance degradation when transferring learned policies to real robots,this research also employs Sim-to-Real methods to bridge the reality gap(learn to transfer).Experimental results validate the whole approach.展开更多
Intelligent Transportation Systems(ITS)leverage Integrated Sensing and Communications(ISAC)to enhance data exchange between vehicles and infrastructure in the Internet of Vehicles(IoV).This integration inevitably incr...Intelligent Transportation Systems(ITS)leverage Integrated Sensing and Communications(ISAC)to enhance data exchange between vehicles and infrastructure in the Internet of Vehicles(IoV).This integration inevitably increases computing demands,risking real-time system stability.Vehicle Edge Computing(VEC)addresses this by offloading tasks to Road Side Units(RSUs),ensuring timely services.Our previous work,the FLSimCo algorithm,which uses local resources for federated Self-Supervised Learning(SSL),has a limitation:vehicles often can’t complete all iteration tasks.Our improved algorithm offloads partial tasks to RSUs and optimizes energy consumption by adjusting transmission power,CPU frequency,and task assignment ratios,balancing local and RSU-based training.Meanwhile,setting an offloading threshold further prevents inefficiencies.Simulation results show that the enhanced algorithm reduces energy consumption and improves offloading efficiency and accuracy of federated SSL.展开更多
基金Supported by Sichuan Science and Technology Program(2023YFSY0026,2023YFH0004)Supported by the Institute of Information&Communications Technology Planning&Evaluation(IITP)grant funded by the Korean government(MSIT)(No.RS-2022-00155885,Artificial Intelligence Convergence Innovation Human Resources Development(Hanyang University ERICA)).
文摘Two-dimensional endoscopic images are susceptible to interferences such as specular reflections and monotonous texture illumination,hindering accurate three-dimensional lesion reconstruction by surgical robots.This study proposes a novel end-to-end disparity estimation model to address these challenges.Our approach combines a Pseudo-Siamese neural network architecture with pyramid dilated convolutions,integrating multi-scale image information to enhance robustness against lighting interferences.This study introduces a Pseudo-Siamese structure-based disparity regression model that simplifies left-right image comparison,improving accuracy and efficiency.The model was evaluated using a dataset of stereo endoscopic videos captured by the Da Vinci surgical robot,comprising simulated silicone heart sequences and real heart video data.Experimental results demonstrate significant improvement in the network’s resistance to lighting interference without substantially increasing parameters.Moreover,the model exhibited faster convergence during training,contributing to overall performance enhancement.This study advances endoscopic image processing accuracy and has potential implications for surgical robot applications in complex environments.
基金supported by the National Natural Science Foundation of China(62276092,62303167)the Postdoctoral Fellowship Program(Grade C)of China Postdoctoral Science Foundation(GZC20230707)+3 种基金the Key Science and Technology Program of Henan Province,China(242102211051,242102211042,212102310084)Key Scientiffc Research Projects of Colleges and Universities in Henan Province,China(25A520009)the China Postdoctoral Science Foundation(2024M760808)the Henan Province medical science and technology research plan joint construction project(LHGJ2024069).
文摘Feature fusion is an important technique in medical image classification that can improve diagnostic accuracy by integrating complementary information from multiple sources.Recently,Deep Learning(DL)has been widely used in pulmonary disease diagnosis,such as pneumonia and tuberculosis.However,traditional feature fusion methods often suffer from feature disparity,information loss,redundancy,and increased complexity,hindering the further extension of DL algorithms.To solve this problem,we propose a Graph-Convolution Fusion Network with Self-Supervised Feature Alignment(Self-FAGCFN)to address the limitations of traditional feature fusion methods in deep learning-based medical image classification for respiratory diseases such as pneumonia and tuberculosis.The network integrates Convolutional Neural Networks(CNNs)for robust feature extraction from two-dimensional grid structures and Graph Convolutional Networks(GCNs)within a Graph Neural Network branch to capture features based on graph structure,focusing on significant node representations.Additionally,an Attention-Embedding Ensemble Block is included to capture critical features from GCN outputs.To ensure effective feature alignment between pre-and post-fusion stages,we introduce a feature alignment loss that minimizes disparities.Moreover,to address the limitations of proposed methods,such as inappropriate centroid discrepancies during feature alignment and class imbalance in the dataset,we develop a Feature-Centroid Fusion(FCF)strategy and a Multi-Level Feature-Centroid Update(MLFCU)algorithm,respectively.Extensive experiments on public datasets LungVision and Chest-Xray demonstrate that the Self-FAGCFN model significantly outperforms existing methods in diagnosing pneumonia and tuberculosis,highlighting its potential for practical medical applications.
基金funded by theNational Science and TechnologyCouncil(NSTC),Taiwan,under grant numbers NSTC 112-2634-F-019-001 and NSTC 113-2634-F-A49-007.
文摘Few-shot learning has emerged as a crucial technique for coral species classification,addressing the challenge of limited labeled data in underwater environments.This study introduces an optimized few-shot learning model that enhances classification accuracy while minimizing reliance on extensive data collection.The proposed model integrates a hybrid similarity measure combining Euclidean distance and cosine similarity,effectively capturing both feature magnitude and directional relationships.This approach achieves a notable accuracy of 71.8%under a 5-way 5-shot evaluation,outperforming state-of-the-art models such as Prototypical Networks,FEAT,and ESPT by up to 10%.Notably,the model demonstrates high precision in classifying Siderastreidae(87.52%)and Fungiidae(88.95%),underscoring its effectiveness in distinguishing subtle morphological differences.To further enhance performance,we incorporate a self-supervised learning mechanism based on contrastive learning,enabling the model to extract robust representations by leveraging local structural patterns in corals.This enhancement significantly improves classification accuracy,particularly for species with high intra-class variation,leading to an overall accuracy of 76.52%under a 5-way 10-shot evaluation.Additionally,the model exploits the repetitive structures inherent in corals,introducing a local feature aggregation strategy that refines classification through spatial information integration.Beyond its technical contributions,this study presents a scalable and efficient approach for automated coral reef monitoring,reducing annotation costs while maintaining high classification accuracy.By improving few-shot learning performance in underwater environments,our model enhances monitoring accuracy by up to 15%compared to traditional methods,offering a practical solution for large-scale coral conservation efforts.
基金supported in part by the National Natural Science Foundation of China under Grants 62472434 and 62402171in part by the National Key Research and Development Program of China under Grant 2022YFF1203001+1 种基金in part by the Science and Technology Innovation Program of Hunan Province under Grant 2022RC3061in part by the Sci-Tech Innovation 2030 Agenda under Grant 2023ZD0508600.
文摘Computed Tomography(CT)reconstruction is essential inmedical imaging and other engineering fields.However,blurring of the projection during CT imaging can lead to artifacts in the reconstructed images.Projection blur combines factors such as larger ray sources,scattering and imaging system vibration.To address the problem,we propose DeblurTomo,a novel self-supervised learning-based deblurring and reconstruction algorithm that efficiently reconstructs sharp CT images from blurry input without needing external data and blur measurement.Specifically,we constructed a coordinate-based implicit neural representation reconstruction network,which can map the coordinates to the attenuation coefficient in the reconstructed space formore convenient ray representation.Then,wemodel the blur as aweighted sumof offset rays and design the RayCorrectionNetwork(RCN)andWeight ProposalNetwork(WPN)to fit these rays and their weights bymulti-view consistency and geometric information,thereby extending 2D deblurring to 3D space.In the training phase,we use the blurry input as the supervision signal to optimize the reconstruction network,the RCN,and the WPN simultaneously.Extensive experiments on the widely used synthetic dataset show that DeblurTomo performs superiorly on the limited-angle and sparse-view in the simulated blurred scenarios.Further experiments on real datasets demonstrate the superiority of our method in practical scenarios.
基金supported in part by the National Natural Science Foundation of China under Grants 62071345。
文摘Self-supervised monocular depth estimation has emerged as a major research focus in recent years,primarily due to the elimination of ground-truth depth dependence.However,the prevailing architectures in this domain suffer from inherent limitations:existing pose network branches infer camera ego-motion exclusively under static-scene and Lambertian-surface assumptions.These assumptions are often violated in real-world scenarios due to dynamic objects,non-Lambertian reflectance,and unstructured background elements,leading to pervasive artifacts such as depth discontinuities(“holes”),structural collapse,and ambiguous reconstruction.To address these challenges,we propose a novel framework that integrates scene dynamic pose estimation into the conventional self-supervised depth network,enhancing its ability to model complex scene dynamics.Our contributions are threefold:(1)a pixel-wise dynamic pose estimation module that jointly resolves the pose transformations of moving objects and localized scene perturbations;(2)a physically-informed loss function that couples dynamic pose and depth predictions,designed to mitigate depth errors arising from high-speed distant objects and geometrically inconsistent motion profiles;(3)an efficient SE(3)transformation parameterization that streamlines network complexity and temporal pre-processing.Extensive experiments on the KITTI and NYU-V2 benchmarks show that our framework achieves state-of-the-art performance in both quantitative metrics and qualitative visual fidelity,significantly improving the robustness and generalization of monocular depth estimation under dynamic conditions.
基金supported by the National Natural Science Foundation of China(42374134,42304125,U20B6005)the Science and Technology Commission of Shanghai Municipality(23JC1400502)the Fundamental Research Funds for the Central Universities.
文摘Blended acquisition offers efficiency improvements over conventional seismic data acquisition, at the cost of introducing blending noise effects. Besides, seismic data often suffers from irregularly missing shots caused by artificial or natural effects during blended acquisition. Therefore, blending noise attenuation and missing shots reconstruction are essential for providing high-quality seismic data for further seismic processing and interpretation. The iterative shrinkage thresholding algorithm can help obtain deblended data based on sparsity assumptions of complete unblended data, and it characterizes seismic data linearly. Supervised learning algorithms can effectively capture the nonlinear relationship between incomplete pseudo-deblended data and complete unblended data. However, the dependence on complete unblended labels limits their practicality in field applications. Consequently, a self-supervised algorithm is presented for simultaneous deblending and interpolation of incomplete blended data, which minimizes the difference between simulated and observed incomplete pseudo-deblended data. The used blind-trace U-Net (BTU-Net) prevents identity mapping during complete unblended data estimation. Furthermore, a multistep process with blending noise simulation-subtraction and missing traces reconstruction-insertion is used in each step to improve the deblending and interpolation performance. Experiments with synthetic and field incomplete blended data demonstrate the effectiveness of the multistep self-supervised BTU-Net algorithm.
基金supported by the King Abdullah University of Science and Technology(KAUST)。
文摘Seismic data denoising is a critical process usually applied at various stages of the seismic processing workflow,as our ability to mitigate noise in seismic data affects the quality of our subsequent analyses.However,finding an optimal balance between preserving seismic signals and effectively reducing seismic noise presents a substantial challenge.In this study,we introduce a multi-stage deep learning model,trained in a self-supervised manner,designed specifically to suppress seismic noise while minimizing signal leakage.This model operates as a patch-based approach,extracting overlapping patches from the noisy data and converting them into 1D vectors for input.It consists of two identical sub-networks,each configured differently.Inspired by the transformer architecture,each sub-network features an embedded block that comprises two fully connected layers,which are utilized for feature extraction from the input patches.After reshaping,a multi-head attention module enhances the model’s focus on significant features by assigning higher attention weights to them.The key difference between the two sub-networks lies in the number of neurons within their fully connected layers.The first sub-network serves as a strong denoiser with a small number of neurons,effectively attenuating seismic noise;in contrast,the second sub-network functions as a signal-add-back model,using a larger number of neurons to retrieve some of the signal that was not preserved in the output of the first sub-network.The proposed model produces two outputs,each corresponding to one of the sub-networks,and both sub-networks are optimized simultaneously using the noisy data as the label for both outputs.Evaluations conducted on both synthetic and field data demonstrate the model’s effectiveness in suppressing seismic noise with minimal signal leakage,outperforming some benchmark methods.
基金supported by the research project‘‘SafeDaBatt”(03EMF0409A)funded by the German Federal Ministry for Digital and Transport(BMDV)+2 种基金the National Key Research and Development Program of China(2022YFE0102700)the Key Research and Development Program of Shaanxi Province(2023-GHYB-05,2023-YBSF-104)the financial support from the China Scholarship Council(CSC)(202206567008)。
文摘Accurate aging diagnosis is crucial for the health and safety management of lithium-ion batteries in electric vehicles.Despite significant advancements achieved by data-driven methods,diagnosis accuracy remains constrained by the high costs of check-up tests and the scarcity of labeled data.This paper presents a framework utilizing self-supervised machine learning to harness the potential of unlabeled data for diagnosing battery aging in electric vehicles during field operations.We validate our method using battery degradation datasets collected over more than two years from twenty real-world electric vehicles.Our analysis comprehensively addresses cell inconsistencies,physical interpretations,and charging uncertainties in real-world applications.This is achieved through self-supervised feature extraction using random short charging sequences in the main peak of incremental capacity curves.By leveraging inexpensive unlabeled data in a self-supervised approach,our method demonstrates improvements in average root mean square errors of 74.54%and 60.50%in the best and worst cases,respectively,compared to the supervised benchmark.This work underscores the potential of employing low-cost unlabeled data with self-supervised machine learning for effective battery health and safety management in realworld scenarios.
基金supported by the National Natural Science Foundation of China(62276192)。
文摘Low-light images suffer from low quality due to poor lighting conditions,noise pollution,and improper settings of cameras.To enhance low-light images,most existing methods rely on normal-light images for guidance but the collection of suitable normal-light images is difficult.In contrast,a self-supervised method breaks free from the reliance on normal-light data,resulting in more convenience and better generalization.Existing self-supervised methods primarily focus on illumination adjustment and design pixel-based adjustment methods,resulting in remnants of other degradations,uneven brightness and artifacts.In response,this paper proposes a self-supervised enhancement method,termed as SLIE.It can handle multiple degradations including illumination attenuation,noise pollution,and color shift,all in a self-supervised manner.Illumination attenuation is estimated based on physical principles and local neighborhood information.The removal and correction of noise and color shift removal are solely realized with noisy images and images with color shifts.Finally,the comprehensive and fully self-supervised approach can achieve better adaptability and generalization.It is applicable to various low light conditions,and can reproduce the original color of scenes in natural light.Extensive experiments conducted on four public datasets demonstrate the superiority of SLIE to thirteen state-of-the-art methods.Our code is available at https://github.com/hanna-xu/SLIE.
基金Supported by the Zhejiang Provincial"Jianbing"and"Lingyan"R&D Programs(2023C03012,2024C01126)。
文摘The encoding aperture snapshot spectral imaging system,based on the compressive sensing theory,can be regarded as an encoder,which can efficiently obtain compressed two-dimensional spectral data and then decode it into three-dimensional spectral data through deep neural networks.However,training the deep neural net⁃works requires a large amount of clean data that is difficult to obtain.To address the problem of insufficient training data for deep neural networks,a self-supervised hyperspectral denoising neural network based on neighbor⁃hood sampling is proposed.This network is integrated into a deep plug-and-play framework to achieve self-supervised spectral reconstruction.The study also examines the impact of different noise degradation models on the fi⁃nal reconstruction quality.Experimental results demonstrate that the self-supervised learning method enhances the average peak signal-to-noise ratio by 1.18 dB and improves the structural similarity by 0.009 compared with the supervised learning method.Additionally,it achieves better visual reconstruction results.
基金supported by the National Key Research and Development Program of China(Grant No.2022YFE0118700)the National Natural Science Foundation of China(Grant No.62375101)the Fundamental Research Funds for the Central Universities(Grant No.YCJJ20230216).
文摘Spectroscopy,especially for plasma spectroscopy,provides a powerful platform for biological and material analysis with its elemental and molecular fingerprinting capability.Artificial intelligence(AI)has the tremendous potential to build a universal quantitative framework covering all branches of plasma spectroscopy based on its unmatched representation and generalization ability.Herein,we introduce an AI-based unified method called self-supervised image-spectrum twin information fusion detection(SISTIFD)to collect twin co-occurrence signals of the plasma and to intelligently predict the physical parameters for improving the performances of all plasma spectroscopic techniques.It can fuse the spectra and plasma images in synchronization,derive the plasma parameters(total number density,plasma temperature,electron density,and other implicit factors),and provide accurate results.The experimental data demonstrate their excellent utility and capacity,with a reduction of 98%in evaluation indices(root mean square error,relative standard deviation,etc.)and an analysis frequency of 143 Hz(much faster than the mainstream detection frame rate of 1 Hz).In addition,as a completely end-to-end and self-supervised framework,the SISTIFD enables automatic detection without manual preprocessing or intervention.With these advantages,it has remarkably enhanced various plasma spectroscopic techniques with state-of-the-art performance and unsealed their possibility in industry,especially in the regions that require both capability and efficiency.This scheme brings new inspiration to the whole field of plasma spectroscopy and enables in situ analysis with a real-world scenario of high throughput,cross-interference,various analyte complexity,and diverse applications.
文摘The federated self-supervised framework is a distributed machine learning method that combines federated learning and self-supervised learning, which can effectively solve the problem of traditional federated learning being difficult to process large-scale unlabeled data. The existing federated self-supervision framework has problems with low communication efficiency and high communication delay between clients and central servers. Therefore, we added edge servers to the federated self-supervision framework to reduce the pressure on the central server caused by frequent communication between both ends. A communication compression scheme using gradient quantization and sparsification was proposed to optimize the communication of the entire framework, and the algorithm of the sparse communication compression module was improved. Experiments have proved that the learning rate changes of the improved sparse communication compression module are smoother and more stable. Our communication compression scheme effectively reduced the overall communication overhead.
基金by National Natural Science Foundation of China(Nos.61822204 and 61521002).
文摘Learning discriminative representations with deep neural networks often relies on massive labeled data, which is expensive and difficult to obtain in many real scenarios. As an alternative, self-supervised learning that leverages input itself as supervision is strongly preferred for its soaring performance on visual representation learning. This paper introduces a contrastive self-supervised framework for learning generalizable representations on the synthetic data that can be obtained easily with complete controllability.Specifically, we propose to optimize a contrastive learning task and a physical property prediction task simultaneously. Given the synthetic scene, the first task aims to maximize agreement between a pair of synthetic images generated by our proposed view sampling module, while the second task aims to predict three physical property maps, i.e., depth, instance contour maps, and surface normal maps. In addition, a feature-level domain adaptation technique with adversarial training is applied to reduce the domain difference between the realistic and the synthetic data. Experiments demonstrate that our proposed method achieves state-of-the-art performance on several visual recognition datasets.
基金funded by the “SMART BATTERY” project, granted by Villum Foundation in 2021 (project number 222860)。
文摘State of health(SoH) estimation plays a key role in smart battery health prognostic and management.However,poor generalization,lack of labeled data,and unused measurements during aging are still major challenges to accurate SoH estimation.Toward this end,this paper proposes a self-supervised learning framework to boost the performance of battery SoH estimation.Different from traditional data-driven methods which rely on a considerable training dataset obtained from numerous battery cells,the proposed method achieves accurate and robust estimations using limited labeled data.A filter-based data preprocessing technique,which enables the extraction of partial capacity-voltage curves under dynamic charging profiles,is applied at first.Unsupervised learning is then used to learn the aging characteristics from the unlabeled data through an auto-encoder-decoder.The learned network parameters are transferred to the downstream SoH estimation task and are fine-tuned with very few sparsely labeled data,which boosts the performance of the estimation framework.The proposed method has been validated under different battery chemistries,formats,operating conditions,and ambient.The estimation accuracy can be guaranteed by using only three labeled data from the initial 20% life cycles,with overall errors less than 1.14% and error distribution of all testing scenarios maintaining less than 4%,and robustness increases with aging.Comparisons with other pure supervised machine learning methods demonstrate the superiority of the proposed method.This simple and data-efficient estimation framework is promising in real-world applications under a variety of scenarios.
基金Supported by the National Key Research and Development Program of China(2019YFB1706401)。
文摘Time series data has attached extensive attention as multi-domain data, but it is difficult to analyze due to its high dimension and few labels. Self-supervised representation learning provides an effective way for processing such data. Considering the frequency domain features of the time series data itself and the contextual feature in the classification task, this paper proposes an unsupervised Long Short-Term Memory(LSTM) and contrastive transformer-based time series representation model using contrastive learning. Firstly, transforming data with frequency domainbased augmentation increases the ability to represent features in the frequency domain. Secondly, the encoder module with three layers of LSTM and convolution maps the augmented data to the latent space and calculates the temporal loss with a contrastive transformer module and contextual loss. Finally, after selfsupervised training, the representation vector of the original data can be got from the pre-trained encoder. Our model achieves satisfied performances on Human Activity Recognition(HAR) and sleepEDF real-life datasets.
文摘Noise suppression is an essential step in many seismic processing workflows.A portion of this noise,particularly in land datasets,presents itself as random noise.In recent years,neural networks have been successfully used to denoise seismic data in a supervised fashion.However,supervised learning always comes with the often unachievable requirement of having noisy-clean data pairs for training.Using blind-spot networks,we redefine the denoising task as a self-supervised procedure where the network uses the surrounding noisy samples to estimate the noise-free value of a central sample.Based on the assumption that noise is statistically independent between samples,the network struggles to predict the noise component of the sample due to its randomicity,whilst the signal component is accurately predicted due to its spatio-temporal coherency.Illustrated on synthetic examples,the blind-spot network is shown to be an efficient denoiser of seismic data contaminated by random noise with minimal damage to the signal;therefore,providing improvements in both the image domain and down-the-line tasks,such as post-stack inversion.To conclude our study,the suggested approach is applied to field data and the results are compared with two commonly used random denoising techniques:FX-deconvolution and sparsity-promoting inversion by Curvelet transform.By demonstrating that blind-spot networks are an efficient suppressor of random noise,we believe this is just the beginning of utilising self-supervised learning in seismic applications.
文摘In recent years,self-supervised learning which does not require a large number of manual labels generate supervised signals through the data itself to attain the characterization learning of samples.Self-supervised learning solves the problem of learning semantic features from unlabeled data,and realizes pre-training of models in large data sets.Its significant advantages have been extensively studied by scholars in recent years.There are usually three types of self-supervised learning:"Generative,Contrastive,and GeneTative-Contrastive."The model of the comparative learning method is relatively simple,and the performance of the current downstream task is comparable to that of the supervised learning method.Therefore,we propose a conceptual analysis framework:data augmentation pipeline,architectures,pretext tasks,comparison methods,semisupervised fine-tuning.Based on this conceptual framework,we qualitatively analyze the existing comparative self-supervised learning methods for computer vision,and then further analyze its performance at different stages,and finally summarize the research status of sei supervised comparative learning methods in other fields.
基金supported by the National Natural Science Foundation of China under Grant No.62173317the Key Research and Development Program of Anhui under Grant No.202104a05020064。
文摘In this paper,a cross-sensor generative self-supervised learning network is proposed for fault detection of multi-sensor.By modeling the sensor signals in multiple dimensions to achieve correlation information mining between channels to deal with the pretext task,the shared features between multi-sensor data can be captured,and the gap between channel data features will be reduced.Meanwhile,in order to model fault features in the downstream task,the salience module is developed to optimize cross-sensor data features based on a small amount of labeled data to make warning feature information prominent for improving the separator accuracy.Finally,experimental results on the public datasets FEMTO-ST dataset and the private datasets SMT shock absorber dataset(SMT-SA dataset)show that the proposed method performs favorably against other STATE-of-the-art methods.
基金partly carried out within the MICS(Made in Italy—Circular and Sustainable)Extended Partnership and received funding from Next-Generation EU(Nos.Italian PNRR—M4 C2,Invest 1.3—D.D.1551.11-10-2022,PE00000004),CUP MICS D43C22003120001.
文摘Intelligent robotic manufacturing systems are revolutionizing the production industry.These next-generation systems employ robots as actuators,multi-source sensors for perception,and artificial intelligence for decision-making,aiming to execute routine manufacturing tasks with greater autonomy and flexibility.In footwear manufacturing,sole deburring presents a specific challenge in detecting defects and elaborating deburring paths,which skilled workers traditionally handle.The present research goes beyond solving such problems traditionally with computer vision and hard robot programming.Instead,it focuses on developing a learning structure mimicking human motion planning capability from vision inputs.Like humans who mentally visualize and predict a path before refining it in real-time,we want to give the robot the ability to predetermine the trajectory needed for a finishing task,exploiting only vision data.The system is designed to learn how to identify defects and directly correlate this information with motions by utilizing a latent space representation,transitioning from simple programmed responses to more adaptive and intelligent behaviors.We call it a self-supervised vision-proprioception model,an AI framework that autonomously learns to correlate visual observations to proprioceptive data(end effector trajectories)for effective task execution.This is achieved by integrating a vision-based latent space learning phase(learn to see),followed by a reinforcement learning stage,where the agent learns to associate the latent space with deburring actions in a simulated environment(learn to act).Recognizing the common performance degradation when transferring learned policies to real robots,this research also employs Sim-to-Real methods to bridge the reality gap(learn to transfer).Experimental results validate the whole approach.
文摘Intelligent Transportation Systems(ITS)leverage Integrated Sensing and Communications(ISAC)to enhance data exchange between vehicles and infrastructure in the Internet of Vehicles(IoV).This integration inevitably increases computing demands,risking real-time system stability.Vehicle Edge Computing(VEC)addresses this by offloading tasks to Road Side Units(RSUs),ensuring timely services.Our previous work,the FLSimCo algorithm,which uses local resources for federated Self-Supervised Learning(SSL),has a limitation:vehicles often can’t complete all iteration tasks.Our improved algorithm offloads partial tasks to RSUs and optimizes energy consumption by adjusting transmission power,CPU frequency,and task assignment ratios,balancing local and RSU-based training.Meanwhile,setting an offloading threshold further prevents inefficiencies.Simulation results show that the enhanced algorithm reduces energy consumption and improves offloading efficiency and accuracy of federated SSL.
