Multi-source domain adaptation utilizes multiple source domains to learn the knowledge and transfers it to an unlabeled target domain.To address the problem,most of the existing methods aim to minimize the domain shif...Multi-source domain adaptation utilizes multiple source domains to learn the knowledge and transfers it to an unlabeled target domain.To address the problem,most of the existing methods aim to minimize the domain shift by auxiliary distribution alignment objectives,which reduces the effect of domain-specific features.However,without explicitly modeling the domain-specific features,it is not easy to guarantee that the domain-invariant representation extracted from input domains contains domain-specific information as few as possible.In this work,we present a different perspective on MSDA,which employs the idea of feature elimination to reduce the influence of domain-specific features.We design two different ways to extract domain-specific features and total features and construct the domain-invariant representations by eliminating the domain-specific features from total features.The experimental results on different domain adaptation datasets demonstrate the effectiveness of our method and the generalization ability of our model.展开更多
The goal of decentralized multi-source domain adaptation is to conduct unsupervised multi-source domain adaptation in a data decentralization scenario. The challenge of data decentralization is that the source domains...The goal of decentralized multi-source domain adaptation is to conduct unsupervised multi-source domain adaptation in a data decentralization scenario. The challenge of data decentralization is that the source domains and target domain lack cross-domain collaboration during training. On the unlabeled target domain, the target model needs to transfer supervision knowledge with the collaboration of source models, while the domain gap will lead to limited adaptation performance from source models. On the labeled source domain, the source model tends to overfit its domain data in the data decentralization scenario, which leads to the negative transfer problem. For these challenges, we propose dual collaboration for decentralized multi-source domain adaptation by training and aggregating the local source models and local target model in collaboration with each other. On the target domain, we train the local target model by distilling supervision knowledge and fully using the unlabeled target domain data to alleviate the domain shift problem with the collaboration of local source models. On the source domain, we regularize the local source models in collaboration with the local target model to overcome the negative transfer problem. This forms a dual collaboration between the decentralized source domains and target domain, which improves the domain adaptation performance under the data decentralization scenario. Extensive experiments indicate that our method outperforms the state-of-the-art methods by a large margin on standard multi-source domain adaptation datasets.展开更多
AIM:To address the challenges of data labeling difficulties,data privacy,and necessary large amount of labeled data for deep learning methods in diabetic retinopathy(DR)identification,the aim of this study is to devel...AIM:To address the challenges of data labeling difficulties,data privacy,and necessary large amount of labeled data for deep learning methods in diabetic retinopathy(DR)identification,the aim of this study is to develop a source-free domain adaptation(SFDA)method for efficient and effective DR identification from unlabeled data.METHODS:A multi-SFDA method was proposed for DR identification.This method integrates multiple source models,which are trained from the same source domain,to generate synthetic pseudo labels for the unlabeled target domain.Besides,a softmax-consistence minimization term is utilized to minimize the intra-class distances between the source and target domains and maximize the inter-class distances.Validation is performed using three color fundus photograph datasets(APTOS2019,DDR,and EyePACS).RESULTS:The proposed model was evaluated and provided promising results with respectively 0.8917 and 0.9795 F1-scores on referable and normal/abnormal DR identification tasks.It demonstrated effective DR identification through minimizing intra-class distances and maximizing inter-class distances between source and target domains.CONCLUSION:The multi-SFDA method provides an effective approach to overcome the challenges in DR identification.The method not only addresses difficulties in data labeling and privacy issues,but also reduces the need for large amounts of labeled data required by deep learning methods,making it a practical tool for early detection and preservation of vision in diabetic patients.展开更多
In recent years,the heterogeneous SAR image classification task of"training on simulated data and testing on measured data"has garnered increasing attention in the field of Synthetic Aperture Radar Automatic...In recent years,the heterogeneous SAR image classification task of"training on simulated data and testing on measured data"has garnered increasing attention in the field of Synthetic Aperture Radar Automatic Target Recognition(SAR-ATR).Although current mainstream domain adaptation methods have made significant breakthroughs in addressing domain shift problems,the escalating model complexity and task complexity have constrained their deployment in real-world applications.To tackle this challenge,this paper proposes a domain adaptation framework based on linear-kernel Maximum Mean Discrepancy(MMD),integrated with a near-zero-cost pseudo-label denoising technique leveraging deep feature clustering.Our method completely eliminates the need for data augmentation and handcrafted feature design,achieving endto-end pseudo-label self-training.Competitive performance is demonstrated across three typical scenarios in the SAMPLE dataset,with the highest accuracy of 98.65%achieved in ScenarioⅢ.The relevant code is available at:https://github.com/TheGreatTreatsby/SAMPLE_MMD.展开更多
Landslide susceptibility evaluation plays an important role in disaster prevention and reduction.Feature-based transfer learning(TL)is an effective method for solving landslide susceptibility mapping(LSM)in target reg...Landslide susceptibility evaluation plays an important role in disaster prevention and reduction.Feature-based transfer learning(TL)is an effective method for solving landslide susceptibility mapping(LSM)in target regions with no available samples.However,as the study area expands,the distribution of land-slide types and triggering mechanisms becomes more diverse,leading to performance degradation in models relying on landslide evaluation knowledge from a single source domain due to domain feature shift.To address this,this study proposes a Multi-source Domain Adaptation Convolutional Neural Network(MDACNN),which combines the landslide prediction knowledge learned from two source domains to perform cross-regional LSM in complex large-scale areas.The method is validated through case studies in three regions located in southeastern coastal China and compared with single-source domain TL models(TCA-based models).The results demonstrate that MDACNN effectively integrates transfer knowledge from multiple source domains to learn diverse landslide-triggering mechanisms,thereby significantly reducing prediction bias inherent to single-source domain TL models,achieving an average improvement of 16.58%across all metrics.Moreover,the landslide susceptibility maps gener-ated by MDACNN accurately quantify the spatial distribution of landslide risks in the target area,provid-ing a powerful scientific and technological tool for landslide disaster management and prevention.展开更多
Quantum machine learning is an important application of quantum computing in the era of noisy intermediate-scale quantum devices.Domain adaptation(DA)is an effective method for addressing the distribution discrepancy ...Quantum machine learning is an important application of quantum computing in the era of noisy intermediate-scale quantum devices.Domain adaptation(DA)is an effective method for addressing the distribution discrepancy problem between the training data and the real data when the neural network model is deployed.In this paper,we propose a variational quantum domain adaptation method inspired by the quantum convolutional neural network,named variational quantum domain adaptation(VQDA).The data are first uploaded by a‘quantum coding module',then the feature information is extracted by several‘quantum convolution layers'and‘quantum pooling layers',which is named‘Feature Extractor'.Subsequently,the labels and the domains of the samples are obtained by the‘quantum fully connected layer'.With a gradient reversal module,the trained‘Feature Extractor'can extract the features that cannot be distinguished from the source and target domains.The simulations on the local computer and IBM Quantum Experience(IBM Q)platform by Qiskit show the effectiveness of the proposed method.The results show that VQDA(with 8 quantum bits)has 91.46%average classification accuracy for DA task between MNIST→USPS(USPS→MNIST),achieves 91.16%average classification accuracy for gray-scale and color images(with 10 quantum bits),and has 69.25%average classification accuracy on the DA task for color images(also with 10 quantum bits).VQDA achieves a 9.14%improvement in average classification accuracy compared to its corresponding classical domain adaptation method with the same parameter scale for different DA tasks.Simultaneously,the parameters scale is reduced to 43%by using VQDA when both quantum and classical DA methods have similar classification accuracies.展开更多
Deep learning networks are increasingly exploited in the field of neuronal soma segmentation.However,annotating dataset is also an expensive and time-consuming task.Unsupervised domain adaptation is an effective metho...Deep learning networks are increasingly exploited in the field of neuronal soma segmentation.However,annotating dataset is also an expensive and time-consuming task.Unsupervised domain adaptation is an effective method to mitigate the problem,which is able to learn an adaptive segmentation model by transferring knowledge from a rich-labeled source domain.In this paper,we propose a multi-level distribution alignment-based unsupervised domain adaptation network(MDA-Net)for segmentation of 3D neuronal soma images.Distribution alignment is performed in both feature space and output space.In the feature space,features from different scales are adaptively fused to enhance the feature extraction capability for small target somata and con-strained to be domain invariant by adversarial adaptation strategy.In the output space,local discrepancy maps that can reveal the spatial structures of somata are constructed on the predicted segmentation results.Then thedistribution alignment is performed on the local discrepancies maps across domains to obtain a superior discrepancy map in the target domain,achieving refined segmentation performance of neuronal somata.Additionally,after a period of distribution align-ment procedure,a portion of target samples with high confident pseudo-labels are selected as training data,which assist in learning a more adaptive segmentation network.We verified the superiority of the proposed algorithm by comparing several domain adaptation networks on two 3D mouse brain neuronal somata datasets and one macaque brain neuronal soma dataset.展开更多
Artificial intelligence has potential for forecasting reactor conditions in the nuclear industry.Owing to economic and security concerns,a common method is to train data generated by simulators.However,achieving a sat...Artificial intelligence has potential for forecasting reactor conditions in the nuclear industry.Owing to economic and security concerns,a common method is to train data generated by simulators.However,achieving a satisfactory performance in practical applications is difficult because simulators imperfectly emulate reality.To bridge this gap,we propose a novel framework called simulation-to-reality domain adaptation(SRDA)for forecasting the operating parameters of nuclear reactors.The SRDA model employs a transformer-based feature extractor to capture dynamic characteristics and temporal dependencies.A parameter predictor with an improved logarithmic loss function is specifically designed to adapt to varying reactor powers.To fuse prior reactor knowledge from simulations with reality,the domain discriminator utilizes an adversarial strategy to ensure the learning of deep domain-invariant features,and the multiple kernel maximum mean discrepancy minimizes their discrepancies.Experiments on neutron fluxes and temperatures from a pressurized water reactor illustrate that the SRDA model surpasses various advanced methods in terms of predictive performance.This study is the first to use domain adaptation for real-world reactor prediction and presents a feasible solution for enhancing the transferability and generalizability of simulated data.展开更多
Smart manufacturing suffers from the heterogeneity of local data distribution across parties,mutual information silos and lack of privacy protection in the process of industry chain collaboration.To address these prob...Smart manufacturing suffers from the heterogeneity of local data distribution across parties,mutual information silos and lack of privacy protection in the process of industry chain collaboration.To address these problems,we propose a federated domain adaptation algorithm based on knowledge distillation and contrastive learning.Knowledge distillation is used to extract transferable integration knowledge from the different source domains and the quality of the extracted integration knowledge is used to assign reasonable weights to each source domain.A more rational weighted average aggregation is used in the aggregation phase of the center server to optimize the global model,while the local model of the source domain is trained with the help of contrastive learning to constrain the local model optimum towards the global model optimum,mitigating the inherent heterogeneity between local data.Our experiments are conducted on the largest domain adaptation dataset,and the results show that compared with other traditional federated domain adaptation algorithms,the algorithm we proposed trains a more accurate model,requires fewer communication rounds,makes more effective use of imbalanced data in the industrial area,and protects data privacy.展开更多
The variations in gas path parameter deviations can fully reflect the healthy state of aeroengine gas path components and units;therefore,airlines usually take them as key parameters for monitoring the aero-engine gas...The variations in gas path parameter deviations can fully reflect the healthy state of aeroengine gas path components and units;therefore,airlines usually take them as key parameters for monitoring the aero-engine gas path performance state and conducting fault diagnosis.In the past,the airlines could not obtain deviations autonomously.At present,a data-driven method based on an aero-engine dataset with a large sample size can be utilized to obtain the deviations.However,it is still difficult to utilize aero-engine datasets with small sample sizes to establish regression models for deviations based on deep neural networks.To obtain monitoring autonomy of each aero-engine model,it is crucial to transfer and reuse the relevant knowledge of deviation modelling learned from different aero-engine models.This paper adopts the Residual-Back Propagation Neural Network(Res-BPNN)to deeply extract high-level features and stacks multi-layer Multi-Kernel Maximum Mean Discrepancy(MK-MMD)adaptation layers to map the extracted high-level features to the Reproduce Kernel Hilbert Space(RKHS)for discrepancy measurement.To further reduce the distribution discrepancy of each aero-engine model,the method of maximizing domain-confusion loss based on an adversarial mechanism is introduced to make the features learned from different domains as close as possible,and then the learned features can be confused.Through the above methods,domain-invariant features can be extracted,and the optimal adaptation effect can be achieved.Finally,the effectiveness of the proposed method is verified by using cruise data from different civil aero-engine models and compared with other transfer learning algorithms.展开更多
In practical mechanical fault detection and diagnosis,it is difficult and expensive to collect enough large-scale supervised data to train deep networks.Transfer learning can reuse the knowledge obtained from the sour...In practical mechanical fault detection and diagnosis,it is difficult and expensive to collect enough large-scale supervised data to train deep networks.Transfer learning can reuse the knowledge obtained from the source task to improve the performance of the target task,which performs well on small data and reduces the demand for high computation power.However,the detection performance is significantly reduced by the direct transfer due to the domain difference.Domain adaptation(DA)can transfer the distribution information from the source domain to the target domain and solve a series of problems caused by the distribution difference of data.In this survey,we review various current DA strategies combined with deep learning(DL)and analyze the principles,advantages,and disadvantages of each method.We also summarize the application of DA combined with DL in the field of fault diagnosis.This paper provides a summary of the research results and proposes future work based on analysis of the key technologies.展开更多
Traditional electroencephalograph(EEG)-based emotion recognition requires a large number of calibration samples to build a model for a specific subject,which restricts the application of the affective brain computer i...Traditional electroencephalograph(EEG)-based emotion recognition requires a large number of calibration samples to build a model for a specific subject,which restricts the application of the affective brain computer interface(BCI)in practice.We attempt to use the multi-modal data from the past session to realize emotion recognition in the case of a small amount of calibration samples.To solve this problem,we propose a multimodal domain adaptive variational autoencoder(MMDA-VAE)method,which learns shared cross-domain latent representations of the multi-modal data.Our method builds a multi-modal variational autoencoder(MVAE)to project the data of multiple modalities into a common space.Through adversarial learning and cycle-consistency regularization,our method can reduce the distribution difference of each domain on the shared latent representation layer and realize the transfer of knowledge.Extensive experiments are conducted on two public datasets,SEED and SEED-IV,and the results show the superiority of our proposed method.Our work can effectively improve the performance of emotion recognition with a small amount of labelled multi-modal data.展开更多
A common necessity for prior unsupervised domain adaptation methods that can improve the domain adaptation in unlabeled target domain dataset is access to source domain data-set and target domain dataset simultaneousl...A common necessity for prior unsupervised domain adaptation methods that can improve the domain adaptation in unlabeled target domain dataset is access to source domain data-set and target domain dataset simultaneously.However,data privacy makes it not always possible to access source domain dataset and target domain dataset in actual industrial equipment simulta-neously,especially for aviation component like Electro-Mechanical Actuator(EMA)whose dataset are often not shareable due to the data copyright and confidentiality.To address this problem,this paper proposes a source free unsupervised domain adaptation framework for EMA fault diagnosis.The proposed framework is a combination of feature network and classifier.Firstly,source domain datasets are only applied to train a source model.Secondly,the well-trained source model is trans-ferred to target domain and classifier is frozen based on source domain hypothesis.Thirdly,nearest centroid filtering is introduced to filter the reliable pseudo labels for unlabeled target domain data-set,and finally,supervised learning and pseudo label clustering are applied to fine-tune the trans-ferred model.In comparison with several traditional unsupervised domain adaptation methods,case studies based on low-and high-frequency monitoring signals on EMA indicate the effectiveness of the proposed method.展开更多
In machinery fault diagnosis,labeled data are always difficult or even impossible to obtain.Transfer learning can leverage related fault diagnosis knowledge from fully labeled source domain to enhance the fault diagno...In machinery fault diagnosis,labeled data are always difficult or even impossible to obtain.Transfer learning can leverage related fault diagnosis knowledge from fully labeled source domain to enhance the fault diagnosis performance in sparsely labeled or unlabeled target domain,which has been widely used for cross domain fault diagnosis.However,existing methods focus on either marginal distribution adaptation(MDA)or conditional distribution adaptation(CDA).In practice,marginal and conditional distributions discrepancies both have significant but different influences on the domain divergence.In this paper,a dynamic distribution adaptation based transfer network(DDATN)is proposed for cross domain bearing fault diagnosis.DDATN utilizes the proposed instance-weighted dynamic maximum mean discrepancy(IDMMD)for dynamic distribution adaptation(DDA),which can dynamically estimate the influences of marginal and conditional distribution and adapt target domain with source domain.The experimental evaluation on cross domain bearing fault diagnosis demonstrates that DDATN can outperformance the state-of-the-art cross domain fault diagnosis methods.展开更多
Sea fog detection with remote sensing images is a challenging task. Driven by the different image characteristics between fog and other types of clouds, such as textures and colors, it can be achieved by using image p...Sea fog detection with remote sensing images is a challenging task. Driven by the different image characteristics between fog and other types of clouds, such as textures and colors, it can be achieved by using image processing methods. Currently, most of the available methods are datadriven and relying on manual annotations. However, because few meteorological observations and buoys over the sea can be realized, obtaining visibility information to help the annotations is difficult. Considering the feasibility of obtaining abundant visible information over the land and the similarity between land fog and sea fog, we propose an unsupervised domain adaptation method to bridge the abundant labeled land fog data and the unlabeled sea fog data to realize the sea fog detection. We used a seeded region growing module to obtain pixel-level masks from roughlabels generated by the unsupervised domain adaptation model. Experimental results demonstrate that our proposed method achieves an accuracy of sea fog recognition up to 99.17%, which is nearly 3% higher than those vanilla methods.展开更多
Data fusion can effectively process multi-sensor information to obtain more accurate and reliable results than a single sensor.The data of water quality in the environment comes from different sensors,thus the data mu...Data fusion can effectively process multi-sensor information to obtain more accurate and reliable results than a single sensor.The data of water quality in the environment comes from different sensors,thus the data must be fused.In our research,self-adaptive weighted data fusion method is used to respectively integrate the data from the PH value,temperature,oxygen dissolved and NH3 concentration of water quality environment.Based on the fusion,the Grubbs method is used to detect the abnormal data so as to provide data support for estimation,prediction and early warning of the water quality.展开更多
To avoid the laborious annotation process for dense prediction tasks like semantic segmentation,unsupervised domain adaptation(UDA)methods have been proposed to leverage the abundant annotations from a source domain,s...To avoid the laborious annotation process for dense prediction tasks like semantic segmentation,unsupervised domain adaptation(UDA)methods have been proposed to leverage the abundant annotations from a source domain,such as virtual world(e.g.,3D games),and adapt models to the target domain(the real world)by narrowing the domain discrepancies.However,because of the large domain gap,directly aligning two distinct domains without considering the intermediates leads to inefficient alignment and inferior adaptation.To address this issue,we propose a novel learnable evolutionary Category Intermediates(CIs)guided UDA model named Leci,which enables the information transfer between the two domains via two processes,i.e.,Distilling and Blending.Starting from a random initialization,the CIs learn shared category-wise semantics automatically from two domains in the Distilling process.Then,the learned semantics in the CIs are sent back to blend the domain features through a residual attentive fusion(RAF)module,such that the categorywise features of both domains shift towards each other.As the CIs progressively and consistently learn from the varying feature distributions during training,they are evolutionary to guide the model to achieve category-wise feature alignment.Experiments on both GTA5 and SYNTHIA datasets demonstrate Leci's superiority over prior representative methods.展开更多
In this paper,we study the problem of domain adaptation,which is a crucial ingredient in transfer learning with two domains,that is,the source domain with labeled data and the target domain with none or few labels.Dom...In this paper,we study the problem of domain adaptation,which is a crucial ingredient in transfer learning with two domains,that is,the source domain with labeled data and the target domain with none or few labels.Domain adaptation aims to extract knowledge from the source domain to improve the performance of the learning task in the target domain.A popular approach to handle this problem is via adversarial training,which is explained by the H△H-distance theory.However,traditional adversarial network architectures just align the marginal feature distribution in the feature space.The alignment of class condition distribution is not guaranteed.Therefore,we proposed a novel method based on pseudo labels and the cluster assumption to avoid the incorrect class alignment in the feature space.The experiments demonstrate that our framework improves the accuracy on typical transfer learning tasks.展开更多
The state of health(SOH)is a critical factor in evaluating the performance of the lithium-ion batteries(LIBs).Due to various end-user behaviors,the LIBs exhibit different degradation modes,which makes it challenging t...The state of health(SOH)is a critical factor in evaluating the performance of the lithium-ion batteries(LIBs).Due to various end-user behaviors,the LIBs exhibit different degradation modes,which makes it challenging to estimate the SOHs in a personalized way.In this article,we present a novel particle swarm optimization-assisted deep domain adaptation(PSO-DDA)method to estimate the SOH of LIBs in a personalized manner,where a new domain adaptation strategy is put forward to reduce cross-domain distribution discrepancy.The standard PSO algorithm is exploited to automatically adjust the chosen hyperparameters of developed DDA-based method.The proposed PSODDA method is validated by extensive experiments on two LIB datasets with different battery chemistry materials,ambient temperatures and charge-discharge configurations.Experimental results indicate that the proposed PSO-DDA method surpasses the convolutional neural network-based method and the standard DDA-based method.The Py Torch implementation of the proposed PSO-DDA method is available at https://github.com/mxt0607/PSO-DDA.展开更多
Most existing domain adaptation(DA) methods aim to explore favorable performance under complicated environments by sampling.However,there are three unsolved problems that limit their efficiencies:ⅰ) they adopt global...Most existing domain adaptation(DA) methods aim to explore favorable performance under complicated environments by sampling.However,there are three unsolved problems that limit their efficiencies:ⅰ) they adopt global sampling but neglect to exploit global and local sampling simultaneously;ⅱ)they either transfer knowledge from a global perspective or a local perspective,while overlooking transmission of confident knowledge from both perspectives;and ⅲ) they apply repeated sampling during iteration,which takes a lot of time.To address these problems,knowledge transfer learning via dual density sampling(KTL-DDS) is proposed in this study,which consists of three parts:ⅰ) Dual density sampling(DDS) that jointly leverages two sampling methods associated with different views,i.e.,global density sampling that extracts representative samples with the most common features and local density sampling that selects representative samples with critical boundary information;ⅱ)Consistent maximum mean discrepancy(CMMD) that reduces intra-and cross-domain risks and guarantees high consistency of knowledge by shortening the distances of every two subsets among the four subsets collected by DDS;and ⅲ) Knowledge dissemination(KD) that transmits confident and consistent knowledge from the representative target samples with global and local properties to the whole target domain by preserving the neighboring relationships of the target domain.Mathematical analyses show that DDS avoids repeated sampling during the iteration.With the above three actions,confident knowledge with both global and local properties is transferred,and the memory and running time are greatly reduced.In addition,a general framework named dual density sampling approximation(DDSA) is extended,which can be easily applied to other DA algorithms.Extensive experiments on five datasets in clean,label corruption(LC),feature missing(FM),and LC&FM environments demonstrate the encouraging performance of KTL-DDS.展开更多
基金supported by the National Natural Science Foundation of China(NSFC)(Grant Nos.61876130 and 61932009).
文摘Multi-source domain adaptation utilizes multiple source domains to learn the knowledge and transfers it to an unlabeled target domain.To address the problem,most of the existing methods aim to minimize the domain shift by auxiliary distribution alignment objectives,which reduces the effect of domain-specific features.However,without explicitly modeling the domain-specific features,it is not easy to guarantee that the domain-invariant representation extracted from input domains contains domain-specific information as few as possible.In this work,we present a different perspective on MSDA,which employs the idea of feature elimination to reduce the influence of domain-specific features.We design two different ways to extract domain-specific features and total features and construct the domain-invariant representations by eliminating the domain-specific features from total features.The experimental results on different domain adaptation datasets demonstrate the effectiveness of our method and the generalization ability of our model.
基金Project supported by the National Nature Science Foundation of China (Nos. 61876130 and 61932009)the Starry Night Science Fund of Zhejiang University Shanghai Institute for Advanced Study。
文摘The goal of decentralized multi-source domain adaptation is to conduct unsupervised multi-source domain adaptation in a data decentralization scenario. The challenge of data decentralization is that the source domains and target domain lack cross-domain collaboration during training. On the unlabeled target domain, the target model needs to transfer supervision knowledge with the collaboration of source models, while the domain gap will lead to limited adaptation performance from source models. On the labeled source domain, the source model tends to overfit its domain data in the data decentralization scenario, which leads to the negative transfer problem. For these challenges, we propose dual collaboration for decentralized multi-source domain adaptation by training and aggregating the local source models and local target model in collaboration with each other. On the target domain, we train the local target model by distilling supervision knowledge and fully using the unlabeled target domain data to alleviate the domain shift problem with the collaboration of local source models. On the source domain, we regularize the local source models in collaboration with the local target model to overcome the negative transfer problem. This forms a dual collaboration between the decentralized source domains and target domain, which improves the domain adaptation performance under the data decentralization scenario. Extensive experiments indicate that our method outperforms the state-of-the-art methods by a large margin on standard multi-source domain adaptation datasets.
基金Supported by the Fund for Shanxi“1331 Project”and Supported by Fundamental Research Program of Shanxi Province(No.202203021211006)the Key Research,Development Program of Shanxi Province(No.201903D311009)+4 种基金the Key Research Program of Taiyuan University(No.21TYKZ01)the Open Fund of Shanxi Province Key Laboratory of Ophthalmology(No.2023SXKLOS04)Shenzhen Fund for Guangdong Provincial High-Level Clinical Key Specialties(No.SZGSP014)Sanming Project of Medicine in Shenzhen(No.SZSM202311012)Shenzhen Science and Technology Planning Project(No.KCXFZ20211020163813019).
文摘AIM:To address the challenges of data labeling difficulties,data privacy,and necessary large amount of labeled data for deep learning methods in diabetic retinopathy(DR)identification,the aim of this study is to develop a source-free domain adaptation(SFDA)method for efficient and effective DR identification from unlabeled data.METHODS:A multi-SFDA method was proposed for DR identification.This method integrates multiple source models,which are trained from the same source domain,to generate synthetic pseudo labels for the unlabeled target domain.Besides,a softmax-consistence minimization term is utilized to minimize the intra-class distances between the source and target domains and maximize the inter-class distances.Validation is performed using three color fundus photograph datasets(APTOS2019,DDR,and EyePACS).RESULTS:The proposed model was evaluated and provided promising results with respectively 0.8917 and 0.9795 F1-scores on referable and normal/abnormal DR identification tasks.It demonstrated effective DR identification through minimizing intra-class distances and maximizing inter-class distances between source and target domains.CONCLUSION:The multi-SFDA method provides an effective approach to overcome the challenges in DR identification.The method not only addresses difficulties in data labeling and privacy issues,but also reduces the need for large amounts of labeled data required by deep learning methods,making it a practical tool for early detection and preservation of vision in diabetic patients.
文摘In recent years,the heterogeneous SAR image classification task of"training on simulated data and testing on measured data"has garnered increasing attention in the field of Synthetic Aperture Radar Automatic Target Recognition(SAR-ATR).Although current mainstream domain adaptation methods have made significant breakthroughs in addressing domain shift problems,the escalating model complexity and task complexity have constrained their deployment in real-world applications.To tackle this challenge,this paper proposes a domain adaptation framework based on linear-kernel Maximum Mean Discrepancy(MMD),integrated with a near-zero-cost pseudo-label denoising technique leveraging deep feature clustering.Our method completely eliminates the need for data augmentation and handcrafted feature design,achieving endto-end pseudo-label self-training.Competitive performance is demonstrated across three typical scenarios in the SAMPLE dataset,with the highest accuracy of 98.65%achieved in ScenarioⅢ.The relevant code is available at:https://github.com/TheGreatTreatsby/SAMPLE_MMD.
基金the National Natural Science Foundation of China(Grant No.42301002,and 52109118)Fujian Provincial Water Resources Science and Technology Project(Grant No.MSK202524)Guidance fund for Science and Technology Program,Fujian province(Grant No.2024Y0002).
文摘Landslide susceptibility evaluation plays an important role in disaster prevention and reduction.Feature-based transfer learning(TL)is an effective method for solving landslide susceptibility mapping(LSM)in target regions with no available samples.However,as the study area expands,the distribution of land-slide types and triggering mechanisms becomes more diverse,leading to performance degradation in models relying on landslide evaluation knowledge from a single source domain due to domain feature shift.To address this,this study proposes a Multi-source Domain Adaptation Convolutional Neural Network(MDACNN),which combines the landslide prediction knowledge learned from two source domains to perform cross-regional LSM in complex large-scale areas.The method is validated through case studies in three regions located in southeastern coastal China and compared with single-source domain TL models(TCA-based models).The results demonstrate that MDACNN effectively integrates transfer knowledge from multiple source domains to learn diverse landslide-triggering mechanisms,thereby significantly reducing prediction bias inherent to single-source domain TL models,achieving an average improvement of 16.58%across all metrics.Moreover,the landslide susceptibility maps gener-ated by MDACNN accurately quantify the spatial distribution of landslide risks in the target area,provid-ing a powerful scientific and technological tool for landslide disaster management and prevention.
基金Project supported by the National Natural Science Foundation of China(Grant Nos.62375140 and 61871234)。
文摘Quantum machine learning is an important application of quantum computing in the era of noisy intermediate-scale quantum devices.Domain adaptation(DA)is an effective method for addressing the distribution discrepancy problem between the training data and the real data when the neural network model is deployed.In this paper,we propose a variational quantum domain adaptation method inspired by the quantum convolutional neural network,named variational quantum domain adaptation(VQDA).The data are first uploaded by a‘quantum coding module',then the feature information is extracted by several‘quantum convolution layers'and‘quantum pooling layers',which is named‘Feature Extractor'.Subsequently,the labels and the domains of the samples are obtained by the‘quantum fully connected layer'.With a gradient reversal module,the trained‘Feature Extractor'can extract the features that cannot be distinguished from the source and target domains.The simulations on the local computer and IBM Quantum Experience(IBM Q)platform by Qiskit show the effectiveness of the proposed method.The results show that VQDA(with 8 quantum bits)has 91.46%average classification accuracy for DA task between MNIST→USPS(USPS→MNIST),achieves 91.16%average classification accuracy for gray-scale and color images(with 10 quantum bits),and has 69.25%average classification accuracy on the DA task for color images(also with 10 quantum bits).VQDA achieves a 9.14%improvement in average classification accuracy compared to its corresponding classical domain adaptation method with the same parameter scale for different DA tasks.Simultaneously,the parameters scale is reduced to 43%by using VQDA when both quantum and classical DA methods have similar classification accuracies.
基金supported by the Fund of Key Laboratory of Biomedical Engineering of Hainan Province(No.BME20240001)the STI2030-Major Projects(No.2021ZD0200104)the National Natural Science Foundations of China under Grant 61771437.
文摘Deep learning networks are increasingly exploited in the field of neuronal soma segmentation.However,annotating dataset is also an expensive and time-consuming task.Unsupervised domain adaptation is an effective method to mitigate the problem,which is able to learn an adaptive segmentation model by transferring knowledge from a rich-labeled source domain.In this paper,we propose a multi-level distribution alignment-based unsupervised domain adaptation network(MDA-Net)for segmentation of 3D neuronal soma images.Distribution alignment is performed in both feature space and output space.In the feature space,features from different scales are adaptively fused to enhance the feature extraction capability for small target somata and con-strained to be domain invariant by adversarial adaptation strategy.In the output space,local discrepancy maps that can reveal the spatial structures of somata are constructed on the predicted segmentation results.Then thedistribution alignment is performed on the local discrepancies maps across domains to obtain a superior discrepancy map in the target domain,achieving refined segmentation performance of neuronal somata.Additionally,after a period of distribution align-ment procedure,a portion of target samples with high confident pseudo-labels are selected as training data,which assist in learning a more adaptive segmentation network.We verified the superiority of the proposed algorithm by comparing several domain adaptation networks on two 3D mouse brain neuronal somata datasets and one macaque brain neuronal soma dataset.
基金supported by the Industry-University Cooperation Project in Fujian Province University(No.2023H6006)the State Key Laboratory of Reliability and Intelligence of Electrical Equipment(No.EERI-KF20200005)。
文摘Artificial intelligence has potential for forecasting reactor conditions in the nuclear industry.Owing to economic and security concerns,a common method is to train data generated by simulators.However,achieving a satisfactory performance in practical applications is difficult because simulators imperfectly emulate reality.To bridge this gap,we propose a novel framework called simulation-to-reality domain adaptation(SRDA)for forecasting the operating parameters of nuclear reactors.The SRDA model employs a transformer-based feature extractor to capture dynamic characteristics and temporal dependencies.A parameter predictor with an improved logarithmic loss function is specifically designed to adapt to varying reactor powers.To fuse prior reactor knowledge from simulations with reality,the domain discriminator utilizes an adversarial strategy to ensure the learning of deep domain-invariant features,and the multiple kernel maximum mean discrepancy minimizes their discrepancies.Experiments on neutron fluxes and temperatures from a pressurized water reactor illustrate that the SRDA model surpasses various advanced methods in terms of predictive performance.This study is the first to use domain adaptation for real-world reactor prediction and presents a feasible solution for enhancing the transferability and generalizability of simulated data.
基金Supported by the Scientific and Technological Innovation 2030—Major Project of"New Generation Artificial Intelligence"(2020AAA0109300)。
文摘Smart manufacturing suffers from the heterogeneity of local data distribution across parties,mutual information silos and lack of privacy protection in the process of industry chain collaboration.To address these problems,we propose a federated domain adaptation algorithm based on knowledge distillation and contrastive learning.Knowledge distillation is used to extract transferable integration knowledge from the different source domains and the quality of the extracted integration knowledge is used to assign reasonable weights to each source domain.A more rational weighted average aggregation is used in the aggregation phase of the center server to optimize the global model,while the local model of the source domain is trained with the help of contrastive learning to constrain the local model optimum towards the global model optimum,mitigating the inherent heterogeneity between local data.Our experiments are conducted on the largest domain adaptation dataset,and the results show that compared with other traditional federated domain adaptation algorithms,the algorithm we proposed trains a more accurate model,requires fewer communication rounds,makes more effective use of imbalanced data in the industrial area,and protects data privacy.
基金supported by the Shandong Provincial Natural Science Foundation(No.ZR2019MEE096)the Key National Natural Science Foundation of China(No.U1733201)。
文摘The variations in gas path parameter deviations can fully reflect the healthy state of aeroengine gas path components and units;therefore,airlines usually take them as key parameters for monitoring the aero-engine gas path performance state and conducting fault diagnosis.In the past,the airlines could not obtain deviations autonomously.At present,a data-driven method based on an aero-engine dataset with a large sample size can be utilized to obtain the deviations.However,it is still difficult to utilize aero-engine datasets with small sample sizes to establish regression models for deviations based on deep neural networks.To obtain monitoring autonomy of each aero-engine model,it is crucial to transfer and reuse the relevant knowledge of deviation modelling learned from different aero-engine models.This paper adopts the Residual-Back Propagation Neural Network(Res-BPNN)to deeply extract high-level features and stacks multi-layer Multi-Kernel Maximum Mean Discrepancy(MK-MMD)adaptation layers to map the extracted high-level features to the Reproduce Kernel Hilbert Space(RKHS)for discrepancy measurement.To further reduce the distribution discrepancy of each aero-engine model,the method of maximizing domain-confusion loss based on an adversarial mechanism is introduced to make the features learned from different domains as close as possible,and then the learned features can be confused.Through the above methods,domain-invariant features can be extracted,and the optimal adaptation effect can be achieved.Finally,the effectiveness of the proposed method is verified by using cruise data from different civil aero-engine models and compared with other transfer learning algorithms.
基金supported by the National Natural Science Foundation of China(Grant Nos.52175096,51775243,11902124),the fellowship of China Postdoctoral Science Foundation(Grant No.2021T140279)111 Project(Grant No.B18027).
文摘In practical mechanical fault detection and diagnosis,it is difficult and expensive to collect enough large-scale supervised data to train deep networks.Transfer learning can reuse the knowledge obtained from the source task to improve the performance of the target task,which performs well on small data and reduces the demand for high computation power.However,the detection performance is significantly reduced by the direct transfer due to the domain difference.Domain adaptation(DA)can transfer the distribution information from the source domain to the target domain and solve a series of problems caused by the distribution difference of data.In this survey,we review various current DA strategies combined with deep learning(DL)and analyze the principles,advantages,and disadvantages of each method.We also summarize the application of DA combined with DL in the field of fault diagnosis.This paper provides a summary of the research results and proposes future work based on analysis of the key technologies.
基金National Natural Science Foundation of China(61976209,62020106015,U21A20388)in part by the CAS International Collaboration Key Project(173211KYSB20190024)in part by the Strategic Priority Research Program of CAS(XDB32040000)。
文摘Traditional electroencephalograph(EEG)-based emotion recognition requires a large number of calibration samples to build a model for a specific subject,which restricts the application of the affective brain computer interface(BCI)in practice.We attempt to use the multi-modal data from the past session to realize emotion recognition in the case of a small amount of calibration samples.To solve this problem,we propose a multimodal domain adaptive variational autoencoder(MMDA-VAE)method,which learns shared cross-domain latent representations of the multi-modal data.Our method builds a multi-modal variational autoencoder(MVAE)to project the data of multiple modalities into a common space.Through adversarial learning and cycle-consistency regularization,our method can reduce the distribution difference of each domain on the shared latent representation layer and realize the transfer of knowledge.Extensive experiments are conducted on two public datasets,SEED and SEED-IV,and the results show the superiority of our proposed method.Our work can effectively improve the performance of emotion recognition with a small amount of labelled multi-modal data.
基金supported by the National Natural Science Foundation of China(No.52075349)the Aeronautical Science Foundation of China(No.201905019001)the China Scholarship Council(No.202106240078).
文摘A common necessity for prior unsupervised domain adaptation methods that can improve the domain adaptation in unlabeled target domain dataset is access to source domain data-set and target domain dataset simultaneously.However,data privacy makes it not always possible to access source domain dataset and target domain dataset in actual industrial equipment simulta-neously,especially for aviation component like Electro-Mechanical Actuator(EMA)whose dataset are often not shareable due to the data copyright and confidentiality.To address this problem,this paper proposes a source free unsupervised domain adaptation framework for EMA fault diagnosis.The proposed framework is a combination of feature network and classifier.Firstly,source domain datasets are only applied to train a source model.Secondly,the well-trained source model is trans-ferred to target domain and classifier is frozen based on source domain hypothesis.Thirdly,nearest centroid filtering is introduced to filter the reliable pseudo labels for unlabeled target domain data-set,and finally,supervised learning and pseudo label clustering are applied to fine-tune the trans-ferred model.In comparison with several traditional unsupervised domain adaptation methods,case studies based on low-and high-frequency monitoring signals on EMA indicate the effectiveness of the proposed method.
基金Supported by National Natural Science Foundation of China(Grant Nos.51875208,51475170)National Key Research and Development Program of China(Grant No.2018YFB1702400).
文摘In machinery fault diagnosis,labeled data are always difficult or even impossible to obtain.Transfer learning can leverage related fault diagnosis knowledge from fully labeled source domain to enhance the fault diagnosis performance in sparsely labeled or unlabeled target domain,which has been widely used for cross domain fault diagnosis.However,existing methods focus on either marginal distribution adaptation(MDA)or conditional distribution adaptation(CDA).In practice,marginal and conditional distributions discrepancies both have significant but different influences on the domain divergence.In this paper,a dynamic distribution adaptation based transfer network(DDATN)is proposed for cross domain bearing fault diagnosis.DDATN utilizes the proposed instance-weighted dynamic maximum mean discrepancy(IDMMD)for dynamic distribution adaptation(DDA),which can dynamically estimate the influences of marginal and conditional distribution and adapt target domain with source domain.The experimental evaluation on cross domain bearing fault diagnosis demonstrates that DDATN can outperformance the state-of-the-art cross domain fault diagnosis methods.
基金supported in part by the Ministry of Education-China Mobile Communication Corp(MoE-CMCC)Artificial Intelligence Project,China(No.MCM20190701)。
文摘Sea fog detection with remote sensing images is a challenging task. Driven by the different image characteristics between fog and other types of clouds, such as textures and colors, it can be achieved by using image processing methods. Currently, most of the available methods are datadriven and relying on manual annotations. However, because few meteorological observations and buoys over the sea can be realized, obtaining visibility information to help the annotations is difficult. Considering the feasibility of obtaining abundant visible information over the land and the similarity between land fog and sea fog, we propose an unsupervised domain adaptation method to bridge the abundant labeled land fog data and the unlabeled sea fog data to realize the sea fog detection. We used a seeded region growing module to obtain pixel-level masks from roughlabels generated by the unsupervised domain adaptation model. Experimental results demonstrate that our proposed method achieves an accuracy of sea fog recognition up to 99.17%, which is nearly 3% higher than those vanilla methods.
基金This study was supported by National Key Research and Development Project(Project No.2017YFD0301506)National Social Science Foundation(Project No.71774052)+1 种基金Hunan Education Department Scientific Research Project(Project No.17K04417A092).
文摘Data fusion can effectively process multi-sensor information to obtain more accurate and reliable results than a single sensor.The data of water quality in the environment comes from different sensors,thus the data must be fused.In our research,self-adaptive weighted data fusion method is used to respectively integrate the data from the PH value,temperature,oxygen dissolved and NH3 concentration of water quality environment.Based on the fusion,the Grubbs method is used to detect the abnormal data so as to provide data support for estimation,prediction and early warning of the water quality.
基金Australian Research Council Project(FL-170100117).
文摘To avoid the laborious annotation process for dense prediction tasks like semantic segmentation,unsupervised domain adaptation(UDA)methods have been proposed to leverage the abundant annotations from a source domain,such as virtual world(e.g.,3D games),and adapt models to the target domain(the real world)by narrowing the domain discrepancies.However,because of the large domain gap,directly aligning two distinct domains without considering the intermediates leads to inefficient alignment and inferior adaptation.To address this issue,we propose a novel learnable evolutionary Category Intermediates(CIs)guided UDA model named Leci,which enables the information transfer between the two domains via two processes,i.e.,Distilling and Blending.Starting from a random initialization,the CIs learn shared category-wise semantics automatically from two domains in the Distilling process.Then,the learned semantics in the CIs are sent back to blend the domain features through a residual attentive fusion(RAF)module,such that the categorywise features of both domains shift towards each other.As the CIs progressively and consistently learn from the varying feature distributions during training,they are evolutionary to guide the model to achieve category-wise feature alignment.Experiments on both GTA5 and SYNTHIA datasets demonstrate Leci's superiority over prior representative methods.
基金supported by the National Key Research and Development Program of China(No.2016YFB0901902)the National Natural Science Foundation of China(No.61733018).
文摘In this paper,we study the problem of domain adaptation,which is a crucial ingredient in transfer learning with two domains,that is,the source domain with labeled data and the target domain with none or few labels.Domain adaptation aims to extract knowledge from the source domain to improve the performance of the learning task in the target domain.A popular approach to handle this problem is via adversarial training,which is explained by the H△H-distance theory.However,traditional adversarial network architectures just align the marginal feature distribution in the feature space.The alignment of class condition distribution is not guaranteed.Therefore,we proposed a novel method based on pseudo labels and the cluster assumption to avoid the incorrect class alignment in the feature space.The experiments demonstrate that our framework improves the accuracy on typical transfer learning tasks.
基金supported in part by the National Natural Science Foundation of China(92167201,62273264,61933007)。
文摘The state of health(SOH)is a critical factor in evaluating the performance of the lithium-ion batteries(LIBs).Due to various end-user behaviors,the LIBs exhibit different degradation modes,which makes it challenging to estimate the SOHs in a personalized way.In this article,we present a novel particle swarm optimization-assisted deep domain adaptation(PSO-DDA)method to estimate the SOH of LIBs in a personalized manner,where a new domain adaptation strategy is put forward to reduce cross-domain distribution discrepancy.The standard PSO algorithm is exploited to automatically adjust the chosen hyperparameters of developed DDA-based method.The proposed PSODDA method is validated by extensive experiments on two LIB datasets with different battery chemistry materials,ambient temperatures and charge-discharge configurations.Experimental results indicate that the proposed PSO-DDA method surpasses the convolutional neural network-based method and the standard DDA-based method.The Py Torch implementation of the proposed PSO-DDA method is available at https://github.com/mxt0607/PSO-DDA.
基金supported in part by the Key-Area Research and Development Program of Guangdong Province (2020B010166006)the National Natural Science Foundation of China (61972102)+1 种基金the Guangzhou Science and Technology Plan Project (023A04J1729)the Science and Technology development fund (FDCT),Macao SAR (015/2020/AMJ)。
文摘Most existing domain adaptation(DA) methods aim to explore favorable performance under complicated environments by sampling.However,there are three unsolved problems that limit their efficiencies:ⅰ) they adopt global sampling but neglect to exploit global and local sampling simultaneously;ⅱ)they either transfer knowledge from a global perspective or a local perspective,while overlooking transmission of confident knowledge from both perspectives;and ⅲ) they apply repeated sampling during iteration,which takes a lot of time.To address these problems,knowledge transfer learning via dual density sampling(KTL-DDS) is proposed in this study,which consists of three parts:ⅰ) Dual density sampling(DDS) that jointly leverages two sampling methods associated with different views,i.e.,global density sampling that extracts representative samples with the most common features and local density sampling that selects representative samples with critical boundary information;ⅱ)Consistent maximum mean discrepancy(CMMD) that reduces intra-and cross-domain risks and guarantees high consistency of knowledge by shortening the distances of every two subsets among the four subsets collected by DDS;and ⅲ) Knowledge dissemination(KD) that transmits confident and consistent knowledge from the representative target samples with global and local properties to the whole target domain by preserving the neighboring relationships of the target domain.Mathematical analyses show that DDS avoids repeated sampling during the iteration.With the above three actions,confident knowledge with both global and local properties is transferred,and the memory and running time are greatly reduced.In addition,a general framework named dual density sampling approximation(DDSA) is extended,which can be easily applied to other DA algorithms.Extensive experiments on five datasets in clean,label corruption(LC),feature missing(FM),and LC&FM environments demonstrate the encouraging performance of KTL-DDS.
