Domain adaptation aims to reduce the distribution gap between the training data(source domain)and the target data.This enables effective predictions even for domains not seen during training.However,most conventional ...Domain adaptation aims to reduce the distribution gap between the training data(source domain)and the target data.This enables effective predictions even for domains not seen during training.However,most conventional domain adaptation methods assume a single source domain,making them less suitable for modern deep learning settings that rely on diverse and large-scale datasets.To address this limitation,recent research has focused on Multi-Source Domain Adaptation(MSDA),which aims to learn effectively from multiple source domains.In this paper,we propose Efficient Domain Transition for Multi-source(EDTM),a novel and efficient framework designed to tackle two major challenges in existing MSDA approaches:(1)integrating knowledge across different source domains and(2)aligning label distributions between source and target domains.EDTM leverages an ensemble-based classifier expert mechanism to enhance the contribution of source domains that are more similar to the target domain.To further stabilize the learning process and improve performance,we incorporate imitation learning into the training of the target model.In addition,Maximum Classifier Discrepancy(MCD)is employed to align class-wise label distributions between the source and target domains.Experiments were conducted using Digits-Five,one of the most representative benchmark datasets for MSDA.The results show that EDTM consistently outperforms existing methods in terms of average classification accuracy.Notably,EDTM achieved significantly higher performance on target domains such as Modified National Institute of Standards and Technolog with blended background images(MNIST-M)and Street View House Numbers(SVHN)datasets,demonstrating enhanced generalization compared to baseline approaches.Furthermore,an ablation study analyzing the contribution of each loss component validated the effectiveness of the framework,highlighting the importance of each module in achieving optimal performance.展开更多
Named Entity Recognition(NER)is vital in natural language processing for the analysis of news texts,as it accurately identifies entities such as locations,persons,and organizations,which is crucial for applications li...Named Entity Recognition(NER)is vital in natural language processing for the analysis of news texts,as it accurately identifies entities such as locations,persons,and organizations,which is crucial for applications like news summarization and event tracking.However,NER in the news domain faces challenges due to insufficient annotated data,complex entity structures,and strong context dependencies.To address these issues,we propose a new Chinesenamed entity recognition method that integrates transfer learning with word embeddings.Our approach leverages the ERNIE pre-trained model for transfer learning and obtaining general language representations and incorporates the Soft-lexicon word embedding technique to handle varied entity structures.This dual-strategy enhances the model’s understanding of context and boosts its ability to process complex texts.Experimental results show that our method achieves an F1 score of 94.72% on a news dataset,surpassing baseline methods by 3%–4%,thereby confirming its effectiveness for Chinese-named entity recognition in the news domain.展开更多
Many applications,including security systems,medical diagnostics,and human-computer interfaces,depend on eye gaze recognition.However,due to factors including individual variations,occlusions,and shifting illumination...Many applications,including security systems,medical diagnostics,and human-computer interfaces,depend on eye gaze recognition.However,due to factors including individual variations,occlusions,and shifting illumination conditions,real-world scenarios continue to provide difficulties for accurate and consistent eye gaze recognition.This work is aimed at investigating the potential benefits of employing transfer learning to improve eye gaze detection ability and efficiency.Transfer learning is the process of fine-tuning pre-trained models on smaller,domain-specific datasets after they have been trained on larger datasets.We study several transfer learning algorithms and evaluate their effectiveness on eye gaze identification,including both Regression and Classification tasks,using a range of deep learning architectures,namely AlexNet,Visual Geometry Group(VGG),InceptionV3,and ResNet.In this study,we evaluate the effectiveness of transfer learning-basedmodels against models that were trained fromscratch using eye-gazing datasets on grounds of various performance and loss metrics such as Precision,Accuracy,and Mean Absolute Error.We investigate the effects of different pre-trainedmodels,dataset sizes,and domain gaps on the transfer learning process,and the findings of our study clarify the efficacy of transfer learning for eye gaze detection and offer suggestions for the most successful transfer learning strategies to apply in real-world situations.展开更多
Despite the big success of transfer learning techniques in anomaly detection,it is still challenging to achieve good transition of detection rules merely based on the preferred data in the anomaly detection with one-c...Despite the big success of transfer learning techniques in anomaly detection,it is still challenging to achieve good transition of detection rules merely based on the preferred data in the anomaly detection with one-class classification,especially for the data with a large distribution difference.To address this challenge,a novel deep one-class transfer learning algorithm with domain-adversarial training is proposed in this paper.First,by integrating a hypersphere adaptation constraint into domainadversarial neural network,a new hypersphere adversarial training mechanism is designed.Second,an alternative optimization method is derived to seek the optimal network parameters while pushing the hyperspheres built in the source domain and target domain to be as identical as possible.Through transferring oneclass detection rule in the adaptive extraction of domain-invariant feature representation,the end-to-end anomaly detection with one-class classification is then enhanced.Furthermore,a theoretical analysis about the model reliability,as well as the strategy of avoiding invalid and negative transfer,is provided.Experiments are conducted on two typical anomaly detection problems,i.e.,image recognition detection and online early fault detection of rolling bearings.The results demonstrate that the proposed algorithm outperforms the state-of-the-art methods in terms of detection accuracy and robustness.展开更多
Most existing domain adaptation(DA) methods aim to explore favorable performance under complicated environments by sampling.However,there are three unsolved problems that limit their efficiencies:ⅰ) they adopt global...Most existing domain adaptation(DA) methods aim to explore favorable performance under complicated environments by sampling.However,there are three unsolved problems that limit their efficiencies:ⅰ) they adopt global sampling but neglect to exploit global and local sampling simultaneously;ⅱ)they either transfer knowledge from a global perspective or a local perspective,while overlooking transmission of confident knowledge from both perspectives;and ⅲ) they apply repeated sampling during iteration,which takes a lot of time.To address these problems,knowledge transfer learning via dual density sampling(KTL-DDS) is proposed in this study,which consists of three parts:ⅰ) Dual density sampling(DDS) that jointly leverages two sampling methods associated with different views,i.e.,global density sampling that extracts representative samples with the most common features and local density sampling that selects representative samples with critical boundary information;ⅱ)Consistent maximum mean discrepancy(CMMD) that reduces intra-and cross-domain risks and guarantees high consistency of knowledge by shortening the distances of every two subsets among the four subsets collected by DDS;and ⅲ) Knowledge dissemination(KD) that transmits confident and consistent knowledge from the representative target samples with global and local properties to the whole target domain by preserving the neighboring relationships of the target domain.Mathematical analyses show that DDS avoids repeated sampling during the iteration.With the above three actions,confident knowledge with both global and local properties is transferred,and the memory and running time are greatly reduced.In addition,a general framework named dual density sampling approximation(DDSA) is extended,which can be easily applied to other DA algorithms.Extensive experiments on five datasets in clean,label corruption(LC),feature missing(FM),and LC&FM environments demonstrate the encouraging performance of KTL-DDS.展开更多
Accurate displacement prediction is critical for the early warning of landslides.The complexity of the coupling relationship between multiple influencing factors and displacement makes the accurate prediction of displ...Accurate displacement prediction is critical for the early warning of landslides.The complexity of the coupling relationship between multiple influencing factors and displacement makes the accurate prediction of displacement difficult.Moreover,in engineering practice,insufficient monitoring data limit the performance of prediction models.To alleviate this problem,a displacement prediction method based on multisource domain transfer learning,which helps accurately predict data in the target domain through the knowledge of one or more source domains,is proposed.First,an optimized variational mode decomposition model based on the minimum sample entropy is used to decompose the cumulative displacement into the trend,periodic,and stochastic components.The trend component is predicted by an autoregressive model,and the periodic component is predicted by the long short-term memory.For the stochastic component,because it is affected by uncertainties,it is predicted by a combination of a Wasserstein generative adversarial network and multisource domain transfer learning for improved prediction accuracy.Considering a real mine slope as a case study,the proposed prediction method was validated.Therefore,this study provides new insights that can be applied to scenarios lacking sample data.展开更多
Transfer learning aims to transfer source models to a target domain.Leveraging the feature matching can alleviate the domain shift effectively,but this process ignores the relationship of the marginal distribution mat...Transfer learning aims to transfer source models to a target domain.Leveraging the feature matching can alleviate the domain shift effectively,but this process ignores the relationship of the marginal distribution matching and the conditional distribution matching.Simultaneously,the discriminative information of both domains is also neglected,which is important for improving the performance on the target domain.In this paper,we propose a novel method called Balanced Discriminative Transfer Feature Learning for Visual Domain Adaptation(BDTFL).The proposed method can adaptively balance the relationship of both distribution matchings and capture the category discriminative information of both domains.Therefore,balanced feature matching can achieve more accurate feature matching and adaptively adjust itself to different scenes.At the same time,discriminative information is exploited to alleviate category confusion during feature matching.And with assistance of the category discriminative information captured from both domains,the source classifier can be transferred to the target domain more accurately and boost the performance of target classification.Extensive experiments show the superiority of BDTFL on popular visual cross-domain benchmarks.展开更多
The non-stationary of the motor imagery electroencephalography(MI-EEG)signal is one of the main limitations for the development of motor imagery brain-computer interfaces(MI-BCI).The nonstationary of the MI-EEG signal...The non-stationary of the motor imagery electroencephalography(MI-EEG)signal is one of the main limitations for the development of motor imagery brain-computer interfaces(MI-BCI).The nonstationary of the MI-EEG signal and the changes of the experimental environment make the feature distribution of the testing set and training set deviates,which reduces the classification accuracy of MI-BCI.In this paper,we propose a Kullback–Leibler divergence(KL)-based transfer learning algorithm to solve the problem of feature transfer,the proposed algorithm uses KL to measure the similarity between the training set and the testing set,adds support vector machine(SVM)classification probability to classify and weight the covariance,and discards the poorly performing samples.The results show that the proposed algorithm can significantly improve the classification accuracy of the testing set compared with the traditional algorithms,especially for subjects with medium classification accuracy.Moreover,the algorithm based on transfer learning has the potential to improve the consistency of feature distribution that the traditional algorithms do not have,which is significant for the application of MI-BCI.展开更多
Recent works have shown that neural networks are promising parameter-free limiters for a variety of numerical schemes(Morgan et al.in A machine learning approach for detect-ing shocks with high-order hydrodynamic meth...Recent works have shown that neural networks are promising parameter-free limiters for a variety of numerical schemes(Morgan et al.in A machine learning approach for detect-ing shocks with high-order hydrodynamic methods.et al.in J Comput Phys 367:166-191.,2018;Veiga et al.in European Conference on Computational Mechanics andⅦEuropean Conference on Computational Fluid Dynamics,vol.1,pp.2525-2550.ECCM.,2018).Following this trend,we train a neural network to serve as a shock-indicator function using simulation data from a Runge-Kutta discontinuous Galer-kin(RKDG)method and a modal high-order limiter(Krivodonova in J Comput Phys 226:879-896.,2007).With this methodology,we obtain one-and two-dimensional black-box shock-indicators which are then coupled to a standard limiter.Furthermore,we describe a strategy to transfer the shock-indicator to a residual distribution(RD)scheme without the need for a full training cycle and large data-set,by finding a mapping between the solution feature spaces from an RD scheme to an RKDG scheme,both in one-and two-dimensional problems,and on Cartesian and unstruc-tured meshes.We report on the quality of the numerical solutions when using the neural network shock-indicator coupled to a limiter,comparing its performance to traditional lim-iters,for both RKDG and RD schemes.展开更多
The wear of metal cutting tools will progressively rise as the cutting time goes on. Wearing heavily on the toolwill generate significant noise and vibration, negatively impacting the accuracy of the forming and the s...The wear of metal cutting tools will progressively rise as the cutting time goes on. Wearing heavily on the toolwill generate significant noise and vibration, negatively impacting the accuracy of the forming and the surfaceintegrity of the workpiece. Hence, during the cutting process, it is imperative to continually monitor the tool wearstate andpromptly replace anyheavilyworn tools toguarantee thequality of the cutting.The conventional tool wearmonitoring models, which are based on machine learning, are specifically built for the intended cutting conditions.However, these models require retraining when the cutting conditions undergo any changes. This method has noapplication value if the cutting conditions frequently change. This manuscript proposes a method for monitoringtool wear basedonunsuperviseddeep transfer learning. Due to the similarity of the tool wear process under varyingworking conditions, a tool wear recognitionmodel that can adapt to both current and previous working conditionshas been developed by utilizing cutting monitoring data from history. To extract and classify cutting vibrationsignals, the unsupervised deep transfer learning network comprises a one-dimensional (1D) convolutional neuralnetwork (CNN) with a multi-layer perceptron (MLP). To achieve distribution alignment of deep features throughthe maximum mean discrepancy algorithm, a domain adaptive layer is embedded in the penultimate layer of thenetwork. A platformformonitoring tool wear during endmilling has been constructed. The proposedmethod wasverified through the execution of a full life test of end milling under multiple working conditions with a Cr12MoVsteel workpiece. Our experiments demonstrate that the transfer learning model maintains a classification accuracyof over 80%. In comparisonwith the most advanced tool wearmonitoring methods, the presentedmodel guaranteessuperior performance in the target domains.展开更多
Machine-learning methodologies have increasingly been embraced in landslide susceptibility assessment.However,the considerable time and financial burdens of landslide inventories often result in persistent data scarci...Machine-learning methodologies have increasingly been embraced in landslide susceptibility assessment.However,the considerable time and financial burdens of landslide inventories often result in persistent data scarcity,which frequently impedes the generation of accurate and informative landslide susceptibility maps.Addressing this challenge,this study compiled a nationwide dataset and developed a transfer learning-based model to evaluate landslide susceptibility in the Chongqing region specifically.Notably,the proposed model,calibrated with the warmup-cosine annealing(WCA)learning rate strategy,demonstrated remarkable predictive capabilities,particularly in scenarios marked by data limitations and when training data were normalized using parameters from the source region.This is evidenced by the area under the receiver operating characteristic curve(AUC)values,which exhibited significant improvements of 51.00%,24.40%and 2.15%,respectively,compared to a deep learning model,in contexts where only 1%,5%and 10%of data from the target region were used for retraining.Simultaneously,there were reductions in loss of 16.12%,27.61%and 15.44%,respectively,in these instances.展开更多
Domain adaptation and adversarial networks are two main approaches for transfer learning.Domain adaptation methods match the mean values of source and target domains,which requires a very large batch size during train...Domain adaptation and adversarial networks are two main approaches for transfer learning.Domain adaptation methods match the mean values of source and target domains,which requires a very large batch size during training.However,adversarial networks are usually unstable when training.In this paper,we propose a joint method of feature matching and adversarial networks to reduce domain discrepancy and mine domaininvariant features from the local and global aspects.At the same time,our method improves the stability of training.Moreover,the method is embedded into a unified convolutional neural network that can be easily optimized by gradient descent.Experimental results show that our joint method can yield the state-of-the-art results on three common public datasets.展开更多
Smart manufacturing suffers from the heterogeneity of local data distribution across parties,mutual information silos and lack of privacy protection in the process of industry chain collaboration.To address these prob...Smart manufacturing suffers from the heterogeneity of local data distribution across parties,mutual information silos and lack of privacy protection in the process of industry chain collaboration.To address these problems,we propose a federated domain adaptation algorithm based on knowledge distillation and contrastive learning.Knowledge distillation is used to extract transferable integration knowledge from the different source domains and the quality of the extracted integration knowledge is used to assign reasonable weights to each source domain.A more rational weighted average aggregation is used in the aggregation phase of the center server to optimize the global model,while the local model of the source domain is trained with the help of contrastive learning to constrain the local model optimum towards the global model optimum,mitigating the inherent heterogeneity between local data.Our experiments are conducted on the largest domain adaptation dataset,and the results show that compared with other traditional federated domain adaptation algorithms,the algorithm we proposed trains a more accurate model,requires fewer communication rounds,makes more effective use of imbalanced data in the industrial area,and protects data privacy.展开更多
Device-free activity recognition plays a crucial role in smart building,security,and human–computer interaction,which shows its strength in its convenience and cost-efficiency.Traditional machine learning has made si...Device-free activity recognition plays a crucial role in smart building,security,and human–computer interaction,which shows its strength in its convenience and cost-efficiency.Traditional machine learning has made significant progress by heuristic hand-crafted features and statistical models,but it suffers from the limitation of manual feature design.Deep learning overcomes such issues by automatic high-level feature extraction,but its performance degrades due to the requirement of massive annotated data and cross-site issues.To deal with these problems,transfer learning helps to transfer knowledge from existing datasets while dealing with the negative effect of background dynamics.This paper surveys the recent progress of deep learning and transfer learning for device-free activity recognition.We begin with the motivation of deep learning and transfer learning,and then introduce the major sensor modalities.Then the deep and transfer learning techniques for device-free human activity recognition are introduced.Eventually,insights on existing works and grand challenges are summarized and presented to promote future research.展开更多
As failure data is usually scarce in practice upon preventive maintenance strategy in prognostics and health management(PHM)domain,transfer learning provides a fundamental solution to enhance generalization of datadri...As failure data is usually scarce in practice upon preventive maintenance strategy in prognostics and health management(PHM)domain,transfer learning provides a fundamental solution to enhance generalization of datadriven methods.In this paper,we briefly discuss general idea and advances of various transfer learning techniques in PHM domain,including domain adaptation,domain generalization,federated learning,and knowledge-driven transfer learning.Based on the observations from state of the art,we provide extensive discussions on possible challenges and opportunities of transfer learning in PHM domain to direct future development.展开更多
Medical imaging diagnosis of rare diseases faces the dual challenge of scarce labeled data and significant differences in equipment.This study proposes a hybrid framework integrating CT and MRI modalities,combining CN...Medical imaging diagnosis of rare diseases faces the dual challenge of scarce labeled data and significant differences in equipment.This study proposes a hybrid framework integrating CT and MRI modalities,combining CNN and Transformer architectures and introducing an adversarial domain adaptation mechanism.The dedicated CNN encoder extracts fine-grained local features,while the Transformer module captures long-range cross-modal dependencies.The gradient inversion domain discriminator aligns the feature distribution of different scanning devices to ensure the device independence of the model.On the two public datasets of neurological diseases,intracranial hemorrhage and demyelinating lesions,the average accuracy rate of this model reached 91.3%,and the F1 score was 0.89,which is 5 to 10 percentage points higher than the single-mode and pure Transformer baseline.Ablation experiments confirmed that the domain-based adversarial transformer component and training contributed to significant performance gains.In the cross-domain(CT to MRI)experiment,the domain adaptation technique increased the F1 score from 0.74 to 0.84.These results highlight the effectiveness of local feature extraction,global context modeling,and collaborative adversarial alignment schemes in multi-agency scenarios with sparse data.Future research will be extended to three-dimensional volume data,integrate semi-supervised learning of unlabeled images,and optimize the reasoning process of real-time clinical decision support.展开更多
In machinery fault diagnosis,labeled data are always difficult or even impossible to obtain.Transfer learning can leverage related fault diagnosis knowledge from fully labeled source domain to enhance the fault diagno...In machinery fault diagnosis,labeled data are always difficult or even impossible to obtain.Transfer learning can leverage related fault diagnosis knowledge from fully labeled source domain to enhance the fault diagnosis performance in sparsely labeled or unlabeled target domain,which has been widely used for cross domain fault diagnosis.However,existing methods focus on either marginal distribution adaptation(MDA)or conditional distribution adaptation(CDA).In practice,marginal and conditional distributions discrepancies both have significant but different influences on the domain divergence.In this paper,a dynamic distribution adaptation based transfer network(DDATN)is proposed for cross domain bearing fault diagnosis.DDATN utilizes the proposed instance-weighted dynamic maximum mean discrepancy(IDMMD)for dynamic distribution adaptation(DDA),which can dynamically estimate the influences of marginal and conditional distribution and adapt target domain with source domain.The experimental evaluation on cross domain bearing fault diagnosis demonstrates that DDATN can outperformance the state-of-the-art cross domain fault diagnosis methods.展开更多
Unsupervised transfer subspace learning is one of the challenging and important topics in domain adaptation,which aims to classify unlabeled target data by using source domain information.The traditional transfer subs...Unsupervised transfer subspace learning is one of the challenging and important topics in domain adaptation,which aims to classify unlabeled target data by using source domain information.The traditional transfer subspace learning methods often impose low-rank constraints,i.e.,trace norm,to preserve data structural information of different domains.However,trace norm is only the convex surrogate to approximate the ideal low-rank constraints and may make their solutions seriously deviate from the original optimums.In addition,the traditional methods directly use the strict labels of source domain,which is difficult to deal with label noise.To solve these problems,we propose a novel nonconvex and discriminative transfer subspace learning method named NDTSL by incorporating Schatten-norm and soft label matrix.Specifically,Schatten-norm can be imposed to approximate the low-rank constraints and obtain a better lowrank representation.Then,we design and adopt soft label matrix in source domain to learn a more flexible classifier and enhance the discriminative ability of target data.Besides,due to the nonconvexity of Schatten-norm,we design an efficient alternative algorithm IALM to solve it.Finally,experimental results on several public transfer tasks demonstrate the effectiveness of NDTSL compared with several state-of-the-art methods.展开更多
基金supported by the National Research Foundation of Korea(NRF)grant funded by the Korea government(MSIT)(No.RS-2024-00406320)the Institute of Information&Communica-tions Technology Planning&Evaluation(IITP)-Innovative Human Resource Development for Local Intellectualization Program Grant funded by the Korea government(MSIT)(IITP-2026-RS-2023-00259678).
文摘Domain adaptation aims to reduce the distribution gap between the training data(source domain)and the target data.This enables effective predictions even for domains not seen during training.However,most conventional domain adaptation methods assume a single source domain,making them less suitable for modern deep learning settings that rely on diverse and large-scale datasets.To address this limitation,recent research has focused on Multi-Source Domain Adaptation(MSDA),which aims to learn effectively from multiple source domains.In this paper,we propose Efficient Domain Transition for Multi-source(EDTM),a novel and efficient framework designed to tackle two major challenges in existing MSDA approaches:(1)integrating knowledge across different source domains and(2)aligning label distributions between source and target domains.EDTM leverages an ensemble-based classifier expert mechanism to enhance the contribution of source domains that are more similar to the target domain.To further stabilize the learning process and improve performance,we incorporate imitation learning into the training of the target model.In addition,Maximum Classifier Discrepancy(MCD)is employed to align class-wise label distributions between the source and target domains.Experiments were conducted using Digits-Five,one of the most representative benchmark datasets for MSDA.The results show that EDTM consistently outperforms existing methods in terms of average classification accuracy.Notably,EDTM achieved significantly higher performance on target domains such as Modified National Institute of Standards and Technolog with blended background images(MNIST-M)and Street View House Numbers(SVHN)datasets,demonstrating enhanced generalization compared to baseline approaches.Furthermore,an ablation study analyzing the contribution of each loss component validated the effectiveness of the framework,highlighting the importance of each module in achieving optimal performance.
基金funded by Advanced Research Project(30209040702).
文摘Named Entity Recognition(NER)is vital in natural language processing for the analysis of news texts,as it accurately identifies entities such as locations,persons,and organizations,which is crucial for applications like news summarization and event tracking.However,NER in the news domain faces challenges due to insufficient annotated data,complex entity structures,and strong context dependencies.To address these issues,we propose a new Chinesenamed entity recognition method that integrates transfer learning with word embeddings.Our approach leverages the ERNIE pre-trained model for transfer learning and obtaining general language representations and incorporates the Soft-lexicon word embedding technique to handle varied entity structures.This dual-strategy enhances the model’s understanding of context and boosts its ability to process complex texts.Experimental results show that our method achieves an F1 score of 94.72% on a news dataset,surpassing baseline methods by 3%–4%,thereby confirming its effectiveness for Chinese-named entity recognition in the news domain.
文摘Many applications,including security systems,medical diagnostics,and human-computer interfaces,depend on eye gaze recognition.However,due to factors including individual variations,occlusions,and shifting illumination conditions,real-world scenarios continue to provide difficulties for accurate and consistent eye gaze recognition.This work is aimed at investigating the potential benefits of employing transfer learning to improve eye gaze detection ability and efficiency.Transfer learning is the process of fine-tuning pre-trained models on smaller,domain-specific datasets after they have been trained on larger datasets.We study several transfer learning algorithms and evaluate their effectiveness on eye gaze identification,including both Regression and Classification tasks,using a range of deep learning architectures,namely AlexNet,Visual Geometry Group(VGG),InceptionV3,and ResNet.In this study,we evaluate the effectiveness of transfer learning-basedmodels against models that were trained fromscratch using eye-gazing datasets on grounds of various performance and loss metrics such as Precision,Accuracy,and Mean Absolute Error.We investigate the effects of different pre-trainedmodels,dataset sizes,and domain gaps on the transfer learning process,and the findings of our study clarify the efficacy of transfer learning for eye gaze detection and offer suggestions for the most successful transfer learning strategies to apply in real-world situations.
基金supported by the National Natural Science Foundation of China(NSFC)(U1704158)Henan Province Technologies Research and Development Project of China(212102210103)+1 种基金the NSFC Development Funding of Henan Normal University(2020PL09)the University of Manitoba Research Grants Program(URGP)。
文摘Despite the big success of transfer learning techniques in anomaly detection,it is still challenging to achieve good transition of detection rules merely based on the preferred data in the anomaly detection with one-class classification,especially for the data with a large distribution difference.To address this challenge,a novel deep one-class transfer learning algorithm with domain-adversarial training is proposed in this paper.First,by integrating a hypersphere adaptation constraint into domainadversarial neural network,a new hypersphere adversarial training mechanism is designed.Second,an alternative optimization method is derived to seek the optimal network parameters while pushing the hyperspheres built in the source domain and target domain to be as identical as possible.Through transferring oneclass detection rule in the adaptive extraction of domain-invariant feature representation,the end-to-end anomaly detection with one-class classification is then enhanced.Furthermore,a theoretical analysis about the model reliability,as well as the strategy of avoiding invalid and negative transfer,is provided.Experiments are conducted on two typical anomaly detection problems,i.e.,image recognition detection and online early fault detection of rolling bearings.The results demonstrate that the proposed algorithm outperforms the state-of-the-art methods in terms of detection accuracy and robustness.
基金supported in part by the Key-Area Research and Development Program of Guangdong Province (2020B010166006)the National Natural Science Foundation of China (61972102)+1 种基金the Guangzhou Science and Technology Plan Project (023A04J1729)the Science and Technology development fund (FDCT),Macao SAR (015/2020/AMJ)。
文摘Most existing domain adaptation(DA) methods aim to explore favorable performance under complicated environments by sampling.However,there are three unsolved problems that limit their efficiencies:ⅰ) they adopt global sampling but neglect to exploit global and local sampling simultaneously;ⅱ)they either transfer knowledge from a global perspective or a local perspective,while overlooking transmission of confident knowledge from both perspectives;and ⅲ) they apply repeated sampling during iteration,which takes a lot of time.To address these problems,knowledge transfer learning via dual density sampling(KTL-DDS) is proposed in this study,which consists of three parts:ⅰ) Dual density sampling(DDS) that jointly leverages two sampling methods associated with different views,i.e.,global density sampling that extracts representative samples with the most common features and local density sampling that selects representative samples with critical boundary information;ⅱ)Consistent maximum mean discrepancy(CMMD) that reduces intra-and cross-domain risks and guarantees high consistency of knowledge by shortening the distances of every two subsets among the four subsets collected by DDS;and ⅲ) Knowledge dissemination(KD) that transmits confident and consistent knowledge from the representative target samples with global and local properties to the whole target domain by preserving the neighboring relationships of the target domain.Mathematical analyses show that DDS avoids repeated sampling during the iteration.With the above three actions,confident knowledge with both global and local properties is transferred,and the memory and running time are greatly reduced.In addition,a general framework named dual density sampling approximation(DDSA) is extended,which can be easily applied to other DA algorithms.Extensive experiments on five datasets in clean,label corruption(LC),feature missing(FM),and LC&FM environments demonstrate the encouraging performance of KTL-DDS.
基金supported by the National Natural Science Foundation of China(Grant No.51674169)Department of Education of Hebei Province of China(Grant No.ZD2019140)+1 种基金Natural Science Foundation of Hebei Province of China(Grant No.F2019210243)S&T Program of Hebei(Grant No.22375413D)School of Electrical and Electronics Engineering。
文摘Accurate displacement prediction is critical for the early warning of landslides.The complexity of the coupling relationship between multiple influencing factors and displacement makes the accurate prediction of displacement difficult.Moreover,in engineering practice,insufficient monitoring data limit the performance of prediction models.To alleviate this problem,a displacement prediction method based on multisource domain transfer learning,which helps accurately predict data in the target domain through the knowledge of one or more source domains,is proposed.First,an optimized variational mode decomposition model based on the minimum sample entropy is used to decompose the cumulative displacement into the trend,periodic,and stochastic components.The trend component is predicted by an autoregressive model,and the periodic component is predicted by the long short-term memory.For the stochastic component,because it is affected by uncertainties,it is predicted by a combination of a Wasserstein generative adversarial network and multisource domain transfer learning for improved prediction accuracy.Considering a real mine slope as a case study,the proposed prediction method was validated.Therefore,this study provides new insights that can be applied to scenarios lacking sample data.
文摘Transfer learning aims to transfer source models to a target domain.Leveraging the feature matching can alleviate the domain shift effectively,but this process ignores the relationship of the marginal distribution matching and the conditional distribution matching.Simultaneously,the discriminative information of both domains is also neglected,which is important for improving the performance on the target domain.In this paper,we propose a novel method called Balanced Discriminative Transfer Feature Learning for Visual Domain Adaptation(BDTFL).The proposed method can adaptively balance the relationship of both distribution matchings and capture the category discriminative information of both domains.Therefore,balanced feature matching can achieve more accurate feature matching and adaptively adjust itself to different scenes.At the same time,discriminative information is exploited to alleviate category confusion during feature matching.And with assistance of the category discriminative information captured from both domains,the source classifier can be transferred to the target domain more accurately and boost the performance of target classification.Extensive experiments show the superiority of BDTFL on popular visual cross-domain benchmarks.
文摘The non-stationary of the motor imagery electroencephalography(MI-EEG)signal is one of the main limitations for the development of motor imagery brain-computer interfaces(MI-BCI).The nonstationary of the MI-EEG signal and the changes of the experimental environment make the feature distribution of the testing set and training set deviates,which reduces the classification accuracy of MI-BCI.In this paper,we propose a Kullback–Leibler divergence(KL)-based transfer learning algorithm to solve the problem of feature transfer,the proposed algorithm uses KL to measure the similarity between the training set and the testing set,adds support vector machine(SVM)classification probability to classify and weight the covariance,and discards the poorly performing samples.The results show that the proposed algorithm can significantly improve the classification accuracy of the testing set compared with the traditional algorithms,especially for subjects with medium classification accuracy.Moreover,the algorithm based on transfer learning has the potential to improve the consistency of feature distribution that the traditional algorithms do not have,which is significant for the application of MI-BCI.
文摘Recent works have shown that neural networks are promising parameter-free limiters for a variety of numerical schemes(Morgan et al.in A machine learning approach for detect-ing shocks with high-order hydrodynamic methods.et al.in J Comput Phys 367:166-191.,2018;Veiga et al.in European Conference on Computational Mechanics andⅦEuropean Conference on Computational Fluid Dynamics,vol.1,pp.2525-2550.ECCM.,2018).Following this trend,we train a neural network to serve as a shock-indicator function using simulation data from a Runge-Kutta discontinuous Galer-kin(RKDG)method and a modal high-order limiter(Krivodonova in J Comput Phys 226:879-896.,2007).With this methodology,we obtain one-and two-dimensional black-box shock-indicators which are then coupled to a standard limiter.Furthermore,we describe a strategy to transfer the shock-indicator to a residual distribution(RD)scheme without the need for a full training cycle and large data-set,by finding a mapping between the solution feature spaces from an RD scheme to an RKDG scheme,both in one-and two-dimensional problems,and on Cartesian and unstruc-tured meshes.We report on the quality of the numerical solutions when using the neural network shock-indicator coupled to a limiter,comparing its performance to traditional lim-iters,for both RKDG and RD schemes.
基金the National Key Research and Development Program of China(No.2020YFB1713500)the Natural Science Basic Research Program of Shaanxi(Grant No.2023JCYB289)+1 种基金the National Natural Science Foundation of China(Grant No.52175112)the Fundamental Research Funds for the Central Universities(Grant No.ZYTS23102).
文摘The wear of metal cutting tools will progressively rise as the cutting time goes on. Wearing heavily on the toolwill generate significant noise and vibration, negatively impacting the accuracy of the forming and the surfaceintegrity of the workpiece. Hence, during the cutting process, it is imperative to continually monitor the tool wearstate andpromptly replace anyheavilyworn tools toguarantee thequality of the cutting.The conventional tool wearmonitoring models, which are based on machine learning, are specifically built for the intended cutting conditions.However, these models require retraining when the cutting conditions undergo any changes. This method has noapplication value if the cutting conditions frequently change. This manuscript proposes a method for monitoringtool wear basedonunsuperviseddeep transfer learning. Due to the similarity of the tool wear process under varyingworking conditions, a tool wear recognitionmodel that can adapt to both current and previous working conditionshas been developed by utilizing cutting monitoring data from history. To extract and classify cutting vibrationsignals, the unsupervised deep transfer learning network comprises a one-dimensional (1D) convolutional neuralnetwork (CNN) with a multi-layer perceptron (MLP). To achieve distribution alignment of deep features throughthe maximum mean discrepancy algorithm, a domain adaptive layer is embedded in the penultimate layer of thenetwork. A platformformonitoring tool wear during endmilling has been constructed. The proposedmethod wasverified through the execution of a full life test of end milling under multiple working conditions with a Cr12MoVsteel workpiece. Our experiments demonstrate that the transfer learning model maintains a classification accuracyof over 80%. In comparisonwith the most advanced tool wearmonitoring methods, the presentedmodel guaranteessuperior performance in the target domains.
基金Project(2301DH09002)supported by the Bureau of Planning and Natural Resources,Chongqing,ChinaProject(2022T3051)supported by the Science and Technology Service Network Initiative,ChinaProject(2018-ZL-01)supported by the Sichuan Transportation Science and Technology,China。
文摘Machine-learning methodologies have increasingly been embraced in landslide susceptibility assessment.However,the considerable time and financial burdens of landslide inventories often result in persistent data scarcity,which frequently impedes the generation of accurate and informative landslide susceptibility maps.Addressing this challenge,this study compiled a nationwide dataset and developed a transfer learning-based model to evaluate landslide susceptibility in the Chongqing region specifically.Notably,the proposed model,calibrated with the warmup-cosine annealing(WCA)learning rate strategy,demonstrated remarkable predictive capabilities,particularly in scenarios marked by data limitations and when training data were normalized using parameters from the source region.This is evidenced by the area under the receiver operating characteristic curve(AUC)values,which exhibited significant improvements of 51.00%,24.40%and 2.15%,respectively,compared to a deep learning model,in contexts where only 1%,5%and 10%of data from the target region were used for retraining.Simultaneously,there were reductions in loss of 16.12%,27.61%and 15.44%,respectively,in these instances.
基金the Aerospace Science and Technology Foundation(No.20115557007)the National Natural Science Foundation of China(No.61673262)the Military Science and Technology Foundation of China(No.18-H863-03-ZT-001-006-06)
文摘Domain adaptation and adversarial networks are two main approaches for transfer learning.Domain adaptation methods match the mean values of source and target domains,which requires a very large batch size during training.However,adversarial networks are usually unstable when training.In this paper,we propose a joint method of feature matching and adversarial networks to reduce domain discrepancy and mine domaininvariant features from the local and global aspects.At the same time,our method improves the stability of training.Moreover,the method is embedded into a unified convolutional neural network that can be easily optimized by gradient descent.Experimental results show that our joint method can yield the state-of-the-art results on three common public datasets.
基金Supported by the Scientific and Technological Innovation 2030—Major Project of"New Generation Artificial Intelligence"(2020AAA0109300)。
文摘Smart manufacturing suffers from the heterogeneity of local data distribution across parties,mutual information silos and lack of privacy protection in the process of industry chain collaboration.To address these problems,we propose a federated domain adaptation algorithm based on knowledge distillation and contrastive learning.Knowledge distillation is used to extract transferable integration knowledge from the different source domains and the quality of the extracted integration knowledge is used to assign reasonable weights to each source domain.A more rational weighted average aggregation is used in the aggregation phase of the center server to optimize the global model,while the local model of the source domain is trained with the help of contrastive learning to constrain the local model optimum towards the global model optimum,mitigating the inherent heterogeneity between local data.Our experiments are conducted on the largest domain adaptation dataset,and the results show that compared with other traditional federated domain adaptation algorithms,the algorithm we proposed trains a more accurate model,requires fewer communication rounds,makes more effective use of imbalanced data in the industrial area,and protects data privacy.
基金This work is supported by NTU Presidential Postdoctoral Fellowship,"Adaptive Multimodal Learning for Robust Sensing and Recognition in Smart Cities"project fund,in Nanyang Technological University,Singapore.
文摘Device-free activity recognition plays a crucial role in smart building,security,and human–computer interaction,which shows its strength in its convenience and cost-efficiency.Traditional machine learning has made significant progress by heuristic hand-crafted features and statistical models,but it suffers from the limitation of manual feature design.Deep learning overcomes such issues by automatic high-level feature extraction,but its performance degrades due to the requirement of massive annotated data and cross-site issues.To deal with these problems,transfer learning helps to transfer knowledge from existing datasets while dealing with the negative effect of background dynamics.This paper surveys the recent progress of deep learning and transfer learning for device-free activity recognition.We begin with the motivation of deep learning and transfer learning,and then introduce the major sensor modalities.Then the deep and transfer learning techniques for device-free human activity recognition are introduced.Eventually,insights on existing works and grand challenges are summarized and presented to promote future research.
文摘As failure data is usually scarce in practice upon preventive maintenance strategy in prognostics and health management(PHM)domain,transfer learning provides a fundamental solution to enhance generalization of datadriven methods.In this paper,we briefly discuss general idea and advances of various transfer learning techniques in PHM domain,including domain adaptation,domain generalization,federated learning,and knowledge-driven transfer learning.Based on the observations from state of the art,we provide extensive discussions on possible challenges and opportunities of transfer learning in PHM domain to direct future development.
文摘Medical imaging diagnosis of rare diseases faces the dual challenge of scarce labeled data and significant differences in equipment.This study proposes a hybrid framework integrating CT and MRI modalities,combining CNN and Transformer architectures and introducing an adversarial domain adaptation mechanism.The dedicated CNN encoder extracts fine-grained local features,while the Transformer module captures long-range cross-modal dependencies.The gradient inversion domain discriminator aligns the feature distribution of different scanning devices to ensure the device independence of the model.On the two public datasets of neurological diseases,intracranial hemorrhage and demyelinating lesions,the average accuracy rate of this model reached 91.3%,and the F1 score was 0.89,which is 5 to 10 percentage points higher than the single-mode and pure Transformer baseline.Ablation experiments confirmed that the domain-based adversarial transformer component and training contributed to significant performance gains.In the cross-domain(CT to MRI)experiment,the domain adaptation technique increased the F1 score from 0.74 to 0.84.These results highlight the effectiveness of local feature extraction,global context modeling,and collaborative adversarial alignment schemes in multi-agency scenarios with sparse data.Future research will be extended to three-dimensional volume data,integrate semi-supervised learning of unlabeled images,and optimize the reasoning process of real-time clinical decision support.
基金Supported by National Natural Science Foundation of China(Grant Nos.51875208,51475170)National Key Research and Development Program of China(Grant No.2018YFB1702400).
文摘In machinery fault diagnosis,labeled data are always difficult or even impossible to obtain.Transfer learning can leverage related fault diagnosis knowledge from fully labeled source domain to enhance the fault diagnosis performance in sparsely labeled or unlabeled target domain,which has been widely used for cross domain fault diagnosis.However,existing methods focus on either marginal distribution adaptation(MDA)or conditional distribution adaptation(CDA).In practice,marginal and conditional distributions discrepancies both have significant but different influences on the domain divergence.In this paper,a dynamic distribution adaptation based transfer network(DDATN)is proposed for cross domain bearing fault diagnosis.DDATN utilizes the proposed instance-weighted dynamic maximum mean discrepancy(IDMMD)for dynamic distribution adaptation(DDA),which can dynamically estimate the influences of marginal and conditional distribution and adapt target domain with source domain.The experimental evaluation on cross domain bearing fault diagnosis demonstrates that DDATN can outperformance the state-of-the-art cross domain fault diagnosis methods.
基金supported by the National Natural Science Foundation of China(Grant No.61922087)the Huxiang Young Talents Program of Hunan Province(2021RC3070).
文摘Unsupervised transfer subspace learning is one of the challenging and important topics in domain adaptation,which aims to classify unlabeled target data by using source domain information.The traditional transfer subspace learning methods often impose low-rank constraints,i.e.,trace norm,to preserve data structural information of different domains.However,trace norm is only the convex surrogate to approximate the ideal low-rank constraints and may make their solutions seriously deviate from the original optimums.In addition,the traditional methods directly use the strict labels of source domain,which is difficult to deal with label noise.To solve these problems,we propose a novel nonconvex and discriminative transfer subspace learning method named NDTSL by incorporating Schatten-norm and soft label matrix.Specifically,Schatten-norm can be imposed to approximate the low-rank constraints and obtain a better lowrank representation.Then,we design and adopt soft label matrix in source domain to learn a more flexible classifier and enhance the discriminative ability of target data.Besides,due to the nonconvexity of Schatten-norm,we design an efficient alternative algorithm IALM to solve it.Finally,experimental results on several public transfer tasks demonstrate the effectiveness of NDTSL compared with several state-of-the-art methods.
