Label distribution learning(LDL)is a new learning paradigm to deal with label ambiguity and many researches have achieved the prominent performances.Compared with traditional supervised learning scenarios,the annotati...Label distribution learning(LDL)is a new learning paradigm to deal with label ambiguity and many researches have achieved the prominent performances.Compared with traditional supervised learning scenarios,the annotation with label distribution is more expensive.Direct use of existing active learning(AL)approaches,which aim to reduce the annotation cost in traditional learning,may lead to the degradation of their performance.To deal with the problem of high annotation cost in LDL,we propose the active label distribution learning via kernel maximum mean discrepancy(ALDL-kMMD)method to tackle this crucial but rarely studied problem.ALDL-kMMD captures the structural information of both data and label,extracts the most representative instances from the unlabeled ones by incorporating the nonlinear model and marginal probability distribution matching.Besides,it is also able to markedly decrease the amount of queried unlabeled instances.Meanwhile,an effective solution is proposed for the original optimization problem of ALDL-kMMD by constructing auxiliary variables.The effectiveness of our method is validated with experiments on the real-world datasets.展开更多
The wear of metal cutting tools will progressively rise as the cutting time goes on. Wearing heavily on the toolwill generate significant noise and vibration, negatively impacting the accuracy of the forming and the s...The wear of metal cutting tools will progressively rise as the cutting time goes on. Wearing heavily on the toolwill generate significant noise and vibration, negatively impacting the accuracy of the forming and the surfaceintegrity of the workpiece. Hence, during the cutting process, it is imperative to continually monitor the tool wearstate andpromptly replace anyheavilyworn tools toguarantee thequality of the cutting.The conventional tool wearmonitoring models, which are based on machine learning, are specifically built for the intended cutting conditions.However, these models require retraining when the cutting conditions undergo any changes. This method has noapplication value if the cutting conditions frequently change. This manuscript proposes a method for monitoringtool wear basedonunsuperviseddeep transfer learning. Due to the similarity of the tool wear process under varyingworking conditions, a tool wear recognitionmodel that can adapt to both current and previous working conditionshas been developed by utilizing cutting monitoring data from history. To extract and classify cutting vibrationsignals, the unsupervised deep transfer learning network comprises a one-dimensional (1D) convolutional neuralnetwork (CNN) with a multi-layer perceptron (MLP). To achieve distribution alignment of deep features throughthe maximum mean discrepancy algorithm, a domain adaptive layer is embedded in the penultimate layer of thenetwork. A platformformonitoring tool wear during endmilling has been constructed. The proposedmethod wasverified through the execution of a full life test of end milling under multiple working conditions with a Cr12MoVsteel workpiece. Our experiments demonstrate that the transfer learning model maintains a classification accuracyof over 80%. In comparisonwith the most advanced tool wearmonitoring methods, the presentedmodel guaranteessuperior performance in the target domains.展开更多
In recent years,a number of intelligent algorithm have been proposed for forecasting the lithium-ion battery state of health(SOH).Due to the varying specifications and operating conditions of batteries,it is difficult...In recent years,a number of intelligent algorithm have been proposed for forecasting the lithium-ion battery state of health(SOH).Due to the varying specifications and operating conditions of batteries,it is difficult to anticipate the health condition of lithium battery as it begins to deteriorate.There are still few studies on health state prediction models for different types of batteries.In this paper,40 battery data from 5 public datasets are selected to carry out research,and a model architecture consisting of Denoising Autoencoder and Transformer is designed.One or two types of battery packs are identified as the source domain,and multiple types of battery packs are identified as the target domain.By employing Maximum Mean Discrepancy(MMD)on the Transformer architecture,the source and target domains were evaluated and found to converge as training continued.Finally,29 transfer learning combination tasks were constructed.Results show that the model built with two kinds of batteries as the target domain has the best prediction accuracy and excels in prediction and is versatile in its application.The experimental results also reveal that this study provides a promising tool for predicting Lithium-ion batteries’SOH and strives to build a generalized model of the Lithium-ion batteries’SOH indicators.展开更多
In Unsupervised Domain Adaptation(UDA)for person re-identification(re-ID),the primary challenge is reducing the distribution discrepancy between the source and target domains.This can be achieved by implicitly or expl...In Unsupervised Domain Adaptation(UDA)for person re-identification(re-ID),the primary challenge is reducing the distribution discrepancy between the source and target domains.This can be achieved by implicitly or explicitly constructing an appropriate intermediate domain to enhance recognition capability on the target domain.Implicit construction is difficult due to the absence of intermediate state supervision,making smooth knowledge transfer from the source to the target domain a challenge.To explicitly construct the most suitable intermediate domain for the model to gradually adapt to the feature distribution changes from the source to the target domain,we propose the Minimal Transfer Cost Framework(MTCF).MTCF considers all scenarios of the intermediate domain during the transfer process,ensuring smoother and more efficient domain alignment.Our framework mainly includes threemodules:Intermediate Domain Generator(IDG),Cross-domain Feature Constraint Module(CFCM),and Residual Channel Space Module(RCSM).First,the IDG Module is introduced to generate all possible intermediate domains,ensuring a smooth transition of knowledge fromthe source to the target domain.To reduce the cross-domain feature distribution discrepancy,we propose the CFCM Module,which quantifies the difficulty of knowledge transfer and ensures the diversity of intermediate domain features and their semantic relevance,achieving alignment between the source and target domains by incorporating mutual information and maximum mean discrepancy.We also design the RCSM,which utilizes attention mechanism to enhance the model’s focus on personnel features in low-resolution images,improving the accuracy and efficiency of person re-ID.Our proposed method outperforms existing technologies in all common UDA re-ID tasks and improves the Mean Average Precision(mAP)by 2.3%in the Market to Duke task compared to the state-of-the-art(SOTA)methods.展开更多
Most existing domain adaptation(DA) methods aim to explore favorable performance under complicated environments by sampling.However,there are three unsolved problems that limit their efficiencies:ⅰ) they adopt global...Most existing domain adaptation(DA) methods aim to explore favorable performance under complicated environments by sampling.However,there are three unsolved problems that limit their efficiencies:ⅰ) they adopt global sampling but neglect to exploit global and local sampling simultaneously;ⅱ)they either transfer knowledge from a global perspective or a local perspective,while overlooking transmission of confident knowledge from both perspectives;and ⅲ) they apply repeated sampling during iteration,which takes a lot of time.To address these problems,knowledge transfer learning via dual density sampling(KTL-DDS) is proposed in this study,which consists of three parts:ⅰ) Dual density sampling(DDS) that jointly leverages two sampling methods associated with different views,i.e.,global density sampling that extracts representative samples with the most common features and local density sampling that selects representative samples with critical boundary information;ⅱ)Consistent maximum mean discrepancy(CMMD) that reduces intra-and cross-domain risks and guarantees high consistency of knowledge by shortening the distances of every two subsets among the four subsets collected by DDS;and ⅲ) Knowledge dissemination(KD) that transmits confident and consistent knowledge from the representative target samples with global and local properties to the whole target domain by preserving the neighboring relationships of the target domain.Mathematical analyses show that DDS avoids repeated sampling during the iteration.With the above three actions,confident knowledge with both global and local properties is transferred,and the memory and running time are greatly reduced.In addition,a general framework named dual density sampling approximation(DDSA) is extended,which can be easily applied to other DA algorithms.Extensive experiments on five datasets in clean,label corruption(LC),feature missing(FM),and LC&FM environments demonstrate the encouraging performance of KTL-DDS.展开更多
This paper proposes a fingerprint matching method integrating transfer learning and online learning to tackle the challenges of environmental adaptability and dynamic interference resistance in photovoltaic(PV)array D...This paper proposes a fingerprint matching method integrating transfer learning and online learning to tackle the challenges of environmental adaptability and dynamic interference resistance in photovoltaic(PV)array DC arc fault location methods based on electromagnetic radiation(EMR)signals.Initially,a comprehensive analysis of the time–frequency characteristics of series arc EMR signals is carried out to pinpoint effective data sources that reflect fault features.Subsequently,a multi-kernel domain-adversarial neural network(MKDANN)is introduced to extract domain-invariant features,and a feature extractor designed specifically for fingerprint matching is devised.To reduce inter-domain distribution differences,a multi-kernel maximum mean discrepancy(MK-MMD)is integrated into the adaptation layer.Moreover,to deal with dynamic environmental changes in real-world situations,the support-class passive aggressive(SPA)algorithm is utilized to adjust model parameters in real time.Finally,MKDANN and SPA technologies are smoothly combined to build a fully operational fault location model.Experimental results indicate that the proposed method attains an overall fault location accuracy of at least 95%,showing strong adaptability to environmental changes and robust interference resistance while maintaining excellent online learning capabilities during model migration.展开更多
The state of health(SOH)and remaining useful life(RUL)of lithium-ion batteries are crucial for health management and diagnosis.However,most data-driven estimation methods heavily rely on scarce labeled data,while trad...The state of health(SOH)and remaining useful life(RUL)of lithium-ion batteries are crucial for health management and diagnosis.However,most data-driven estimation methods heavily rely on scarce labeled data,while traditional transfer learning faces challenges in handling domain shifts across various battery types.This paper proposes an enhanced vision-transformer integrating with semi-supervised transfer learning for SOH and RUL estimation of lithium-ion batteries.A depth-wise separable convolutional vision-transformer is developed to extract local aging details with depth-wise convolutions and establishes global dependencies between aging information using multi-head attention.Maximum mean discrepancy is employed to initially reduce the distribution difference between the source and target domains,providing a superior starting point for fine-tuning the target domain model.Subsequently,the abundant aging data of the same type as the target battery are labeled through semi-supervised learning,compensating for the source model's limitations in capturing target battery aging characteristics.Consistency regularization incorporates the cross-entropy between predictions with and without adversarial perturbations into the gradient backpropagation of the overall model.In particular,across the experimental groups 13–15 for different types of batteries,the root mean square error of SOH estimation was less than 0.66%,and the mean relative error of RUL estimation was 3.86%.Leveraging extensive unlabeled aging data,the proposed method could achieve accurate estimation of SOH and RUL.展开更多
Deep neural networks have been successfully applied to numerous machine learning tasks because of their impressive feature abstraction capabilities.However,conventional deep networks assume that the training and test ...Deep neural networks have been successfully applied to numerous machine learning tasks because of their impressive feature abstraction capabilities.However,conventional deep networks assume that the training and test data are sampled from the same distribution,and this assumption is often violated in real-world scenarios.To address the domain shift or data bias problems,we introduce layer-wise domain correction(LDC),a new unsupervised domain adaptation algorithm which adapts an existing deep network through additive correction layers spaced throughout the network.Through the additive layers,the representations of source and target domains can be perfectly aligned.The corrections that are trained via maximum mean discrepancy,adapt to the target domain while increasing the representational capacity of the network.LDC requires no target labels,achieves state-of-the-art performance across several adaptation benchmarks,and requires significantly less training time than existing adaptation methods.展开更多
Generative adversarial networks(GANs)have shown impressive power in the field of machine learning.Traditional GANs have focused on unsupervised learning tasks.In recent years,conditional GANs that can generate data wi...Generative adversarial networks(GANs)have shown impressive power in the field of machine learning.Traditional GANs have focused on unsupervised learning tasks.In recent years,conditional GANs that can generate data with labels have been proposed in semi-supervised learning and have achieved better image quality than traditional GANs.Conditional GANs,however,generally only minimize the difference between marginal distributions of real and generated data,neglecting the difference with respect to each class of the data.To address this challenge,we propose the GAN with joint distribution moment matching(JDMM-GAN)for matching the joint distribution based on maximum mean discrepancy,which minimizes the differences of both the marginal and conditional distributions.The learning procedure is iteratively conducted by the stochastic gradient descent and back-propagation.We evaluate JDMM-GAN on several benchmark datasets,including MNIST,CIFAR-10 and the Extended Yale Face.Compared with the state-of-the-art GANs,JDMM-GAN generates more realistic images and achieves the best inception score for CIFAR-10 dataset.展开更多
基金partially supported by the National Natural Science Fundation of China(Grant Nos.61922087,61906201 and 62006238)the Science and Technology Innovation Program of Hunan Province(2021RC3070).
文摘Label distribution learning(LDL)is a new learning paradigm to deal with label ambiguity and many researches have achieved the prominent performances.Compared with traditional supervised learning scenarios,the annotation with label distribution is more expensive.Direct use of existing active learning(AL)approaches,which aim to reduce the annotation cost in traditional learning,may lead to the degradation of their performance.To deal with the problem of high annotation cost in LDL,we propose the active label distribution learning via kernel maximum mean discrepancy(ALDL-kMMD)method to tackle this crucial but rarely studied problem.ALDL-kMMD captures the structural information of both data and label,extracts the most representative instances from the unlabeled ones by incorporating the nonlinear model and marginal probability distribution matching.Besides,it is also able to markedly decrease the amount of queried unlabeled instances.Meanwhile,an effective solution is proposed for the original optimization problem of ALDL-kMMD by constructing auxiliary variables.The effectiveness of our method is validated with experiments on the real-world datasets.
基金the National Key Research and Development Program of China(No.2020YFB1713500)the Natural Science Basic Research Program of Shaanxi(Grant No.2023JCYB289)+1 种基金the National Natural Science Foundation of China(Grant No.52175112)the Fundamental Research Funds for the Central Universities(Grant No.ZYTS23102).
文摘The wear of metal cutting tools will progressively rise as the cutting time goes on. Wearing heavily on the toolwill generate significant noise and vibration, negatively impacting the accuracy of the forming and the surfaceintegrity of the workpiece. Hence, during the cutting process, it is imperative to continually monitor the tool wearstate andpromptly replace anyheavilyworn tools toguarantee thequality of the cutting.The conventional tool wearmonitoring models, which are based on machine learning, are specifically built for the intended cutting conditions.However, these models require retraining when the cutting conditions undergo any changes. This method has noapplication value if the cutting conditions frequently change. This manuscript proposes a method for monitoringtool wear basedonunsuperviseddeep transfer learning. Due to the similarity of the tool wear process under varyingworking conditions, a tool wear recognitionmodel that can adapt to both current and previous working conditionshas been developed by utilizing cutting monitoring data from history. To extract and classify cutting vibrationsignals, the unsupervised deep transfer learning network comprises a one-dimensional (1D) convolutional neuralnetwork (CNN) with a multi-layer perceptron (MLP). To achieve distribution alignment of deep features throughthe maximum mean discrepancy algorithm, a domain adaptive layer is embedded in the penultimate layer of thenetwork. A platformformonitoring tool wear during endmilling has been constructed. The proposedmethod wasverified through the execution of a full life test of end milling under multiple working conditions with a Cr12MoVsteel workpiece. Our experiments demonstrate that the transfer learning model maintains a classification accuracyof over 80%. In comparisonwith the most advanced tool wearmonitoring methods, the presentedmodel guaranteessuperior performance in the target domains.
文摘In recent years,a number of intelligent algorithm have been proposed for forecasting the lithium-ion battery state of health(SOH).Due to the varying specifications and operating conditions of batteries,it is difficult to anticipate the health condition of lithium battery as it begins to deteriorate.There are still few studies on health state prediction models for different types of batteries.In this paper,40 battery data from 5 public datasets are selected to carry out research,and a model architecture consisting of Denoising Autoencoder and Transformer is designed.One or two types of battery packs are identified as the source domain,and multiple types of battery packs are identified as the target domain.By employing Maximum Mean Discrepancy(MMD)on the Transformer architecture,the source and target domains were evaluated and found to converge as training continued.Finally,29 transfer learning combination tasks were constructed.Results show that the model built with two kinds of batteries as the target domain has the best prediction accuracy and excels in prediction and is versatile in its application.The experimental results also reveal that this study provides a promising tool for predicting Lithium-ion batteries’SOH and strives to build a generalized model of the Lithium-ion batteries’SOH indicators.
文摘In Unsupervised Domain Adaptation(UDA)for person re-identification(re-ID),the primary challenge is reducing the distribution discrepancy between the source and target domains.This can be achieved by implicitly or explicitly constructing an appropriate intermediate domain to enhance recognition capability on the target domain.Implicit construction is difficult due to the absence of intermediate state supervision,making smooth knowledge transfer from the source to the target domain a challenge.To explicitly construct the most suitable intermediate domain for the model to gradually adapt to the feature distribution changes from the source to the target domain,we propose the Minimal Transfer Cost Framework(MTCF).MTCF considers all scenarios of the intermediate domain during the transfer process,ensuring smoother and more efficient domain alignment.Our framework mainly includes threemodules:Intermediate Domain Generator(IDG),Cross-domain Feature Constraint Module(CFCM),and Residual Channel Space Module(RCSM).First,the IDG Module is introduced to generate all possible intermediate domains,ensuring a smooth transition of knowledge fromthe source to the target domain.To reduce the cross-domain feature distribution discrepancy,we propose the CFCM Module,which quantifies the difficulty of knowledge transfer and ensures the diversity of intermediate domain features and their semantic relevance,achieving alignment between the source and target domains by incorporating mutual information and maximum mean discrepancy.We also design the RCSM,which utilizes attention mechanism to enhance the model’s focus on personnel features in low-resolution images,improving the accuracy and efficiency of person re-ID.Our proposed method outperforms existing technologies in all common UDA re-ID tasks and improves the Mean Average Precision(mAP)by 2.3%in the Market to Duke task compared to the state-of-the-art(SOTA)methods.
基金supported in part by the Key-Area Research and Development Program of Guangdong Province (2020B010166006)the National Natural Science Foundation of China (61972102)+1 种基金the Guangzhou Science and Technology Plan Project (023A04J1729)the Science and Technology development fund (FDCT),Macao SAR (015/2020/AMJ)。
文摘Most existing domain adaptation(DA) methods aim to explore favorable performance under complicated environments by sampling.However,there are three unsolved problems that limit their efficiencies:ⅰ) they adopt global sampling but neglect to exploit global and local sampling simultaneously;ⅱ)they either transfer knowledge from a global perspective or a local perspective,while overlooking transmission of confident knowledge from both perspectives;and ⅲ) they apply repeated sampling during iteration,which takes a lot of time.To address these problems,knowledge transfer learning via dual density sampling(KTL-DDS) is proposed in this study,which consists of three parts:ⅰ) Dual density sampling(DDS) that jointly leverages two sampling methods associated with different views,i.e.,global density sampling that extracts representative samples with the most common features and local density sampling that selects representative samples with critical boundary information;ⅱ)Consistent maximum mean discrepancy(CMMD) that reduces intra-and cross-domain risks and guarantees high consistency of knowledge by shortening the distances of every two subsets among the four subsets collected by DDS;and ⅲ) Knowledge dissemination(KD) that transmits confident and consistent knowledge from the representative target samples with global and local properties to the whole target domain by preserving the neighboring relationships of the target domain.Mathematical analyses show that DDS avoids repeated sampling during the iteration.With the above three actions,confident knowledge with both global and local properties is transferred,and the memory and running time are greatly reduced.In addition,a general framework named dual density sampling approximation(DDSA) is extended,which can be easily applied to other DA algorithms.Extensive experiments on five datasets in clean,label corruption(LC),feature missing(FM),and LC&FM environments demonstrate the encouraging performance of KTL-DDS.
基金financially supported in part by the Natural Science Foundation of Fujian,China,under Grant 2021J01633.
文摘This paper proposes a fingerprint matching method integrating transfer learning and online learning to tackle the challenges of environmental adaptability and dynamic interference resistance in photovoltaic(PV)array DC arc fault location methods based on electromagnetic radiation(EMR)signals.Initially,a comprehensive analysis of the time–frequency characteristics of series arc EMR signals is carried out to pinpoint effective data sources that reflect fault features.Subsequently,a multi-kernel domain-adversarial neural network(MKDANN)is introduced to extract domain-invariant features,and a feature extractor designed specifically for fingerprint matching is devised.To reduce inter-domain distribution differences,a multi-kernel maximum mean discrepancy(MK-MMD)is integrated into the adaptation layer.Moreover,to deal with dynamic environmental changes in real-world situations,the support-class passive aggressive(SPA)algorithm is utilized to adjust model parameters in real time.Finally,MKDANN and SPA technologies are smoothly combined to build a fully operational fault location model.Experimental results indicate that the proposed method attains an overall fault location accuracy of at least 95%,showing strong adaptability to environmental changes and robust interference resistance while maintaining excellent online learning capabilities during model migration.
基金supported by the Science and Technology Major Project of Fujian Province of China(Grant No.2022HZ028018)the National Natural Science Foundation of China(Grant No.51907030).
文摘The state of health(SOH)and remaining useful life(RUL)of lithium-ion batteries are crucial for health management and diagnosis.However,most data-driven estimation methods heavily rely on scarce labeled data,while traditional transfer learning faces challenges in handling domain shifts across various battery types.This paper proposes an enhanced vision-transformer integrating with semi-supervised transfer learning for SOH and RUL estimation of lithium-ion batteries.A depth-wise separable convolutional vision-transformer is developed to extract local aging details with depth-wise convolutions and establishes global dependencies between aging information using multi-head attention.Maximum mean discrepancy is employed to initially reduce the distribution difference between the source and target domains,providing a superior starting point for fine-tuning the target domain model.Subsequently,the abundant aging data of the same type as the target battery are labeled through semi-supervised learning,compensating for the source model's limitations in capturing target battery aging characteristics.Consistency regularization incorporates the cross-entropy between predictions with and without adversarial perturbations into the gradient backpropagation of the overall model.In particular,across the experimental groups 13–15 for different types of batteries,the root mean square error of SOH estimation was less than 0.66%,and the mean relative error of RUL estimation was 3.86%.Leveraging extensive unlabeled aging data,the proposed method could achieve accurate estimation of SOH and RUL.
基金supported by the National Key R&D Program of China(No.2016YFB1200203)the National Natural Science Foundation of China(Nos.41427806 and 61273233)
文摘Deep neural networks have been successfully applied to numerous machine learning tasks because of their impressive feature abstraction capabilities.However,conventional deep networks assume that the training and test data are sampled from the same distribution,and this assumption is often violated in real-world scenarios.To address the domain shift or data bias problems,we introduce layer-wise domain correction(LDC),a new unsupervised domain adaptation algorithm which adapts an existing deep network through additive correction layers spaced throughout the network.Through the additive layers,the representations of source and target domains can be perfectly aligned.The corrections that are trained via maximum mean discrepancy,adapt to the target domain while increasing the representational capacity of the network.LDC requires no target labels,achieves state-of-the-art performance across several adaptation benchmarks,and requires significantly less training time than existing adaptation methods.
基金This work is supported by the National Natural Science Foundation of China(Nos.11771276,11471208,61731009)the Foundation of Science and Technology Commission of Shanghai Municipality(No.14DZ2260800).
文摘Generative adversarial networks(GANs)have shown impressive power in the field of machine learning.Traditional GANs have focused on unsupervised learning tasks.In recent years,conditional GANs that can generate data with labels have been proposed in semi-supervised learning and have achieved better image quality than traditional GANs.Conditional GANs,however,generally only minimize the difference between marginal distributions of real and generated data,neglecting the difference with respect to each class of the data.To address this challenge,we propose the GAN with joint distribution moment matching(JDMM-GAN)for matching the joint distribution based on maximum mean discrepancy,which minimizes the differences of both the marginal and conditional distributions.The learning procedure is iteratively conducted by the stochastic gradient descent and back-propagation.We evaluate JDMM-GAN on several benchmark datasets,including MNIST,CIFAR-10 and the Extended Yale Face.Compared with the state-of-the-art GANs,JDMM-GAN generates more realistic images and achieves the best inception score for CIFAR-10 dataset.
