The field of biometric identification has seen significant advancements over the years,with research focusing on enhancing the accuracy and security of these systems.One of the key developments is the integration of d...The field of biometric identification has seen significant advancements over the years,with research focusing on enhancing the accuracy and security of these systems.One of the key developments is the integration of deep learning techniques in biometric systems.However,despite these advancements,certain challenges persist.One of the most significant challenges is scalability over growing complexity.Traditional methods either require maintaining and securing a growing database,introducing serious security challenges,or relying on retraining the entiremodelwhen new data is introduced-a process that can be computationally expensive and complex.This challenge underscores the need for more efficient methods to scale securely.To this end,we introduce a novel approach that addresses these challenges by integrating multimodal biometrics,cancelable biometrics,and incremental learning techniques.This work is among the first attempts to seamlessly incorporate deep cancelable biometrics with dynamic architectural updates,applied incrementally to the deep learning model as new users are enrolled,achieving high performance with minimal catastrophic forgetting.By leveraging a One-Dimensional Convolutional Neural Network(1D-CNN)architecture combined with a hybrid incremental learning approach,our system achieves high recognition accuracy,averaging 98.98% over incrementing datasets,while ensuring user privacy through cancelable templates generated via a pre-trained CNN model and random projection.The approach demonstrates remarkable adaptability,utilizing the least intrusive biometric traits like facial features and fingerprints,ensuring not only robust performance but also long-term serviceability.展开更多
Current machine learning models for predicting geological conditions during earth pressure balance(EPB)shield tunneling predominantly rely on accurate geological conditions as model label inputs.This study introduces ...Current machine learning models for predicting geological conditions during earth pressure balance(EPB)shield tunneling predominantly rely on accurate geological conditions as model label inputs.This study introduces an innovative approach for the real-time prediction of geological conditions in EPB shield tunneling by utilizing an unsupervised incremental learning model that integrates deep temporal clustering(DTC)with elastic weight consolidation(EWC).The model was trained and tested using data from an EPB shield tunneling project in Nanjing,China.Results demonstrate that the DTC model outperforms nine comparison models by clustering the entire dataset into four distinct groups representing various geological conditions without requiring labeled data.Additionally,integrating EWC into the DTC model significantly enhances its continuous learning capabilities,enabling automatic parameter updates with incoming data and facilitating the real-time recognition of geological conditions.Feature importance was evaluated using the feature elimination method and the Shapley additive explanations(SHAP)method,underscoring the critical roles of earth chamber pressure and cutterhead rotation speed in predicting geological conditions.The proposed EWC-DTC model demonstrates practical utility for EPB shield tunneling in complex environments.展开更多
Ever since the research in machine learning gained traction in recent years,it has been employed to address challenges in a wide variety of domains,including mechanical devices.Most of the machine learning models are ...Ever since the research in machine learning gained traction in recent years,it has been employed to address challenges in a wide variety of domains,including mechanical devices.Most of the machine learning models are built on the assumption of a static learning environment,but in practical situations,the data generated by the process is dynamic.This evolution of the data is termed concept drift.This research paper presents an approach for predictingmechanical failure in real-time using incremental learning based on the statistically calculated parameters of mechanical equipment.The method proposed here is applicable to allmechanical devices that are susceptible to failure or operational degradation.The proposed method in this paper is equipped with the capacity to detect the drift in data generation and adaptation.The proposed approach evaluates the machine learning and deep learning models for their efficacy in handling the errors related to industrial machines due to their dynamic nature.It is observed that,in the settings without concept drift in the data,methods like SVM and Random Forest performed better compared to deep neural networks.However,this resulted in poor sensitivity for the smallest drift in the machine data reported as a drift.In this perspective,DNN generated the stable drift detection method;it reported an accuracy of 84%and an AUC of 0.87 while detecting only a single drift point,indicating the stability to performbetter in detecting and adapting to new data in the drifting environments under industrial measurement settings.展开更多
Deep Convolution Neural Networks(DCNNs)can capture discriminative features from large datasets.However,how to incrementally learn new samples without forgetting old ones and recognize novel classes that arise in the d...Deep Convolution Neural Networks(DCNNs)can capture discriminative features from large datasets.However,how to incrementally learn new samples without forgetting old ones and recognize novel classes that arise in the dynamically changing world,e.g.,classifying newly discovered fish species,remains an open problem.We address an even more challenging and realistic setting of this problem where new class samples are insufficient,i.e.,Few-Shot Class-Incremental Learning(FSCIL).Current FSCIL methods augment the training data to alleviate the overfitting of novel classes.By contrast,we propose Filter Bank Networks(FBNs)that augment the learnable filters to capture fine-detailed features for adapting to future new classes.In the forward pass,FBNs augment each convolutional filter to a virtual filter bank containing the canonical one,i.e.,itself,and multiple transformed versions.During back-propagation,FBNs explicitly stimulate fine-detailed features to emerge and collectively align all gradients of each filter bank to learn the canonical one.FBNs capture pattern variants that do not yet exist in the pretraining session,thus making it easy to incorporate new classes in the incremental learning phase.Moreover,FBNs introduce model-level prior knowledge to efficiently utilize the limited few-shot data.Extensive experiments on MNIST,CIFAR100,CUB200,andMini-ImageNet datasets show that FBNs consistently outperformthe baseline by a significantmargin,reporting new state-of-the-art FSCIL results.In addition,we contribute a challenging FSCIL benchmark,Fishshot1K,which contains 8261 underwater images covering 1000 ocean fish species.The code is included in the supplementary materials.展开更多
Predicting the productivity of multistage fractured horizontal wells plays an important role in exploiting unconventional resources.In recent years,machine learning(ML)models have emerged as a new approach for such st...Predicting the productivity of multistage fractured horizontal wells plays an important role in exploiting unconventional resources.In recent years,machine learning(ML)models have emerged as a new approach for such studies.However,the scarcity of sufficient real data for model training often leads to imprecise predictions,even though the models trained with real data better characterize geological and engineering features.To tackle this issue,we propose an ML model that can obtain reliable results even with a small amount of data samples.Our model integrates the synthetic minority oversampling technique(SMOTE)to expand the data volume,the support vector machine(SVM)for model training,and the particle swarm optimization(PSO)algorithm for optimizing hyperparameters.To enhance the model performance,we conduct feature fusion and dimensionality reduction.Additionally,we examine the influences of different sample sizes and ML models for training.The proposed model demonstrates higher prediction accuracy and generalization ability,achieving a predicted R^(2)value of up to 0.9 for the test set,compared to the traditional ML techniques with an R^(2)of 0.13.This model accurately predicts the production of fractured horizontal wells even with limited samples,supplying an efficient tool for optimizing the production of unconventional resources.Importantly,the model holds the potential applicability to address similar challenges in other fields constrained by scarce data samples.展开更多
Image classification is crucial for various applications,including digital construction,smart manu-facturing,and medical imaging.Focusing on the inadequate model generalization and data privacy concerns in few-shot im...Image classification is crucial for various applications,including digital construction,smart manu-facturing,and medical imaging.Focusing on the inadequate model generalization and data privacy concerns in few-shot image classification,in this paper,we propose a federated learning approach that incorporates privacy-preserving techniques.First,we utilize contrastive learning to train on local few-shot image data and apply various data augmentation methods to expand the sample size,thereby enhancing the model’s generalization capabilities in few-shot contexts.Second,we introduce local differential privacy techniques and weight pruning methods to safeguard model parameters,perturbing the transmitted parameters to ensure user data privacy.Finally,numerical simulations are conducted to demonstrate the effectiveness of our proposed method.The results indicate that our approach significantly enhances model generalization and test accuracy compared to several popular federated learning algorithms while maintaining data privacy,highlighting its effectiveness and practicality in addressing the challenges of model generalization and data privacy in few-shot image scenarios.展开更多
Although conventional object detection methods achieve high accuracy through extensively annotated datasets,acquiring such large-scale labeled data remains challenging and cost-prohibitive in numerous real-world appli...Although conventional object detection methods achieve high accuracy through extensively annotated datasets,acquiring such large-scale labeled data remains challenging and cost-prohibitive in numerous real-world applications.Few-shot object detection presents a new research idea that aims to localize and classify objects in images using only limited annotated examples.However,the inherent challenge in few-shot object detection lies in the insufficient sample diversity to fully characterize the sample feature distribution,which consequently impacts model performance.Inspired by contrastive learning principles,we propose an Implicit Feature Contrastive Learning(IFCL)module to address this limitation and augment feature diversity for more robust representational learning.This module generates augmented support sample features in a mixed feature space and implicitly contrasts them with query Region of Interest(RoI)features.This approach facilitates more comprehensive learning of both intra-class feature similarity and inter-class feature diversity,thereby enhancing the model’s object classification and localization capabilities.Extensive experiments on PASCAL VOC show that our method achieves a respective improvement of 3.2%,1.8%,and 2.3%on 10-shot of three Novel Sets compared to the baseline model FPD.展开更多
Air quality estimation assesses the pollution level in the air,supports public health warnings,and is a valuable tool in environmental management.Although air sensors have proven helpful in this task,sensors are often...Air quality estimation assesses the pollution level in the air,supports public health warnings,and is a valuable tool in environmental management.Although air sensors have proven helpful in this task,sensors are often expensive and difficult to install,while cameras are becoming more popular and accessible,from which images can be collected as data for deep learning models to solve the above task.This leads to another problem:several labeled images are needed to achieve high accuracy when deep-learningmodels predict air quality.In this research,we have threemain contributions:(1)Collect and publish an air quality estimation dataset,namely PTIT_AQED,including environmental image data and air quality;(2)Propose a deep learning model to predict air quality with few data,called PTIT_FAQE(PTIT Few-shot air quality estimation).We build PTIT_FAQE based on EfficientNet-a CNN architecture that ensures high performance in deep learning applications and Few-shot Learning with Prototypical Networks.This helps the model use only a fewtraining data but still achieve high accuracy in air quality estimation.And(3)conduct experiments to prove the superiority of PTIT_FAQE compared to other studies on both PTIT_AQED and APIN datasets.The results show that our model achieves an accuracy of 0.9278 and an F1-Score of 0.9139 on the PTIT_AQED dataset and an accuracy of 0.9467 and an F1-Score of 0.9371 on the APIN dataset,which demonstrate a significant performance improvement compared to previous studies.We also conduct detailed experiments to evaluate the impact of each component on model performance.展开更多
Few-shot point cloud 3D object detection(FS3D)aims to identify and locate objects of novel classes within point clouds using knowledge acquired from annotated base classes and a minimal number of samples from the nove...Few-shot point cloud 3D object detection(FS3D)aims to identify and locate objects of novel classes within point clouds using knowledge acquired from annotated base classes and a minimal number of samples from the novel classes.Due to imbalanced training data,existing FS3D methods based on fully supervised learning can lead to overfitting toward base classes,which impairs the network’s ability to generalize knowledge learned from base classes to novel classes and also prevents the network from extracting distinctive foreground and background representations for novel class objects.To address these issues,this thesis proposes a category-agnostic contrastive learning approach,enhancing the generalization and identification abilities for almost unseen categories through the construction of pseudo-labels and positive-negative sample pairs unrelated to specific classes.Firstly,this thesis designs a proposal-wise context contrastive module(CCM).By reducing the distance between foreground point features and increasing the distance between foreground and background point features within a region proposal,CCM aids the network in extracting more discriminative foreground and background feature representations without reliance on categorical annotations.Secondly,this thesis utilizes a geometric contrastive module(GCM),which enhances the network’s geometric perception capability by employing contrastive learning on the foreground point features associated with various basic geometric components,such as edges,corners,and surfaces,thereby enabling these geometric components to exhibit more distinguishable representations.This thesis also combines category-aware contrastive learning with former modules to maintain categorical distinctiveness.Extensive experimental results on FS-SUNRGBD and FS-ScanNet datasets demonstrate the effectiveness of this method with average precision exceeding the baseline by up to 8%.展开更多
Full ceramic bearings are mission-critical components in oil-free environments,such as food processing,semiconductor manufacturing,and medical applications.Developing effective fault diagnosis methods for these bearin...Full ceramic bearings are mission-critical components in oil-free environments,such as food processing,semiconductor manufacturing,and medical applications.Developing effective fault diagnosis methods for these bearings is essential to ensuring operational reliability and preventing costly failures.Traditional supervised deep learning approaches have demonstrated promise in fault detection,but their dependence on large labeled datasets poses significant challenges in industrial settings where fault-labeled data is scarce.This paper introduces a few-shot learning approach for full ceramic bearing fault diagnosis by leveraging the pre-trained GPT-2 model.Large language models(LLMs)like GPT-2,pre-trained on diverse textual data,exhibit remarkable transfer learning and few-shot learning capabilities,making them ideal for applications with limited labeled data.In this study,acoustic emission(AE)signals from bearings were processed using empirical mode decomposition(EMD),and the extracted AE features were converted into structured text for fine-tuning GPT-2 as a fault classifier.To enhance its performance,we incorporated a modified loss function and softmax activation with cosine similarity,ensuring better generalization in fault identification.Experimental evaluations on a laboratory-collected full ceramic bearing dataset demonstrated that the proposed approach achieved high diagnostic accuracy with as few as five labeled samples,outperforming conventional methods such as k-nearest neighbor(KNN),large memory storage and retrieval(LAMSTAR)neural network,deep neural network(DNN),recurrent neural network(RNN),long short-term memory(LSTM)network,and model-agnostic meta-learning(MAML).The results highlight LLMs’potential to revolutionize fault diagnosis,enabling faster deployment,reduced reliance on extensive labeled datasets,and improved adaptability in industrial monitoring systems.展开更多
In this paper, we explore a novel ensemble method for spectral clustering. In contrast to the traditional clustering ensemble methods that combine all the obtained clustering results, we propose the adaptive spectral ...In this paper, we explore a novel ensemble method for spectral clustering. In contrast to the traditional clustering ensemble methods that combine all the obtained clustering results, we propose the adaptive spectral clustering ensemble method to achieve a better clustering solution. This method can adaptively assess the number of the component members, which is not owned by many other algorithms. The component clusterings of the ensemble system are generated by spectral clustering （SC） which bears some good characteristics to engender the diverse committees. The selection process works by evaluating the generated component spectral clustering through resampling technique and population-based incremental learning algorithm （PBIL）. Experimental results on UCI datasets demonstrate that the proposed algorithm can achieve better results compared with traditional clustering ensemble methods, especially when the number of component clusterings is large.展开更多
A new incremental support vector machine （SVM） algorithm is proposed which is based on multiple kernel learning. Through introducing multiple kernel learning into the SVM incremental learning, large scale data set l...A new incremental support vector machine （SVM） algorithm is proposed which is based on multiple kernel learning. Through introducing multiple kernel learning into the SVM incremental learning, large scale data set learning problem can be solved effectively. Furthermore, different punishments are adopted in allusion to the training subset and the acquired support vectors, which may help to improve the performance of SVM. Simulation results indicate that the proposed algorithm can not only solve the model selection problem in SVM incremental learning, but also improve the classification or prediction precision.展开更多
In view of the incremental learning problem of manifold learning algorithm, an adaptive neighborhood incremental principal component analysis(PCA) and locality preserving projection(LPP) manifold learning algorithm is...In view of the incremental learning problem of manifold learning algorithm, an adaptive neighborhood incremental principal component analysis(PCA) and locality preserving projection(LPP) manifold learning algorithm is presented, and the incremental learning principle of algorithm is introduced. For incremental sample data, the adjacency and covariance matrices are incrementally updated by the existing samples; then the dimensionality reduction results of the incremental samples are estimated by the dimensionality reduction results of the existing samples; finally, the dimensionality reduction results of the incremental and existing samples are updated by subspace iteration method. The adaptive neighborhood incremental PCA-LPP manifold learning algorithm is applied to processing of gearbox fault signals. The dimensionality reduction results by incremental learning have very small error, compared with those by batch learning. Spatial aggregation of the incremental samples is basically stable, and fault identification rate is increased.展开更多
Intelligent seismic facies identification based on deep learning can alleviate the time-consuming and labor-intensive problem of manual interpretation,which has been widely applied.Supervised learning can realize faci...Intelligent seismic facies identification based on deep learning can alleviate the time-consuming and labor-intensive problem of manual interpretation,which has been widely applied.Supervised learning can realize facies identification with high efficiency and accuracy;however,it depends on the usage of a large amount of well-labeled data.To solve this issue,we propose herein an incremental semi-supervised method for intelligent facies identification.Our method considers the continuity of the lateral variation of strata and uses cosine similarity to quantify the similarity of the seismic data feature domain.The maximum-diff erence sample in the neighborhood of the currently used training data is then found to reasonably expand the training sets.This process continuously increases the amount of training data and learns its distribution.We integrate old knowledge while absorbing new ones to realize incremental semi-supervised learning and achieve the purpose of evolving the network models.In this work,accuracy and confusion matrix are employed to jointly control the predicted results of the model from both overall and partial aspects.The obtained values are then applied to a three-dimensional(3D)real dataset and used to quantitatively evaluate the results.Using unlabeled data,our proposed method acquires more accurate and stable testing results compared to conventional supervised learning algorithms that only use well-labeled data.A considerable improvement for small-sample categories is also observed.Using less than 1%of the training data,the proposed method can achieve an average accuracy of over 95%on the 3D dataset.In contrast,the conventional supervised learning algorithm achieved only approximately 85%.展开更多
The maintenance of an aero-engine usually includes three levels,and the maintenance cost and period greatly differ depending on the different maintenance levels.To plan a reasonable maintenance budget program, airline...The maintenance of an aero-engine usually includes three levels,and the maintenance cost and period greatly differ depending on the different maintenance levels.To plan a reasonable maintenance budget program, airlines would like to predict the maintenance level of aero-engine before repairing in terms of performance parameters,which can provide more economic benefits.The maintenance level decision rules are mined using the historical maintenance data of a civil aero-engine based on the rough set theory,and a variety of possible models of updating rules produced by newly increased maintenance cases added to the historical maintenance case database are investigated by the means of incremental machine learning.The continuously updated rules can provide reasonable guidance suggestions for engineers and decision support for planning a maintenance budget program before repairing. The results of an example show that the decision rules become more typical and robust,and they are more accurate to predict the maintenance level of an aero-engine module as the maintenance data increase,which illustrates the feasibility of the represented method.展开更多
According to the classic Karush-Kuhn-Tucker(KKT)theorem,at every step of incremental support vector machine(SVM)learning,the newly adding sample which violates the KKT conditions will be a new support vector(SV)and mi...According to the classic Karush-Kuhn-Tucker(KKT)theorem,at every step of incremental support vector machine(SVM)learning,the newly adding sample which violates the KKT conditions will be a new support vector(SV)and migrate the old samples between SV set and non-support vector(NSV)set,and at the same time the learning model should be updated based on the SVs.However,it is not exactly clear at this moment that which of the old samples would change between SVs and NSVs.Additionally,the learning model will be unnecessarily updated,which will not greatly increase its accuracy but decrease the training speed.Therefore,how to choose the new SVs from old sets during the incremental stages and when to process incremental steps will greatly influence the accuracy and efficiency of incremental SVM learning.In this work,a new algorithm is proposed to select candidate SVs and use the wrongly predicted sample to trigger the incremental processing simultaneously.Experimental results show that the proposed algorithm can achieve good performance with high efficiency,high speed and good accuracy.展开更多
This article presents hourly load forecasting by using an incremental learning model called Online Sequential Extreme Learning Machine(OSELM),which can learn and adapt automatically according to new arrival input.Howe...This article presents hourly load forecasting by using an incremental learning model called Online Sequential Extreme Learning Machine(OSELM),which can learn and adapt automatically according to new arrival input.However,the use of OS-ELM requires a sufficient amount of initial training sample data,which makes OS-ELM inoperable if sufficiently accurate sample data cannot be obtained.To solve this problem,a synthesis of the initial training sample is proposed.The synthesis of the initial sample is achieved by taking the first data received at the start of working and adding random noises to that data to create new and sufficient samples.Then the synthesis samples are used to initial train the OS-ELM.This proposed method is compared with Fully Online Extreme Learning Machine(FOS-ELM),which is an incremental learning model that also does not require the initial training samples.Both the proposed method and FOS-ELM are used for hourly load forecasting from the Hourly Energy Consumption dataset.Experiments have shown that the proposed method with a wide range of noise levels,can forecast hourly load more accurately than the FOS-ELM.展开更多
In order to improve the efficiency of learning the triangular membership functions( TMFs) for mining fuzzy association rule( FAR) in dynamic database,a single-pass fuzzy c means( SPFCM)algorithm is combined with the r...In order to improve the efficiency of learning the triangular membership functions( TMFs) for mining fuzzy association rule( FAR) in dynamic database,a single-pass fuzzy c means( SPFCM)algorithm is combined with the real-coded CHC genetic model to incrementally learn the TMFs. The cluster centers resulting from SPFCM are regarded as the midpoint of TMFs. The population of CHC is generated randomly according to the cluster center and constraint conditions among TMFs. Then a new population for incremental learning is composed of the excellent chromosomes stored in the first genetic process and the chromosomes generated based on the cluster center adjusted by SPFCM. The experiments on real datasets show that the number of generations converging to the solution of the proposed approach is less than that of the existing batch learning approach. The quality of TMFs generated by the approach is comparable to that of the batch learning approach. Compared with the existing incremental learning strategy,the proposed approach is superior in terms of the quality of TMFs and time cost.展开更多
In this study, we propose an incremental learning approach based on a machine-machine interaction via relative attribute feedbacks that exploit comparative relationships among top level image categories. One machine a...In this study, we propose an incremental learning approach based on a machine-machine interaction via relative attribute feedbacks that exploit comparative relationships among top level image categories. One machine acts as ＇Student （S）＇ with initially limited information and it endeavors to capture the task domain gradually by questioning its mentor on a pool of unlabeled data. The other machine is ＇Teacher （T）＇ with the implicit knowledge for helping S on learning the class models. T initiates relative attributes as a communication channel by randomly sorting the classes on attribute space in an unsupervised manner. S starts modeling the categories in this intermediate level by using only a limited number of labeled data. Thereafter, it first selects an entropy-based sample from the pool of unlabeled data and triggers the conversation by propagating the selected image with its belief class in a query. Since T already knows the ground truth labels, it not only decides whether the belief is true or false, but it also provides an attribute-based feedback to S in each case without revealing the true label of the query sample if the belief is false. So the number of training data is increased virtually by dropping the falsely predicted sample back into the unlabeled pool. Next, S updates the attribute space which, in fact, has an impact on T＇s future responses, and then the category models are updated concurrently for the next run. We experience the weakly supervised algorithm on the real world datasets of faces and natural scenes in comparison with direct attribute prediction and semi-supervised learning approaches, and a noteworthy performance increase is achieved.展开更多
The visions of Industry 4.0 and 5.0 have reinforced the industrial environment.They have also made artificial intelligence incorporated as a major facilitator.Diagnosing machine faults has become a solid foundation fo...The visions of Industry 4.0 and 5.0 have reinforced the industrial environment.They have also made artificial intelligence incorporated as a major facilitator.Diagnosing machine faults has become a solid foundation for automatically recognizing machine failure,and thus timely maintenance can ensure safe operations.Transfer learning is a promising solution that can enhance the machine fault diagnosis model by borrowing pre-trained knowledge from the source model and applying it to the target model,which typically involves two datasets.In response to the availability of multiple datasets,this paper proposes using selective and adaptive incremental transfer learning(SA-ITL),which fuses three algorithms,namely,the hybrid selective algorithm,the transferability enhancement algorithm,and the incremental transfer learning algorithm.It is a selective algorithm that enables selecting and ordering appropriate datasets for transfer learning and selecting useful knowledge to avoid negative transfer.The algorithm also adaptively adjusts the portion of training data to balance the learning rate and training time.The proposed algorithm is evaluated and analyzed using ten benchmark datasets.Compared with other algorithms from existing works,SA-ITL improves the accuracy of all datasets.Ablation studies present the accuracy enhancements of the SA-ITL,including the hybrid selective algorithm(1.22%-3.82%),transferability enhancement algorithm(1.91%-4.15%),and incremental transfer learning algorithm(0.605%-2.68%).These also show the benefits of enhancing the target model with heterogeneous image datasets that widen the range of domain selection between source and target domains.展开更多
基金the Deputyship for Research&Innovation,Ministry of Education in Saudi Arabia for funding this research work through project number RI-44-0833.
文摘The field of biometric identification has seen significant advancements over the years,with research focusing on enhancing the accuracy and security of these systems.One of the key developments is the integration of deep learning techniques in biometric systems.However,despite these advancements,certain challenges persist.One of the most significant challenges is scalability over growing complexity.Traditional methods either require maintaining and securing a growing database,introducing serious security challenges,or relying on retraining the entiremodelwhen new data is introduced-a process that can be computationally expensive and complex.This challenge underscores the need for more efficient methods to scale securely.To this end,we introduce a novel approach that addresses these challenges by integrating multimodal biometrics,cancelable biometrics,and incremental learning techniques.This work is among the first attempts to seamlessly incorporate deep cancelable biometrics with dynamic architectural updates,applied incrementally to the deep learning model as new users are enrolled,achieving high performance with minimal catastrophic forgetting.By leveraging a One-Dimensional Convolutional Neural Network(1D-CNN)architecture combined with a hybrid incremental learning approach,our system achieves high recognition accuracy,averaging 98.98% over incrementing datasets,while ensuring user privacy through cancelable templates generated via a pre-trained CNN model and random projection.The approach demonstrates remarkable adaptability,utilizing the least intrusive biometric traits like facial features and fingerprints,ensuring not only robust performance but also long-term serviceability.
基金financially supported by the National Natural Science Foundation of China(Grant Nos.52378392,52408356)Foal Eagle Program Youth Top-notch Talent Project of Fujian Province,China(Grant No.00387088).
文摘Current machine learning models for predicting geological conditions during earth pressure balance(EPB)shield tunneling predominantly rely on accurate geological conditions as model label inputs.This study introduces an innovative approach for the real-time prediction of geological conditions in EPB shield tunneling by utilizing an unsupervised incremental learning model that integrates deep temporal clustering(DTC)with elastic weight consolidation(EWC).The model was trained and tested using data from an EPB shield tunneling project in Nanjing,China.Results demonstrate that the DTC model outperforms nine comparison models by clustering the entire dataset into four distinct groups representing various geological conditions without requiring labeled data.Additionally,integrating EWC into the DTC model significantly enhances its continuous learning capabilities,enabling automatic parameter updates with incoming data and facilitating the real-time recognition of geological conditions.Feature importance was evaluated using the feature elimination method and the Shapley additive explanations(SHAP)method,underscoring the critical roles of earth chamber pressure and cutterhead rotation speed in predicting geological conditions.The proposed EWC-DTC model demonstrates practical utility for EPB shield tunneling in complex environments.
文摘Ever since the research in machine learning gained traction in recent years,it has been employed to address challenges in a wide variety of domains,including mechanical devices.Most of the machine learning models are built on the assumption of a static learning environment,but in practical situations,the data generated by the process is dynamic.This evolution of the data is termed concept drift.This research paper presents an approach for predictingmechanical failure in real-time using incremental learning based on the statistically calculated parameters of mechanical equipment.The method proposed here is applicable to allmechanical devices that are susceptible to failure or operational degradation.The proposed method in this paper is equipped with the capacity to detect the drift in data generation and adaptation.The proposed approach evaluates the machine learning and deep learning models for their efficacy in handling the errors related to industrial machines due to their dynamic nature.It is observed that,in the settings without concept drift in the data,methods like SVM and Random Forest performed better compared to deep neural networks.However,this resulted in poor sensitivity for the smallest drift in the machine data reported as a drift.In this perspective,DNN generated the stable drift detection method;it reported an accuracy of 84%and an AUC of 0.87 while detecting only a single drift point,indicating the stability to performbetter in detecting and adapting to new data in the drifting environments under industrial measurement settings.
基金support from the Strategic Priority Research Program of the Chinese Academy of Sciences under Grant No.XDA27000000.
文摘Deep Convolution Neural Networks(DCNNs)can capture discriminative features from large datasets.However,how to incrementally learn new samples without forgetting old ones and recognize novel classes that arise in the dynamically changing world,e.g.,classifying newly discovered fish species,remains an open problem.We address an even more challenging and realistic setting of this problem where new class samples are insufficient,i.e.,Few-Shot Class-Incremental Learning(FSCIL).Current FSCIL methods augment the training data to alleviate the overfitting of novel classes.By contrast,we propose Filter Bank Networks(FBNs)that augment the learnable filters to capture fine-detailed features for adapting to future new classes.In the forward pass,FBNs augment each convolutional filter to a virtual filter bank containing the canonical one,i.e.,itself,and multiple transformed versions.During back-propagation,FBNs explicitly stimulate fine-detailed features to emerge and collectively align all gradients of each filter bank to learn the canonical one.FBNs capture pattern variants that do not yet exist in the pretraining session,thus making it easy to incorporate new classes in the incremental learning phase.Moreover,FBNs introduce model-level prior knowledge to efficiently utilize the limited few-shot data.Extensive experiments on MNIST,CIFAR100,CUB200,andMini-ImageNet datasets show that FBNs consistently outperformthe baseline by a significantmargin,reporting new state-of-the-art FSCIL results.In addition,we contribute a challenging FSCIL benchmark,Fishshot1K,which contains 8261 underwater images covering 1000 ocean fish species.The code is included in the supplementary materials.
基金supported by the National Natural Science Foundation of China(52274055)the Shandong Provincial Natural Science Foundation(ZR2022YQ50)the Taishan Scholar Program of Shandong Province(tsqn202408088)。
文摘Predicting the productivity of multistage fractured horizontal wells plays an important role in exploiting unconventional resources.In recent years,machine learning(ML)models have emerged as a new approach for such studies.However,the scarcity of sufficient real data for model training often leads to imprecise predictions,even though the models trained with real data better characterize geological and engineering features.To tackle this issue,we propose an ML model that can obtain reliable results even with a small amount of data samples.Our model integrates the synthetic minority oversampling technique(SMOTE)to expand the data volume,the support vector machine(SVM)for model training,and the particle swarm optimization(PSO)algorithm for optimizing hyperparameters.To enhance the model performance,we conduct feature fusion and dimensionality reduction.Additionally,we examine the influences of different sample sizes and ML models for training.The proposed model demonstrates higher prediction accuracy and generalization ability,achieving a predicted R^(2)value of up to 0.9 for the test set,compared to the traditional ML techniques with an R^(2)of 0.13.This model accurately predicts the production of fractured horizontal wells even with limited samples,supplying an efficient tool for optimizing the production of unconventional resources.Importantly,the model holds the potential applicability to address similar challenges in other fields constrained by scarce data samples.
基金supported by Suzhou Science and Technology Plan(Basic Research)Project under Grant SJC2023002Postgraduate Research&Practice Innovation Program of Jiangsu Province under Grant KYCX23_3322.
文摘Image classification is crucial for various applications,including digital construction,smart manu-facturing,and medical imaging.Focusing on the inadequate model generalization and data privacy concerns in few-shot image classification,in this paper,we propose a federated learning approach that incorporates privacy-preserving techniques.First,we utilize contrastive learning to train on local few-shot image data and apply various data augmentation methods to expand the sample size,thereby enhancing the model’s generalization capabilities in few-shot contexts.Second,we introduce local differential privacy techniques and weight pruning methods to safeguard model parameters,perturbing the transmitted parameters to ensure user data privacy.Finally,numerical simulations are conducted to demonstrate the effectiveness of our proposed method.The results indicate that our approach significantly enhances model generalization and test accuracy compared to several popular federated learning algorithms while maintaining data privacy,highlighting its effectiveness and practicality in addressing the challenges of model generalization and data privacy in few-shot image scenarios.
基金funded by the China Chongqing Municipal Science and Technology Bureau,grant numbers CSTB2024TIAD-CYKJCXX0009,CSTB2024NSCQ-LZX0043,CSTB2022NSCQ-MSX0288Chongqing Municipal Commission of Housing and Urban-Rural Development,grant number CKZ2024-87+3 种基金the Chongqing University of Technology Graduate Education High-Quality Development Project,grant number gzlsz202401the Chongqing University of Technology—Chongqing LINGLUE Technology Co.,Ltd.Electronic Information(Artificial Intelligence)Graduate Joint Training Basethe Postgraduate Education and Teaching Reform Research Project in Chongqing,grant number yjg213116the Chongqing University of Technology-CISDI Chongqing Information Technology Co.,Ltd.Computer Technology Graduate Joint Training Base.
文摘Although conventional object detection methods achieve high accuracy through extensively annotated datasets,acquiring such large-scale labeled data remains challenging and cost-prohibitive in numerous real-world applications.Few-shot object detection presents a new research idea that aims to localize and classify objects in images using only limited annotated examples.However,the inherent challenge in few-shot object detection lies in the insufficient sample diversity to fully characterize the sample feature distribution,which consequently impacts model performance.Inspired by contrastive learning principles,we propose an Implicit Feature Contrastive Learning(IFCL)module to address this limitation and augment feature diversity for more robust representational learning.This module generates augmented support sample features in a mixed feature space and implicitly contrasts them with query Region of Interest(RoI)features.This approach facilitates more comprehensive learning of both intra-class feature similarity and inter-class feature diversity,thereby enhancing the model’s object classification and localization capabilities.Extensive experiments on PASCAL VOC show that our method achieves a respective improvement of 3.2%,1.8%,and 2.3%on 10-shot of three Novel Sets compared to the baseline model FPD.
文摘Air quality estimation assesses the pollution level in the air,supports public health warnings,and is a valuable tool in environmental management.Although air sensors have proven helpful in this task,sensors are often expensive and difficult to install,while cameras are becoming more popular and accessible,from which images can be collected as data for deep learning models to solve the above task.This leads to another problem:several labeled images are needed to achieve high accuracy when deep-learningmodels predict air quality.In this research,we have threemain contributions:(1)Collect and publish an air quality estimation dataset,namely PTIT_AQED,including environmental image data and air quality;(2)Propose a deep learning model to predict air quality with few data,called PTIT_FAQE(PTIT Few-shot air quality estimation).We build PTIT_FAQE based on EfficientNet-a CNN architecture that ensures high performance in deep learning applications and Few-shot Learning with Prototypical Networks.This helps the model use only a fewtraining data but still achieve high accuracy in air quality estimation.And(3)conduct experiments to prove the superiority of PTIT_FAQE compared to other studies on both PTIT_AQED and APIN datasets.The results show that our model achieves an accuracy of 0.9278 and an F1-Score of 0.9139 on the PTIT_AQED dataset and an accuracy of 0.9467 and an F1-Score of 0.9371 on the APIN dataset,which demonstrate a significant performance improvement compared to previous studies.We also conduct detailed experiments to evaluate the impact of each component on model performance.
文摘Few-shot point cloud 3D object detection(FS3D)aims to identify and locate objects of novel classes within point clouds using knowledge acquired from annotated base classes and a minimal number of samples from the novel classes.Due to imbalanced training data,existing FS3D methods based on fully supervised learning can lead to overfitting toward base classes,which impairs the network’s ability to generalize knowledge learned from base classes to novel classes and also prevents the network from extracting distinctive foreground and background representations for novel class objects.To address these issues,this thesis proposes a category-agnostic contrastive learning approach,enhancing the generalization and identification abilities for almost unseen categories through the construction of pseudo-labels and positive-negative sample pairs unrelated to specific classes.Firstly,this thesis designs a proposal-wise context contrastive module(CCM).By reducing the distance between foreground point features and increasing the distance between foreground and background point features within a region proposal,CCM aids the network in extracting more discriminative foreground and background feature representations without reliance on categorical annotations.Secondly,this thesis utilizes a geometric contrastive module(GCM),which enhances the network’s geometric perception capability by employing contrastive learning on the foreground point features associated with various basic geometric components,such as edges,corners,and surfaces,thereby enabling these geometric components to exhibit more distinguishable representations.This thesis also combines category-aware contrastive learning with former modules to maintain categorical distinctiveness.Extensive experimental results on FS-SUNRGBD and FS-ScanNet datasets demonstrate the effectiveness of this method with average precision exceeding the baseline by up to 8%.
文摘Full ceramic bearings are mission-critical components in oil-free environments,such as food processing,semiconductor manufacturing,and medical applications.Developing effective fault diagnosis methods for these bearings is essential to ensuring operational reliability and preventing costly failures.Traditional supervised deep learning approaches have demonstrated promise in fault detection,but their dependence on large labeled datasets poses significant challenges in industrial settings where fault-labeled data is scarce.This paper introduces a few-shot learning approach for full ceramic bearing fault diagnosis by leveraging the pre-trained GPT-2 model.Large language models(LLMs)like GPT-2,pre-trained on diverse textual data,exhibit remarkable transfer learning and few-shot learning capabilities,making them ideal for applications with limited labeled data.In this study,acoustic emission(AE)signals from bearings were processed using empirical mode decomposition(EMD),and the extracted AE features were converted into structured text for fine-tuning GPT-2 as a fault classifier.To enhance its performance,we incorporated a modified loss function and softmax activation with cosine similarity,ensuring better generalization in fault identification.Experimental evaluations on a laboratory-collected full ceramic bearing dataset demonstrated that the proposed approach achieved high diagnostic accuracy with as few as five labeled samples,outperforming conventional methods such as k-nearest neighbor(KNN),large memory storage and retrieval(LAMSTAR)neural network,deep neural network(DNN),recurrent neural network(RNN),long short-term memory(LSTM)network,and model-agnostic meta-learning(MAML).The results highlight LLMs’potential to revolutionize fault diagnosis,enabling faster deployment,reduced reliance on extensive labeled datasets,and improved adaptability in industrial monitoring systems.
基金Supported by the National Natural Science Foundation of China (60661003)the Research Project Department of Education of Jiangxi Province (GJJ10566)
文摘In this paper, we explore a novel ensemble method for spectral clustering. In contrast to the traditional clustering ensemble methods that combine all the obtained clustering results, we propose the adaptive spectral clustering ensemble method to achieve a better clustering solution. This method can adaptively assess the number of the component members, which is not owned by many other algorithms. The component clusterings of the ensemble system are generated by spectral clustering （SC） which bears some good characteristics to engender the diverse committees. The selection process works by evaluating the generated component spectral clustering through resampling technique and population-based incremental learning algorithm （PBIL）. Experimental results on UCI datasets demonstrate that the proposed algorithm can achieve better results compared with traditional clustering ensemble methods, especially when the number of component clusterings is large.
基金supported by the National Natural Science Key Foundation of China(69974021)
文摘A new incremental support vector machine （SVM） algorithm is proposed which is based on multiple kernel learning. Through introducing multiple kernel learning into the SVM incremental learning, large scale data set learning problem can be solved effectively. Furthermore, different punishments are adopted in allusion to the training subset and the acquired support vectors, which may help to improve the performance of SVM. Simulation results indicate that the proposed algorithm can not only solve the model selection problem in SVM incremental learning, but also improve the classification or prediction precision.
基金the National Natural Science Foundation of China(No.50775219)
文摘In view of the incremental learning problem of manifold learning algorithm, an adaptive neighborhood incremental principal component analysis(PCA) and locality preserving projection(LPP) manifold learning algorithm is presented, and the incremental learning principle of algorithm is introduced. For incremental sample data, the adjacency and covariance matrices are incrementally updated by the existing samples; then the dimensionality reduction results of the incremental samples are estimated by the dimensionality reduction results of the existing samples; finally, the dimensionality reduction results of the incremental and existing samples are updated by subspace iteration method. The adaptive neighborhood incremental PCA-LPP manifold learning algorithm is applied to processing of gearbox fault signals. The dimensionality reduction results by incremental learning have very small error, compared with those by batch learning. Spatial aggregation of the incremental samples is basically stable, and fault identification rate is increased.
基金financially supported by the National Key R&D Program of China(No.2018YFA0702504)the National Natural Science Foundation of China(No.42174152 and No.41974140)+1 种基金the Science Foundation of China University of Petroleum,Beijing(No.2462020YXZZ008 and No.2462020QZDX003)the Strategic Cooperation Technology Projects of CNPC and CUPB(No.ZLZX2020-03).
文摘Intelligent seismic facies identification based on deep learning can alleviate the time-consuming and labor-intensive problem of manual interpretation,which has been widely applied.Supervised learning can realize facies identification with high efficiency and accuracy;however,it depends on the usage of a large amount of well-labeled data.To solve this issue,we propose herein an incremental semi-supervised method for intelligent facies identification.Our method considers the continuity of the lateral variation of strata and uses cosine similarity to quantify the similarity of the seismic data feature domain.The maximum-diff erence sample in the neighborhood of the currently used training data is then found to reasonably expand the training sets.This process continuously increases the amount of training data and learns its distribution.We integrate old knowledge while absorbing new ones to realize incremental semi-supervised learning and achieve the purpose of evolving the network models.In this work,accuracy and confusion matrix are employed to jointly control the predicted results of the model from both overall and partial aspects.The obtained values are then applied to a three-dimensional(3D)real dataset and used to quantitatively evaluate the results.Using unlabeled data,our proposed method acquires more accurate and stable testing results compared to conventional supervised learning algorithms that only use well-labeled data.A considerable improvement for small-sample categories is also observed.Using less than 1%of the training data,the proposed method can achieve an average accuracy of over 95%on the 3D dataset.In contrast,the conventional supervised learning algorithm achieved only approximately 85%.
基金Supported by the National Natural Science Foundation of China(60939003)
文摘The maintenance of an aero-engine usually includes three levels,and the maintenance cost and period greatly differ depending on the different maintenance levels.To plan a reasonable maintenance budget program, airlines would like to predict the maintenance level of aero-engine before repairing in terms of performance parameters,which can provide more economic benefits.The maintenance level decision rules are mined using the historical maintenance data of a civil aero-engine based on the rough set theory,and a variety of possible models of updating rules produced by newly increased maintenance cases added to the historical maintenance case database are investigated by the means of incremental machine learning.The continuously updated rules can provide reasonable guidance suggestions for engineers and decision support for planning a maintenance budget program before repairing. The results of an example show that the decision rules become more typical and robust,and they are more accurate to predict the maintenance level of an aero-engine module as the maintenance data increase,which illustrates the feasibility of the represented method.
基金supported by the National Natural Science Foundation of China(Nos.U1509207 and 61325019)
文摘According to the classic Karush-Kuhn-Tucker(KKT)theorem,at every step of incremental support vector machine(SVM)learning,the newly adding sample which violates the KKT conditions will be a new support vector(SV)and migrate the old samples between SV set and non-support vector(NSV)set,and at the same time the learning model should be updated based on the SVs.However,it is not exactly clear at this moment that which of the old samples would change between SVs and NSVs.Additionally,the learning model will be unnecessarily updated,which will not greatly increase its accuracy but decrease the training speed.Therefore,how to choose the new SVs from old sets during the incremental stages and when to process incremental steps will greatly influence the accuracy and efficiency of incremental SVM learning.In this work,a new algorithm is proposed to select candidate SVs and use the wrongly predicted sample to trigger the incremental processing simultaneously.Experimental results show that the proposed algorithm can achieve good performance with high efficiency,high speed and good accuracy.
文摘This article presents hourly load forecasting by using an incremental learning model called Online Sequential Extreme Learning Machine(OSELM),which can learn and adapt automatically according to new arrival input.However,the use of OS-ELM requires a sufficient amount of initial training sample data,which makes OS-ELM inoperable if sufficiently accurate sample data cannot be obtained.To solve this problem,a synthesis of the initial training sample is proposed.The synthesis of the initial sample is achieved by taking the first data received at the start of working and adding random noises to that data to create new and sufficient samples.Then the synthesis samples are used to initial train the OS-ELM.This proposed method is compared with Fully Online Extreme Learning Machine(FOS-ELM),which is an incremental learning model that also does not require the initial training samples.Both the proposed method and FOS-ELM are used for hourly load forecasting from the Hourly Energy Consumption dataset.Experiments have shown that the proposed method with a wide range of noise levels,can forecast hourly load more accurately than the FOS-ELM.
基金Supported by the National Natural Science Foundation of China(No.61301245,U1533104)
文摘In order to improve the efficiency of learning the triangular membership functions( TMFs) for mining fuzzy association rule( FAR) in dynamic database,a single-pass fuzzy c means( SPFCM)algorithm is combined with the real-coded CHC genetic model to incrementally learn the TMFs. The cluster centers resulting from SPFCM are regarded as the midpoint of TMFs. The population of CHC is generated randomly according to the cluster center and constraint conditions among TMFs. Then a new population for incremental learning is composed of the excellent chromosomes stored in the first genetic process and the chromosomes generated based on the cluster center adjusted by SPFCM. The experiments on real datasets show that the number of generations converging to the solution of the proposed approach is less than that of the existing batch learning approach. The quality of TMFs generated by the approach is comparable to that of the batch learning approach. Compared with the existing incremental learning strategy,the proposed approach is superior in terms of the quality of TMFs and time cost.
文摘In this study, we propose an incremental learning approach based on a machine-machine interaction via relative attribute feedbacks that exploit comparative relationships among top level image categories. One machine acts as ＇Student （S）＇ with initially limited information and it endeavors to capture the task domain gradually by questioning its mentor on a pool of unlabeled data. The other machine is ＇Teacher （T）＇ with the implicit knowledge for helping S on learning the class models. T initiates relative attributes as a communication channel by randomly sorting the classes on attribute space in an unsupervised manner. S starts modeling the categories in this intermediate level by using only a limited number of labeled data. Thereafter, it first selects an entropy-based sample from the pool of unlabeled data and triggers the conversation by propagating the selected image with its belief class in a query. Since T already knows the ground truth labels, it not only decides whether the belief is true or false, but it also provides an attribute-based feedback to S in each case without revealing the true label of the query sample if the belief is false. So the number of training data is increased virtually by dropping the falsely predicted sample back into the unlabeled pool. Next, S updates the attribute space which, in fact, has an impact on T＇s future responses, and then the category models are updated concurrently for the next run. We experience the weakly supervised algorithm on the real world datasets of faces and natural scenes in comparison with direct attribute prediction and semi-supervised learning approaches, and a noteworthy performance increase is achieved.
文摘The visions of Industry 4.0 and 5.0 have reinforced the industrial environment.They have also made artificial intelligence incorporated as a major facilitator.Diagnosing machine faults has become a solid foundation for automatically recognizing machine failure,and thus timely maintenance can ensure safe operations.Transfer learning is a promising solution that can enhance the machine fault diagnosis model by borrowing pre-trained knowledge from the source model and applying it to the target model,which typically involves two datasets.In response to the availability of multiple datasets,this paper proposes using selective and adaptive incremental transfer learning(SA-ITL),which fuses three algorithms,namely,the hybrid selective algorithm,the transferability enhancement algorithm,and the incremental transfer learning algorithm.It is a selective algorithm that enables selecting and ordering appropriate datasets for transfer learning and selecting useful knowledge to avoid negative transfer.The algorithm also adaptively adjusts the portion of training data to balance the learning rate and training time.The proposed algorithm is evaluated and analyzed using ten benchmark datasets.Compared with other algorithms from existing works,SA-ITL improves the accuracy of all datasets.Ablation studies present the accuracy enhancements of the SA-ITL,including the hybrid selective algorithm(1.22%-3.82%),transferability enhancement algorithm(1.91%-4.15%),and incremental transfer learning algorithm(0.605%-2.68%).These also show the benefits of enhancing the target model with heterogeneous image datasets that widen the range of domain selection between source and target domains.
