Predicting the productivity of multistage fractured horizontal wells plays an important role in exploiting unconventional resources.In recent years,machine learning(ML)models have emerged as a new approach for such st...Predicting the productivity of multistage fractured horizontal wells plays an important role in exploiting unconventional resources.In recent years,machine learning(ML)models have emerged as a new approach for such studies.However,the scarcity of sufficient real data for model training often leads to imprecise predictions,even though the models trained with real data better characterize geological and engineering features.To tackle this issue,we propose an ML model that can obtain reliable results even with a small amount of data samples.Our model integrates the synthetic minority oversampling technique(SMOTE)to expand the data volume,the support vector machine(SVM)for model training,and the particle swarm optimization(PSO)algorithm for optimizing hyperparameters.To enhance the model performance,we conduct feature fusion and dimensionality reduction.Additionally,we examine the influences of different sample sizes and ML models for training.The proposed model demonstrates higher prediction accuracy and generalization ability,achieving a predicted R^(2)value of up to 0.9 for the test set,compared to the traditional ML techniques with an R^(2)of 0.13.This model accurately predicts the production of fractured horizontal wells even with limited samples,supplying an efficient tool for optimizing the production of unconventional resources.Importantly,the model holds the potential applicability to address similar challenges in other fields constrained by scarce data samples.展开更多
Full ceramic bearings are mission-critical components in oil-free environments,such as food processing,semiconductor manufacturing,and medical applications.Developing effective fault diagnosis methods for these bearin...Full ceramic bearings are mission-critical components in oil-free environments,such as food processing,semiconductor manufacturing,and medical applications.Developing effective fault diagnosis methods for these bearings is essential to ensuring operational reliability and preventing costly failures.Traditional supervised deep learning approaches have demonstrated promise in fault detection,but their dependence on large labeled datasets poses significant challenges in industrial settings where fault-labeled data is scarce.This paper introduces a few-shot learning approach for full ceramic bearing fault diagnosis by leveraging the pre-trained GPT-2 model.Large language models(LLMs)like GPT-2,pre-trained on diverse textual data,exhibit remarkable transfer learning and few-shot learning capabilities,making them ideal for applications with limited labeled data.In this study,acoustic emission(AE)signals from bearings were processed using empirical mode decomposition(EMD),and the extracted AE features were converted into structured text for fine-tuning GPT-2 as a fault classifier.To enhance its performance,we incorporated a modified loss function and softmax activation with cosine similarity,ensuring better generalization in fault identification.Experimental evaluations on a laboratory-collected full ceramic bearing dataset demonstrated that the proposed approach achieved high diagnostic accuracy with as few as five labeled samples,outperforming conventional methods such as k-nearest neighbor(KNN),large memory storage and retrieval(LAMSTAR)neural network,deep neural network(DNN),recurrent neural network(RNN),long short-term memory(LSTM)network,and model-agnostic meta-learning(MAML).The results highlight LLMs’potential to revolutionize fault diagnosis,enabling faster deployment,reduced reliance on extensive labeled datasets,and improved adaptability in industrial monitoring systems.展开更多
Air quality estimation assesses the pollution level in the air,supports public health warnings,and is a valuable tool in environmental management.Although air sensors have proven helpful in this task,sensors are often...Air quality estimation assesses the pollution level in the air,supports public health warnings,and is a valuable tool in environmental management.Although air sensors have proven helpful in this task,sensors are often expensive and difficult to install,while cameras are becoming more popular and accessible,from which images can be collected as data for deep learning models to solve the above task.This leads to another problem:several labeled images are needed to achieve high accuracy when deep-learningmodels predict air quality.In this research,we have threemain contributions:(1)Collect and publish an air quality estimation dataset,namely PTIT_AQED,including environmental image data and air quality;(2)Propose a deep learning model to predict air quality with few data,called PTIT_FAQE(PTIT Few-shot air quality estimation).We build PTIT_FAQE based on EfficientNet-a CNN architecture that ensures high performance in deep learning applications and Few-shot Learning with Prototypical Networks.This helps the model use only a fewtraining data but still achieve high accuracy in air quality estimation.And(3)conduct experiments to prove the superiority of PTIT_FAQE compared to other studies on both PTIT_AQED and APIN datasets.The results show that our model achieves an accuracy of 0.9278 and an F1-Score of 0.9139 on the PTIT_AQED dataset and an accuracy of 0.9467 and an F1-Score of 0.9371 on the APIN dataset,which demonstrate a significant performance improvement compared to previous studies.We also conduct detailed experiments to evaluate the impact of each component on model performance.展开更多
Hydrogen generation and related energy applications heavily rely on the hydrogen evolution reaction(HER),which faces challenges of slow kinetics and high overpotential.Efficient electrocatalysts,particularly single-at...Hydrogen generation and related energy applications heavily rely on the hydrogen evolution reaction(HER),which faces challenges of slow kinetics and high overpotential.Efficient electrocatalysts,particularly single-atom catalysts (SACs) on two-dimensional (2D) materials,are essential.This study presents a few-shot machine learning (ML) assisted high-throughput screening of 2D septuple-atomic-layer Ga_(2)CoS_(4-x)supported SACs to predict HER catalytic activity.Initially,density functional theory (DFT)calculations showed that 2D Ga_(2)CoS4is inactive for HER.However,defective Ga_(2)CoS_(4-x)(x=0–0.25)monolayers exhibit excellent HER activity due to surface sulfur vacancies (SVs),with predicted overpotentials (0–60 mV) comparable to or lower than commercial Pt/C,which typically exhibits an overpotential of around 50 m V in the acidic electrolyte,when the concentration of surface SV is lower than 8.3%.SVs generate spin-polarized states near the Fermi level,making them effective HER sites.We demonstrate ML-accelerated HER overpotential predictions for all transition metal SACs on 2D Ga_(2)CoS_(4-x).Using DFT data from 18 SACs,an ML model with high prediction accuracy and reduced computation time was developed.An intrinsic descriptor linking SAC atomic properties to HER overpotential was identified.This study thus provides a framework for screening SACs on 2D materials,enhancing catalyst design.展开更多
Due to the limited computational capability and the diversity of the Internet of Things devices working in different environment,we consider fewshot learning-based automatic modulation classification(AMC)to improve it...Due to the limited computational capability and the diversity of the Internet of Things devices working in different environment,we consider fewshot learning-based automatic modulation classification(AMC)to improve its reliability.A data enhancement module(DEM)is designed by a convolutional layer to supplement frequency-domain information as well as providing nonlinear mapping that is beneficial for AMC.Multimodal network is designed to have multiple residual blocks,where each residual block has multiple convolutional kernels of different sizes for diverse feature extraction.Moreover,a deep supervised loss function is designed to supervise all parts of the network including the hidden layers and the DEM.Since different model may output different results,cooperative classifier is designed to avoid the randomness of single model and improve the reliability.Simulation results show that this few-shot learning-based AMC method can significantly improve the AMC accuracy compared to the existing methods.展开更多
Intrusion Detection Systems(IDSs)have a great interest these days to discover complex attack events and protect the critical infrastructures of the Internet of Things(IoT)networks.Existing IDSs based on shallow and de...Intrusion Detection Systems(IDSs)have a great interest these days to discover complex attack events and protect the critical infrastructures of the Internet of Things(IoT)networks.Existing IDSs based on shallow and deep network architectures demand high computational resources and high volumes of data to establish an adaptive detection engine that discovers new families of attacks from the edge of IoT networks.However,attackers exploit network gateways at the edge using new attacking scenarios(i.e.,zero-day attacks),such as ransomware and Distributed Denial of Service(DDoS)attacks.This paper proposes new IDS based on Few-Shot Deep Learning,named CNN-IDS,which can automatically identify zero-day attacks from the edge of a network and protect its IoT systems.The proposed system comprises two-methodological stages:1)a filtered Information Gain method is to select the most useful features from network data,and 2)one-dimensional Convolutional Neural Network(CNN)algorithm is to recognize new attack types from a network’s edge.The proposed model is trained and validated using two datasets of the UNSW-NB15 and Bot-IoT.The experimental results showed that it enhances about a 3%detection rate and around a 3%–4%falsepositive rate with the UNSW-NB15 dataset and about an 8%detection rate using the BoT-IoT dataset.展开更多
Objective To develop a few-shot learning(FSL) approach for classifying optical coherence tomography(OCT) images in patients with inherited retinal disorders(IRDs).Methods In this study, an FSL model based on a student...Objective To develop a few-shot learning(FSL) approach for classifying optical coherence tomography(OCT) images in patients with inherited retinal disorders(IRDs).Methods In this study, an FSL model based on a student–teacher learning framework was designed to classify images. 2,317 images from 189 participants were included. Of these, 1,126 images revealed IRDs, 533 were normal samples, and 658 were control samples.Results The FSL model achieved a total accuracy of 0.974–0.983, total sensitivity of 0.934–0.957, total specificity of 0.984–0.990, and total F1 score of 0.935–0.957, which were superior to the total accuracy of the baseline model of 0.943–0.954, total sensitivity of 0.866–0.886, total specificity of 0.962–0.971,and total F1 score of 0.859–0.885. The performance of most subclassifications also exhibited advantages. Moreover, the FSL model had a higher area under curves(AUC) of the receiver operating characteristic(ROC) curves in most subclassifications.Conclusion This study demonstrates the effective use of the FSL model for the classification of OCT images from patients with IRDs, normal, and control participants with a smaller volume of data. The general principle and similar network architectures can also be applied to other retinal diseases with a low prevalence.展开更多
Few-shot learning is becoming more and more popular in many fields,especially in the computer vision field.This inspires us to introduce few-shot learning to the genomic field,which faces a typical few-shot problem be...Few-shot learning is becoming more and more popular in many fields,especially in the computer vision field.This inspires us to introduce few-shot learning to the genomic field,which faces a typical few-shot problem because some tasks only have a limited number of samples with high-dimensions.The goal of this study was to investigate the few-shot disease sub-type prediction problem and identify patient subgroups through training on small data.Accurate disease subtype classification allows clinicians to efficiently deliver investigations and interventions in clinical practice.We propose the SW-Net,which simulates the clinical process of extracting the shared knowledge from a range of interrelated tasks and generalizes it to unseen data.Our model is built upon a simple baseline,and we modified it for genomic data.Supportbased initialization for the classifier and transductive fine-tuning techniques were applied in our model to improve prediction accuracy,and an Entropy regularization term on the query set was appended to reduce over-fitting.Moreover,to address the high dimension and high noise issue,we future extended a feature selection module to adaptively select important features and a sample weighting module to prioritize high-confidence samples.Experiments on simulated data and The Cancer Genome Atlas meta-dataset show that our new baseline model gets higher prediction accuracy compared to other competing algorithms.展开更多
At present,deep learning has been well applied in many fields.However,due to the high complexity of hypothesis space,numerous training samples are usually required to ensure the reliability of minimizing experience ri...At present,deep learning has been well applied in many fields.However,due to the high complexity of hypothesis space,numerous training samples are usually required to ensure the reliability of minimizing experience risk.Therefore,training a classifier with a small number of training examples is a challenging task.From a biological point of view,based on the assumption that rich prior knowledge and analogical association should enable human beings to quickly distinguish novel things from a few or even one example,we proposed a dynamic analogical association algorithm to make the model use only a few labeled samples for classification.To be specific,the algorithm search for knowledge structures similar to existing tasks in prior knowledge based on manifold matching,and combine sampling distributions to generate offsets instead of two sample points,thereby ensuring high confidence and significant contribution to the classification.The comparative results on two common benchmark datasets substantiate the superiority of the proposed method compared to existing data generation approaches for few-shot learning,and the effectiveness of the algorithm has been proved through ablation experiments.展开更多
Numerous meta-learning methods focus on the few-shot learning issue,yet most of them assume that various tasks have a shared embedding space,so the generalization ability of the trained model is limited.In order to so...Numerous meta-learning methods focus on the few-shot learning issue,yet most of them assume that various tasks have a shared embedding space,so the generalization ability of the trained model is limited.In order to solve the aforementioned problem,a task-adaptive meta-learning method based on graph neural network(TAGN) is proposed in this paper,where the characterization ability of the original feature extraction network is ameliorated and the classification accuracy is remarkably improved.Firstly,a task-adaptation module based on the self-attention mechanism is employed,where the generalization ability of the model is enhanced on the new task.Secondly,images are classified in non-Euclidean domain,where the disadvantages of poor adaptability of the traditional distance function are overcome.A large number of experiments are conducted and the results show that the proposed methodology has a better performance than traditional task-independent classification methods on two real-word datasets.展开更多
Recent advances in OCR show that end-to-end(E2E)training pipelines including detection and identification can achieve the best results.However,many existing methods usually focus on case insensitive English characters...Recent advances in OCR show that end-to-end(E2E)training pipelines including detection and identification can achieve the best results.However,many existing methods usually focus on case insensitive English characters.In this paper,we apply an E2E approach,the multiplex multilingual mask TextSpotter,which performs script recognition at the word level and uses different recognition headers to process different scripts while maintaining uniform loss,thus optimizing script recognition and multiple recognition headers simultaneously.Experiments show that this method is superior to the single-head model with similar number of parameters in endto-end identification tasks.展开更多
We propose to use a Few-Shot Learning(FSL)method for the pre-stack seismic inversion problem in obtaining a high resolution reservoir model from recorded seismic data.Recently,artificial neural network(ANN)demonstrate...We propose to use a Few-Shot Learning(FSL)method for the pre-stack seismic inversion problem in obtaining a high resolution reservoir model from recorded seismic data.Recently,artificial neural network(ANN)demonstrates great advantages for seismic inversion because of its powerful feature extraction and parameter learning ability.Hence,ANN method could provide a high resolution inversion result that are critical for reservoir characterization.However,the ANN approach requires plenty of labeled samples for training in order to obtain a satisfactory result.For the common problem of scarce samples in the ANN seismic inversion,we create a novel pre-stack seismic inversion method that takes advantage of the FSL.The results of conventional inversion are used as the auxiliary dataset for ANN based on FSL,while the well log is regarded the scarce training dataset.According to the characteristics of seismic inversion(large amount and high dimensional),we construct an arch network(A-Net)architecture to implement this method.An example shows that this method can improve the accuracy and resolution of inversion results.展开更多
Few-shot learning attempts to identify novel categories by exploiting limited labeled training data,while the performances of existing methods still have much room for improvement.Thanks to a very low cost,many recent...Few-shot learning attempts to identify novel categories by exploiting limited labeled training data,while the performances of existing methods still have much room for improvement.Thanks to a very low cost,many recent methods resort to additional unlabeled training data to boost performance,known as semi-supervised few-shot learning(SSFSL).The general idea of SSFSL methods is to first generate pseudo labels for all unlabeled data and then augment the labeled training set with selected pseudo-labeled data.However,almost all previous SSFSL methods only take supervision signal from pseudo-labeling,ignoring that the distribution of training data can also be utilized as an effective unsupervised regularization.In this paper,we propose a simple yet effective SSFSL method named feature reconstruction based regression method(TENET),which takes low-rank feature reconstruction as the unsupervised objective function and pseudo labels as the supervised constraint.We provide several theoretical insights on why TENET can mitigate overfitting on low-quality training data,and why it can enhance the robustness against inaccurate pseudo labels.Extensive experiments on four popular datasets validate the effectiveness of TENET.展开更多
This paper presents a novel approach for tire-pattern classification,aimed at conducting forensic analysis on tire marks discovered at crime scenes.The classification model proposed in this study accounts for the intr...This paper presents a novel approach for tire-pattern classification,aimed at conducting forensic analysis on tire marks discovered at crime scenes.The classification model proposed in this study accounts for the intricate and dynamic nature of tire prints found in real-world scenarios,including accident sites.To address this complexity,the classifier model was developed to harness the meta-learning capabilities of few-shot learning algorithms(learning-to-learn).The model is meticulously designed and optimized to effectively classify both tire patterns exhibited on wheels and tire-indentation marks visible on surfaces due to friction.This is achieved by employing a semantic segmentation model to extract the tire pattern marks within the image.These marks are subsequently used as a mask channel,combined with the original image,and fed into the classifier to perform classification.Overall,The proposed model follows a three-step process:(i)the Bilateral Segmentation Network is employed to derive the semantic segmentation of the tire pattern within a given image.(ii)utilizing the semantic image in conjunction with the original image,the model learns and clusters groups to generate vectors that define the relative position of the image in the test set.(iii)the model performs predictions based on these learned features.Empirical verification demonstrates usage of semantic model to extract the tire patterns before performing classification increases the overall accuracy of classification by∼4%.展开更多
Recent advancements in deep learning have led to significant breakthroughs across various fields. However, these methods often require extensive labeled data for optimal performance, posing challenges and high costs i...Recent advancements in deep learning have led to significant breakthroughs across various fields. However, these methods often require extensive labeled data for optimal performance, posing challenges and high costs in practical applications. Addressing this issue, Few-Shot Learning (FSL) is introduced. FSL aims to learn effectively from limited labeled samples and generalize well during testing. This paper provides a comprehensive survey of FSL, reviewing prominent deep learning based approaches of FSL. We define FSL through literature review in machine learning and specify the “N-way K-shot” paradigm to distinguish it from related learning challenges. Next, we classify FSL methods by analyzing the Vapnik−Chervonenkis dimension of neural networks. It underscores the necessity for models with abundant labeled examples and finite hypothesis space to generalize well to new and unseen instances. We categorize FSL methods into three types based on strategies to increase labeled samples or reduce hypothesis space: data augmentation, model-based methods, and algorithm-optimized approaches. Using this taxonomy, we review various methods and evaluate their strengths and weaknesses. We also present a comparison of these techniques as summarized in this paper, using benchmark datasets. Moreover, we delve into specific sub-tasks within FSL, such as applications in computer vision and robotics. Lastly, we examine the limitations, unique challenges, and future directions of FSL, aiming to offer a thorough understanding of this rapidly evolving field.展开更多
Although conventional object detection methods achieve high accuracy through extensively annotated datasets,acquiring such large-scale labeled data remains challenging and cost-prohibitive in numerous real-world appli...Although conventional object detection methods achieve high accuracy through extensively annotated datasets,acquiring such large-scale labeled data remains challenging and cost-prohibitive in numerous real-world applications.Few-shot object detection presents a new research idea that aims to localize and classify objects in images using only limited annotated examples.However,the inherent challenge in few-shot object detection lies in the insufficient sample diversity to fully characterize the sample feature distribution,which consequently impacts model performance.Inspired by contrastive learning principles,we propose an Implicit Feature Contrastive Learning(IFCL)module to address this limitation and augment feature diversity for more robust representational learning.This module generates augmented support sample features in a mixed feature space and implicitly contrasts them with query Region of Interest(RoI)features.This approach facilitates more comprehensive learning of both intra-class feature similarity and inter-class feature diversity,thereby enhancing the model’s object classification and localization capabilities.Extensive experiments on PASCAL VOC show that our method achieves a respective improvement of 3.2%,1.8%,and 2.3%on 10-shot of three Novel Sets compared to the baseline model FPD.展开更多
Few-shot learning is a challenging task that aims to train a classifier with very limited training samples.Most existing few-shot learning methods mainly focus on mining knowledge from limited training samples as much...Few-shot learning is a challenging task that aims to train a classifier with very limited training samples.Most existing few-shot learning methods mainly focus on mining knowledge from limited training samples as much as possible and ignore the learning order.Inspired by human learning,people select useful knowledge and follow a learning path to enhance their learning ability.In this paper.we propose a novel few-shot learning model called dynamic knowledge path learning(DKPL)to guide the few-shot learning task to learn useful selected knowledge with suitable learning paths.Specifically,we simultaneously consider the importance,direction,and diversity of knowledge and propose a dynamic path learning strategy in the dynamic path construction module.Furthermore,we design a new learner to absorb knowledge,step by step,according to each class’s learning path in the knowledge path propagation module.As far as we know,this is the first few-shot learning work to consider dynamic path learning to improve classification accuracy.Experiments and visual case studies demonstrate the effectiveness and superiority of the DKPL model on four real-world image datasets.展开更多
Few-shot point cloud 3D object detection(FS3D)aims to identify and locate objects of novel classes within point clouds using knowledge acquired from annotated base classes and a minimal number of samples from the nove...Few-shot point cloud 3D object detection(FS3D)aims to identify and locate objects of novel classes within point clouds using knowledge acquired from annotated base classes and a minimal number of samples from the novel classes.Due to imbalanced training data,existing FS3D methods based on fully supervised learning can lead to overfitting toward base classes,which impairs the network’s ability to generalize knowledge learned from base classes to novel classes and also prevents the network from extracting distinctive foreground and background representations for novel class objects.To address these issues,this thesis proposes a category-agnostic contrastive learning approach,enhancing the generalization and identification abilities for almost unseen categories through the construction of pseudo-labels and positive-negative sample pairs unrelated to specific classes.Firstly,this thesis designs a proposal-wise context contrastive module(CCM).By reducing the distance between foreground point features and increasing the distance between foreground and background point features within a region proposal,CCM aids the network in extracting more discriminative foreground and background feature representations without reliance on categorical annotations.Secondly,this thesis utilizes a geometric contrastive module(GCM),which enhances the network’s geometric perception capability by employing contrastive learning on the foreground point features associated with various basic geometric components,such as edges,corners,and surfaces,thereby enabling these geometric components to exhibit more distinguishable representations.This thesis also combines category-aware contrastive learning with former modules to maintain categorical distinctiveness.Extensive experimental results on FS-SUNRGBD and FS-ScanNet datasets demonstrate the effectiveness of this method with average precision exceeding the baseline by up to 8%.展开更多
Image classification is crucial for various applications,including digital construction,smart manu-facturing,and medical imaging.Focusing on the inadequate model generalization and data privacy concerns in few-shot im...Image classification is crucial for various applications,including digital construction,smart manu-facturing,and medical imaging.Focusing on the inadequate model generalization and data privacy concerns in few-shot image classification,in this paper,we propose a federated learning approach that incorporates privacy-preserving techniques.First,we utilize contrastive learning to train on local few-shot image data and apply various data augmentation methods to expand the sample size,thereby enhancing the model’s generalization capabilities in few-shot contexts.Second,we introduce local differential privacy techniques and weight pruning methods to safeguard model parameters,perturbing the transmitted parameters to ensure user data privacy.Finally,numerical simulations are conducted to demonstrate the effectiveness of our proposed method.The results indicate that our approach significantly enhances model generalization and test accuracy compared to several popular federated learning algorithms while maintaining data privacy,highlighting its effectiveness and practicality in addressing the challenges of model generalization and data privacy in few-shot image scenarios.展开更多
The distributed fiber optic sensing system,known for its high sensitivity and wide-ranging measurement capabilities,has been widely used in monitoring underground gas pipelines.It primarily serves to perceive vibratio...The distributed fiber optic sensing system,known for its high sensitivity and wide-ranging measurement capabilities,has been widely used in monitoring underground gas pipelines.It primarily serves to perceive vibration signals induced by external events and to effectively provide early warnings of potential intrusion activities.Due to the complexity and diversity of external intrusion events,traditional deep learning methods can achieve event recognition with an average accuracy exceeding 90%.However,these methods rely on large-scale datasets,leading to significant time and labor costs during the data collection process.Additionally,traditional methods perform poorly when faced with the scarcity of low-frequency event samples,making it challenging to address these rare occurrences.To address this issue,this paper proposes a small-sample learning model based on triplet learning for intrusion event recognition.The model employs a 6-way 20-shot support set configuration and utilizes the KNN clustering algorithm to assess the model's performance.Experimental results indicate that the model achieves an average accuracy of 91.6%,further validating the superior performance of the triplet learning model in classifying external intrusion events.Compared to traditional methods,this approach not only effectively reduces the dependence on large-scale datasets but also better addresses the classification of low-frequency event samples,demonstrating significant application potential.展开更多
基金supported by the National Natural Science Foundation of China(52274055)the Shandong Provincial Natural Science Foundation(ZR2022YQ50)the Taishan Scholar Program of Shandong Province(tsqn202408088)。
文摘Predicting the productivity of multistage fractured horizontal wells plays an important role in exploiting unconventional resources.In recent years,machine learning(ML)models have emerged as a new approach for such studies.However,the scarcity of sufficient real data for model training often leads to imprecise predictions,even though the models trained with real data better characterize geological and engineering features.To tackle this issue,we propose an ML model that can obtain reliable results even with a small amount of data samples.Our model integrates the synthetic minority oversampling technique(SMOTE)to expand the data volume,the support vector machine(SVM)for model training,and the particle swarm optimization(PSO)algorithm for optimizing hyperparameters.To enhance the model performance,we conduct feature fusion and dimensionality reduction.Additionally,we examine the influences of different sample sizes and ML models for training.The proposed model demonstrates higher prediction accuracy and generalization ability,achieving a predicted R^(2)value of up to 0.9 for the test set,compared to the traditional ML techniques with an R^(2)of 0.13.This model accurately predicts the production of fractured horizontal wells even with limited samples,supplying an efficient tool for optimizing the production of unconventional resources.Importantly,the model holds the potential applicability to address similar challenges in other fields constrained by scarce data samples.
文摘Full ceramic bearings are mission-critical components in oil-free environments,such as food processing,semiconductor manufacturing,and medical applications.Developing effective fault diagnosis methods for these bearings is essential to ensuring operational reliability and preventing costly failures.Traditional supervised deep learning approaches have demonstrated promise in fault detection,but their dependence on large labeled datasets poses significant challenges in industrial settings where fault-labeled data is scarce.This paper introduces a few-shot learning approach for full ceramic bearing fault diagnosis by leveraging the pre-trained GPT-2 model.Large language models(LLMs)like GPT-2,pre-trained on diverse textual data,exhibit remarkable transfer learning and few-shot learning capabilities,making them ideal for applications with limited labeled data.In this study,acoustic emission(AE)signals from bearings were processed using empirical mode decomposition(EMD),and the extracted AE features were converted into structured text for fine-tuning GPT-2 as a fault classifier.To enhance its performance,we incorporated a modified loss function and softmax activation with cosine similarity,ensuring better generalization in fault identification.Experimental evaluations on a laboratory-collected full ceramic bearing dataset demonstrated that the proposed approach achieved high diagnostic accuracy with as few as five labeled samples,outperforming conventional methods such as k-nearest neighbor(KNN),large memory storage and retrieval(LAMSTAR)neural network,deep neural network(DNN),recurrent neural network(RNN),long short-term memory(LSTM)network,and model-agnostic meta-learning(MAML).The results highlight LLMs’potential to revolutionize fault diagnosis,enabling faster deployment,reduced reliance on extensive labeled datasets,and improved adaptability in industrial monitoring systems.
文摘Air quality estimation assesses the pollution level in the air,supports public health warnings,and is a valuable tool in environmental management.Although air sensors have proven helpful in this task,sensors are often expensive and difficult to install,while cameras are becoming more popular and accessible,from which images can be collected as data for deep learning models to solve the above task.This leads to another problem:several labeled images are needed to achieve high accuracy when deep-learningmodels predict air quality.In this research,we have threemain contributions:(1)Collect and publish an air quality estimation dataset,namely PTIT_AQED,including environmental image data and air quality;(2)Propose a deep learning model to predict air quality with few data,called PTIT_FAQE(PTIT Few-shot air quality estimation).We build PTIT_FAQE based on EfficientNet-a CNN architecture that ensures high performance in deep learning applications and Few-shot Learning with Prototypical Networks.This helps the model use only a fewtraining data but still achieve high accuracy in air quality estimation.And(3)conduct experiments to prove the superiority of PTIT_FAQE compared to other studies on both PTIT_AQED and APIN datasets.The results show that our model achieves an accuracy of 0.9278 and an F1-Score of 0.9139 on the PTIT_AQED dataset and an accuracy of 0.9467 and an F1-Score of 0.9371 on the APIN dataset,which demonstrate a significant performance improvement compared to previous studies.We also conduct detailed experiments to evaluate the impact of each component on model performance.
文摘Hydrogen generation and related energy applications heavily rely on the hydrogen evolution reaction(HER),which faces challenges of slow kinetics and high overpotential.Efficient electrocatalysts,particularly single-atom catalysts (SACs) on two-dimensional (2D) materials,are essential.This study presents a few-shot machine learning (ML) assisted high-throughput screening of 2D septuple-atomic-layer Ga_(2)CoS_(4-x)supported SACs to predict HER catalytic activity.Initially,density functional theory (DFT)calculations showed that 2D Ga_(2)CoS4is inactive for HER.However,defective Ga_(2)CoS_(4-x)(x=0–0.25)monolayers exhibit excellent HER activity due to surface sulfur vacancies (SVs),with predicted overpotentials (0–60 mV) comparable to or lower than commercial Pt/C,which typically exhibits an overpotential of around 50 m V in the acidic electrolyte,when the concentration of surface SV is lower than 8.3%.SVs generate spin-polarized states near the Fermi level,making them effective HER sites.We demonstrate ML-accelerated HER overpotential predictions for all transition metal SACs on 2D Ga_(2)CoS_(4-x).Using DFT data from 18 SACs,an ML model with high prediction accuracy and reduced computation time was developed.An intrinsic descriptor linking SAC atomic properties to HER overpotential was identified.This study thus provides a framework for screening SACs on 2D materials,enhancing catalyst design.
基金supported in part by National Key Research and Development Program of China under Grant 2021YFB2900404.
文摘Due to the limited computational capability and the diversity of the Internet of Things devices working in different environment,we consider fewshot learning-based automatic modulation classification(AMC)to improve its reliability.A data enhancement module(DEM)is designed by a convolutional layer to supplement frequency-domain information as well as providing nonlinear mapping that is beneficial for AMC.Multimodal network is designed to have multiple residual blocks,where each residual block has multiple convolutional kernels of different sizes for diverse feature extraction.Moreover,a deep supervised loss function is designed to supervise all parts of the network including the hidden layers and the DEM.Since different model may output different results,cooperative classifier is designed to avoid the randomness of single model and improve the reliability.Simulation results show that this few-shot learning-based AMC method can significantly improve the AMC accuracy compared to the existing methods.
基金This work has been supported by the Australian Research Data Common(ARDC),project code–RG192500.
文摘Intrusion Detection Systems(IDSs)have a great interest these days to discover complex attack events and protect the critical infrastructures of the Internet of Things(IoT)networks.Existing IDSs based on shallow and deep network architectures demand high computational resources and high volumes of data to establish an adaptive detection engine that discovers new families of attacks from the edge of IoT networks.However,attackers exploit network gateways at the edge using new attacking scenarios(i.e.,zero-day attacks),such as ransomware and Distributed Denial of Service(DDoS)attacks.This paper proposes new IDS based on Few-Shot Deep Learning,named CNN-IDS,which can automatically identify zero-day attacks from the edge of a network and protect its IoT systems.The proposed system comprises two-methodological stages:1)a filtered Information Gain method is to select the most useful features from network data,and 2)one-dimensional Convolutional Neural Network(CNN)algorithm is to recognize new attack types from a network’s edge.The proposed model is trained and validated using two datasets of the UNSW-NB15 and Bot-IoT.The experimental results showed that it enhances about a 3%detection rate and around a 3%–4%falsepositive rate with the UNSW-NB15 dataset and about an 8%detection rate using the BoT-IoT dataset.
基金supported by National Natural Science Foundation of China [No.82171073]。
文摘Objective To develop a few-shot learning(FSL) approach for classifying optical coherence tomography(OCT) images in patients with inherited retinal disorders(IRDs).Methods In this study, an FSL model based on a student–teacher learning framework was designed to classify images. 2,317 images from 189 participants were included. Of these, 1,126 images revealed IRDs, 533 were normal samples, and 658 were control samples.Results The FSL model achieved a total accuracy of 0.974–0.983, total sensitivity of 0.934–0.957, total specificity of 0.984–0.990, and total F1 score of 0.935–0.957, which were superior to the total accuracy of the baseline model of 0.943–0.954, total sensitivity of 0.866–0.886, total specificity of 0.962–0.971,and total F1 score of 0.859–0.885. The performance of most subclassifications also exhibited advantages. Moreover, the FSL model had a higher area under curves(AUC) of the receiver operating characteristic(ROC) curves in most subclassifications.Conclusion This study demonstrates the effective use of the FSL model for the classification of OCT images from patients with IRDs, normal, and control participants with a smaller volume of data. The general principle and similar network architectures can also be applied to other retinal diseases with a low prevalence.
基金supported by the Macao Science and Technology Development Funds Grands No.0158/2019/A3 from the Macao Special Administrative Region of the People’s Republic of China.
文摘Few-shot learning is becoming more and more popular in many fields,especially in the computer vision field.This inspires us to introduce few-shot learning to the genomic field,which faces a typical few-shot problem because some tasks only have a limited number of samples with high-dimensions.The goal of this study was to investigate the few-shot disease sub-type prediction problem and identify patient subgroups through training on small data.Accurate disease subtype classification allows clinicians to efficiently deliver investigations and interventions in clinical practice.We propose the SW-Net,which simulates the clinical process of extracting the shared knowledge from a range of interrelated tasks and generalizes it to unseen data.Our model is built upon a simple baseline,and we modified it for genomic data.Supportbased initialization for the classifier and transductive fine-tuning techniques were applied in our model to improve prediction accuracy,and an Entropy regularization term on the query set was appended to reduce over-fitting.Moreover,to address the high dimension and high noise issue,we future extended a feature selection module to adaptively select important features and a sample weighting module to prioritize high-confidence samples.Experiments on simulated data and The Cancer Genome Atlas meta-dataset show that our new baseline model gets higher prediction accuracy compared to other competing algorithms.
基金This work was supported by The National Natural Science Foundation of China(No.61402537)Sichuan Science and Technology Program(Nos.2019ZDZX0006,2020YFQ0056)+1 种基金the West Light Foundation of Chinese Academy of Sciences(201899)the Talents by Sichuan provincial Party Committee Organization Department,and Science and Technology Service Network Initiative(KFJ-STS-QYZD-2021-21-001).
文摘At present,deep learning has been well applied in many fields.However,due to the high complexity of hypothesis space,numerous training samples are usually required to ensure the reliability of minimizing experience risk.Therefore,training a classifier with a small number of training examples is a challenging task.From a biological point of view,based on the assumption that rich prior knowledge and analogical association should enable human beings to quickly distinguish novel things from a few or even one example,we proposed a dynamic analogical association algorithm to make the model use only a few labeled samples for classification.To be specific,the algorithm search for knowledge structures similar to existing tasks in prior knowledge based on manifold matching,and combine sampling distributions to generate offsets instead of two sample points,thereby ensuring high confidence and significant contribution to the classification.The comparative results on two common benchmark datasets substantiate the superiority of the proposed method compared to existing data generation approaches for few-shot learning,and the effectiveness of the algorithm has been proved through ablation experiments.
基金Supported by the National High Technology Research and Development Program of China(20-H863-05-XXX-XX)the National Natural Science Foundation of China(61171131)+1 种基金Shandong Province Key Research and Development Program(YD01033)the China Scholarship Council Program(201608370049)。
文摘Numerous meta-learning methods focus on the few-shot learning issue,yet most of them assume that various tasks have a shared embedding space,so the generalization ability of the trained model is limited.In order to solve the aforementioned problem,a task-adaptive meta-learning method based on graph neural network(TAGN) is proposed in this paper,where the characterization ability of the original feature extraction network is ameliorated and the classification accuracy is remarkably improved.Firstly,a task-adaptation module based on the self-attention mechanism is employed,where the generalization ability of the model is enhanced on the new task.Secondly,images are classified in non-Euclidean domain,where the disadvantages of poor adaptability of the traditional distance function are overcome.A large number of experiments are conducted and the results show that the proposed methodology has a better performance than traditional task-independent classification methods on two real-word datasets.
基金supported by the Advanced Training Project of the Professional Leaders in Jiangsu Higher Vocational Colleges (2020GRFX006).
文摘Recent advances in OCR show that end-to-end(E2E)training pipelines including detection and identification can achieve the best results.However,many existing methods usually focus on case insensitive English characters.In this paper,we apply an E2E approach,the multiplex multilingual mask TextSpotter,which performs script recognition at the word level and uses different recognition headers to process different scripts while maintaining uniform loss,thus optimizing script recognition and multiple recognition headers simultaneously.Experiments show that this method is superior to the single-head model with similar number of parameters in endto-end identification tasks.
基金Wupport from the National Natural Science Foundation of China(Grant No.42130812,42174151,and 41804126).
文摘We propose to use a Few-Shot Learning(FSL)method for the pre-stack seismic inversion problem in obtaining a high resolution reservoir model from recorded seismic data.Recently,artificial neural network(ANN)demonstrates great advantages for seismic inversion because of its powerful feature extraction and parameter learning ability.Hence,ANN method could provide a high resolution inversion result that are critical for reservoir characterization.However,the ANN approach requires plenty of labeled samples for training in order to obtain a satisfactory result.For the common problem of scarce samples in the ANN seismic inversion,we create a novel pre-stack seismic inversion method that takes advantage of the FSL.The results of conventional inversion are used as the auxiliary dataset for ANN based on FSL,while the well log is regarded the scarce training dataset.According to the characteristics of seismic inversion(large amount and high dimensional),we construct an arch network(A-Net)architecture to implement this method.An example shows that this method can improve the accuracy and resolution of inversion results.
基金supported in part by the Beijing Natural Science Foundation,China(No.L221013)in part by the National Science Foundation of China(Nos.U20B2070 and 61832016).
文摘Few-shot learning attempts to identify novel categories by exploiting limited labeled training data,while the performances of existing methods still have much room for improvement.Thanks to a very low cost,many recent methods resort to additional unlabeled training data to boost performance,known as semi-supervised few-shot learning(SSFSL).The general idea of SSFSL methods is to first generate pseudo labels for all unlabeled data and then augment the labeled training set with selected pseudo-labeled data.However,almost all previous SSFSL methods only take supervision signal from pseudo-labeling,ignoring that the distribution of training data can also be utilized as an effective unsupervised regularization.In this paper,we propose a simple yet effective SSFSL method named feature reconstruction based regression method(TENET),which takes low-rank feature reconstruction as the unsupervised objective function and pseudo labels as the supervised constraint.We provide several theoretical insights on why TENET can mitigate overfitting on low-quality training data,and why it can enhance the robustness against inaccurate pseudo labels.Extensive experiments on four popular datasets validate the effectiveness of TENET.
文摘This paper presents a novel approach for tire-pattern classification,aimed at conducting forensic analysis on tire marks discovered at crime scenes.The classification model proposed in this study accounts for the intricate and dynamic nature of tire prints found in real-world scenarios,including accident sites.To address this complexity,the classifier model was developed to harness the meta-learning capabilities of few-shot learning algorithms(learning-to-learn).The model is meticulously designed and optimized to effectively classify both tire patterns exhibited on wheels and tire-indentation marks visible on surfaces due to friction.This is achieved by employing a semantic segmentation model to extract the tire pattern marks within the image.These marks are subsequently used as a mask channel,combined with the original image,and fed into the classifier to perform classification.Overall,The proposed model follows a three-step process:(i)the Bilateral Segmentation Network is employed to derive the semantic segmentation of the tire pattern within a given image.(ii)utilizing the semantic image in conjunction with the original image,the model learns and clusters groups to generate vectors that define the relative position of the image in the test set.(iii)the model performs predictions based on these learned features.Empirical verification demonstrates usage of semantic model to extract the tire patterns before performing classification increases the overall accuracy of classification by∼4%.
基金supported by the Fundamental Research Funds for the Central Universities(No.1082204112364).
文摘Recent advancements in deep learning have led to significant breakthroughs across various fields. However, these methods often require extensive labeled data for optimal performance, posing challenges and high costs in practical applications. Addressing this issue, Few-Shot Learning (FSL) is introduced. FSL aims to learn effectively from limited labeled samples and generalize well during testing. This paper provides a comprehensive survey of FSL, reviewing prominent deep learning based approaches of FSL. We define FSL through literature review in machine learning and specify the “N-way K-shot” paradigm to distinguish it from related learning challenges. Next, we classify FSL methods by analyzing the Vapnik−Chervonenkis dimension of neural networks. It underscores the necessity for models with abundant labeled examples and finite hypothesis space to generalize well to new and unseen instances. We categorize FSL methods into three types based on strategies to increase labeled samples or reduce hypothesis space: data augmentation, model-based methods, and algorithm-optimized approaches. Using this taxonomy, we review various methods and evaluate their strengths and weaknesses. We also present a comparison of these techniques as summarized in this paper, using benchmark datasets. Moreover, we delve into specific sub-tasks within FSL, such as applications in computer vision and robotics. Lastly, we examine the limitations, unique challenges, and future directions of FSL, aiming to offer a thorough understanding of this rapidly evolving field.
基金funded by the China Chongqing Municipal Science and Technology Bureau,grant numbers CSTB2024TIAD-CYKJCXX0009,CSTB2024NSCQ-LZX0043,CSTB2022NSCQ-MSX0288Chongqing Municipal Commission of Housing and Urban-Rural Development,grant number CKZ2024-87+3 种基金the Chongqing University of Technology Graduate Education High-Quality Development Project,grant number gzlsz202401the Chongqing University of Technology—Chongqing LINGLUE Technology Co.,Ltd.Electronic Information(Artificial Intelligence)Graduate Joint Training Basethe Postgraduate Education and Teaching Reform Research Project in Chongqing,grant number yjg213116the Chongqing University of Technology-CISDI Chongqing Information Technology Co.,Ltd.Computer Technology Graduate Joint Training Base.
文摘Although conventional object detection methods achieve high accuracy through extensively annotated datasets,acquiring such large-scale labeled data remains challenging and cost-prohibitive in numerous real-world applications.Few-shot object detection presents a new research idea that aims to localize and classify objects in images using only limited annotated examples.However,the inherent challenge in few-shot object detection lies in the insufficient sample diversity to fully characterize the sample feature distribution,which consequently impacts model performance.Inspired by contrastive learning principles,we propose an Implicit Feature Contrastive Learning(IFCL)module to address this limitation and augment feature diversity for more robust representational learning.This module generates augmented support sample features in a mixed feature space and implicitly contrasts them with query Region of Interest(RoI)features.This approach facilitates more comprehensive learning of both intra-class feature similarity and inter-class feature diversity,thereby enhancing the model’s object classification and localization capabilities.Extensive experiments on PASCAL VOC show that our method achieves a respective improvement of 3.2%,1.8%,and 2.3%on 10-shot of three Novel Sets compared to the baseline model FPD.
基金supported in part by the National Natural Science Foundation of China(No.62227808)the Shenzhen Science and Technology Program(No.ZDSYS20210623091809029).
文摘Few-shot learning is a challenging task that aims to train a classifier with very limited training samples.Most existing few-shot learning methods mainly focus on mining knowledge from limited training samples as much as possible and ignore the learning order.Inspired by human learning,people select useful knowledge and follow a learning path to enhance their learning ability.In this paper.we propose a novel few-shot learning model called dynamic knowledge path learning(DKPL)to guide the few-shot learning task to learn useful selected knowledge with suitable learning paths.Specifically,we simultaneously consider the importance,direction,and diversity of knowledge and propose a dynamic path learning strategy in the dynamic path construction module.Furthermore,we design a new learner to absorb knowledge,step by step,according to each class’s learning path in the knowledge path propagation module.As far as we know,this is the first few-shot learning work to consider dynamic path learning to improve classification accuracy.Experiments and visual case studies demonstrate the effectiveness and superiority of the DKPL model on four real-world image datasets.
文摘Few-shot point cloud 3D object detection(FS3D)aims to identify and locate objects of novel classes within point clouds using knowledge acquired from annotated base classes and a minimal number of samples from the novel classes.Due to imbalanced training data,existing FS3D methods based on fully supervised learning can lead to overfitting toward base classes,which impairs the network’s ability to generalize knowledge learned from base classes to novel classes and also prevents the network from extracting distinctive foreground and background representations for novel class objects.To address these issues,this thesis proposes a category-agnostic contrastive learning approach,enhancing the generalization and identification abilities for almost unseen categories through the construction of pseudo-labels and positive-negative sample pairs unrelated to specific classes.Firstly,this thesis designs a proposal-wise context contrastive module(CCM).By reducing the distance between foreground point features and increasing the distance between foreground and background point features within a region proposal,CCM aids the network in extracting more discriminative foreground and background feature representations without reliance on categorical annotations.Secondly,this thesis utilizes a geometric contrastive module(GCM),which enhances the network’s geometric perception capability by employing contrastive learning on the foreground point features associated with various basic geometric components,such as edges,corners,and surfaces,thereby enabling these geometric components to exhibit more distinguishable representations.This thesis also combines category-aware contrastive learning with former modules to maintain categorical distinctiveness.Extensive experimental results on FS-SUNRGBD and FS-ScanNet datasets demonstrate the effectiveness of this method with average precision exceeding the baseline by up to 8%.
基金supported by Suzhou Science and Technology Plan(Basic Research)Project under Grant SJC2023002Postgraduate Research&Practice Innovation Program of Jiangsu Province under Grant KYCX23_3322.
文摘Image classification is crucial for various applications,including digital construction,smart manu-facturing,and medical imaging.Focusing on the inadequate model generalization and data privacy concerns in few-shot image classification,in this paper,we propose a federated learning approach that incorporates privacy-preserving techniques.First,we utilize contrastive learning to train on local few-shot image data and apply various data augmentation methods to expand the sample size,thereby enhancing the model’s generalization capabilities in few-shot contexts.Second,we introduce local differential privacy techniques and weight pruning methods to safeguard model parameters,perturbing the transmitted parameters to ensure user data privacy.Finally,numerical simulations are conducted to demonstrate the effectiveness of our proposed method.The results indicate that our approach significantly enhances model generalization and test accuracy compared to several popular federated learning algorithms while maintaining data privacy,highlighting its effectiveness and practicality in addressing the challenges of model generalization and data privacy in few-shot image scenarios.
基金Supported by the Scientific Research and Technology Development Project of Petrochina Southwest Oil and Gas Field Company(20230307-02)。
文摘The distributed fiber optic sensing system,known for its high sensitivity and wide-ranging measurement capabilities,has been widely used in monitoring underground gas pipelines.It primarily serves to perceive vibration signals induced by external events and to effectively provide early warnings of potential intrusion activities.Due to the complexity and diversity of external intrusion events,traditional deep learning methods can achieve event recognition with an average accuracy exceeding 90%.However,these methods rely on large-scale datasets,leading to significant time and labor costs during the data collection process.Additionally,traditional methods perform poorly when faced with the scarcity of low-frequency event samples,making it challenging to address these rare occurrences.To address this issue,this paper proposes a small-sample learning model based on triplet learning for intrusion event recognition.The model employs a 6-way 20-shot support set configuration and utilizes the KNN clustering algorithm to assess the model's performance.Experimental results indicate that the model achieves an average accuracy of 91.6%,further validating the superior performance of the triplet learning model in classifying external intrusion events.Compared to traditional methods,this approach not only effectively reduces the dependence on large-scale datasets but also better addresses the classification of low-frequency event samples,demonstrating significant application potential.
