The Reliability-Based Design Optimization(RBDO)of complex engineering structures considering uncertainties has problems of being high-dimensional,highly nonlinear,and timeconsuming,which requires a significant amount ...The Reliability-Based Design Optimization(RBDO)of complex engineering structures considering uncertainties has problems of being high-dimensional,highly nonlinear,and timeconsuming,which requires a significant amount of sampling simulation computation.In this paper,a basis-adaptive Polynomial Chaos(PC)-Kriging surrogate model is proposed,in order to relieve the computational burden and enhance the predictive accuracy of a metamodel.The active learning basis-adaptive PC-Kriging model is combined with a quantile-based RBDO framework.Finally,five engineering cases have been implemented,including a benchmark RBDO problem,three high-dimensional explicit problems,and a high-dimensional implicit problem.Compared with Support Vector Regression(SVR),Kriging,and polynomial chaos expansion models,results show that the proposed basis-adaptive PC-Kriging model is more accurate and efficient for RBDO problems of complex engineering structures.展开更多
In materials science,a significant correlation often exists between material input parameters and their corresponding performance attributes.Nevertheless,the inherent challenges associated with small data obscure thes...In materials science,a significant correlation often exists between material input parameters and their corresponding performance attributes.Nevertheless,the inherent challenges associated with small data obscure these statistical correlations,impeding machine learning models from effectively capturing the underlying patterns,thereby hampering efficient optimization of material properties.This work presents a novel active learning framework that integrates generative adversarial networks(GAN)with a directionally constrained expected absolute improvement(EAI)acquisition function to accelerate the discovery of ultra-high temperature ceramics(UHTCs)using small data.The framework employs GAN for data augmentation,symbolic regression for feature weight derivation,and a self-developed EAI function that incorporates input feature importance weighting to quantify bidirectional deviations from zero ablation rate.Through only two iterations,this framework successfully identified the optimal composition of HfB_(2)-3.52SiC-5.23TaSi_(2),which exhibits robust near-zero ablation rates under plasma ablation at 2500℃ for 200 s,demonstrating superior sampling efficiency compared to conventional active learning approaches.Microstructural analysis reveals that the exceptional performance stems from the formation of a highly viscous HfO_(2)-SiO_(2)-Ta_(2)O_(5)-HfSiO_(4)-Hf_(3)(BO_(3))_(4) oxide layer,which provides effective oxygen barrier protection.This work demonstrates an efficient and universal approach for rapid materials discovery using small data.展开更多
Industrial decarbonization is critical for achieving net-zero goals.The carbon dioxide electrochemical reduction reaction(CO_(2)RR)is a promising approach for converting CO_(2)into high-value chemicals,offering the po...Industrial decarbonization is critical for achieving net-zero goals.The carbon dioxide electrochemical reduction reaction(CO_(2)RR)is a promising approach for converting CO_(2)into high-value chemicals,offering the potential for decarbonizing industrial processes toward a sustainable,carbon-neutral future.However,developing CO_(2)RR catalysts with high selectivity and activity remains a challenge due to the complexity of finding such catalysts and the inefficiency of traditional computational or experimental approaches.Here,we present a methodology integrating density functional theory(DFT)calculations,deep learning models,and an active learning strategy to rapidly screen high-performance catalysts.The proposed methodology is then demonstrated on graphene-based single-atom catalysts for selective CO_(2)electroreduction to methanol.First,we conduct systematic binding energy calculations for 3045 single-atom catalysts to identify thermodynamically stable catalysts as the design space.We then use a graph neural network,fine-tuned with a specialized adsorption energy database,to predict the relative activity and selectivity of the candidate catalysts.An autonomous active learning framework is used to facilitate the exploration of designs.After six learning cycles and 2180 adsorption calculations across 15 intermediates,we develop a surrogate model that identifies four novel catalysts on the Pareto front of activity and selectivity.Our work demonstrates the effectiveness of leveraging a domain foundation model with an active learning framework and holds potential to significantly accelerate the discovery of high-performance CO_(2)RR catalysts.展开更多
Objective:Deep learning(DL)has become the prevailing method in chest radiograph analysis,yet its performance heavily depends on large quantities of annotated images.To mitigate the cost,cold-start active learning(AL),...Objective:Deep learning(DL)has become the prevailing method in chest radiograph analysis,yet its performance heavily depends on large quantities of annotated images.To mitigate the cost,cold-start active learning(AL),comprising an initialization followed by subsequent learning,selects a small subset of informative data points for labeling.Recent advancements in pretrained models by supervised or self-supervised learning tailored to chest radiograph have shown broad applicability to diverse downstream tasks.However,their potential in cold-start AL remains unexplored.Methods:To validate the efficacy of domain-specific pretraining,we compared two foundation models:supervised TXRV and self-supervised REMEDIS with their general domain counterparts pretrained on ImageNet.Model performance was evaluated at both initialization and subsequent learning stages on two diagnostic tasks:psychiatric pneumonia and COVID-19.For initialization,we assessed their integration with three strategies:diversity,uncertainty,and hybrid sampling.For subsequent learning,we focused on uncertainty sampling powered by different pretrained models.We also conducted statistical tests to compare the foundation models with ImageNet counterparts,investigate the relationship between initialization and subsequent learning,examine the performance of one-shot initialization against the full AL process,and investigate the influence of class balance in initialization samples on initialization and subsequent learning.Results:First,domain-specific foundation models failed to outperform ImageNet counterparts in six out of eight experiments on informative sample selection.Both domain-specific and general pretrained models were unable to generate representations that could substitute for the original images as model inputs in seven of the eight scenarios.However,pretrained model-based initialization surpassed random sampling,the default approach in cold-start AL.Second,initialization performance was positively correlated with subsequent learning performance,highlighting the importance of initialization strategies.Third,one-shot initialization performed comparably to the full AL process,demonstrating the potential of reducing experts'repeated waiting during AL iterations.Last,a U-shaped correlation was observed between the class balance of initialization samples and model performance,suggesting that the class balance is more strongly associated with performance at middle budget levels than at low or high budgets.Conclusions:In this study,we highlighted the limitations of medical pretraining compared to general pretraining in the context of cold-start AL.We also identified promising outcomes related to cold-start AL,including initialization based on pretrained models,the positive influence of initialization on subsequent learning,the potential for one-shot initialization,and the influence of class balance on middle-budget AL.Researchers are encouraged to improve medical pretraining for versatile DL foundations and explore novel AL methods.展开更多
Human Activity Recognition(HAR)has become increasingly critical in civic surveillance,medical care monitoring,and institutional protection.Current deep learning-based approaches often suffer from excessive computation...Human Activity Recognition(HAR)has become increasingly critical in civic surveillance,medical care monitoring,and institutional protection.Current deep learning-based approaches often suffer from excessive computational complexity,limited generalizability under varying conditions,and compromised real-time performance.To counter these,this paper introduces an Active Learning-aided Heuristic Deep Spatio-Textural Ensemble Learning(ALH-DSEL)framework.The model initially identifies keyframes from the surveillance videos with a Multi-Constraint Active Learning(MCAL)approach,with features extracted from DenseNet121.The frames are then segmented employing an optimized Fuzzy C-Means clustering algorithm with Firefly to identify areas of interest.A deep ensemble feature extractor,comprising DenseNet121,EfficientNet-B7,MobileNet,and GLCM,extracts varied spatial and textural features.Fused characteristics are enhanced through PCA and Min-Max normalization and discriminated by a maximum voting ensemble of RF,AdaBoost,and XGBoost.The experimental results show that ALH-DSEL provides higher accuracy,precision,recall,and F1-score,validating its superiority for real-time HAR in surveillance scenarios.展开更多
To capture the nonlinear dynamics and gain evolution in chirped pulse amplification(CPA)systems,the split-step Fourier method and the fourth-order Runge–Kutta method are integrated to iteratively address the generali...To capture the nonlinear dynamics and gain evolution in chirped pulse amplification(CPA)systems,the split-step Fourier method and the fourth-order Runge–Kutta method are integrated to iteratively address the generalized nonlinear Schrödinger equation and the rate equations.However,this approach is burdened by substantial computational demands,resulting in significant time expenditures.In the context of intelligent laser optimization and inverse design,the necessity for numerous simulations further exacerbates this issue,highlighting the need for fast and accurate simulation methodologies.Here,we introduce an end-to-end model augmented with active learning(E2E-AL)with decent generalization through different dedicated embedding methods over various parameters.On an identical computational platform,the artificial intelligence–driven model is 2000 times faster than the conventional simulation method.Benefiting from the active learning strategy,the E2E-AL model achieves decent precision with only two-thirds of the training samples compared with the case without such a strategy.Furthermore,we demonstrate a multi-objective inverse design of the CPA systems enabled by the E2E-AL model.The E2E-AL framework manifests the potential of becoming a standard approach for the rapid and accurate modeling of ultrafast lasers and is readily extended to simulate other complex systems.展开更多
Dynamical systems often exhibit multiple attractors representing significantly different functioning conditions.A global map of attraction basins can offer valuable guidance for stabilizing or transitioning system sta...Dynamical systems often exhibit multiple attractors representing significantly different functioning conditions.A global map of attraction basins can offer valuable guidance for stabilizing or transitioning system states.Such a map can be constructed without prior system knowledge by identifying attractors across a sufficient number of points in the state space.However,determining the attractor for each initial state can be a laborious task.Here,we tackle the challenge of reconstructing attraction basins using as few initial points as possible.In each iteration of our approach,informative points are selected through random seeding and are driven along the current classification boundary,promoting the eventual selection of points that are both diverse and enlightening.The results across various experimental dynamical systems demonstrate that our approach requires fewer points than baseline methods while achieving comparable mapping accuracy.Additionally,the reconstructed map allows us to accurately estimate the minimum escape distance required to transition the system state to a target basin.展开更多
Support vector machines（SVMs） are a popular class of supervised learning algorithms, and are particularly applicable to large and high-dimensional classification problems. Like most machine learning methods for data...Support vector machines（SVMs） are a popular class of supervised learning algorithms, and are particularly applicable to large and high-dimensional classification problems. Like most machine learning methods for data classification and information retrieval, they require manually labeled data samples in the training stage. However, manual labeling is a time consuming and errorprone task. One possible solution to this issue is to exploit the large number of unlabeled samples that are easily accessible via the internet. This paper presents a novel active learning method for text categorization. The main objective of active learning is to reduce the labeling effort, without compromising the accuracy of classification, by intelligently selecting which samples should be labeled.The proposed method selects a batch of informative samples using the posterior probabilities provided by a set of multi-class SVM classifiers, and these samples are then manually labeled by an expert. Experimental results indicate that the proposed active learning method significantly reduces the labeling effort, while simultaneously enhancing the classification accuracy.展开更多
This paper describes a new method for active learning in content-based image retrieval. The proposed method firstly uses support vector machine (SVM) classifiers to learn an initial query concept. Then the proposed ac...This paper describes a new method for active learning in content-based image retrieval. The proposed method firstly uses support vector machine (SVM) classifiers to learn an initial query concept. Then the proposed active learning scheme employs similarity measure to check the current version space and selects images with maximum expected information gain to solicit user's label. Finally, the learned query is refined based on the user's further feedback. With the combination of SVM classifier and similarity measure, the proposed method can alleviate model bias existing in each of them. Our experiments on several query concepts show that the proposed method can learn the user's query concept quickly and effectively only with several iterations.展开更多
For multi-objective optimization problems,particle swarm optimization(PSO)algorithm generally needs a large number of fitness evaluations to obtain the Pareto optimal solutions.However,it will become substantially tim...For multi-objective optimization problems,particle swarm optimization(PSO)algorithm generally needs a large number of fitness evaluations to obtain the Pareto optimal solutions.However,it will become substantially time-consuming when handling computationally expensive fitness functions.In order to save the computational cost,a surrogate-assisted PSO with Pareto active learning is proposed.In real physical space(the objective functions are computationally expensive),PSO is used as an optimizer,and its optimization results are used to construct the surrogate models.In virtual space,objective functions are replaced by the cheaper surrogate models,PSO is viewed as a sampler to produce the candidate solutions.To enhance the quality of candidate solutions,a hybrid mutation sampling method based on the simulated evolution is proposed,which combines the advantage of fast convergence of PSO and implements mutation to increase diversity.Furthermore,ε-Pareto active learning(ε-PAL)method is employed to pre-select candidate solutions to guide PSO in the real physical space.However,little work has considered the method of determining parameterε.Therefore,a greedy search method is presented to determine the value ofεwhere the number of active sampling is employed as the evaluation criteria of classification cost.Experimental studies involving application on a number of benchmark test problems and parameter determination for multi-input multi-output least squares support vector machines(MLSSVM)are given,in which the results demonstrate promising performance of the proposed algorithm compared with other representative multi-objective particle swarm optimization(MOPSO)algorithms.展开更多
In this paper, we present a novel Support Vector Machine active learning algorithm for effective 3D model retrieval using the concept of relevance feedback. The proposed method learns from the most informative objects...In this paper, we present a novel Support Vector Machine active learning algorithm for effective 3D model retrieval using the concept of relevance feedback. The proposed method learns from the most informative objects which are marked by the user, and then creates a boundary separating the relevant models from irrelevant ones. What it needs is only a small number of 3D models labelled by the user. It can grasp the user＇s semantic knowledge rapidly and accurately. Experimental results showed that the proposed algorithm significantly improves the retrieval effectiveness. Compared with four state-of-the-art query refinement schemes for 3D model retrieval, it provides superior retrieval performance after no more than two rounds of relevance feedback.展开更多
Active learning has been widely utilized to reduce the labeling cost of supervised learning.By selecting specific instances to train the model,the performance of the model was improved within limited steps.However,rar...Active learning has been widely utilized to reduce the labeling cost of supervised learning.By selecting specific instances to train the model,the performance of the model was improved within limited steps.However,rare work paid attention to the effectiveness of active learning on it.In this paper,we proposed a deep active learning model with bidirectional encoder representations from transformers(BERT)for text classification.BERT takes advantage of the self-attention mechanism to integrate contextual information,which is beneficial to accelerate the convergence of training.As for the process of active learning,we design an instance selection strategy based on posterior probabilities Margin,Intra-correlation and Inter-correlation(MII).Selected instances are characterized by small margin,low intra-cohesion and high inter-cohesion.We conduct extensive experiments and analytics with our methods.The effect of learner is compared while the effect of sampling strategy and text classification is assessed from three real datasets.The results show that our method outperforms the baselines in terms of accuracy.展开更多
This paper is devoted to the probabilistic stability analysis of a tunnel face excavated in a two-layer soil. The interface of the soil layers is assumed to be positioned above the tunnel roof. In the framework of lim...This paper is devoted to the probabilistic stability analysis of a tunnel face excavated in a two-layer soil. The interface of the soil layers is assumed to be positioned above the tunnel roof. In the framework of limit analysis, a rotational failure mechanism is adopted to describe the face failure considering different shear strength parameters in the two layers. The surrogate Kriging model is introduced to replace the actual performance function to perform a Monte Carlo simulation. An active learning function is used to train the Kriging model which can ensure an efficient tunnel face failure probability prediction without loss of accuracy. The deterministic stability analysis is given to validate the proposed tunnel face failure model. Subsequently, the number of initial sampling points, the correlation coefficient, the distribution type and the coefficient of variability of random variables are discussed to show their influences on the failure probability. The proposed approach is an advisable alternative for the tunnel face stability assessment and can provide guidance for tunnel design.展开更多
Owing to the continuous barrage of cyber threats,there is a massive amount of cyber threat intelligence.However,a great deal of cyber threat intelligence come from textual sources.For analysis of cyber threat intellig...Owing to the continuous barrage of cyber threats,there is a massive amount of cyber threat intelligence.However,a great deal of cyber threat intelligence come from textual sources.For analysis of cyber threat intelligence,many security analysts rely on cumbersome and time-consuming manual efforts.Cybersecurity knowledge graph plays a significant role in automatics analysis of cyber threat intelligence.As the foundation for constructing cybersecurity knowledge graph,named entity recognition(NER)is required for identifying critical threat-related elements from textual cyber threat intelligence.Recently,deep neural network-based models have attained very good results in NER.However,the performance of these models relies heavily on the amount of labeled data.Since labeled data in cybersecurity is scarce,in this paper,we propose an adversarial active learning framework to effectively select the informative samples for further annotation.In addition,leveraging the long short-term memory(LSTM)network and the bidirectional LSTM(BiLSTM)network,we propose a novel NER model by introducing a dynamic attention mechanism into the BiLSTM-LSTM encoderdecoder.With the selected informative samples annotated,the proposed NER model is retrained.As a result,the performance of the NER model is incrementally enhanced with low labeling cost.Experimental results show the effectiveness of the proposed method.展开更多
The majority of big data analytics applied to transportation datasets suffer from being too domain-specific,that is,they draw conclusions for a dataset based on analytics on the same dataset.This makes models trained ...The majority of big data analytics applied to transportation datasets suffer from being too domain-specific,that is,they draw conclusions for a dataset based on analytics on the same dataset.This makes models trained from one domain(e.g.taxi data)applies badly to a different domain(e.g.Uber data).To achieve accurate analyses on a new domain,substantial amounts of data must be available,which limits practical applications.To remedy this,we propose to use semi-supervised and active learning of big data to accomplish the domain adaptation task:Selectively choosing a small amount of datapoints from a new domain while achieving comparable performances to using all the datapoints.We choose the New York City(NYC)transportation data of taxi and Uber as our dataset,simulating different domains with 90%as the source data domain for training and the remaining 10%as the target data domain for evaluation.We propose semi-supervised and active learning strategies and apply it to the source domain for selecting datapoints.Experimental results show that our adaptation achieves a comparable performance of using all datapoints while using only a fraction of them,substantially reducing the amount of data required.Our approach has two major advantages:It can make accurate analytics and predictions when big datasets are not available,and even if big datasets are available,our approach chooses the most informative datapoints out of the dataset,making the process much more efficient without having to process huge amounts of data.展开更多
AIM:To conduct a classification study of high myopic maculopathy(HMM)using limited datasets,including tessellated fundus,diffuse chorioretinal atrophy,patchy chorioretinal atrophy,and macular atrophy,and minimize anno...AIM:To conduct a classification study of high myopic maculopathy(HMM)using limited datasets,including tessellated fundus,diffuse chorioretinal atrophy,patchy chorioretinal atrophy,and macular atrophy,and minimize annotation costs,and to optimize the ALFA-Mix active learning algorithm and apply it to HMM classification.METHODS:The optimized ALFA-Mix algorithm(ALFAMix+)was compared with five algorithms,including ALFA-Mix.Four models,including Res Net18,were established.Each algorithm was combined with four models for experiments on the HMM dataset.Each experiment consisted of 20 active learning rounds,with 100 images selected per round.The algorithm was evaluated by comparing the number of rounds in which ALFA-Mix+outperformed other algorithms.Finally,this study employed six models,including Efficient Former,to classify HMM.The best-performing model among these models was selected as the baseline model and combined with the ALFA-Mix+algorithm to achieve satisfactor y classification results with a small dataset.RESULTS:ALFA-Mix+outperforms other algorithms with an average superiority of 16.6,14.75,16.8,and 16.7 rounds in terms of accuracy,sensitivity,specificity,and Kappa value,respectively.This study conducted experiments on classifying HMM using several advanced deep learning models with a complete training set of 4252 images.The Efficient Former achieved the best results with an accuracy,sensitivity,specificity,and Kappa value of 0.8821,0.8334,0.9693,and 0.8339,respectively.Therefore,by combining ALFA-Mix+with Efficient Former,this study achieved results with an accuracy,sensitivity,specificity,and Kappa value of 0.8964,0.8643,0.9721,and 0.8537,respectively.CONCLUSION:The ALFA-Mix+algorithm reduces the required samples without compromising accuracy.Compared to other algorithms,ALFA-Mix+outperforms in more rounds of experiments.It effectively selects valuable samples compared to other algorithms.In HMM classification,combining ALFA-Mix+with Efficient Former enhances model performance,further demonstrating the effectiveness of ALFA-Mix+.展开更多
In speech recognition, acoustic modeling always requires tremendous transcribed samples, and the transcription becomes intensively time-consuming and costly. In order to aid this labor-intensive process, Active Learni...In speech recognition, acoustic modeling always requires tremendous transcribed samples, and the transcription becomes intensively time-consuming and costly. In order to aid this labor-intensive process, Active Learning （AL） is adopted for speech recognition, where only the most informative training samples are selected for manual annotation. In this paper, we propose a novel active learning method for Chinese acoustic modeling, the methods for initial training set selection based on Kullback-Leibler Divergence （KLD） and sample evaluation based on multi-level confusion networks are proposed and adopted in our active learning system, respectively. Our experiments show that our proposed method can achieve satisfying performances.展开更多
Electronic medical record (EMR) containing rich biomedical information has a great potential in disease diagnosis and biomedical research. However, the EMR information is usually in the form of unstructured text, whic...Electronic medical record (EMR) containing rich biomedical information has a great potential in disease diagnosis and biomedical research. However, the EMR information is usually in the form of unstructured text, which increases the use cost and hinders its applications. In this work, an effective named entity recognition (NER) method is presented for information extraction on Chinese EMR, which is achieved by word embedding bootstrapped deep active learning to promote the acquisition of medical information from Chinese EMR and to release its value. In this work, deep active learning of bi-directional long short-term memory followed by conditional random field (Bi-LSTM+CRF) is used to capture the characteristics of different information from labeled corpus, and the word embedding models of contiguous bag of words and skip-gram are combined in the above model to respectively capture the text feature of Chinese EMR from unlabeled corpus. To evaluate the performance of above method, the tasks of NER on Chinese EMR with “medical history” content were used. Experimental results show that the word embedding bootstrapped deep active learning method using unlabeled medical corpus can achieve a better performance compared with other models.展开更多
Objective： To assess the outcome of the application of active learning during practicum among nursing students using clinical assessment and evaluation scores as a measurement. Methods： Nursing students were instruc...Objective： To assess the outcome of the application of active learning during practicum among nursing students using clinical assessment and evaluation scores as a measurement. Methods： Nursing students were instructed on the basics of active learning prior to the initiation of their clinical experience. The participants were divided into 5groups of nursing students （ n = 56） across three levels （years 2-4） in a public academic institute of a bachelor degree program in Macao. Final clinical evaluation was averaged and compared between groups with and without intervention. Results： These nursing students were given higher appraisals in verbal and written comments than previous students without interventian. The groups with the invention achieved higher clinical assessment and evaluation scores on average than comparable groups without the active learning intervention. One group of sophomore nursing students （year 2） did not receive as high of evaluations as the other groups, receiving an average score of above 80. Conclusions＂ Nursing students must engage in active learning to demonstrate that they are willing to gain knowledge of theory, nursing skills and communication skills during the clinical practicum.展开更多
For complex engineering problems,multi-fidelity modeling has been used to achieve efficient reliability analysis by leveraging multiple information sources.However,most methods require nested training samples to captu...For complex engineering problems,multi-fidelity modeling has been used to achieve efficient reliability analysis by leveraging multiple information sources.However,most methods require nested training samples to capture the correlation between different fidelity data,which may lead to a significant increase in low-fidelity samples.In addition,it is difficult to build accurate surrogate models because current methods do not fully consider the nonlinearity between different fidelity samples.To address these problems,a novel multi-fidelity modeling method with active learning is proposed in this paper.Firstly,a nonlinear autoregressive multi-fidelity Kriging(NAMK)model is used to build a surrogate model.To avoid introducing redundant samples in the process of NAMK model updating,a collective learning function is then developed by a combination of a U-learning function,the correlation between different fidelity samples,and the sampling cost.Furthermore,a residual model is constructed to automatically generate low-fidelity samples when high-fidelity samples are selected.The efficiency and accuracy of the proposed method are demonstrated using three numerical examples and an engineering case.展开更多
基金supported by the National Key R&D Program of China(No.2021YFB1715000)the National Natural Science Foundation of China(No.52375073)。
文摘The Reliability-Based Design Optimization(RBDO)of complex engineering structures considering uncertainties has problems of being high-dimensional,highly nonlinear,and timeconsuming,which requires a significant amount of sampling simulation computation.In this paper,a basis-adaptive Polynomial Chaos(PC)-Kriging surrogate model is proposed,in order to relieve the computational burden and enhance the predictive accuracy of a metamodel.The active learning basis-adaptive PC-Kriging model is combined with a quantile-based RBDO framework.Finally,five engineering cases have been implemented,including a benchmark RBDO problem,three high-dimensional explicit problems,and a high-dimensional implicit problem.Compared with Support Vector Regression(SVR),Kriging,and polynomial chaos expansion models,results show that the proposed basis-adaptive PC-Kriging model is more accurate and efficient for RBDO problems of complex engineering structures.
基金supported by the Natural Science Foundation of China[grant numbers 52302093]Natural Science Foundation of Jiangxi Province[grant numbers 20224BAB204021].
文摘In materials science,a significant correlation often exists between material input parameters and their corresponding performance attributes.Nevertheless,the inherent challenges associated with small data obscure these statistical correlations,impeding machine learning models from effectively capturing the underlying patterns,thereby hampering efficient optimization of material properties.This work presents a novel active learning framework that integrates generative adversarial networks(GAN)with a directionally constrained expected absolute improvement(EAI)acquisition function to accelerate the discovery of ultra-high temperature ceramics(UHTCs)using small data.The framework employs GAN for data augmentation,symbolic regression for feature weight derivation,and a self-developed EAI function that incorporates input feature importance weighting to quantify bidirectional deviations from zero ablation rate.Through only two iterations,this framework successfully identified the optimal composition of HfB_(2)-3.52SiC-5.23TaSi_(2),which exhibits robust near-zero ablation rates under plasma ablation at 2500℃ for 200 s,demonstrating superior sampling efficiency compared to conventional active learning approaches.Microstructural analysis reveals that the exceptional performance stems from the formation of a highly viscous HfO_(2)-SiO_(2)-Ta_(2)O_(5)-HfSiO_(4)-Hf_(3)(BO_(3))_(4) oxide layer,which provides effective oxygen barrier protection.This work demonstrates an efficient and universal approach for rapid materials discovery using small data.
基金supported by the National Key Research and Development Program of China(2022ZD0117501)the Scientific Research Innovation Capability Support Project for Young Faculty(ZYGXQNJSKYCXNLZCXM-E7)the Tsinghua University Initiative Scientific Research Program and the Carbon Neutrality and Energy System Transformation(CNEST)Program led by Tsinghua University.
文摘Industrial decarbonization is critical for achieving net-zero goals.The carbon dioxide electrochemical reduction reaction(CO_(2)RR)is a promising approach for converting CO_(2)into high-value chemicals,offering the potential for decarbonizing industrial processes toward a sustainable,carbon-neutral future.However,developing CO_(2)RR catalysts with high selectivity and activity remains a challenge due to the complexity of finding such catalysts and the inefficiency of traditional computational or experimental approaches.Here,we present a methodology integrating density functional theory(DFT)calculations,deep learning models,and an active learning strategy to rapidly screen high-performance catalysts.The proposed methodology is then demonstrated on graphene-based single-atom catalysts for selective CO_(2)electroreduction to methanol.First,we conduct systematic binding energy calculations for 3045 single-atom catalysts to identify thermodynamically stable catalysts as the design space.We then use a graph neural network,fine-tuned with a specialized adsorption energy database,to predict the relative activity and selectivity of the candidate catalysts.An autonomous active learning framework is used to facilitate the exploration of designs.After six learning cycles and 2180 adsorption calculations across 15 intermediates,we develop a surrogate model that identifies four novel catalysts on the Pareto front of activity and selectivity.Our work demonstrates the effectiveness of leveraging a domain foundation model with an active learning framework and holds potential to significantly accelerate the discovery of high-performance CO_(2)RR catalysts.
文摘Objective:Deep learning(DL)has become the prevailing method in chest radiograph analysis,yet its performance heavily depends on large quantities of annotated images.To mitigate the cost,cold-start active learning(AL),comprising an initialization followed by subsequent learning,selects a small subset of informative data points for labeling.Recent advancements in pretrained models by supervised or self-supervised learning tailored to chest radiograph have shown broad applicability to diverse downstream tasks.However,their potential in cold-start AL remains unexplored.Methods:To validate the efficacy of domain-specific pretraining,we compared two foundation models:supervised TXRV and self-supervised REMEDIS with their general domain counterparts pretrained on ImageNet.Model performance was evaluated at both initialization and subsequent learning stages on two diagnostic tasks:psychiatric pneumonia and COVID-19.For initialization,we assessed their integration with three strategies:diversity,uncertainty,and hybrid sampling.For subsequent learning,we focused on uncertainty sampling powered by different pretrained models.We also conducted statistical tests to compare the foundation models with ImageNet counterparts,investigate the relationship between initialization and subsequent learning,examine the performance of one-shot initialization against the full AL process,and investigate the influence of class balance in initialization samples on initialization and subsequent learning.Results:First,domain-specific foundation models failed to outperform ImageNet counterparts in six out of eight experiments on informative sample selection.Both domain-specific and general pretrained models were unable to generate representations that could substitute for the original images as model inputs in seven of the eight scenarios.However,pretrained model-based initialization surpassed random sampling,the default approach in cold-start AL.Second,initialization performance was positively correlated with subsequent learning performance,highlighting the importance of initialization strategies.Third,one-shot initialization performed comparably to the full AL process,demonstrating the potential of reducing experts'repeated waiting during AL iterations.Last,a U-shaped correlation was observed between the class balance of initialization samples and model performance,suggesting that the class balance is more strongly associated with performance at middle budget levels than at low or high budgets.Conclusions:In this study,we highlighted the limitations of medical pretraining compared to general pretraining in the context of cold-start AL.We also identified promising outcomes related to cold-start AL,including initialization based on pretrained models,the positive influence of initialization on subsequent learning,the potential for one-shot initialization,and the influence of class balance on middle-budget AL.Researchers are encouraged to improve medical pretraining for versatile DL foundations and explore novel AL methods.
文摘Human Activity Recognition(HAR)has become increasingly critical in civic surveillance,medical care monitoring,and institutional protection.Current deep learning-based approaches often suffer from excessive computational complexity,limited generalizability under varying conditions,and compromised real-time performance.To counter these,this paper introduces an Active Learning-aided Heuristic Deep Spatio-Textural Ensemble Learning(ALH-DSEL)framework.The model initially identifies keyframes from the surveillance videos with a Multi-Constraint Active Learning(MCAL)approach,with features extracted from DenseNet121.The frames are then segmented employing an optimized Fuzzy C-Means clustering algorithm with Firefly to identify areas of interest.A deep ensemble feature extractor,comprising DenseNet121,EfficientNet-B7,MobileNet,and GLCM,extracts varied spatial and textural features.Fused characteristics are enhanced through PCA and Min-Max normalization and discriminated by a maximum voting ensemble of RF,AdaBoost,and XGBoost.The experimental results show that ALH-DSEL provides higher accuracy,precision,recall,and F1-score,validating its superiority for real-time HAR in surveillance scenarios.
基金supported by the National Natural Science Foundation of China(Grant Nos.62227821,62025503,and 62205199).
文摘To capture the nonlinear dynamics and gain evolution in chirped pulse amplification(CPA)systems,the split-step Fourier method and the fourth-order Runge–Kutta method are integrated to iteratively address the generalized nonlinear Schrödinger equation and the rate equations.However,this approach is burdened by substantial computational demands,resulting in significant time expenditures.In the context of intelligent laser optimization and inverse design,the necessity for numerous simulations further exacerbates this issue,highlighting the need for fast and accurate simulation methodologies.Here,we introduce an end-to-end model augmented with active learning(E2E-AL)with decent generalization through different dedicated embedding methods over various parameters.On an identical computational platform,the artificial intelligence–driven model is 2000 times faster than the conventional simulation method.Benefiting from the active learning strategy,the E2E-AL model achieves decent precision with only two-thirds of the training samples compared with the case without such a strategy.Furthermore,we demonstrate a multi-objective inverse design of the CPA systems enabled by the E2E-AL model.The E2E-AL framework manifests the potential of becoming a standard approach for the rapid and accurate modeling of ultrafast lasers and is readily extended to simulate other complex systems.
基金supported by the National Natural Science Foundation of China(Grant Nos.T2225022,12350710786,62088101,and 12161141016)Shuguang Program of Shanghai Education Development Foundation(Grant No.22SG21)Shanghai Municipal Education Commission,and the Fundamental Research Funds for the Central Universities。
文摘Dynamical systems often exhibit multiple attractors representing significantly different functioning conditions.A global map of attraction basins can offer valuable guidance for stabilizing or transitioning system states.Such a map can be constructed without prior system knowledge by identifying attractors across a sufficient number of points in the state space.However,determining the attractor for each initial state can be a laborious task.Here,we tackle the challenge of reconstructing attraction basins using as few initial points as possible.In each iteration of our approach,informative points are selected through random seeding and are driven along the current classification boundary,promoting the eventual selection of points that are both diverse and enlightening.The results across various experimental dynamical systems demonstrate that our approach requires fewer points than baseline methods while achieving comparable mapping accuracy.Additionally,the reconstructed map allows us to accurately estimate the minimum escape distance required to transition the system state to a target basin.
文摘Support vector machines（SVMs） are a popular class of supervised learning algorithms, and are particularly applicable to large and high-dimensional classification problems. Like most machine learning methods for data classification and information retrieval, they require manually labeled data samples in the training stage. However, manual labeling is a time consuming and errorprone task. One possible solution to this issue is to exploit the large number of unlabeled samples that are easily accessible via the internet. This paper presents a novel active learning method for text categorization. The main objective of active learning is to reduce the labeling effort, without compromising the accuracy of classification, by intelligently selecting which samples should be labeled.The proposed method selects a batch of informative samples using the posterior probabilities provided by a set of multi-class SVM classifiers, and these samples are then manually labeled by an expert. Experimental results indicate that the proposed active learning method significantly reduces the labeling effort, while simultaneously enhancing the classification accuracy.
文摘This paper describes a new method for active learning in content-based image retrieval. The proposed method firstly uses support vector machine (SVM) classifiers to learn an initial query concept. Then the proposed active learning scheme employs similarity measure to check the current version space and selects images with maximum expected information gain to solicit user's label. Finally, the learned query is refined based on the user's further feedback. With the combination of SVM classifier and similarity measure, the proposed method can alleviate model bias existing in each of them. Our experiments on several query concepts show that the proposed method can learn the user's query concept quickly and effectively only with several iterations.
基金supported by the National Natural Sciences Foundation of China(61603069,61533005,61522304,U1560102)the National Key Research and Development Program of China(2017YFA0700300)
文摘For multi-objective optimization problems,particle swarm optimization(PSO)algorithm generally needs a large number of fitness evaluations to obtain the Pareto optimal solutions.However,it will become substantially time-consuming when handling computationally expensive fitness functions.In order to save the computational cost,a surrogate-assisted PSO with Pareto active learning is proposed.In real physical space(the objective functions are computationally expensive),PSO is used as an optimizer,and its optimization results are used to construct the surrogate models.In virtual space,objective functions are replaced by the cheaper surrogate models,PSO is viewed as a sampler to produce the candidate solutions.To enhance the quality of candidate solutions,a hybrid mutation sampling method based on the simulated evolution is proposed,which combines the advantage of fast convergence of PSO and implements mutation to increase diversity.Furthermore,ε-Pareto active learning(ε-PAL)method is employed to pre-select candidate solutions to guide PSO in the real physical space.However,little work has considered the method of determining parameterε.Therefore,a greedy search method is presented to determine the value ofεwhere the number of active sampling is employed as the evaluation criteria of classification cost.Experimental studies involving application on a number of benchmark test problems and parameter determination for multi-input multi-output least squares support vector machines(MLSSVM)are given,in which the results demonstrate promising performance of the proposed algorithm compared with other representative multi-objective particle swarm optimization(MOPSO)algorithms.
基金the National Basic Research Program (973) of China (No. 2004CB719401)the National Research Foundation for the Doctoral Program of Higher Education of China (No.20060003060)
文摘In this paper, we present a novel Support Vector Machine active learning algorithm for effective 3D model retrieval using the concept of relevance feedback. The proposed method learns from the most informative objects which are marked by the user, and then creates a boundary separating the relevant models from irrelevant ones. What it needs is only a small number of 3D models labelled by the user. It can grasp the user＇s semantic knowledge rapidly and accurately. Experimental results showed that the proposed algorithm significantly improves the retrieval effectiveness. Compared with four state-of-the-art query refinement schemes for 3D model retrieval, it provides superior retrieval performance after no more than two rounds of relevance feedback.
基金This work is supported by National Natural Science Foundation of China(61402225,61728204)Innovation Funding(NJ20160028,NT2018028,NS2018057)+1 种基金Aeronautical Science Foundation of China(2016551500)State Key Laboratory for smart grid protection and operation control Foundation,and the Science and Technology Funds from National State Grid Ltd.,China degree and Graduate Education Fund.
文摘Active learning has been widely utilized to reduce the labeling cost of supervised learning.By selecting specific instances to train the model,the performance of the model was improved within limited steps.However,rare work paid attention to the effectiveness of active learning on it.In this paper,we proposed a deep active learning model with bidirectional encoder representations from transformers(BERT)for text classification.BERT takes advantage of the self-attention mechanism to integrate contextual information,which is beneficial to accelerate the convergence of training.As for the process of active learning,we design an instance selection strategy based on posterior probabilities Margin,Intra-correlation and Inter-correlation(MII).Selected instances are characterized by small margin,low intra-cohesion and high inter-cohesion.We conduct extensive experiments and analytics with our methods.The effect of learner is compared while the effect of sampling strategy and text classification is assessed from three real datasets.The results show that our method outperforms the baselines in terms of accuracy.
基金Projects supported by the China Scholarship Council
文摘This paper is devoted to the probabilistic stability analysis of a tunnel face excavated in a two-layer soil. The interface of the soil layers is assumed to be positioned above the tunnel roof. In the framework of limit analysis, a rotational failure mechanism is adopted to describe the face failure considering different shear strength parameters in the two layers. The surrogate Kriging model is introduced to replace the actual performance function to perform a Monte Carlo simulation. An active learning function is used to train the Kriging model which can ensure an efficient tunnel face failure probability prediction without loss of accuracy. The deterministic stability analysis is given to validate the proposed tunnel face failure model. Subsequently, the number of initial sampling points, the correlation coefficient, the distribution type and the coefficient of variability of random variables are discussed to show their influences on the failure probability. The proposed approach is an advisable alternative for the tunnel face stability assessment and can provide guidance for tunnel design.
基金the National Natural Science Foundation of China undergrant 61501515.
文摘Owing to the continuous barrage of cyber threats,there is a massive amount of cyber threat intelligence.However,a great deal of cyber threat intelligence come from textual sources.For analysis of cyber threat intelligence,many security analysts rely on cumbersome and time-consuming manual efforts.Cybersecurity knowledge graph plays a significant role in automatics analysis of cyber threat intelligence.As the foundation for constructing cybersecurity knowledge graph,named entity recognition(NER)is required for identifying critical threat-related elements from textual cyber threat intelligence.Recently,deep neural network-based models have attained very good results in NER.However,the performance of these models relies heavily on the amount of labeled data.Since labeled data in cybersecurity is scarce,in this paper,we propose an adversarial active learning framework to effectively select the informative samples for further annotation.In addition,leveraging the long short-term memory(LSTM)network and the bidirectional LSTM(BiLSTM)network,we propose a novel NER model by introducing a dynamic attention mechanism into the BiLSTM-LSTM encoderdecoder.With the selected informative samples annotated,the proposed NER model is retrained.As a result,the performance of the NER model is incrementally enhanced with low labeling cost.Experimental results show the effectiveness of the proposed method.
文摘The majority of big data analytics applied to transportation datasets suffer from being too domain-specific,that is,they draw conclusions for a dataset based on analytics on the same dataset.This makes models trained from one domain(e.g.taxi data)applies badly to a different domain(e.g.Uber data).To achieve accurate analyses on a new domain,substantial amounts of data must be available,which limits practical applications.To remedy this,we propose to use semi-supervised and active learning of big data to accomplish the domain adaptation task:Selectively choosing a small amount of datapoints from a new domain while achieving comparable performances to using all the datapoints.We choose the New York City(NYC)transportation data of taxi and Uber as our dataset,simulating different domains with 90%as the source data domain for training and the remaining 10%as the target data domain for evaluation.We propose semi-supervised and active learning strategies and apply it to the source domain for selecting datapoints.Experimental results show that our adaptation achieves a comparable performance of using all datapoints while using only a fraction of them,substantially reducing the amount of data required.Our approach has two major advantages:It can make accurate analytics and predictions when big datasets are not available,and even if big datasets are available,our approach chooses the most informative datapoints out of the dataset,making the process much more efficient without having to process huge amounts of data.
基金Supported by the National Natural Science Foundation of China(No.61906066)the Zhejiang Provincial Philosophy and Social Science Planning Project(No.21NDJC021Z)+4 种基金Shenzhen Fund for Guangdong Provincial High-level Clinical Key Specialties(No.SZGSP014)Sanming Project of Medicine in Shenzhen(No.SZSM202011015)Shenzhen Science and Technology Planning Project(No.KCXFZ20211020163813019)the Natural Science Foundation of Ningbo City(No.202003N4072)the Postgraduate Research and Innovation Project of Huzhou University(No.2023KYCX52)。
文摘AIM:To conduct a classification study of high myopic maculopathy(HMM)using limited datasets,including tessellated fundus,diffuse chorioretinal atrophy,patchy chorioretinal atrophy,and macular atrophy,and minimize annotation costs,and to optimize the ALFA-Mix active learning algorithm and apply it to HMM classification.METHODS:The optimized ALFA-Mix algorithm(ALFAMix+)was compared with five algorithms,including ALFA-Mix.Four models,including Res Net18,were established.Each algorithm was combined with four models for experiments on the HMM dataset.Each experiment consisted of 20 active learning rounds,with 100 images selected per round.The algorithm was evaluated by comparing the number of rounds in which ALFA-Mix+outperformed other algorithms.Finally,this study employed six models,including Efficient Former,to classify HMM.The best-performing model among these models was selected as the baseline model and combined with the ALFA-Mix+algorithm to achieve satisfactor y classification results with a small dataset.RESULTS:ALFA-Mix+outperforms other algorithms with an average superiority of 16.6,14.75,16.8,and 16.7 rounds in terms of accuracy,sensitivity,specificity,and Kappa value,respectively.This study conducted experiments on classifying HMM using several advanced deep learning models with a complete training set of 4252 images.The Efficient Former achieved the best results with an accuracy,sensitivity,specificity,and Kappa value of 0.8821,0.8334,0.9693,and 0.8339,respectively.Therefore,by combining ALFA-Mix+with Efficient Former,this study achieved results with an accuracy,sensitivity,specificity,and Kappa value of 0.8964,0.8643,0.9721,and 0.8537,respectively.CONCLUSION:The ALFA-Mix+algorithm reduces the required samples without compromising accuracy.Compared to other algorithms,ALFA-Mix+outperforms in more rounds of experiments.It effectively selects valuable samples compared to other algorithms.In HMM classification,combining ALFA-Mix+with Efficient Former enhances model performance,further demonstrating the effectiveness of ALFA-Mix+.
基金Acknowledgements This study is supported by the National Natural Science Foundation of China （60705019）, the National High-Tech Research and Development Plan of China （ 2006AA010102 and 2007AA01Z417）, the NOKIA project, and the 111 Project of China under Grant No. 1308004.
文摘In speech recognition, acoustic modeling always requires tremendous transcribed samples, and the transcription becomes intensively time-consuming and costly. In order to aid this labor-intensive process, Active Learning （AL） is adopted for speech recognition, where only the most informative training samples are selected for manual annotation. In this paper, we propose a novel active learning method for Chinese acoustic modeling, the methods for initial training set selection based on Kullback-Leibler Divergence （KLD） and sample evaluation based on multi-level confusion networks are proposed and adopted in our active learning system, respectively. Our experiments show that our proposed method can achieve satisfying performances.
基金the Artificial Intelligence Innovation and Development Project of Shanghai Municipal Commission of Economy and Information (No. 2019-RGZN-01081)。
文摘Electronic medical record (EMR) containing rich biomedical information has a great potential in disease diagnosis and biomedical research. However, the EMR information is usually in the form of unstructured text, which increases the use cost and hinders its applications. In this work, an effective named entity recognition (NER) method is presented for information extraction on Chinese EMR, which is achieved by word embedding bootstrapped deep active learning to promote the acquisition of medical information from Chinese EMR and to release its value. In this work, deep active learning of bi-directional long short-term memory followed by conditional random field (Bi-LSTM+CRF) is used to capture the characteristics of different information from labeled corpus, and the word embedding models of contiguous bag of words and skip-gram are combined in the above model to respectively capture the text feature of Chinese EMR from unlabeled corpus. To evaluate the performance of above method, the tasks of NER on Chinese EMR with “medical history” content were used. Experimental results show that the word embedding bootstrapped deep active learning method using unlabeled medical corpus can achieve a better performance compared with other models.
基金supported by the research fund of Macao Polytechnic Institute(RP/ESS-04/2012)
文摘Objective： To assess the outcome of the application of active learning during practicum among nursing students using clinical assessment and evaluation scores as a measurement. Methods： Nursing students were instructed on the basics of active learning prior to the initiation of their clinical experience. The participants were divided into 5groups of nursing students （ n = 56） across three levels （years 2-4） in a public academic institute of a bachelor degree program in Macao. Final clinical evaluation was averaged and compared between groups with and without intervention. Results： These nursing students were given higher appraisals in verbal and written comments than previous students without interventian. The groups with the invention achieved higher clinical assessment and evaluation scores on average than comparable groups without the active learning intervention. One group of sophomore nursing students （year 2） did not receive as high of evaluations as the other groups, receiving an average score of above 80. Conclusions＂ Nursing students must engage in active learning to demonstrate that they are willing to gain knowledge of theory, nursing skills and communication skills during the clinical practicum.
基金supported by the Major Projects of Zhejiang Provincial Natural Science Foundation of China(No.LD22E050009)the National Natural Science Foundation of China(No.51475425)the College Student’s Science and Technology Innovation Project of Zhejiang Province(No.2022R403B060),China.
文摘For complex engineering problems,multi-fidelity modeling has been used to achieve efficient reliability analysis by leveraging multiple information sources.However,most methods require nested training samples to capture the correlation between different fidelity data,which may lead to a significant increase in low-fidelity samples.In addition,it is difficult to build accurate surrogate models because current methods do not fully consider the nonlinearity between different fidelity samples.To address these problems,a novel multi-fidelity modeling method with active learning is proposed in this paper.Firstly,a nonlinear autoregressive multi-fidelity Kriging(NAMK)model is used to build a surrogate model.To avoid introducing redundant samples in the process of NAMK model updating,a collective learning function is then developed by a combination of a U-learning function,the correlation between different fidelity samples,and the sampling cost.Furthermore,a residual model is constructed to automatically generate low-fidelity samples when high-fidelity samples are selected.The efficiency and accuracy of the proposed method are demonstrated using three numerical examples and an engineering case.
