As a data-driven approach, Deep Learning(DL)-based fault diagnosis methods need to collect the relatively comprehensive data on machine fault types to achieve satisfactory performance. A mechanical system may include ...As a data-driven approach, Deep Learning(DL)-based fault diagnosis methods need to collect the relatively comprehensive data on machine fault types to achieve satisfactory performance. A mechanical system may include multiple submachines in the real-world. During condition monitoring of a mechanical system, fault data are distributed in a continuous flow of constantly generated information and new faults will inevitably occur in unconsidered submachines, which are also called machine increments. Therefore, adequately collecting fault data in advance is difficult. Limited by the characteristics of DL, training existing models directly with new fault data of new submachines leads to catastrophic forgetting of old tasks, while the cost of collecting all known data to retrain the models is excessively high. DL-based fault diagnosis methods cannot learn continually and adaptively in dynamic environments. A new Continual Learning Fault Diagnosis method(CLFD) is proposed in this paper to solve a series of fault diagnosis tasks with machine increments. The stability–plasticity dilemma is an intrinsic issue in continual learning. The core of CLFD is the proposed Dual-branch Adaptive Aggregation Residual Network(DAARN).Two types of residual blocks are created in each block layer of DAARN: steady and dynamic blocks. The stability–plasticity dilemma is solved by assigning them with adaptive aggregation weights to balance stability and plasticity, and a bi-level optimization program is used to optimize adaptive aggregation weights and model parameters. In addition, a feature-level knowledge distillation loss function is proposed to further overcome catastrophic forgetting. CLFD is then applied to the fault diagnosis case with machine increments. Results demonstrate that CLFD outperforms other continual learning methods and has satisfactory robustness.展开更多
Continual learning(CL)studies the problem of learning to accumulate knowledge over time from a stream of data.A crucial challenge is that neural networks suffer from performance degradation on previously seen data,kno...Continual learning(CL)studies the problem of learning to accumulate knowledge over time from a stream of data.A crucial challenge is that neural networks suffer from performance degradation on previously seen data,known as catastrophic forgetting,due to allowing parameter sharing.In this work,we consider a more practical online class-incremental CL setting,where the model learns new samples in an online manner and may continuously experience new classes.Moreover,prior knowledge is unavailable during training and evaluation.Existing works usually explore sample usages from a single dimension,which ignores a lot of valuable supervisory information.To better tackle the setting,we propose a novel replay-based CL method,which leverages multi-level representations produced by the intermediate process of training samples for replay and strengthens supervision to consolidate previous knowledge.Specifically,besides the previous raw samples,we store the corresponding logits and features in the memory.Furthermore,to imitate the prediction of the past model,we construct extra constraints by leveraging multi-level information stored in the memory.With the same number of samples for replay,our method can use more past knowledge to prevent interference.We conduct extensive evaluations on several popular CL datasets,and experiments show that our method consistently outperforms state-of-the-art methods with various sizes of episodic memory.We further provide a detailed analysis of these results and demonstrate that our method is more viable in practical scenarios.展开更多
Bayesian-based methods have emerged as an effective approach in continual learning(CL) to solve catastrophic forgetting. One prominent example is Variational Continual Learning(VCL), which demonstrates remarkable perf...Bayesian-based methods have emerged as an effective approach in continual learning(CL) to solve catastrophic forgetting. One prominent example is Variational Continual Learning(VCL), which demonstrates remarkable performance in task-incremental learning(task-IL).However, class-incremental learning(class-IL) is still challenging for VCL, and the reasons behind this limitation remain unclear. Relying on the sophisticated neural mechanisms, particularly the mechanism of memory consolidation during sleep, the human brain possesses inherent advantages for both task-IL and class-IL scenarios, which provides insight for a braininspired VCL. To identify the reasons for the inadequacy of VCL in class-IL, we first conduct a comprehensive theoretical analysis of VCL. On this basis, we propose a novel Bayesian framework named as Learning within Sleeping(Lw S) by leveraging the memory consolidation.By simulating the distribution integration and generalization observed during memory consolidation in sleep, Lw S achieves the idea of prior knowledge guiding posterior knowledge learning as in VCL. In addition, with emulating the process of memory reactivation of the brain,Lw S imposes a constraint on feature invariance to mitigate forgetting learned knowledge. Experimental results demonstrate that Lw S outperforms both Bayesian and non-Bayesian methods in task-IL and class-IL scenarios, which further indicates the effectiveness of incorporating brain mechanisms on designing novel approaches for CL.展开更多
Climate change poses significant challenges to agricultural management,particularly in adapting to extreme weather conditions that impact agricultural production.Existing works with traditional Reinforcement Learning(...Climate change poses significant challenges to agricultural management,particularly in adapting to extreme weather conditions that impact agricultural production.Existing works with traditional Reinforcement Learning(RL)methods often falter under such extreme conditions.To address this challenge,our study introduces a novel approach by integrating Continual Learning(CL)with RL to form Continual Reinforcement Learning(CRL),enhancing the adaptability of agricultural management strategies.Leveraging the Gym-DSSAT simulation environment,our research enables RL agents to learn optimal fertilization strategies based on variable weather conditions.By incorporating CL algorithms,such as Elastic Weight Consolidation(EWC),with established RL techniques like Deep Q-Networks(DQN),we developed a framework in which agents can learn and retain knowledge across diverse weather scenarios.The CRL approach was tested under climate variability to assess the robustness and adaptability of the induced policies,particularly under extreme weather events like severe droughts.Our results showed that continually learned policies exhibited superior adaptability and performance compared to optimal policies learned through the conventional RL methods,especially in challenging conditions of reduced rainfall and increased temperatures.This pioneering work,which combines CL with RL to generate adaptive policies for agricultural management,is expected to make significant advancements in precision agriculture in the era of climate change.展开更多
Digital twin(DT)technology is currently pervasive in industrial Internet of things(IoT)applications,notably in predictive maintenance scenarios.Prevailing digital twin-based predictive maintenance methodologies are co...Digital twin(DT)technology is currently pervasive in industrial Internet of things(IoT)applications,notably in predictive maintenance scenarios.Prevailing digital twin-based predictive maintenance methodologies are constrained by a narrow focus on singular physical modeling paradigms,impeding comprehensive analysis of diverse factory data at scale.This paper introduces an improved method,federated continual learning with authentication for distributed digital twin-based industrial IoT(FCLA-DT).This decentralized strategy ensures the continual learning capability vital for adaptive and real-time decision-making in complex industrial predictive maintenance systems.An authentication scheme based on group signature is introduced to enable the verification of digital twin identities during inter-twin collaborations,avoiding unauthorized access and potential model theft.Security analysis shows that FCLA-DT can enable numerous nodes to collaborate learning without compromising individual twin privacy,thereby ensuring group authentication in the cooperative distributed industrial IoT.Performance analysis shows that FCLA-DT outperforms traditional federated learning methods with over 95% fault diagnosis accuracy and ensures the privacy and authentication of digital twins in multi-client task learning.展开更多
In natural language processing(NLP),managing multiple downstream tasks through fine-tuning pre-trained models often requires maintaining separate task-specific models,leading to practical inefficiencies.To address thi...In natural language processing(NLP),managing multiple downstream tasks through fine-tuning pre-trained models often requires maintaining separate task-specific models,leading to practical inefficiencies.To address this challenge,we introduce AdaptForever,a novel approach that enables continuous mastery of NLP tasks through the integration of elastic and mutual learning strategies with a stochastic expert mechanism.Our method freezes the pre-trained model weights while incorporating adapters enhanced with mutual learning capabilities,facilitating effective knowledge transfer from previous tasks to new ones.By combining Elastic Weight Consolidation(EWC)for knowledge preservation with specialized regularization terms,AdaptForever successfully maintains performance on earlier tasks while acquiring new capabilities.Experimental results demonstrate that AdaptForever achieves superior performance across a continuous sequence of NLP tasks compared to existing parameter-efficient methods,while effectively preventing catastrophic forgetting and enabling positive knowledge transfer between tasks.展开更多
Deep learning has shown its human-level performance in various applications.However,current deep learning models are characterized by catastrophic forgetting of old knowledge when learning new classes.This poses a cha...Deep learning has shown its human-level performance in various applications.However,current deep learning models are characterized by catastrophic forgetting of old knowledge when learning new classes.This poses a challenge such as in intelligent diagnosis systems where initially only training data of a limited number of diseases are available.In this case,updating the intelligent system with data of new diseases would inevitably downgrade its performance on previously learned diseases.Inspired by the process of learning new knowledge in human brains,we propose a Bayesian generative model for continual learning built on afixed pre-trained feature extractor.In this model,knowledge of each old class can be compactly represented by a collection of statistical distributions,e.g.,with Gaussian mixture models,and naturally kept from forgetting in continual learning over time.Unlike existing class-incremental learning methods,the proposed approach is not sensitive to the continual learning process and can be additionally well applied to the data-incremental learning scenario.Experiments on multiple medical and natural image classification tasks reveal that the proposed approach outperforms state-of-the-art approaches that even keep some images of old classes during continual learning of new classes.展开更多
Although modulation classification based on deep neural network can achieve high Modulation Classification(MC)accuracies,catastrophic forgetting will occur when the neural network model continues to learn new tasks.In...Although modulation classification based on deep neural network can achieve high Modulation Classification(MC)accuracies,catastrophic forgetting will occur when the neural network model continues to learn new tasks.In this paper,we simulate the dynamic wireless communication environment and focus on breaking the learning paradigm of isolated automatic MC.We innovate a research algorithm for continuous automatic MC.Firstly,a memory for storing representative old task modulation signals is built,which is employed to limit the gradient update direction of new tasks in the continuous learning stage to ensure that the loss of old tasks is also in a downward trend.Secondly,in order to better simulate the dynamic wireless communication environment,we employ the mini-batch gradient algorithm which is more suitable for continuous learning.Finally,the signal in the memory can be replayed to further strengthen the characteristics of the old task signal in the model.Simulation results verify the effectiveness of the method.展开更多
Objectives： To analyse motivation and preferences of pharmacists who participate in CE （continuing education） to develop suitable lifelong learning programmes for pharmacists. Methods： An online questionnaire, whi...Objectives： To analyse motivation and preferences of pharmacists who participate in CE （continuing education） to develop suitable lifelong learning programmes for pharmacists. Methods： An online questionnaire, which explored the motivation and preferences of the pharmacists to lifelong learning, was sent to all members of the Royal Dutch Pharmaceutical Society （4321） in the Netherlands. The data were analysed using a non-hierarchical clustering technique. Key findings： Two clusters of pharmacists were discovered. Cluster A pharmacists （n = 474） were more motivated by credit points （63.5% vs. 47.2%）, personal interest （84.1% vs. 56.3%）, updating knowledge （73.8% vs. 56.8%） and topicality of CE courses （47.7% vs. 26.1%）. Cluster B pharmacists （n = 199） were predominantly motivated by the aspect ＂duty as a care-giver＂ （97.0% vs. 0 % in cluster A）. Pharmacists who belonged to cluster A tended to be women （60.5%）, often worked part-time （29.3%） and mostly preferred lectures （71.1%）. Cluster B pharmacists consisted of statistically significantly more male pharmacists （52.8%, p = 0.001）, worked more full time （77.4%, p = 0.009） and mostly preferred blended learning （62.3%, p = 0.047）. Conclusions： These results suggest the use of different education formats for different kinds of pharmacists to participate in CE activities.展开更多
The network security analyzers use intrusion detection systems(IDSes)to distinguish malicious traffic from benign ones.The deep learning-based(DL-based)IDSes are proposed to auto-extract high-level features and elimin...The network security analyzers use intrusion detection systems(IDSes)to distinguish malicious traffic from benign ones.The deep learning-based(DL-based)IDSes are proposed to auto-extract high-level features and eliminate the time-consuming and costly signature extraction process.However,this new generation of IDSes still needs to overcome a number of challenges to be employed in practical environments.One of the main issues of an applicable IDS is facing traffic concept drift,which manifests itself as new(i.e.,zero-day)attacks,in addition to the changing behavior of benign users/applications.Furthermore,a practical DL-based IDS needs to be conformed to a distributed(i.e.,multi-sensor)architecture in order to yield more accurate detections,create a collective attack knowledge based on the observations of different sensors,and also handle big data challenges for supporting high throughput networks.This paper proposes a novel multi-agent network intrusion detection framework to address the above shortcomings,considering a more practical scenario(i.e.,online adaptable IDSes).This framework employs continual deep anomaly detectors for adapting each agent to the changing attack/benign patterns in its local traffic.In addition,a federated learning approach is proposed for sharing and exchanging local knowledge between different agents.Furthermore,the proposed framework implements sequential packet labeling for each flow,which provides an attack probability score for the flow by gradually observing each flow packet and updating its estimation.We evaluate the proposed framework by employing different deep models(including CNN-based and LSTM-based)over the CICIDS2017 and CSE-CIC-IDS2018 datasets.Through extensive evaluations and experiments,we show that the proposed distributed framework is well adapted to the traffic concept drift.More precisely,our results indicate that the CNNbased models are well suited for continually adapting to the traffic concept drift(i.e.,achieving an average detection rate of above 95%while needing just 128 new flows for the updating phase),and the LSTM-based models are a good candidate for sequential packet labeling in practical online IDSes(i.e.,detecting intrusions by just observing their first 15 packets).展开更多
Continual learning(CL)has emerged as a crucial paradigm for learning from sequential data while retaining previous knowledge.Continual graph learning(CGL),characterized by dynamically evolving graphs from streaming da...Continual learning(CL)has emerged as a crucial paradigm for learning from sequential data while retaining previous knowledge.Continual graph learning(CGL),characterized by dynamically evolving graphs from streaming data,presents distinct challenges that demand efficient algorithms to prevent catastrophic forgetting.The first challenge stems from the interdependencies between different graph data,in which previous graphs infuence new data distributions.The second challenge is handling large graphs in an efficient manner.To address these challenges,we propose an eficient continual graph learner(E-CGL)in this paper.We address the interdependence issue by demonstrating the effectiveness of replay strategies and introducing a combined sampling approach that considers both node importance and diversity.To improve efficiency,E-CGL leverages a simple yet effective multilayer perceptron(MLP)model that shares weights with a graph neural network(GNN)during training,thereby accelerating computation by circumventing the expensive message-passing process.Our method achieves state-ofthe-art results on four CGL datasets under two settings,while significantly lowering the catastrophic forgetting value to an average of-1.1%.Additionally,E-CGL achieves the training and inference speedup by an average of 15.83x and 4.89x,respectively,across four datasets.These results indicate that E-CGL not only effectively manages correlations between different graph data during continual training but also enhances efficiency in large-scale CGL.展开更多
The automatic colorization of anime line drawings is a challenging problem in production pipelines.Recent advances in deep neural networks have addressed this problem;however,collectingmany images of colorization targ...The automatic colorization of anime line drawings is a challenging problem in production pipelines.Recent advances in deep neural networks have addressed this problem;however,collectingmany images of colorization targets in novel anime work before the colorization process starts leads to chicken-and-egg problems and has become an obstacle to using them in production pipelines.To overcome this obstacle,we propose a new patch-based learning method for few-shot anime-style colorization.The learning method adopts an efficient patch sampling technique with position embedding according to the characteristics of anime line drawings.We also present a continuous learning strategy that continuously updates our colorization model using new samples colorized by human artists.The advantage of our method is that it can learn our colorization model from scratch or pre-trained weights using only a few pre-and post-colorized line drawings that are created by artists in their usual colorization work.Therefore,our method can be easily incorporated within existing production pipelines.We quantitatively demonstrate that our colorizationmethod outperforms state-of-the-art methods.展开更多
Modelling,predicting and analysing driver behaviours are essential to advanced driver assistance systems(ADAS)and the comprehensive understanding of complex driving scenarios.Recently,with the development of deep lear...Modelling,predicting and analysing driver behaviours are essential to advanced driver assistance systems(ADAS)and the comprehensive understanding of complex driving scenarios.Recently,with the development of deep learning(DL),numerous driver behaviour learning(DBL)methods have been proposed and applied in connected vehicles(CV)and intelligent transportation systems(ITS).This study provides a review of DBL,which mainly focuses on typical applications in CV and ITS.First,a comprehensive review of the state-of-the-art DBL is presented.Next,Given the constantly changing nature of real driving scenarios,most existing learning-based models may suffer from the so-called“catastrophic forgetting,”which refers to their inability to perform well in previously learned scenarios after acquiring new ones.As a solution to the aforementioned issue,this paper presents a framework for continual driver behaviour learning(CDBL)by leveraging continual learning technology.The proposed CDBL framework is demonstrated to outperform existing methods in behaviour prediction through a case study.Finally,future works,potential challenges and emerging trends in this area are highlighted.展开更多
Purpose:During Japan's initial pandemic prevention and control period,the Ministry of Education,Culture,Sports,Science,and Technology of Japan(MEXT)issued several notifications to ensure students'continuous le...Purpose:During Japan's initial pandemic prevention and control period,the Ministry of Education,Culture,Sports,Science,and Technology of Japan(MEXT)issued several notifications to ensure students'continuous learning according to the"leave no one behind"philosophy.This study focused on the comprehensive measures comprising top-level arrangements to implement the"suspending classes without stopping learning"initiative.Design/Approach/Methods:The study reviewed MEXT's guide on ensuring learning for Japanese primary and secondary school students during the pandemic.Findings:The pandemic has accelerated the development of the information and communications technology educational environment and the implementation of the Global and Innovation Gateway for All(GIGA)schools initiative.These developments laid the foundation for the evolution of diverse pedagogical models and feasible methods for promoting equity.The home schooling experience during the pandemic provided a natural base for students to practice and foster survivability.Originality/Value:This paper provides scholars with an understanding of the opportunities and challenges encountered in educational innovation in Japan.It provides insights into the future direction of pedagogical development,capability development,and effective educational arrangements for extraordinary circumstances to facilitate educational development from a broader perspective,particularly under the new normal.展开更多
1 Introduction Unsupervised Federated Continual Learning(UFCL)is a new learning paradigm that embeds unsupervised representation techniques into the Federated Learning(FL)framework,which enables continuous training of...1 Introduction Unsupervised Federated Continual Learning(UFCL)is a new learning paradigm that embeds unsupervised representation techniques into the Federated Learning(FL)framework,which enables continuous training of a shared representation model without compromising individual participants’data privacy[1,2].However,the continuous learning process may cause catastrophic forgetting in the model,reducing generated representations’performance.展开更多
Recently,the catastrophic forgetting problem of neural networks in the process of continual learning(CL)has attracted more and more attention with the development of deep learning.The orthogonal weight modification(OW...Recently,the catastrophic forgetting problem of neural networks in the process of continual learning(CL)has attracted more and more attention with the development of deep learning.The orthogonal weight modification(OWM)algorithm to some extent overcomes the catastrophic forgetting problem in CL.It is well-known that the mapping rule learned by the network is usually not accurate in the early stage of neural network training.Our main idea is to establish an immune mechanism in CL,which rejects unreliable mapping rules at the beginning of the training until those are reliable enough.Our algorithm showed a very good competitive advantage in the permuted and disjoint MNIST tasks and disjoint CIFAR-10 tasks.As for the more challenging task of Chinese handwriting character recognition,our algorithm showed a notable improvement compared with the OWM algorithm.In view of the context-dependent processing(CDP)module in[37],we revealed that the module may result in a loss of information and we proposed a modified CDP module to overcome this weakness.The performance of the system with the modified CDP module outperforms the original one in the CelebFaces attributed recognition task.Besides continual multi-task,we also considered a single task,where the immunity-based OWM(IOWM)algorithm was designed as an optimization solver of neural networks for low-dose computed tomography(CT)denoising task.展开更多
The power market is a typical imperfectly competitive market where power suppliers gain higher profits through strategic bidding behaviors.Most existing studies assume that a power supplier is accessible to the suffic...The power market is a typical imperfectly competitive market where power suppliers gain higher profits through strategic bidding behaviors.Most existing studies assume that a power supplier is accessible to the sufficient market information to derive an optimal bidding strategy.However,this assumption may not be true in reality,particularly when a power market is newly launched.To help power suppliers bid with the limited information,a modified continuous action reinforcement learning automata algorithm is proposed.This algorithm introduces the discretization and Dyna structure into continuous action reinforcement learning automata algorithm for easy implementation in a repeated game.Simulation results verify the effectiveness of the proposed learning algorithm.展开更多
基金supported by the National Natural Science Foundation of China(Nos.52272440,51875375)the China Postdoctoral Science Foundation Funded Project(No.2021M701503).
文摘As a data-driven approach, Deep Learning(DL)-based fault diagnosis methods need to collect the relatively comprehensive data on machine fault types to achieve satisfactory performance. A mechanical system may include multiple submachines in the real-world. During condition monitoring of a mechanical system, fault data are distributed in a continuous flow of constantly generated information and new faults will inevitably occur in unconsidered submachines, which are also called machine increments. Therefore, adequately collecting fault data in advance is difficult. Limited by the characteristics of DL, training existing models directly with new fault data of new submachines leads to catastrophic forgetting of old tasks, while the cost of collecting all known data to retrain the models is excessively high. DL-based fault diagnosis methods cannot learn continually and adaptively in dynamic environments. A new Continual Learning Fault Diagnosis method(CLFD) is proposed in this paper to solve a series of fault diagnosis tasks with machine increments. The stability–plasticity dilemma is an intrinsic issue in continual learning. The core of CLFD is the proposed Dual-branch Adaptive Aggregation Residual Network(DAARN).Two types of residual blocks are created in each block layer of DAARN: steady and dynamic blocks. The stability–plasticity dilemma is solved by assigning them with adaptive aggregation weights to balance stability and plasticity, and a bi-level optimization program is used to optimize adaptive aggregation weights and model parameters. In addition, a feature-level knowledge distillation loss function is proposed to further overcome catastrophic forgetting. CLFD is then applied to the fault diagnosis case with machine increments. Results demonstrate that CLFD outperforms other continual learning methods and has satisfactory robustness.
基金supported in part by the National Natura Science Foundation of China(U2013602,61876181,51521003)the Nationa Key R&D Program of China(2020YFB13134)+2 种基金Shenzhen Science and Technology Research and Development Foundation(JCYJ20190813171009236)Beijing Nova Program of Science and Technology(Z191100001119043)the Youth Innovation Promotion Association,Chinese Academy of Sciences。
文摘Continual learning(CL)studies the problem of learning to accumulate knowledge over time from a stream of data.A crucial challenge is that neural networks suffer from performance degradation on previously seen data,known as catastrophic forgetting,due to allowing parameter sharing.In this work,we consider a more practical online class-incremental CL setting,where the model learns new samples in an online manner and may continuously experience new classes.Moreover,prior knowledge is unavailable during training and evaluation.Existing works usually explore sample usages from a single dimension,which ignores a lot of valuable supervisory information.To better tackle the setting,we propose a novel replay-based CL method,which leverages multi-level representations produced by the intermediate process of training samples for replay and strengthens supervision to consolidate previous knowledge.Specifically,besides the previous raw samples,we store the corresponding logits and features in the memory.Furthermore,to imitate the prediction of the past model,we construct extra constraints by leveraging multi-level information stored in the memory.With the same number of samples for replay,our method can use more past knowledge to prevent interference.We conduct extensive evaluations on several popular CL datasets,and experiments show that our method consistently outperforms state-of-the-art methods with various sizes of episodic memory.We further provide a detailed analysis of these results and demonstrate that our method is more viable in practical scenarios.
基金supported by the National Natural Science Foundation of China under Grant 62236001 and Grant 62325601
文摘Bayesian-based methods have emerged as an effective approach in continual learning(CL) to solve catastrophic forgetting. One prominent example is Variational Continual Learning(VCL), which demonstrates remarkable performance in task-incremental learning(task-IL).However, class-incremental learning(class-IL) is still challenging for VCL, and the reasons behind this limitation remain unclear. Relying on the sophisticated neural mechanisms, particularly the mechanism of memory consolidation during sleep, the human brain possesses inherent advantages for both task-IL and class-IL scenarios, which provides insight for a braininspired VCL. To identify the reasons for the inadequacy of VCL in class-IL, we first conduct a comprehensive theoretical analysis of VCL. On this basis, we propose a novel Bayesian framework named as Learning within Sleeping(Lw S) by leveraging the memory consolidation.By simulating the distribution integration and generalization observed during memory consolidation in sleep, Lw S achieves the idea of prior knowledge guiding posterior knowledge learning as in VCL. In addition, with emulating the process of memory reactivation of the brain,Lw S imposes a constraint on feature invariance to mitigate forgetting learned knowledge. Experimental results demonstrate that Lw S outperforms both Bayesian and non-Bayesian methods in task-IL and class-IL scenarios, which further indicates the effectiveness of incorporating brain mechanisms on designing novel approaches for CL.
基金support from the University of Iowa OVPR Interdisciplinary Scholars Program and the US Department of Education(ED#P116S210005)for this study.Kishlay Jha’s work is supported in part by the US National Institute of Health(NIH)and National Science Foundation(NSF)under grants R01LM014012-01A1 and ITE-2333740.
文摘Climate change poses significant challenges to agricultural management,particularly in adapting to extreme weather conditions that impact agricultural production.Existing works with traditional Reinforcement Learning(RL)methods often falter under such extreme conditions.To address this challenge,our study introduces a novel approach by integrating Continual Learning(CL)with RL to form Continual Reinforcement Learning(CRL),enhancing the adaptability of agricultural management strategies.Leveraging the Gym-DSSAT simulation environment,our research enables RL agents to learn optimal fertilization strategies based on variable weather conditions.By incorporating CL algorithms,such as Elastic Weight Consolidation(EWC),with established RL techniques like Deep Q-Networks(DQN),we developed a framework in which agents can learn and retain knowledge across diverse weather scenarios.The CRL approach was tested under climate variability to assess the robustness and adaptability of the induced policies,particularly under extreme weather events like severe droughts.Our results showed that continually learned policies exhibited superior adaptability and performance compared to optimal policies learned through the conventional RL methods,especially in challenging conditions of reduced rainfall and increased temperatures.This pioneering work,which combines CL with RL to generate adaptive policies for agricultural management,is expected to make significant advancements in precision agriculture in the era of climate change.
基金supported by the National Natural Science Foundation of China under Grant 62472132Natural Science Foundation of Zhejiang Province under Grant LZ22F030004Key Research and Development Program Project of Zhejiang Province under Grant 2024C01179.
文摘Digital twin(DT)technology is currently pervasive in industrial Internet of things(IoT)applications,notably in predictive maintenance scenarios.Prevailing digital twin-based predictive maintenance methodologies are constrained by a narrow focus on singular physical modeling paradigms,impeding comprehensive analysis of diverse factory data at scale.This paper introduces an improved method,federated continual learning with authentication for distributed digital twin-based industrial IoT(FCLA-DT).This decentralized strategy ensures the continual learning capability vital for adaptive and real-time decision-making in complex industrial predictive maintenance systems.An authentication scheme based on group signature is introduced to enable the verification of digital twin identities during inter-twin collaborations,avoiding unauthorized access and potential model theft.Security analysis shows that FCLA-DT can enable numerous nodes to collaborate learning without compromising individual twin privacy,thereby ensuring group authentication in the cooperative distributed industrial IoT.Performance analysis shows that FCLA-DT outperforms traditional federated learning methods with over 95% fault diagnosis accuracy and ensures the privacy and authentication of digital twins in multi-client task learning.
基金supported by the National Key R&D Program of China(No.2023YFB3308601)Sichuan Science and Technology Program(2024NSFJQ0035,2024NSFSC0004)the Talents by Sichuan provincial Party Committee Organization Department.
文摘In natural language processing(NLP),managing multiple downstream tasks through fine-tuning pre-trained models often requires maintaining separate task-specific models,leading to practical inefficiencies.To address this challenge,we introduce AdaptForever,a novel approach that enables continuous mastery of NLP tasks through the integration of elastic and mutual learning strategies with a stochastic expert mechanism.Our method freezes the pre-trained model weights while incorporating adapters enhanced with mutual learning capabilities,facilitating effective knowledge transfer from previous tasks to new ones.By combining Elastic Weight Consolidation(EWC)for knowledge preservation with specialized regularization terms,AdaptForever successfully maintains performance on earlier tasks while acquiring new capabilities.Experimental results demonstrate that AdaptForever achieves superior performance across a continuous sequence of NLP tasks compared to existing parameter-efficient methods,while effectively preventing catastrophic forgetting and enabling positive knowledge transfer between tasks.
基金supported in part by the National Natural Science Foundation of China(Grant Nos.62071502,U1811461)the Guangdong Key Research and Development Program(Grant No.2020B1111190001).
文摘Deep learning has shown its human-level performance in various applications.However,current deep learning models are characterized by catastrophic forgetting of old knowledge when learning new classes.This poses a challenge such as in intelligent diagnosis systems where initially only training data of a limited number of diseases are available.In this case,updating the intelligent system with data of new diseases would inevitably downgrade its performance on previously learned diseases.Inspired by the process of learning new knowledge in human brains,we propose a Bayesian generative model for continual learning built on afixed pre-trained feature extractor.In this model,knowledge of each old class can be compactly represented by a collection of statistical distributions,e.g.,with Gaussian mixture models,and naturally kept from forgetting in continual learning over time.Unlike existing class-incremental learning methods,the proposed approach is not sensitive to the continual learning process and can be additionally well applied to the data-incremental learning scenario.Experiments on multiple medical and natural image classification tasks reveal that the proposed approach outperforms state-of-the-art approaches that even keep some images of old classes during continual learning of new classes.
文摘Although modulation classification based on deep neural network can achieve high Modulation Classification(MC)accuracies,catastrophic forgetting will occur when the neural network model continues to learn new tasks.In this paper,we simulate the dynamic wireless communication environment and focus on breaking the learning paradigm of isolated automatic MC.We innovate a research algorithm for continuous automatic MC.Firstly,a memory for storing representative old task modulation signals is built,which is employed to limit the gradient update direction of new tasks in the continuous learning stage to ensure that the loss of old tasks is also in a downward trend.Secondly,in order to better simulate the dynamic wireless communication environment,we employ the mini-batch gradient algorithm which is more suitable for continuous learning.Finally,the signal in the memory can be replayed to further strengthen the characteristics of the old task signal in the model.Simulation results verify the effectiveness of the method.
文摘Objectives： To analyse motivation and preferences of pharmacists who participate in CE （continuing education） to develop suitable lifelong learning programmes for pharmacists. Methods： An online questionnaire, which explored the motivation and preferences of the pharmacists to lifelong learning, was sent to all members of the Royal Dutch Pharmaceutical Society （4321） in the Netherlands. The data were analysed using a non-hierarchical clustering technique. Key findings： Two clusters of pharmacists were discovered. Cluster A pharmacists （n = 474） were more motivated by credit points （63.5% vs. 47.2%）, personal interest （84.1% vs. 56.3%）, updating knowledge （73.8% vs. 56.8%） and topicality of CE courses （47.7% vs. 26.1%）. Cluster B pharmacists （n = 199） were predominantly motivated by the aspect ＂duty as a care-giver＂ （97.0% vs. 0 % in cluster A）. Pharmacists who belonged to cluster A tended to be women （60.5%）, often worked part-time （29.3%） and mostly preferred lectures （71.1%）. Cluster B pharmacists consisted of statistically significantly more male pharmacists （52.8%, p = 0.001）, worked more full time （77.4%, p = 0.009） and mostly preferred blended learning （62.3%, p = 0.047）. Conclusions： These results suggest the use of different education formats for different kinds of pharmacists to participate in CE activities.
文摘The network security analyzers use intrusion detection systems(IDSes)to distinguish malicious traffic from benign ones.The deep learning-based(DL-based)IDSes are proposed to auto-extract high-level features and eliminate the time-consuming and costly signature extraction process.However,this new generation of IDSes still needs to overcome a number of challenges to be employed in practical environments.One of the main issues of an applicable IDS is facing traffic concept drift,which manifests itself as new(i.e.,zero-day)attacks,in addition to the changing behavior of benign users/applications.Furthermore,a practical DL-based IDS needs to be conformed to a distributed(i.e.,multi-sensor)architecture in order to yield more accurate detections,create a collective attack knowledge based on the observations of different sensors,and also handle big data challenges for supporting high throughput networks.This paper proposes a novel multi-agent network intrusion detection framework to address the above shortcomings,considering a more practical scenario(i.e.,online adaptable IDSes).This framework employs continual deep anomaly detectors for adapting each agent to the changing attack/benign patterns in its local traffic.In addition,a federated learning approach is proposed for sharing and exchanging local knowledge between different agents.Furthermore,the proposed framework implements sequential packet labeling for each flow,which provides an attack probability score for the flow by gradually observing each flow packet and updating its estimation.We evaluate the proposed framework by employing different deep models(including CNN-based and LSTM-based)over the CICIDS2017 and CSE-CIC-IDS2018 datasets.Through extensive evaluations and experiments,we show that the proposed distributed framework is well adapted to the traffic concept drift.More precisely,our results indicate that the CNNbased models are well suited for continually adapting to the traffic concept drift(i.e.,achieving an average detection rate of above 95%while needing just 128 new flows for the updating phase),and the LSTM-based models are a good candidate for sequential packet labeling in practical online IDSes(i.e.,detecting intrusions by just observing their first 15 packets).
基金Project supported by the National Natural Science Foundation of China(No.62272411)the Key R&D Projects in Zhejiang Province(Nos.2024C01106 and 2025C01030)the Zhejiang Natural Science Foundation(No.LRG25F020001)。
文摘Continual learning(CL)has emerged as a crucial paradigm for learning from sequential data while retaining previous knowledge.Continual graph learning(CGL),characterized by dynamically evolving graphs from streaming data,presents distinct challenges that demand efficient algorithms to prevent catastrophic forgetting.The first challenge stems from the interdependencies between different graph data,in which previous graphs infuence new data distributions.The second challenge is handling large graphs in an efficient manner.To address these challenges,we propose an eficient continual graph learner(E-CGL)in this paper.We address the interdependence issue by demonstrating the effectiveness of replay strategies and introducing a combined sampling approach that considers both node importance and diversity.To improve efficiency,E-CGL leverages a simple yet effective multilayer perceptron(MLP)model that shares weights with a graph neural network(GNN)during training,thereby accelerating computation by circumventing the expensive message-passing process.Our method achieves state-ofthe-art results on four CGL datasets under two settings,while significantly lowering the catastrophic forgetting value to an average of-1.1%.Additionally,E-CGL achieves the training and inference speedup by an average of 15.83x and 4.89x,respectively,across four datasets.These results indicate that E-CGL not only effectively manages correlations between different graph data during continual training but also enhances efficiency in large-scale CGL.
文摘The automatic colorization of anime line drawings is a challenging problem in production pipelines.Recent advances in deep neural networks have addressed this problem;however,collectingmany images of colorization targets in novel anime work before the colorization process starts leads to chicken-and-egg problems and has become an obstacle to using them in production pipelines.To overcome this obstacle,we propose a new patch-based learning method for few-shot anime-style colorization.The learning method adopts an efficient patch sampling technique with position embedding according to the characteristics of anime line drawings.We also present a continuous learning strategy that continuously updates our colorization model using new samples colorized by human artists.The advantage of our method is that it can learn our colorization model from scratch or pre-trained weights using only a few pre-and post-colorized line drawings that are created by artists in their usual colorization work.Therefore,our method can be easily incorporated within existing production pipelines.We quantitatively demonstrate that our colorizationmethod outperforms state-of-the-art methods.
基金Supported by the National Key Research and Development Program of China(No.2022ZD0115503).
文摘Modelling,predicting and analysing driver behaviours are essential to advanced driver assistance systems(ADAS)and the comprehensive understanding of complex driving scenarios.Recently,with the development of deep learning(DL),numerous driver behaviour learning(DBL)methods have been proposed and applied in connected vehicles(CV)and intelligent transportation systems(ITS).This study provides a review of DBL,which mainly focuses on typical applications in CV and ITS.First,a comprehensive review of the state-of-the-art DBL is presented.Next,Given the constantly changing nature of real driving scenarios,most existing learning-based models may suffer from the so-called“catastrophic forgetting,”which refers to their inability to perform well in previously learned scenarios after acquiring new ones.As a solution to the aforementioned issue,this paper presents a framework for continual driver behaviour learning(CDBL)by leveraging continual learning technology.The proposed CDBL framework is demonstrated to outperform existing methods in behaviour prediction through a case study.Finally,future works,potential challenges and emerging trends in this area are highlighted.
基金supported by the Chinese Ministry of Education Project for the Key Research Institute of Humanities and Social Sciences at Universities,entitled"Cross-Boundary Curriculum Partnerships Between Schooling and Shadow Education"(Project number:22JJD880028).
文摘Purpose:During Japan's initial pandemic prevention and control period,the Ministry of Education,Culture,Sports,Science,and Technology of Japan(MEXT)issued several notifications to ensure students'continuous learning according to the"leave no one behind"philosophy.This study focused on the comprehensive measures comprising top-level arrangements to implement the"suspending classes without stopping learning"initiative.Design/Approach/Methods:The study reviewed MEXT's guide on ensuring learning for Japanese primary and secondary school students during the pandemic.Findings:The pandemic has accelerated the development of the information and communications technology educational environment and the implementation of the Global and Innovation Gateway for All(GIGA)schools initiative.These developments laid the foundation for the evolution of diverse pedagogical models and feasible methods for promoting equity.The home schooling experience during the pandemic provided a natural base for students to practice and foster survivability.Originality/Value:This paper provides scholars with an understanding of the opportunities and challenges encountered in educational innovation in Japan.It provides insights into the future direction of pedagogical development,capability development,and effective educational arrangements for extraordinary circumstances to facilitate educational development from a broader perspective,particularly under the new normal.
基金supported by the National Science and Technology Major Project(2022ZD0120203).
文摘1 Introduction Unsupervised Federated Continual Learning(UFCL)is a new learning paradigm that embeds unsupervised representation techniques into the Federated Learning(FL)framework,which enables continuous training of a shared representation model without compromising individual participants’data privacy[1,2].However,the continuous learning process may cause catastrophic forgetting in the model,reducing generated representations’performance.
基金supported by the Beijing Natural Science Foundation,China(Grant/Award Number:Z210003)the National Nature Science Foundation of China(NSFC)(Grant/Award Numbers:12071313,61827809)+1 种基金the key research project of the Academy for Multidisciplinary Studies,Capital Normal University,China,the National Key Research and Development Program of China(Grant/Award Number:2020YFA0712200)the Major Technologies R&D Program of Shenzhen,China(JSGGZD20220822095600001).
文摘Recently,the catastrophic forgetting problem of neural networks in the process of continual learning(CL)has attracted more and more attention with the development of deep learning.The orthogonal weight modification(OWM)algorithm to some extent overcomes the catastrophic forgetting problem in CL.It is well-known that the mapping rule learned by the network is usually not accurate in the early stage of neural network training.Our main idea is to establish an immune mechanism in CL,which rejects unreliable mapping rules at the beginning of the training until those are reliable enough.Our algorithm showed a very good competitive advantage in the permuted and disjoint MNIST tasks and disjoint CIFAR-10 tasks.As for the more challenging task of Chinese handwriting character recognition,our algorithm showed a notable improvement compared with the OWM algorithm.In view of the context-dependent processing(CDP)module in[37],we revealed that the module may result in a loss of information and we proposed a modified CDP module to overcome this weakness.The performance of the system with the modified CDP module outperforms the original one in the CelebFaces attributed recognition task.Besides continual multi-task,we also considered a single task,where the immunity-based OWM(IOWM)algorithm was designed as an optimization solver of neural networks for low-dose computed tomography(CT)denoising task.
基金This work was supported by the National Natural Science Foundation of China(No.U1866206).
文摘The power market is a typical imperfectly competitive market where power suppliers gain higher profits through strategic bidding behaviors.Most existing studies assume that a power supplier is accessible to the sufficient market information to derive an optimal bidding strategy.However,this assumption may not be true in reality,particularly when a power market is newly launched.To help power suppliers bid with the limited information,a modified continuous action reinforcement learning automata algorithm is proposed.This algorithm introduces the discretization and Dyna structure into continuous action reinforcement learning automata algorithm for easy implementation in a repeated game.Simulation results verify the effectiveness of the proposed learning algorithm.
