Cardiovascular diseases(CVDs)continue to present a leading cause ofmortalityworldwide,emphasizing the importance of early and accurate prediction.Electrocardiogram(ECG)signals,central to cardiac monitoring,have increa...Cardiovascular diseases(CVDs)continue to present a leading cause ofmortalityworldwide,emphasizing the importance of early and accurate prediction.Electrocardiogram(ECG)signals,central to cardiac monitoring,have increasingly been integratedwithDeep Learning(DL)for real-time prediction of CVDs.However,DL models are prone to performance degradation due to concept drift and to catastrophic forgetting.To address this issue,we propose a realtime CVDs prediction approach,referred to as ADWIN-GFR that combines Convolutional Neural Network(CNN)layers,for spatial feature extraction,with Gated Recurrent Units(GRU),for temporal modeling,alongside adaptive drift detection and mitigation mechanisms.The proposed approach integratesAdaptiveWindowing(ADWIN)for realtime concept drift detection,a fine-tuning strategy based on Generative Features Replay(GFR)to preserve previously acquired knowledge,and a dynamic replay buffer ensuring variance,diversity,and data distribution coverage.Extensive experiments conducted on the MIT-BIH arrhythmia dataset demonstrate that ADWIN-GFR outperforms standard fine-tuning techniques,achieving an average post-drift accuracy of 95.4%,amacro F1-score of 93.9%,and a remarkably low forgetting score of 0.9%.It also exhibits an average drift detection delay of 12 steps and achieves an adaptation gain of 17.2%.These findings underscore the potential of ADWIN-GFR for deployment in real-world cardiac monitoring systems,including wearable ECG devices and hospital-based patient monitoring platforms.展开更多
Many real⁃world machine learning applications face the challenge of dealing with changing data over time,known as concept drift,and the issue of data indeterminacy,where all the true labels available are unrealistic.T...Many real⁃world machine learning applications face the challenge of dealing with changing data over time,known as concept drift,and the issue of data indeterminacy,where all the true labels available are unrealistic.This can lead to a decrease in the accuracy of the prediction models.The aim of this study is to introduce a new approach for detecting drift,which is based on neutrosophic set theory.This approach takes into account uncertainty in the prediction model and is able to handle indeterminate information,considering its impact on the models performance.The proposed method reads data into windows and calculates a set of values based on the concept of neutrosophic membership.These values are then used in the Neutrosophic Support Vector Machine(N⁃SVM).To address the issue of indeterminate true label data,the values issued by N⁃SVM are expressed as entropy and used as input for the ADWIN(Adaptive Windowing)change detector.When a drift is detected,the prediction model is retrained by including only the most recent instances with the original training data set.The proposed method gives promising results in terms of drift detection accuracy compared to the state of existing drift detection methods such as KSWIN,ADWIN,and DWM.展开更多
Ever since the research in machine learning gained traction in recent years,it has been employed to address challenges in a wide variety of domains,including mechanical devices.Most of the machine learning models are ...Ever since the research in machine learning gained traction in recent years,it has been employed to address challenges in a wide variety of domains,including mechanical devices.Most of the machine learning models are built on the assumption of a static learning environment,but in practical situations,the data generated by the process is dynamic.This evolution of the data is termed concept drift.This research paper presents an approach for predictingmechanical failure in real-time using incremental learning based on the statistically calculated parameters of mechanical equipment.The method proposed here is applicable to allmechanical devices that are susceptible to failure or operational degradation.The proposed method in this paper is equipped with the capacity to detect the drift in data generation and adaptation.The proposed approach evaluates the machine learning and deep learning models for their efficacy in handling the errors related to industrial machines due to their dynamic nature.It is observed that,in the settings without concept drift in the data,methods like SVM and Random Forest performed better compared to deep neural networks.However,this resulted in poor sensitivity for the smallest drift in the machine data reported as a drift.In this perspective,DNN generated the stable drift detection method;it reported an accuracy of 84%and an AUC of 0.87 while detecting only a single drift point,indicating the stability to performbetter in detecting and adapting to new data in the drifting environments under industrial measurement settings.展开更多
Cybersecurity increasingly relies on machine learning(ML)models to respond to and detect attacks.However,the rapidly changing data environment makes model life-cycle management after deployment essential.Real-time det...Cybersecurity increasingly relies on machine learning(ML)models to respond to and detect attacks.However,the rapidly changing data environment makes model life-cycle management after deployment essential.Real-time detection of drift signals from various threats is fundamental for effectively managing deployed models.However,detecting drift in unsupervised environments can be challenging.This study introduces a novel approach leveraging Shapley additive explanations(SHAP),a widely recognized explainability technique in ML,to address drift detection in unsupervised settings.The proposed method incorporates a range of plots and statistical techniques to enhance drift detection reliability and introduces a drift suspicion metric that considers the explanatory aspects absent in the current approaches.To validate the effectiveness of the proposed approach in a real-world scenario,we applied it to an environment designed to detect domain generation algorithms(DGAs).The dataset was obtained from various types of DGAs provided by NetLab.Based on this dataset composition,we sought to validate the proposed SHAP-based approach through drift scenarios that occur when a previously deployed model detects new data types in an environment that detects real-world DGAs.The results revealed that more than 90%of the drift data exceeded the threshold,demonstrating the high reliability of the approach to detect drift in an unsupervised environment.The proposed method distinguishes itself fromexisting approaches by employing explainable artificial intelligence(XAI)-based detection,which is not limited by model or system environment constraints.In conclusion,this paper proposes a novel approach to detect drift in unsupervised ML settings for cybersecurity.The proposed method employs SHAP-based XAI and a drift suspicion metric to improve drift detection reliability.It is versatile and suitable for various realtime data analysis contexts beyond DGA detection environments.This study significantly contributes to theMLcommunity by addressing the critical issue of managing ML models in real-world cybersecurity settings.Our approach is distinguishable from existing techniques by employing XAI-based detection,which is not limited by model or system environment constraints.As a result,our method can be applied in critical domains that require adaptation to continuous changes,such as cybersecurity.Through extensive validation across diverse settings beyond DGA detection environments,the proposed method will emerge as a versatile drift detection technique suitable for a wide range of real-time data analysis contexts.It is also anticipated to emerge as a new approach to protect essential systems and infrastructures from attacks.展开更多
Accurate energy consumption forecasting is crucial for reducing operational costs, achieving net-zero carbon emissions, and ensuring sustainable buildings and cities of the future. Despite the frequent use of Artifici...Accurate energy consumption forecasting is crucial for reducing operational costs, achieving net-zero carbon emissions, and ensuring sustainable buildings and cities of the future. Despite the frequent use of Artificial Intelligence (AI) algorithms for learning energy consumption patterns and predictions in Building Science, relying solely on these techniques for energy demand prediction addresses only a fraction of the challenge. A drift in energy usage can lead to inaccuracies in these AI models and subsequently to poor decision-making and interventions. While drift detection techniques have been reported, a reliable and robust approach capable of explaining identified discrepancies with actionable insights has not been discussed in extant literature. Hence, this paper presents an Artificial Intelligence framework for energy consumption forecasting with explainable drift detection, aimed at addressing these challenges. The proposed framework is composed of energy embeddings, an optimized dimensional model integrated within a data warehouse, and scalable cloud implementation for effective drift detection with explainability capability. The framework is empirically evaluated in the real-world setting of a multi-campus, mixed-use tertiary education setting in Victoria, Australia. The results of these experiments highlight its capabilities in detecting concept drift, adapting forecast predictions, and providing an interpretation of the changes using energy embeddings.展开更多
Purpose:This research addresses the challenge of concept drift in AI-enabled software,particularly within autonomous vehicle systems where concept drift in object recognition(like pedestrian detection)can lead to misc...Purpose:This research addresses the challenge of concept drift in AI-enabled software,particularly within autonomous vehicle systems where concept drift in object recognition(like pedestrian detection)can lead to misclassifications and safety risks.This study introduces a proactive framework to detect early signs of domain-specific concept drift by leveraging domain analysis and natural language processing techniques.This method is designed to help maintain the relevance of domain knowledge and prevent potential failures in AI systems due to evolving concept definitions.Design/methodology/approach:The proposed framework integrates natural language processing and image analysis to continuously update and monitor key domain concepts against evolving external data sources,such as social media and news.By identifying terms and features closely associated with core concepts,the system anticipates and flags significant changes.This was tested in the automotive domain on the pedestrian concept,where the framework was evaluated for its capacity to detect shifts in the recognition of pedestrians,particularly during events like Halloween and specific car accidents.Findings:The framework demonstrated an ability to detect shifts in the domain concept of pedestrians,as evidenced by contextual changes around major events.While it successfully identified pedestrian-related drift,the system’s accuracy varied when overlapping with larger social events.The results indicate the model’s potential to foresee relevant shifts before they impact autonomous systems,although further refinement is needed to handle high-impact concurrent events.Research limitations:This study focused on detecting concept drift in the pedestrian domain within autonomous vehicles,with results varying across domains.To assess generalizability,we tested the framework for airplane-related incidents and demonstrated adaptability.However,unpredictable events and data biases from social media and news may obscure domain-specific drifts.Further evaluation across diverse applications is needed to enhance robustness in evolving AI environments.Practical implications:The proactive detection of concept drift has significant implications for AI-driven domains,especially in safety-critical applications like autonomous driving.By identifying early signs of drift,this framework provides actionable insights for AI system updates,potentially reducing misclassification risks and enhancing public safety.Moreover,it enables timely interventions,reducing costly and labor-intensive retraining requirements by focusing only on the relevant aspects of evolving concepts.This method offers a streamlined approach for maintaining AI system performance in environments where domain knowledge rapidly changes.Originality/value:This study contributes a novel domain-agnostic framework that combines natural language processing with image analysis to predict concept drift early.This unique approach,which is focused on real-time data sources,offers an effective and scalable solution for addressing the evolving nature of domain-specific concepts in AI applications.展开更多
The packet loss classification has always been a hot and difficult issue in TCP congestion control research.Compared with the terrestrial network,the probability of packet loss in LEO satellite network increases drama...The packet loss classification has always been a hot and difficult issue in TCP congestion control research.Compared with the terrestrial network,the probability of packet loss in LEO satellite network increases dramatically.What’s more,the problem of concept drifting is also more serious,which greatly affects the accuracy of the loss classification model.In this paper,we propose a new loss classification scheme based on concept drift detection and hybrid integration learning for LEO satellite networks,named LDM-Satellite,which consists of three modules:concept drift detection,lost packet cache and hybrid integration classification.As far,this is the first paper to consider the influence of concept drift on the loss classification model in satellite networks.We also innovatively use multiple base classifiers and a naive Bayes classifier as the final hybrid classifier.And a new weight algorithm for these classifiers is given.In ns-2 simulation,LDM-Satellite has a better AUC(0.9885)than the single-model machine learning classification algorithms.The accuracy of loss classification even exceeds 98%,higher than traditional TCP protocols.Moreover,compared with the existing protocols used for satellite networks,LDM-Satellite not only improves the throughput rate but also has good fairness.展开更多
基金supported by Princess Nourah bint Abdulrahman University Researchers Supporting Project number(PNURSP2025R196)Princess Nourah bint Abdulrahman University,Riyadh,Saudi Arabia.
文摘Cardiovascular diseases(CVDs)continue to present a leading cause ofmortalityworldwide,emphasizing the importance of early and accurate prediction.Electrocardiogram(ECG)signals,central to cardiac monitoring,have increasingly been integratedwithDeep Learning(DL)for real-time prediction of CVDs.However,DL models are prone to performance degradation due to concept drift and to catastrophic forgetting.To address this issue,we propose a realtime CVDs prediction approach,referred to as ADWIN-GFR that combines Convolutional Neural Network(CNN)layers,for spatial feature extraction,with Gated Recurrent Units(GRU),for temporal modeling,alongside adaptive drift detection and mitigation mechanisms.The proposed approach integratesAdaptiveWindowing(ADWIN)for realtime concept drift detection,a fine-tuning strategy based on Generative Features Replay(GFR)to preserve previously acquired knowledge,and a dynamic replay buffer ensuring variance,diversity,and data distribution coverage.Extensive experiments conducted on the MIT-BIH arrhythmia dataset demonstrate that ADWIN-GFR outperforms standard fine-tuning techniques,achieving an average post-drift accuracy of 95.4%,amacro F1-score of 93.9%,and a remarkably low forgetting score of 0.9%.It also exhibits an average drift detection delay of 12 steps and achieves an adaptation gain of 17.2%.These findings underscore the potential of ADWIN-GFR for deployment in real-world cardiac monitoring systems,including wearable ECG devices and hospital-based patient monitoring platforms.
文摘Many real⁃world machine learning applications face the challenge of dealing with changing data over time,known as concept drift,and the issue of data indeterminacy,where all the true labels available are unrealistic.This can lead to a decrease in the accuracy of the prediction models.The aim of this study is to introduce a new approach for detecting drift,which is based on neutrosophic set theory.This approach takes into account uncertainty in the prediction model and is able to handle indeterminate information,considering its impact on the models performance.The proposed method reads data into windows and calculates a set of values based on the concept of neutrosophic membership.These values are then used in the Neutrosophic Support Vector Machine(N⁃SVM).To address the issue of indeterminate true label data,the values issued by N⁃SVM are expressed as entropy and used as input for the ADWIN(Adaptive Windowing)change detector.When a drift is detected,the prediction model is retrained by including only the most recent instances with the original training data set.The proposed method gives promising results in terms of drift detection accuracy compared to the state of existing drift detection methods such as KSWIN,ADWIN,and DWM.
文摘Ever since the research in machine learning gained traction in recent years,it has been employed to address challenges in a wide variety of domains,including mechanical devices.Most of the machine learning models are built on the assumption of a static learning environment,but in practical situations,the data generated by the process is dynamic.This evolution of the data is termed concept drift.This research paper presents an approach for predictingmechanical failure in real-time using incremental learning based on the statistically calculated parameters of mechanical equipment.The method proposed here is applicable to allmechanical devices that are susceptible to failure or operational degradation.The proposed method in this paper is equipped with the capacity to detect the drift in data generation and adaptation.The proposed approach evaluates the machine learning and deep learning models for their efficacy in handling the errors related to industrial machines due to their dynamic nature.It is observed that,in the settings without concept drift in the data,methods like SVM and Random Forest performed better compared to deep neural networks.However,this resulted in poor sensitivity for the smallest drift in the machine data reported as a drift.In this perspective,DNN generated the stable drift detection method;it reported an accuracy of 84%and an AUC of 0.87 while detecting only a single drift point,indicating the stability to performbetter in detecting and adapting to new data in the drifting environments under industrial measurement settings.
基金supported by the Institute of Information and Communications Technology Planning and Evaluation(IITP)grant funded by the Korean government(MSIT)(No.2022-0-00089,Development of clustering and analysis technology to identify cyber attack groups based on life cycle)the Institute of Civil Military Technology Cooperation funded by the Defense Acquisition Program Administration and Ministry of Trade,Industry and Energy of Korean government under Grant No.21-CM-EC-07.
文摘Cybersecurity increasingly relies on machine learning(ML)models to respond to and detect attacks.However,the rapidly changing data environment makes model life-cycle management after deployment essential.Real-time detection of drift signals from various threats is fundamental for effectively managing deployed models.However,detecting drift in unsupervised environments can be challenging.This study introduces a novel approach leveraging Shapley additive explanations(SHAP),a widely recognized explainability technique in ML,to address drift detection in unsupervised settings.The proposed method incorporates a range of plots and statistical techniques to enhance drift detection reliability and introduces a drift suspicion metric that considers the explanatory aspects absent in the current approaches.To validate the effectiveness of the proposed approach in a real-world scenario,we applied it to an environment designed to detect domain generation algorithms(DGAs).The dataset was obtained from various types of DGAs provided by NetLab.Based on this dataset composition,we sought to validate the proposed SHAP-based approach through drift scenarios that occur when a previously deployed model detects new data types in an environment that detects real-world DGAs.The results revealed that more than 90%of the drift data exceeded the threshold,demonstrating the high reliability of the approach to detect drift in an unsupervised environment.The proposed method distinguishes itself fromexisting approaches by employing explainable artificial intelligence(XAI)-based detection,which is not limited by model or system environment constraints.In conclusion,this paper proposes a novel approach to detect drift in unsupervised ML settings for cybersecurity.The proposed method employs SHAP-based XAI and a drift suspicion metric to improve drift detection reliability.It is versatile and suitable for various realtime data analysis contexts beyond DGA detection environments.This study significantly contributes to theMLcommunity by addressing the critical issue of managing ML models in real-world cybersecurity settings.Our approach is distinguishable from existing techniques by employing XAI-based detection,which is not limited by model or system environment constraints.As a result,our method can be applied in critical domains that require adaptation to continuous changes,such as cybersecurity.Through extensive validation across diverse settings beyond DGA detection environments,the proposed method will emerge as a versatile drift detection technique suitable for a wide range of real-time data analysis contexts.It is also anticipated to emerge as a new approach to protect essential systems and infrastructures from attacks.
基金supported by the Department of Climate Change,Energy,the Environment and Water of the Australian Federal Government,as part of the International Clean Innovation Researcher Networks(ICIRN)program,grant number ICIRN000077.
文摘Accurate energy consumption forecasting is crucial for reducing operational costs, achieving net-zero carbon emissions, and ensuring sustainable buildings and cities of the future. Despite the frequent use of Artificial Intelligence (AI) algorithms for learning energy consumption patterns and predictions in Building Science, relying solely on these techniques for energy demand prediction addresses only a fraction of the challenge. A drift in energy usage can lead to inaccuracies in these AI models and subsequently to poor decision-making and interventions. While drift detection techniques have been reported, a reliable and robust approach capable of explaining identified discrepancies with actionable insights has not been discussed in extant literature. Hence, this paper presents an Artificial Intelligence framework for energy consumption forecasting with explainable drift detection, aimed at addressing these challenges. The proposed framework is composed of energy embeddings, an optimized dimensional model integrated within a data warehouse, and scalable cloud implementation for effective drift detection with explainability capability. The framework is empirically evaluated in the real-world setting of a multi-campus, mixed-use tertiary education setting in Victoria, Australia. The results of these experiments highlight its capabilities in detecting concept drift, adapting forecast predictions, and providing an interpretation of the changes using energy embeddings.
基金supported by U.S.Office of Naval Research(ONR)Grant number G2A62826.
文摘Purpose:This research addresses the challenge of concept drift in AI-enabled software,particularly within autonomous vehicle systems where concept drift in object recognition(like pedestrian detection)can lead to misclassifications and safety risks.This study introduces a proactive framework to detect early signs of domain-specific concept drift by leveraging domain analysis and natural language processing techniques.This method is designed to help maintain the relevance of domain knowledge and prevent potential failures in AI systems due to evolving concept definitions.Design/methodology/approach:The proposed framework integrates natural language processing and image analysis to continuously update and monitor key domain concepts against evolving external data sources,such as social media and news.By identifying terms and features closely associated with core concepts,the system anticipates and flags significant changes.This was tested in the automotive domain on the pedestrian concept,where the framework was evaluated for its capacity to detect shifts in the recognition of pedestrians,particularly during events like Halloween and specific car accidents.Findings:The framework demonstrated an ability to detect shifts in the domain concept of pedestrians,as evidenced by contextual changes around major events.While it successfully identified pedestrian-related drift,the system’s accuracy varied when overlapping with larger social events.The results indicate the model’s potential to foresee relevant shifts before they impact autonomous systems,although further refinement is needed to handle high-impact concurrent events.Research limitations:This study focused on detecting concept drift in the pedestrian domain within autonomous vehicles,with results varying across domains.To assess generalizability,we tested the framework for airplane-related incidents and demonstrated adaptability.However,unpredictable events and data biases from social media and news may obscure domain-specific drifts.Further evaluation across diverse applications is needed to enhance robustness in evolving AI environments.Practical implications:The proactive detection of concept drift has significant implications for AI-driven domains,especially in safety-critical applications like autonomous driving.By identifying early signs of drift,this framework provides actionable insights for AI system updates,potentially reducing misclassification risks and enhancing public safety.Moreover,it enables timely interventions,reducing costly and labor-intensive retraining requirements by focusing only on the relevant aspects of evolving concepts.This method offers a streamlined approach for maintaining AI system performance in environments where domain knowledge rapidly changes.Originality/value:This study contributes a novel domain-agnostic framework that combines natural language processing with image analysis to predict concept drift early.This unique approach,which is focused on real-time data sources,offers an effective and scalable solution for addressing the evolving nature of domain-specific concepts in AI applications.
基金the Wireless Network Positioning and Communication Integration Research Center in BUPT for financial support
文摘The packet loss classification has always been a hot and difficult issue in TCP congestion control research.Compared with the terrestrial network,the probability of packet loss in LEO satellite network increases dramatically.What’s more,the problem of concept drifting is also more serious,which greatly affects the accuracy of the loss classification model.In this paper,we propose a new loss classification scheme based on concept drift detection and hybrid integration learning for LEO satellite networks,named LDM-Satellite,which consists of three modules:concept drift detection,lost packet cache and hybrid integration classification.As far,this is the first paper to consider the influence of concept drift on the loss classification model in satellite networks.We also innovatively use multiple base classifiers and a naive Bayes classifier as the final hybrid classifier.And a new weight algorithm for these classifiers is given.In ns-2 simulation,LDM-Satellite has a better AUC(0.9885)than the single-model machine learning classification algorithms.The accuracy of loss classification even exceeds 98%,higher than traditional TCP protocols.Moreover,compared with the existing protocols used for satellite networks,LDM-Satellite not only improves the throughput rate but also has good fairness.
