Studies show that Graph Neural Networks(GNNs)are susceptible to minor perturbations.Therefore,analyzing adversarial attacks on GNNs is crucial in current research.Previous studies used Generative Adversarial Networks ...Studies show that Graph Neural Networks(GNNs)are susceptible to minor perturbations.Therefore,analyzing adversarial attacks on GNNs is crucial in current research.Previous studies used Generative Adversarial Networks to generate a set of fake nodes,injecting them into a clean GNNs to poison the graph structure and evaluate the robustness of GNNs.In the attack process,the computation of new node connections and the attack loss are independent,which affects the attack on the GNN.To improve this,a Fake Node Camouflage Attack based on Mutual Information(FNCAMI)algorithm is proposed.By incorporating Mutual Information(MI)loss,the distribution of nodes injected into the GNNs become more similar to the original nodes,achieving better attack results.Since the loss ratios of GNNs and MI affect performance,we also design an adaptive weighting method.By adjusting the loss weights in real-time through rate changes,larger loss values are obtained,eliminating local optima.The feasibility,effectiveness,and stealthiness of this algorithm are validated on four real datasets.Additionally,we use both global and targeted attacks to test the algorithm’s performance.Comparisons with baseline attack algorithms and ablation experiments demonstrate the efficiency of the FNCAMI algorithm.展开更多
The defense techniques for machine learning are critical yet challenging due tothe number and type of attacks for widely applied machine learning algorithms aresignificantly increasing. Among these attacks, the poison...The defense techniques for machine learning are critical yet challenging due tothe number and type of attacks for widely applied machine learning algorithms aresignificantly increasing. Among these attacks, the poisoning attack, which disturbsmachine learning algorithms by injecting poisoning samples, is an attack with the greatestthreat. In this paper, we focus on analyzing the characteristics of positioning samples andpropose a novel sample evaluation method to defend against the poisoning attack cateringfor the characteristics of poisoning samples. To capture the intrinsic data characteristicsfrom heterogeneous aspects, we first evaluate training data by multiple criteria, each ofwhich is reformulated from a spectral clustering. Then, we integrate the multipleevaluation scores generated by the multiple criteria through the proposed multiplespectral clustering aggregation (MSCA) method. Finally, we use the unified score as theindicator of poisoning attack samples. Experimental results on intrusion detection datasets show that MSCA significantly outperforms the K-means outlier detection in terms ofdata legality evaluation and poisoning attack detection.展开更多
The development of Intelligent Railway Transportation Systems necessitates incorporating privacy-preserving mechanisms into AI models to protect sensitive information and enhance system efficiency.Federated learning o...The development of Intelligent Railway Transportation Systems necessitates incorporating privacy-preserving mechanisms into AI models to protect sensitive information and enhance system efficiency.Federated learning offers a promising solution by allowing multiple clients to train models collaboratively without sharing private data.However,despite its privacy benefits,federated learning systems are vulnerable to poisoning attacks,where adversaries alter local model parameters on compromised clients and send malicious updates to the server,potentially compromising the global model’s accuracy.In this study,we introduce PMM(Perturbation coefficient Multiplied by Maximum value),a new poisoning attack method that perturbs model updates layer by layer,demonstrating the threat of poisoning attacks faced by federated learning.Extensive experiments across three distinct datasets have demonstrated PMM’s ability to significantly reduce the global model’s accuracy.Additionally,we propose an effective defense method,namely CLBL(Cluster Layer By Layer).Experiment results on three datasets have confirmed CLBL’s effectiveness.展开更多
Bayesian networks are a powerful class of graphical decision models used to represent causal relationships among variables.However,the reliability and integrity of learned Bayesian network models are highly dependent ...Bayesian networks are a powerful class of graphical decision models used to represent causal relationships among variables.However,the reliability and integrity of learned Bayesian network models are highly dependent on the quality of incoming data streams.One of the primary challenges with Bayesian networks is their vulnerability to adversarial data poisoning attacks,wherein malicious data is injected into the training dataset to negatively influence the Bayesian network models and impair their performance.In this research paper,we propose an efficient framework for detecting data poisoning attacks against Bayesian network structure learning algorithms.Our framework utilizes latent variables to quantify the amount of belief between every two nodes in each causal model over time.We use our innovative methodology to tackle an important issue with data poisoning assaults in the context of Bayesian networks.With regard to four different forms of data poisoning attacks,we specifically aim to strengthen the security and dependability of Bayesian network structure learning techniques,such as the PC algorithm.By doing this,we explore the complexity of this area and offer workablemethods for identifying and reducing these sneaky dangers.Additionally,our research investigates one particular use case,the“Visit to Asia Network.”The practical consequences of using uncertainty as a way to spot cases of data poisoning are explored in this inquiry,which is of utmost relevance.Our results demonstrate the promising efficacy of latent variables in detecting and mitigating the threat of data poisoning attacks.Additionally,our proposed latent-based framework proves to be sensitive in detecting malicious data poisoning attacks in the context of stream data.展开更多
Over the past years,the emergence of intelligent networks empowered by machine learning techniques has brought great facilitates to different aspects of human life.However,using machine learning in intelligent network...Over the past years,the emergence of intelligent networks empowered by machine learning techniques has brought great facilitates to different aspects of human life.However,using machine learning in intelligent networks also presents potential security and privacy threats.A common practice is the so-called poisoning attacks where malicious users inject fake training data with the aim of corrupting the learned model.In this survey,we comprehensively review existing poisoning attacks as well as the countermeasures in intelligent networks for the first time.We emphasize and compare the principles of the formal poisoning attacks employed in different categories of learning algorithms,and analyze the strengths and limitations of corresponding defense methods in a compact form.We also highlight some remaining challenges and future directions in the attack-defense confrontation to promote further research in this emerging yet promising area.展开更多
Machine Learning(ML)systems often involve a re-training process to make better predictions and classifications.This re-training process creates a loophole and poses a security threat for ML systems.Adversaries leverag...Machine Learning(ML)systems often involve a re-training process to make better predictions and classifications.This re-training process creates a loophole and poses a security threat for ML systems.Adversaries leverage this loophole and design data poisoning attacks against ML systems.Data poisoning attacks are a type of attack in which an adversary manipulates the training dataset to degrade the ML system’s performance.Data poisoning attacks are challenging to detect,and even more difficult to respond to,particularly in the Internet of Things(IoT)environment.To address this problem,we proposed DISTINIT,the first proactive data poisoning attack detection framework using distancemeasures.We found that Jaccard Distance(JD)can be used in the DISTINIT(among other distance measures)and we finally improved the JD to attain an Optimized JD(OJD)with lower time and space complexity.Our security analysis shows that the DISTINIT is secure against data poisoning attacks by considering key features of adversarial attacks.We conclude that the proposed OJD-based DISTINIT is effective and efficient against data poisoning attacks where in-time detection is critical for IoT applications with large volumes of streaming data.展开更多
In recent years,we have witnessed a surge in mobile devices such as smartphones,tablets,smart watches,etc.,most of which are based on the Android operating system.However,because these Android-based mobile devices are...In recent years,we have witnessed a surge in mobile devices such as smartphones,tablets,smart watches,etc.,most of which are based on the Android operating system.However,because these Android-based mobile devices are becoming increasingly popular,they are now the primary target of mobile malware,which could lead to both privacy leakage and property loss.To address the rapidly deteriorating security issues caused by mobile malware,various research efforts have been made to develop novel and effective detection mechanisms to identify and combat them.Nevertheless,in order to avoid being caught by these malware detection mechanisms,malware authors are inclined to initiate adversarial example attacks by tampering with mobile applications.In this paper,several types of adversarial example attacks are investigated and a feasible approach is proposed to fight against them.First,we look at adversarial example attacks on the Android system and prior solutions that have been proposed to address these attacks.Then,we specifically focus on the data poisoning attack and evasion attack models,which may mutate various application features,such as API calls,permissions and the class label,to produce adversarial examples.Then,we propose and design a malware detection approach that is resistant to adversarial examples.To observe and investigate how the malware detection system is influenced by the adversarial example attacks,we conduct experiments on some real Android application datasets which are composed of both malware and benign applications.Experimental results clearly indicate that the performance of Android malware detection is severely degraded when facing adversarial example attacks.展开更多
The security of Federated Learning(FL)/Distributed Machine Learning(DML)is gravely threatened by data poisoning attacks,which destroy the usability of the model by contaminating training samples,so such attacks are ca...The security of Federated Learning(FL)/Distributed Machine Learning(DML)is gravely threatened by data poisoning attacks,which destroy the usability of the model by contaminating training samples,so such attacks are called causative availability indiscriminate attacks.Facing the problem that existing data sanitization methods are hard to apply to real-time applications due to their tedious process and heavy computations,we propose a new supervised batch detection method for poison,which can fleetly sanitize the training dataset before the local model training.We design a training dataset generation method that helps to enhance accuracy and uses data complexity features to train a detection model,which will be used in an efficient batch hierarchical detection process.Our model stockpiles knowledge about poison,which can be expanded by retraining to adapt to new attacks.Being neither attack-specific nor scenario-specific,our method is applicable to FL/DML or other online or offline scenarios.展开更多
在物联网(Internet of Things,IoT)环境中,联邦学习由于其无需集中存储数据即可实现模型训练的特点被广泛应用于隐私保护和分布式计算.然而,物联网设备的分布式特性和多样化的安全需求使得联邦学习系统容易受到数据中毒攻击,攻击者可能...在物联网(Internet of Things,IoT)环境中,联邦学习由于其无需集中存储数据即可实现模型训练的特点被广泛应用于隐私保护和分布式计算.然而,物联网设备的分布式特性和多样化的安全需求使得联邦学习系统容易受到数据中毒攻击,攻击者可能通过上传恶意梯度来干扰全局模型的训练过程,从而威胁系统的安全性.尽管已有多种防御策略针对数据中毒攻击,但如何在保护隐私的同时确保系统对这些攻击的鲁棒性仍然是一个挑战.本文提出了一种新的名为DPI的防御方案,旨在解决这一问题.DPI通过设计一种无损聚合方案,有效地检测并隔离恶意梯度,避免了参与者实际梯度泄露.具体而言,DPI首先为每个参与者的梯度应用提供可移动的掩码,然后通过奇异值分解对掩码后的数据进行聚合与降维.接着,采用聚类算法从低维数据中检测并剔除中毒梯度.大量实验结果表明,DPI在检测有毒梯度方面表现优异,相较于现有最先进的方法,能够在隐私保护和数据安全方面实现更好的平衡,提升了联邦学习系统在物联网环境中的鲁棒性.DPI不仅能够有效应对中毒攻击,还能确保参与者的梯度隐私不被泄露,满足物联网场景下的安全需求.展开更多
基金supported by the Natural Science Basic Research Plan in Shaanxi Province of China(Program No.2022JM-381,2017JQ6070)National Natural Science Foundation of China(Grant No.61703256),Foundation of State Key Laboratory of Public Big Data(No.PBD2022-08)the Fundamental Research Funds for the Central Universities,China(Program No.GK202201014,GK202202003,GK201803020).
文摘Studies show that Graph Neural Networks(GNNs)are susceptible to minor perturbations.Therefore,analyzing adversarial attacks on GNNs is crucial in current research.Previous studies used Generative Adversarial Networks to generate a set of fake nodes,injecting them into a clean GNNs to poison the graph structure and evaluate the robustness of GNNs.In the attack process,the computation of new node connections and the attack loss are independent,which affects the attack on the GNN.To improve this,a Fake Node Camouflage Attack based on Mutual Information(FNCAMI)algorithm is proposed.By incorporating Mutual Information(MI)loss,the distribution of nodes injected into the GNNs become more similar to the original nodes,achieving better attack results.Since the loss ratios of GNNs and MI affect performance,we also design an adaptive weighting method.By adjusting the loss weights in real-time through rate changes,larger loss values are obtained,eliminating local optima.The feasibility,effectiveness,and stealthiness of this algorithm are validated on four real datasets.Additionally,we use both global and targeted attacks to test the algorithm’s performance.Comparisons with baseline attack algorithms and ablation experiments demonstrate the efficiency of the FNCAMI algorithm.
文摘The defense techniques for machine learning are critical yet challenging due tothe number and type of attacks for widely applied machine learning algorithms aresignificantly increasing. Among these attacks, the poisoning attack, which disturbsmachine learning algorithms by injecting poisoning samples, is an attack with the greatestthreat. In this paper, we focus on analyzing the characteristics of positioning samples andpropose a novel sample evaluation method to defend against the poisoning attack cateringfor the characteristics of poisoning samples. To capture the intrinsic data characteristicsfrom heterogeneous aspects, we first evaluate training data by multiple criteria, each ofwhich is reformulated from a spectral clustering. Then, we integrate the multipleevaluation scores generated by the multiple criteria through the proposed multiplespectral clustering aggregation (MSCA) method. Finally, we use the unified score as theindicator of poisoning attack samples. Experimental results on intrusion detection datasets show that MSCA significantly outperforms the K-means outlier detection in terms ofdata legality evaluation and poisoning attack detection.
基金supported by Systematic Major Project of China State Railway Group Corporation Limited(Grant Number:P2023W002).
文摘The development of Intelligent Railway Transportation Systems necessitates incorporating privacy-preserving mechanisms into AI models to protect sensitive information and enhance system efficiency.Federated learning offers a promising solution by allowing multiple clients to train models collaboratively without sharing private data.However,despite its privacy benefits,federated learning systems are vulnerable to poisoning attacks,where adversaries alter local model parameters on compromised clients and send malicious updates to the server,potentially compromising the global model’s accuracy.In this study,we introduce PMM(Perturbation coefficient Multiplied by Maximum value),a new poisoning attack method that perturbs model updates layer by layer,demonstrating the threat of poisoning attacks faced by federated learning.Extensive experiments across three distinct datasets have demonstrated PMM’s ability to significantly reduce the global model’s accuracy.Additionally,we propose an effective defense method,namely CLBL(Cluster Layer By Layer).Experiment results on three datasets have confirmed CLBL’s effectiveness.
文摘Bayesian networks are a powerful class of graphical decision models used to represent causal relationships among variables.However,the reliability and integrity of learned Bayesian network models are highly dependent on the quality of incoming data streams.One of the primary challenges with Bayesian networks is their vulnerability to adversarial data poisoning attacks,wherein malicious data is injected into the training dataset to negatively influence the Bayesian network models and impair their performance.In this research paper,we propose an efficient framework for detecting data poisoning attacks against Bayesian network structure learning algorithms.Our framework utilizes latent variables to quantify the amount of belief between every two nodes in each causal model over time.We use our innovative methodology to tackle an important issue with data poisoning assaults in the context of Bayesian networks.With regard to four different forms of data poisoning attacks,we specifically aim to strengthen the security and dependability of Bayesian network structure learning techniques,such as the PC algorithm.By doing this,we explore the complexity of this area and offer workablemethods for identifying and reducing these sneaky dangers.Additionally,our research investigates one particular use case,the“Visit to Asia Network.”The practical consequences of using uncertainty as a way to spot cases of data poisoning are explored in this inquiry,which is of utmost relevance.Our results demonstrate the promising efficacy of latent variables in detecting and mitigating the threat of data poisoning attacks.Additionally,our proposed latent-based framework proves to be sensitive in detecting malicious data poisoning attacks in the context of stream data.
基金This work was supported in part by the National Natural Science Foundation of China under Grants 62002104 and 61872416the Natural Science Foundation of Hubei Province of China under Grant 2019CFB191the special fund for Wuhan Yellow Crane Talents(Excellent Young Scholar).
文摘Over the past years,the emergence of intelligent networks empowered by machine learning techniques has brought great facilitates to different aspects of human life.However,using machine learning in intelligent networks also presents potential security and privacy threats.A common practice is the so-called poisoning attacks where malicious users inject fake training data with the aim of corrupting the learned model.In this survey,we comprehensively review existing poisoning attacks as well as the countermeasures in intelligent networks for the first time.We emphasize and compare the principles of the formal poisoning attacks employed in different categories of learning algorithms,and analyze the strengths and limitations of corresponding defense methods in a compact form.We also highlight some remaining challenges and future directions in the attack-defense confrontation to promote further research in this emerging yet promising area.
基金This work was supported by a National Research Foundation of Korea(NRF)grant funded by the Korea Government(MSIT)under Grant 2020R1A2B5B01002145.
文摘Machine Learning(ML)systems often involve a re-training process to make better predictions and classifications.This re-training process creates a loophole and poses a security threat for ML systems.Adversaries leverage this loophole and design data poisoning attacks against ML systems.Data poisoning attacks are a type of attack in which an adversary manipulates the training dataset to degrade the ML system’s performance.Data poisoning attacks are challenging to detect,and even more difficult to respond to,particularly in the Internet of Things(IoT)environment.To address this problem,we proposed DISTINIT,the first proactive data poisoning attack detection framework using distancemeasures.We found that Jaccard Distance(JD)can be used in the DISTINIT(among other distance measures)and we finally improved the JD to attain an Optimized JD(OJD)with lower time and space complexity.Our security analysis shows that the DISTINIT is secure against data poisoning attacks by considering key features of adversarial attacks.We conclude that the proposed OJD-based DISTINIT is effective and efficient against data poisoning attacks where in-time detection is critical for IoT applications with large volumes of streaming data.
文摘In recent years,we have witnessed a surge in mobile devices such as smartphones,tablets,smart watches,etc.,most of which are based on the Android operating system.However,because these Android-based mobile devices are becoming increasingly popular,they are now the primary target of mobile malware,which could lead to both privacy leakage and property loss.To address the rapidly deteriorating security issues caused by mobile malware,various research efforts have been made to develop novel and effective detection mechanisms to identify and combat them.Nevertheless,in order to avoid being caught by these malware detection mechanisms,malware authors are inclined to initiate adversarial example attacks by tampering with mobile applications.In this paper,several types of adversarial example attacks are investigated and a feasible approach is proposed to fight against them.First,we look at adversarial example attacks on the Android system and prior solutions that have been proposed to address these attacks.Then,we specifically focus on the data poisoning attack and evasion attack models,which may mutate various application features,such as API calls,permissions and the class label,to produce adversarial examples.Then,we propose and design a malware detection approach that is resistant to adversarial examples.To observe and investigate how the malware detection system is influenced by the adversarial example attacks,we conduct experiments on some real Android application datasets which are composed of both malware and benign applications.Experimental results clearly indicate that the performance of Android malware detection is severely degraded when facing adversarial example attacks.
基金supported in part by the“Pioneer”and“Leading Goose”R&D Program of Zhejiang(Grant No.2022C03174)the National Natural Science Foundation of China(No.92067103)+4 种基金the Key Research and Development Program of Shaanxi,China(No.2021ZDLGY06-02)the Natural Science Foundation of Shaanxi Province(No.2019ZDLGY12-02)the Shaanxi Innovation Team Project(No.2018TD-007)the Xi'an Science and technology Innovation Plan(No.201809168CX9JC10)the Fundamental Research Funds for the Central Universities(No.YJS2212)and National 111 Program of China B16037.
文摘The security of Federated Learning(FL)/Distributed Machine Learning(DML)is gravely threatened by data poisoning attacks,which destroy the usability of the model by contaminating training samples,so such attacks are called causative availability indiscriminate attacks.Facing the problem that existing data sanitization methods are hard to apply to real-time applications due to their tedious process and heavy computations,we propose a new supervised batch detection method for poison,which can fleetly sanitize the training dataset before the local model training.We design a training dataset generation method that helps to enhance accuracy and uses data complexity features to train a detection model,which will be used in an efficient batch hierarchical detection process.Our model stockpiles knowledge about poison,which can be expanded by retraining to adapt to new attacks.Being neither attack-specific nor scenario-specific,our method is applicable to FL/DML or other online or offline scenarios.
文摘在物联网(Internet of Things,IoT)环境中,联邦学习由于其无需集中存储数据即可实现模型训练的特点被广泛应用于隐私保护和分布式计算.然而,物联网设备的分布式特性和多样化的安全需求使得联邦学习系统容易受到数据中毒攻击,攻击者可能通过上传恶意梯度来干扰全局模型的训练过程,从而威胁系统的安全性.尽管已有多种防御策略针对数据中毒攻击,但如何在保护隐私的同时确保系统对这些攻击的鲁棒性仍然是一个挑战.本文提出了一种新的名为DPI的防御方案,旨在解决这一问题.DPI通过设计一种无损聚合方案,有效地检测并隔离恶意梯度,避免了参与者实际梯度泄露.具体而言,DPI首先为每个参与者的梯度应用提供可移动的掩码,然后通过奇异值分解对掩码后的数据进行聚合与降维.接着,采用聚类算法从低维数据中检测并剔除中毒梯度.大量实验结果表明,DPI在检测有毒梯度方面表现优异,相较于现有最先进的方法,能够在隐私保护和数据安全方面实现更好的平衡,提升了联邦学习系统在物联网环境中的鲁棒性.DPI不仅能够有效应对中毒攻击,还能确保参与者的梯度隐私不被泄露,满足物联网场景下的安全需求.
