The emergence of adversarial examples has revealed the inadequacies in the robustness of image classification models based on Convolutional Neural Networks (CNNs). Particularly in recent years, the discovery of natura...The emergence of adversarial examples has revealed the inadequacies in the robustness of image classification models based on Convolutional Neural Networks (CNNs). Particularly in recent years, the discovery of natural adversarial examples has posed significant challenges, as traditional defense methods against adversarial attacks have proven to be largely ineffective against these natural adversarial examples. This paper explores defenses against these natural adversarial examples from three perspectives: adversarial examples, model architecture, and dataset. First, it employs Class Activation Mapping (CAM) to visualize how models classify natural adversarial examples, identifying several typical attack patterns. Next, various common CNN models are analyzed to evaluate their susceptibility to these attacks, revealing that different architectures exhibit varying defensive capabilities. The study finds that as the depth of a network increases, its defenses against natural adversarial examples strengthen. Lastly, Finally, the impact of dataset class distribution on the defense capability of models is examined, focusing on two aspects: the number of classes in the training set and the number of predicted classes. This study investigates how these factors influence the model’s ability to defend against natural adversarial examples. Results indicate that reducing the number of training classes enhances the model’s defense against natural adversarial examples. Additionally, under a fixed number of training classes, some CNN models show an optimal range of predicted classes for achieving the best defense performance against these adversarial examples.展开更多
Recent years have witnessed the ever-increasing performance of Deep Neural Networks(DNNs)in computer vision tasks.However,researchers have identified a potential vulnerability:carefully crafted adversarial examples ca...Recent years have witnessed the ever-increasing performance of Deep Neural Networks(DNNs)in computer vision tasks.However,researchers have identified a potential vulnerability:carefully crafted adversarial examples can easily mislead DNNs into incorrect behavior via the injection of imperceptible modification to the input data.In this survey,we focus on(1)adversarial attack algorithms to generate adversarial examples,(2)adversarial defense techniques to secure DNNs against adversarial examples,and(3)important problems in the realm of adversarial examples beyond attack and defense,including the theoretical explanations,trade-off issues and benign attacks in adversarial examples.Additionally,we draw a brief comparison between recently published surveys on adversarial examples,and identify the future directions for the research of adversarial examples,such as the generalization of methods and the understanding of transferability,that might be solutions to the open problems in this field.展开更多
Adversarial examples are hot topics in the field of security in deep learning.The feature,generation methods,attack and defense methods of the adversarial examples are focuses of the current research on adversarial ex...Adversarial examples are hot topics in the field of security in deep learning.The feature,generation methods,attack and defense methods of the adversarial examples are focuses of the current research on adversarial examples.This article explains the key technologies and theories of adversarial examples from the concept of adversarial examples,the occurrences of the adversarial examples,the attacking methods of adversarial examples.This article lists the possible reasons for the adversarial examples.This article also analyzes several typical generation methods of adversarial examples in detail:Limited-memory BFGS(L-BFGS),Fast Gradient Sign Method(FGSM),Basic Iterative Method(BIM),Iterative Least-likely Class Method(LLC),etc.Furthermore,in the perspective of the attack methods and reasons of the adversarial examples,the main defense techniques for the adversarial examples are listed:preprocessing,regularization and adversarial training method,distillation method,etc.,which application scenarios and deficiencies of different defense measures are pointed out.This article further discusses the application of adversarial examples which currently is mainly used in adversarial evaluation and adversarial training.Finally,the overall research direction of the adversarial examples is prospected to completely solve the adversarial attack problem.There are still a lot of practical and theoretical problems that need to be solved.Finding out the characteristics of the adversarial examples,giving a mathematical description of its practical application prospects,exploring the universal method of adversarial example generation and the generation mechanism of the adversarial examples are the main research directions of the adversarial examples in the future.展开更多
Antivirus vendors and the research community employ Machine Learning(ML)or Deep Learning(DL)-based static analysis techniques for efficient identification of new threats,given the continual emergence of novel malware ...Antivirus vendors and the research community employ Machine Learning(ML)or Deep Learning(DL)-based static analysis techniques for efficient identification of new threats,given the continual emergence of novel malware variants.On the other hand,numerous researchers have reported that Adversarial Examples(AEs),generated by manipulating previously detected malware,can successfully evade ML/DL-based classifiers.Commercial antivirus systems,in particular,have been identified as vulnerable to such AEs.This paper firstly focuses on conducting black-box attacks to circumvent ML/DL-based malware classifiers.Our attack method utilizes seven different perturbations,including Overlay Append,Section Append,and Break Checksum,capitalizing on the ambiguities present in the PE format,as previously employed in evasion attack research.By directly applying the perturbation techniques to PE binaries,our attack method eliminates the need to grapple with the problem-feature space dilemma,a persistent challenge in many evasion attack studies.Being a black-box attack,our method can generate AEs that successfully evade both DL-based and ML-based classifiers.Also,AEs generated by the attack method retain their executability and malicious behavior,eliminating the need for functionality verification.Through thorogh evaluations,we confirmed that the attack method achieves an evasion rate of 65.6%against well-known ML-based malware detectors and can reach a remarkable 99%evasion rate against well-known DL-based malware detectors.Furthermore,our AEs demonstrated the capability to bypass detection by 17%of vendors out of the 64 on VirusTotal(VT).In addition,we propose a defensive approach that utilizes Trend Locality Sensitive Hashing(TLSH)to construct a similarity-based defense model.Through several experiments on the approach,we verified that our defense model can effectively counter AEs generated by the perturbation techniques.In conclusion,our defense model alleviates the limitation of the most promising defense method,adversarial training,which is only effective against the AEs that are included in the training classifiers.展开更多
A quantum variational circuit is a quantum machine learning model similar to a neural network.A crafted adversarial example can lead to incorrect results for the model.Using adversarial examples to train the model wil...A quantum variational circuit is a quantum machine learning model similar to a neural network.A crafted adversarial example can lead to incorrect results for the model.Using adversarial examples to train the model will greatly improve its robustness.The existing method is to use automatic differentials or finite difference to obtain a gradient and use it to construct adversarial examples.This paper proposes an innovative method for constructing adversarial examples of quantum variational circuits.In this method,the gradient can be obtained by measuring the expected value of a quantum bit respectively in a series quantum circuit.This method can be used to construct the adversarial examples for a quantum variational circuit classifier.The implementation results prove the effectiveness of the proposed method.Compared with the existing method,our method requires fewer resources and is more efficient.展开更多
Deep learning-based systems have succeeded in many computer vision tasks.However,it is found that the latest study indicates that these systems are in danger in the presence of adversarial attacks.These attacks can qu...Deep learning-based systems have succeeded in many computer vision tasks.However,it is found that the latest study indicates that these systems are in danger in the presence of adversarial attacks.These attacks can quickly spoil deep learning models,e.g.,different convolutional neural networks(CNNs),used in various computer vision tasks from image classification to object detection.The adversarial examples are carefully designed by injecting a slight perturbation into the clean images.The proposed CRU-Net defense model is inspired by state-of-the-art defense mechanisms such as MagNet defense,Generative Adversarial Net-work Defense,Deep Regret Analytic Generative Adversarial Networks Defense,Deep Denoising Sparse Autoencoder Defense,and Condtional Generattive Adversarial Network Defense.We have experimentally proved that our approach is better than previous defensive techniques.Our proposed CRU-Net model maps the adversarial image examples into clean images by eliminating the adversarial perturbation.The proposed defensive approach is based on residual and U-Net learning.Many experiments are done on the datasets MNIST and CIFAR10 to prove that our proposed CRU-Net defense model prevents adversarial example attacks in WhiteBox and BlackBox settings and improves the robustness of the deep learning algorithms especially in the computer visionfield.We have also reported similarity(SSIM and PSNR)between the original and restored clean image examples by the proposed CRU-Net defense model.展开更多
Image-denoising techniques are widely used to defend against Adversarial Examples(AEs).However,denoising alone cannot completely eliminate adversarial perturbations.The remaining perturbations tend to amplify as they ...Image-denoising techniques are widely used to defend against Adversarial Examples(AEs).However,denoising alone cannot completely eliminate adversarial perturbations.The remaining perturbations tend to amplify as they propagate through deeper layers of the network,leading to misclassifications.Moreover,image denoising compromises the classification accuracy of original examples.To address these challenges in AE defense through image denoising,this paper proposes a novel AE detection technique.The proposed technique combines multiple traditional image-denoising algorithms and Convolutional Neural Network(CNN)network structures.The used detector model integrates the classification results of different models as the input to the detector and calculates the final output of the detector based on a machine-learning voting algorithm.By analyzing the discrepancy between predictions made by the model on original examples and denoised examples,AEs are detected effectively.This technique reduces computational overhead without modifying the model structure or parameters,effectively avoiding the error amplification caused by denoising.The proposed approach demonstrates excellent detection performance against mainstream AE attacks.Experimental results show outstanding detection performance in well-known AE attacks,including Fast Gradient Sign Method(FGSM),Basic Iteration Method(BIM),DeepFool,and Carlini&Wagner(C&W),achieving a 94%success rate in FGSM detection,while only reducing the accuracy of clean examples by 4%.展开更多
Deep neural networks(DNNs)are poten-tially susceptible to adversarial examples that are ma-liciously manipulated by adding imperceptible pertur-bations to legitimate inputs,leading to abnormal be-havior of models.Plen...Deep neural networks(DNNs)are poten-tially susceptible to adversarial examples that are ma-liciously manipulated by adding imperceptible pertur-bations to legitimate inputs,leading to abnormal be-havior of models.Plenty of methods have been pro-posed to defend against adversarial examples.How-ever,the majority of them are suffering the follow-ing weaknesses:1)lack of generalization and prac-ticality.2)fail to deal with unknown attacks.To ad-dress the above issues,we design the adversarial na-ture eraser(ANE)and feature map detector(FMD)to detect fragile and high-intensity adversarial examples,respectively.Then,we apply the ensemble learning method to compose our detector,dealing with adver-sarial examples with diverse magnitudes in a divide-and-conquer manner.Experimental results show that our approach achieves 99.30%and 99.62%Area un-der Curve(AUC)scores on average when tested with various Lp norm-based attacks on CIFAR-10 and Im-ageNet,respectively.Furthermore,our approach also shows its potential in detecting unknown attacks.展开更多
Speech is easily leaked imperceptibly.When people use their phones,the personal voice assistant is constantly listening and waiting to be activated.Private content in speech may be maliciously extracted through automa...Speech is easily leaked imperceptibly.When people use their phones,the personal voice assistant is constantly listening and waiting to be activated.Private content in speech may be maliciously extracted through automatic speech recognition(ASR)technology by some applications on phone devices.To guarantee that the recognized speech content is accurate,speech enhancement technology is used to denoise the input speech.Speech enhancement technology has developed rapidly along with deep neural networks(DNNs),but adversarial examples can cause DNNs to fail.Considering that the vulnerability of DNN can be used to protect the privacy in speech.In this work,we propose an adversarial method to degrade speech enhancement systems,which can prevent the malicious extraction of private information in speech.Experimental results show that the generated enhanced adversarial examples can be removed most content of the target speech or replaced with target speech content by speech enhancement.The word error rate(WER)between the enhanced original example and enhanced adversarial example recognition result can reach 89.0%.WER of target attack between enhanced adversarial example and target example is low at 33.75%.The adversarial perturbation in the adversarial example can bring much more change than itself.The rate of difference between two enhanced examples and adversarial perturbation can reach more than 1.4430.Meanwhile,the transferability between different speech enhancement models is also investigated.The low transferability of the method can be used to ensure the content in the adversarial example is not damaged,the useful information can be extracted by the friendly ASR.This work can prevent the malicious extraction of speech.展开更多
Transfer-based Adversarial Attacks(TAAs)can deceive a victim model even without prior knowledge.This is achieved by leveraging the property of adversarial examples.That is,when generated from a surrogate model,they re...Transfer-based Adversarial Attacks(TAAs)can deceive a victim model even without prior knowledge.This is achieved by leveraging the property of adversarial examples.That is,when generated from a surrogate model,they retain their features if applied to other models due to their good transferability.However,adversarial examples often exhibit overfitting,as they are tailored to exploit the particular architecture and feature representation of source models.Consequently,when attempting black-box transfer attacks on different target models,their effectiveness is decreased.To solve this problem,this study proposes an approach based on a Regularized Constrained Feature Layer(RCFL).The proposed method first uses regularization constraints to attenuate the initial examples of low-frequency components.Perturbations are then added to a pre-specified layer of the source model using the back-propagation technique,in order to modify the original adversarial examples.Afterward,a regularized loss function is used to enhance the black-box transferability between different target models.The proposed method is finally tested on the ImageNet,CIFAR-100,and Stanford Car datasets with various target models,The obtained results demonstrate that it achieves a significantly higher transfer-based adversarial attack success rate compared with baseline techniques.展开更多
Adversarial Malware Example(AME)-based adversarial training can effectively enhance the robustness of Machine Learning(ML)-based malware detectors against AME.AME quality is a key factor to the robustness enhancement....Adversarial Malware Example(AME)-based adversarial training can effectively enhance the robustness of Machine Learning(ML)-based malware detectors against AME.AME quality is a key factor to the robustness enhancement.Generative Adversarial Network(GAN)is a kind of AME generation method,but the existing GAN-based AME generation methods have the issues of inadequate optimization,mode collapse and training instability.In this paper,we propose a novel approach(denote as LSGAN-AT)to enhance ML-based malware detector robustness against Adversarial Examples,which includes LSGAN module and AT module.LSGAN module can generate more effective and smoother AME by utilizing brand-new network structures and Least Square(LS)loss to optimize boundary samples.AT module makes adversarial training using AME generated by LSGAN to generate ML-based Robust Malware Detector(RMD).Extensive experiment results validate the better transferability of AME in terms of attacking 6 ML detectors and the RMD transferability in terms of resisting the MalGAN black-box attack.The results also verify the performance of the generated RMD in the recognition rate of AME.展开更多
Low-rank matrix decomposition with first-order total variation(TV)regularization exhibits excellent performance in exploration of image structure.Taking advantage of its excellent performance in image denoising,we app...Low-rank matrix decomposition with first-order total variation(TV)regularization exhibits excellent performance in exploration of image structure.Taking advantage of its excellent performance in image denoising,we apply it to improve the robustness of deep neural networks.However,although TV regularization can improve the robustness of the model,it reduces the accuracy of normal samples due to its over-smoothing.In our work,we develop a new low-rank matrix recovery model,called LRTGV,which incorporates total generalized variation(TGV)regularization into the reweighted low-rank matrix recovery model.In the proposed model,TGV is used to better reconstruct texture information without over-smoothing.The reweighted nuclear norm and Li-norm can enhance the global structure information.Thus,the proposed LRTGV can destroy the structure of adversarial noise while re-enhancing the global structure and local texture of the image.To solve the challenging optimal model issue,we propose an algorithm based on the alternating direction method of multipliers.Experimental results show that the proposed algorithm has a certain defense capability against black-box attacks,and outperforms state-of-the-art low-rank matrix recovery methods in image restoration.展开更多
Adversarial Malware Example(AME)-based adversarial training can effectively enhance the robustness of Machine Learning(ML)-based malware detectors against AME.AME quality is a key factor to the robustness enhancement....Adversarial Malware Example(AME)-based adversarial training can effectively enhance the robustness of Machine Learning(ML)-based malware detectors against AME.AME quality is a key factor to the robustness enhancement.Generative Adversarial Network(GAN)is a kind of AME generation method,but the existing GAN-based AME generation methods have the issues of inadequate optimization,mode collapse and training instability.In this paper,we propose a novel approach(denote as LSGAN-AT)to enhance ML-based malware detector robustness against Adversarial Examples,which includes LSGAN module and AT module.LSGAN module can generate more effective and smoother AME by utilizing brand-new network structures and Least Square(LS)loss to optimize boundary samples.AT module makes adversarial training using AME generated by LSGAN to generate ML-based Robust Malware Detector(RMD).Extensive experiment results validate the better transferability of AME in terms of attacking 6 ML detectors and the RMD transferability in terms of resisting the MalGAN black-box attack.The results also verify the performance of the generated RMD in the recognition rate of AME.展开更多
Adding subtle perturbations to an image can cause the classification model to misclassify,and such images are called adversarial examples.Adversar-ial examples threaten the safe use of deep neural networks,but when com...Adding subtle perturbations to an image can cause the classification model to misclassify,and such images are called adversarial examples.Adversar-ial examples threaten the safe use of deep neural networks,but when combined with reversible data hiding(RDH)technology,they can protect images from being correctly identified by unauthorized models and recover the image lossless under authorized models.Based on this,the reversible adversarial example(RAE)is ris-ing.However,existing RAE technology focuses on feasibility,attack success rate and image quality,but ignores transferability and time complexity.In this paper,we optimize the data hiding structure and combine data augmentation technology,whichflips the input image in probability to avoid overfitting phenomenon on the dataset.On the premise of maintaining a high success rate of white-box attacks and the image’s visual quality,the proposed method improves the transferability of reversible adversarial examples by approximately 16%and reduces the com-putational cost by approximately 43%compared to the state-of-the-art method.In addition,the appropriateflip probability can be selected for different application scenarios.展开更多
In recent work,adversarial stickers are widely used to attack face recognition(FR)systems in the physical world.However,it is difficult to evaluate the performance of physical attacks because of the lack of volunteers...In recent work,adversarial stickers are widely used to attack face recognition(FR)systems in the physical world.However,it is difficult to evaluate the performance of physical attacks because of the lack of volunteers in the experiment.In this paper,a simple attack method called incomplete physical adversarial attack(IPAA)is proposed to simulate physical attacks.Different from the process of physical attacks,when an IPAA is conducted,a photo of the adversarial sticker is embedded into a facial image as the input to attack FR systems,which can obtain results similar to those of physical attacks without inviting any volunteers.The results show that IPAA has a higher similarity with physical attacks than digital attacks,indicating that IPAA is able to evaluate the performance of physical attacks.IPAA is effective in quantitatively measuring the impact of the sticker location on the results of attacks.展开更多
These days,deep learning and computer vision are much-growing fields in this modern world of information technology.Deep learning algorithms and computer vision have achieved great success in different applications li...These days,deep learning and computer vision are much-growing fields in this modern world of information technology.Deep learning algorithms and computer vision have achieved great success in different applications like image classification,speech recognition,self-driving vehicles,disease diagnostics,and many more.Despite success in various applications,it is found that these learning algorithms face severe threats due to adversarial attacks.Adversarial examples are inputs like images in the computer vision field,which are intentionally slightly changed or perturbed.These changes are humanly imperceptible.But are misclassified by a model with high probability and severely affects the performance or prediction.In this scenario,we present a deep image restoration model that restores adversarial examples so that the target model is classified correctly again.We proved that our defense method against adversarial attacks based on a deep image restoration model is simple and state-of-the-art by providing strong experimental results evidence.We have used MNIST and CIFAR10 datasets for experiments and analysis of our defense method.In the end,we have compared our method to other state-ofthe-art defense methods and proved that our results are better than other rival methods.展开更多
The license plate recognition system(LPRS)has been widely adopted in daily life due to its efficiency and high accuracy.Deep neural networks are commonly used in the LPRS to improve the recognition accuracy.However,re...The license plate recognition system(LPRS)has been widely adopted in daily life due to its efficiency and high accuracy.Deep neural networks are commonly used in the LPRS to improve the recognition accuracy.However,researchers have found that deep neural networks have their own security problems that may lead to unexpected results.Specifically,they can be easily attacked by the adversarial examples that are generated by adding small perturbations to the original images,resulting in incorrect license plate recognition.There are some classic methods to generate adversarial examples,but they cannot be adopted on LPRS directly.In this paper,we modify some classic methods to generate adversarial examples that could mislead the LPRS.We conduct extensive evaluations on the HyperLPR system and the results show that the system could be easily attacked by such adversarial examples.In addition,we show that the generated images could also attack the black-box systems;we show some examples that the Baidu LPR system also makes incorrect recognitions.We hope this paper could help improve the LPRS by realizing the existence of such adversarial attacks.展开更多
Attackers inject the designed adversarial sample into the target recommendation system to achieve illegal goals,seriously affecting the security and reliability of the recommendation system.It is difficult for attacke...Attackers inject the designed adversarial sample into the target recommendation system to achieve illegal goals,seriously affecting the security and reliability of the recommendation system.It is difficult for attackers to obtain detailed knowledge of the target model in actual scenarios,so using gradient optimization to generate adversarial samples in the local surrogate model has become an effective black‐box attack strategy.However,these methods suffer from gradients falling into local minima,limiting the transferability of the adversarial samples.This reduces the attack's effectiveness and often ignores the imperceptibility of the generated adversarial samples.To address these challenges,we propose a novel attack algorithm called PGMRS‐KL that combines pre‐gradient‐guided momentum gradient optimization strategy and fake user generation constrained by Kullback‐Leibler divergence.Specifically,the algorithm combines the accumulated gradient direction with the previous step's gradient direction to iteratively update the adversarial samples.It uses KL loss to minimize the distribution distance between fake and real user data,achieving high transferability and imperceptibility of the adversarial samples.Experimental results demonstrate the superiority of our approach over state‐of‐the‐art gradient‐based attack algorithms in terms of attack transferability and the generation of imperceptible fake user data.展开更多
In recent years,with the rapid development of natural language processing,the security issues related to it have attracted more and more attention.Character perturbation is a common security problem.It can try to comp...In recent years,with the rapid development of natural language processing,the security issues related to it have attracted more and more attention.Character perturbation is a common security problem.It can try to completely modify the input classification judgment of the target program without people’s attention by adding,deleting,or replacing several characters,which can reduce the effectiveness of the classifier.Although the current research has provided various methods of perturbation attacks on characters,the success rate of some methods is still not ideal.This paper mainly studies the sample generation of optimal perturbation characters and proposes a characterlevel text adversarial sample generation method.The goal is to use this method to achieve the best effect on character perturbation.After sentiment classification experiments,this model has a higher perturbation success rate on the IMDB dataset,which proves the effectiveness and rationality of this method for text perturbation and provides a reference for future research work.展开更多
With the rapid developments of artificial intelligence(AI)and deep learning(DL)techniques,it is critical to ensure the security and robustness of the deployed algorithms.Recently,the security vulnerability of DL algor...With the rapid developments of artificial intelligence(AI)and deep learning(DL)techniques,it is critical to ensure the security and robustness of the deployed algorithms.Recently,the security vulnerability of DL algorithms to adversarial samples has been widely recognized.The fabricated samples can lead to various misbehaviors of the DL models while being perceived as benign by humans.Successful implementations of adversarial attacks in real physical-world scenarios further demonstrate their practicality.Hence,adversarial attack and defense techniques have attracted increasing attention from both machine learning and security communities and have become a hot research topic in recent years.In this paper,we first introduce the theoretical foundations,algorithms,and applications of adversarial attack techniques.We then describe a few research efforts on the defense techniques,which cover the broad frontier in the field.Several open problems and challenges are subsequently discussed,which we hope will provoke further research efforts in this critical area.展开更多
文摘The emergence of adversarial examples has revealed the inadequacies in the robustness of image classification models based on Convolutional Neural Networks (CNNs). Particularly in recent years, the discovery of natural adversarial examples has posed significant challenges, as traditional defense methods against adversarial attacks have proven to be largely ineffective against these natural adversarial examples. This paper explores defenses against these natural adversarial examples from three perspectives: adversarial examples, model architecture, and dataset. First, it employs Class Activation Mapping (CAM) to visualize how models classify natural adversarial examples, identifying several typical attack patterns. Next, various common CNN models are analyzed to evaluate their susceptibility to these attacks, revealing that different architectures exhibit varying defensive capabilities. The study finds that as the depth of a network increases, its defenses against natural adversarial examples strengthen. Lastly, Finally, the impact of dataset class distribution on the defense capability of models is examined, focusing on two aspects: the number of classes in the training set and the number of predicted classes. This study investigates how these factors influence the model’s ability to defend against natural adversarial examples. Results indicate that reducing the number of training classes enhances the model’s defense against natural adversarial examples. Additionally, under a fixed number of training classes, some CNN models show an optimal range of predicted classes for achieving the best defense performance against these adversarial examples.
基金Supported by the National Natural Science Foundation of China(U1903214,62372339,62371350,61876135)the Ministry of Education Industry University Cooperative Education Project(202102246004,220800006041043,202002142012)the Fundamental Research Funds for the Central Universities(2042023kf1033)。
文摘Recent years have witnessed the ever-increasing performance of Deep Neural Networks(DNNs)in computer vision tasks.However,researchers have identified a potential vulnerability:carefully crafted adversarial examples can easily mislead DNNs into incorrect behavior via the injection of imperceptible modification to the input data.In this survey,we focus on(1)adversarial attack algorithms to generate adversarial examples,(2)adversarial defense techniques to secure DNNs against adversarial examples,and(3)important problems in the realm of adversarial examples beyond attack and defense,including the theoretical explanations,trade-off issues and benign attacks in adversarial examples.Additionally,we draw a brief comparison between recently published surveys on adversarial examples,and identify the future directions for the research of adversarial examples,such as the generalization of methods and the understanding of transferability,that might be solutions to the open problems in this field.
基金This work is supported by the NSFC[Grant Nos.61772281,61703212]the Priority Academic Program Development of Jiangsu Higher Education Institutions(PAPD)and Jiangsu Collaborative Innovation Center on Atmospheric Environment and Equipment Technology(CICAEET).
文摘Adversarial examples are hot topics in the field of security in deep learning.The feature,generation methods,attack and defense methods of the adversarial examples are focuses of the current research on adversarial examples.This article explains the key technologies and theories of adversarial examples from the concept of adversarial examples,the occurrences of the adversarial examples,the attacking methods of adversarial examples.This article lists the possible reasons for the adversarial examples.This article also analyzes several typical generation methods of adversarial examples in detail:Limited-memory BFGS(L-BFGS),Fast Gradient Sign Method(FGSM),Basic Iterative Method(BIM),Iterative Least-likely Class Method(LLC),etc.Furthermore,in the perspective of the attack methods and reasons of the adversarial examples,the main defense techniques for the adversarial examples are listed:preprocessing,regularization and adversarial training method,distillation method,etc.,which application scenarios and deficiencies of different defense measures are pointed out.This article further discusses the application of adversarial examples which currently is mainly used in adversarial evaluation and adversarial training.Finally,the overall research direction of the adversarial examples is prospected to completely solve the adversarial attack problem.There are still a lot of practical and theoretical problems that need to be solved.Finding out the characteristics of the adversarial examples,giving a mathematical description of its practical application prospects,exploring the universal method of adversarial example generation and the generation mechanism of the adversarial examples are the main research directions of the adversarial examples in the future.
基金supported by Institute of Information&Communications Technology Planning&Evaluation(IITP)Grant funded by the Korea government,Ministry of Science and ICT(MSIT)(No.2017-0-00168,Automatic Deep Malware Analysis Technology for Cyber Threat Intelligence).
文摘Antivirus vendors and the research community employ Machine Learning(ML)or Deep Learning(DL)-based static analysis techniques for efficient identification of new threats,given the continual emergence of novel malware variants.On the other hand,numerous researchers have reported that Adversarial Examples(AEs),generated by manipulating previously detected malware,can successfully evade ML/DL-based classifiers.Commercial antivirus systems,in particular,have been identified as vulnerable to such AEs.This paper firstly focuses on conducting black-box attacks to circumvent ML/DL-based malware classifiers.Our attack method utilizes seven different perturbations,including Overlay Append,Section Append,and Break Checksum,capitalizing on the ambiguities present in the PE format,as previously employed in evasion attack research.By directly applying the perturbation techniques to PE binaries,our attack method eliminates the need to grapple with the problem-feature space dilemma,a persistent challenge in many evasion attack studies.Being a black-box attack,our method can generate AEs that successfully evade both DL-based and ML-based classifiers.Also,AEs generated by the attack method retain their executability and malicious behavior,eliminating the need for functionality verification.Through thorogh evaluations,we confirmed that the attack method achieves an evasion rate of 65.6%against well-known ML-based malware detectors and can reach a remarkable 99%evasion rate against well-known DL-based malware detectors.Furthermore,our AEs demonstrated the capability to bypass detection by 17%of vendors out of the 64 on VirusTotal(VT).In addition,we propose a defensive approach that utilizes Trend Locality Sensitive Hashing(TLSH)to construct a similarity-based defense model.Through several experiments on the approach,we verified that our defense model can effectively counter AEs generated by the perturbation techniques.In conclusion,our defense model alleviates the limitation of the most promising defense method,adversarial training,which is only effective against the AEs that are included in the training classifiers.
基金Project supported by the National Natural Science Foundation of China(Grant Nos.62076042 and 62102049)the Natural Science Foundation of Sichuan Province(Grant No.2022NSFSC0535)+2 种基金the Key Research and Development Project of Sichuan Province(Grant Nos.2021YFSY0012 and 2021YFG0332)the Key Research and Development Project of Chengdu(Grant No.2021-YF05-02424-GX)the Innovation Team of Quantum Security Communication of Sichuan Province(Grant No.17TD0009).
文摘A quantum variational circuit is a quantum machine learning model similar to a neural network.A crafted adversarial example can lead to incorrect results for the model.Using adversarial examples to train the model will greatly improve its robustness.The existing method is to use automatic differentials or finite difference to obtain a gradient and use it to construct adversarial examples.This paper proposes an innovative method for constructing adversarial examples of quantum variational circuits.In this method,the gradient can be obtained by measuring the expected value of a quantum bit respectively in a series quantum circuit.This method can be used to construct the adversarial examples for a quantum variational circuit classifier.The implementation results prove the effectiveness of the proposed method.Compared with the existing method,our method requires fewer resources and is more efficient.
文摘Deep learning-based systems have succeeded in many computer vision tasks.However,it is found that the latest study indicates that these systems are in danger in the presence of adversarial attacks.These attacks can quickly spoil deep learning models,e.g.,different convolutional neural networks(CNNs),used in various computer vision tasks from image classification to object detection.The adversarial examples are carefully designed by injecting a slight perturbation into the clean images.The proposed CRU-Net defense model is inspired by state-of-the-art defense mechanisms such as MagNet defense,Generative Adversarial Net-work Defense,Deep Regret Analytic Generative Adversarial Networks Defense,Deep Denoising Sparse Autoencoder Defense,and Condtional Generattive Adversarial Network Defense.We have experimentally proved that our approach is better than previous defensive techniques.Our proposed CRU-Net model maps the adversarial image examples into clean images by eliminating the adversarial perturbation.The proposed defensive approach is based on residual and U-Net learning.Many experiments are done on the datasets MNIST and CIFAR10 to prove that our proposed CRU-Net defense model prevents adversarial example attacks in WhiteBox and BlackBox settings and improves the robustness of the deep learning algorithms especially in the computer visionfield.We have also reported similarity(SSIM and PSNR)between the original and restored clean image examples by the proposed CRU-Net defense model.
基金supported in part by the Natural Science Foundation of Hunan Province under Grant Nos.2023JJ30316 and 2022JJ2029in part by a project supported by Scientific Research Fund of Hunan Provincial Education Department under Grant No.22A0686+1 种基金in part by the National Natural Science Foundation of China under Grant No.62172058Researchers Supporting Project(No.RSP2023R102)King Saud University,Riyadh,Saudi Arabia.
文摘Image-denoising techniques are widely used to defend against Adversarial Examples(AEs).However,denoising alone cannot completely eliminate adversarial perturbations.The remaining perturbations tend to amplify as they propagate through deeper layers of the network,leading to misclassifications.Moreover,image denoising compromises the classification accuracy of original examples.To address these challenges in AE defense through image denoising,this paper proposes a novel AE detection technique.The proposed technique combines multiple traditional image-denoising algorithms and Convolutional Neural Network(CNN)network structures.The used detector model integrates the classification results of different models as the input to the detector and calculates the final output of the detector based on a machine-learning voting algorithm.By analyzing the discrepancy between predictions made by the model on original examples and denoised examples,AEs are detected effectively.This technique reduces computational overhead without modifying the model structure or parameters,effectively avoiding the error amplification caused by denoising.The proposed approach demonstrates excellent detection performance against mainstream AE attacks.Experimental results show outstanding detection performance in well-known AE attacks,including Fast Gradient Sign Method(FGSM),Basic Iteration Method(BIM),DeepFool,and Carlini&Wagner(C&W),achieving a 94%success rate in FGSM detection,while only reducing the accuracy of clean examples by 4%.
基金This work was partly supported by the National Natural Science Foundation of China under No.62372334,61876134,and U1836112.
文摘Deep neural networks(DNNs)are poten-tially susceptible to adversarial examples that are ma-liciously manipulated by adding imperceptible pertur-bations to legitimate inputs,leading to abnormal be-havior of models.Plenty of methods have been pro-posed to defend against adversarial examples.How-ever,the majority of them are suffering the follow-ing weaknesses:1)lack of generalization and prac-ticality.2)fail to deal with unknown attacks.To ad-dress the above issues,we design the adversarial na-ture eraser(ANE)and feature map detector(FMD)to detect fragile and high-intensity adversarial examples,respectively.Then,we apply the ensemble learning method to compose our detector,dealing with adver-sarial examples with diverse magnitudes in a divide-and-conquer manner.Experimental results show that our approach achieves 99.30%and 99.62%Area un-der Curve(AUC)scores on average when tested with various Lp norm-based attacks on CIFAR-10 and Im-ageNet,respectively.Furthermore,our approach also shows its potential in detecting unknown attacks.
基金This work was supported by the National Natural Science Foundation of China(Grant No.61300055)Zhejiang Natural Science Foundation(Grant No.LY20F020010)+2 种基金Ningbo Science and Technology Innovation Project(Grant No.2022Z075)Ningbo Natural Science Foundation(Grant No.202003N4089)K.C.Wong Magna Fund in Ningbo University.
文摘Speech is easily leaked imperceptibly.When people use their phones,the personal voice assistant is constantly listening and waiting to be activated.Private content in speech may be maliciously extracted through automatic speech recognition(ASR)technology by some applications on phone devices.To guarantee that the recognized speech content is accurate,speech enhancement technology is used to denoise the input speech.Speech enhancement technology has developed rapidly along with deep neural networks(DNNs),but adversarial examples can cause DNNs to fail.Considering that the vulnerability of DNN can be used to protect the privacy in speech.In this work,we propose an adversarial method to degrade speech enhancement systems,which can prevent the malicious extraction of private information in speech.Experimental results show that the generated enhanced adversarial examples can be removed most content of the target speech or replaced with target speech content by speech enhancement.The word error rate(WER)between the enhanced original example and enhanced adversarial example recognition result can reach 89.0%.WER of target attack between enhanced adversarial example and target example is low at 33.75%.The adversarial perturbation in the adversarial example can bring much more change than itself.The rate of difference between two enhanced examples and adversarial perturbation can reach more than 1.4430.Meanwhile,the transferability between different speech enhancement models is also investigated.The low transferability of the method can be used to ensure the content in the adversarial example is not damaged,the useful information can be extracted by the friendly ASR.This work can prevent the malicious extraction of speech.
基金supported by the Intelligent Policing Key Laboratory of Sichuan Province(No.ZNJW2022KFZD002)This work was supported by the Scientific and Technological Research Program of Chongqing Municipal Education Commission(Grant Nos.KJQN202302403,KJQN202303111).
文摘Transfer-based Adversarial Attacks(TAAs)can deceive a victim model even without prior knowledge.This is achieved by leveraging the property of adversarial examples.That is,when generated from a surrogate model,they retain their features if applied to other models due to their good transferability.However,adversarial examples often exhibit overfitting,as they are tailored to exploit the particular architecture and feature representation of source models.Consequently,when attempting black-box transfer attacks on different target models,their effectiveness is decreased.To solve this problem,this study proposes an approach based on a Regularized Constrained Feature Layer(RCFL).The proposed method first uses regularization constraints to attenuate the initial examples of low-frequency components.Perturbations are then added to a pre-specified layer of the source model using the back-propagation technique,in order to modify the original adversarial examples.Afterward,a regularized loss function is used to enhance the black-box transferability between different target models.The proposed method is finally tested on the ImageNet,CIFAR-100,and Stanford Car datasets with various target models,The obtained results demonstrate that it achieves a significantly higher transfer-based adversarial attack success rate compared with baseline techniques.
基金Chinese National Key Laboratory of Science and Technology on Information System Security and National Natural Science Foundation of China under Grant No.U1836105The research of R.J.Rodríguez and X.Chang has been supported in part by the University of Zaragoza and the Fundación Ibercaja under Grant JIUZ-2020-TIC-08The research of R.J.Rodríguez has also been supported in part by the University,Industry and Innovation Department of the Aragonese Government under Programa de Proyectos Estratégicos de Grupos de Investigación(DisCo research group,ref.T21-20R).
文摘Adversarial Malware Example(AME)-based adversarial training can effectively enhance the robustness of Machine Learning(ML)-based malware detectors against AME.AME quality is a key factor to the robustness enhancement.Generative Adversarial Network(GAN)is a kind of AME generation method,but the existing GAN-based AME generation methods have the issues of inadequate optimization,mode collapse and training instability.In this paper,we propose a novel approach(denote as LSGAN-AT)to enhance ML-based malware detector robustness against Adversarial Examples,which includes LSGAN module and AT module.LSGAN module can generate more effective and smoother AME by utilizing brand-new network structures and Least Square(LS)loss to optimize boundary samples.AT module makes adversarial training using AME generated by LSGAN to generate ML-based Robust Malware Detector(RMD).Extensive experiment results validate the better transferability of AME in terms of attacking 6 ML detectors and the RMD transferability in terms of resisting the MalGAN black-box attack.The results also verify the performance of the generated RMD in the recognition rate of AME.
基金Project supported by the National Natural Science Foundation of China(No.62072024)the Outstanding Youth Program of Beijing University of Civil Engineering and Architecture,China(No.JDJQ20220805)the Shenzhen Stability Support General Project(Type A),China(No.20200826104014001)。
文摘Low-rank matrix decomposition with first-order total variation(TV)regularization exhibits excellent performance in exploration of image structure.Taking advantage of its excellent performance in image denoising,we apply it to improve the robustness of deep neural networks.However,although TV regularization can improve the robustness of the model,it reduces the accuracy of normal samples due to its over-smoothing.In our work,we develop a new low-rank matrix recovery model,called LRTGV,which incorporates total generalized variation(TGV)regularization into the reweighted low-rank matrix recovery model.In the proposed model,TGV is used to better reconstruct texture information without over-smoothing.The reweighted nuclear norm and Li-norm can enhance the global structure information.Thus,the proposed LRTGV can destroy the structure of adversarial noise while re-enhancing the global structure and local texture of the image.To solve the challenging optimal model issue,we propose an algorithm based on the alternating direction method of multipliers.Experimental results show that the proposed algorithm has a certain defense capability against black-box attacks,and outperforms state-of-the-art low-rank matrix recovery methods in image restoration.
基金The research of J.Wang,X.Chang,Y.Wang and J.Zhang was supported in part by Project supported by Chinese National Key Laboratory of Science and Technology on Information System Security and National Natural Science Foundation of China under Grant No.U1836105The research of R.J.Rodriguez and X.Chang has been supported in part by the University of Zaragoza and the Fundacion Ibercaja under Grant JIUZ-2020-TIC-08The research of R.J.Rodriguez has also been supported in part by the University,Industry and Innovation Department of the Aragonese Government under Programa de Proyectos Estrategicos de Grupos de Investigacidn(DisCo research group,ref.T21-20R).
文摘Adversarial Malware Example(AME)-based adversarial training can effectively enhance the robustness of Machine Learning(ML)-based malware detectors against AME.AME quality is a key factor to the robustness enhancement.Generative Adversarial Network(GAN)is a kind of AME generation method,but the existing GAN-based AME generation methods have the issues of inadequate optimization,mode collapse and training instability.In this paper,we propose a novel approach(denote as LSGAN-AT)to enhance ML-based malware detector robustness against Adversarial Examples,which includes LSGAN module and AT module.LSGAN module can generate more effective and smoother AME by utilizing brand-new network structures and Least Square(LS)loss to optimize boundary samples.AT module makes adversarial training using AME generated by LSGAN to generate ML-based Robust Malware Detector(RMD).Extensive experiment results validate the better transferability of AME in terms of attacking 6 ML detectors and the RMD transferability in terms of resisting the MalGAN black-box attack.The results also verify the performance of the generated RMD in the recognition rate of AME.
基金This research work is partly supported by the National Natural Science Foundation of China(62172001)the Provincial Colleges Quality Project of Anhui Province(2020xsxxkc047)the National Undergraduate Innovation and Entrepreneurship Training Program(202210357077).
文摘Adding subtle perturbations to an image can cause the classification model to misclassify,and such images are called adversarial examples.Adversar-ial examples threaten the safe use of deep neural networks,but when combined with reversible data hiding(RDH)technology,they can protect images from being correctly identified by unauthorized models and recover the image lossless under authorized models.Based on this,the reversible adversarial example(RAE)is ris-ing.However,existing RAE technology focuses on feasibility,attack success rate and image quality,but ignores transferability and time complexity.In this paper,we optimize the data hiding structure and combine data augmentation technology,whichflips the input image in probability to avoid overfitting phenomenon on the dataset.On the premise of maintaining a high success rate of white-box attacks and the image’s visual quality,the proposed method improves the transferability of reversible adversarial examples by approximately 16%and reduces the com-putational cost by approximately 43%compared to the state-of-the-art method.In addition,the appropriateflip probability can be selected for different application scenarios.
文摘In recent work,adversarial stickers are widely used to attack face recognition(FR)systems in the physical world.However,it is difficult to evaluate the performance of physical attacks because of the lack of volunteers in the experiment.In this paper,a simple attack method called incomplete physical adversarial attack(IPAA)is proposed to simulate physical attacks.Different from the process of physical attacks,when an IPAA is conducted,a photo of the adversarial sticker is embedded into a facial image as the input to attack FR systems,which can obtain results similar to those of physical attacks without inviting any volunteers.The results show that IPAA has a higher similarity with physical attacks than digital attacks,indicating that IPAA is able to evaluate the performance of physical attacks.IPAA is effective in quantitatively measuring the impact of the sticker location on the results of attacks.
文摘These days,deep learning and computer vision are much-growing fields in this modern world of information technology.Deep learning algorithms and computer vision have achieved great success in different applications like image classification,speech recognition,self-driving vehicles,disease diagnostics,and many more.Despite success in various applications,it is found that these learning algorithms face severe threats due to adversarial attacks.Adversarial examples are inputs like images in the computer vision field,which are intentionally slightly changed or perturbed.These changes are humanly imperceptible.But are misclassified by a model with high probability and severely affects the performance or prediction.In this scenario,we present a deep image restoration model that restores adversarial examples so that the target model is classified correctly again.We proved that our defense method against adversarial attacks based on a deep image restoration model is simple and state-of-the-art by providing strong experimental results evidence.We have used MNIST and CIFAR10 datasets for experiments and analysis of our defense method.In the end,we have compared our method to other state-ofthe-art defense methods and proved that our results are better than other rival methods.
基金This work is supported by the National Natural Science Foundation of China under Grant Nos.U1636215,61902082the Guangdong Key R&D Program of China 2019B010136003National Key R&D Program of China 2019YFB1706003.
文摘The license plate recognition system(LPRS)has been widely adopted in daily life due to its efficiency and high accuracy.Deep neural networks are commonly used in the LPRS to improve the recognition accuracy.However,researchers have found that deep neural networks have their own security problems that may lead to unexpected results.Specifically,they can be easily attacked by the adversarial examples that are generated by adding small perturbations to the original images,resulting in incorrect license plate recognition.There are some classic methods to generate adversarial examples,but they cannot be adopted on LPRS directly.In this paper,we modify some classic methods to generate adversarial examples that could mislead the LPRS.We conduct extensive evaluations on the HyperLPR system and the results show that the system could be easily attacked by such adversarial examples.In addition,we show that the generated images could also attack the black-box systems;we show some examples that the Baidu LPR system also makes incorrect recognitions.We hope this paper could help improve the LPRS by realizing the existence of such adversarial attacks.
基金The National Natural Science Foundation of China (61876001)Opening Foundation of State Key Laboratory of Cognitive Intelligence,Opening Foundation of State Key Laboratory of Cognitive Intelligence(iED2022-006)Scientific Research Planning Project of Anhui Province (2022AH050072)
文摘Attackers inject the designed adversarial sample into the target recommendation system to achieve illegal goals,seriously affecting the security and reliability of the recommendation system.It is difficult for attackers to obtain detailed knowledge of the target model in actual scenarios,so using gradient optimization to generate adversarial samples in the local surrogate model has become an effective black‐box attack strategy.However,these methods suffer from gradients falling into local minima,limiting the transferability of the adversarial samples.This reduces the attack's effectiveness and often ignores the imperceptibility of the generated adversarial samples.To address these challenges,we propose a novel attack algorithm called PGMRS‐KL that combines pre‐gradient‐guided momentum gradient optimization strategy and fake user generation constrained by Kullback‐Leibler divergence.Specifically,the algorithm combines the accumulated gradient direction with the previous step's gradient direction to iteratively update the adversarial samples.It uses KL loss to minimize the distribution distance between fake and real user data,achieving high transferability and imperceptibility of the adversarial samples.Experimental results demonstrate the superiority of our approach over state‐of‐the‐art gradient‐based attack algorithms in terms of attack transferability and the generation of imperceptible fake user data.
基金This work was supported by the National Key Research and Development Plan(Grant Nos.2018YFB1800302 and 2019YFA0706404)the Natural Science Foundation of China(Grant No.61702013)+2 种基金Joint of Beijing Natural Science Foundation and Education Commission(Grant No.KZ201810009011)Beijing Natural Science Foundation(Grant Nos.4202020,19L2021)Science and Technology Innovation Project of North China University of Technology(Grant No.19XN108).
文摘In recent years,with the rapid development of natural language processing,the security issues related to it have attracted more and more attention.Character perturbation is a common security problem.It can try to completely modify the input classification judgment of the target program without people’s attention by adding,deleting,or replacing several characters,which can reduce the effectiveness of the classifier.Although the current research has provided various methods of perturbation attacks on characters,the success rate of some methods is still not ideal.This paper mainly studies the sample generation of optimal perturbation characters and proposes a characterlevel text adversarial sample generation method.The goal is to use this method to achieve the best effect on character perturbation.After sentiment classification experiments,this model has a higher perturbation success rate on the IMDB dataset,which proves the effectiveness and rationality of this method for text perturbation and provides a reference for future research work.
基金Ant Financial,Zhejiang University Financial Technology Research Center.
文摘With the rapid developments of artificial intelligence(AI)and deep learning(DL)techniques,it is critical to ensure the security and robustness of the deployed algorithms.Recently,the security vulnerability of DL algorithms to adversarial samples has been widely recognized.The fabricated samples can lead to various misbehaviors of the DL models while being perceived as benign by humans.Successful implementations of adversarial attacks in real physical-world scenarios further demonstrate their practicality.Hence,adversarial attack and defense techniques have attracted increasing attention from both machine learning and security communities and have become a hot research topic in recent years.In this paper,we first introduce the theoretical foundations,algorithms,and applications of adversarial attack techniques.We then describe a few research efforts on the defense techniques,which cover the broad frontier in the field.Several open problems and challenges are subsequently discussed,which we hope will provoke further research efforts in this critical area.
