The extensive proliferation of modern information services and ubiquitous digitization of society have raised cybersecurity challenges to new levels.With the massive number of connected devices,opportunities for poten...The extensive proliferation of modern information services and ubiquitous digitization of society have raised cybersecurity challenges to new levels.With the massive number of connected devices,opportunities for potential network attacks are nearly unlimited.An additional problem is that many low-cost devices are not equippedwith effective security protection so that they are easily hacked and applied within a network of bots(botnet)to perform distributed denial of service(DDoS)attacks.In this paper,we propose a novel intrusion detection system(IDS)based on deep learning that aims to identify suspicious behavior in modern heterogeneous information systems.The proposed approach is based on a deep recurrent autoencoder that learns time series of normal network behavior and detects notable network anomalies.An additional feature of the proposed IDS is that it is trained with an optimized dataset,where the number of features is reduced by 94%without classification accuracy loss.Thus,the proposed IDS remains stable in response to slight system perturbations,which do not represent network anomalies.The proposed approach is evaluated under different simulation scenarios and provides a 99%detection accuracy over known datasets while reducing the training time by an order of magnitude.展开更多
Federated semi-supervised learning(FSSL)faces two major challenges:the scarcity of labeled data across clients and the non-independent and identically distributed(Non-IID)nature of data among clients.To address these ...Federated semi-supervised learning(FSSL)faces two major challenges:the scarcity of labeled data across clients and the non-independent and identically distributed(Non-IID)nature of data among clients.To address these issues,we propose diffusion model-based data synthesis aided FSSL(DDSA-FSSL),a novel approach that leverages diffusion model(DM)to generate synthetic data,thereby bridging the gap between heterogeneous local data distributions and the global data distribution.In the proposed DDSA-FSSL,each client addresses the scarcity of labeled data by utilizing a federated learningtrained classifier to perform pseudo labeling for unlabeled data.The DM is then collaboratively trained using both labeled and precision-optimized pseudolabeled data,enabling clients to generate synthetic samples for classes that are absent in their labeled datasets.As a result,the disparity between local and global distributions is reduced and clients can create enriched synthetic datasets that better align with the global data distribution.Extensive experiments on various datasets and Non-IID scenarios demonstrate the effectiveness of DDSA-FSSL,achieving significant performance improvements,such as increasing accuracy from 38.46%to 52.14%on CIFAR-10 datasets with 10%labeled data.展开更多
Depth estimation is a fundamental computer vision problem that infers three-dimensional(3D)structures from a given scene.As it is an ill-posed problem,to fit the projection function from the given scene to the 3D stru...Depth estimation is a fundamental computer vision problem that infers three-dimensional(3D)structures from a given scene.As it is an ill-posed problem,to fit the projection function from the given scene to the 3D structure,traditional methods generally require mass amounts of annotated data.Such pixel-level annotation is quite labor consuming,especially when addressing reflective surfaces such as mirrors or water.The widespread application of deep learning further intensifies the demand for large amounts of annotated data.Therefore,it is urgent and necessary to propose a framework that is able to reduce the requirement on the amount of data.In this paper,we propose a novel semisupervised learning framework to infer the 3D structure from the given scene.First,semantic information is employed to make the depth inference more accurate.Second,we make both the depth estimation and semantic segmentation coarse-to-fine frameworks;thus,the depth estimation can be gradually guided by semantic segmentation.We compare our model with state-of-the-art methods.The experimental results demonstrate that our method is better than many supervised learning-based methods,which proves the effectiveness of the proposed method.展开更多
The rapidly increasing popularity of mobile devices has changed the methods with which people access various network services and increased net-work traffic markedly.Over the past few decades,network traffic identific...The rapidly increasing popularity of mobile devices has changed the methods with which people access various network services and increased net-work traffic markedly.Over the past few decades,network traffic identification has been a research hotspot in the field of network management and security mon-itoring.However,as more network services use encryption technology,network traffic identification faces many challenges.Although classic machine learning methods can solve many problems that cannot be solved by port-and payload-based methods,manually extract features that are frequently updated is time-consuming and labor-intensive.Deep learning has good automatic feature learning capabilities and is an ideal method for network traffic identification,particularly encrypted traffic identification;Existing recognition methods based on deep learning primarily use supervised learning methods and rely on many labeled samples.However,in real scenarios,labeled samples are often difficult to obtain.This paper adjusts the structure of the auxiliary classification generation adversarial network(ACGAN)so that it can use unlabeled samples for training,and use the wasserstein distance instead of the original cross entropy as the loss function to achieve semisupervised learning.Experimental results show that the identification accuracy of ISCX and USTC data sets using the proposed method yields markedly better performance when the number of labeled samples is small compared to that of convolutional neural network(CNN)based classifier.展开更多
Most unsupervised or semisupervised hyperspectral anomaly detection(HAD)methods train background reconstruction models in the original spectral domain.However,due to the noise and spatial resolution limitations,there ...Most unsupervised or semisupervised hyperspectral anomaly detection(HAD)methods train background reconstruction models in the original spectral domain.However,due to the noise and spatial resolution limitations,there may be a lack of discrimination between backgrounds and anomalies.This makes it easy for the autoencoder to capture the lowlevel features shared between the two,thereby increasing the difficulty of separating anomalies from the backgrounds,which runs counter to the purpose of HAD.To this end,the authors map the original spectrums to the fractional Fourier domain(FrFD)and reformulate it as a mapping task in which restoration errors are employed to distinguish background and anomaly.This study proposes a novel frequency‐to‐spectrum mapping generative adversarial network for HAD.Specifically,the depth separable features of backgrounds and anomalies are enhanced in the FrFD.Due to the semisupervised approach,FTSGAN needs to learn the embedded features of the backgrounds,thus mapping and restoring them from the FrFD to the original spectral domain.This strategy effectively prevents the model from focussing on the numerical equivalence of input and output,and restricts the ability of FTSGAN to restore anomalies.The comparison and analysis of the experiments verify that the proposed method is competitive.展开更多
Radiation source identification plays an important role in non-cooperative communication scene and numerous methods have been proposed in this field. Deep learning has gained popularity in a variety of computer vision...Radiation source identification plays an important role in non-cooperative communication scene and numerous methods have been proposed in this field. Deep learning has gained popularity in a variety of computer vision tasks. Recently, it has also been successfully applied for radiation source identification.However, training deep neural networks for classification requires a large number of labeled samples, and in non-cooperative applications, it is unrealistic. This paper proposes a method for the unlabeled samples of unknown radiation source. It uses semi-supervised learning to detect unlabeled samples and label new samples automatically. It avoids retraining the neural network with parameter-transfer learning. The results show that compared with the traditional algorithms, the proposed algorithm can offer better accuracy.展开更多
Deep learning based on labeled data has brought massive success in computer vision, speech recognition, and natural language processing. Nevertheless, labeled data is just a drop in the ocean compared with unlabeled d...Deep learning based on labeled data has brought massive success in computer vision, speech recognition, and natural language processing. Nevertheless, labeled data is just a drop in the ocean compared with unlabeled data. How can people utilize the unlabeled data efectively? Research has focused on unsupervised and semi-supervised learning to solve such a problem. Some theoretical and empirical studies have proved that unlabeled data can help boost the generalization ability and robustness under adversarial attacks. However, current theoretical research on the relationship between robustness and unlabeled data limits its scope to toy datasets. Meanwhile, the visual models in autonomous driving need a significant improvement in robustness to guarantee security and safety. This paper proposes a semi-supervised learning framework for object detection in autonomous vehicles, improving the robustness with unlabeled data. Firstly, we build a baseline with the transfer learning of an unsupervised contrastive learning method—Momentum Contrast(MoCo). Secondly,we propose a semi-supervised co-training method to label the unlabeled data for retraining,which improves generalization on the autonomous driving dataset. Thirdly, we apply the unsupervised Bounding Box data augmentation(BBAug) method based on a search algorithm, which uses reinforcement learning to improve the robustness of object detection for autonomous driving. We present an empirical study on the KITTI dataset with diverse adversarial attack methods. Our proposed method realizes the state-of-the-art generalization and robustness under white-box attacks(DPatch and Contextual Patch) and black-box attacks(Gaussian noise, Rain, Fog, and so on). Our proposed method and empirical study show that using more unlabeled data benefits the robustness of perception systems in autonomous driving.展开更多
Various problems are encountered when adopting ordinary vector space algorithms for high-order tensor data input. Namely, one must overcome the Small Sample Size （SSS） and overfitting problems. In addition, the stru...Various problems are encountered when adopting ordinary vector space algorithms for high-order tensor data input. Namely, one must overcome the Small Sample Size （SSS） and overfitting problems. In addition, the structural information of the original tensor signal is lost during the vectorization process. Therefore, comparable methods using a direct tensor input are more appropriate. In the case of electrocardiograms （ECGs）, another problem must be overcome; the manual diagnosis of ECG data is expensive and time consuming, rendering it difficult to acquire data with diagnosis labels. However, when effective features for classification in the original data are very sparse, we propose a semisupervised sparse multilinear discriminant analysis （SSSMDA） method. This method uses the distribution of both the labeled and the unlabeled data together with labels discovered through a label propagation Mgorithm. In practice, we use 12-lead ECGs collected from a remote diagnosis system and apply a short-time-fourier transformation （STFT） to obtain third-order tensors. The experimental results highlight the sparsity of the ECG data and the ability of our method to extract sparse and effective features that can be used for classification.展开更多
基金This work was supported by the Slovak Research and Development Agency,project number APVV-18-0214by the Scientific Grant Agency of the Ministry of Education,science,research and sport of the Slovak Republic under the contract:1/0268/19by the Ukrainian government projects No.0120U102201“Development the methods and unified software-hardware means for the deployment of the energy efficient intent-based multi-purpose information and communication networks,”and No.0120U100674,“Designing the novel decentralized mobile network based on blockchain architecture and artificial intelligence for 5G/6G development in Ukraine.”。
文摘The extensive proliferation of modern information services and ubiquitous digitization of society have raised cybersecurity challenges to new levels.With the massive number of connected devices,opportunities for potential network attacks are nearly unlimited.An additional problem is that many low-cost devices are not equippedwith effective security protection so that they are easily hacked and applied within a network of bots(botnet)to perform distributed denial of service(DDoS)attacks.In this paper,we propose a novel intrusion detection system(IDS)based on deep learning that aims to identify suspicious behavior in modern heterogeneous information systems.The proposed approach is based on a deep recurrent autoencoder that learns time series of normal network behavior and detects notable network anomalies.An additional feature of the proposed IDS is that it is trained with an optimized dataset,where the number of features is reduced by 94%without classification accuracy loss.Thus,the proposed IDS remains stable in response to slight system perturbations,which do not represent network anomalies.The proposed approach is evaluated under different simulation scenarios and provides a 99%detection accuracy over known datasets while reducing the training time by an order of magnitude.
基金supported in part by NSF of China under Grant 62222111 and Grant 62431015in part by the Science and Technology Commission Foundation of Shanghai under Grant 24DP1500702.
文摘Federated semi-supervised learning(FSSL)faces two major challenges:the scarcity of labeled data across clients and the non-independent and identically distributed(Non-IID)nature of data among clients.To address these issues,we propose diffusion model-based data synthesis aided FSSL(DDSA-FSSL),a novel approach that leverages diffusion model(DM)to generate synthetic data,thereby bridging the gap between heterogeneous local data distributions and the global data distribution.In the proposed DDSA-FSSL,each client addresses the scarcity of labeled data by utilizing a federated learningtrained classifier to perform pseudo labeling for unlabeled data.The DM is then collaboratively trained using both labeled and precision-optimized pseudolabeled data,enabling clients to generate synthetic samples for classes that are absent in their labeled datasets.As a result,the disparity between local and global distributions is reduced and clients can create enriched synthetic datasets that better align with the global data distribution.Extensive experiments on various datasets and Non-IID scenarios demonstrate the effectiveness of DDSA-FSSL,achieving significant performance improvements,such as increasing accuracy from 38.46%to 52.14%on CIFAR-10 datasets with 10%labeled data.
基金supported in part by the National High Technology Research and Development Program of China(Grant No.2021YFF0900500)the National Natural Science Foundation of China(Grant Nos.61972115 and 61872116)。
文摘Depth estimation is a fundamental computer vision problem that infers three-dimensional(3D)structures from a given scene.As it is an ill-posed problem,to fit the projection function from the given scene to the 3D structure,traditional methods generally require mass amounts of annotated data.Such pixel-level annotation is quite labor consuming,especially when addressing reflective surfaces such as mirrors or water.The widespread application of deep learning further intensifies the demand for large amounts of annotated data.Therefore,it is urgent and necessary to propose a framework that is able to reduce the requirement on the amount of data.In this paper,we propose a novel semisupervised learning framework to infer the 3D structure from the given scene.First,semantic information is employed to make the depth inference more accurate.Second,we make both the depth estimation and semantic segmentation coarse-to-fine frameworks;thus,the depth estimation can be gradually guided by semantic segmentation.We compare our model with state-of-the-art methods.The experimental results demonstrate that our method is better than many supervised learning-based methods,which proves the effectiveness of the proposed method.
基金This work is supported by the Science and Technology Project of State Grid Jiangsu Electric Power Co.,Ltd.under Grant No.J2020068.
文摘The rapidly increasing popularity of mobile devices has changed the methods with which people access various network services and increased net-work traffic markedly.Over the past few decades,network traffic identification has been a research hotspot in the field of network management and security mon-itoring.However,as more network services use encryption technology,network traffic identification faces many challenges.Although classic machine learning methods can solve many problems that cannot be solved by port-and payload-based methods,manually extract features that are frequently updated is time-consuming and labor-intensive.Deep learning has good automatic feature learning capabilities and is an ideal method for network traffic identification,particularly encrypted traffic identification;Existing recognition methods based on deep learning primarily use supervised learning methods and rely on many labeled samples.However,in real scenarios,labeled samples are often difficult to obtain.This paper adjusts the structure of the auxiliary classification generation adversarial network(ACGAN)so that it can use unlabeled samples for training,and use the wasserstein distance instead of the original cross entropy as the loss function to achieve semisupervised learning.Experimental results show that the identification accuracy of ISCX and USTC data sets using the proposed method yields markedly better performance when the number of labeled samples is small compared to that of convolutional neural network(CNN)based classifier.
基金supported by the National Natural Science Foundation of China under Grant 62161160336Grant 41871245in part by the Belgium Vlaio project(AI ICON‐2021‐0599:Smart industrial spectral cameras via artificial intelligence).
文摘Most unsupervised or semisupervised hyperspectral anomaly detection(HAD)methods train background reconstruction models in the original spectral domain.However,due to the noise and spatial resolution limitations,there may be a lack of discrimination between backgrounds and anomalies.This makes it easy for the autoencoder to capture the lowlevel features shared between the two,thereby increasing the difficulty of separating anomalies from the backgrounds,which runs counter to the purpose of HAD.To this end,the authors map the original spectrums to the fractional Fourier domain(FrFD)and reformulate it as a mapping task in which restoration errors are employed to distinguish background and anomaly.This study proposes a novel frequency‐to‐spectrum mapping generative adversarial network for HAD.Specifically,the depth separable features of backgrounds and anomalies are enhanced in the FrFD.Due to the semisupervised approach,FTSGAN needs to learn the embedded features of the backgrounds,thus mapping and restoring them from the FrFD to the original spectral domain.This strategy effectively prevents the model from focussing on the numerical equivalence of input and output,and restricts the ability of FTSGAN to restore anomalies.The comparison and analysis of the experiments verify that the proposed method is competitive.
基金supported by the National Key R&D Program of China (2018YFB2101300)。
文摘Radiation source identification plays an important role in non-cooperative communication scene and numerous methods have been proposed in this field. Deep learning has gained popularity in a variety of computer vision tasks. Recently, it has also been successfully applied for radiation source identification.However, training deep neural networks for classification requires a large number of labeled samples, and in non-cooperative applications, it is unrealistic. This paper proposes a method for the unlabeled samples of unknown radiation source. It uses semi-supervised learning to detect unlabeled samples and label new samples automatically. It avoids retraining the neural network with parameter-transfer learning. The results show that compared with the traditional algorithms, the proposed algorithm can offer better accuracy.
基金supported by the National Natural Science Foundation of China under Grant No. 61701348 and No. 62133011
文摘Deep learning based on labeled data has brought massive success in computer vision, speech recognition, and natural language processing. Nevertheless, labeled data is just a drop in the ocean compared with unlabeled data. How can people utilize the unlabeled data efectively? Research has focused on unsupervised and semi-supervised learning to solve such a problem. Some theoretical and empirical studies have proved that unlabeled data can help boost the generalization ability and robustness under adversarial attacks. However, current theoretical research on the relationship between robustness and unlabeled data limits its scope to toy datasets. Meanwhile, the visual models in autonomous driving need a significant improvement in robustness to guarantee security and safety. This paper proposes a semi-supervised learning framework for object detection in autonomous vehicles, improving the robustness with unlabeled data. Firstly, we build a baseline with the transfer learning of an unsupervised contrastive learning method—Momentum Contrast(MoCo). Secondly,we propose a semi-supervised co-training method to label the unlabeled data for retraining,which improves generalization on the autonomous driving dataset. Thirdly, we apply the unsupervised Bounding Box data augmentation(BBAug) method based on a search algorithm, which uses reinforcement learning to improve the robustness of object detection for autonomous driving. We present an empirical study on the KITTI dataset with diverse adversarial attack methods. Our proposed method realizes the state-of-the-art generalization and robustness under white-box attacks(DPatch and Contextual Patch) and black-box attacks(Gaussian noise, Rain, Fog, and so on). Our proposed method and empirical study show that using more unlabeled data benefits the robustness of perception systems in autonomous driving.
基金supported by the National Natural Science Foundation of China under Grant Nos.91120305,61272251the National Basic Research 973 Program of China under Grant No.2015CB856004
文摘Various problems are encountered when adopting ordinary vector space algorithms for high-order tensor data input. Namely, one must overcome the Small Sample Size （SSS） and overfitting problems. In addition, the structural information of the original tensor signal is lost during the vectorization process. Therefore, comparable methods using a direct tensor input are more appropriate. In the case of electrocardiograms （ECGs）, another problem must be overcome; the manual diagnosis of ECG data is expensive and time consuming, rendering it difficult to acquire data with diagnosis labels. However, when effective features for classification in the original data are very sparse, we propose a semisupervised sparse multilinear discriminant analysis （SSSMDA） method. This method uses the distribution of both the labeled and the unlabeled data together with labels discovered through a label propagation Mgorithm. In practice, we use 12-lead ECGs collected from a remote diagnosis system and apply a short-time-fourier transformation （STFT） to obtain third-order tensors. The experimental results highlight the sparsity of the ECG data and the ability of our method to extract sparse and effective features that can be used for classification.
