Tremendous amount of data are being generated and saved in many complex engineering and social systems every day.It is significant and feasible to utilize the big data to make better decisions by machine learning tech...Tremendous amount of data are being generated and saved in many complex engineering and social systems every day.It is significant and feasible to utilize the big data to make better decisions by machine learning techniques. In this paper, we focus on batch reinforcement learning(RL) algorithms for discounted Markov decision processes(MDPs) with large discrete or continuous state spaces, aiming to learn the best possible policy given a fixed amount of training data. The batch RL algorithms with handcrafted feature representations work well for low-dimensional MDPs. However, for many real-world RL tasks which often involve high-dimensional state spaces, it is difficult and even infeasible to use feature engineering methods to design features for value function approximation. To cope with high-dimensional RL problems, the desire to obtain data-driven features has led to a lot of works in incorporating feature selection and feature learning into traditional batch RL algorithms. In this paper, we provide a comprehensive survey on automatic feature selection and unsupervised feature learning for high-dimensional batch RL. Moreover, we present recent theoretical developments on applying statistical learning to establish finite-sample error bounds for batch RL algorithms based on weighted Lpnorms. Finally, we derive some future directions in the research of RL algorithms, theories and applications.展开更多
This paper presents an end-to-end deep learning method to solve geometry problems via feature learning and contrastive learning of multimodal data.A key challenge in solving geometry problems using deep learning is to...This paper presents an end-to-end deep learning method to solve geometry problems via feature learning and contrastive learning of multimodal data.A key challenge in solving geometry problems using deep learning is to automatically adapt to the task of understanding single-modal and multimodal problems.Existing methods either focus on single-modal ormultimodal problems,and they cannot fit each other.A general geometry problem solver shouldobviouslybe able toprocess variousmodalproblems at the same time.Inthispaper,a shared feature-learning model of multimodal data is adopted to learn the unified feature representation of text and image,which can solve the heterogeneity issue between multimodal geometry problems.A contrastive learning model of multimodal data enhances the semantic relevance betweenmultimodal features and maps them into a unified semantic space,which can effectively adapt to both single-modal and multimodal downstream tasks.Based on the feature extraction and fusion of multimodal data,a proposed geometry problem solver uses relation extraction,theorem reasoning,and problem solving to present solutions in a readable way.Experimental results show the effectiveness of the method.展开更多
Industrial Control Systems (ICS) or SCADA networks are increasingly targeted by cyber-attacks as their architectures shifted from proprietary hardware, software and protocols to standard and open sources ones. Further...Industrial Control Systems (ICS) or SCADA networks are increasingly targeted by cyber-attacks as their architectures shifted from proprietary hardware, software and protocols to standard and open sources ones. Furthermore, these systems which used to be isolated are now interconnected to corporate networks and to the Internet. Among the countermeasures to mitigate the threats, anomaly detection systems play an important role as they can help detect even unknown attacks. Deep learning which has gained a great attention in the last few years due to excellent results in image, video and natural language processing is being used for anomaly detection in information security, particularly in SCADA networks. The salient features of the data from SCADA networks are learnt as hierarchical representation using deep architectures, and those learnt features are used to classify the data into normal or anomalous ones. This article is a review of various architectures such as Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), Stacked Autoencoder (SAE), Long Short Term Memory (LSTM), or a combination of those architectures, for anomaly detection purpose in SCADA networks.展开更多
With the increasing complexity of industrial processes, the high-dimensional industrial data exhibit a strong nonlinearity, bringing considerable challenges to the fault diagnosis of industrial processes. To efficient...With the increasing complexity of industrial processes, the high-dimensional industrial data exhibit a strong nonlinearity, bringing considerable challenges to the fault diagnosis of industrial processes. To efficiently extract deep meaningful features that are crucial for fault diagnosis, a sparse Gaussian feature extractor(SGFE) is designed to learn a nonlinear mapping that projects the raw data into the feature space with the fault label dimension. The feature space is described by the one-hot encoding of the fault category label as an orthogonal basis. In this way, the deep sparse Gaussian features related to fault categories can be gradually learned from the raw data by SGFE. In the feature space,the sparse Gaussian(SG) loss function is designed to constrain the distribution of features to multiple sparse multivariate Gaussian distributions. The sparse Gaussian features are linearly separable in the feature space, which is conducive to improving the accuracy of the downstream fault classification task. The feasibility and practical utility of the proposed SGFE are verified by the handwritten digits MNIST benchmark and Tennessee-Eastman(TE) benchmark process,respectively.展开更多
Joint Multimodal Aspect-based Sentiment Analysis(JMASA)is a significant task in the research of multimodal fine-grained sentiment analysis,which combines two subtasks:Multimodal Aspect Term Extraction(MATE)and Multimo...Joint Multimodal Aspect-based Sentiment Analysis(JMASA)is a significant task in the research of multimodal fine-grained sentiment analysis,which combines two subtasks:Multimodal Aspect Term Extraction(MATE)and Multimodal Aspect-oriented Sentiment Classification(MASC).Currently,most existing models for JMASA only perform text and image feature encoding from a basic level,but often neglect the in-depth analysis of unimodal intrinsic features,which may lead to the low accuracy of aspect term extraction and the poor ability of sentiment prediction due to the insufficient learning of intra-modal features.Given this problem,we propose a Text-Image Feature Fine-grained Learning(TIFFL)model for JMASA.First,we construct an enhanced adjacency matrix of word dependencies and adopt graph convolutional network to learn the syntactic structure features for text,which addresses the context interference problem of identifying different aspect terms.Then,the adjective-noun pairs extracted from image are introduced to enable the semantic representation of visual features more intuitive,which addresses the ambiguous semantic extraction problem during image feature learning.Thereby,the model performance of aspect term extraction and sentiment polarity prediction can be further optimized and enhanced.Experiments on two Twitter benchmark datasets demonstrate that TIFFL achieves competitive results for JMASA,MATE and MASC,thus validating the effectiveness of our proposed methods.展开更多
In this study,we devised a computational framework called Supervised Feature Learning and Scoring(SuperFeat)which enables the training of a machine learning model and evaluates the canonical cellular statuses/features...In this study,we devised a computational framework called Supervised Feature Learning and Scoring(SuperFeat)which enables the training of a machine learning model and evaluates the canonical cellular statuses/features in pathological tissues that underlie the progression of disease.This framework also enables the identification of potential drugs that target the presumed detrimental cellular features.This framework was constructed on the basis of an artificial neural network with the gene expression profiles serving as input nodes.The training data comprised single-cell RNA sequencing datasets that encompassed the specific cell lineage during the developmental progression of cell features.A few models of the canonical cancer-involved cellular statuses/features were tested by such framework.Finally,we illustrated the drug repurposing pipeline,utilizing the training parameters derived from the adverse cellular statuses/features,which yielded successful validation results both in vitro and in vivo.SuperFeat is accessible at https://github.com/weilin-genomics/rSuperFeat.展开更多
In the Visual Place Recognition(VPR)task,existing research has leveraged large-scale pre-trained models to improve the performance of place recognition.However,when there are significant environmental differences betw...In the Visual Place Recognition(VPR)task,existing research has leveraged large-scale pre-trained models to improve the performance of place recognition.However,when there are significant environmental differences between query images and reference images,a large number of ineffective local features will interfere with the extraction of key landmark features,leading to the retrieval of visually similar but geographically different images.To address this perceptual aliasing problem caused by environmental condition changes,we propose a novel Visual Place Recognition method with Cross-Environment Robust Feature Enhancement(CerfeVPR).This method uses the GAN network to generate similar images of the original images under different environmental conditions,thereby enhancing the learning of robust features of the original images.This enables the global descriptor to effectively ignore appearance changes caused by environmental factors such as seasons and lighting,showing better place recognition accuracy than other methods.Meanwhile,we introduce a large kernel convolution adapter to fine tune the pre-trained model,obtaining a better image feature representation for subsequent robust feature learning.Then,we process the information of different local regions in the general features through a 3-layer pyramid scene parsing network and fuse it with a tag that retains global information to construct a multi-dimensional image feature representation.Based on this,we use the fused features of similar images to drive the robust feature learning of the original images and complete the feature matching between query images and retrieved images.Experiments on multiple commonly used datasets show that our method exhibits excellent performance.On average,CerfeVPR achieves the highest results,with all Recall@N values exceeding 90%.In particular,on the highly challenging Nordland dataset,the R@1 metric is improved by 4.6%,significantly outperforming other methods,which fully verifies the superiority of CerfeVPR in visual place recognition under complex environments.展开更多
In multi-label learning,the label-specific features learning framework can effectively solve the dimensional catastrophe problem brought by high-dimensional data.The classification performance and robustness of the mo...In multi-label learning,the label-specific features learning framework can effectively solve the dimensional catastrophe problem brought by high-dimensional data.The classification performance and robustness of the model are effectively improved.Most existing label-specific features learning utilizes the cosine similarity method to measure label correlation.It is well known that the correlation between labels is asymmetric.However,existing label-specific features learning only considers the private features of labels in classification and does not take into account the common features of labels.Based on this,this paper proposes a Causality-driven Common and Label-specific Features Learning,named CCSF algorithm.Firstly,the causal learning algorithm GSBN is used to calculate the asymmetric correlation between labels.Then,in the optimization,both l_(2,1)-norm and l_(1)-norm are used to select the corresponding features,respectively.Finally,it is compared with six state-of-the-art algorithms on nine datasets.The experimental results prove the effectiveness of the algorithm in this paper.展开更多
Emotion-based features are critical for achieving high performance in a speech emotion recognition(SER) system. In general, it is difficult to develop these features due to the ambiguity of the ground-truth. In this p...Emotion-based features are critical for achieving high performance in a speech emotion recognition(SER) system. In general, it is difficult to develop these features due to the ambiguity of the ground-truth. In this paper, we apply several unsupervised feature learning algorithms(including K-means clustering, the sparse auto-encoder, and sparse restricted Boltzmann machines), which have promise for learning task-related features by using unlabeled data, to speech emotion recognition. We then evaluate the performance of the proposed approach and present a detailed analysis of the effect of two important factors in the model setup, the content window size and the number of hidden layer nodes. Experimental results show that larger content windows and more hidden nodes contribute to higher performance. We also show that the two-layer network cannot explicitly improve performance compared to a single-layer network.展开更多
In some image classification tasks, similarities among different categories are different and the samples are usually misclassified as highly similar categories. To distinguish highly similar categories, more specific...In some image classification tasks, similarities among different categories are different and the samples are usually misclassified as highly similar categories. To distinguish highly similar categories, more specific features are required so that the classifier can improve the classification performance. In this paper, we propose a novel two-level hierarchical feature learning framework based on the deep convolutional neural network(CNN), which is simple and effective. First, the deep feature extractors of different levels are trained using the transfer learning method that fine-tunes the pre-trained deep CNN model toward the new target dataset. Second, the general feature extracted from all the categories and the specific feature extracted from highly similar categories are fused into a feature vector. Then the final feature representation is fed into a linear classifier. Finally, experiments using the Caltech-256, Oxford Flower-102, and Tasmania Coral Point Count(CPC) datasets demonstrate that the expression ability of the deep features resulting from two-level hierarchical feature learning is powerful. Our proposed method effectively increases the classification accuracy in comparison with flat multiple classification methods.展开更多
Fine-grained image search is one of the most challenging tasks in computer vision that aims to retrieve similar images at the fine-grained level for a given query image.The key objective is to learn discriminative fin...Fine-grained image search is one of the most challenging tasks in computer vision that aims to retrieve similar images at the fine-grained level for a given query image.The key objective is to learn discriminative fine-grained features by training deep models such that similar images are clustered,and dissimilar images are separated in the low embedding space.Previous works primarily focused on defining local structure loss functions like triplet loss,pairwise loss,etc.However,training via these approaches takes a long training time,and they have poor accuracy.Additionally,representations learned through it tend to tighten up in the embedded space and lose generalizability to unseen classes.This paper proposes a noise-assisted representation learning method for fine-grained image retrieval to mitigate these issues.In the proposed work,class manifold learning is performed in which positive pairs are created with noise insertion operation instead of tightening class clusters.And other instances are treated as negatives within the same cluster.Then a loss function is defined to penalize when the distance between instances of the same class becomes too small relative to the noise pair in that class in embedded space.The proposed approach is validated on CARS-196 and CUB-200 datasets and achieved better retrieval results(85.38%recall@1 for CARS-196%and 70.13%recall@1 for CUB-200)compared to other existing methods.展开更多
Time series anomaly detection is crucial in various industrial applications to identify unusual behaviors within the time series data.Due to the challenges associated with annotating anomaly events,time series reconst...Time series anomaly detection is crucial in various industrial applications to identify unusual behaviors within the time series data.Due to the challenges associated with annotating anomaly events,time series reconstruction has become a prevalent approach for unsupervised anomaly detection.However,effectively learning representations and achieving accurate detection results remain challenging due to the intricate temporal patterns and dependencies in real-world time series.In this paper,we propose a cross-dimension attentive feature fusion network for time series anomaly detection,referred to as CAFFN.Specifically,a series and feature mixing block is introduced to learn representations in 1D space.Additionally,a fast Fourier transform is employed to convert the time series into 2D space,providing the capability for 2D feature extraction.Finally,a cross-dimension attentive feature fusion mechanism is designed that adaptively integrates features across different dimensions for anomaly detection.Experimental results on real-world time series datasets demonstrate that CAFFN performs better than other competing methods in time series anomaly detection.展开更多
Existing fault diagnosis methods usually assume that there are balanced training data for every machine health state.However,the collection of fault signals is very difficult and expensive,resulting in the problem of ...Existing fault diagnosis methods usually assume that there are balanced training data for every machine health state.However,the collection of fault signals is very difficult and expensive,resulting in the problem of imbalanced training dataset.It will degrade the performance of fault diagnosis methods significantly.To address this problem,an imbalanced fault diagnosis of rotating machinery using autoencoder-based SuperGraph feature learning is proposed in this paper.Unsupervised autoencoder is firstly used to compress every monitoring signal into a low-dimensional vector as the node attribute in the SuperGraph.And the edge connections in the graph depend on the relationship between signals.On the basis,graph convolution is performed on the constructed SuperGraph to achieve imbalanced training dataset fault diagnosis for rotating machinery.Comprehensive experiments are conducted on a benchmarking publicized dataset and a practical experimental platform,and the results show that the proposed method can effectively achieve rotating machinery fault diagnosis towards imbalanced training dataset through graph feature learning.展开更多
In this paper we consider the problem of“end-to-end”digital camera identification by considering sequence of images obtained from the cameras.The problem of digital camera identification is harder than the problem o...In this paper we consider the problem of“end-to-end”digital camera identification by considering sequence of images obtained from the cameras.The problem of digital camera identification is harder than the problem of identifying its analog counterpart since the process of analog to digital conversion smooths out the intrinsic noise in the analog signal.However it is known that identifying a digital camera is possible by analyzing the camera’s intrinsic sensor artifacts that are introduced into the images/videos during the process of photo/video capture.It is known that such methods are computationally intensive requiring expensive pre-processing steps.In this paper we propose an end-to-end deep feature learning framework for identifying cameras using images obtained from them.We conduct experiments using three custom datasets:the first containing two cameras in an indoor environment where each camera may observe different scenes having no overlapping features,the second containing images from four cameras in an outdoor setting but where each camera observes scenes having overlapping features and the third containing images from two cameras observing the same checkerboard pattern in an indoor setting.Our results show that it is possible to capture the intrinsic hardware signature of the cameras using deep feature representations in an end-to-end framework.These deep feature maps can in turn be used to disambiguate the cameras from each another.Our system is end-to-end,requires no complicated pre-processing steps and the trained model is computationally efficient during testing,paving a way to have near instantaneous decisions for the problem of digital camera identification in production environments.Finally we present comparisons against the current state-of-the-art in digital camera identification which clearly establishes the superiority of the end-to-end solution.展开更多
In this paper,we report upon our recent work aimed at improving and adapting machine learning algorithms to automatically classify nanoscience images acquired by the Scanning Electron Microscope(SEM).This is done by c...In this paper,we report upon our recent work aimed at improving and adapting machine learning algorithms to automatically classify nanoscience images acquired by the Scanning Electron Microscope(SEM).This is done by coupling supervised and unsupervised learning approaches.We first investigate supervised learning on a ten-category data set of images and compare the performance of the different models in terms of training accuracy.Then,we reduce the dimensionality of the features through autoencoders to perform unsupervised learning on a subset of images in a selected range of scales(from 1μm to 2μm).Finally,we compare different clustering methods to uncover intrinsic structures in the images.展开更多
The quick spread of the CoronavirusDisease(COVID-19)infection around the world considered a real danger for global health.The biological structure and symptoms of COVID-19 are similar to other viral chest maladies,whi...The quick spread of the CoronavirusDisease(COVID-19)infection around the world considered a real danger for global health.The biological structure and symptoms of COVID-19 are similar to other viral chest maladies,which makes it challenging and a big issue to improve approaches for efficient identification of COVID-19 disease.In this study,an automatic prediction of COVID-19 identification is proposed to automatically discriminate between healthy and COVID-19 infected subjects in X-ray images using two successful moderns are traditional machine learning methods(e.g.,artificial neural network(ANN),support vector machine(SVM),linear kernel and radial basis function(RBF),k-nearest neighbor(k-NN),Decision Tree(DT),andCN2 rule inducer techniques)and deep learningmodels(e.g.,MobileNets V2,ResNet50,GoogleNet,DarkNet andXception).A largeX-ray dataset has been created and developed,namely the COVID-19 vs.Normal(400 healthy cases,and 400 COVID cases).To the best of our knowledge,it is currently the largest publicly accessible COVID-19 dataset with the largest number of X-ray images of confirmed COVID-19 infection cases.Based on the results obtained from the experiments,it can be concluded that all the models performed well,deep learning models had achieved the optimum accuracy of 98.8%in ResNet50 model.In comparison,in traditional machine learning techniques, the SVM demonstrated the best result for an accuracy of 95% and RBFaccuracy 94% for the prediction of coronavirus disease 2019.展开更多
In the era of Big data,learning discriminant feature representation from network traffic is identified has as an invariably essential task for improving the detection ability of an intrusion detection system(IDS).Owin...In the era of Big data,learning discriminant feature representation from network traffic is identified has as an invariably essential task for improving the detection ability of an intrusion detection system(IDS).Owing to the lack of accurately labeled network traffic data,many unsupervised feature representation learning models have been proposed with state-of-theart performance.Yet,these models fail to consider the classification error while learning the feature representation.Intuitively,the learnt feature representation may degrade the performance of the classification task.For the first time in the field of intrusion detection,this paper proposes an unsupervised IDS model leveraging the benefits of deep autoencoder(DAE)for learning the robust feature representation and one-class support vector machine(OCSVM)for finding the more compact decision hyperplane for intrusion detection.Specially,the proposed model defines a new unified objective function to minimize the reconstruction and classification error simultaneously.This unique contribution not only enables the model to support joint learning for feature representation and classifier training but also guides to learn the robust feature representation which can improve the discrimination ability of the classifier for intrusion detection.Three set of evaluation experiments are conducted to demonstrate the potential of the proposed model.First,the ablation evaluation on benchmark dataset,NSL-KDD validates the design decision of the proposed model.Next,the performance evaluation on recent intrusion dataset,UNSW-NB15 signifies the stable performance of the proposed model.Finally,the comparative evaluation verifies the efficacy of the proposed model against recently published state-of-the-art methods.展开更多
Gait recognition is an active research area that uses a walking theme to identify the subject correctly.Human Gait Recognition(HGR)is performed without any cooperation from the individual.However,in practice,it remain...Gait recognition is an active research area that uses a walking theme to identify the subject correctly.Human Gait Recognition(HGR)is performed without any cooperation from the individual.However,in practice,it remains a challenging task under diverse walking sequences due to the covariant factors such as normal walking and walking with wearing a coat.Researchers,over the years,have worked on successfully identifying subjects using different techniques,but there is still room for improvement in accuracy due to these covariant factors.This paper proposes an automated model-free framework for human gait recognition in this article.There are a few critical steps in the proposed method.Firstly,optical flow-based motion region esti-mation and dynamic coordinates-based cropping are performed.The second step involves training a fine-tuned pre-trained MobileNetV2 model on both original and optical flow cropped frames;the training has been conducted using static hyperparameters.The third step proposed a fusion technique known as normal distribution serially fusion.In the fourth step,a better optimization algorithm is applied to select the best features,which are then classified using a Bi-Layered neural network.Three publicly available datasets,CASIA A,CASIA B,and CASIA C,were used in the experimental process and obtained average accuracies of 99.6%,91.6%,and 95.02%,respectively.The proposed framework has achieved improved accuracy compared to the other methods.展开更多
Dear Sir,Iam Dr.Kavitha S,from the Department of Electronics and Communication Engineering,Nandha Engineering College,Erode,Tamil Nadu,India.I write to present the detection of glaucoma using extreme learning machine(...Dear Sir,Iam Dr.Kavitha S,from the Department of Electronics and Communication Engineering,Nandha Engineering College,Erode,Tamil Nadu,India.I write to present the detection of glaucoma using extreme learning machine(ELM)and fractal feature analysis.Glaucoma is the second most frequent cause of permanent blindness in industrial展开更多
Fault diagnosis occupies a pivotal position within the domain of machine and equipment management.Existing methods,however,often exhibit limitations in their scope of application,typically focusing on specific types o...Fault diagnosis occupies a pivotal position within the domain of machine and equipment management.Existing methods,however,often exhibit limitations in their scope of application,typically focusing on specific types of signals or faults in individual mechanical components while being constrained by data types and inherent characteristics.To address the limitations of existing methods,we propose a fault diagnosis method based on graph neural networks(GNNs)embedded with multirelationships of intrinsic mode functions(MIMF).The approach introduces a novel graph topological structure constructed from the features of intrinsic mode functions(IMFs)of monitored signals and their multirelationships.Additionally,a graph-level based fault diagnosis network model is designed to enhance feature learning capabilities for graph samples and enable flexible application across diverse signal sources and devices.Experimental validation with datasets including independent vibration signals for gear fault detection,mixed vibration signals for concurrent gear and bearing faults,and pressure signals for hydraulic cylinder leakage characterization demonstrates the model's adaptability and superior diagnostic accuracy across various types of signals and mechanical systems.展开更多
基金supported by National Natural Science Foundation of China(Nos.61034002,61233001 and 61273140)
文摘Tremendous amount of data are being generated and saved in many complex engineering and social systems every day.It is significant and feasible to utilize the big data to make better decisions by machine learning techniques. In this paper, we focus on batch reinforcement learning(RL) algorithms for discounted Markov decision processes(MDPs) with large discrete or continuous state spaces, aiming to learn the best possible policy given a fixed amount of training data. The batch RL algorithms with handcrafted feature representations work well for low-dimensional MDPs. However, for many real-world RL tasks which often involve high-dimensional state spaces, it is difficult and even infeasible to use feature engineering methods to design features for value function approximation. To cope with high-dimensional RL problems, the desire to obtain data-driven features has led to a lot of works in incorporating feature selection and feature learning into traditional batch RL algorithms. In this paper, we provide a comprehensive survey on automatic feature selection and unsupervised feature learning for high-dimensional batch RL. Moreover, we present recent theoretical developments on applying statistical learning to establish finite-sample error bounds for batch RL algorithms based on weighted Lpnorms. Finally, we derive some future directions in the research of RL algorithms, theories and applications.
基金supported by the NationalNatural Science Foundation of China (No.62107014,Jian P.,62177025,He B.)the Key R&D and Promotion Projects of Henan Province (No.212102210147,Jian P.)Innovative Education Program for Graduate Students at North China University of Water Resources and Electric Power,China (No.YK-2021-99,Guo F.).
文摘This paper presents an end-to-end deep learning method to solve geometry problems via feature learning and contrastive learning of multimodal data.A key challenge in solving geometry problems using deep learning is to automatically adapt to the task of understanding single-modal and multimodal problems.Existing methods either focus on single-modal ormultimodal problems,and they cannot fit each other.A general geometry problem solver shouldobviouslybe able toprocess variousmodalproblems at the same time.Inthispaper,a shared feature-learning model of multimodal data is adopted to learn the unified feature representation of text and image,which can solve the heterogeneity issue between multimodal geometry problems.A contrastive learning model of multimodal data enhances the semantic relevance betweenmultimodal features and maps them into a unified semantic space,which can effectively adapt to both single-modal and multimodal downstream tasks.Based on the feature extraction and fusion of multimodal data,a proposed geometry problem solver uses relation extraction,theorem reasoning,and problem solving to present solutions in a readable way.Experimental results show the effectiveness of the method.
文摘Industrial Control Systems (ICS) or SCADA networks are increasingly targeted by cyber-attacks as their architectures shifted from proprietary hardware, software and protocols to standard and open sources ones. Furthermore, these systems which used to be isolated are now interconnected to corporate networks and to the Internet. Among the countermeasures to mitigate the threats, anomaly detection systems play an important role as they can help detect even unknown attacks. Deep learning which has gained a great attention in the last few years due to excellent results in image, video and natural language processing is being used for anomaly detection in information security, particularly in SCADA networks. The salient features of the data from SCADA networks are learnt as hierarchical representation using deep architectures, and those learnt features are used to classify the data into normal or anomalous ones. This article is a review of various architectures such as Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), Stacked Autoencoder (SAE), Long Short Term Memory (LSTM), or a combination of those architectures, for anomaly detection purpose in SCADA networks.
基金Projects(62125306, 62133003) supported by the National Natural Science Foundation of ChinaProject(TPL2019C03) supported by the Open Fund of Science and Technology on Thermal Energy and Power Laboratory,ChinaProject supported by the Fundamental Research Funds for the Central Universities(Zhejiang University NGICS Platform),China。
文摘With the increasing complexity of industrial processes, the high-dimensional industrial data exhibit a strong nonlinearity, bringing considerable challenges to the fault diagnosis of industrial processes. To efficiently extract deep meaningful features that are crucial for fault diagnosis, a sparse Gaussian feature extractor(SGFE) is designed to learn a nonlinear mapping that projects the raw data into the feature space with the fault label dimension. The feature space is described by the one-hot encoding of the fault category label as an orthogonal basis. In this way, the deep sparse Gaussian features related to fault categories can be gradually learned from the raw data by SGFE. In the feature space,the sparse Gaussian(SG) loss function is designed to constrain the distribution of features to multiple sparse multivariate Gaussian distributions. The sparse Gaussian features are linearly separable in the feature space, which is conducive to improving the accuracy of the downstream fault classification task. The feasibility and practical utility of the proposed SGFE are verified by the handwritten digits MNIST benchmark and Tennessee-Eastman(TE) benchmark process,respectively.
基金supported by the Science and Technology Project of Henan Province(No.222102210081).
文摘Joint Multimodal Aspect-based Sentiment Analysis(JMASA)is a significant task in the research of multimodal fine-grained sentiment analysis,which combines two subtasks:Multimodal Aspect Term Extraction(MATE)and Multimodal Aspect-oriented Sentiment Classification(MASC).Currently,most existing models for JMASA only perform text and image feature encoding from a basic level,but often neglect the in-depth analysis of unimodal intrinsic features,which may lead to the low accuracy of aspect term extraction and the poor ability of sentiment prediction due to the insufficient learning of intra-modal features.Given this problem,we propose a Text-Image Feature Fine-grained Learning(TIFFL)model for JMASA.First,we construct an enhanced adjacency matrix of word dependencies and adopt graph convolutional network to learn the syntactic structure features for text,which addresses the context interference problem of identifying different aspect terms.Then,the adjective-noun pairs extracted from image are introduced to enable the semantic representation of visual features more intuitive,which addresses the ambiguous semantic extraction problem during image feature learning.Thereby,the model performance of aspect term extraction and sentiment polarity prediction can be further optimized and enhanced.Experiments on two Twitter benchmark datasets demonstrate that TIFFL achieves competitive results for JMASA,MATE and MASC,thus validating the effectiveness of our proposed methods.
基金supported by grants from the Shanghai Jiao Tong University,the Renji Hospital Start-up funding for New PI,the Natural Science Foundation of Shanghai Science and Technology Innovation Action Plan(Grant No.21ZR1441500)the Young Talent of Hunan(Grant No.2020RC3066)+1 种基金the Hunan Natural Science Fund for Excellent Young Scholars(Grant No.2021JJ20003)the China Postdoctoral Science Foundation(Grant No.2021T140197).
文摘In this study,we devised a computational framework called Supervised Feature Learning and Scoring(SuperFeat)which enables the training of a machine learning model and evaluates the canonical cellular statuses/features in pathological tissues that underlie the progression of disease.This framework also enables the identification of potential drugs that target the presumed detrimental cellular features.This framework was constructed on the basis of an artificial neural network with the gene expression profiles serving as input nodes.The training data comprised single-cell RNA sequencing datasets that encompassed the specific cell lineage during the developmental progression of cell features.A few models of the canonical cancer-involved cellular statuses/features were tested by such framework.Finally,we illustrated the drug repurposing pipeline,utilizing the training parameters derived from the adverse cellular statuses/features,which yielded successful validation results both in vitro and in vivo.SuperFeat is accessible at https://github.com/weilin-genomics/rSuperFeat.
基金supported by Postgraduate Scientific Research Innovation Project of Hunan Province CX20230915National Natural Science Foundation of China under Grant 62472440.
文摘In the Visual Place Recognition(VPR)task,existing research has leveraged large-scale pre-trained models to improve the performance of place recognition.However,when there are significant environmental differences between query images and reference images,a large number of ineffective local features will interfere with the extraction of key landmark features,leading to the retrieval of visually similar but geographically different images.To address this perceptual aliasing problem caused by environmental condition changes,we propose a novel Visual Place Recognition method with Cross-Environment Robust Feature Enhancement(CerfeVPR).This method uses the GAN network to generate similar images of the original images under different environmental conditions,thereby enhancing the learning of robust features of the original images.This enables the global descriptor to effectively ignore appearance changes caused by environmental factors such as seasons and lighting,showing better place recognition accuracy than other methods.Meanwhile,we introduce a large kernel convolution adapter to fine tune the pre-trained model,obtaining a better image feature representation for subsequent robust feature learning.Then,we process the information of different local regions in the general features through a 3-layer pyramid scene parsing network and fuse it with a tag that retains global information to construct a multi-dimensional image feature representation.Based on this,we use the fused features of similar images to drive the robust feature learning of the original images and complete the feature matching between query images and retrieved images.Experiments on multiple commonly used datasets show that our method exhibits excellent performance.On average,CerfeVPR achieves the highest results,with all Recall@N values exceeding 90%.In particular,on the highly challenging Nordland dataset,the R@1 metric is improved by 4.6%,significantly outperforming other methods,which fully verifies the superiority of CerfeVPR in visual place recognition under complex environments.
基金2022 University Research Priorities,No.2022AH051989.
文摘In multi-label learning,the label-specific features learning framework can effectively solve the dimensional catastrophe problem brought by high-dimensional data.The classification performance and robustness of the model are effectively improved.Most existing label-specific features learning utilizes the cosine similarity method to measure label correlation.It is well known that the correlation between labels is asymmetric.However,existing label-specific features learning only considers the private features of labels in classification and does not take into account the common features of labels.Based on this,this paper proposes a Causality-driven Common and Label-specific Features Learning,named CCSF algorithm.Firstly,the causal learning algorithm GSBN is used to calculate the asymmetric correlation between labels.Then,in the optimization,both l_(2,1)-norm and l_(1)-norm are used to select the corresponding features,respectively.Finally,it is compared with six state-of-the-art algorithms on nine datasets.The experimental results prove the effectiveness of the algorithm in this paper.
基金supported by the National Natural Science Foundation of China(Nos.61272211 and 61170126)the Six Talent Peaks Foundation of Jiangsu Province,China(No.DZXX027)
文摘Emotion-based features are critical for achieving high performance in a speech emotion recognition(SER) system. In general, it is difficult to develop these features due to the ambiguity of the ground-truth. In this paper, we apply several unsupervised feature learning algorithms(including K-means clustering, the sparse auto-encoder, and sparse restricted Boltzmann machines), which have promise for learning task-related features by using unlabeled data, to speech emotion recognition. We then evaluate the performance of the proposed approach and present a detailed analysis of the effect of two important factors in the model setup, the content window size and the number of hidden layer nodes. Experimental results show that larger content windows and more hidden nodes contribute to higher performance. We also show that the two-layer network cannot explicitly improve performance compared to a single-layer network.
基金Project supported by the National Natural Science Foundation of China(No.61379074)the Zhejiang Provincial Natural Science Foundation of China(Nos.LZ12F02003 and LY15F020035)
文摘In some image classification tasks, similarities among different categories are different and the samples are usually misclassified as highly similar categories. To distinguish highly similar categories, more specific features are required so that the classifier can improve the classification performance. In this paper, we propose a novel two-level hierarchical feature learning framework based on the deep convolutional neural network(CNN), which is simple and effective. First, the deep feature extractors of different levels are trained using the transfer learning method that fine-tunes the pre-trained deep CNN model toward the new target dataset. Second, the general feature extracted from all the categories and the specific feature extracted from highly similar categories are fused into a feature vector. Then the final feature representation is fed into a linear classifier. Finally, experiments using the Caltech-256, Oxford Flower-102, and Tasmania Coral Point Count(CPC) datasets demonstrate that the expression ability of the deep features resulting from two-level hierarchical feature learning is powerful. Our proposed method effectively increases the classification accuracy in comparison with flat multiple classification methods.
文摘Fine-grained image search is one of the most challenging tasks in computer vision that aims to retrieve similar images at the fine-grained level for a given query image.The key objective is to learn discriminative fine-grained features by training deep models such that similar images are clustered,and dissimilar images are separated in the low embedding space.Previous works primarily focused on defining local structure loss functions like triplet loss,pairwise loss,etc.However,training via these approaches takes a long training time,and they have poor accuracy.Additionally,representations learned through it tend to tighten up in the embedded space and lose generalizability to unseen classes.This paper proposes a noise-assisted representation learning method for fine-grained image retrieval to mitigate these issues.In the proposed work,class manifold learning is performed in which positive pairs are created with noise insertion operation instead of tightening class clusters.And other instances are treated as negatives within the same cluster.Then a loss function is defined to penalize when the distance between instances of the same class becomes too small relative to the noise pair in that class in embedded space.The proposed approach is validated on CARS-196 and CUB-200 datasets and achieved better retrieval results(85.38%recall@1 for CARS-196%and 70.13%recall@1 for CUB-200)compared to other existing methods.
基金supported in part by the National Natural Science Foundation of China(Grants 62376172,62006163,62376043)in part by the National Postdoctoral Program for Innovative Talents(Grant BX20200226)in part by Sichuan Science and Technology Planning Project(Grants 2022YFSY0047,2022YFQ0014,2023ZYD0143,2022YFH0021,2023YFQ0020,24QYCX0354,24NSFTD0025).
文摘Time series anomaly detection is crucial in various industrial applications to identify unusual behaviors within the time series data.Due to the challenges associated with annotating anomaly events,time series reconstruction has become a prevalent approach for unsupervised anomaly detection.However,effectively learning representations and achieving accurate detection results remain challenging due to the intricate temporal patterns and dependencies in real-world time series.In this paper,we propose a cross-dimension attentive feature fusion network for time series anomaly detection,referred to as CAFFN.Specifically,a series and feature mixing block is introduced to learn representations in 1D space.Additionally,a fast Fourier transform is employed to convert the time series into 2D space,providing the capability for 2D feature extraction.Finally,a cross-dimension attentive feature fusion mechanism is designed that adaptively integrates features across different dimensions for anomaly detection.Experimental results on real-world time series datasets demonstrate that CAFFN performs better than other competing methods in time series anomaly detection.
基金This work was supported by the National Key R&D Program of China(Grant No.2020YFB1711203).
文摘Existing fault diagnosis methods usually assume that there are balanced training data for every machine health state.However,the collection of fault signals is very difficult and expensive,resulting in the problem of imbalanced training dataset.It will degrade the performance of fault diagnosis methods significantly.To address this problem,an imbalanced fault diagnosis of rotating machinery using autoencoder-based SuperGraph feature learning is proposed in this paper.Unsupervised autoencoder is firstly used to compress every monitoring signal into a low-dimensional vector as the node attribute in the SuperGraph.And the edge connections in the graph depend on the relationship between signals.On the basis,graph convolution is performed on the constructed SuperGraph to achieve imbalanced training dataset fault diagnosis for rotating machinery.Comprehensive experiments are conducted on a benchmarking publicized dataset and a practical experimental platform,and the results show that the proposed method can effectively achieve rotating machinery fault diagnosis towards imbalanced training dataset through graph feature learning.
文摘In this paper we consider the problem of“end-to-end”digital camera identification by considering sequence of images obtained from the cameras.The problem of digital camera identification is harder than the problem of identifying its analog counterpart since the process of analog to digital conversion smooths out the intrinsic noise in the analog signal.However it is known that identifying a digital camera is possible by analyzing the camera’s intrinsic sensor artifacts that are introduced into the images/videos during the process of photo/video capture.It is known that such methods are computationally intensive requiring expensive pre-processing steps.In this paper we propose an end-to-end deep feature learning framework for identifying cameras using images obtained from them.We conduct experiments using three custom datasets:the first containing two cameras in an indoor environment where each camera may observe different scenes having no overlapping features,the second containing images from four cameras in an outdoor setting but where each camera observes scenes having overlapping features and the third containing images from two cameras observing the same checkerboard pattern in an indoor setting.Our results show that it is possible to capture the intrinsic hardware signature of the cameras using deep feature representations in an end-to-end framework.These deep feature maps can in turn be used to disambiguate the cameras from each another.Our system is end-to-end,requires no complicated pre-processing steps and the trained model is computationally efficient during testing,paving a way to have near instantaneous decisions for the problem of digital camera identification in production environments.Finally we present comparisons against the current state-of-the-art in digital camera identification which clearly establishes the superiority of the end-to-end solution.
基金This work has been done within the NFFA-EUROPE project and has received funding from the European Union’s Horizon 2020 Research and Innovation Program under grant agreement No.654360 NFFA-EUROPE.
文摘In this paper,we report upon our recent work aimed at improving and adapting machine learning algorithms to automatically classify nanoscience images acquired by the Scanning Electron Microscope(SEM).This is done by coupling supervised and unsupervised learning approaches.We first investigate supervised learning on a ten-category data set of images and compare the performance of the different models in terms of training accuracy.Then,we reduce the dimensionality of the features through autoencoders to perform unsupervised learning on a subset of images in a selected range of scales(from 1μm to 2μm).Finally,we compare different clustering methods to uncover intrinsic structures in the images.
文摘The quick spread of the CoronavirusDisease(COVID-19)infection around the world considered a real danger for global health.The biological structure and symptoms of COVID-19 are similar to other viral chest maladies,which makes it challenging and a big issue to improve approaches for efficient identification of COVID-19 disease.In this study,an automatic prediction of COVID-19 identification is proposed to automatically discriminate between healthy and COVID-19 infected subjects in X-ray images using two successful moderns are traditional machine learning methods(e.g.,artificial neural network(ANN),support vector machine(SVM),linear kernel and radial basis function(RBF),k-nearest neighbor(k-NN),Decision Tree(DT),andCN2 rule inducer techniques)and deep learningmodels(e.g.,MobileNets V2,ResNet50,GoogleNet,DarkNet andXception).A largeX-ray dataset has been created and developed,namely the COVID-19 vs.Normal(400 healthy cases,and 400 COVID cases).To the best of our knowledge,it is currently the largest publicly accessible COVID-19 dataset with the largest number of X-ray images of confirmed COVID-19 infection cases.Based on the results obtained from the experiments,it can be concluded that all the models performed well,deep learning models had achieved the optimum accuracy of 98.8%in ResNet50 model.In comparison,in traditional machine learning techniques, the SVM demonstrated the best result for an accuracy of 95% and RBFaccuracy 94% for the prediction of coronavirus disease 2019.
基金This work was supported by the Research Deanship of Prince Sattam Bin Abdulaziz University,Al-Kharj,Saudi Arabia(Grant No.2020/01/17215).Also,the author thanks Deanship of college of computer engineering and sciences for technical support provided to complete the project successfully。
文摘In the era of Big data,learning discriminant feature representation from network traffic is identified has as an invariably essential task for improving the detection ability of an intrusion detection system(IDS).Owing to the lack of accurately labeled network traffic data,many unsupervised feature representation learning models have been proposed with state-of-theart performance.Yet,these models fail to consider the classification error while learning the feature representation.Intuitively,the learnt feature representation may degrade the performance of the classification task.For the first time in the field of intrusion detection,this paper proposes an unsupervised IDS model leveraging the benefits of deep autoencoder(DAE)for learning the robust feature representation and one-class support vector machine(OCSVM)for finding the more compact decision hyperplane for intrusion detection.Specially,the proposed model defines a new unified objective function to minimize the reconstruction and classification error simultaneously.This unique contribution not only enables the model to support joint learning for feature representation and classifier training but also guides to learn the robust feature representation which can improve the discrimination ability of the classifier for intrusion detection.Three set of evaluation experiments are conducted to demonstrate the potential of the proposed model.First,the ablation evaluation on benchmark dataset,NSL-KDD validates the design decision of the proposed model.Next,the performance evaluation on recent intrusion dataset,UNSW-NB15 signifies the stable performance of the proposed model.Finally,the comparative evaluation verifies the efficacy of the proposed model against recently published state-of-the-art methods.
基金supported by“Human Resources Program in Energy Technology”of the Korea Institute of Energy Technology Evaluation and Planning(KETEP)granted financial resources from the Ministry of Trade,Industry&Energy,Republic of Korea.(No.20204010600090).
文摘Gait recognition is an active research area that uses a walking theme to identify the subject correctly.Human Gait Recognition(HGR)is performed without any cooperation from the individual.However,in practice,it remains a challenging task under diverse walking sequences due to the covariant factors such as normal walking and walking with wearing a coat.Researchers,over the years,have worked on successfully identifying subjects using different techniques,but there is still room for improvement in accuracy due to these covariant factors.This paper proposes an automated model-free framework for human gait recognition in this article.There are a few critical steps in the proposed method.Firstly,optical flow-based motion region esti-mation and dynamic coordinates-based cropping are performed.The second step involves training a fine-tuned pre-trained MobileNetV2 model on both original and optical flow cropped frames;the training has been conducted using static hyperparameters.The third step proposed a fusion technique known as normal distribution serially fusion.In the fourth step,a better optimization algorithm is applied to select the best features,which are then classified using a Bi-Layered neural network.Three publicly available datasets,CASIA A,CASIA B,and CASIA C,were used in the experimental process and obtained average accuracies of 99.6%,91.6%,and 95.02%,respectively.The proposed framework has achieved improved accuracy compared to the other methods.
文摘Dear Sir,Iam Dr.Kavitha S,from the Department of Electronics and Communication Engineering,Nandha Engineering College,Erode,Tamil Nadu,India.I write to present the detection of glaucoma using extreme learning machine(ELM)and fractal feature analysis.Glaucoma is the second most frequent cause of permanent blindness in industrial
文摘Fault diagnosis occupies a pivotal position within the domain of machine and equipment management.Existing methods,however,often exhibit limitations in their scope of application,typically focusing on specific types of signals or faults in individual mechanical components while being constrained by data types and inherent characteristics.To address the limitations of existing methods,we propose a fault diagnosis method based on graph neural networks(GNNs)embedded with multirelationships of intrinsic mode functions(MIMF).The approach introduces a novel graph topological structure constructed from the features of intrinsic mode functions(IMFs)of monitored signals and their multirelationships.Additionally,a graph-level based fault diagnosis network model is designed to enhance feature learning capabilities for graph samples and enable flexible application across diverse signal sources and devices.Experimental validation with datasets including independent vibration signals for gear fault detection,mixed vibration signals for concurrent gear and bearing faults,and pressure signals for hydraulic cylinder leakage characterization demonstrates the model's adaptability and superior diagnostic accuracy across various types of signals and mechanical systems.
