Long-termpetroleum production forecasting is essential for the effective development andmanagement of oilfields.Due to its ability to extract complex patterns,deep learning has gained popularity for production forecas...Long-termpetroleum production forecasting is essential for the effective development andmanagement of oilfields.Due to its ability to extract complex patterns,deep learning has gained popularity for production forecasting.However,existing deep learning models frequently overlook the selective utilization of information from other production wells,resulting in suboptimal performance in long-term production forecasting across multiple wells.To achieve accurate long-term petroleum production forecast,we propose a spatial-geological perception graph convolutional neural network(SGP-GCN)that accounts for the temporal,spatial,and geological dependencies inherent in petroleum production.Utilizing the attention mechanism,the SGP-GCN effectively captures intricate correlations within production and geological data,forming the representations of each production well.Based on the spatial distances and geological feature correlations,we construct a spatial-geological matrix as the weight matrix to enable differential utilization of information from other wells.Additionally,a matrix sparsification algorithm based on production clustering(SPC)is also proposed to optimize the weight distribution within the spatial-geological matrix,thereby enhancing long-term forecasting performance.Empirical evaluations have shown that the SGP-GCN outperforms existing deep learning models,such as CNN-LSTM-SA,in long-term petroleum production forecasting.This demonstrates the potential of the SGP-GCN as a valuable tool for long-term petroleum production forecasting across multiple wells.展开更多
The data generated from non-Euclidean domains and its graphical representation(with complex-relationship object interdependence)applications has observed an exponential growth.The sophistication of graph data has pose...The data generated from non-Euclidean domains and its graphical representation(with complex-relationship object interdependence)applications has observed an exponential growth.The sophistication of graph data has posed consequential obstacles to the existing machine learning algorithms.In this study,we have considered a revamped version of a semi-supervised learning algorithm for graph-structured data to address the issue of expanding deep learning approaches to represent the graph data.Additionally,the quantum information theory has been applied through Graph Neural Networks(GNNs)to generate Riemannian metrics in closed-form of several graph layers.In further,to pre-process the adjacency matrix of graphs,a new formulation is established to incorporate high order proximities.The proposed scheme has shown outstanding improvements to overcome the deficiencies in Graph Convolutional Network(GCN),particularly,the information loss and imprecise information representation with acceptable computational overhead.Moreover,the proposed Quantum Graph Convolutional Network(QGCN)has significantly strengthened the GCN on semi-supervised node classification tasks.In parallel,it expands the generalization process with a significant difference by making small random perturbationsG of the graph during the training process.The evaluation results are provided on three benchmark datasets,including Citeseer,Cora,and PubMed,that distinctly delineate the superiority of the proposed model in terms of computational accuracy against state-of-the-art GCN and three other methods based on the same algorithms in the existing literature.展开更多
Self-powered neutron detectors(SPNDs)play a critical role in monitoring the safety margins and overall health of reactors,directly affecting safe operation within the reactor.In this work,a novel fault identification ...Self-powered neutron detectors(SPNDs)play a critical role in monitoring the safety margins and overall health of reactors,directly affecting safe operation within the reactor.In this work,a novel fault identification method based on graph convolutional networks(GCN)and Stacking ensemble learning is proposed for SPNDs.The GCN is employed to extract the spatial neighborhood information of SPNDs at different positions,and residuals are obtained by nonlinear fitting of SPND signals.In order to completely extract the time-varying features from residual sequences,the Stacking fusion model,integrated with various algorithms,is developed and enables the identification of five conditions for SPNDs:normal,drift,bias,precision degradation,and complete failure.The results demonstrate that the integration of diverse base-learners in the GCN-Stacking model exhibits advantages over a single model as well as enhances the stability and reliability in fault identification.Additionally,the GCN-Stacking model maintains higher accuracy in identifying faults at different reactor power levels.展开更多
To achieve good results in convolutional neural networks(CNN) for text classification task, term-based pooling operation in CNNs is proposed. Firstly, the convolution results of several convolution kernels are combine...To achieve good results in convolutional neural networks(CNN) for text classification task, term-based pooling operation in CNNs is proposed. Firstly, the convolution results of several convolution kernels are combined by this method, and then the results after combination are made pooling operation, three sorts of CNN models(we named TBCNN, MCT-CNN and MMCT-CNN respectively) are constructed and then corresponding algorithmic thought are detailed on this basis. Secondly, relevant experiments and analyses are respectively designed to show the effects of three key parameters(convolution kernel, combination kernel number and word embedding) on three kinds of CNN models and to further demonstrate the effect of the models proposed. The experimental results show that compared with the traditional method of text classification in CNNs, term-based pooling method is addressed that not only the availability of the way is proved, but also the performance shows good superiority.展开更多
With the high-speed development of the Internet,a growing number of Internet users like giving their subjective comments in the BBS,blog and shopping website.These comments contains critics’attitudes,emotions,views a...With the high-speed development of the Internet,a growing number of Internet users like giving their subjective comments in the BBS,blog and shopping website.These comments contains critics’attitudes,emotions,views and other information.Using these information reasonablely can help understand the social public opinion and make a timely response and help dealer to improve quality and service of products and make consumers know merchandise.This paper mainly discusses using convolutional neural network(CNN)for the operation of the text feature extraction.The concrete realization are discussed.Then combining with other text classifier make class operation.The experiment result shows the effectiveness of the method which is proposed in this paper.展开更多
Graph convolutional networks(GCNs)have received significant attention from various research fields due to the excellent performance in learning graph representations.Although GCN performs well compared with other meth...Graph convolutional networks(GCNs)have received significant attention from various research fields due to the excellent performance in learning graph representations.Although GCN performs well compared with other methods,it still faces challenges.Training a GCN model for large-scale graphs in a conventional way requires high computation and storage costs.Therefore,motivated by an urgent need in terms of efficiency and scalability in training GCN,sampling methods have been proposed and achieved a significant effect.In this paper,we categorize sampling methods based on the sampling mechanisms and provide a comprehensive survey of sampling methods for efficient training of GCN.To highlight the characteristics and differences of sampling methods,we present a detailed comparison within each category and further give an overall comparative analysis for the sampling methods in all categories.Finally,we discuss some challenges and future research directions of the sampling methods.展开更多
Existing solutions do not work well when multi-targets coexist in a sentence.The reason is that the existing solution is usually to separate multiple targets and process them separately.If the original sentence has N ...Existing solutions do not work well when multi-targets coexist in a sentence.The reason is that the existing solution is usually to separate multiple targets and process them separately.If the original sentence has N target,the original sentence will be repeated for N times,and only one target will be processed each time.To some extent,this approach degenerates the fine-grained sentiment classification task into the sentence-level sentiment classification task,and the research method of processing the target separately ignores the internal relation and interaction between the targets.Based on the above considerations,we proposes to use Graph Convolutional Network(GCN)to model and process multi-targets appearing in sentences at the same time based on the positional relationship,and then to construct a graph of the sentiment relationship between targets based on the difference of the sentiment polarity between target words.In addition to the standard target-dependent sentiment classification task,an auxiliary node relation classification task is constructed.Experiments demonstrate that our model achieves new comparable performance on the benchmark datasets:SemEval-2014 Task 4,i.e.,reviews for restaurants and laptops.Furthermore,the method of dividing the target words into isolated individuals has disadvantages,and the multi-task learning model is beneficial to enhance the feature extraction ability and expression ability of the model.展开更多
Existing traffic flow prediction frameworks have already achieved enormous success due to large traffic datasets and capability of deep learning models.However,data privacy and security are always a challenge in every...Existing traffic flow prediction frameworks have already achieved enormous success due to large traffic datasets and capability of deep learning models.However,data privacy and security are always a challenge in every field where data need to be uploaded to the cloud.Federated learning(FL)is an emerging trend for distributed training of data.The primary goal of FL is to train an efficient communication model without compromising data privacy.The traffic data have a robust spatio-temporal correlation,but various approaches proposed earlier have not considered spatial correlation of the traffic data.This paper presents FL-based traffic flow prediction with spatio-temporal correlation.This work uses a differential privacy(DP)scheme for privacy preservation of participant's data.To the best of our knowledge,this is the first time that FL is used for vehicular traffic prediction while considering the spatio-temporal correlation of traffic data with DP preservation.The proposed framework trains the data locally at the client-side with DP.It then uses the model aggregation mechanism federated graph convolutional network(FedGCN)at the server-side to find the average of locally trained models.The results of the proposed work show that the FedGCN model accurately predicts the traffic.DP scheme at client-side helps clients to set a budget for privacy loss.展开更多
The continuous improvement of the cyber threat intelligence sharing mechanism provides new ideas to deal with Advanced Persistent Threats(APT).Extracting attack behaviors,i.e.,Tactics,Techniques,Procedures(TTP)from Cy...The continuous improvement of the cyber threat intelligence sharing mechanism provides new ideas to deal with Advanced Persistent Threats(APT).Extracting attack behaviors,i.e.,Tactics,Techniques,Procedures(TTP)from Cyber Threat Intelligence(CTI)can facilitate APT actors’profiling for an immediate response.However,it is difficult for traditional manual methods to analyze attack behaviors from cyber threat intelligence due to its heterogeneous nature.Based on the Adversarial Tactics,Techniques and Common Knowledge(ATT&CK)of threat behavior description,this paper proposes a threat behavioral knowledge extraction framework that integrates Heterogeneous Text Network(HTN)and Graph Convolutional Network(GCN)to solve this issue.It leverages the hierarchical correlation relationships of attack techniques and tactics in the ATT&CK to construct a text network of heterogeneous cyber threat intelligence.With the help of the Bidirectional EncoderRepresentation fromTransformers(BERT)pretraining model to analyze the contextual semantics of cyber threat intelligence,the task of threat behavior identification is transformed into a text classification task,which automatically extracts attack behavior in CTI,then identifies the malware and advanced threat actors.The experimental results show that F1 achieve 94.86%and 92.15%for the multi-label classification tasks of tactics and techniques.Extend the experiment to verify the method’s effectiveness in identifying the malware and threat actors in APT attacks.The F1 for malware and advanced threat actors identification task reached 98.45%and 99.48%,which are better than the benchmark model in the experiment and achieve state of the art.The model can effectivelymodel threat intelligence text data and acquire knowledge and experience migration by correlating implied features with a priori knowledge to compensate for insufficient sample data and improve the classification performance and recognition ability of threat behavior in text.展开更多
Micro-expressions are spontaneous, unconscious movements that reveal true emotions.Accurate facial movement information and network training learning methods are crucial for micro-expression recognition.However, most ...Micro-expressions are spontaneous, unconscious movements that reveal true emotions.Accurate facial movement information and network training learning methods are crucial for micro-expression recognition.However, most existing micro-expression recognition technologies so far focus on modeling the single category of micro-expression images and neural network structure.Aiming at the problems of low recognition rate and weak model generalization ability in micro-expression recognition, a micro-expression recognition algorithm is proposed based on graph convolution network(GCN) and Transformer model.Firstly, action unit(AU) feature detection is extracted and facial muscle nodes in the neighborhood are divided into three subsets for recognition.Then, graph convolution layer is used to find the layout of dependencies between AU nodes of micro-expression classification.Finally, multiple attentional features of each facial action are enriched with Transformer model to include more sequence information before calculating the overall correlation of each region.The proposed method is validated in CASME II and CAS(ME)^2 datasets, and the recognition rate reached 69.85%.展开更多
Objective To construct symptom-formula-herb heterogeneous graphs structured Treatise on Febrile Diseases(Shang Han Lun,《伤寒论》)dataset and explore an optimal learning method represented with node attributes based o...Objective To construct symptom-formula-herb heterogeneous graphs structured Treatise on Febrile Diseases(Shang Han Lun,《伤寒论》)dataset and explore an optimal learning method represented with node attributes based on graph convolutional network(GCN).Methods Clauses that contain symptoms,formulas,and herbs were abstracted from Treatise on Febrile Diseases to construct symptom-formula-herb heterogeneous graphs,which were used to propose a node representation learning method based on GCN−the Traditional Chinese Medicine Graph Convolution Network(TCM-GCN).The symptom-formula,symptom-herb,and formula-herb heterogeneous graphs were processed with the TCM-GCN to realize high-order propagating message passing and neighbor aggregation to obtain new node representation attributes,and thus acquiring the nodes’sum-aggregations of symptoms,formulas,and herbs to lay a foundation for the downstream tasks of the prediction models.Results Comparisons among the node representations with multi-hot encoding,non-fusion encoding,and fusion encoding showed that the Precision@10,Recall@10,and F1-score@10 of the fusion encoding were 9.77%,6.65%,and 8.30%,respectively,higher than those of the non-fusion encoding in the prediction studies of the model.Conclusion Node representations by fusion encoding achieved comparatively ideal results,indicating the TCM-GCN is effective in realizing node-level representations of heterogeneous graph structured Treatise on Febrile Diseases dataset and is able to elevate the performance of the downstream tasks of the diagnosis model.展开更多
The traditional malware research is mainly based on its recognition and detection as a breakthrough point,without focusing on its propagation trends or predicting the subsequently infected nodes.The complexity of netw...The traditional malware research is mainly based on its recognition and detection as a breakthrough point,without focusing on its propagation trends or predicting the subsequently infected nodes.The complexity of network structure,diversity of network nodes,and sparsity of data all pose difficulties in predicting propagation.This paper proposes a malware propagation prediction model based on representation learning and Graph Convolutional Networks(GCN)to address the aforementioned problems.First,to solve the problem of the inaccuracy of infection intensity calculation caused by the sparsity of node interaction behavior data in the malware propagation network,a mechanism based on a tensor to mine the infection intensity among nodes is proposed to retain the network structure information.The influence of the relationship between nodes on the infection intensity is also analyzed.Second,given the diversity and complexity of the content and structure of infected and normal nodes in the network,considering the advantages of representation learning in data feature extraction,the corresponding representation learning method is adopted for the characteristics of infection intensity among nodes.This can efficiently calculate the relationship between entities and relationships in low dimensional space to achieve the goal of low dimensional,dense,and real-valued representation learning for the characteristics of propagation spatial data.We also design a new method,Tensor2vec,to learn the potential structural features of malware propagation.Finally,considering the convolution ability of GCN for non-Euclidean data,we propose a dynamic prediction model of malware propagation based on representation learning and GCN to solve the time effectiveness problem of the malware propagation carrier.The experimental results show that the proposed model can effectively predict the behaviors of the nodes in the network and discover the influence of different characteristics of nodes on the malware propagation situation.展开更多
Objective To develop a model based on a graph convolutional network(GCN)to achieve ef-ficient classification of the cold and hot medicinal properties of Chinese herbal medicines(CHMs).Methods After screening the datas...Objective To develop a model based on a graph convolutional network(GCN)to achieve ef-ficient classification of the cold and hot medicinal properties of Chinese herbal medicines(CHMs).Methods After screening the dataset provided in the published literature,this study includ-ed 495 CHMs and their 8075 compounds.Three molecular descriptors were used to repre-sent the compounds:the molecular access system(MACCS),extended connectivity finger-print(ECFP),and two-dimensional(2D)molecular descriptors computed by the RDKit open-source toolkit(RDKit_2D).A homogeneous graph with CHMs as nodes was constructed and a classification model for the cold and hot medicinal properties of CHMs was developed based on a GCN using the molecular descriptor information of the compounds as node features.Fi-nally,using accuracy and F1 score to evaluate model performance,the GCN model was ex-perimentally compared with the traditional machine learning approaches,including decision tree(DT),random forest(RF),k-nearest neighbor(KNN),Naïve Bayes classifier(NBC),and support vector machine(SVM).MACCS,ECFP,and RDKit_2D molecular descriptors were al-so adopted as features for comparison.Results The experimental results show that the GCN achieved better performance than the traditional machine learning approach when using MACCS as features,with the accuracy and F1 score reaching 0.8364 and 0.8453,respectively.The accuracy and F1 score have increased by 0.8690 and 0.8120,respectively,compared with the lowest performing feature combina-tion OMER(only the combination of MACCS,ECFP,and RDKit_2D).The accuracy and F1 score of DT,RF,KNN,NBC,and SVM are 0.5051 and 0.5018,0.6162 and 0.6015,0.6768 and 0.6243,0.6162 and 0.6071,0.6364 and 0.6225,respectively.Conclusion In this study,by introducing molecular descriptors as features,it is verified that molecular descriptors and fingerprints play a key role in classifying the cold and hot medici-nal properties of CHMs.Meanwhile,excellent classification performance was achieved using the GCN model,providing an important algorithmic basis for the in-depth study of the“struc-ture-property”relationship of CHMs.展开更多
The past decade has seen the rapid development of text detection based on deep learning.However,current methods of Chinese character detection and recognition have proven to be poor.The accuracy of segmenting text box...The past decade has seen the rapid development of text detection based on deep learning.However,current methods of Chinese character detection and recognition have proven to be poor.The accuracy of segmenting text boxes in natural scenes is not impressive.The reasons for this strait can be summarized into two points:the complexity of natural scenes and numerous types of Chinese characters.In response to these problems,we proposed a lightweight neural network architecture named CTSF.It consists of two modules,one is a text detection network that combines CTPN and the image feature extraction modules of PVANet,named CDSE.The other is a literacy network based on spatial pyramid pool and fusion of Chinese character skeleton features named SPPCNN-SF,so as to realize the text detection and recognition,respectively.Our model performs much better than the original model on ICDAR2011 and ICDAR2013(achieved 85%and 88%F-measures)and enhanced the processing speed in training phase.In addition,our method achieves extremely performance on three Chinese datasets,with accuracy of 95.12%,95.56%and 96.01%.展开更多
基金funded by National Natural Science Foundation of China,grant number 62071491.
文摘Long-termpetroleum production forecasting is essential for the effective development andmanagement of oilfields.Due to its ability to extract complex patterns,deep learning has gained popularity for production forecasting.However,existing deep learning models frequently overlook the selective utilization of information from other production wells,resulting in suboptimal performance in long-term production forecasting across multiple wells.To achieve accurate long-term petroleum production forecast,we propose a spatial-geological perception graph convolutional neural network(SGP-GCN)that accounts for the temporal,spatial,and geological dependencies inherent in petroleum production.Utilizing the attention mechanism,the SGP-GCN effectively captures intricate correlations within production and geological data,forming the representations of each production well.Based on the spatial distances and geological feature correlations,we construct a spatial-geological matrix as the weight matrix to enable differential utilization of information from other wells.Additionally,a matrix sparsification algorithm based on production clustering(SPC)is also proposed to optimize the weight distribution within the spatial-geological matrix,thereby enhancing long-term forecasting performance.Empirical evaluations have shown that the SGP-GCN outperforms existing deep learning models,such as CNN-LSTM-SA,in long-term petroleum production forecasting.This demonstrates the potential of the SGP-GCN as a valuable tool for long-term petroleum production forecasting across multiple wells.
基金supported by the National Key Research and Development Program of China(2018YFB1600600)the National Natural Science Foundation of China under(61976034,U1808206)the Dalian Science and Technology Innovation Fund(2019J12GX035).
文摘The data generated from non-Euclidean domains and its graphical representation(with complex-relationship object interdependence)applications has observed an exponential growth.The sophistication of graph data has posed consequential obstacles to the existing machine learning algorithms.In this study,we have considered a revamped version of a semi-supervised learning algorithm for graph-structured data to address the issue of expanding deep learning approaches to represent the graph data.Additionally,the quantum information theory has been applied through Graph Neural Networks(GNNs)to generate Riemannian metrics in closed-form of several graph layers.In further,to pre-process the adjacency matrix of graphs,a new formulation is established to incorporate high order proximities.The proposed scheme has shown outstanding improvements to overcome the deficiencies in Graph Convolutional Network(GCN),particularly,the information loss and imprecise information representation with acceptable computational overhead.Moreover,the proposed Quantum Graph Convolutional Network(QGCN)has significantly strengthened the GCN on semi-supervised node classification tasks.In parallel,it expands the generalization process with a significant difference by making small random perturbationsG of the graph during the training process.The evaluation results are provided on three benchmark datasets,including Citeseer,Cora,and PubMed,that distinctly delineate the superiority of the proposed model in terms of computational accuracy against state-of-the-art GCN and three other methods based on the same algorithms in the existing literature.
基金the Industry-University Cooperation Project in Fujian Province University(No.2022H6020)。
文摘Self-powered neutron detectors(SPNDs)play a critical role in monitoring the safety margins and overall health of reactors,directly affecting safe operation within the reactor.In this work,a novel fault identification method based on graph convolutional networks(GCN)and Stacking ensemble learning is proposed for SPNDs.The GCN is employed to extract the spatial neighborhood information of SPNDs at different positions,and residuals are obtained by nonlinear fitting of SPND signals.In order to completely extract the time-varying features from residual sequences,the Stacking fusion model,integrated with various algorithms,is developed and enables the identification of five conditions for SPNDs:normal,drift,bias,precision degradation,and complete failure.The results demonstrate that the integration of diverse base-learners in the GCN-Stacking model exhibits advantages over a single model as well as enhances the stability and reliability in fault identification.Additionally,the GCN-Stacking model maintains higher accuracy in identifying faults at different reactor power levels.
文摘To achieve good results in convolutional neural networks(CNN) for text classification task, term-based pooling operation in CNNs is proposed. Firstly, the convolution results of several convolution kernels are combined by this method, and then the results after combination are made pooling operation, three sorts of CNN models(we named TBCNN, MCT-CNN and MMCT-CNN respectively) are constructed and then corresponding algorithmic thought are detailed on this basis. Secondly, relevant experiments and analyses are respectively designed to show the effects of three key parameters(convolution kernel, combination kernel number and word embedding) on three kinds of CNN models and to further demonstrate the effect of the models proposed. The experimental results show that compared with the traditional method of text classification in CNNs, term-based pooling method is addressed that not only the availability of the way is proved, but also the performance shows good superiority.
文摘With the high-speed development of the Internet,a growing number of Internet users like giving their subjective comments in the BBS,blog and shopping website.These comments contains critics’attitudes,emotions,views and other information.Using these information reasonablely can help understand the social public opinion and make a timely response and help dealer to improve quality and service of products and make consumers know merchandise.This paper mainly discusses using convolutional neural network(CNN)for the operation of the text feature extraction.The concrete realization are discussed.Then combining with other text classifier make class operation.The experiment result shows the effectiveness of the method which is proposed in this paper.
基金supported by the National Natural Science Foundation of China(61732018,61872335,61802367,61876215)the Strategic Priority Research Program of Chinese Academy of Sciences(XDC05000000)+1 种基金Beijing Academy of Artificial Intelligence(BAAI),the Open Project Program of the State Key Laboratory of Mathematical Engineering and Advanced Computing(2019A07)the Open Project of Zhejiang Laboratory,and a grant from the Institute for Guo Qiang,Tsinghua University.Recommended by Associate Editor Long Chen.
文摘Graph convolutional networks(GCNs)have received significant attention from various research fields due to the excellent performance in learning graph representations.Although GCN performs well compared with other methods,it still faces challenges.Training a GCN model for large-scale graphs in a conventional way requires high computation and storage costs.Therefore,motivated by an urgent need in terms of efficiency and scalability in training GCN,sampling methods have been proposed and achieved a significant effect.In this paper,we categorize sampling methods based on the sampling mechanisms and provide a comprehensive survey of sampling methods for efficient training of GCN.To highlight the characteristics and differences of sampling methods,we present a detailed comparison within each category and further give an overall comparative analysis for the sampling methods in all categories.Finally,we discuss some challenges and future research directions of the sampling methods.
基金This study was supported in part by the Research Innovation Team Fund(Award No.18TD0026)from the Department of Educationin part by the Sichuan Key Research&Development Project(Project No.2020YFG0168)from the Science Technology Department,Sichuan Province.
文摘Existing solutions do not work well when multi-targets coexist in a sentence.The reason is that the existing solution is usually to separate multiple targets and process them separately.If the original sentence has N target,the original sentence will be repeated for N times,and only one target will be processed each time.To some extent,this approach degenerates the fine-grained sentiment classification task into the sentence-level sentiment classification task,and the research method of processing the target separately ignores the internal relation and interaction between the targets.Based on the above considerations,we proposes to use Graph Convolutional Network(GCN)to model and process multi-targets appearing in sentences at the same time based on the positional relationship,and then to construct a graph of the sentiment relationship between targets based on the difference of the sentiment polarity between target words.In addition to the standard target-dependent sentiment classification task,an auxiliary node relation classification task is constructed.Experiments demonstrate that our model achieves new comparable performance on the benchmark datasets:SemEval-2014 Task 4,i.e.,reviews for restaurants and laptops.Furthermore,the method of dividing the target words into isolated individuals has disadvantages,and the multi-task learning model is beneficial to enhance the feature extraction ability and expression ability of the model.
文摘Existing traffic flow prediction frameworks have already achieved enormous success due to large traffic datasets and capability of deep learning models.However,data privacy and security are always a challenge in every field where data need to be uploaded to the cloud.Federated learning(FL)is an emerging trend for distributed training of data.The primary goal of FL is to train an efficient communication model without compromising data privacy.The traffic data have a robust spatio-temporal correlation,but various approaches proposed earlier have not considered spatial correlation of the traffic data.This paper presents FL-based traffic flow prediction with spatio-temporal correlation.This work uses a differential privacy(DP)scheme for privacy preservation of participant's data.To the best of our knowledge,this is the first time that FL is used for vehicular traffic prediction while considering the spatio-temporal correlation of traffic data with DP preservation.The proposed framework trains the data locally at the client-side with DP.It then uses the model aggregation mechanism federated graph convolutional network(FedGCN)at the server-side to find the average of locally trained models.The results of the proposed work show that the FedGCN model accurately predicts the traffic.DP scheme at client-side helps clients to set a budget for privacy loss.
基金supported by China’s National Key R&D Program,No.2019QY1404the National Natural Science Foundation of China,Grant No.U20A20161,U1836103the Basic Strengthening Program Project,No.2019-JCJQ-ZD-113.
文摘The continuous improvement of the cyber threat intelligence sharing mechanism provides new ideas to deal with Advanced Persistent Threats(APT).Extracting attack behaviors,i.e.,Tactics,Techniques,Procedures(TTP)from Cyber Threat Intelligence(CTI)can facilitate APT actors’profiling for an immediate response.However,it is difficult for traditional manual methods to analyze attack behaviors from cyber threat intelligence due to its heterogeneous nature.Based on the Adversarial Tactics,Techniques and Common Knowledge(ATT&CK)of threat behavior description,this paper proposes a threat behavioral knowledge extraction framework that integrates Heterogeneous Text Network(HTN)and Graph Convolutional Network(GCN)to solve this issue.It leverages the hierarchical correlation relationships of attack techniques and tactics in the ATT&CK to construct a text network of heterogeneous cyber threat intelligence.With the help of the Bidirectional EncoderRepresentation fromTransformers(BERT)pretraining model to analyze the contextual semantics of cyber threat intelligence,the task of threat behavior identification is transformed into a text classification task,which automatically extracts attack behavior in CTI,then identifies the malware and advanced threat actors.The experimental results show that F1 achieve 94.86%and 92.15%for the multi-label classification tasks of tactics and techniques.Extend the experiment to verify the method’s effectiveness in identifying the malware and threat actors in APT attacks.The F1 for malware and advanced threat actors identification task reached 98.45%and 99.48%,which are better than the benchmark model in the experiment and achieve state of the art.The model can effectivelymodel threat intelligence text data and acquire knowledge and experience migration by correlating implied features with a priori knowledge to compensate for insufficient sample data and improve the classification performance and recognition ability of threat behavior in text.
基金Supported by Shaanxi Province Key Research and Development Project (2021GY-280)the National Natural Science Foundation of China (No.61834005,61772417,61802304)。
文摘Micro-expressions are spontaneous, unconscious movements that reveal true emotions.Accurate facial movement information and network training learning methods are crucial for micro-expression recognition.However, most existing micro-expression recognition technologies so far focus on modeling the single category of micro-expression images and neural network structure.Aiming at the problems of low recognition rate and weak model generalization ability in micro-expression recognition, a micro-expression recognition algorithm is proposed based on graph convolution network(GCN) and Transformer model.Firstly, action unit(AU) feature detection is extracted and facial muscle nodes in the neighborhood are divided into three subsets for recognition.Then, graph convolution layer is used to find the layout of dependencies between AU nodes of micro-expression classification.Finally, multiple attentional features of each facial action are enriched with Transformer model to include more sequence information before calculating the overall correlation of each region.The proposed method is validated in CASME II and CAS(ME)^2 datasets, and the recognition rate reached 69.85%.
基金New-Generation Artificial Intelligence-Major Program in the Sci-Tech Innovation 2030 Agenda from the Ministry of Science and Technology of China(2018AAA0102100)Hunan Provincial Department of Education key project(21A0250)The First Class Discipline Open Fund of Hunan University of Traditional Chinese Medicine(2022ZYX08)。
文摘Objective To construct symptom-formula-herb heterogeneous graphs structured Treatise on Febrile Diseases(Shang Han Lun,《伤寒论》)dataset and explore an optimal learning method represented with node attributes based on graph convolutional network(GCN).Methods Clauses that contain symptoms,formulas,and herbs were abstracted from Treatise on Febrile Diseases to construct symptom-formula-herb heterogeneous graphs,which were used to propose a node representation learning method based on GCN−the Traditional Chinese Medicine Graph Convolution Network(TCM-GCN).The symptom-formula,symptom-herb,and formula-herb heterogeneous graphs were processed with the TCM-GCN to realize high-order propagating message passing and neighbor aggregation to obtain new node representation attributes,and thus acquiring the nodes’sum-aggregations of symptoms,formulas,and herbs to lay a foundation for the downstream tasks of the prediction models.Results Comparisons among the node representations with multi-hot encoding,non-fusion encoding,and fusion encoding showed that the Precision@10,Recall@10,and F1-score@10 of the fusion encoding were 9.77%,6.65%,and 8.30%,respectively,higher than those of the non-fusion encoding in the prediction studies of the model.Conclusion Node representations by fusion encoding achieved comparatively ideal results,indicating the TCM-GCN is effective in realizing node-level representations of heterogeneous graph structured Treatise on Febrile Diseases dataset and is able to elevate the performance of the downstream tasks of the diagnosis model.
基金This research is partially supported by the National Natural Science Foundation of China(Grant No.61772098)Chongqing Technology Innovation and Application Development Project(Grant No.cstc2020jscxmsxmX0150)+2 种基金Chongqing Science and Technology Innovation Leading Talent Support Program(CSTCCXLJRC201908)Basic and Advanced Research Projects of CSTC(No.cstc2019jcyj-zdxmX0008)Science and Technology Research Program of Chongqing Municipal Education Commission(Grant No.KJZD-K201900605).
文摘The traditional malware research is mainly based on its recognition and detection as a breakthrough point,without focusing on its propagation trends or predicting the subsequently infected nodes.The complexity of network structure,diversity of network nodes,and sparsity of data all pose difficulties in predicting propagation.This paper proposes a malware propagation prediction model based on representation learning and Graph Convolutional Networks(GCN)to address the aforementioned problems.First,to solve the problem of the inaccuracy of infection intensity calculation caused by the sparsity of node interaction behavior data in the malware propagation network,a mechanism based on a tensor to mine the infection intensity among nodes is proposed to retain the network structure information.The influence of the relationship between nodes on the infection intensity is also analyzed.Second,given the diversity and complexity of the content and structure of infected and normal nodes in the network,considering the advantages of representation learning in data feature extraction,the corresponding representation learning method is adopted for the characteristics of infection intensity among nodes.This can efficiently calculate the relationship between entities and relationships in low dimensional space to achieve the goal of low dimensional,dense,and real-valued representation learning for the characteristics of propagation spatial data.We also design a new method,Tensor2vec,to learn the potential structural features of malware propagation.Finally,considering the convolution ability of GCN for non-Euclidean data,we propose a dynamic prediction model of malware propagation based on representation learning and GCN to solve the time effectiveness problem of the malware propagation carrier.The experimental results show that the proposed model can effectively predict the behaviors of the nodes in the network and discover the influence of different characteristics of nodes on the malware propagation situation.
基金Hunan Provincial Natural Science Foundation(2022JJ30438)Natural Science Foundation of Changsha(kq2202260)Hunan Province Traditional Chinese Medicine Research Project(B2023039).
文摘Objective To develop a model based on a graph convolutional network(GCN)to achieve ef-ficient classification of the cold and hot medicinal properties of Chinese herbal medicines(CHMs).Methods After screening the dataset provided in the published literature,this study includ-ed 495 CHMs and their 8075 compounds.Three molecular descriptors were used to repre-sent the compounds:the molecular access system(MACCS),extended connectivity finger-print(ECFP),and two-dimensional(2D)molecular descriptors computed by the RDKit open-source toolkit(RDKit_2D).A homogeneous graph with CHMs as nodes was constructed and a classification model for the cold and hot medicinal properties of CHMs was developed based on a GCN using the molecular descriptor information of the compounds as node features.Fi-nally,using accuracy and F1 score to evaluate model performance,the GCN model was ex-perimentally compared with the traditional machine learning approaches,including decision tree(DT),random forest(RF),k-nearest neighbor(KNN),Naïve Bayes classifier(NBC),and support vector machine(SVM).MACCS,ECFP,and RDKit_2D molecular descriptors were al-so adopted as features for comparison.Results The experimental results show that the GCN achieved better performance than the traditional machine learning approach when using MACCS as features,with the accuracy and F1 score reaching 0.8364 and 0.8453,respectively.The accuracy and F1 score have increased by 0.8690 and 0.8120,respectively,compared with the lowest performing feature combina-tion OMER(only the combination of MACCS,ECFP,and RDKit_2D).The accuracy and F1 score of DT,RF,KNN,NBC,and SVM are 0.5051 and 0.5018,0.6162 and 0.6015,0.6768 and 0.6243,0.6162 and 0.6071,0.6364 and 0.6225,respectively.Conclusion In this study,by introducing molecular descriptors as features,it is verified that molecular descriptors and fingerprints play a key role in classifying the cold and hot medici-nal properties of CHMs.Meanwhile,excellent classification performance was achieved using the GCN model,providing an important algorithmic basis for the in-depth study of the“struc-ture-property”relationship of CHMs.
基金This work is supported by the National Natural Science Foundation of China(61872231,61701297).
文摘The past decade has seen the rapid development of text detection based on deep learning.However,current methods of Chinese character detection and recognition have proven to be poor.The accuracy of segmenting text boxes in natural scenes is not impressive.The reasons for this strait can be summarized into two points:the complexity of natural scenes and numerous types of Chinese characters.In response to these problems,we proposed a lightweight neural network architecture named CTSF.It consists of two modules,one is a text detection network that combines CTPN and the image feature extraction modules of PVANet,named CDSE.The other is a literacy network based on spatial pyramid pool and fusion of Chinese character skeleton features named SPPCNN-SF,so as to realize the text detection and recognition,respectively.Our model performs much better than the original model on ICDAR2011 and ICDAR2013(achieved 85%and 88%F-measures)and enhanced the processing speed in training phase.In addition,our method achieves extremely performance on three Chinese datasets,with accuracy of 95.12%,95.56%and 96.01%.
