Long-termpetroleum production forecasting is essential for the effective development andmanagement of oilfields.Due to its ability to extract complex patterns,deep learning has gained popularity for production forecas...Long-termpetroleum production forecasting is essential for the effective development andmanagement of oilfields.Due to its ability to extract complex patterns,deep learning has gained popularity for production forecasting.However,existing deep learning models frequently overlook the selective utilization of information from other production wells,resulting in suboptimal performance in long-term production forecasting across multiple wells.To achieve accurate long-term petroleum production forecast,we propose a spatial-geological perception graph convolutional neural network(SGP-GCN)that accounts for the temporal,spatial,and geological dependencies inherent in petroleum production.Utilizing the attention mechanism,the SGP-GCN effectively captures intricate correlations within production and geological data,forming the representations of each production well.Based on the spatial distances and geological feature correlations,we construct a spatial-geological matrix as the weight matrix to enable differential utilization of information from other wells.Additionally,a matrix sparsification algorithm based on production clustering(SPC)is also proposed to optimize the weight distribution within the spatial-geological matrix,thereby enhancing long-term forecasting performance.Empirical evaluations have shown that the SGP-GCN outperforms existing deep learning models,such as CNN-LSTM-SA,in long-term petroleum production forecasting.This demonstrates the potential of the SGP-GCN as a valuable tool for long-term petroleum production forecasting across multiple wells.展开更多
Accurate traffic flow prediction has a profound impact on modern traffic management. Traffic flow has complex spatial-temporal correlations and periodicity, which poses difficulties for precise prediction. To address ...Accurate traffic flow prediction has a profound impact on modern traffic management. Traffic flow has complex spatial-temporal correlations and periodicity, which poses difficulties for precise prediction. To address this problem, a Multi-head Self-attention and Spatial-Temporal Graph Convolutional Network (MSSTGCN) for multiscale traffic flow prediction is proposed. Firstly, to capture the hidden traffic periodicity of traffic flow, traffic flow is divided into three kinds of periods, including hourly, daily, and weekly data. Secondly, a graph attention residual layer is constructed to learn the global spatial features across regions. Local spatial-temporal dependence is captured by using a T-GCN module. Thirdly, a transformer layer is introduced to learn the long-term dependence in time. A position embedding mechanism is introduced to label position information for all traffic sequences. Thus, this multi-head self-attention mechanism can recognize the sequence order and allocate weights for different time nodes. Experimental results on four real-world datasets show that the MSSTGCN performs better than the baseline methods and can be successfully adapted to traffic prediction tasks.展开更多
In the burgeoning field of anomaly detection within attributed networks,traditional methodologies often encounter the intricacies of network complexity,particularly in capturing nonlinearity and sparsity.This study in...In the burgeoning field of anomaly detection within attributed networks,traditional methodologies often encounter the intricacies of network complexity,particularly in capturing nonlinearity and sparsity.This study introduces an innovative approach that synergizes the strengths of graph convolutional networks with advanced deep residual learning and a unique residual-based attention mechanism,thereby creating a more nuanced and efficient method for anomaly detection in complex networks.The heart of our model lies in the integration of graph convolutional networks that capture complex structural relationships within the network data.This is further bolstered by deep residual learning,which is employed to model intricate nonlinear connections directly from input data.A pivotal innovation in our approach is the incorporation of a residual-based attention mech-anism.This mechanism dynamically adjusts the importance of nodes based on their residual information,thereby significantly enhancing the sensitivity of the model to subtle anomalies.Furthermore,we introduce a novel hypersphere mapping technique in the latent space to distinctly separate normal and anomalous data.This mapping is the key to our model’s ability to pinpoint anomalies with greater precision.An extensive experimental setup was used to validate the efficacy of the proposed model.Using attributed social network datasets,we demonstrate that our model not only competes with but also surpasses existing state-of-the-art methods in anomaly detection.The results show the exceptional capability of our model to handle the multifaceted nature of real-world networks.展开更多
Self-powered neutron detectors(SPNDs)play a critical role in monitoring the safety margins and overall health of reactors,directly affecting safe operation within the reactor.In this work,a novel fault identification ...Self-powered neutron detectors(SPNDs)play a critical role in monitoring the safety margins and overall health of reactors,directly affecting safe operation within the reactor.In this work,a novel fault identification method based on graph convolutional networks(GCN)and Stacking ensemble learning is proposed for SPNDs.The GCN is employed to extract the spatial neighborhood information of SPNDs at different positions,and residuals are obtained by nonlinear fitting of SPND signals.In order to completely extract the time-varying features from residual sequences,the Stacking fusion model,integrated with various algorithms,is developed and enables the identification of five conditions for SPNDs:normal,drift,bias,precision degradation,and complete failure.The results demonstrate that the integration of diverse base-learners in the GCN-Stacking model exhibits advantages over a single model as well as enhances the stability and reliability in fault identification.Additionally,the GCN-Stacking model maintains higher accuracy in identifying faults at different reactor power levels.展开更多
Existing solutions do not work well when multi-targets coexist in a sentence.The reason is that the existing solution is usually to separate multiple targets and process them separately.If the original sentence has N ...Existing solutions do not work well when multi-targets coexist in a sentence.The reason is that the existing solution is usually to separate multiple targets and process them separately.If the original sentence has N target,the original sentence will be repeated for N times,and only one target will be processed each time.To some extent,this approach degenerates the fine-grained sentiment classification task into the sentence-level sentiment classification task,and the research method of processing the target separately ignores the internal relation and interaction between the targets.Based on the above considerations,we proposes to use Graph Convolutional Network(GCN)to model and process multi-targets appearing in sentences at the same time based on the positional relationship,and then to construct a graph of the sentiment relationship between targets based on the difference of the sentiment polarity between target words.In addition to the standard target-dependent sentiment classification task,an auxiliary node relation classification task is constructed.Experiments demonstrate that our model achieves new comparable performance on the benchmark datasets:SemEval-2014 Task 4,i.e.,reviews for restaurants and laptops.Furthermore,the method of dividing the target words into isolated individuals has disadvantages,and the multi-task learning model is beneficial to enhance the feature extraction ability and expression ability of the model.展开更多
Micro-expressions are spontaneous, unconscious movements that reveal true emotions.Accurate facial movement information and network training learning methods are crucial for micro-expression recognition.However, most ...Micro-expressions are spontaneous, unconscious movements that reveal true emotions.Accurate facial movement information and network training learning methods are crucial for micro-expression recognition.However, most existing micro-expression recognition technologies so far focus on modeling the single category of micro-expression images and neural network structure.Aiming at the problems of low recognition rate and weak model generalization ability in micro-expression recognition, a micro-expression recognition algorithm is proposed based on graph convolution network(GCN) and Transformer model.Firstly, action unit(AU) feature detection is extracted and facial muscle nodes in the neighborhood are divided into three subsets for recognition.Then, graph convolution layer is used to find the layout of dependencies between AU nodes of micro-expression classification.Finally, multiple attentional features of each facial action are enriched with Transformer model to include more sequence information before calculating the overall correlation of each region.The proposed method is validated in CASME II and CAS(ME)^2 datasets, and the recognition rate reached 69.85%.展开更多
The traditional malware research is mainly based on its recognition and detection as a breakthrough point,without focusing on its propagation trends or predicting the subsequently infected nodes.The complexity of netw...The traditional malware research is mainly based on its recognition and detection as a breakthrough point,without focusing on its propagation trends or predicting the subsequently infected nodes.The complexity of network structure,diversity of network nodes,and sparsity of data all pose difficulties in predicting propagation.This paper proposes a malware propagation prediction model based on representation learning and Graph Convolutional Networks(GCN)to address the aforementioned problems.First,to solve the problem of the inaccuracy of infection intensity calculation caused by the sparsity of node interaction behavior data in the malware propagation network,a mechanism based on a tensor to mine the infection intensity among nodes is proposed to retain the network structure information.The influence of the relationship between nodes on the infection intensity is also analyzed.Second,given the diversity and complexity of the content and structure of infected and normal nodes in the network,considering the advantages of representation learning in data feature extraction,the corresponding representation learning method is adopted for the characteristics of infection intensity among nodes.This can efficiently calculate the relationship between entities and relationships in low dimensional space to achieve the goal of low dimensional,dense,and real-valued representation learning for the characteristics of propagation spatial data.We also design a new method,Tensor2vec,to learn the potential structural features of malware propagation.Finally,considering the convolution ability of GCN for non-Euclidean data,we propose a dynamic prediction model of malware propagation based on representation learning and GCN to solve the time effectiveness problem of the malware propagation carrier.The experimental results show that the proposed model can effectively predict the behaviors of the nodes in the network and discover the influence of different characteristics of nodes on the malware propagation situation.展开更多
Graph convolutional networks(GCNs)have received significant attention from various research fields due to the excellent performance in learning graph representations.Although GCN performs well compared with other meth...Graph convolutional networks(GCNs)have received significant attention from various research fields due to the excellent performance in learning graph representations.Although GCN performs well compared with other methods,it still faces challenges.Training a GCN model for large-scale graphs in a conventional way requires high computation and storage costs.Therefore,motivated by an urgent need in terms of efficiency and scalability in training GCN,sampling methods have been proposed and achieved a significant effect.In this paper,we categorize sampling methods based on the sampling mechanisms and provide a comprehensive survey of sampling methods for efficient training of GCN.To highlight the characteristics and differences of sampling methods,we present a detailed comparison within each category and further give an overall comparative analysis for the sampling methods in all categories.Finally,we discuss some challenges and future research directions of the sampling methods.展开更多
Existing traffic flow prediction frameworks have already achieved enormous success due to large traffic datasets and capability of deep learning models.However,data privacy and security are always a challenge in every...Existing traffic flow prediction frameworks have already achieved enormous success due to large traffic datasets and capability of deep learning models.However,data privacy and security are always a challenge in every field where data need to be uploaded to the cloud.Federated learning(FL)is an emerging trend for distributed training of data.The primary goal of FL is to train an efficient communication model without compromising data privacy.The traffic data have a robust spatio-temporal correlation,but various approaches proposed earlier have not considered spatial correlation of the traffic data.This paper presents FL-based traffic flow prediction with spatio-temporal correlation.This work uses a differential privacy(DP)scheme for privacy preservation of participant's data.To the best of our knowledge,this is the first time that FL is used for vehicular traffic prediction while considering the spatio-temporal correlation of traffic data with DP preservation.The proposed framework trains the data locally at the client-side with DP.It then uses the model aggregation mechanism federated graph convolutional network(FedGCN)at the server-side to find the average of locally trained models.The results of the proposed work show that the FedGCN model accurately predicts the traffic.DP scheme at client-side helps clients to set a budget for privacy loss.展开更多
The continuous improvement of the cyber threat intelligence sharing mechanism provides new ideas to deal with Advanced Persistent Threats(APT).Extracting attack behaviors,i.e.,Tactics,Techniques,Procedures(TTP)from Cy...The continuous improvement of the cyber threat intelligence sharing mechanism provides new ideas to deal with Advanced Persistent Threats(APT).Extracting attack behaviors,i.e.,Tactics,Techniques,Procedures(TTP)from Cyber Threat Intelligence(CTI)can facilitate APT actors’profiling for an immediate response.However,it is difficult for traditional manual methods to analyze attack behaviors from cyber threat intelligence due to its heterogeneous nature.Based on the Adversarial Tactics,Techniques and Common Knowledge(ATT&CK)of threat behavior description,this paper proposes a threat behavioral knowledge extraction framework that integrates Heterogeneous Text Network(HTN)and Graph Convolutional Network(GCN)to solve this issue.It leverages the hierarchical correlation relationships of attack techniques and tactics in the ATT&CK to construct a text network of heterogeneous cyber threat intelligence.With the help of the Bidirectional EncoderRepresentation fromTransformers(BERT)pretraining model to analyze the contextual semantics of cyber threat intelligence,the task of threat behavior identification is transformed into a text classification task,which automatically extracts attack behavior in CTI,then identifies the malware and advanced threat actors.The experimental results show that F1 achieve 94.86%and 92.15%for the multi-label classification tasks of tactics and techniques.Extend the experiment to verify the method’s effectiveness in identifying the malware and threat actors in APT attacks.The F1 for malware and advanced threat actors identification task reached 98.45%and 99.48%,which are better than the benchmark model in the experiment and achieve state of the art.The model can effectivelymodel threat intelligence text data and acquire knowledge and experience migration by correlating implied features with a priori knowledge to compensate for insufficient sample data and improve the classification performance and recognition ability of threat behavior in text.展开更多
As one of the main characteristics of atmospheric pollutants,PM_(2.5) severely affects human health and has received widespread attention in recent years.How to predict the variations of PM_(2.5) concentrations with h...As one of the main characteristics of atmospheric pollutants,PM_(2.5) severely affects human health and has received widespread attention in recent years.How to predict the variations of PM_(2.5) concentrations with high accuracy is an important topic.The PM_(2.5) monitoring stations in Xinjiang Uygur Autonomous Region,China,are unevenly distributed,which makes it challenging to conduct comprehensive analyses and predictions.Therefore,this study primarily addresses the limitations mentioned above and the poor generalization ability of PM_(2.5) concentration prediction models across different monitoring stations.We chose the northern slope of the Tianshan Mountains as the study area and took the January−December in 2019 as the research period.On the basis of data from 21 PM_(2.5) monitoring stations as well as meteorological data(temperature,instantaneous wind speed,and pressure),we developed an improved model,namely GCN−TCN−AR(where GCN is the graph convolution network,TCN is the temporal convolutional network,and AR is the autoregression),for predicting PM_(2.5) concentrations on the northern slope of the Tianshan Mountains.The GCN−TCN−AR model is composed of an improved GCN model,a TCN model,and an AR model.The results revealed that the R2 values predicted by the GCN−TCN−AR model at the four monitoring stations(Urumqi,Wujiaqu,Shihezi,and Changji)were 0.93,0.91,0.93,and 0.92,respectively,and the RMSE(root mean square error)values were 6.85,7.52,7.01,and 7.28μg/m^(3),respectively.The performance of the GCN−TCN−AR model was also compared with the currently neural network models,including the GCN−TCN,GCN,TCN,Support Vector Regression(SVR),and AR.The GCN−TCN−AR outperformed the other current neural network models,with high prediction accuracy and good stability,making it especially suitable for the predictions of PM_(2.5)concentrations.This study revealed the significant spatiotemporal variations of PM_(2.5)concentrations.First,the PM_(2.5) concentrations exhibited clear seasonal fluctuations,with higher levels typically observed in winter and differences presented between months.Second,the spatial distribution analysis revealed that cities such as Urumqi and Wujiaqu have high PM_(2.5) concentrations,with a noticeable geographical clustering of pollutions.Understanding the variations in PM_(2.5) concentrations is highly important for the sustainable development of ecological environment in arid areas.展开更多
Objective To construct symptom-formula-herb heterogeneous graphs structured Treatise on Febrile Diseases(Shang Han Lun,《伤寒论》)dataset and explore an optimal learning method represented with node attributes based o...Objective To construct symptom-formula-herb heterogeneous graphs structured Treatise on Febrile Diseases(Shang Han Lun,《伤寒论》)dataset and explore an optimal learning method represented with node attributes based on graph convolutional network(GCN).Methods Clauses that contain symptoms,formulas,and herbs were abstracted from Treatise on Febrile Diseases to construct symptom-formula-herb heterogeneous graphs,which were used to propose a node representation learning method based on GCN−the Traditional Chinese Medicine Graph Convolution Network(TCM-GCN).The symptom-formula,symptom-herb,and formula-herb heterogeneous graphs were processed with the TCM-GCN to realize high-order propagating message passing and neighbor aggregation to obtain new node representation attributes,and thus acquiring the nodes’sum-aggregations of symptoms,formulas,and herbs to lay a foundation for the downstream tasks of the prediction models.Results Comparisons among the node representations with multi-hot encoding,non-fusion encoding,and fusion encoding showed that the Precision@10,Recall@10,and F1-score@10 of the fusion encoding were 9.77%,6.65%,and 8.30%,respectively,higher than those of the non-fusion encoding in the prediction studies of the model.Conclusion Node representations by fusion encoding achieved comparatively ideal results,indicating the TCM-GCN is effective in realizing node-level representations of heterogeneous graph structured Treatise on Febrile Diseases dataset and is able to elevate the performance of the downstream tasks of the diagnosis model.展开更多
Objective To develop a model based on a graph convolutional network(GCN)to achieve ef-ficient classification of the cold and hot medicinal properties of Chinese herbal medicines(CHMs).Methods After screening the datas...Objective To develop a model based on a graph convolutional network(GCN)to achieve ef-ficient classification of the cold and hot medicinal properties of Chinese herbal medicines(CHMs).Methods After screening the dataset provided in the published literature,this study includ-ed 495 CHMs and their 8075 compounds.Three molecular descriptors were used to repre-sent the compounds:the molecular access system(MACCS),extended connectivity finger-print(ECFP),and two-dimensional(2D)molecular descriptors computed by the RDKit open-source toolkit(RDKit_2D).A homogeneous graph with CHMs as nodes was constructed and a classification model for the cold and hot medicinal properties of CHMs was developed based on a GCN using the molecular descriptor information of the compounds as node features.Fi-nally,using accuracy and F1 score to evaluate model performance,the GCN model was ex-perimentally compared with the traditional machine learning approaches,including decision tree(DT),random forest(RF),k-nearest neighbor(KNN),Naïve Bayes classifier(NBC),and support vector machine(SVM).MACCS,ECFP,and RDKit_2D molecular descriptors were al-so adopted as features for comparison.Results The experimental results show that the GCN achieved better performance than the traditional machine learning approach when using MACCS as features,with the accuracy and F1 score reaching 0.8364 and 0.8453,respectively.The accuracy and F1 score have increased by 0.8690 and 0.8120,respectively,compared with the lowest performing feature combina-tion OMER(only the combination of MACCS,ECFP,and RDKit_2D).The accuracy and F1 score of DT,RF,KNN,NBC,and SVM are 0.5051 and 0.5018,0.6162 and 0.6015,0.6768 and 0.6243,0.6162 and 0.6071,0.6364 and 0.6225,respectively.Conclusion In this study,by introducing molecular descriptors as features,it is verified that molecular descriptors and fingerprints play a key role in classifying the cold and hot medici-nal properties of CHMs.Meanwhile,excellent classification performance was achieved using the GCN model,providing an important algorithmic basis for the in-depth study of the“struc-ture-property”relationship of CHMs.展开更多
引入基于风向相似度自适应的GCN-LSTM模型进行昆山市PM_(2.5)预测,并与GCN模型和LSTM模型预测结果进行比较。结果显示,风向相似度自适应GCN-LSTM模型对昆山市PM_(2.5)浓度模拟的整体平均绝对误差、均方根误差和平均绝对百分比误差分别为...引入基于风向相似度自适应的GCN-LSTM模型进行昆山市PM_(2.5)预测,并与GCN模型和LSTM模型预测结果进行比较。结果显示,风向相似度自适应GCN-LSTM模型对昆山市PM_(2.5)浓度模拟的整体平均绝对误差、均方根误差和平均绝对百分比误差分别为3.30μg/m^(3)、5.16μg/m^(3)和15.6%,低于GCN模型和LSTM模型的对应指标。对于未来1 h PM_(2.5)浓度预测,风向相似度自适应GCN-LSTM模型在多个方面均比GCN模型和LSTM模型表现更好。展开更多
基金funded by National Natural Science Foundation of China,grant number 62071491.
文摘Long-termpetroleum production forecasting is essential for the effective development andmanagement of oilfields.Due to its ability to extract complex patterns,deep learning has gained popularity for production forecasting.However,existing deep learning models frequently overlook the selective utilization of information from other production wells,resulting in suboptimal performance in long-term production forecasting across multiple wells.To achieve accurate long-term petroleum production forecast,we propose a spatial-geological perception graph convolutional neural network(SGP-GCN)that accounts for the temporal,spatial,and geological dependencies inherent in petroleum production.Utilizing the attention mechanism,the SGP-GCN effectively captures intricate correlations within production and geological data,forming the representations of each production well.Based on the spatial distances and geological feature correlations,we construct a spatial-geological matrix as the weight matrix to enable differential utilization of information from other wells.Additionally,a matrix sparsification algorithm based on production clustering(SPC)is also proposed to optimize the weight distribution within the spatial-geological matrix,thereby enhancing long-term forecasting performance.Empirical evaluations have shown that the SGP-GCN outperforms existing deep learning models,such as CNN-LSTM-SA,in long-term petroleum production forecasting.This demonstrates the potential of the SGP-GCN as a valuable tool for long-term petroleum production forecasting across multiple wells.
基金supported by the National Natural Science Foundation of China(Grant Nos.62472149,62376089,62202147)Hubei Provincial Science and Technology Plan Project(2023BCB04100).
文摘Accurate traffic flow prediction has a profound impact on modern traffic management. Traffic flow has complex spatial-temporal correlations and periodicity, which poses difficulties for precise prediction. To address this problem, a Multi-head Self-attention and Spatial-Temporal Graph Convolutional Network (MSSTGCN) for multiscale traffic flow prediction is proposed. Firstly, to capture the hidden traffic periodicity of traffic flow, traffic flow is divided into three kinds of periods, including hourly, daily, and weekly data. Secondly, a graph attention residual layer is constructed to learn the global spatial features across regions. Local spatial-temporal dependence is captured by using a T-GCN module. Thirdly, a transformer layer is introduced to learn the long-term dependence in time. A position embedding mechanism is introduced to label position information for all traffic sequences. Thus, this multi-head self-attention mechanism can recognize the sequence order and allocate weights for different time nodes. Experimental results on four real-world datasets show that the MSSTGCN performs better than the baseline methods and can be successfully adapted to traffic prediction tasks.
文摘In the burgeoning field of anomaly detection within attributed networks,traditional methodologies often encounter the intricacies of network complexity,particularly in capturing nonlinearity and sparsity.This study introduces an innovative approach that synergizes the strengths of graph convolutional networks with advanced deep residual learning and a unique residual-based attention mechanism,thereby creating a more nuanced and efficient method for anomaly detection in complex networks.The heart of our model lies in the integration of graph convolutional networks that capture complex structural relationships within the network data.This is further bolstered by deep residual learning,which is employed to model intricate nonlinear connections directly from input data.A pivotal innovation in our approach is the incorporation of a residual-based attention mech-anism.This mechanism dynamically adjusts the importance of nodes based on their residual information,thereby significantly enhancing the sensitivity of the model to subtle anomalies.Furthermore,we introduce a novel hypersphere mapping technique in the latent space to distinctly separate normal and anomalous data.This mapping is the key to our model’s ability to pinpoint anomalies with greater precision.An extensive experimental setup was used to validate the efficacy of the proposed model.Using attributed social network datasets,we demonstrate that our model not only competes with but also surpasses existing state-of-the-art methods in anomaly detection.The results show the exceptional capability of our model to handle the multifaceted nature of real-world networks.
基金the Industry-University Cooperation Project in Fujian Province University(No.2022H6020)。
文摘Self-powered neutron detectors(SPNDs)play a critical role in monitoring the safety margins and overall health of reactors,directly affecting safe operation within the reactor.In this work,a novel fault identification method based on graph convolutional networks(GCN)and Stacking ensemble learning is proposed for SPNDs.The GCN is employed to extract the spatial neighborhood information of SPNDs at different positions,and residuals are obtained by nonlinear fitting of SPND signals.In order to completely extract the time-varying features from residual sequences,the Stacking fusion model,integrated with various algorithms,is developed and enables the identification of five conditions for SPNDs:normal,drift,bias,precision degradation,and complete failure.The results demonstrate that the integration of diverse base-learners in the GCN-Stacking model exhibits advantages over a single model as well as enhances the stability and reliability in fault identification.Additionally,the GCN-Stacking model maintains higher accuracy in identifying faults at different reactor power levels.
基金This study was supported in part by the Research Innovation Team Fund(Award No.18TD0026)from the Department of Educationin part by the Sichuan Key Research&Development Project(Project No.2020YFG0168)from the Science Technology Department,Sichuan Province.
文摘Existing solutions do not work well when multi-targets coexist in a sentence.The reason is that the existing solution is usually to separate multiple targets and process them separately.If the original sentence has N target,the original sentence will be repeated for N times,and only one target will be processed each time.To some extent,this approach degenerates the fine-grained sentiment classification task into the sentence-level sentiment classification task,and the research method of processing the target separately ignores the internal relation and interaction between the targets.Based on the above considerations,we proposes to use Graph Convolutional Network(GCN)to model and process multi-targets appearing in sentences at the same time based on the positional relationship,and then to construct a graph of the sentiment relationship between targets based on the difference of the sentiment polarity between target words.In addition to the standard target-dependent sentiment classification task,an auxiliary node relation classification task is constructed.Experiments demonstrate that our model achieves new comparable performance on the benchmark datasets:SemEval-2014 Task 4,i.e.,reviews for restaurants and laptops.Furthermore,the method of dividing the target words into isolated individuals has disadvantages,and the multi-task learning model is beneficial to enhance the feature extraction ability and expression ability of the model.
基金Supported by Shaanxi Province Key Research and Development Project (2021GY-280)the National Natural Science Foundation of China (No.61834005,61772417,61802304)。
文摘Micro-expressions are spontaneous, unconscious movements that reveal true emotions.Accurate facial movement information and network training learning methods are crucial for micro-expression recognition.However, most existing micro-expression recognition technologies so far focus on modeling the single category of micro-expression images and neural network structure.Aiming at the problems of low recognition rate and weak model generalization ability in micro-expression recognition, a micro-expression recognition algorithm is proposed based on graph convolution network(GCN) and Transformer model.Firstly, action unit(AU) feature detection is extracted and facial muscle nodes in the neighborhood are divided into three subsets for recognition.Then, graph convolution layer is used to find the layout of dependencies between AU nodes of micro-expression classification.Finally, multiple attentional features of each facial action are enriched with Transformer model to include more sequence information before calculating the overall correlation of each region.The proposed method is validated in CASME II and CAS(ME)^2 datasets, and the recognition rate reached 69.85%.
基金This research is partially supported by the National Natural Science Foundation of China(Grant No.61772098)Chongqing Technology Innovation and Application Development Project(Grant No.cstc2020jscxmsxmX0150)+2 种基金Chongqing Science and Technology Innovation Leading Talent Support Program(CSTCCXLJRC201908)Basic and Advanced Research Projects of CSTC(No.cstc2019jcyj-zdxmX0008)Science and Technology Research Program of Chongqing Municipal Education Commission(Grant No.KJZD-K201900605).
文摘The traditional malware research is mainly based on its recognition and detection as a breakthrough point,without focusing on its propagation trends or predicting the subsequently infected nodes.The complexity of network structure,diversity of network nodes,and sparsity of data all pose difficulties in predicting propagation.This paper proposes a malware propagation prediction model based on representation learning and Graph Convolutional Networks(GCN)to address the aforementioned problems.First,to solve the problem of the inaccuracy of infection intensity calculation caused by the sparsity of node interaction behavior data in the malware propagation network,a mechanism based on a tensor to mine the infection intensity among nodes is proposed to retain the network structure information.The influence of the relationship between nodes on the infection intensity is also analyzed.Second,given the diversity and complexity of the content and structure of infected and normal nodes in the network,considering the advantages of representation learning in data feature extraction,the corresponding representation learning method is adopted for the characteristics of infection intensity among nodes.This can efficiently calculate the relationship between entities and relationships in low dimensional space to achieve the goal of low dimensional,dense,and real-valued representation learning for the characteristics of propagation spatial data.We also design a new method,Tensor2vec,to learn the potential structural features of malware propagation.Finally,considering the convolution ability of GCN for non-Euclidean data,we propose a dynamic prediction model of malware propagation based on representation learning and GCN to solve the time effectiveness problem of the malware propagation carrier.The experimental results show that the proposed model can effectively predict the behaviors of the nodes in the network and discover the influence of different characteristics of nodes on the malware propagation situation.
基金supported by the National Natural Science Foundation of China(61732018,61872335,61802367,61876215)the Strategic Priority Research Program of Chinese Academy of Sciences(XDC05000000)+1 种基金Beijing Academy of Artificial Intelligence(BAAI),the Open Project Program of the State Key Laboratory of Mathematical Engineering and Advanced Computing(2019A07)the Open Project of Zhejiang Laboratory,and a grant from the Institute for Guo Qiang,Tsinghua University.Recommended by Associate Editor Long Chen.
文摘Graph convolutional networks(GCNs)have received significant attention from various research fields due to the excellent performance in learning graph representations.Although GCN performs well compared with other methods,it still faces challenges.Training a GCN model for large-scale graphs in a conventional way requires high computation and storage costs.Therefore,motivated by an urgent need in terms of efficiency and scalability in training GCN,sampling methods have been proposed and achieved a significant effect.In this paper,we categorize sampling methods based on the sampling mechanisms and provide a comprehensive survey of sampling methods for efficient training of GCN.To highlight the characteristics and differences of sampling methods,we present a detailed comparison within each category and further give an overall comparative analysis for the sampling methods in all categories.Finally,we discuss some challenges and future research directions of the sampling methods.
文摘Existing traffic flow prediction frameworks have already achieved enormous success due to large traffic datasets and capability of deep learning models.However,data privacy and security are always a challenge in every field where data need to be uploaded to the cloud.Federated learning(FL)is an emerging trend for distributed training of data.The primary goal of FL is to train an efficient communication model without compromising data privacy.The traffic data have a robust spatio-temporal correlation,but various approaches proposed earlier have not considered spatial correlation of the traffic data.This paper presents FL-based traffic flow prediction with spatio-temporal correlation.This work uses a differential privacy(DP)scheme for privacy preservation of participant's data.To the best of our knowledge,this is the first time that FL is used for vehicular traffic prediction while considering the spatio-temporal correlation of traffic data with DP preservation.The proposed framework trains the data locally at the client-side with DP.It then uses the model aggregation mechanism federated graph convolutional network(FedGCN)at the server-side to find the average of locally trained models.The results of the proposed work show that the FedGCN model accurately predicts the traffic.DP scheme at client-side helps clients to set a budget for privacy loss.
基金supported by China’s National Key R&D Program,No.2019QY1404the National Natural Science Foundation of China,Grant No.U20A20161,U1836103the Basic Strengthening Program Project,No.2019-JCJQ-ZD-113.
文摘The continuous improvement of the cyber threat intelligence sharing mechanism provides new ideas to deal with Advanced Persistent Threats(APT).Extracting attack behaviors,i.e.,Tactics,Techniques,Procedures(TTP)from Cyber Threat Intelligence(CTI)can facilitate APT actors’profiling for an immediate response.However,it is difficult for traditional manual methods to analyze attack behaviors from cyber threat intelligence due to its heterogeneous nature.Based on the Adversarial Tactics,Techniques and Common Knowledge(ATT&CK)of threat behavior description,this paper proposes a threat behavioral knowledge extraction framework that integrates Heterogeneous Text Network(HTN)and Graph Convolutional Network(GCN)to solve this issue.It leverages the hierarchical correlation relationships of attack techniques and tactics in the ATT&CK to construct a text network of heterogeneous cyber threat intelligence.With the help of the Bidirectional EncoderRepresentation fromTransformers(BERT)pretraining model to analyze the contextual semantics of cyber threat intelligence,the task of threat behavior identification is transformed into a text classification task,which automatically extracts attack behavior in CTI,then identifies the malware and advanced threat actors.The experimental results show that F1 achieve 94.86%and 92.15%for the multi-label classification tasks of tactics and techniques.Extend the experiment to verify the method’s effectiveness in identifying the malware and threat actors in APT attacks.The F1 for malware and advanced threat actors identification task reached 98.45%and 99.48%,which are better than the benchmark model in the experiment and achieve state of the art.The model can effectivelymodel threat intelligence text data and acquire knowledge and experience migration by correlating implied features with a priori knowledge to compensate for insufficient sample data and improve the classification performance and recognition ability of threat behavior in text.
基金supported by the Program of Support Xinjiang by Technology(2024E02028,B2-2024-0359)Xinjiang Tianchi Talent Program of 2024,the Foundation of Chinese Academy of Sciences(B2-2023-0239)the Youth Foundation of Shandong Natural Science(ZR2023QD070).
文摘As one of the main characteristics of atmospheric pollutants,PM_(2.5) severely affects human health and has received widespread attention in recent years.How to predict the variations of PM_(2.5) concentrations with high accuracy is an important topic.The PM_(2.5) monitoring stations in Xinjiang Uygur Autonomous Region,China,are unevenly distributed,which makes it challenging to conduct comprehensive analyses and predictions.Therefore,this study primarily addresses the limitations mentioned above and the poor generalization ability of PM_(2.5) concentration prediction models across different monitoring stations.We chose the northern slope of the Tianshan Mountains as the study area and took the January−December in 2019 as the research period.On the basis of data from 21 PM_(2.5) monitoring stations as well as meteorological data(temperature,instantaneous wind speed,and pressure),we developed an improved model,namely GCN−TCN−AR(where GCN is the graph convolution network,TCN is the temporal convolutional network,and AR is the autoregression),for predicting PM_(2.5) concentrations on the northern slope of the Tianshan Mountains.The GCN−TCN−AR model is composed of an improved GCN model,a TCN model,and an AR model.The results revealed that the R2 values predicted by the GCN−TCN−AR model at the four monitoring stations(Urumqi,Wujiaqu,Shihezi,and Changji)were 0.93,0.91,0.93,and 0.92,respectively,and the RMSE(root mean square error)values were 6.85,7.52,7.01,and 7.28μg/m^(3),respectively.The performance of the GCN−TCN−AR model was also compared with the currently neural network models,including the GCN−TCN,GCN,TCN,Support Vector Regression(SVR),and AR.The GCN−TCN−AR outperformed the other current neural network models,with high prediction accuracy and good stability,making it especially suitable for the predictions of PM_(2.5)concentrations.This study revealed the significant spatiotemporal variations of PM_(2.5)concentrations.First,the PM_(2.5) concentrations exhibited clear seasonal fluctuations,with higher levels typically observed in winter and differences presented between months.Second,the spatial distribution analysis revealed that cities such as Urumqi and Wujiaqu have high PM_(2.5) concentrations,with a noticeable geographical clustering of pollutions.Understanding the variations in PM_(2.5) concentrations is highly important for the sustainable development of ecological environment in arid areas.
基金New-Generation Artificial Intelligence-Major Program in the Sci-Tech Innovation 2030 Agenda from the Ministry of Science and Technology of China(2018AAA0102100)Hunan Provincial Department of Education key project(21A0250)The First Class Discipline Open Fund of Hunan University of Traditional Chinese Medicine(2022ZYX08)。
文摘Objective To construct symptom-formula-herb heterogeneous graphs structured Treatise on Febrile Diseases(Shang Han Lun,《伤寒论》)dataset and explore an optimal learning method represented with node attributes based on graph convolutional network(GCN).Methods Clauses that contain symptoms,formulas,and herbs were abstracted from Treatise on Febrile Diseases to construct symptom-formula-herb heterogeneous graphs,which were used to propose a node representation learning method based on GCN−the Traditional Chinese Medicine Graph Convolution Network(TCM-GCN).The symptom-formula,symptom-herb,and formula-herb heterogeneous graphs were processed with the TCM-GCN to realize high-order propagating message passing and neighbor aggregation to obtain new node representation attributes,and thus acquiring the nodes’sum-aggregations of symptoms,formulas,and herbs to lay a foundation for the downstream tasks of the prediction models.Results Comparisons among the node representations with multi-hot encoding,non-fusion encoding,and fusion encoding showed that the Precision@10,Recall@10,and F1-score@10 of the fusion encoding were 9.77%,6.65%,and 8.30%,respectively,higher than those of the non-fusion encoding in the prediction studies of the model.Conclusion Node representations by fusion encoding achieved comparatively ideal results,indicating the TCM-GCN is effective in realizing node-level representations of heterogeneous graph structured Treatise on Febrile Diseases dataset and is able to elevate the performance of the downstream tasks of the diagnosis model.
基金Hunan Provincial Natural Science Foundation(2022JJ30438)Natural Science Foundation of Changsha(kq2202260)Hunan Province Traditional Chinese Medicine Research Project(B2023039).
文摘Objective To develop a model based on a graph convolutional network(GCN)to achieve ef-ficient classification of the cold and hot medicinal properties of Chinese herbal medicines(CHMs).Methods After screening the dataset provided in the published literature,this study includ-ed 495 CHMs and their 8075 compounds.Three molecular descriptors were used to repre-sent the compounds:the molecular access system(MACCS),extended connectivity finger-print(ECFP),and two-dimensional(2D)molecular descriptors computed by the RDKit open-source toolkit(RDKit_2D).A homogeneous graph with CHMs as nodes was constructed and a classification model for the cold and hot medicinal properties of CHMs was developed based on a GCN using the molecular descriptor information of the compounds as node features.Fi-nally,using accuracy and F1 score to evaluate model performance,the GCN model was ex-perimentally compared with the traditional machine learning approaches,including decision tree(DT),random forest(RF),k-nearest neighbor(KNN),Naïve Bayes classifier(NBC),and support vector machine(SVM).MACCS,ECFP,and RDKit_2D molecular descriptors were al-so adopted as features for comparison.Results The experimental results show that the GCN achieved better performance than the traditional machine learning approach when using MACCS as features,with the accuracy and F1 score reaching 0.8364 and 0.8453,respectively.The accuracy and F1 score have increased by 0.8690 and 0.8120,respectively,compared with the lowest performing feature combina-tion OMER(only the combination of MACCS,ECFP,and RDKit_2D).The accuracy and F1 score of DT,RF,KNN,NBC,and SVM are 0.5051 and 0.5018,0.6162 and 0.6015,0.6768 and 0.6243,0.6162 and 0.6071,0.6364 and 0.6225,respectively.Conclusion In this study,by introducing molecular descriptors as features,it is verified that molecular descriptors and fingerprints play a key role in classifying the cold and hot medici-nal properties of CHMs.Meanwhile,excellent classification performance was achieved using the GCN model,providing an important algorithmic basis for the in-depth study of the“struc-ture-property”relationship of CHMs.
文摘引入基于风向相似度自适应的GCN-LSTM模型进行昆山市PM_(2.5)预测,并与GCN模型和LSTM模型预测结果进行比较。结果显示,风向相似度自适应GCN-LSTM模型对昆山市PM_(2.5)浓度模拟的整体平均绝对误差、均方根误差和平均绝对百分比误差分别为3.30μg/m^(3)、5.16μg/m^(3)和15.6%,低于GCN模型和LSTM模型的对应指标。对于未来1 h PM_(2.5)浓度预测,风向相似度自适应GCN-LSTM模型在多个方面均比GCN模型和LSTM模型表现更好。
