This study proposes a novel forecasting framework that simultaneously captures the strong periodicity and irregular meteorological fluctuations inherent in solar radiation time series.Existing approaches typically def...This study proposes a novel forecasting framework that simultaneously captures the strong periodicity and irregular meteorological fluctuations inherent in solar radiation time series.Existing approaches typically define inter-regional correlations using either simple correlation coefficients or distance-based measures when applying spatio-temporal graph neural networks(STGNNs).However,such definitions are prone to generating spurious correlations due to the dominance of periodic structures.To address this limitation,we adopt the Elastic-Band Transform(EBT)to decompose solar radiation into periodic and amplitude-modulated components,which are then modeled independently with separate graph neural networks.The periodic component,characterized by strong nationwide correlations,is learned with a relatively simple architecture,whereas the amplitude-modulated component is modeled with more complex STGNNs that capture climatological similarities between regions.The predictions from the two components are subsequently recombined to yield final forecasts that integrate both periodic patterns and aperiodic variability.The proposed framework is validated with multiple STGNN architectures,and experimental results demonstrate improved predictive accuracy and interpretability compared to conventional methods.展开更多
The ability to accurately predict urban traffic flows is crucial for optimising city operations.Consequently,various methods for forecasting urban traffic have been developed,focusing on analysing historical data to u...The ability to accurately predict urban traffic flows is crucial for optimising city operations.Consequently,various methods for forecasting urban traffic have been developed,focusing on analysing historical data to understand complex mobility patterns.Deep learning techniques,such as graph neural networks(GNNs),are popular for their ability to capture spatio-temporal dependencies.However,these models often become overly complex due to the large number of hyper-parameters involved.In this study,we introduce Dynamic Multi-Graph Spatial-Temporal Graph Neural Ordinary Differential Equation Networks(DMST-GNODE),a framework based on ordinary differential equations(ODEs)that autonomously discovers effective spatial-temporal graph neural network(STGNN)architectures for traffic prediction tasks.The comparative analysis of DMST-GNODE and baseline models indicates that DMST-GNODE model demonstrates superior performance across multiple datasets,consistently achieving the lowest Root Mean Square Error(RMSE)and Mean Absolute Error(MAE)values,alongside the highest accuracy.On the BKK(Bangkok)dataset,it outperformed other models with an RMSE of 3.3165 and an accuracy of 0.9367 for a 20-min interval,maintaining this trend across 40 and 60 min.Similarly,on the PeMS08 dataset,DMST-GNODE achieved the best performance with an RMSE of 19.4863 and an accuracy of 0.9377 at 20 min,demonstrating its effectiveness over longer periods.The Los_Loop dataset results further emphasise this model’s advantage,with an RMSE of 3.3422 and an accuracy of 0.7643 at 20 min,consistently maintaining superiority across all time intervals.These numerical highlights indicate that DMST-GNODE not only outperforms baseline models but also achieves higher accuracy and lower errors across different time intervals and datasets.展开更多
Lateral interaction in the biological brain is a key mechanism that underlies higher cognitive functions.Linear self‐organising map(SOM)introduces lateral interaction in a general form in which signals of any modalit...Lateral interaction in the biological brain is a key mechanism that underlies higher cognitive functions.Linear self‐organising map(SOM)introduces lateral interaction in a general form in which signals of any modality can be used.Some approaches directly incorporate SOM learning rules into neural networks,but incur complex operations and poor extendibility.The efficient way to implement lateral interaction in deep neural networks is not well established.The use of Laplacian Matrix‐based Smoothing(LS)regularisation is proposed for implementing lateral interaction in a concise form.The authors’derivation and experiments show that lateral interaction implemented by SOM model is a special case of LS‐regulated k‐means,and they both show the topology‐preserving capability.The authors also verify that LS‐regularisation can be used in conjunction with the end‐to‐end training paradigm in deep auto‐encoders.Additionally,the benefits of LS‐regularisation in relaxing the requirement of parameter initialisation in various models and improving the classification performance of prototype classifiers are evaluated.Furthermore,the topologically ordered structure introduced by LS‐regularisation in feature extractor can improve the generalisation performance on classification tasks.Overall,LS‐regularisation is an effective and efficient way to implement lateral interaction and can be easily extended to different models.展开更多
Appropriately characterising the mixed space-time relations of the contagion process caused by hybrid space and time factors remains the primary challenge in COVID-19 forecasting.However,in previous deep learning mode...Appropriately characterising the mixed space-time relations of the contagion process caused by hybrid space and time factors remains the primary challenge in COVID-19 forecasting.However,in previous deep learning models for epidemic forecasting,spatial and temporal variations are captured separately.A unified model is developed to cover all spatio-temporal relations.However,this measure is insufficient for modelling the complex spatio-temporal relations of infectious disease transmission.A dynamic adaptive spatio-temporal graph network(DASTGN)is proposed based on attention mechanisms to improve prediction accuracy.In DASTGN,complex spatio-temporal relations are depicted by adaptively fusing the mixed space-time effects and dynamic space-time dependency structure.This dual-scale model considers the time-specific,space-specific,and direct effects of the propagation process at the fine-grained level.Furthermore,the model characterises impacts from various space-time neighbour blocks under time-varying interventions at the coarse-grained level.The performance comparisons on the three COVID-19 datasets reveal that DASTGN achieves state-of-the-art results with a maximum improvement of 17.092%in the root mean-square error and 11.563%in the mean absolute error.Experimental results indicate that the mechanisms of designing DASTGN can effectively detect some spreading characteristics of COVID-19.The spatio-temporal weight matrices learned in each proposed module reveal diffusion patterns in various scenarios.In conclusion,DASTGN has successfully captured the dynamic spatio-temporal variations of COVID-19,and considering multiple dynamic space-time relationships is essential in epidemic forecasting.展开更多
Due to the increasingly severe challenges brought by various epidemic diseases,people urgently need intelligent outbreak trend prediction.Predicting disease onset is very important to assist decision-making.Most of th...Due to the increasingly severe challenges brought by various epidemic diseases,people urgently need intelligent outbreak trend prediction.Predicting disease onset is very important to assist decision-making.Most of the exist-ing work fails to make full use of the temporal and spatial characteristics of epidemics,and also relies on multi-variate data for prediction.In this paper,we propose a Multi-Scale Location Attention Graph Neural Networks(MSLAGNN)based on a large number of Centers for Disease Control and Prevention(CDC)patient electronic medical records research sequence source data sets.In order to understand the geography and timeliness of infec-tious diseases,specific neural networks are used to extract the geography and timeliness of infectious diseases.In the model framework,the features of different periods are extracted by a multi-scale convolution module.At the same time,the propagation effects between regions are simulated by graph convolution and attention mechan-isms.We compare the proposed method with the most advanced statistical methods and deep learning models.Meanwhile,we conduct comparative experiments on data sets with different time lengths to observe the predic-tion performance of the model in the face of different degrees of data collection.We conduct extensive experi-ments on real-world epidemic-related data sets.The method has strong prediction performance and can be readily used for epidemic prediction.展开更多
In order to address the issues of predefined adjacency matrices inadequately representing information in road networks,insufficiently capturing spatial dependencies of traffic networks,and the potential problem of exc...In order to address the issues of predefined adjacency matrices inadequately representing information in road networks,insufficiently capturing spatial dependencies of traffic networks,and the potential problem of excessive smoothing or neglecting initial node information as the layers of graph convolutional neural networks increase,thus affecting traffic prediction performance,this paper proposes a prediction model based on Adaptive Multi-channel Graph Convolutional Neural Networks(AMGCN).The model utilizes an adaptive adjacency matrix to automatically learn implicit graph structures from data,introduces a mixed skip propagation graph convolutional neural network model,which retains the original node states and selectively acquires outputs of convolutional layers,thus avoiding the loss of node initial states and comprehensively capturing spatial correlations of traffic flow.Finally,the output is fed into Long Short-Term Memory networks to capture temporal correlations.Comparative experiments on two real datasets validate the effectiveness of the proposed model.展开更多
随着会话推荐的广泛应用,如何充分利用语义信息、建模用户跨会话兴趣以及抑制数据噪声成为提升推荐性能的关键。为此提出一种新颖的会话推荐增强框架LGSBR,通过整合大语言模型(large language model,LLM)的语义理解能力与图神经网络(gra...随着会话推荐的广泛应用,如何充分利用语义信息、建模用户跨会话兴趣以及抑制数据噪声成为提升推荐性能的关键。为此提出一种新颖的会话推荐增强框架LGSBR,通过整合大语言模型(large language model,LLM)的语义理解能力与图神经网络(graph neural network,GNN)的结构建模能力,实现语义增强与个性化推荐。具体而言,利用大语言模型及微调的语言模型生成项目补充文本嵌入和用户跨会话兴趣嵌入,通过软注意力机制融合文本与ID嵌入,生成语义丰富的项目表示;引入用户兴趣嵌入,结合对齐损失实现个性化推荐;最后通过两阶段权重学习过滤噪声项目,优化会话表示。实验结果表明,在Beauty数据集上,LGSBR的P@20达到21.38%,MRR@20达到6.76%,分别较SR-GNN基线提升23.3%和50.56%;在MovieLen-1M数据集上,P@20为25.86%,MRR@20为7.58%,分别提升12.63%和10.98%;研究验证了LGSBR在多种GNN模型上的通用性和有效性。展开更多
Due to the many types of distributed denial-of-service attacks(DDoS)attacks and the large amount of data generated,it becomes a chal-lenge to manage and apply the malicious behavior knowledge generated by DDoS attacks...Due to the many types of distributed denial-of-service attacks(DDoS)attacks and the large amount of data generated,it becomes a chal-lenge to manage and apply the malicious behavior knowledge generated by DDoS attacks.We propose a malicious behavior knowledge base framework for DDoS attacks,which completes the construction and application of a multi-domain malicious behavior knowledge base.First,we collected mali-cious behavior traffic generated by five mainstream DDoS attacks.At the same time,we completed the knowledge collection mechanism through data pre-processing and dataset design.Then,we designed a malicious behavior category graph and malicious behavior structure graph for the characteristic information and spatial structure of DDoS attacks and completed the knowl-edge learning mechanism using a graph neural network model.To protect the data privacy of multiple multi-domain malicious behavior knowledge bases,we implement the knowledge-sharing mechanism based on federated learning.Finally,we store the constructed knowledge graphs,graph neural network model,and Federated model into the malicious behavior knowledge base to complete the knowledge management mechanism.The experimental results show that our proposed system architecture can effectively construct and apply the malicious behavior knowledge base,and the detection capability of multiple DDoS attacks occurring in the network reaches above 0.95,while there exists a certain anti-interference capability for data poisoning cases.展开更多
基金supported by Basic Science Research Program through the National Research Foundation of Korea(NRF)funded by the Ministry of Education(RS-2023-00249743).
文摘This study proposes a novel forecasting framework that simultaneously captures the strong periodicity and irregular meteorological fluctuations inherent in solar radiation time series.Existing approaches typically define inter-regional correlations using either simple correlation coefficients or distance-based measures when applying spatio-temporal graph neural networks(STGNNs).However,such definitions are prone to generating spurious correlations due to the dominance of periodic structures.To address this limitation,we adopt the Elastic-Band Transform(EBT)to decompose solar radiation into periodic and amplitude-modulated components,which are then modeled independently with separate graph neural networks.The periodic component,characterized by strong nationwide correlations,is learned with a relatively simple architecture,whereas the amplitude-modulated component is modeled with more complex STGNNs that capture climatological similarities between regions.The predictions from the two components are subsequently recombined to yield final forecasts that integrate both periodic patterns and aperiodic variability.The proposed framework is validated with multiple STGNN architectures,and experimental results demonstrate improved predictive accuracy and interpretability compared to conventional methods.
文摘The ability to accurately predict urban traffic flows is crucial for optimising city operations.Consequently,various methods for forecasting urban traffic have been developed,focusing on analysing historical data to understand complex mobility patterns.Deep learning techniques,such as graph neural networks(GNNs),are popular for their ability to capture spatio-temporal dependencies.However,these models often become overly complex due to the large number of hyper-parameters involved.In this study,we introduce Dynamic Multi-Graph Spatial-Temporal Graph Neural Ordinary Differential Equation Networks(DMST-GNODE),a framework based on ordinary differential equations(ODEs)that autonomously discovers effective spatial-temporal graph neural network(STGNN)architectures for traffic prediction tasks.The comparative analysis of DMST-GNODE and baseline models indicates that DMST-GNODE model demonstrates superior performance across multiple datasets,consistently achieving the lowest Root Mean Square Error(RMSE)and Mean Absolute Error(MAE)values,alongside the highest accuracy.On the BKK(Bangkok)dataset,it outperformed other models with an RMSE of 3.3165 and an accuracy of 0.9367 for a 20-min interval,maintaining this trend across 40 and 60 min.Similarly,on the PeMS08 dataset,DMST-GNODE achieved the best performance with an RMSE of 19.4863 and an accuracy of 0.9377 at 20 min,demonstrating its effectiveness over longer periods.The Los_Loop dataset results further emphasise this model’s advantage,with an RMSE of 3.3422 and an accuracy of 0.7643 at 20 min,consistently maintaining superiority across all time intervals.These numerical highlights indicate that DMST-GNODE not only outperforms baseline models but also achieves higher accuracy and lower errors across different time intervals and datasets.
基金supported by the National Natural Science Foundation of China grants 61836014 to CL,and the STI2030‐Major Projects(2022ZD0205100)the Strategic Priority Research Program of Chinese Academy of Science,Grant No.XDB32010300+1 种基金Shanghai Municipal Science and Technology Major Project(Grant No.2018SHZDZX05)the Innovation Academy of Artificial Intelligence,Chinese Academy of Sciences to ZW.
文摘Lateral interaction in the biological brain is a key mechanism that underlies higher cognitive functions.Linear self‐organising map(SOM)introduces lateral interaction in a general form in which signals of any modality can be used.Some approaches directly incorporate SOM learning rules into neural networks,but incur complex operations and poor extendibility.The efficient way to implement lateral interaction in deep neural networks is not well established.The use of Laplacian Matrix‐based Smoothing(LS)regularisation is proposed for implementing lateral interaction in a concise form.The authors’derivation and experiments show that lateral interaction implemented by SOM model is a special case of LS‐regulated k‐means,and they both show the topology‐preserving capability.The authors also verify that LS‐regularisation can be used in conjunction with the end‐to‐end training paradigm in deep auto‐encoders.Additionally,the benefits of LS‐regularisation in relaxing the requirement of parameter initialisation in various models and improving the classification performance of prototype classifiers are evaluated.Furthermore,the topologically ordered structure introduced by LS‐regularisation in feature extractor can improve the generalisation performance on classification tasks.Overall,LS‐regularisation is an effective and efficient way to implement lateral interaction and can be easily extended to different models.
基金Youth Innovation Promotion Association CAS,Grant/Award Number:2021103Strategic Priority Research Program of Chinese Academy of Sciences,Grant/Award Number:XDC02060500。
文摘Appropriately characterising the mixed space-time relations of the contagion process caused by hybrid space and time factors remains the primary challenge in COVID-19 forecasting.However,in previous deep learning models for epidemic forecasting,spatial and temporal variations are captured separately.A unified model is developed to cover all spatio-temporal relations.However,this measure is insufficient for modelling the complex spatio-temporal relations of infectious disease transmission.A dynamic adaptive spatio-temporal graph network(DASTGN)is proposed based on attention mechanisms to improve prediction accuracy.In DASTGN,complex spatio-temporal relations are depicted by adaptively fusing the mixed space-time effects and dynamic space-time dependency structure.This dual-scale model considers the time-specific,space-specific,and direct effects of the propagation process at the fine-grained level.Furthermore,the model characterises impacts from various space-time neighbour blocks under time-varying interventions at the coarse-grained level.The performance comparisons on the three COVID-19 datasets reveal that DASTGN achieves state-of-the-art results with a maximum improvement of 17.092%in the root mean-square error and 11.563%in the mean absolute error.Experimental results indicate that the mechanisms of designing DASTGN can effectively detect some spreading characteristics of COVID-19.The spatio-temporal weight matrices learned in each proposed module reveal diffusion patterns in various scenarios.In conclusion,DASTGN has successfully captured the dynamic spatio-temporal variations of COVID-19,and considering multiple dynamic space-time relationships is essential in epidemic forecasting.
文摘Due to the increasingly severe challenges brought by various epidemic diseases,people urgently need intelligent outbreak trend prediction.Predicting disease onset is very important to assist decision-making.Most of the exist-ing work fails to make full use of the temporal and spatial characteristics of epidemics,and also relies on multi-variate data for prediction.In this paper,we propose a Multi-Scale Location Attention Graph Neural Networks(MSLAGNN)based on a large number of Centers for Disease Control and Prevention(CDC)patient electronic medical records research sequence source data sets.In order to understand the geography and timeliness of infec-tious diseases,specific neural networks are used to extract the geography and timeliness of infectious diseases.In the model framework,the features of different periods are extracted by a multi-scale convolution module.At the same time,the propagation effects between regions are simulated by graph convolution and attention mechan-isms.We compare the proposed method with the most advanced statistical methods and deep learning models.Meanwhile,we conduct comparative experiments on data sets with different time lengths to observe the predic-tion performance of the model in the face of different degrees of data collection.We conduct extensive experi-ments on real-world epidemic-related data sets.The method has strong prediction performance and can be readily used for epidemic prediction.
文摘In order to address the issues of predefined adjacency matrices inadequately representing information in road networks,insufficiently capturing spatial dependencies of traffic networks,and the potential problem of excessive smoothing or neglecting initial node information as the layers of graph convolutional neural networks increase,thus affecting traffic prediction performance,this paper proposes a prediction model based on Adaptive Multi-channel Graph Convolutional Neural Networks(AMGCN).The model utilizes an adaptive adjacency matrix to automatically learn implicit graph structures from data,introduces a mixed skip propagation graph convolutional neural network model,which retains the original node states and selectively acquires outputs of convolutional layers,thus avoiding the loss of node initial states and comprehensively capturing spatial correlations of traffic flow.Finally,the output is fed into Long Short-Term Memory networks to capture temporal correlations.Comparative experiments on two real datasets validate the effectiveness of the proposed model.
文摘随着会话推荐的广泛应用,如何充分利用语义信息、建模用户跨会话兴趣以及抑制数据噪声成为提升推荐性能的关键。为此提出一种新颖的会话推荐增强框架LGSBR,通过整合大语言模型(large language model,LLM)的语义理解能力与图神经网络(graph neural network,GNN)的结构建模能力,实现语义增强与个性化推荐。具体而言,利用大语言模型及微调的语言模型生成项目补充文本嵌入和用户跨会话兴趣嵌入,通过软注意力机制融合文本与ID嵌入,生成语义丰富的项目表示;引入用户兴趣嵌入,结合对齐损失实现个性化推荐;最后通过两阶段权重学习过滤噪声项目,优化会话表示。实验结果表明,在Beauty数据集上,LGSBR的P@20达到21.38%,MRR@20达到6.76%,分别较SR-GNN基线提升23.3%和50.56%;在MovieLen-1M数据集上,P@20为25.86%,MRR@20为7.58%,分别提升12.63%和10.98%;研究验证了LGSBR在多种GNN模型上的通用性和有效性。
基金supported by the NationalKeyR&DProgramof China underGrant No.2018YFA0701604.
文摘Due to the many types of distributed denial-of-service attacks(DDoS)attacks and the large amount of data generated,it becomes a chal-lenge to manage and apply the malicious behavior knowledge generated by DDoS attacks.We propose a malicious behavior knowledge base framework for DDoS attacks,which completes the construction and application of a multi-domain malicious behavior knowledge base.First,we collected mali-cious behavior traffic generated by five mainstream DDoS attacks.At the same time,we completed the knowledge collection mechanism through data pre-processing and dataset design.Then,we designed a malicious behavior category graph and malicious behavior structure graph for the characteristic information and spatial structure of DDoS attacks and completed the knowl-edge learning mechanism using a graph neural network model.To protect the data privacy of multiple multi-domain malicious behavior knowledge bases,we implement the knowledge-sharing mechanism based on federated learning.Finally,we store the constructed knowledge graphs,graph neural network model,and Federated model into the malicious behavior knowledge base to complete the knowledge management mechanism.The experimental results show that our proposed system architecture can effectively construct and apply the malicious behavior knowledge base,and the detection capability of multiple DDoS attacks occurring in the network reaches above 0.95,while there exists a certain anti-interference capability for data poisoning cases.
