Extrapolation on Temporal Knowledge Graphs(TKGs)aims to predict future knowledge from a set of historical Knowledge Graphs in chronological order.The temporally adjacent facts in TKGs naturally form event sequences,ca...Extrapolation on Temporal Knowledge Graphs(TKGs)aims to predict future knowledge from a set of historical Knowledge Graphs in chronological order.The temporally adjacent facts in TKGs naturally form event sequences,called event evolution patterns,implying informative temporal dependencies between events.Recently,many extrapolation works on TKGs have been devoted to modelling these evolutional patterns,but the task is still far from resolved because most existing works simply rely on encoding these patterns into entity representations while overlooking the significant information implied by relations of evolutional patterns.However,the authors realise that the temporal dependencies inherent in the relations of these event evolution patterns may guide the follow-up event prediction to some extent.To this end,a Temporal Relational Context-based Temporal Dependencies Learning Network(TRenD)is proposed to explore the temporal context of relations for more comprehensive learning of event evolution patterns,especially those temporal dependencies caused by interactive patterns of relations.Trend incorporates a semantic context unit to capture semantic correlations between relations,and a structural context unit to learn the interaction pattern of relations.By learning the temporal contexts of relations semantically and structurally,the authors gain insights into the underlying event evolution patterns,enabling to extract comprehensive historical information for future prediction better.Experimental results on benchmark datasets demonstrate the superiority of the model.展开更多
Accurate prediction of future events brings great benefits and reduces losses for society in many domains,such as civil unrest,pandemics,and crimes.Knowledge graph is a general language for describing and modeling com...Accurate prediction of future events brings great benefits and reduces losses for society in many domains,such as civil unrest,pandemics,and crimes.Knowledge graph is a general language for describing and modeling complex systems.Different types of events continually occur,which are often related to historical and concurrent events.In this paper,we formalize the future event prediction as a temporal knowledge graph reasoning problem.Most existing studies either conduct reasoning on static knowledge graphs or assume knowledges graphs of all timestamps are available during the training process.As a result,they cannot effectively reason over temporal knowledge graphs and predict events happening in the future.To address this problem,some recent works learn to infer future events based on historical eventbased temporal knowledge graphs.However,these methods do not comprehensively consider the latent patterns and influences behind historical events and concurrent events simultaneously.This paper proposes a new graph representation learning model,namely Recurrent Event Graph ATtention Network(RE-GAT),based on a novel historical and concurrent events attention-aware mechanism by modeling the event knowledge graph sequence recurrently.More specifically,our RE-GAT uses an attention-based historical events embedding module to encode past events,and employs an attention-based concurrent events embedding module to model the associations of events at the same timestamp.A translation-based decoder module and a learning objective are developed to optimize the embeddings of entities and relations.We evaluate our proposed method on four benchmark datasets.Extensive experimental results demonstrate the superiority of our RE-GAT model comparing to various base-lines,which proves that our method can more accurately predict what events are going to happen.展开更多
Predicting potential facts in the future,Temporal Knowledge Graph(TKG)extrapolation remains challenging because of the deep dependence between the temporal association and semantic patterns of facts.Intuitively,facts(...Predicting potential facts in the future,Temporal Knowledge Graph(TKG)extrapolation remains challenging because of the deep dependence between the temporal association and semantic patterns of facts.Intuitively,facts(events)that happened at different timestamps have different influences on future events,which can be attributed to a hierarchy among not only facts but also relevant entities.Therefore,it is crucial to pay more attention to important entities and events when forecasting the future.However,most existing methods focus on reasoning over temporally evolving facts or mining evolutional patterns from known facts,which may be affected by the diversity and variability of the evolution,and they might fail to attach importance to facts that matter.Hyperbolic geometry was proved to be effective in capturing hierarchical patterns among data,which is considered to be a solution for modelling hierarchical relations among facts.To this end,we propose ReTIN,a novel model integrating real-time influence of historical facts for TKG reasoning based on hyperbolic geometry,which provides low-dimensional embeddings to capture latent hierarchical structures and other rich semantic patterns of the existing TKG.Considering both real-time and global features of TKG boosts the adaptation of ReTIN to the ever-changing dynamics and inherent constraints.Extensive experiments on benchmarks demonstrate the superiority of ReTIN over various baselines.The ablation study further supports the value of exploiting temporal information.展开更多
As the research of knowledge graph(KG)is deepened and widely used,knowledge graph com-pletion(KGC)has attracted more and more attentions from researchers,especially in scenarios of in-telligent search,social networks ...As the research of knowledge graph(KG)is deepened and widely used,knowledge graph com-pletion(KGC)has attracted more and more attentions from researchers,especially in scenarios of in-telligent search,social networks and deep question and answer(Q&A).Current research mainly fo-cuses on the completion of static knowledge graphs,and the temporal information in temporal knowl-edge graphs(TKGs)is ignored.However,the temporal information is definitely very helpful for the completion.Note that existing researches on temporal knowledge graph completion are difficult to process temporal information and to integrate entities,relations and time well.In this work,a rotation and scaling(RotatS)model is proposed,which learns rotation and scaling transformations from head entity embedding to tail entity embedding in 3D spaces to capture the information of time and rela-tions in the temporal knowledge graph.The performance of the proposed RotatS model have been evaluated by comparison with several baselines under similar experimental conditions and space com-plexity on four typical knowl good graph completion datasets publicly available online.The study shows that RotatS can achieve good results in terms of prediction accuracy.展开更多
Due to the structural dependencies among concurrent events in the knowledge graph and the substantial amount of sequential correlation information carried by temporally adjacent events,we propose an Independent Recurr...Due to the structural dependencies among concurrent events in the knowledge graph and the substantial amount of sequential correlation information carried by temporally adjacent events,we propose an Independent Recurrent Temporal Graph Convolution Networks(IndRT-GCNets)framework to efficiently and accurately capture event attribute information.The framework models the knowledge graph sequences to learn the evolutionary represen-tations of entities and relations within each period.Firstly,by utilizing the temporal graph convolution module in the evolutionary representation unit,the framework captures the structural dependency relationships within the knowledge graph in each period.Meanwhile,to achieve better event representation and establish effective correlations,an independent recurrent neural network is employed to implement auto-regressive modeling.Furthermore,static attributes of entities in the entity-relation events are constrained andmerged using a static graph constraint to obtain optimal entity representations.Finally,the evolution of entity and relation representations is utilized to predict events in the next subsequent step.On multiple real-world datasets such as Freebase13(FB13),Freebase 15k(FB15K),WordNet11(WN11),WordNet18(WN18),FB15K-237,WN18RR,YAGO3-10,and Nell-995,the results of multiple evaluation indicators show that our proposed IndRT-GCNets framework outperforms most existing models on knowledge reasoning tasks,which validates the effectiveness and robustness.展开更多
Temporal knowledge graph(TKG)reasoning has emerged as a pivotal approach in event prediction.An important yet challenging task in TKG reasoning is to predict future events by extrapolating from historical events and t...Temporal knowledge graph(TKG)reasoning has emerged as a pivotal approach in event prediction.An important yet challenging task in TKG reasoning is to predict future events by extrapolating from historical events and their correlations.Existing methods either overlook the modeling of long-term dependencies between entities or are ineffective in aggregating long-term information with recent facts.Motivated by dual process theory in cognitive sciences,we introduce TKG-LDG,an approach enhancing TKG for future entity prediction with long-term dense graph,to model event evolution in an adaptive manner.We first construct a unified dense graph from historical data to capture long-term dependencies,reflecting cumulative knowledge of entity interactions over time.This unified dense graph is compatible with any graph neural network and facilitates entity interaction learning from a long-term perspective.Then we initialize a TKG encoder from the unified dense graph to enhance short-term event interaction modeling.TKG-LDG effectively marries global context with local adaptability to recent temporal changes through its short-term recurrent encoders,in a way that mirrors human reasoning by integrating both long-term and short-term event dynamics.Extensive experiments conducted on six widely used TKG datasets demonstrate that our model outperforms strong baselines in future event prediction.展开更多
Temporal knowledge graph completion(TKGC),which merges temporal information into traditional static knowledge graph completion(SKGC),has garnered increasing attention recently.Among numerous emerging approaches,transl...Temporal knowledge graph completion(TKGC),which merges temporal information into traditional static knowledge graph completion(SKGC),has garnered increasing attention recently.Among numerous emerging approaches,translation-based embedding models constitute a prominent approach in TKGC research.However,existing translation-based methods typically incorporate timestamps into entities or relations,rather than utilizing them independently.This practice fails to fully exploit the rich semantics inherent in temporal information,thereby weakening the expressive capability of models.To address this limitation,we propose embedding timestamps,like entities and relations,in one or more dedicated semantic spaces.After projecting all embeddings into a shared space,we use the relation-timestamp pair instead of the conventional relation embedding as the translation vector between head and tail entities.Our method elevates timestamps to the same representational significance as entities and relations.Based on this strategy,we introduce two novel translation-based embedding models:TE-TransR and TE-TransT.With the independent representation of timestamps,our method not only enhances capabilities in link prediction but also facilitates a relatively underexplored task,namely time prediction.To further bolster the precision and reliability of time prediction,we introduce a granular,time unit-based timestamp setting and a relation-specific evaluation protocol.Extensive experiments demonstrate that our models achieve strong performance on link prediction benchmarks,with TE-TransR outperforming existing baselines in the time prediction task.展开更多
The ability to forecast future events brings great benefits for society and cyberspace in many public safety domains,such as civil unrest,pandemics and crimes.The occurrences of new events are often correlated or depe...The ability to forecast future events brings great benefits for society and cyberspace in many public safety domains,such as civil unrest,pandemics and crimes.The occurrences of new events are often correlated or dependent on historical and concurrent events.Many existing studies learn event-occurring processes with sequential and structural models,which,however,suffer from inefficient and inaccurate prediction problems.To better understand the event forecasting task and characterize the occurrence of new events,we exploit the human cognitive theory from the cognitive neuroscience discipline to find available cues for algorithm design and event prediction.Motivated by the dual process theory,we propose a two-stage learning scheme for event knowledge mining and prediction.First,we screen out event candidates based on historical inherent knowledge.Then we re-rank event candidates by probing into the newest relative events.Our proposed model mimics a sociological phenomenon called“the chameleon effect”and consists of a new target attentive graph collaborative learning mechanism to ensure a better understanding of sophisticated evolution patterns associated with events.In addition,self-supervised contrastive learning is employed to alleviate the over-smoothing problem that existed in graph learning while improving the model’s interpretability.Experiments show the effectiveness of our approach.展开更多
1 Introduction Temporal Knowledge Graphs(TKGs)provide a dynamic framework for modeling evolving events and relationships over time,with applications ranging from stock market to international politics.As to stock mark...1 Introduction Temporal Knowledge Graphs(TKGs)provide a dynamic framework for modeling evolving events and relationships over time,with applications ranging from stock market to international politics.As to stock market,TKGs can model how these relationships change over time,enabling the prediction of stock price movements,market trends,and potential risks.While graph-based methods such as Graph Neural Networks(GNNs)[1,2]have been widely adopted for TKG extrapolation,we argue that their structural focus often overshadows the critical role of historical information.Historical periodicity and temporal patterns serve as the foundation for effective temporal reasoning,particularly in forecasting future events.展开更多
基金supported in part by the National Natural Science Foundation of China(No.62302507)and the funding of Harbin Institute of Technology(Shenzhen)(No.20210035).
文摘Extrapolation on Temporal Knowledge Graphs(TKGs)aims to predict future knowledge from a set of historical Knowledge Graphs in chronological order.The temporally adjacent facts in TKGs naturally form event sequences,called event evolution patterns,implying informative temporal dependencies between events.Recently,many extrapolation works on TKGs have been devoted to modelling these evolutional patterns,but the task is still far from resolved because most existing works simply rely on encoding these patterns into entity representations while overlooking the significant information implied by relations of evolutional patterns.However,the authors realise that the temporal dependencies inherent in the relations of these event evolution patterns may guide the follow-up event prediction to some extent.To this end,a Temporal Relational Context-based Temporal Dependencies Learning Network(TRenD)is proposed to explore the temporal context of relations for more comprehensive learning of event evolution patterns,especially those temporal dependencies caused by interactive patterns of relations.Trend incorporates a semantic context unit to capture semantic correlations between relations,and a structural context unit to learn the interaction pattern of relations.By learning the temporal contexts of relations semantically and structurally,the authors gain insights into the underlying event evolution patterns,enabling to extract comprehensive historical information for future prediction better.Experimental results on benchmark datasets demonstrate the superiority of the model.
基金supported by the National Natural Science Foundation of China under grants U19B2044National Key Research and Development Program of China(2021YFC3300500).
文摘Accurate prediction of future events brings great benefits and reduces losses for society in many domains,such as civil unrest,pandemics,and crimes.Knowledge graph is a general language for describing and modeling complex systems.Different types of events continually occur,which are often related to historical and concurrent events.In this paper,we formalize the future event prediction as a temporal knowledge graph reasoning problem.Most existing studies either conduct reasoning on static knowledge graphs or assume knowledges graphs of all timestamps are available during the training process.As a result,they cannot effectively reason over temporal knowledge graphs and predict events happening in the future.To address this problem,some recent works learn to infer future events based on historical eventbased temporal knowledge graphs.However,these methods do not comprehensively consider the latent patterns and influences behind historical events and concurrent events simultaneously.This paper proposes a new graph representation learning model,namely Recurrent Event Graph ATtention Network(RE-GAT),based on a novel historical and concurrent events attention-aware mechanism by modeling the event knowledge graph sequence recurrently.More specifically,our RE-GAT uses an attention-based historical events embedding module to encode past events,and employs an attention-based concurrent events embedding module to model the associations of events at the same timestamp.A translation-based decoder module and a learning objective are developed to optimize the embeddings of entities and relations.We evaluate our proposed method on four benchmark datasets.Extensive experimental results demonstrate the superiority of our RE-GAT model comparing to various base-lines,which proves that our method can more accurately predict what events are going to happen.
基金Major Key Project of Pengcheng Laboratory,Grant/Award Number:PCL2022A03。
文摘Predicting potential facts in the future,Temporal Knowledge Graph(TKG)extrapolation remains challenging because of the deep dependence between the temporal association and semantic patterns of facts.Intuitively,facts(events)that happened at different timestamps have different influences on future events,which can be attributed to a hierarchy among not only facts but also relevant entities.Therefore,it is crucial to pay more attention to important entities and events when forecasting the future.However,most existing methods focus on reasoning over temporally evolving facts or mining evolutional patterns from known facts,which may be affected by the diversity and variability of the evolution,and they might fail to attach importance to facts that matter.Hyperbolic geometry was proved to be effective in capturing hierarchical patterns among data,which is considered to be a solution for modelling hierarchical relations among facts.To this end,we propose ReTIN,a novel model integrating real-time influence of historical facts for TKG reasoning based on hyperbolic geometry,which provides low-dimensional embeddings to capture latent hierarchical structures and other rich semantic patterns of the existing TKG.Considering both real-time and global features of TKG boosts the adaptation of ReTIN to the ever-changing dynamics and inherent constraints.Extensive experiments on benchmarks demonstrate the superiority of ReTIN over various baselines.The ablation study further supports the value of exploiting temporal information.
基金the National Natural Science Foundation of China(No.6187022153).
文摘As the research of knowledge graph(KG)is deepened and widely used,knowledge graph com-pletion(KGC)has attracted more and more attentions from researchers,especially in scenarios of in-telligent search,social networks and deep question and answer(Q&A).Current research mainly fo-cuses on the completion of static knowledge graphs,and the temporal information in temporal knowl-edge graphs(TKGs)is ignored.However,the temporal information is definitely very helpful for the completion.Note that existing researches on temporal knowledge graph completion are difficult to process temporal information and to integrate entities,relations and time well.In this work,a rotation and scaling(RotatS)model is proposed,which learns rotation and scaling transformations from head entity embedding to tail entity embedding in 3D spaces to capture the information of time and rela-tions in the temporal knowledge graph.The performance of the proposed RotatS model have been evaluated by comparison with several baselines under similar experimental conditions and space com-plexity on four typical knowl good graph completion datasets publicly available online.The study shows that RotatS can achieve good results in terms of prediction accuracy.
基金the National Natural Science Founda-tion of China(62062062)hosted by Gulila Altenbek.
文摘Due to the structural dependencies among concurrent events in the knowledge graph and the substantial amount of sequential correlation information carried by temporally adjacent events,we propose an Independent Recurrent Temporal Graph Convolution Networks(IndRT-GCNets)framework to efficiently and accurately capture event attribute information.The framework models the knowledge graph sequences to learn the evolutionary represen-tations of entities and relations within each period.Firstly,by utilizing the temporal graph convolution module in the evolutionary representation unit,the framework captures the structural dependency relationships within the knowledge graph in each period.Meanwhile,to achieve better event representation and establish effective correlations,an independent recurrent neural network is employed to implement auto-regressive modeling.Furthermore,static attributes of entities in the entity-relation events are constrained andmerged using a static graph constraint to obtain optimal entity representations.Finally,the evolution of entity and relation representations is utilized to predict events in the next subsequent step.On multiple real-world datasets such as Freebase13(FB13),Freebase 15k(FB15K),WordNet11(WN11),WordNet18(WN18),FB15K-237,WN18RR,YAGO3-10,and Nell-995,the results of multiple evaluation indicators show that our proposed IndRT-GCNets framework outperforms most existing models on knowledge reasoning tasks,which validates the effectiveness and robustness.
基金supported by the National Natural Science Foundation of China(Nos.62176043 and 62072077)the Intelligent Terminal Key Laboratory of Sichuan Province(No.SCITLAB-30002).
文摘Temporal knowledge graph(TKG)reasoning has emerged as a pivotal approach in event prediction.An important yet challenging task in TKG reasoning is to predict future events by extrapolating from historical events and their correlations.Existing methods either overlook the modeling of long-term dependencies between entities or are ineffective in aggregating long-term information with recent facts.Motivated by dual process theory in cognitive sciences,we introduce TKG-LDG,an approach enhancing TKG for future entity prediction with long-term dense graph,to model event evolution in an adaptive manner.We first construct a unified dense graph from historical data to capture long-term dependencies,reflecting cumulative knowledge of entity interactions over time.This unified dense graph is compatible with any graph neural network and facilitates entity interaction learning from a long-term perspective.Then we initialize a TKG encoder from the unified dense graph to enhance short-term event interaction modeling.TKG-LDG effectively marries global context with local adaptability to recent temporal changes through its short-term recurrent encoders,in a way that mirrors human reasoning by integrating both long-term and short-term event dynamics.Extensive experiments conducted on six widely used TKG datasets demonstrate that our model outperforms strong baselines in future event prediction.
基金supported by the National Natural Science Foundation of China under Grant No.72293575.
文摘Temporal knowledge graph completion(TKGC),which merges temporal information into traditional static knowledge graph completion(SKGC),has garnered increasing attention recently.Among numerous emerging approaches,translation-based embedding models constitute a prominent approach in TKGC research.However,existing translation-based methods typically incorporate timestamps into entities or relations,rather than utilizing them independently.This practice fails to fully exploit the rich semantics inherent in temporal information,thereby weakening the expressive capability of models.To address this limitation,we propose embedding timestamps,like entities and relations,in one or more dedicated semantic spaces.After projecting all embeddings into a shared space,we use the relation-timestamp pair instead of the conventional relation embedding as the translation vector between head and tail entities.Our method elevates timestamps to the same representational significance as entities and relations.Based on this strategy,we introduce two novel translation-based embedding models:TE-TransR and TE-TransT.With the independent representation of timestamps,our method not only enhances capabilities in link prediction but also facilitates a relatively underexplored task,namely time prediction.To further bolster the precision and reliability of time prediction,we introduce a granular,time unit-based timestamp setting and a relation-specific evaluation protocol.Extensive experiments demonstrate that our models achieve strong performance on link prediction benchmarks,with TE-TransR outperforming existing baselines in the time prediction task.
基金supported by the National Natural Science Foundation of China(Nos.62176043 and 62072077)the Grant SCITLAB-30002 of Intelligent Terminal Key Laboratory of Sichuan Province.
文摘The ability to forecast future events brings great benefits for society and cyberspace in many public safety domains,such as civil unrest,pandemics and crimes.The occurrences of new events are often correlated or dependent on historical and concurrent events.Many existing studies learn event-occurring processes with sequential and structural models,which,however,suffer from inefficient and inaccurate prediction problems.To better understand the event forecasting task and characterize the occurrence of new events,we exploit the human cognitive theory from the cognitive neuroscience discipline to find available cues for algorithm design and event prediction.Motivated by the dual process theory,we propose a two-stage learning scheme for event knowledge mining and prediction.First,we screen out event candidates based on historical inherent knowledge.Then we re-rank event candidates by probing into the newest relative events.Our proposed model mimics a sociological phenomenon called“the chameleon effect”and consists of a new target attentive graph collaborative learning mechanism to ensure a better understanding of sophisticated evolution patterns associated with events.In addition,self-supervised contrastive learning is employed to alleviate the over-smoothing problem that existed in graph learning while improving the model’s interpretability.Experiments show the effectiveness of our approach.
基金supported by the National Natural Science Foundation of China(Grant Nos.62020106005,42050105,62061146002)Shanghai Pilot Program for Basic Research-Shanghai Jiao Tong University.
文摘1 Introduction Temporal Knowledge Graphs(TKGs)provide a dynamic framework for modeling evolving events and relationships over time,with applications ranging from stock market to international politics.As to stock market,TKGs can model how these relationships change over time,enabling the prediction of stock price movements,market trends,and potential risks.While graph-based methods such as Graph Neural Networks(GNNs)[1,2]have been widely adopted for TKG extrapolation,we argue that their structural focus often overshadows the critical role of historical information.Historical periodicity and temporal patterns serve as the foundation for effective temporal reasoning,particularly in forecasting future events.
