In the domain of knowledge graph embedding,conventional approaches typically transform entities and relations into continuous vector spaces.However,parameter efficiency becomes increasingly crucial when dealing with l...In the domain of knowledge graph embedding,conventional approaches typically transform entities and relations into continuous vector spaces.However,parameter efficiency becomes increasingly crucial when dealing with large-scale knowledge graphs that contain vast numbers of entities and relations.In particular,resource-intensive embeddings often lead to increased computational costs,and may limit scalability and adaptability in practical environ-ments,such as in low-resource settings or real-world applications.This paper explores an approach to knowledge graph representation learning that leverages small,reserved entities and relation sets for parameter-efficient embedding.We introduce a hierarchical attention network designed to refine and maximize the representational quality of embeddings by selectively focusing on these reserved sets,thereby reducing model complexity.Empirical assessments validate that our model achieves high performance on the benchmark dataset with fewer parameters and smaller embedding dimensions.The ablation studies further highlight the impact and contribution of each component in the proposed hierarchical attention structure.展开更多
Purpose:Due to the incompleteness nature of knowledge graphs(KGs),the task of predicting missing links between entities becomes important.Many previous approaches are static,this posed a notable problem that all meani...Purpose:Due to the incompleteness nature of knowledge graphs(KGs),the task of predicting missing links between entities becomes important.Many previous approaches are static,this posed a notable problem that all meanings of a polysemous entity share one embedding vector.This study aims to propose a polysemous embedding approach,named KG embedding under relational contexts(ContE for short),for missing link prediction.Design/methodology/approach:ContE models and infers different relationship patterns by considering the context of the relationship,which is implicit in the local neighborhood of the relationship.The forward and backward impacts of the relationship in ContE are mapped to two different embedding vectors,which represent the contextual information of the relationship.Then,according to the position of the entity,the entity’s polysemous representation is obtained by adding its static embedding vector to the corresponding context vector of the relationship.Findings:ContE is a fully expressive,that is,given any ground truth over the triples,there are embedding assignments to entities and relations that can precisely separate the true triples from false ones.ContE is capable of modeling four connectivity patterns such as symmetry,antisymmetry,inversion and composition.Research limitations:ContE needs to do a grid search to find best parameters to get best performance in practice,which is a time-consuming task.Sometimes,it requires longer entity vectors to get better performance than some other models.Practical implications:ContE is a bilinear model,which is a quite simple model that could be applied to large-scale KGs.By considering contexts of relations,ContE can distinguish the exact meaning of an entity in different triples so that when performing compositional reasoning,it is capable to infer the connectivity patterns of relations and achieves good performance on link prediction tasks.Originality/value:ContE considers the contexts of entities in terms of their positions in triples and the relationships they link to.It decomposes a relation vector into two vectors,namely,forward impact vector and backward impact vector in order to capture the relational contexts.ContE has the same low computational complexity as TransE.Therefore,it provides a new approach for contextualized knowledge graph embedding.展开更多
Accurate prediction of future events brings great benefits and reduces losses for society in many domains,such as civil unrest,pandemics,and crimes.Knowledge graph is a general language for describing and modeling com...Accurate prediction of future events brings great benefits and reduces losses for society in many domains,such as civil unrest,pandemics,and crimes.Knowledge graph is a general language for describing and modeling complex systems.Different types of events continually occur,which are often related to historical and concurrent events.In this paper,we formalize the future event prediction as a temporal knowledge graph reasoning problem.Most existing studies either conduct reasoning on static knowledge graphs or assume knowledges graphs of all timestamps are available during the training process.As a result,they cannot effectively reason over temporal knowledge graphs and predict events happening in the future.To address this problem,some recent works learn to infer future events based on historical eventbased temporal knowledge graphs.However,these methods do not comprehensively consider the latent patterns and influences behind historical events and concurrent events simultaneously.This paper proposes a new graph representation learning model,namely Recurrent Event Graph ATtention Network(RE-GAT),based on a novel historical and concurrent events attention-aware mechanism by modeling the event knowledge graph sequence recurrently.More specifically,our RE-GAT uses an attention-based historical events embedding module to encode past events,and employs an attention-based concurrent events embedding module to model the associations of events at the same timestamp.A translation-based decoder module and a learning objective are developed to optimize the embeddings of entities and relations.We evaluate our proposed method on four benchmark datasets.Extensive experimental results demonstrate the superiority of our RE-GAT model comparing to various base-lines,which proves that our method can more accurately predict what events are going to happen.展开更多
Predicting potential facts in the future,Temporal Knowledge Graph(TKG)extrapolation remains challenging because of the deep dependence between the temporal association and semantic patterns of facts.Intuitively,facts(...Predicting potential facts in the future,Temporal Knowledge Graph(TKG)extrapolation remains challenging because of the deep dependence between the temporal association and semantic patterns of facts.Intuitively,facts(events)that happened at different timestamps have different influences on future events,which can be attributed to a hierarchy among not only facts but also relevant entities.Therefore,it is crucial to pay more attention to important entities and events when forecasting the future.However,most existing methods focus on reasoning over temporally evolving facts or mining evolutional patterns from known facts,which may be affected by the diversity and variability of the evolution,and they might fail to attach importance to facts that matter.Hyperbolic geometry was proved to be effective in capturing hierarchical patterns among data,which is considered to be a solution for modelling hierarchical relations among facts.To this end,we propose ReTIN,a novel model integrating real-time influence of historical facts for TKG reasoning based on hyperbolic geometry,which provides low-dimensional embeddings to capture latent hierarchical structures and other rich semantic patterns of the existing TKG.Considering both real-time and global features of TKG boosts the adaptation of ReTIN to the ever-changing dynamics and inherent constraints.Extensive experiments on benchmarks demonstrate the superiority of ReTIN over various baselines.The ablation study further supports the value of exploiting temporal information.展开更多
To solve the low efficiency of approximate queries caused by the large sizes of the knowledge graphs in the real world,an embedding-based approximate query method is proposed.First,the nodes in the query graph are cla...To solve the low efficiency of approximate queries caused by the large sizes of the knowledge graphs in the real world,an embedding-based approximate query method is proposed.First,the nodes in the query graph are classified according to the degrees of approximation required for different types of nodes.This classification transforms the query problem into three constraints,from which approximate information is extracted.Second,candidates are generated by calculating the similarity between embeddings.Finally,a deep neural network model is designed,incorporating a loss function based on the high-dimensional ellipsoidal diffusion distance.This model identifies the distance between nodes using their embeddings and constructs a score function.k nodes are returned as the query results.The results show that the proposed method can return both exact results and approximate matching results.On datasets DBLP(DataBase systems and Logic Programming)and FUA-S(Flight USA Airports-Sparse),this method exhibits superior performance in terms of precision and recall,returning results in 0.10 and 0.03 s,respectively.This indicates greater efficiency compared to PathSim and other comparative methods.展开更多
Fault sensing in wind turbine(WT)generator bearings is essential for ensuring reliability and holding down maintenance costs.Feeding raw sensor data to machine learning(ML)model often overlooks the enveloping interdep...Fault sensing in wind turbine(WT)generator bearings is essential for ensuring reliability and holding down maintenance costs.Feeding raw sensor data to machine learning(ML)model often overlooks the enveloping interdependencies between system elements.This study proposes a new hybrid method that combines the domain knowledge via knowledge graphs(KGs)and the traditional feature-based data.Incorporation of contextual relationships through construction of graph embedding methods,such as Node2Vec,can capture meaningful information,such as the relationships among key parameters(e.g.wind speed,rotor Revolutions Per Minute(RPM),and temperature)in the enriched feature representations.These node embeddings,when augmented with the original data,can be used to allow the model to learn and generalize better.As shown in results achieved on experimental data,the augmented ML model(with KG)is much better at predicting with the help of accuracy and error measure compared to traditional ML methods.Paired t-test analysis proves the statistical validity of this improvement.Moreover,graph-based feature importance increases the interpretability of the model and helps to uncover the structurally significant variables that are otherwise ignored by the common methods.The approach provides an excellent,knowledge-guided manner through which intelligent fault detection can be executed on WT systems.展开更多
Knowledge graph embedding aims at embedding entities and relations in a knowledge graph into a continuous, dense, low-dimensional and realvalued vector space. Among various embedding models appeared in recent years, t...Knowledge graph embedding aims at embedding entities and relations in a knowledge graph into a continuous, dense, low-dimensional and realvalued vector space. Among various embedding models appeared in recent years, translation-based models such as TransE, TransH and TransR achieve state-of-the-art performance. However, in these models, negative triples used for training phase are generated by replacing each positive entity in positive triples with negative entities from the entity set with the same probability;as a result, a large number of invalid negative triples will be generated and used in the training process. In this paper, a method named adaptive negative sampling (ANS) is proposed to generate valid negative triples. In this method, it first divided all the entities into a number of groups which consist of similar entities by some clustering algorithms such as K-Means. Then, corresponding to each positive triple, the head entity was replaced by a negative entity from the cluster in which the head entity was located and the tail entity was replaced in a similar approach. As a result, it generated a set of high-quality negative triples which benefit for improving the effectiveness of embedding models. The ANS method was combined with the TransE model and the resulted model was named as TransE-ANS. Experimental results show that TransE-ANS achieves significant improvement in the link prediction task.展开更多
Knowledge graphs(KGs)have been widely accepted as powerful tools for modeling the complex relationships between concepts and developing knowledge-based services.In recent years,researchers in the field of power system...Knowledge graphs(KGs)have been widely accepted as powerful tools for modeling the complex relationships between concepts and developing knowledge-based services.In recent years,researchers in the field of power systems have explored KGs to develop intelligent dispatching systems for increasingly large power grids.With multiple power grid dispatching knowledge graphs(PDKGs)constructed by different agencies,the knowledge fusion of different PDKGs is useful for providing more accurate decision supports.To achieve this,entity alignment that aims at connecting different KGs by identifying equivalent entities is a critical step.Existing entity alignment methods cannot integrate useful structural,attribute,and relational information while calculating entities’similarities and are prone to making many-to-one alignments,thus can hardly achieve the best performance.To address these issues,this paper proposes a collective entity alignment model that integrates three kinds of available information and makes collective counterpart assignments.This model proposes a novel knowledge graph attention network(KGAT)to learn the embeddings of entities and relations explicitly and calculates entities’similarities by adaptively incorporating the structural,attribute,and relational similarities.Then,we formulate the counterpart assignment task as an integer programming(IP)problem to obtain one-to-one alignments.We not only conduct experiments on a pair of PDKGs but also evaluate o ur model on three commonly used cross-lingual KGs.Experimental comparisons indicate that our model outperforms other methods and provides an effective tool for the knowledge fusion of PDKGs.展开更多
To solve the problem of missing many valid triples in knowledge graphs(KGs),a novel model based on a convolutional neural network(CNN)called ConvKG is proposed,which employs a joint learning strategy for knowledge gra...To solve the problem of missing many valid triples in knowledge graphs(KGs),a novel model based on a convolutional neural network(CNN)called ConvKG is proposed,which employs a joint learning strategy for knowledge graph completion(KGC).Related research work has shown the superiority of convolutional neural networks(CNNs)in extracting semantic features of triple embeddings.However,these researches use only one single-shaped filter and fail to extract semantic features of different granularity.To solve this problem,ConvKG exploits multi-shaped filters to co-convolute on the triple embeddings,joint learning semantic features of different granularity.Different shaped filters cover different sizes on the triple embeddings and capture pairwise interactions of different granularity among triple elements.Experimental results confirm the strength of joint learning,and compared with state-of-the-art CNN-based KGC models,ConvKG achieves the better mean rank(MR)and Hits@10 metrics on dataset WN18 RR,and the better MR on dataset FB15k-237.展开更多
Knowledge tracing is the key component in online individualized learning,which is capable of assessing the users'mastery of skills and predicting the probability that the users can solve specific problems.Availabl...Knowledge tracing is the key component in online individualized learning,which is capable of assessing the users'mastery of skills and predicting the probability that the users can solve specific problems.Available knowledge tracing models have the problem that the assessments are not directly used in the predictions.To make full use of the assessments during predictions,a novel model,named deep knowledge tracing embedding neural network(DKTENN),is proposed in this work.DKTENN is a synthesis of deep knowledge tracing(DKT)and knowledge graph embedding(KGE).DKT utilizes sophisticated long short-term memory(LSTM)to assess the users and track the mastery of skills according to the users'interaction sequences with skill-level tags,and KGE is applied to predict the probability on the basis of both the embedded problems and DKT's assessments.DKTENN outperforms performance factors analysis and the other knowledge tracing models based on deep learning in the experiments.展开更多
The recommendation algorithm based on collaborative filtering is currently the most successful recommendation method. It recommends items to theuser based on the known historical interaction data of the target user. F...The recommendation algorithm based on collaborative filtering is currently the most successful recommendation method. It recommends items to theuser based on the known historical interaction data of the target user. Furthermore,the combination of the recommended algorithm based on collaborative filtrationand other auxiliary knowledge base is an effective way to improve the performance of the recommended system, of which the Co-Factorization Model(CoFM) is one representative research. CoFM, a fusion recommendation modelcombining the collaborative filtering model FM and the graph embeddingmodel TransE, introduces the information of many entities and their relationsin the knowledge graph into the recommendation system as effective auxiliaryinformation. It can effectively improve the accuracy of recommendations andalleviate the problem of sparse user historical interaction data. Unfortunately,the graph-embedded model TransE used in the CoFM model cannot solve the1-N, N-1, and N-N problems well. To tackle this problem, a novel fusion recommendation model Joint Factorization Machines and TransH Model (JFMH) isproposed, which improves CoFM by replacing the TransE model with TransHmodel. A large number of experiments on two widely used benchmark data setsshow that compared with CoFM, JFMH has improved performance in terms ofitem recommendation and knowledge graph completion, and is more competitivethan multiple baseline methods.展开更多
With the widespread use of Internet,the amount of data in the field of traditional Chinese medicine(TCM)is growing exponentially.Consequently,there is much attention on the collection of useful knowledge as well as it...With the widespread use of Internet,the amount of data in the field of traditional Chinese medicine(TCM)is growing exponentially.Consequently,there is much attention on the collection of useful knowledge as well as its effective organization and expression.Knowledge graphs have thus emerged,and knowledge reasoning based on this tool has become one of the hot spots of research.This paper first presents a brief introduction to the development of knowledge graphs and knowledge reasoning,and explores the significance of knowledge reasoning.Secondly,the mainstream knowledge reasoning methods,including knowledge reasoning based on traditional rules,knowledge reasoning based on distributed feature representation,and knowledge reasoning based on neural networks are introduced.Then,using stroke as an example,the knowledge reasoning methods are expounded,the principles and characteristics of commonly used knowledge reasoning methods are summarized,and the research and applications of knowledge reasoning techniques in TCM in recent years are sorted out.Finally,we summarize the problems faced in the development of knowledge reasoning in TCM,and put forward the importance of constructing a knowledge reasoning model suitable for the field of TCM.展开更多
Based on the well logging knowledge graph of hydrocarbon-bearing formation(HBF),a Knowledge-Powered Neural Network Formation Evaluation model(KPNFE)has been proposed.It has the following functions:(1)extracting charac...Based on the well logging knowledge graph of hydrocarbon-bearing formation(HBF),a Knowledge-Powered Neural Network Formation Evaluation model(KPNFE)has been proposed.It has the following functions:(1)extracting characteristic parameters describing HBF in multiple dimensions and multiple scales;(2)showing the characteristic parameter-related entities,relationships,and attributes as vectors via graph embedding technique;(3)intelligently identifying HBF;(4)seamlessly integrating expertise into the intelligent computing to establish the assessment system and ranking algorithm for potential pay recommendation.Taking 547 wells encountered the low porosity and low permeability Chang 6 Member of Triassic in the Jiyuan Block of Ordos Basin,NW China as objects,80%of the wells were randomly selected as the training dataset and the remainder as the validation dataset.The KPNFE prediction results on the validation dataset had a coincidence rate of 94.43%with the expert interpretation results and a coincidence rate of 84.38%for all the oil testing layers,which is 13 percentage points higher in accuracy and over 100 times faster than the primary conventional interpretation.In addition,a number of potential pays likely to produce industrial oil were recommended.The KPNFE model effectively inherits,carries forward and improves the expert knowledge,nicely solving the robustness problem in HBF identification.The KPNFE,with good interpretability and high accuracy of computation results,is a powerful technical means for efficient and high-quality well logging re-evaluation of old wells in mature oilfields.展开更多
Traffic accident data analysis and reasoning are crucial for accident prevention and control.Constructing an accident knowledge graph from hybrid datasets of Chinese and English accidents is a valuable task.However,it...Traffic accident data analysis and reasoning are crucial for accident prevention and control.Constructing an accident knowledge graph from hybrid datasets of Chinese and English accidents is a valuable task.However,it is challenging due to the need to consider multiple perspectives and infer implicit relationships between actors and factors in complex traffic accidents.To address these challenges,this paper proposes an accident learning attention embeddings based multi-view accident knowledge graph for traffic accident reasoning named BrightAccidentGraph.First,this paper proposes a multi-source traffic accident dataset construction and preprocessing method for traffic accident judgement records published by the China Judgement Document Network and traffic accident records published by the UK’s Ministry of Transport.Then,traffic accident graph construction and portrait method is proposed,we demonstrate the efficiency of the proposed method by constructing several multi-view traffic accident portraits using a multi-source dataset.Furthermore,accident learning attention embeddings based multi-view accident knowledge graph construction and traffic accident reasoning method using deep learning are introduced.Experiments on two hybrid datasets verify the efficiency and merits of our method.展开更多
Link prediction,also known as Knowledge Graph Completion(KGC),is the common task in Knowledge Graphs(KGs)to predict missing connections between entities.Most existing methods focus on designing shallow,scalable models...Link prediction,also known as Knowledge Graph Completion(KGC),is the common task in Knowledge Graphs(KGs)to predict missing connections between entities.Most existing methods focus on designing shallow,scalable models,which have less expressive than deep,multi-layer models.Furthermore,most operations like addition,matrix multiplications or factorization are handcrafted based on a few known relation patterns in several wellknown datasets,such as FB15k,WN18,etc.However,due to the diversity and complex nature of real-world data distribution,it is inherently difficult to preset all latent patterns.To address this issue,we proposeKGE-ANS,a novel knowledge graph embedding framework for general link prediction tasks using automatic network search.KGEANS can learn a deep,multi-layer effective architecture to adapt to different datasets through neural architecture search.In addition,the general search spacewe designed is tailored forKGtasks.We performextensive experiments on benchmark datasets and the dataset constructed in this paper.The results show that our KGE-ANS outperforms several state-of-the-art methods,especially on these datasets with complex relation patterns.展开更多
Utilizing graph neural networks for knowledge embedding to accomplish the task of knowledge graph completion(KGC)has become an important research area in knowledge graph completion.However,the number of nodes in the k...Utilizing graph neural networks for knowledge embedding to accomplish the task of knowledge graph completion(KGC)has become an important research area in knowledge graph completion.However,the number of nodes in the knowledge graph increases exponentially with the depth of the tree,whereas the distances of nodes in Euclidean space are second-order polynomial distances,whereby knowledge embedding using graph neural networks in Euclidean space will not represent the distances between nodes well.This paper introduces a novel approach called hyperbolic hierarchical graph attention network(H2GAT)to rectify this limitation.Firstly,the paper conducts knowledge representation in the hyperbolic space,effectively mitigating the issue of exponential growth of nodes with tree depth and consequent information loss.Secondly,it introduces a hierarchical graph atten-tion mechanism specifically designed for the hyperbolic space,allowing for enhanced capture of the network structure inherent in the knowledge graph.Finally,the efficacy of the proposed H2GAT model is evaluated on benchmark datasets,namely WN18RR and FB15K-237,thereby validating its effectiveness.The H2GAT model achieved 0.445,0.515,and 0.586 in the Hits@1,Hits@3 and Hits@10 metrics respectively on the WN18RR dataset and 0.243,0.367 and 0.518 on the FB15K-237 dataset.By incorporating hyperbolic space embedding and hierarchical graph attention,the H2GAT model successfully addresses the limitations of existing hyperbolic knowledge embedding models,exhibiting its competence in knowledge graph completion tasks.展开更多
Temporal knowledge graph completion(TKGC),which merges temporal information into traditional static knowledge graph completion(SKGC),has garnered increasing attention recently.Among numerous emerging approaches,transl...Temporal knowledge graph completion(TKGC),which merges temporal information into traditional static knowledge graph completion(SKGC),has garnered increasing attention recently.Among numerous emerging approaches,translation-based embedding models constitute a prominent approach in TKGC research.However,existing translation-based methods typically incorporate timestamps into entities or relations,rather than utilizing them independently.This practice fails to fully exploit the rich semantics inherent in temporal information,thereby weakening the expressive capability of models.To address this limitation,we propose embedding timestamps,like entities and relations,in one or more dedicated semantic spaces.After projecting all embeddings into a shared space,we use the relation-timestamp pair instead of the conventional relation embedding as the translation vector between head and tail entities.Our method elevates timestamps to the same representational significance as entities and relations.Based on this strategy,we introduce two novel translation-based embedding models:TE-TransR and TE-TransT.With the independent representation of timestamps,our method not only enhances capabilities in link prediction but also facilitates a relatively underexplored task,namely time prediction.To further bolster the precision and reliability of time prediction,we introduce a granular,time unit-based timestamp setting and a relation-specific evaluation protocol.Extensive experiments demonstrate that our models achieve strong performance on link prediction benchmarks,with TE-TransR outperforming existing baselines in the time prediction task.展开更多
基金supported by the National Science and Technology Council(NSTC),Taiwan,under Grants Numbers 112-2622-E-029-009 and 112-2221-E-029-019.
文摘In the domain of knowledge graph embedding,conventional approaches typically transform entities and relations into continuous vector spaces.However,parameter efficiency becomes increasingly crucial when dealing with large-scale knowledge graphs that contain vast numbers of entities and relations.In particular,resource-intensive embeddings often lead to increased computational costs,and may limit scalability and adaptability in practical environ-ments,such as in low-resource settings or real-world applications.This paper explores an approach to knowledge graph representation learning that leverages small,reserved entities and relation sets for parameter-efficient embedding.We introduce a hierarchical attention network designed to refine and maximize the representational quality of embeddings by selectively focusing on these reserved sets,thereby reducing model complexity.Empirical assessments validate that our model achieves high performance on the benchmark dataset with fewer parameters and smaller embedding dimensions.The ablation studies further highlight the impact and contribution of each component in the proposed hierarchical attention structure.
基金supported by the Key R&D Program Project of Zhejiang Province under Grant no.2019 C01004 and 2021C02004.
文摘Purpose:Due to the incompleteness nature of knowledge graphs(KGs),the task of predicting missing links between entities becomes important.Many previous approaches are static,this posed a notable problem that all meanings of a polysemous entity share one embedding vector.This study aims to propose a polysemous embedding approach,named KG embedding under relational contexts(ContE for short),for missing link prediction.Design/methodology/approach:ContE models and infers different relationship patterns by considering the context of the relationship,which is implicit in the local neighborhood of the relationship.The forward and backward impacts of the relationship in ContE are mapped to two different embedding vectors,which represent the contextual information of the relationship.Then,according to the position of the entity,the entity’s polysemous representation is obtained by adding its static embedding vector to the corresponding context vector of the relationship.Findings:ContE is a fully expressive,that is,given any ground truth over the triples,there are embedding assignments to entities and relations that can precisely separate the true triples from false ones.ContE is capable of modeling four connectivity patterns such as symmetry,antisymmetry,inversion and composition.Research limitations:ContE needs to do a grid search to find best parameters to get best performance in practice,which is a time-consuming task.Sometimes,it requires longer entity vectors to get better performance than some other models.Practical implications:ContE is a bilinear model,which is a quite simple model that could be applied to large-scale KGs.By considering contexts of relations,ContE can distinguish the exact meaning of an entity in different triples so that when performing compositional reasoning,it is capable to infer the connectivity patterns of relations and achieves good performance on link prediction tasks.Originality/value:ContE considers the contexts of entities in terms of their positions in triples and the relationships they link to.It decomposes a relation vector into two vectors,namely,forward impact vector and backward impact vector in order to capture the relational contexts.ContE has the same low computational complexity as TransE.Therefore,it provides a new approach for contextualized knowledge graph embedding.
基金supported by the National Natural Science Foundation of China under grants U19B2044National Key Research and Development Program of China(2021YFC3300500).
文摘Accurate prediction of future events brings great benefits and reduces losses for society in many domains,such as civil unrest,pandemics,and crimes.Knowledge graph is a general language for describing and modeling complex systems.Different types of events continually occur,which are often related to historical and concurrent events.In this paper,we formalize the future event prediction as a temporal knowledge graph reasoning problem.Most existing studies either conduct reasoning on static knowledge graphs or assume knowledges graphs of all timestamps are available during the training process.As a result,they cannot effectively reason over temporal knowledge graphs and predict events happening in the future.To address this problem,some recent works learn to infer future events based on historical eventbased temporal knowledge graphs.However,these methods do not comprehensively consider the latent patterns and influences behind historical events and concurrent events simultaneously.This paper proposes a new graph representation learning model,namely Recurrent Event Graph ATtention Network(RE-GAT),based on a novel historical and concurrent events attention-aware mechanism by modeling the event knowledge graph sequence recurrently.More specifically,our RE-GAT uses an attention-based historical events embedding module to encode past events,and employs an attention-based concurrent events embedding module to model the associations of events at the same timestamp.A translation-based decoder module and a learning objective are developed to optimize the embeddings of entities and relations.We evaluate our proposed method on four benchmark datasets.Extensive experimental results demonstrate the superiority of our RE-GAT model comparing to various base-lines,which proves that our method can more accurately predict what events are going to happen.
基金Major Key Project of Pengcheng Laboratory,Grant/Award Number:PCL2022A03。
文摘Predicting potential facts in the future,Temporal Knowledge Graph(TKG)extrapolation remains challenging because of the deep dependence between the temporal association and semantic patterns of facts.Intuitively,facts(events)that happened at different timestamps have different influences on future events,which can be attributed to a hierarchy among not only facts but also relevant entities.Therefore,it is crucial to pay more attention to important entities and events when forecasting the future.However,most existing methods focus on reasoning over temporally evolving facts or mining evolutional patterns from known facts,which may be affected by the diversity and variability of the evolution,and they might fail to attach importance to facts that matter.Hyperbolic geometry was proved to be effective in capturing hierarchical patterns among data,which is considered to be a solution for modelling hierarchical relations among facts.To this end,we propose ReTIN,a novel model integrating real-time influence of historical facts for TKG reasoning based on hyperbolic geometry,which provides low-dimensional embeddings to capture latent hierarchical structures and other rich semantic patterns of the existing TKG.Considering both real-time and global features of TKG boosts the adaptation of ReTIN to the ever-changing dynamics and inherent constraints.Extensive experiments on benchmarks demonstrate the superiority of ReTIN over various baselines.The ablation study further supports the value of exploiting temporal information.
基金The State Grid Technology Project(No.5108202340042A-1-1-ZN).
文摘To solve the low efficiency of approximate queries caused by the large sizes of the knowledge graphs in the real world,an embedding-based approximate query method is proposed.First,the nodes in the query graph are classified according to the degrees of approximation required for different types of nodes.This classification transforms the query problem into three constraints,from which approximate information is extracted.Second,candidates are generated by calculating the similarity between embeddings.Finally,a deep neural network model is designed,incorporating a loss function based on the high-dimensional ellipsoidal diffusion distance.This model identifies the distance between nodes using their embeddings and constructs a score function.k nodes are returned as the query results.The results show that the proposed method can return both exact results and approximate matching results.On datasets DBLP(DataBase systems and Logic Programming)and FUA-S(Flight USA Airports-Sparse),this method exhibits superior performance in terms of precision and recall,returning results in 0.10 and 0.03 s,respectively.This indicates greater efficiency compared to PathSim and other comparative methods.
文摘Fault sensing in wind turbine(WT)generator bearings is essential for ensuring reliability and holding down maintenance costs.Feeding raw sensor data to machine learning(ML)model often overlooks the enveloping interdependencies between system elements.This study proposes a new hybrid method that combines the domain knowledge via knowledge graphs(KGs)and the traditional feature-based data.Incorporation of contextual relationships through construction of graph embedding methods,such as Node2Vec,can capture meaningful information,such as the relationships among key parameters(e.g.wind speed,rotor Revolutions Per Minute(RPM),and temperature)in the enriched feature representations.These node embeddings,when augmented with the original data,can be used to allow the model to learn and generalize better.As shown in results achieved on experimental data,the augmented ML model(with KG)is much better at predicting with the help of accuracy and error measure compared to traditional ML methods.Paired t-test analysis proves the statistical validity of this improvement.Moreover,graph-based feature importance increases the interpretability of the model and helps to uncover the structurally significant variables that are otherwise ignored by the common methods.The approach provides an excellent,knowledge-guided manner through which intelligent fault detection can be executed on WT systems.
基金the National Natural Science Foundation of China (Nos. U1501252, 61572146 and U1711263)the Natural Science Foundation of Guangxi Province (No. 2016GXNSFDA380006)+1 种基金the Guangxi Innovation-Driven Development Project (No. AA17202024)the Guangxi Universities Young and Middle-aged Teacher Basic Ability Enhancement Project (No. 2018KY0203).
文摘Knowledge graph embedding aims at embedding entities and relations in a knowledge graph into a continuous, dense, low-dimensional and realvalued vector space. Among various embedding models appeared in recent years, translation-based models such as TransE, TransH and TransR achieve state-of-the-art performance. However, in these models, negative triples used for training phase are generated by replacing each positive entity in positive triples with negative entities from the entity set with the same probability;as a result, a large number of invalid negative triples will be generated and used in the training process. In this paper, a method named adaptive negative sampling (ANS) is proposed to generate valid negative triples. In this method, it first divided all the entities into a number of groups which consist of similar entities by some clustering algorithms such as K-Means. Then, corresponding to each positive triple, the head entity was replaced by a negative entity from the cluster in which the head entity was located and the tail entity was replaced in a similar approach. As a result, it generated a set of high-quality negative triples which benefit for improving the effectiveness of embedding models. The ANS method was combined with the TransE model and the resulted model was named as TransE-ANS. Experimental results show that TransE-ANS achieves significant improvement in the link prediction task.
基金supported by the National Key R&D Program of China(2018AAA0101502)the Science and Technology Project of SGCC(State Grid Corporation of China):Fundamental Theory of Human-in-the-Loop Hybrid-Augmented Intelligence for Power Grid Dispatch and Control。
文摘Knowledge graphs(KGs)have been widely accepted as powerful tools for modeling the complex relationships between concepts and developing knowledge-based services.In recent years,researchers in the field of power systems have explored KGs to develop intelligent dispatching systems for increasingly large power grids.With multiple power grid dispatching knowledge graphs(PDKGs)constructed by different agencies,the knowledge fusion of different PDKGs is useful for providing more accurate decision supports.To achieve this,entity alignment that aims at connecting different KGs by identifying equivalent entities is a critical step.Existing entity alignment methods cannot integrate useful structural,attribute,and relational information while calculating entities’similarities and are prone to making many-to-one alignments,thus can hardly achieve the best performance.To address these issues,this paper proposes a collective entity alignment model that integrates three kinds of available information and makes collective counterpart assignments.This model proposes a novel knowledge graph attention network(KGAT)to learn the embeddings of entities and relations explicitly and calculates entities’similarities by adaptively incorporating the structural,attribute,and relational similarities.Then,we formulate the counterpart assignment task as an integer programming(IP)problem to obtain one-to-one alignments.We not only conduct experiments on a pair of PDKGs but also evaluate o ur model on three commonly used cross-lingual KGs.Experimental comparisons indicate that our model outperforms other methods and provides an effective tool for the knowledge fusion of PDKGs.
基金Supported by the National Natural Science Foundation of China(No.61876144)。
文摘To solve the problem of missing many valid triples in knowledge graphs(KGs),a novel model based on a convolutional neural network(CNN)called ConvKG is proposed,which employs a joint learning strategy for knowledge graph completion(KGC).Related research work has shown the superiority of convolutional neural networks(CNNs)in extracting semantic features of triple embeddings.However,these researches use only one single-shaped filter and fail to extract semantic features of different granularity.To solve this problem,ConvKG exploits multi-shaped filters to co-convolute on the triple embeddings,joint learning semantic features of different granularity.Different shaped filters cover different sizes on the triple embeddings and capture pairwise interactions of different granularity among triple elements.Experimental results confirm the strength of joint learning,and compared with state-of-the-art CNN-based KGC models,ConvKG achieves the better mean rank(MR)and Hits@10 metrics on dataset WN18 RR,and the better MR on dataset FB15k-237.
文摘Knowledge tracing is the key component in online individualized learning,which is capable of assessing the users'mastery of skills and predicting the probability that the users can solve specific problems.Available knowledge tracing models have the problem that the assessments are not directly used in the predictions.To make full use of the assessments during predictions,a novel model,named deep knowledge tracing embedding neural network(DKTENN),is proposed in this work.DKTENN is a synthesis of deep knowledge tracing(DKT)and knowledge graph embedding(KGE).DKT utilizes sophisticated long short-term memory(LSTM)to assess the users and track the mastery of skills according to the users'interaction sequences with skill-level tags,and KGE is applied to predict the probability on the basis of both the embedded problems and DKT's assessments.DKTENN outperforms performance factors analysis and the other knowledge tracing models based on deep learning in the experiments.
基金funded by State Grid Shandong Electric Power Company Science and Technology Project Funding under Grant no.520613200001,520613180002,62061318C002Weihai Scientific Research and Innovation Fund(2020).
文摘The recommendation algorithm based on collaborative filtering is currently the most successful recommendation method. It recommends items to theuser based on the known historical interaction data of the target user. Furthermore,the combination of the recommended algorithm based on collaborative filtrationand other auxiliary knowledge base is an effective way to improve the performance of the recommended system, of which the Co-Factorization Model(CoFM) is one representative research. CoFM, a fusion recommendation modelcombining the collaborative filtering model FM and the graph embeddingmodel TransE, introduces the information of many entities and their relationsin the knowledge graph into the recommendation system as effective auxiliaryinformation. It can effectively improve the accuracy of recommendations andalleviate the problem of sparse user historical interaction data. Unfortunately,the graph-embedded model TransE used in the CoFM model cannot solve the1-N, N-1, and N-N problems well. To tackle this problem, a novel fusion recommendation model Joint Factorization Machines and TransH Model (JFMH) isproposed, which improves CoFM by replacing the TransE model with TransHmodel. A large number of experiments on two widely used benchmark data setsshow that compared with CoFM, JFMH has improved performance in terms ofitem recommendation and knowledge graph completion, and is more competitivethan multiple baseline methods.
基金The National Key R&D Program of China(2018AAA0102100)Hunan Provincial Department of Education Outstanding Youth Project(22B0385)+2 种基金Open Fund of the Domestic First-class Discipline Construction Project of Chinese Medicine of Hunan University of Chinese Medicine(2018ZYX17)Electronic Science and Technology Discipline Open Fund Project of School of Information Science and Engineering,Hunan University of Chinese Medicine(2018-2)Hunan University of Chinese Medicine Graduate Innovation Project(2022CX122)。
文摘With the widespread use of Internet,the amount of data in the field of traditional Chinese medicine(TCM)is growing exponentially.Consequently,there is much attention on the collection of useful knowledge as well as its effective organization and expression.Knowledge graphs have thus emerged,and knowledge reasoning based on this tool has become one of the hot spots of research.This paper first presents a brief introduction to the development of knowledge graphs and knowledge reasoning,and explores the significance of knowledge reasoning.Secondly,the mainstream knowledge reasoning methods,including knowledge reasoning based on traditional rules,knowledge reasoning based on distributed feature representation,and knowledge reasoning based on neural networks are introduced.Then,using stroke as an example,the knowledge reasoning methods are expounded,the principles and characteristics of commonly used knowledge reasoning methods are summarized,and the research and applications of knowledge reasoning techniques in TCM in recent years are sorted out.Finally,we summarize the problems faced in the development of knowledge reasoning in TCM,and put forward the importance of constructing a knowledge reasoning model suitable for the field of TCM.
基金Supported by the National Science and Technology Major Project(2016ZX05007-004)。
文摘Based on the well logging knowledge graph of hydrocarbon-bearing formation(HBF),a Knowledge-Powered Neural Network Formation Evaluation model(KPNFE)has been proposed.It has the following functions:(1)extracting characteristic parameters describing HBF in multiple dimensions and multiple scales;(2)showing the characteristic parameter-related entities,relationships,and attributes as vectors via graph embedding technique;(3)intelligently identifying HBF;(4)seamlessly integrating expertise into the intelligent computing to establish the assessment system and ranking algorithm for potential pay recommendation.Taking 547 wells encountered the low porosity and low permeability Chang 6 Member of Triassic in the Jiyuan Block of Ordos Basin,NW China as objects,80%of the wells were randomly selected as the training dataset and the remainder as the validation dataset.The KPNFE prediction results on the validation dataset had a coincidence rate of 94.43%with the expert interpretation results and a coincidence rate of 84.38%for all the oil testing layers,which is 13 percentage points higher in accuracy and over 100 times faster than the primary conventional interpretation.In addition,a number of potential pays likely to produce industrial oil were recommended.The KPNFE model effectively inherits,carries forward and improves the expert knowledge,nicely solving the robustness problem in HBF identification.The KPNFE,with good interpretability and high accuracy of computation results,is a powerful technical means for efficient and high-quality well logging re-evaluation of old wells in mature oilfields.
基金supported by the Key Project of College-bureau Cooperation of Beijing Police College(No.2022KXJ12)the Key Project of Ministry of Public Security Police Equipment R&D Program(No.2023ZB12)+3 种基金the National Key R&D Program of China(No.2022YFB2703500)the Opening Project of Key Laboratory of Evidence Science(China University of Political Science and Law),the Ministry of Education(No.2022KFKT09)the National Natural Science Foundation of China(No.62272026)the Beijing University of Aeronautics and Astronautics teaching and research projects“Research on teaching methods of research methods course driven by process-based practice ability training”and“Research on teaching methods of introduction to mobile computing driven by practice ability training”.
文摘Traffic accident data analysis and reasoning are crucial for accident prevention and control.Constructing an accident knowledge graph from hybrid datasets of Chinese and English accidents is a valuable task.However,it is challenging due to the need to consider multiple perspectives and infer implicit relationships between actors and factors in complex traffic accidents.To address these challenges,this paper proposes an accident learning attention embeddings based multi-view accident knowledge graph for traffic accident reasoning named BrightAccidentGraph.First,this paper proposes a multi-source traffic accident dataset construction and preprocessing method for traffic accident judgement records published by the China Judgement Document Network and traffic accident records published by the UK’s Ministry of Transport.Then,traffic accident graph construction and portrait method is proposed,we demonstrate the efficiency of the proposed method by constructing several multi-view traffic accident portraits using a multi-source dataset.Furthermore,accident learning attention embeddings based multi-view accident knowledge graph construction and traffic accident reasoning method using deep learning are introduced.Experiments on two hybrid datasets verify the efficiency and merits of our method.
基金supported in part by the Major Scientific and Technological Projects of CNPC under Grant ZD2019-183-006.
文摘Link prediction,also known as Knowledge Graph Completion(KGC),is the common task in Knowledge Graphs(KGs)to predict missing connections between entities.Most existing methods focus on designing shallow,scalable models,which have less expressive than deep,multi-layer models.Furthermore,most operations like addition,matrix multiplications or factorization are handcrafted based on a few known relation patterns in several wellknown datasets,such as FB15k,WN18,etc.However,due to the diversity and complex nature of real-world data distribution,it is inherently difficult to preset all latent patterns.To address this issue,we proposeKGE-ANS,a novel knowledge graph embedding framework for general link prediction tasks using automatic network search.KGEANS can learn a deep,multi-layer effective architecture to adapt to different datasets through neural architecture search.In addition,the general search spacewe designed is tailored forKGtasks.We performextensive experiments on benchmark datasets and the dataset constructed in this paper.The results show that our KGE-ANS outperforms several state-of-the-art methods,especially on these datasets with complex relation patterns.
基金the Beijing Municipal Science and Technology Program(No.Z231100001323004).
文摘Utilizing graph neural networks for knowledge embedding to accomplish the task of knowledge graph completion(KGC)has become an important research area in knowledge graph completion.However,the number of nodes in the knowledge graph increases exponentially with the depth of the tree,whereas the distances of nodes in Euclidean space are second-order polynomial distances,whereby knowledge embedding using graph neural networks in Euclidean space will not represent the distances between nodes well.This paper introduces a novel approach called hyperbolic hierarchical graph attention network(H2GAT)to rectify this limitation.Firstly,the paper conducts knowledge representation in the hyperbolic space,effectively mitigating the issue of exponential growth of nodes with tree depth and consequent information loss.Secondly,it introduces a hierarchical graph atten-tion mechanism specifically designed for the hyperbolic space,allowing for enhanced capture of the network structure inherent in the knowledge graph.Finally,the efficacy of the proposed H2GAT model is evaluated on benchmark datasets,namely WN18RR and FB15K-237,thereby validating its effectiveness.The H2GAT model achieved 0.445,0.515,and 0.586 in the Hits@1,Hits@3 and Hits@10 metrics respectively on the WN18RR dataset and 0.243,0.367 and 0.518 on the FB15K-237 dataset.By incorporating hyperbolic space embedding and hierarchical graph attention,the H2GAT model successfully addresses the limitations of existing hyperbolic knowledge embedding models,exhibiting its competence in knowledge graph completion tasks.
基金supported by the National Natural Science Foundation of China under Grant No.72293575.
文摘Temporal knowledge graph completion(TKGC),which merges temporal information into traditional static knowledge graph completion(SKGC),has garnered increasing attention recently.Among numerous emerging approaches,translation-based embedding models constitute a prominent approach in TKGC research.However,existing translation-based methods typically incorporate timestamps into entities or relations,rather than utilizing them independently.This practice fails to fully exploit the rich semantics inherent in temporal information,thereby weakening the expressive capability of models.To address this limitation,we propose embedding timestamps,like entities and relations,in one or more dedicated semantic spaces.After projecting all embeddings into a shared space,we use the relation-timestamp pair instead of the conventional relation embedding as the translation vector between head and tail entities.Our method elevates timestamps to the same representational significance as entities and relations.Based on this strategy,we introduce two novel translation-based embedding models:TE-TransR and TE-TransT.With the independent representation of timestamps,our method not only enhances capabilities in link prediction but also facilitates a relatively underexplored task,namely time prediction.To further bolster the precision and reliability of time prediction,we introduce a granular,time unit-based timestamp setting and a relation-specific evaluation protocol.Extensive experiments demonstrate that our models achieve strong performance on link prediction benchmarks,with TE-TransR outperforming existing baselines in the time prediction task.
