Medical visual question answering(MedVQA)aims to enhance diagnostic confidence and deepen patientsunderstanding of their health conditions.While the Transformer architecture is widely used in multimodal fields,its app...Medical visual question answering(MedVQA)aims to enhance diagnostic confidence and deepen patientsunderstanding of their health conditions.While the Transformer architecture is widely used in multimodal fields,its application in MedVQA requires further enhancement.A critical limitation of contemporary MedVQA systems lies in the inability to integrate lifelong knowledge with specific patient data to generate human-like responses.Existing Transformer-based MedVQA models require enhancing their capabitities for interpreting answers through the applications of medical image knowledge.The introduction of the medical knowledge graph visual language transformer(MKGViLT),designed for joint medical knowledge graphs(KGs),addresses this challenge.MKGViLT incorporates an enhanced Transformer structure to effectively extract features and combine modalities for MedVQA tasks.The MKGViLT model delivers answers based on richer background knowledge,thereby enhancing performance.The efficacy of MKGViLT is evaluated using the SLAKE and P-VQA datasets.Experimental results show that MKGViLT surpasses the most advanced methods on the SLAKE dataset.展开更多
In the domain of knowledge graph embedding,conventional approaches typically transform entities and relations into continuous vector spaces.However,parameter efficiency becomes increasingly crucial when dealing with l...In the domain of knowledge graph embedding,conventional approaches typically transform entities and relations into continuous vector spaces.However,parameter efficiency becomes increasingly crucial when dealing with large-scale knowledge graphs that contain vast numbers of entities and relations.In particular,resource-intensive embeddings often lead to increased computational costs,and may limit scalability and adaptability in practical environ-ments,such as in low-resource settings or real-world applications.This paper explores an approach to knowledge graph representation learning that leverages small,reserved entities and relation sets for parameter-efficient embedding.We introduce a hierarchical attention network designed to refine and maximize the representational quality of embeddings by selectively focusing on these reserved sets,thereby reducing model complexity.Empirical assessments validate that our model achieves high performance on the benchmark dataset with fewer parameters and smaller embedding dimensions.The ablation studies further highlight the impact and contribution of each component in the proposed hierarchical attention structure.展开更多
Purpose:Due to the incompleteness nature of knowledge graphs(KGs),the task of predicting missing links between entities becomes important.Many previous approaches are static,this posed a notable problem that all meani...Purpose:Due to the incompleteness nature of knowledge graphs(KGs),the task of predicting missing links between entities becomes important.Many previous approaches are static,this posed a notable problem that all meanings of a polysemous entity share one embedding vector.This study aims to propose a polysemous embedding approach,named KG embedding under relational contexts(ContE for short),for missing link prediction.Design/methodology/approach:ContE models and infers different relationship patterns by considering the context of the relationship,which is implicit in the local neighborhood of the relationship.The forward and backward impacts of the relationship in ContE are mapped to two different embedding vectors,which represent the contextual information of the relationship.Then,according to the position of the entity,the entity’s polysemous representation is obtained by adding its static embedding vector to the corresponding context vector of the relationship.Findings:ContE is a fully expressive,that is,given any ground truth over the triples,there are embedding assignments to entities and relations that can precisely separate the true triples from false ones.ContE is capable of modeling four connectivity patterns such as symmetry,antisymmetry,inversion and composition.Research limitations:ContE needs to do a grid search to find best parameters to get best performance in practice,which is a time-consuming task.Sometimes,it requires longer entity vectors to get better performance than some other models.Practical implications:ContE is a bilinear model,which is a quite simple model that could be applied to large-scale KGs.By considering contexts of relations,ContE can distinguish the exact meaning of an entity in different triples so that when performing compositional reasoning,it is capable to infer the connectivity patterns of relations and achieves good performance on link prediction tasks.Originality/value:ContE considers the contexts of entities in terms of their positions in triples and the relationships they link to.It decomposes a relation vector into two vectors,namely,forward impact vector and backward impact vector in order to capture the relational contexts.ContE has the same low computational complexity as TransE.Therefore,it provides a new approach for contextualized knowledge graph embedding.展开更多
Tracking and analyzing data from research projects is critical for understanding research trends and supporting the development of science and technology strategies.However,the data from these projects is often comple...Tracking and analyzing data from research projects is critical for understanding research trends and supporting the development of science and technology strategies.However,the data from these projects is often complex and inadequate,making it challenging for researchers to conduct in-depth data mining to improve policies or management.To address this problem,this paper adopts a top-down approach to construct a knowledge graph(KG)for research projects.Firstly,we construct an integrated ontology by referring to the metamodel of various architectures,which is called the meta-model integration conceptual reference model.Subsequently,we use the dependency parsing method to extract knowledge from unstructured textual data and use the entity alignment method based on weakly supervised learning to classify the extracted entities,completing the construction of the KG for the research projects.In addition,a knowledge inference model based on representation learning is employed to achieve knowledge completion and improve the KG.Finally,experiments are conducted on the KG for research projects and the results demonstrate the effectiveness of the proposed method in enriching incomplete data within the KG.展开更多
Accurate prediction of future events brings great benefits and reduces losses for society in many domains,such as civil unrest,pandemics,and crimes.Knowledge graph is a general language for describing and modeling com...Accurate prediction of future events brings great benefits and reduces losses for society in many domains,such as civil unrest,pandemics,and crimes.Knowledge graph is a general language for describing and modeling complex systems.Different types of events continually occur,which are often related to historical and concurrent events.In this paper,we formalize the future event prediction as a temporal knowledge graph reasoning problem.Most existing studies either conduct reasoning on static knowledge graphs or assume knowledges graphs of all timestamps are available during the training process.As a result,they cannot effectively reason over temporal knowledge graphs and predict events happening in the future.To address this problem,some recent works learn to infer future events based on historical eventbased temporal knowledge graphs.However,these methods do not comprehensively consider the latent patterns and influences behind historical events and concurrent events simultaneously.This paper proposes a new graph representation learning model,namely Recurrent Event Graph ATtention Network(RE-GAT),based on a novel historical and concurrent events attention-aware mechanism by modeling the event knowledge graph sequence recurrently.More specifically,our RE-GAT uses an attention-based historical events embedding module to encode past events,and employs an attention-based concurrent events embedding module to model the associations of events at the same timestamp.A translation-based decoder module and a learning objective are developed to optimize the embeddings of entities and relations.We evaluate our proposed method on four benchmark datasets.Extensive experimental results demonstrate the superiority of our RE-GAT model comparing to various base-lines,which proves that our method can more accurately predict what events are going to happen.展开更多
Predicting potential facts in the future,Temporal Knowledge Graph(TKG)extrapolation remains challenging because of the deep dependence between the temporal association and semantic patterns of facts.Intuitively,facts(...Predicting potential facts in the future,Temporal Knowledge Graph(TKG)extrapolation remains challenging because of the deep dependence between the temporal association and semantic patterns of facts.Intuitively,facts(events)that happened at different timestamps have different influences on future events,which can be attributed to a hierarchy among not only facts but also relevant entities.Therefore,it is crucial to pay more attention to important entities and events when forecasting the future.However,most existing methods focus on reasoning over temporally evolving facts or mining evolutional patterns from known facts,which may be affected by the diversity and variability of the evolution,and they might fail to attach importance to facts that matter.Hyperbolic geometry was proved to be effective in capturing hierarchical patterns among data,which is considered to be a solution for modelling hierarchical relations among facts.To this end,we propose ReTIN,a novel model integrating real-time influence of historical facts for TKG reasoning based on hyperbolic geometry,which provides low-dimensional embeddings to capture latent hierarchical structures and other rich semantic patterns of the existing TKG.Considering both real-time and global features of TKG boosts the adaptation of ReTIN to the ever-changing dynamics and inherent constraints.Extensive experiments on benchmarks demonstrate the superiority of ReTIN over various baselines.The ablation study further supports the value of exploiting temporal information.展开更多
To solve the low efficiency of approximate queries caused by the large sizes of the knowledge graphs in the real world,an embedding-based approximate query method is proposed.First,the nodes in the query graph are cla...To solve the low efficiency of approximate queries caused by the large sizes of the knowledge graphs in the real world,an embedding-based approximate query method is proposed.First,the nodes in the query graph are classified according to the degrees of approximation required for different types of nodes.This classification transforms the query problem into three constraints,from which approximate information is extracted.Second,candidates are generated by calculating the similarity between embeddings.Finally,a deep neural network model is designed,incorporating a loss function based on the high-dimensional ellipsoidal diffusion distance.This model identifies the distance between nodes using their embeddings and constructs a score function.k nodes are returned as the query results.The results show that the proposed method can return both exact results and approximate matching results.On datasets DBLP(DataBase systems and Logic Programming)and FUA-S(Flight USA Airports-Sparse),this method exhibits superior performance in terms of precision and recall,returning results in 0.10 and 0.03 s,respectively.This indicates greater efficiency compared to PathSim and other comparative methods.展开更多
Fault sensing in wind turbine(WT)generator bearings is essential for ensuring reliability and holding down maintenance costs.Feeding raw sensor data to machine learning(ML)model often overlooks the enveloping interdep...Fault sensing in wind turbine(WT)generator bearings is essential for ensuring reliability and holding down maintenance costs.Feeding raw sensor data to machine learning(ML)model often overlooks the enveloping interdependencies between system elements.This study proposes a new hybrid method that combines the domain knowledge via knowledge graphs(KGs)and the traditional feature-based data.Incorporation of contextual relationships through construction of graph embedding methods,such as Node2Vec,can capture meaningful information,such as the relationships among key parameters(e.g.wind speed,rotor Revolutions Per Minute(RPM),and temperature)in the enriched feature representations.These node embeddings,when augmented with the original data,can be used to allow the model to learn and generalize better.As shown in results achieved on experimental data,the augmented ML model(with KG)is much better at predicting with the help of accuracy and error measure compared to traditional ML methods.Paired t-test analysis proves the statistical validity of this improvement.Moreover,graph-based feature importance increases the interpretability of the model and helps to uncover the structurally significant variables that are otherwise ignored by the common methods.The approach provides an excellent,knowledge-guided manner through which intelligent fault detection can be executed on WT systems.展开更多
Knowledge graph embedding aims at embedding entities and relations in a knowledge graph into a continuous, dense, low-dimensional and realvalued vector space. Among various embedding models appeared in recent years, t...Knowledge graph embedding aims at embedding entities and relations in a knowledge graph into a continuous, dense, low-dimensional and realvalued vector space. Among various embedding models appeared in recent years, translation-based models such as TransE, TransH and TransR achieve state-of-the-art performance. However, in these models, negative triples used for training phase are generated by replacing each positive entity in positive triples with negative entities from the entity set with the same probability;as a result, a large number of invalid negative triples will be generated and used in the training process. In this paper, a method named adaptive negative sampling (ANS) is proposed to generate valid negative triples. In this method, it first divided all the entities into a number of groups which consist of similar entities by some clustering algorithms such as K-Means. Then, corresponding to each positive triple, the head entity was replaced by a negative entity from the cluster in which the head entity was located and the tail entity was replaced in a similar approach. As a result, it generated a set of high-quality negative triples which benefit for improving the effectiveness of embedding models. The ANS method was combined with the TransE model and the resulted model was named as TransE-ANS. Experimental results show that TransE-ANS achieves significant improvement in the link prediction task.展开更多
Knowledge graphs(KGs)have been widely accepted as powerful tools for modeling the complex relationships between concepts and developing knowledge-based services.In recent years,researchers in the field of power system...Knowledge graphs(KGs)have been widely accepted as powerful tools for modeling the complex relationships between concepts and developing knowledge-based services.In recent years,researchers in the field of power systems have explored KGs to develop intelligent dispatching systems for increasingly large power grids.With multiple power grid dispatching knowledge graphs(PDKGs)constructed by different agencies,the knowledge fusion of different PDKGs is useful for providing more accurate decision supports.To achieve this,entity alignment that aims at connecting different KGs by identifying equivalent entities is a critical step.Existing entity alignment methods cannot integrate useful structural,attribute,and relational information while calculating entities’similarities and are prone to making many-to-one alignments,thus can hardly achieve the best performance.To address these issues,this paper proposes a collective entity alignment model that integrates three kinds of available information and makes collective counterpart assignments.This model proposes a novel knowledge graph attention network(KGAT)to learn the embeddings of entities and relations explicitly and calculates entities’similarities by adaptively incorporating the structural,attribute,and relational similarities.Then,we formulate the counterpart assignment task as an integer programming(IP)problem to obtain one-to-one alignments.We not only conduct experiments on a pair of PDKGs but also evaluate o ur model on three commonly used cross-lingual KGs.Experimental comparisons indicate that our model outperforms other methods and provides an effective tool for the knowledge fusion of PDKGs.展开更多
Knowledge graph(KG)serves as a specialized semantic network that encapsulates intricate relationships among real-world entities within a structured framework.This framework facilitates a transformation in information ...Knowledge graph(KG)serves as a specialized semantic network that encapsulates intricate relationships among real-world entities within a structured framework.This framework facilitates a transformation in information retrieval,transitioning it from mere string matching to far more sophisticated entity matching.In this transformative process,the advancement of artificial intelligence and intelligent information services is invigorated.Meanwhile,the role ofmachine learningmethod in the construction of KG is important,and these techniques have already achieved initial success.This article embarks on a comprehensive journey through the last strides in the field of KG via machine learning.With a profound amalgamation of cutting-edge research in machine learning,this article undertakes a systematical exploration of KG construction methods in three distinct phases:entity learning,ontology learning,and knowledge reasoning.Especially,a meticulous dissection of machine learningdriven algorithms is conducted,spotlighting their contributions to critical facets such as entity extraction,relation extraction,entity linking,and link prediction.Moreover,this article also provides an analysis of the unresolved challenges and emerging trajectories that beckon within the expansive application of machine learning-fueled,large-scale KG construction.展开更多
The recommendation algorithm based on collaborative filtering is currently the most successful recommendation method. It recommends items to theuser based on the known historical interaction data of the target user. F...The recommendation algorithm based on collaborative filtering is currently the most successful recommendation method. It recommends items to theuser based on the known historical interaction data of the target user. Furthermore,the combination of the recommended algorithm based on collaborative filtrationand other auxiliary knowledge base is an effective way to improve the performance of the recommended system, of which the Co-Factorization Model(CoFM) is one representative research. CoFM, a fusion recommendation modelcombining the collaborative filtering model FM and the graph embeddingmodel TransE, introduces the information of many entities and their relationsin the knowledge graph into the recommendation system as effective auxiliaryinformation. It can effectively improve the accuracy of recommendations andalleviate the problem of sparse user historical interaction data. Unfortunately,the graph-embedded model TransE used in the CoFM model cannot solve the1-N, N-1, and N-N problems well. To tackle this problem, a novel fusion recommendation model Joint Factorization Machines and TransH Model (JFMH) isproposed, which improves CoFM by replacing the TransE model with TransHmodel. A large number of experiments on two widely used benchmark data setsshow that compared with CoFM, JFMH has improved performance in terms ofitem recommendation and knowledge graph completion, and is more competitivethan multiple baseline methods.展开更多
To solve the problem of missing many valid triples in knowledge graphs(KGs),a novel model based on a convolutional neural network(CNN)called ConvKG is proposed,which employs a joint learning strategy for knowledge gra...To solve the problem of missing many valid triples in knowledge graphs(KGs),a novel model based on a convolutional neural network(CNN)called ConvKG is proposed,which employs a joint learning strategy for knowledge graph completion(KGC).Related research work has shown the superiority of convolutional neural networks(CNNs)in extracting semantic features of triple embeddings.However,these researches use only one single-shaped filter and fail to extract semantic features of different granularity.To solve this problem,ConvKG exploits multi-shaped filters to co-convolute on the triple embeddings,joint learning semantic features of different granularity.Different shaped filters cover different sizes on the triple embeddings and capture pairwise interactions of different granularity among triple elements.Experimental results confirm the strength of joint learning,and compared with state-of-the-art CNN-based KGC models,ConvKG achieves the better mean rank(MR)and Hits@10 metrics on dataset WN18 RR,and the better MR on dataset FB15k-237.展开更多
With the widespread use of Internet,the amount of data in the field of traditional Chinese medicine(TCM)is growing exponentially.Consequently,there is much attention on the collection of useful knowledge as well as it...With the widespread use of Internet,the amount of data in the field of traditional Chinese medicine(TCM)is growing exponentially.Consequently,there is much attention on the collection of useful knowledge as well as its effective organization and expression.Knowledge graphs have thus emerged,and knowledge reasoning based on this tool has become one of the hot spots of research.This paper first presents a brief introduction to the development of knowledge graphs and knowledge reasoning,and explores the significance of knowledge reasoning.Secondly,the mainstream knowledge reasoning methods,including knowledge reasoning based on traditional rules,knowledge reasoning based on distributed feature representation,and knowledge reasoning based on neural networks are introduced.Then,using stroke as an example,the knowledge reasoning methods are expounded,the principles and characteristics of commonly used knowledge reasoning methods are summarized,and the research and applications of knowledge reasoning techniques in TCM in recent years are sorted out.Finally,we summarize the problems faced in the development of knowledge reasoning in TCM,and put forward the importance of constructing a knowledge reasoning model suitable for the field of TCM.展开更多
Based on the well logging knowledge graph of hydrocarbon-bearing formation(HBF),a Knowledge-Powered Neural Network Formation Evaluation model(KPNFE)has been proposed.It has the following functions:(1)extracting charac...Based on the well logging knowledge graph of hydrocarbon-bearing formation(HBF),a Knowledge-Powered Neural Network Formation Evaluation model(KPNFE)has been proposed.It has the following functions:(1)extracting characteristic parameters describing HBF in multiple dimensions and multiple scales;(2)showing the characteristic parameter-related entities,relationships,and attributes as vectors via graph embedding technique;(3)intelligently identifying HBF;(4)seamlessly integrating expertise into the intelligent computing to establish the assessment system and ranking algorithm for potential pay recommendation.Taking 547 wells encountered the low porosity and low permeability Chang 6 Member of Triassic in the Jiyuan Block of Ordos Basin,NW China as objects,80%of the wells were randomly selected as the training dataset and the remainder as the validation dataset.The KPNFE prediction results on the validation dataset had a coincidence rate of 94.43%with the expert interpretation results and a coincidence rate of 84.38%for all the oil testing layers,which is 13 percentage points higher in accuracy and over 100 times faster than the primary conventional interpretation.In addition,a number of potential pays likely to produce industrial oil were recommended.The KPNFE model effectively inherits,carries forward and improves the expert knowledge,nicely solving the robustness problem in HBF identification.The KPNFE,with good interpretability and high accuracy of computation results,is a powerful technical means for efficient and high-quality well logging re-evaluation of old wells in mature oilfields.展开更多
Knowledge graph(KG)conflict resolution is to solve knowledge conflicts problem in the construction of KG.Aiming at the problem of KG conflict resolution,a KG conflict resolution algorithm NGDcrm is proposed,which is a...Knowledge graph(KG)conflict resolution is to solve knowledge conflicts problem in the construction of KG.Aiming at the problem of KG conflict resolution,a KG conflict resolution algorithm NGDcrm is proposed,which is a numeric graph dependency-based conflict resolution method.NGDcrm utilizes the dependency graph to perform arithmetic calculation and predicate comparison of numerical entity knowledge in the KG.NGDcrm first uses a parallel segmentation method to segment the KG;then,it extracts the features of the KG according to KG embedding;finally,it uses numerical graph dependencies to detect and correct the wrong facts in the KG based on the extracted features.The experimental results on real data show that NGDcrm is better than the state-of-the-art knowledge conflict resolution method.Among them,the AUC value of NGDcrm on the DBpedia dataset is 15.4%higher than the state-of-the-art method.展开更多
Accurately recommending candidate news to users is a basic challenge of personalized news recommendation systems.Traditional methods are usually difficult to learn and acquire complex semantic information in news text...Accurately recommending candidate news to users is a basic challenge of personalized news recommendation systems.Traditional methods are usually difficult to learn and acquire complex semantic information in news texts,resulting in unsatisfactory recommendation results.Besides,these traditional methods are more friendly to active users with rich historical behaviors.However,they can not effectively solve the long tail problem of inactive users.To address these issues,this research presents a novel general framework that combines Large Language Models(LLM)and Knowledge Graphs(KG)into traditional methods.To learn the contextual information of news text,we use LLMs’powerful text understanding ability to generate news representations with rich semantic information,and then,the generated news representations are used to enhance the news encoding in traditional methods.In addition,multi-hops relationship of news entities is mined and the structural information of news is encoded using KG,thus alleviating the challenge of long-tail distribution.Experimental results demonstrate that compared with various traditional models,on evaluation indicators such as AUC,MRR,nDCG@5 and nDCG@10,the framework significantly improves the recommendation performance.The successful integration of LLM and KG in our framework has established a feasible way for achieving more accurate personalized news recommendation.Our code is available at https://github.com/Xuan-ZW/LKPNR.展开更多
Knowledge tracing is the key component in online individualized learning,which is capable of assessing the users'mastery of skills and predicting the probability that the users can solve specific problems.Availabl...Knowledge tracing is the key component in online individualized learning,which is capable of assessing the users'mastery of skills and predicting the probability that the users can solve specific problems.Available knowledge tracing models have the problem that the assessments are not directly used in the predictions.To make full use of the assessments during predictions,a novel model,named deep knowledge tracing embedding neural network(DKTENN),is proposed in this work.DKTENN is a synthesis of deep knowledge tracing(DKT)and knowledge graph embedding(KGE).DKT utilizes sophisticated long short-term memory(LSTM)to assess the users and track the mastery of skills according to the users'interaction sequences with skill-level tags,and KGE is applied to predict the probability on the basis of both the embedded problems and DKT's assessments.DKTENN outperforms performance factors analysis and the other knowledge tracing models based on deep learning in the experiments.展开更多
基金Supported by the National Natural Science Foundation of China(No.62001313)the Liaoning Professional Talent Protect(No.XLYC2203046)the Shenyang Municipal Medical Engineering Cross Research Foundation of China(No.22-321-32-09).
文摘Medical visual question answering(MedVQA)aims to enhance diagnostic confidence and deepen patientsunderstanding of their health conditions.While the Transformer architecture is widely used in multimodal fields,its application in MedVQA requires further enhancement.A critical limitation of contemporary MedVQA systems lies in the inability to integrate lifelong knowledge with specific patient data to generate human-like responses.Existing Transformer-based MedVQA models require enhancing their capabitities for interpreting answers through the applications of medical image knowledge.The introduction of the medical knowledge graph visual language transformer(MKGViLT),designed for joint medical knowledge graphs(KGs),addresses this challenge.MKGViLT incorporates an enhanced Transformer structure to effectively extract features and combine modalities for MedVQA tasks.The MKGViLT model delivers answers based on richer background knowledge,thereby enhancing performance.The efficacy of MKGViLT is evaluated using the SLAKE and P-VQA datasets.Experimental results show that MKGViLT surpasses the most advanced methods on the SLAKE dataset.
基金supported by the National Science and Technology Council(NSTC),Taiwan,under Grants Numbers 112-2622-E-029-009 and 112-2221-E-029-019.
文摘In the domain of knowledge graph embedding,conventional approaches typically transform entities and relations into continuous vector spaces.However,parameter efficiency becomes increasingly crucial when dealing with large-scale knowledge graphs that contain vast numbers of entities and relations.In particular,resource-intensive embeddings often lead to increased computational costs,and may limit scalability and adaptability in practical environ-ments,such as in low-resource settings or real-world applications.This paper explores an approach to knowledge graph representation learning that leverages small,reserved entities and relation sets for parameter-efficient embedding.We introduce a hierarchical attention network designed to refine and maximize the representational quality of embeddings by selectively focusing on these reserved sets,thereby reducing model complexity.Empirical assessments validate that our model achieves high performance on the benchmark dataset with fewer parameters and smaller embedding dimensions.The ablation studies further highlight the impact and contribution of each component in the proposed hierarchical attention structure.
基金supported by the Key R&D Program Project of Zhejiang Province under Grant no.2019 C01004 and 2021C02004.
文摘Purpose:Due to the incompleteness nature of knowledge graphs(KGs),the task of predicting missing links between entities becomes important.Many previous approaches are static,this posed a notable problem that all meanings of a polysemous entity share one embedding vector.This study aims to propose a polysemous embedding approach,named KG embedding under relational contexts(ContE for short),for missing link prediction.Design/methodology/approach:ContE models and infers different relationship patterns by considering the context of the relationship,which is implicit in the local neighborhood of the relationship.The forward and backward impacts of the relationship in ContE are mapped to two different embedding vectors,which represent the contextual information of the relationship.Then,according to the position of the entity,the entity’s polysemous representation is obtained by adding its static embedding vector to the corresponding context vector of the relationship.Findings:ContE is a fully expressive,that is,given any ground truth over the triples,there are embedding assignments to entities and relations that can precisely separate the true triples from false ones.ContE is capable of modeling four connectivity patterns such as symmetry,antisymmetry,inversion and composition.Research limitations:ContE needs to do a grid search to find best parameters to get best performance in practice,which is a time-consuming task.Sometimes,it requires longer entity vectors to get better performance than some other models.Practical implications:ContE is a bilinear model,which is a quite simple model that could be applied to large-scale KGs.By considering contexts of relations,ContE can distinguish the exact meaning of an entity in different triples so that when performing compositional reasoning,it is capable to infer the connectivity patterns of relations and achieves good performance on link prediction tasks.Originality/value:ContE considers the contexts of entities in terms of their positions in triples and the relationships they link to.It decomposes a relation vector into two vectors,namely,forward impact vector and backward impact vector in order to capture the relational contexts.ContE has the same low computational complexity as TransE.Therefore,it provides a new approach for contextualized knowledge graph embedding.
基金supported by the National Natural Science Foundation of China(72101263).
文摘Tracking and analyzing data from research projects is critical for understanding research trends and supporting the development of science and technology strategies.However,the data from these projects is often complex and inadequate,making it challenging for researchers to conduct in-depth data mining to improve policies or management.To address this problem,this paper adopts a top-down approach to construct a knowledge graph(KG)for research projects.Firstly,we construct an integrated ontology by referring to the metamodel of various architectures,which is called the meta-model integration conceptual reference model.Subsequently,we use the dependency parsing method to extract knowledge from unstructured textual data and use the entity alignment method based on weakly supervised learning to classify the extracted entities,completing the construction of the KG for the research projects.In addition,a knowledge inference model based on representation learning is employed to achieve knowledge completion and improve the KG.Finally,experiments are conducted on the KG for research projects and the results demonstrate the effectiveness of the proposed method in enriching incomplete data within the KG.
基金supported by the National Natural Science Foundation of China under grants U19B2044National Key Research and Development Program of China(2021YFC3300500).
文摘Accurate prediction of future events brings great benefits and reduces losses for society in many domains,such as civil unrest,pandemics,and crimes.Knowledge graph is a general language for describing and modeling complex systems.Different types of events continually occur,which are often related to historical and concurrent events.In this paper,we formalize the future event prediction as a temporal knowledge graph reasoning problem.Most existing studies either conduct reasoning on static knowledge graphs or assume knowledges graphs of all timestamps are available during the training process.As a result,they cannot effectively reason over temporal knowledge graphs and predict events happening in the future.To address this problem,some recent works learn to infer future events based on historical eventbased temporal knowledge graphs.However,these methods do not comprehensively consider the latent patterns and influences behind historical events and concurrent events simultaneously.This paper proposes a new graph representation learning model,namely Recurrent Event Graph ATtention Network(RE-GAT),based on a novel historical and concurrent events attention-aware mechanism by modeling the event knowledge graph sequence recurrently.More specifically,our RE-GAT uses an attention-based historical events embedding module to encode past events,and employs an attention-based concurrent events embedding module to model the associations of events at the same timestamp.A translation-based decoder module and a learning objective are developed to optimize the embeddings of entities and relations.We evaluate our proposed method on four benchmark datasets.Extensive experimental results demonstrate the superiority of our RE-GAT model comparing to various base-lines,which proves that our method can more accurately predict what events are going to happen.
基金Major Key Project of Pengcheng Laboratory,Grant/Award Number:PCL2022A03。
文摘Predicting potential facts in the future,Temporal Knowledge Graph(TKG)extrapolation remains challenging because of the deep dependence between the temporal association and semantic patterns of facts.Intuitively,facts(events)that happened at different timestamps have different influences on future events,which can be attributed to a hierarchy among not only facts but also relevant entities.Therefore,it is crucial to pay more attention to important entities and events when forecasting the future.However,most existing methods focus on reasoning over temporally evolving facts or mining evolutional patterns from known facts,which may be affected by the diversity and variability of the evolution,and they might fail to attach importance to facts that matter.Hyperbolic geometry was proved to be effective in capturing hierarchical patterns among data,which is considered to be a solution for modelling hierarchical relations among facts.To this end,we propose ReTIN,a novel model integrating real-time influence of historical facts for TKG reasoning based on hyperbolic geometry,which provides low-dimensional embeddings to capture latent hierarchical structures and other rich semantic patterns of the existing TKG.Considering both real-time and global features of TKG boosts the adaptation of ReTIN to the ever-changing dynamics and inherent constraints.Extensive experiments on benchmarks demonstrate the superiority of ReTIN over various baselines.The ablation study further supports the value of exploiting temporal information.
基金The State Grid Technology Project(No.5108202340042A-1-1-ZN).
文摘To solve the low efficiency of approximate queries caused by the large sizes of the knowledge graphs in the real world,an embedding-based approximate query method is proposed.First,the nodes in the query graph are classified according to the degrees of approximation required for different types of nodes.This classification transforms the query problem into three constraints,from which approximate information is extracted.Second,candidates are generated by calculating the similarity between embeddings.Finally,a deep neural network model is designed,incorporating a loss function based on the high-dimensional ellipsoidal diffusion distance.This model identifies the distance between nodes using their embeddings and constructs a score function.k nodes are returned as the query results.The results show that the proposed method can return both exact results and approximate matching results.On datasets DBLP(DataBase systems and Logic Programming)and FUA-S(Flight USA Airports-Sparse),this method exhibits superior performance in terms of precision and recall,returning results in 0.10 and 0.03 s,respectively.This indicates greater efficiency compared to PathSim and other comparative methods.
文摘Fault sensing in wind turbine(WT)generator bearings is essential for ensuring reliability and holding down maintenance costs.Feeding raw sensor data to machine learning(ML)model often overlooks the enveloping interdependencies between system elements.This study proposes a new hybrid method that combines the domain knowledge via knowledge graphs(KGs)and the traditional feature-based data.Incorporation of contextual relationships through construction of graph embedding methods,such as Node2Vec,can capture meaningful information,such as the relationships among key parameters(e.g.wind speed,rotor Revolutions Per Minute(RPM),and temperature)in the enriched feature representations.These node embeddings,when augmented with the original data,can be used to allow the model to learn and generalize better.As shown in results achieved on experimental data,the augmented ML model(with KG)is much better at predicting with the help of accuracy and error measure compared to traditional ML methods.Paired t-test analysis proves the statistical validity of this improvement.Moreover,graph-based feature importance increases the interpretability of the model and helps to uncover the structurally significant variables that are otherwise ignored by the common methods.The approach provides an excellent,knowledge-guided manner through which intelligent fault detection can be executed on WT systems.
基金the National Natural Science Foundation of China (Nos. U1501252, 61572146 and U1711263)the Natural Science Foundation of Guangxi Province (No. 2016GXNSFDA380006)+1 种基金the Guangxi Innovation-Driven Development Project (No. AA17202024)the Guangxi Universities Young and Middle-aged Teacher Basic Ability Enhancement Project (No. 2018KY0203).
文摘Knowledge graph embedding aims at embedding entities and relations in a knowledge graph into a continuous, dense, low-dimensional and realvalued vector space. Among various embedding models appeared in recent years, translation-based models such as TransE, TransH and TransR achieve state-of-the-art performance. However, in these models, negative triples used for training phase are generated by replacing each positive entity in positive triples with negative entities from the entity set with the same probability;as a result, a large number of invalid negative triples will be generated and used in the training process. In this paper, a method named adaptive negative sampling (ANS) is proposed to generate valid negative triples. In this method, it first divided all the entities into a number of groups which consist of similar entities by some clustering algorithms such as K-Means. Then, corresponding to each positive triple, the head entity was replaced by a negative entity from the cluster in which the head entity was located and the tail entity was replaced in a similar approach. As a result, it generated a set of high-quality negative triples which benefit for improving the effectiveness of embedding models. The ANS method was combined with the TransE model and the resulted model was named as TransE-ANS. Experimental results show that TransE-ANS achieves significant improvement in the link prediction task.
基金supported by the National Key R&D Program of China(2018AAA0101502)the Science and Technology Project of SGCC(State Grid Corporation of China):Fundamental Theory of Human-in-the-Loop Hybrid-Augmented Intelligence for Power Grid Dispatch and Control。
文摘Knowledge graphs(KGs)have been widely accepted as powerful tools for modeling the complex relationships between concepts and developing knowledge-based services.In recent years,researchers in the field of power systems have explored KGs to develop intelligent dispatching systems for increasingly large power grids.With multiple power grid dispatching knowledge graphs(PDKGs)constructed by different agencies,the knowledge fusion of different PDKGs is useful for providing more accurate decision supports.To achieve this,entity alignment that aims at connecting different KGs by identifying equivalent entities is a critical step.Existing entity alignment methods cannot integrate useful structural,attribute,and relational information while calculating entities’similarities and are prone to making many-to-one alignments,thus can hardly achieve the best performance.To address these issues,this paper proposes a collective entity alignment model that integrates three kinds of available information and makes collective counterpart assignments.This model proposes a novel knowledge graph attention network(KGAT)to learn the embeddings of entities and relations explicitly and calculates entities’similarities by adaptively incorporating the structural,attribute,and relational similarities.Then,we formulate the counterpart assignment task as an integer programming(IP)problem to obtain one-to-one alignments.We not only conduct experiments on a pair of PDKGs but also evaluate o ur model on three commonly used cross-lingual KGs.Experimental comparisons indicate that our model outperforms other methods and provides an effective tool for the knowledge fusion of PDKGs.
基金supported in part by the Beijing Natural Science Foundation under Grants L211020 and M21032in part by the National Natural Science Foundation of China under Grants U1836106 and 62271045in part by the Scientific and Technological Innovation Foundation of Foshan under Grants BK21BF001 and BK20BF010。
文摘Knowledge graph(KG)serves as a specialized semantic network that encapsulates intricate relationships among real-world entities within a structured framework.This framework facilitates a transformation in information retrieval,transitioning it from mere string matching to far more sophisticated entity matching.In this transformative process,the advancement of artificial intelligence and intelligent information services is invigorated.Meanwhile,the role ofmachine learningmethod in the construction of KG is important,and these techniques have already achieved initial success.This article embarks on a comprehensive journey through the last strides in the field of KG via machine learning.With a profound amalgamation of cutting-edge research in machine learning,this article undertakes a systematical exploration of KG construction methods in three distinct phases:entity learning,ontology learning,and knowledge reasoning.Especially,a meticulous dissection of machine learningdriven algorithms is conducted,spotlighting their contributions to critical facets such as entity extraction,relation extraction,entity linking,and link prediction.Moreover,this article also provides an analysis of the unresolved challenges and emerging trajectories that beckon within the expansive application of machine learning-fueled,large-scale KG construction.
基金funded by State Grid Shandong Electric Power Company Science and Technology Project Funding under Grant no.520613200001,520613180002,62061318C002Weihai Scientific Research and Innovation Fund(2020).
文摘The recommendation algorithm based on collaborative filtering is currently the most successful recommendation method. It recommends items to theuser based on the known historical interaction data of the target user. Furthermore,the combination of the recommended algorithm based on collaborative filtrationand other auxiliary knowledge base is an effective way to improve the performance of the recommended system, of which the Co-Factorization Model(CoFM) is one representative research. CoFM, a fusion recommendation modelcombining the collaborative filtering model FM and the graph embeddingmodel TransE, introduces the information of many entities and their relationsin the knowledge graph into the recommendation system as effective auxiliaryinformation. It can effectively improve the accuracy of recommendations andalleviate the problem of sparse user historical interaction data. Unfortunately,the graph-embedded model TransE used in the CoFM model cannot solve the1-N, N-1, and N-N problems well. To tackle this problem, a novel fusion recommendation model Joint Factorization Machines and TransH Model (JFMH) isproposed, which improves CoFM by replacing the TransE model with TransHmodel. A large number of experiments on two widely used benchmark data setsshow that compared with CoFM, JFMH has improved performance in terms ofitem recommendation and knowledge graph completion, and is more competitivethan multiple baseline methods.
基金Supported by the National Natural Science Foundation of China(No.61876144)。
文摘To solve the problem of missing many valid triples in knowledge graphs(KGs),a novel model based on a convolutional neural network(CNN)called ConvKG is proposed,which employs a joint learning strategy for knowledge graph completion(KGC).Related research work has shown the superiority of convolutional neural networks(CNNs)in extracting semantic features of triple embeddings.However,these researches use only one single-shaped filter and fail to extract semantic features of different granularity.To solve this problem,ConvKG exploits multi-shaped filters to co-convolute on the triple embeddings,joint learning semantic features of different granularity.Different shaped filters cover different sizes on the triple embeddings and capture pairwise interactions of different granularity among triple elements.Experimental results confirm the strength of joint learning,and compared with state-of-the-art CNN-based KGC models,ConvKG achieves the better mean rank(MR)and Hits@10 metrics on dataset WN18 RR,and the better MR on dataset FB15k-237.
基金The National Key R&D Program of China(2018AAA0102100)Hunan Provincial Department of Education Outstanding Youth Project(22B0385)+2 种基金Open Fund of the Domestic First-class Discipline Construction Project of Chinese Medicine of Hunan University of Chinese Medicine(2018ZYX17)Electronic Science and Technology Discipline Open Fund Project of School of Information Science and Engineering,Hunan University of Chinese Medicine(2018-2)Hunan University of Chinese Medicine Graduate Innovation Project(2022CX122)。
文摘With the widespread use of Internet,the amount of data in the field of traditional Chinese medicine(TCM)is growing exponentially.Consequently,there is much attention on the collection of useful knowledge as well as its effective organization and expression.Knowledge graphs have thus emerged,and knowledge reasoning based on this tool has become one of the hot spots of research.This paper first presents a brief introduction to the development of knowledge graphs and knowledge reasoning,and explores the significance of knowledge reasoning.Secondly,the mainstream knowledge reasoning methods,including knowledge reasoning based on traditional rules,knowledge reasoning based on distributed feature representation,and knowledge reasoning based on neural networks are introduced.Then,using stroke as an example,the knowledge reasoning methods are expounded,the principles and characteristics of commonly used knowledge reasoning methods are summarized,and the research and applications of knowledge reasoning techniques in TCM in recent years are sorted out.Finally,we summarize the problems faced in the development of knowledge reasoning in TCM,and put forward the importance of constructing a knowledge reasoning model suitable for the field of TCM.
基金Supported by the National Science and Technology Major Project(2016ZX05007-004)。
文摘Based on the well logging knowledge graph of hydrocarbon-bearing formation(HBF),a Knowledge-Powered Neural Network Formation Evaluation model(KPNFE)has been proposed.It has the following functions:(1)extracting characteristic parameters describing HBF in multiple dimensions and multiple scales;(2)showing the characteristic parameter-related entities,relationships,and attributes as vectors via graph embedding technique;(3)intelligently identifying HBF;(4)seamlessly integrating expertise into the intelligent computing to establish the assessment system and ranking algorithm for potential pay recommendation.Taking 547 wells encountered the low porosity and low permeability Chang 6 Member of Triassic in the Jiyuan Block of Ordos Basin,NW China as objects,80%of the wells were randomly selected as the training dataset and the remainder as the validation dataset.The KPNFE prediction results on the validation dataset had a coincidence rate of 94.43%with the expert interpretation results and a coincidence rate of 84.38%for all the oil testing layers,which is 13 percentage points higher in accuracy and over 100 times faster than the primary conventional interpretation.In addition,a number of potential pays likely to produce industrial oil were recommended.The KPNFE model effectively inherits,carries forward and improves the expert knowledge,nicely solving the robustness problem in HBF identification.The KPNFE,with good interpretability and high accuracy of computation results,is a powerful technical means for efficient and high-quality well logging re-evaluation of old wells in mature oilfields.
基金Supported by the Henan Province Science and Technology Department Foundation(No.202102310237,192102210133,202102310295)the Doctoral Research Fund of Zhengzhou University of Light Industry(No.2018BSJJ039)the Internet Medical and Health Service Henan Collaborative Innovation Center Open Project Fund(No.IH2019006).
文摘Knowledge graph(KG)conflict resolution is to solve knowledge conflicts problem in the construction of KG.Aiming at the problem of KG conflict resolution,a KG conflict resolution algorithm NGDcrm is proposed,which is a numeric graph dependency-based conflict resolution method.NGDcrm utilizes the dependency graph to perform arithmetic calculation and predicate comparison of numerical entity knowledge in the KG.NGDcrm first uses a parallel segmentation method to segment the KG;then,it extracts the features of the KG according to KG embedding;finally,it uses numerical graph dependencies to detect and correct the wrong facts in the KG based on the extracted features.The experimental results on real data show that NGDcrm is better than the state-of-the-art knowledge conflict resolution method.Among them,the AUC value of NGDcrm on the DBpedia dataset is 15.4%higher than the state-of-the-art method.
基金supported by National Key R&D Program of China(2022QY2000-02).
文摘Accurately recommending candidate news to users is a basic challenge of personalized news recommendation systems.Traditional methods are usually difficult to learn and acquire complex semantic information in news texts,resulting in unsatisfactory recommendation results.Besides,these traditional methods are more friendly to active users with rich historical behaviors.However,they can not effectively solve the long tail problem of inactive users.To address these issues,this research presents a novel general framework that combines Large Language Models(LLM)and Knowledge Graphs(KG)into traditional methods.To learn the contextual information of news text,we use LLMs’powerful text understanding ability to generate news representations with rich semantic information,and then,the generated news representations are used to enhance the news encoding in traditional methods.In addition,multi-hops relationship of news entities is mined and the structural information of news is encoded using KG,thus alleviating the challenge of long-tail distribution.Experimental results demonstrate that compared with various traditional models,on evaluation indicators such as AUC,MRR,nDCG@5 and nDCG@10,the framework significantly improves the recommendation performance.The successful integration of LLM and KG in our framework has established a feasible way for achieving more accurate personalized news recommendation.Our code is available at https://github.com/Xuan-ZW/LKPNR.
文摘Knowledge tracing is the key component in online individualized learning,which is capable of assessing the users'mastery of skills and predicting the probability that the users can solve specific problems.Available knowledge tracing models have the problem that the assessments are not directly used in the predictions.To make full use of the assessments during predictions,a novel model,named deep knowledge tracing embedding neural network(DKTENN),is proposed in this work.DKTENN is a synthesis of deep knowledge tracing(DKT)and knowledge graph embedding(KGE).DKT utilizes sophisticated long short-term memory(LSTM)to assess the users and track the mastery of skills according to the users'interaction sequences with skill-level tags,and KGE is applied to predict the probability on the basis of both the embedded problems and DKT's assessments.DKTENN outperforms performance factors analysis and the other knowledge tracing models based on deep learning in the experiments.
