Medical visual question answering(MedVQA)aims to enhance diagnostic confidence and deepen patientsunderstanding of their health conditions.While the Transformer architecture is widely used in multimodal fields,its app...Medical visual question answering(MedVQA)aims to enhance diagnostic confidence and deepen patientsunderstanding of their health conditions.While the Transformer architecture is widely used in multimodal fields,its application in MedVQA requires further enhancement.A critical limitation of contemporary MedVQA systems lies in the inability to integrate lifelong knowledge with specific patient data to generate human-like responses.Existing Transformer-based MedVQA models require enhancing their capabitities for interpreting answers through the applications of medical image knowledge.The introduction of the medical knowledge graph visual language transformer(MKGViLT),designed for joint medical knowledge graphs(KGs),addresses this challenge.MKGViLT incorporates an enhanced Transformer structure to effectively extract features and combine modalities for MedVQA tasks.The MKGViLT model delivers answers based on richer background knowledge,thereby enhancing performance.The efficacy of MKGViLT is evaluated using the SLAKE and P-VQA datasets.Experimental results show that MKGViLT surpasses the most advanced methods on the SLAKE dataset.展开更多
Tracking and analyzing data from research projects is critical for understanding research trends and supporting the development of science and technology strategies.However,the data from these projects is often comple...Tracking and analyzing data from research projects is critical for understanding research trends and supporting the development of science and technology strategies.However,the data from these projects is often complex and inadequate,making it challenging for researchers to conduct in-depth data mining to improve policies or management.To address this problem,this paper adopts a top-down approach to construct a knowledge graph(KG)for research projects.Firstly,we construct an integrated ontology by referring to the metamodel of various architectures,which is called the meta-model integration conceptual reference model.Subsequently,we use the dependency parsing method to extract knowledge from unstructured textual data and use the entity alignment method based on weakly supervised learning to classify the extracted entities,completing the construction of the KG for the research projects.In addition,a knowledge inference model based on representation learning is employed to achieve knowledge completion and improve the KG.Finally,experiments are conducted on the KG for research projects and the results demonstrate the effectiveness of the proposed method in enriching incomplete data within the KG.展开更多
Knowledge graph(KG)serves as a specialized semantic network that encapsulates intricate relationships among real-world entities within a structured framework.This framework facilitates a transformation in information ...Knowledge graph(KG)serves as a specialized semantic network that encapsulates intricate relationships among real-world entities within a structured framework.This framework facilitates a transformation in information retrieval,transitioning it from mere string matching to far more sophisticated entity matching.In this transformative process,the advancement of artificial intelligence and intelligent information services is invigorated.Meanwhile,the role ofmachine learningmethod in the construction of KG is important,and these techniques have already achieved initial success.This article embarks on a comprehensive journey through the last strides in the field of KG via machine learning.With a profound amalgamation of cutting-edge research in machine learning,this article undertakes a systematical exploration of KG construction methods in three distinct phases:entity learning,ontology learning,and knowledge reasoning.Especially,a meticulous dissection of machine learningdriven algorithms is conducted,spotlighting their contributions to critical facets such as entity extraction,relation extraction,entity linking,and link prediction.Moreover,this article also provides an analysis of the unresolved challenges and emerging trajectories that beckon within the expansive application of machine learning-fueled,large-scale KG construction.展开更多
Knowledge graph(KG) link prediction aims to address the problem of missing multiple valid triples in KGs. Existing approaches either struggle to efficiently model the message passing process of multi-hop paths or lack...Knowledge graph(KG) link prediction aims to address the problem of missing multiple valid triples in KGs. Existing approaches either struggle to efficiently model the message passing process of multi-hop paths or lack transparency of model prediction principles. In this paper,a new graph convolutional network path semantic-aware graph convolution network(PSGCN) is proposed to achieve modeling the semantic information of multi-hop paths. PSGCN first uses a random walk strategy to obtain all-hop paths in KGs,then captures the semantics of the paths by Word2Sec and long shortterm memory(LSTM) models,and finally converts them into a potential representation for the graph convolution network(GCN) messaging process. PSGCN combines path-based inference methods and graph neural networks to achieve better interpretability and scalability. In addition,to ensure the robustness of the model,the value of the path thresholdKis experimented on the FB15K-237 and WN18RR datasets,and the final results prove the effectiveness of the model.展开更多
Knowledge graph(KG)conflict resolution is to solve knowledge conflicts problem in the construction of KG.Aiming at the problem of KG conflict resolution,a KG conflict resolution algorithm NGDcrm is proposed,which is a...Knowledge graph(KG)conflict resolution is to solve knowledge conflicts problem in the construction of KG.Aiming at the problem of KG conflict resolution,a KG conflict resolution algorithm NGDcrm is proposed,which is a numeric graph dependency-based conflict resolution method.NGDcrm utilizes the dependency graph to perform arithmetic calculation and predicate comparison of numerical entity knowledge in the KG.NGDcrm first uses a parallel segmentation method to segment the KG;then,it extracts the features of the KG according to KG embedding;finally,it uses numerical graph dependencies to detect and correct the wrong facts in the KG based on the extracted features.The experimental results on real data show that NGDcrm is better than the state-of-the-art knowledge conflict resolution method.Among them,the AUC value of NGDcrm on the DBpedia dataset is 15.4%higher than the state-of-the-art method.展开更多
At present,knowledge embedding methods are widely used in the field of knowledge graph(KG)reasoning,and have been successfully applied to those with large entities and relationships.However,in research and production ...At present,knowledge embedding methods are widely used in the field of knowledge graph(KG)reasoning,and have been successfully applied to those with large entities and relationships.However,in research and production environments,there are a large number of KGs with a small number of entities and relations,which are called sparse KGs.Limited by the performance of knowledge extraction methods or some other reasons(some common-sense information does not appear in the natural corpus),the relation between entities is often incomplete.To solve this problem,a method of the graph neural network and information enhancement is proposed.The improved method increases the mean reciprocal rank(MRR)and Hit@3 by 1.6%and 1.7%,respectively,when the sparsity of the FB15K-237 dataset is 10%.When the sparsity is 50%,the evaluation indexes MRR and Hit@10 are increased by 0.8%and 1.8%,respectively.展开更多
Accurately recommending candidate news to users is a basic challenge of personalized news recommendation systems.Traditional methods are usually difficult to learn and acquire complex semantic information in news text...Accurately recommending candidate news to users is a basic challenge of personalized news recommendation systems.Traditional methods are usually difficult to learn and acquire complex semantic information in news texts,resulting in unsatisfactory recommendation results.Besides,these traditional methods are more friendly to active users with rich historical behaviors.However,they can not effectively solve the long tail problem of inactive users.To address these issues,this research presents a novel general framework that combines Large Language Models(LLM)and Knowledge Graphs(KG)into traditional methods.To learn the contextual information of news text,we use LLMs’powerful text understanding ability to generate news representations with rich semantic information,and then,the generated news representations are used to enhance the news encoding in traditional methods.In addition,multi-hops relationship of news entities is mined and the structural information of news is encoded using KG,thus alleviating the challenge of long-tail distribution.Experimental results demonstrate that compared with various traditional models,on evaluation indicators such as AUC,MRR,nDCG@5 and nDCG@10,the framework significantly improves the recommendation performance.The successful integration of LLM and KG in our framework has established a feasible way for achieving more accurate personalized news recommendation.Our code is available at https://github.com/Xuan-ZW/LKPNR.展开更多
As the research of knowledge graph(KG)is deepened and widely used,knowledge graph com-pletion(KGC)has attracted more and more attentions from researchers,especially in scenarios of in-telligent search,social networks ...As the research of knowledge graph(KG)is deepened and widely used,knowledge graph com-pletion(KGC)has attracted more and more attentions from researchers,especially in scenarios of in-telligent search,social networks and deep question and answer(Q&A).Current research mainly fo-cuses on the completion of static knowledge graphs,and the temporal information in temporal knowl-edge graphs(TKGs)is ignored.However,the temporal information is definitely very helpful for the completion.Note that existing researches on temporal knowledge graph completion are difficult to process temporal information and to integrate entities,relations and time well.In this work,a rotation and scaling(RotatS)model is proposed,which learns rotation and scaling transformations from head entity embedding to tail entity embedding in 3D spaces to capture the information of time and rela-tions in the temporal knowledge graph.The performance of the proposed RotatS model have been evaluated by comparison with several baselines under similar experimental conditions and space com-plexity on four typical knowl good graph completion datasets publicly available online.The study shows that RotatS can achieve good results in terms of prediction accuracy.展开更多
大语言模型(large language model,LLM)随着不断发展,在开放领域取得了出色的表现.然而,由于缺乏专业知识,LLM在垂直领域问答任务上效果较差.这一问题引发了研究者的广泛关注.现有研究通过“检索-问答”的方式,将领域知识注入大语言模型...大语言模型(large language model,LLM)随着不断发展,在开放领域取得了出色的表现.然而,由于缺乏专业知识,LLM在垂直领域问答任务上效果较差.这一问题引发了研究者的广泛关注.现有研究通过“检索-问答”的方式,将领域知识注入大语言模型,以增强其性能.然而该方式通常会检索到额外的噪声数据而导致LLM的性能损失.为了解决该问题,提出基于知识相关性的知识图谱问答方法.具体而言,将噪声数据与回答问题所需要的知识进行区分,在“检索-相关性评估-问答”的框架下,引导大语言模型选择合理的知识做出正确的回答.此外,提出一个机械领域知识图谱问答的数据集Mecha-QA,包含传统机械制造以及增材制造两个子领域,以推进该领域大语言模型与知识图谱问答相关的研究.为了验证所提方法的有效性,在Mecha-QA和航空航天领域数据集Aero-QA上进行实验.结果表明,该方法可以显著提升大语言模型在垂直领域知识图谱问答的性能.展开更多
基金Supported by the National Natural Science Foundation of China(No.62001313)the Liaoning Professional Talent Protect(No.XLYC2203046)the Shenyang Municipal Medical Engineering Cross Research Foundation of China(No.22-321-32-09).
文摘Medical visual question answering(MedVQA)aims to enhance diagnostic confidence and deepen patientsunderstanding of their health conditions.While the Transformer architecture is widely used in multimodal fields,its application in MedVQA requires further enhancement.A critical limitation of contemporary MedVQA systems lies in the inability to integrate lifelong knowledge with specific patient data to generate human-like responses.Existing Transformer-based MedVQA models require enhancing their capabitities for interpreting answers through the applications of medical image knowledge.The introduction of the medical knowledge graph visual language transformer(MKGViLT),designed for joint medical knowledge graphs(KGs),addresses this challenge.MKGViLT incorporates an enhanced Transformer structure to effectively extract features and combine modalities for MedVQA tasks.The MKGViLT model delivers answers based on richer background knowledge,thereby enhancing performance.The efficacy of MKGViLT is evaluated using the SLAKE and P-VQA datasets.Experimental results show that MKGViLT surpasses the most advanced methods on the SLAKE dataset.
基金supported by the National Natural Science Foundation of China(72101263).
文摘Tracking and analyzing data from research projects is critical for understanding research trends and supporting the development of science and technology strategies.However,the data from these projects is often complex and inadequate,making it challenging for researchers to conduct in-depth data mining to improve policies or management.To address this problem,this paper adopts a top-down approach to construct a knowledge graph(KG)for research projects.Firstly,we construct an integrated ontology by referring to the metamodel of various architectures,which is called the meta-model integration conceptual reference model.Subsequently,we use the dependency parsing method to extract knowledge from unstructured textual data and use the entity alignment method based on weakly supervised learning to classify the extracted entities,completing the construction of the KG for the research projects.In addition,a knowledge inference model based on representation learning is employed to achieve knowledge completion and improve the KG.Finally,experiments are conducted on the KG for research projects and the results demonstrate the effectiveness of the proposed method in enriching incomplete data within the KG.
基金supported in part by the Beijing Natural Science Foundation under Grants L211020 and M21032in part by the National Natural Science Foundation of China under Grants U1836106 and 62271045in part by the Scientific and Technological Innovation Foundation of Foshan under Grants BK21BF001 and BK20BF010。
文摘Knowledge graph(KG)serves as a specialized semantic network that encapsulates intricate relationships among real-world entities within a structured framework.This framework facilitates a transformation in information retrieval,transitioning it from mere string matching to far more sophisticated entity matching.In this transformative process,the advancement of artificial intelligence and intelligent information services is invigorated.Meanwhile,the role ofmachine learningmethod in the construction of KG is important,and these techniques have already achieved initial success.This article embarks on a comprehensive journey through the last strides in the field of KG via machine learning.With a profound amalgamation of cutting-edge research in machine learning,this article undertakes a systematical exploration of KG construction methods in three distinct phases:entity learning,ontology learning,and knowledge reasoning.Especially,a meticulous dissection of machine learningdriven algorithms is conducted,spotlighting their contributions to critical facets such as entity extraction,relation extraction,entity linking,and link prediction.Moreover,this article also provides an analysis of the unresolved challenges and emerging trajectories that beckon within the expansive application of machine learning-fueled,large-scale KG construction.
基金Supported by the National Natural Science Foundation of China(No.61876144).
文摘Knowledge graph(KG) link prediction aims to address the problem of missing multiple valid triples in KGs. Existing approaches either struggle to efficiently model the message passing process of multi-hop paths or lack transparency of model prediction principles. In this paper,a new graph convolutional network path semantic-aware graph convolution network(PSGCN) is proposed to achieve modeling the semantic information of multi-hop paths. PSGCN first uses a random walk strategy to obtain all-hop paths in KGs,then captures the semantics of the paths by Word2Sec and long shortterm memory(LSTM) models,and finally converts them into a potential representation for the graph convolution network(GCN) messaging process. PSGCN combines path-based inference methods and graph neural networks to achieve better interpretability and scalability. In addition,to ensure the robustness of the model,the value of the path thresholdKis experimented on the FB15K-237 and WN18RR datasets,and the final results prove the effectiveness of the model.
基金Supported by the Henan Province Science and Technology Department Foundation(No.202102310237,192102210133,202102310295)the Doctoral Research Fund of Zhengzhou University of Light Industry(No.2018BSJJ039)the Internet Medical and Health Service Henan Collaborative Innovation Center Open Project Fund(No.IH2019006).
文摘Knowledge graph(KG)conflict resolution is to solve knowledge conflicts problem in the construction of KG.Aiming at the problem of KG conflict resolution,a KG conflict resolution algorithm NGDcrm is proposed,which is a numeric graph dependency-based conflict resolution method.NGDcrm utilizes the dependency graph to perform arithmetic calculation and predicate comparison of numerical entity knowledge in the KG.NGDcrm first uses a parallel segmentation method to segment the KG;then,it extracts the features of the KG according to KG embedding;finally,it uses numerical graph dependencies to detect and correct the wrong facts in the KG based on the extracted features.The experimental results on real data show that NGDcrm is better than the state-of-the-art knowledge conflict resolution method.Among them,the AUC value of NGDcrm on the DBpedia dataset is 15.4%higher than the state-of-the-art method.
基金supported by the Sichuan Science and Technology Program under Grants No.2022YFQ0052 and No.2021YFQ0009.
文摘At present,knowledge embedding methods are widely used in the field of knowledge graph(KG)reasoning,and have been successfully applied to those with large entities and relationships.However,in research and production environments,there are a large number of KGs with a small number of entities and relations,which are called sparse KGs.Limited by the performance of knowledge extraction methods or some other reasons(some common-sense information does not appear in the natural corpus),the relation between entities is often incomplete.To solve this problem,a method of the graph neural network and information enhancement is proposed.The improved method increases the mean reciprocal rank(MRR)and Hit@3 by 1.6%and 1.7%,respectively,when the sparsity of the FB15K-237 dataset is 10%.When the sparsity is 50%,the evaluation indexes MRR and Hit@10 are increased by 0.8%and 1.8%,respectively.
基金supported by National Key R&D Program of China(2022QY2000-02).
文摘Accurately recommending candidate news to users is a basic challenge of personalized news recommendation systems.Traditional methods are usually difficult to learn and acquire complex semantic information in news texts,resulting in unsatisfactory recommendation results.Besides,these traditional methods are more friendly to active users with rich historical behaviors.However,they can not effectively solve the long tail problem of inactive users.To address these issues,this research presents a novel general framework that combines Large Language Models(LLM)and Knowledge Graphs(KG)into traditional methods.To learn the contextual information of news text,we use LLMs’powerful text understanding ability to generate news representations with rich semantic information,and then,the generated news representations are used to enhance the news encoding in traditional methods.In addition,multi-hops relationship of news entities is mined and the structural information of news is encoded using KG,thus alleviating the challenge of long-tail distribution.Experimental results demonstrate that compared with various traditional models,on evaluation indicators such as AUC,MRR,nDCG@5 and nDCG@10,the framework significantly improves the recommendation performance.The successful integration of LLM and KG in our framework has established a feasible way for achieving more accurate personalized news recommendation.Our code is available at https://github.com/Xuan-ZW/LKPNR.
基金the National Natural Science Foundation of China(No.6187022153).
文摘As the research of knowledge graph(KG)is deepened and widely used,knowledge graph com-pletion(KGC)has attracted more and more attentions from researchers,especially in scenarios of in-telligent search,social networks and deep question and answer(Q&A).Current research mainly fo-cuses on the completion of static knowledge graphs,and the temporal information in temporal knowl-edge graphs(TKGs)is ignored.However,the temporal information is definitely very helpful for the completion.Note that existing researches on temporal knowledge graph completion are difficult to process temporal information and to integrate entities,relations and time well.In this work,a rotation and scaling(RotatS)model is proposed,which learns rotation and scaling transformations from head entity embedding to tail entity embedding in 3D spaces to capture the information of time and rela-tions in the temporal knowledge graph.The performance of the proposed RotatS model have been evaluated by comparison with several baselines under similar experimental conditions and space com-plexity on four typical knowl good graph completion datasets publicly available online.The study shows that RotatS can achieve good results in terms of prediction accuracy.
文摘大语言模型(large language model,LLM)随着不断发展,在开放领域取得了出色的表现.然而,由于缺乏专业知识,LLM在垂直领域问答任务上效果较差.这一问题引发了研究者的广泛关注.现有研究通过“检索-问答”的方式,将领域知识注入大语言模型,以增强其性能.然而该方式通常会检索到额外的噪声数据而导致LLM的性能损失.为了解决该问题,提出基于知识相关性的知识图谱问答方法.具体而言,将噪声数据与回答问题所需要的知识进行区分,在“检索-相关性评估-问答”的框架下,引导大语言模型选择合理的知识做出正确的回答.此外,提出一个机械领域知识图谱问答的数据集Mecha-QA,包含传统机械制造以及增材制造两个子领域,以推进该领域大语言模型与知识图谱问答相关的研究.为了验证所提方法的有效性,在Mecha-QA和航空航天领域数据集Aero-QA上进行实验.结果表明,该方法可以显著提升大语言模型在垂直领域知识图谱问答的性能.
