Medical visual question answering(MedVQA)aims to enhance diagnostic confidence and deepen patientsunderstanding of their health conditions.While the Transformer architecture is widely used in multimodal fields,its app...Medical visual question answering(MedVQA)aims to enhance diagnostic confidence and deepen patientsunderstanding of their health conditions.While the Transformer architecture is widely used in multimodal fields,its application in MedVQA requires further enhancement.A critical limitation of contemporary MedVQA systems lies in the inability to integrate lifelong knowledge with specific patient data to generate human-like responses.Existing Transformer-based MedVQA models require enhancing their capabitities for interpreting answers through the applications of medical image knowledge.The introduction of the medical knowledge graph visual language transformer(MKGViLT),designed for joint medical knowledge graphs(KGs),addresses this challenge.MKGViLT incorporates an enhanced Transformer structure to effectively extract features and combine modalities for MedVQA tasks.The MKGViLT model delivers answers based on richer background knowledge,thereby enhancing performance.The efficacy of MKGViLT is evaluated using the SLAKE and P-VQA datasets.Experimental results show that MKGViLT surpasses the most advanced methods on the SLAKE dataset.展开更多
Temporal alignment of multisensor time series(MTS)is a critical prerequisite for accurate modeling and optimal control in subsequent data-driven applications.Nevertheless,many approaches frequently neglect to consider...Temporal alignment of multisensor time series(MTS)is a critical prerequisite for accurate modeling and optimal control in subsequent data-driven applications.Nevertheless,many approaches frequently neglect to consider the complex interdependencies between different sensors in MTS,and temporal alignment in many methods is typically treated as an isolated task disconnected from the downstream objectives,leading to unsatisfactory performances in follow-up applications.To address these challenges,this paper proposes a novel knowledge graph(KG)-guided iterative-updating graph neural network(GNN)for time-delay estimation(TDE)in MTS.Initially,a domain-specific KG is constructed from domain mechanism knowledge,providing a foundation for GNN's initialization.Next,capitalizing on the inherent structure of the graph topology,a GNN-based TDE method is developed.Then,a customized loss function is constructed,which synthesizes both the performances of downstream tasks and graph-based constraints.Moreover,an innovative algorithm for GNN structure learning and iterative-updating is proposed to renovate the graph structure further.Finally,experimental results across various regression and classification tasks on numerical simulation,public datasets,and the real blast furnace ironmaking dataset demonstrate that the proposed method can achieve accurate temporal alignment of MTS.展开更多
Tracking and analyzing data from research projects is critical for understanding research trends and supporting the development of science and technology strategies.However,the data from these projects is often comple...Tracking and analyzing data from research projects is critical for understanding research trends and supporting the development of science and technology strategies.However,the data from these projects is often complex and inadequate,making it challenging for researchers to conduct in-depth data mining to improve policies or management.To address this problem,this paper adopts a top-down approach to construct a knowledge graph(KG)for research projects.Firstly,we construct an integrated ontology by referring to the metamodel of various architectures,which is called the meta-model integration conceptual reference model.Subsequently,we use the dependency parsing method to extract knowledge from unstructured textual data and use the entity alignment method based on weakly supervised learning to classify the extracted entities,completing the construction of the KG for the research projects.In addition,a knowledge inference model based on representation learning is employed to achieve knowledge completion and improve the KG.Finally,experiments are conducted on the KG for research projects and the results demonstrate the effectiveness of the proposed method in enriching incomplete data within the KG.展开更多
现有的检索增强生成(RAG)问答系统在特定领域应用时,存在检索路径单一、用户潜在意图覆盖不足和召回文段质量低导致的系统回答准确性低与不全面的问题。因此,提出一种双阶段优化方法——预回答与召回过滤(PARF)。首先,通过结合领域知识...现有的检索增强生成(RAG)问答系统在特定领域应用时,存在检索路径单一、用户潜在意图覆盖不足和召回文段质量低导致的系统回答准确性低与不全面的问题。因此,提出一种双阶段优化方法——预回答与召回过滤(PARF)。首先,通过结合领域知识图谱与提示工程技术,引导大语言模型(LLM)生成预回答,构建“原始查询→预回答→相关文段”的多向检索路径,从而扩展原始查询的语义空间;其次,利用BERT(Bidirectional Encoder Representations from Transformers)模型对召回文段进行相关性评分与过滤,实现检索与生成阶段的协同优化,提升有效信息的密度。实验结果表明,相较于基线方法DPR-LLM(Dense Passage Retrieval with LLM)构建的RAG问答系统,PARF方法构建的RAG问答系统的一致性指标F1和ROUGE-L(Recall-Oriented Understudy for Gisting Evaluation-L)在轨道交通问答数据集上分别提升19.8和41.5个百分点,在医药问答数据集上分别提升16.1和17.6个百分点,效果指标正确率分别提升10.2和8.8个百分点。展开更多
随着大语言模型(Large Language Model,简称LLM)技术的飞速发展,衍生出了对文档的检索增强生成(Retrieval-Augmented Generation,简称RAG)方法,但对于特定领域内多样化异构文档,传统RAG方法仍难以运用。面向对知识推理准确性有苛刻要求...随着大语言模型(Large Language Model,简称LLM)技术的飞速发展,衍生出了对文档的检索增强生成(Retrieval-Augmented Generation,简称RAG)方法,但对于特定领域内多样化异构文档,传统RAG方法仍难以运用。面向对知识推理准确性有苛刻要求的特定领域(如军事、医疗等),本文针对远距离知识无法关联、答案不完整、特殊领域推理能力匮乏等问题,提出了基于远亲知识联结推理的异构文档问答增强方法(简称KiRAG),该方法通过解析异构文档,抽取、构建领域知识图谱,根据问题在图谱中推理并动态召回文档分片,使LLM理解问题和生成答案时,具备完备的信息视野,以确保最终答案的完整性、合理性及准确性。实验验证表明,该方法能够有效解决军事领域异构文档的知识问答运用问题,并具备在其他相似领域泛化运用的潜力。展开更多
Knowledge graph(KG)serves as a specialized semantic network that encapsulates intricate relationships among real-world entities within a structured framework.This framework facilitates a transformation in information ...Knowledge graph(KG)serves as a specialized semantic network that encapsulates intricate relationships among real-world entities within a structured framework.This framework facilitates a transformation in information retrieval,transitioning it from mere string matching to far more sophisticated entity matching.In this transformative process,the advancement of artificial intelligence and intelligent information services is invigorated.Meanwhile,the role ofmachine learningmethod in the construction of KG is important,and these techniques have already achieved initial success.This article embarks on a comprehensive journey through the last strides in the field of KG via machine learning.With a profound amalgamation of cutting-edge research in machine learning,this article undertakes a systematical exploration of KG construction methods in three distinct phases:entity learning,ontology learning,and knowledge reasoning.Especially,a meticulous dissection of machine learningdriven algorithms is conducted,spotlighting their contributions to critical facets such as entity extraction,relation extraction,entity linking,and link prediction.Moreover,this article also provides an analysis of the unresolved challenges and emerging trajectories that beckon within the expansive application of machine learning-fueled,large-scale KG construction.展开更多
Knowledge graph(KG) link prediction aims to address the problem of missing multiple valid triples in KGs. Existing approaches either struggle to efficiently model the message passing process of multi-hop paths or lack...Knowledge graph(KG) link prediction aims to address the problem of missing multiple valid triples in KGs. Existing approaches either struggle to efficiently model the message passing process of multi-hop paths or lack transparency of model prediction principles. In this paper,a new graph convolutional network path semantic-aware graph convolution network(PSGCN) is proposed to achieve modeling the semantic information of multi-hop paths. PSGCN first uses a random walk strategy to obtain all-hop paths in KGs,then captures the semantics of the paths by Word2Sec and long shortterm memory(LSTM) models,and finally converts them into a potential representation for the graph convolution network(GCN) messaging process. PSGCN combines path-based inference methods and graph neural networks to achieve better interpretability and scalability. In addition,to ensure the robustness of the model,the value of the path thresholdKis experimented on the FB15K-237 and WN18RR datasets,and the final results prove the effectiveness of the model.展开更多
Accurately recommending candidate news to users is a basic challenge of personalized news recommendation systems.Traditional methods are usually difficult to learn and acquire complex semantic information in news text...Accurately recommending candidate news to users is a basic challenge of personalized news recommendation systems.Traditional methods are usually difficult to learn and acquire complex semantic information in news texts,resulting in unsatisfactory recommendation results.Besides,these traditional methods are more friendly to active users with rich historical behaviors.However,they can not effectively solve the long tail problem of inactive users.To address these issues,this research presents a novel general framework that combines Large Language Models(LLM)and Knowledge Graphs(KG)into traditional methods.To learn the contextual information of news text,we use LLMs’powerful text understanding ability to generate news representations with rich semantic information,and then,the generated news representations are used to enhance the news encoding in traditional methods.In addition,multi-hops relationship of news entities is mined and the structural information of news is encoded using KG,thus alleviating the challenge of long-tail distribution.Experimental results demonstrate that compared with various traditional models,on evaluation indicators such as AUC,MRR,nDCG@5 and nDCG@10,the framework significantly improves the recommendation performance.The successful integration of LLM and KG in our framework has established a feasible way for achieving more accurate personalized news recommendation.Our code is available at https://github.com/Xuan-ZW/LKPNR.展开更多
Knowledge graph(KG)conflict resolution is to solve knowledge conflicts problem in the construction of KG.Aiming at the problem of KG conflict resolution,a KG conflict resolution algorithm NGDcrm is proposed,which is a...Knowledge graph(KG)conflict resolution is to solve knowledge conflicts problem in the construction of KG.Aiming at the problem of KG conflict resolution,a KG conflict resolution algorithm NGDcrm is proposed,which is a numeric graph dependency-based conflict resolution method.NGDcrm utilizes the dependency graph to perform arithmetic calculation and predicate comparison of numerical entity knowledge in the KG.NGDcrm first uses a parallel segmentation method to segment the KG;then,it extracts the features of the KG according to KG embedding;finally,it uses numerical graph dependencies to detect and correct the wrong facts in the KG based on the extracted features.The experimental results on real data show that NGDcrm is better than the state-of-the-art knowledge conflict resolution method.Among them,the AUC value of NGDcrm on the DBpedia dataset is 15.4%higher than the state-of-the-art method.展开更多
At present,knowledge embedding methods are widely used in the field of knowledge graph(KG)reasoning,and have been successfully applied to those with large entities and relationships.However,in research and production ...At present,knowledge embedding methods are widely used in the field of knowledge graph(KG)reasoning,and have been successfully applied to those with large entities and relationships.However,in research and production environments,there are a large number of KGs with a small number of entities and relations,which are called sparse KGs.Limited by the performance of knowledge extraction methods or some other reasons(some common-sense information does not appear in the natural corpus),the relation between entities is often incomplete.To solve this problem,a method of the graph neural network and information enhancement is proposed.The improved method increases the mean reciprocal rank(MRR)and Hit@3 by 1.6%and 1.7%,respectively,when the sparsity of the FB15K-237 dataset is 10%.When the sparsity is 50%,the evaluation indexes MRR and Hit@10 are increased by 0.8%and 1.8%,respectively.展开更多
As the research of knowledge graph(KG)is deepened and widely used,knowledge graph com-pletion(KGC)has attracted more and more attentions from researchers,especially in scenarios of in-telligent search,social networks ...As the research of knowledge graph(KG)is deepened and widely used,knowledge graph com-pletion(KGC)has attracted more and more attentions from researchers,especially in scenarios of in-telligent search,social networks and deep question and answer(Q&A).Current research mainly fo-cuses on the completion of static knowledge graphs,and the temporal information in temporal knowl-edge graphs(TKGs)is ignored.However,the temporal information is definitely very helpful for the completion.Note that existing researches on temporal knowledge graph completion are difficult to process temporal information and to integrate entities,relations and time well.In this work,a rotation and scaling(RotatS)model is proposed,which learns rotation and scaling transformations from head entity embedding to tail entity embedding in 3D spaces to capture the information of time and rela-tions in the temporal knowledge graph.The performance of the proposed RotatS model have been evaluated by comparison with several baselines under similar experimental conditions and space com-plexity on four typical knowl good graph completion datasets publicly available online.The study shows that RotatS can achieve good results in terms of prediction accuracy.展开更多
基金Supported by the National Natural Science Foundation of China(No.62001313)the Liaoning Professional Talent Protect(No.XLYC2203046)the Shenyang Municipal Medical Engineering Cross Research Foundation of China(No.22-321-32-09).
文摘Medical visual question answering(MedVQA)aims to enhance diagnostic confidence and deepen patientsunderstanding of their health conditions.While the Transformer architecture is widely used in multimodal fields,its application in MedVQA requires further enhancement.A critical limitation of contemporary MedVQA systems lies in the inability to integrate lifelong knowledge with specific patient data to generate human-like responses.Existing Transformer-based MedVQA models require enhancing their capabitities for interpreting answers through the applications of medical image knowledge.The introduction of the medical knowledge graph visual language transformer(MKGViLT),designed for joint medical knowledge graphs(KGs),addresses this challenge.MKGViLT incorporates an enhanced Transformer structure to effectively extract features and combine modalities for MedVQA tasks.The MKGViLT model delivers answers based on richer background knowledge,thereby enhancing performance.The efficacy of MKGViLT is evaluated using the SLAKE and P-VQA datasets.Experimental results show that MKGViLT surpasses the most advanced methods on the SLAKE dataset.
基金supported by the Young Scientists Fund of the National Natural Science Foundation of China(62303491)the Major Program of Xiangjiang Laboratory(22XJ01005)+1 种基金the Science and Technology Innovation Program of Hunan Province(2024RC1007)the Natural Science Foundation of Hunan Province(2025JJ10007)。
文摘Temporal alignment of multisensor time series(MTS)is a critical prerequisite for accurate modeling and optimal control in subsequent data-driven applications.Nevertheless,many approaches frequently neglect to consider the complex interdependencies between different sensors in MTS,and temporal alignment in many methods is typically treated as an isolated task disconnected from the downstream objectives,leading to unsatisfactory performances in follow-up applications.To address these challenges,this paper proposes a novel knowledge graph(KG)-guided iterative-updating graph neural network(GNN)for time-delay estimation(TDE)in MTS.Initially,a domain-specific KG is constructed from domain mechanism knowledge,providing a foundation for GNN's initialization.Next,capitalizing on the inherent structure of the graph topology,a GNN-based TDE method is developed.Then,a customized loss function is constructed,which synthesizes both the performances of downstream tasks and graph-based constraints.Moreover,an innovative algorithm for GNN structure learning and iterative-updating is proposed to renovate the graph structure further.Finally,experimental results across various regression and classification tasks on numerical simulation,public datasets,and the real blast furnace ironmaking dataset demonstrate that the proposed method can achieve accurate temporal alignment of MTS.
基金supported by the National Natural Science Foundation of China(72101263).
文摘Tracking and analyzing data from research projects is critical for understanding research trends and supporting the development of science and technology strategies.However,the data from these projects is often complex and inadequate,making it challenging for researchers to conduct in-depth data mining to improve policies or management.To address this problem,this paper adopts a top-down approach to construct a knowledge graph(KG)for research projects.Firstly,we construct an integrated ontology by referring to the metamodel of various architectures,which is called the meta-model integration conceptual reference model.Subsequently,we use the dependency parsing method to extract knowledge from unstructured textual data and use the entity alignment method based on weakly supervised learning to classify the extracted entities,completing the construction of the KG for the research projects.In addition,a knowledge inference model based on representation learning is employed to achieve knowledge completion and improve the KG.Finally,experiments are conducted on the KG for research projects and the results demonstrate the effectiveness of the proposed method in enriching incomplete data within the KG.
文摘现有的检索增强生成(RAG)问答系统在特定领域应用时,存在检索路径单一、用户潜在意图覆盖不足和召回文段质量低导致的系统回答准确性低与不全面的问题。因此,提出一种双阶段优化方法——预回答与召回过滤(PARF)。首先,通过结合领域知识图谱与提示工程技术,引导大语言模型(LLM)生成预回答,构建“原始查询→预回答→相关文段”的多向检索路径,从而扩展原始查询的语义空间;其次,利用BERT(Bidirectional Encoder Representations from Transformers)模型对召回文段进行相关性评分与过滤,实现检索与生成阶段的协同优化,提升有效信息的密度。实验结果表明,相较于基线方法DPR-LLM(Dense Passage Retrieval with LLM)构建的RAG问答系统,PARF方法构建的RAG问答系统的一致性指标F1和ROUGE-L(Recall-Oriented Understudy for Gisting Evaluation-L)在轨道交通问答数据集上分别提升19.8和41.5个百分点,在医药问答数据集上分别提升16.1和17.6个百分点,效果指标正确率分别提升10.2和8.8个百分点。
文摘随着大语言模型(Large Language Model,简称LLM)技术的飞速发展,衍生出了对文档的检索增强生成(Retrieval-Augmented Generation,简称RAG)方法,但对于特定领域内多样化异构文档,传统RAG方法仍难以运用。面向对知识推理准确性有苛刻要求的特定领域(如军事、医疗等),本文针对远距离知识无法关联、答案不完整、特殊领域推理能力匮乏等问题,提出了基于远亲知识联结推理的异构文档问答增强方法(简称KiRAG),该方法通过解析异构文档,抽取、构建领域知识图谱,根据问题在图谱中推理并动态召回文档分片,使LLM理解问题和生成答案时,具备完备的信息视野,以确保最终答案的完整性、合理性及准确性。实验验证表明,该方法能够有效解决军事领域异构文档的知识问答运用问题,并具备在其他相似领域泛化运用的潜力。
基金supported in part by the Beijing Natural Science Foundation under Grants L211020 and M21032in part by the National Natural Science Foundation of China under Grants U1836106 and 62271045in part by the Scientific and Technological Innovation Foundation of Foshan under Grants BK21BF001 and BK20BF010。
文摘Knowledge graph(KG)serves as a specialized semantic network that encapsulates intricate relationships among real-world entities within a structured framework.This framework facilitates a transformation in information retrieval,transitioning it from mere string matching to far more sophisticated entity matching.In this transformative process,the advancement of artificial intelligence and intelligent information services is invigorated.Meanwhile,the role ofmachine learningmethod in the construction of KG is important,and these techniques have already achieved initial success.This article embarks on a comprehensive journey through the last strides in the field of KG via machine learning.With a profound amalgamation of cutting-edge research in machine learning,this article undertakes a systematical exploration of KG construction methods in three distinct phases:entity learning,ontology learning,and knowledge reasoning.Especially,a meticulous dissection of machine learningdriven algorithms is conducted,spotlighting their contributions to critical facets such as entity extraction,relation extraction,entity linking,and link prediction.Moreover,this article also provides an analysis of the unresolved challenges and emerging trajectories that beckon within the expansive application of machine learning-fueled,large-scale KG construction.
基金Supported by the National Natural Science Foundation of China(No.61876144).
文摘Knowledge graph(KG) link prediction aims to address the problem of missing multiple valid triples in KGs. Existing approaches either struggle to efficiently model the message passing process of multi-hop paths or lack transparency of model prediction principles. In this paper,a new graph convolutional network path semantic-aware graph convolution network(PSGCN) is proposed to achieve modeling the semantic information of multi-hop paths. PSGCN first uses a random walk strategy to obtain all-hop paths in KGs,then captures the semantics of the paths by Word2Sec and long shortterm memory(LSTM) models,and finally converts them into a potential representation for the graph convolution network(GCN) messaging process. PSGCN combines path-based inference methods and graph neural networks to achieve better interpretability and scalability. In addition,to ensure the robustness of the model,the value of the path thresholdKis experimented on the FB15K-237 and WN18RR datasets,and the final results prove the effectiveness of the model.
基金supported by National Key R&D Program of China(2022QY2000-02).
文摘Accurately recommending candidate news to users is a basic challenge of personalized news recommendation systems.Traditional methods are usually difficult to learn and acquire complex semantic information in news texts,resulting in unsatisfactory recommendation results.Besides,these traditional methods are more friendly to active users with rich historical behaviors.However,they can not effectively solve the long tail problem of inactive users.To address these issues,this research presents a novel general framework that combines Large Language Models(LLM)and Knowledge Graphs(KG)into traditional methods.To learn the contextual information of news text,we use LLMs’powerful text understanding ability to generate news representations with rich semantic information,and then,the generated news representations are used to enhance the news encoding in traditional methods.In addition,multi-hops relationship of news entities is mined and the structural information of news is encoded using KG,thus alleviating the challenge of long-tail distribution.Experimental results demonstrate that compared with various traditional models,on evaluation indicators such as AUC,MRR,nDCG@5 and nDCG@10,the framework significantly improves the recommendation performance.The successful integration of LLM and KG in our framework has established a feasible way for achieving more accurate personalized news recommendation.Our code is available at https://github.com/Xuan-ZW/LKPNR.
基金Supported by the Henan Province Science and Technology Department Foundation(No.202102310237,192102210133,202102310295)the Doctoral Research Fund of Zhengzhou University of Light Industry(No.2018BSJJ039)the Internet Medical and Health Service Henan Collaborative Innovation Center Open Project Fund(No.IH2019006).
文摘Knowledge graph(KG)conflict resolution is to solve knowledge conflicts problem in the construction of KG.Aiming at the problem of KG conflict resolution,a KG conflict resolution algorithm NGDcrm is proposed,which is a numeric graph dependency-based conflict resolution method.NGDcrm utilizes the dependency graph to perform arithmetic calculation and predicate comparison of numerical entity knowledge in the KG.NGDcrm first uses a parallel segmentation method to segment the KG;then,it extracts the features of the KG according to KG embedding;finally,it uses numerical graph dependencies to detect and correct the wrong facts in the KG based on the extracted features.The experimental results on real data show that NGDcrm is better than the state-of-the-art knowledge conflict resolution method.Among them,the AUC value of NGDcrm on the DBpedia dataset is 15.4%higher than the state-of-the-art method.
基金supported by the Sichuan Science and Technology Program under Grants No.2022YFQ0052 and No.2021YFQ0009.
文摘At present,knowledge embedding methods are widely used in the field of knowledge graph(KG)reasoning,and have been successfully applied to those with large entities and relationships.However,in research and production environments,there are a large number of KGs with a small number of entities and relations,which are called sparse KGs.Limited by the performance of knowledge extraction methods or some other reasons(some common-sense information does not appear in the natural corpus),the relation between entities is often incomplete.To solve this problem,a method of the graph neural network and information enhancement is proposed.The improved method increases the mean reciprocal rank(MRR)and Hit@3 by 1.6%and 1.7%,respectively,when the sparsity of the FB15K-237 dataset is 10%.When the sparsity is 50%,the evaluation indexes MRR and Hit@10 are increased by 0.8%and 1.8%,respectively.
基金the National Natural Science Foundation of China(No.6187022153).
文摘As the research of knowledge graph(KG)is deepened and widely used,knowledge graph com-pletion(KGC)has attracted more and more attentions from researchers,especially in scenarios of in-telligent search,social networks and deep question and answer(Q&A).Current research mainly fo-cuses on the completion of static knowledge graphs,and the temporal information in temporal knowl-edge graphs(TKGs)is ignored.However,the temporal information is definitely very helpful for the completion.Note that existing researches on temporal knowledge graph completion are difficult to process temporal information and to integrate entities,relations and time well.In this work,a rotation and scaling(RotatS)model is proposed,which learns rotation and scaling transformations from head entity embedding to tail entity embedding in 3D spaces to capture the information of time and rela-tions in the temporal knowledge graph.The performance of the proposed RotatS model have been evaluated by comparison with several baselines under similar experimental conditions and space com-plexity on four typical knowl good graph completion datasets publicly available online.The study shows that RotatS can achieve good results in terms of prediction accuracy.
