Objective To improve the accuracy and professionalism of question-answering(QA)model in traditional Chinese medicine(TCM)lung cancer by integrating large language models with structured knowledge graphs using the know...Objective To improve the accuracy and professionalism of question-answering(QA)model in traditional Chinese medicine(TCM)lung cancer by integrating large language models with structured knowledge graphs using the knowledge graph(KG)to text-enhanced retrievalaugmented generation(KG2TRAG)method.Methods The TCM lung cancer model(TCMLCM)was constructed by fine-tuning Chat-GLM2-6B on the specialized datasets Tianchi TCM,HuangDi,and ShenNong-TCM-Dataset,as well as a TCM lung cancer KG.The KG2TRAG method was applied to enhance the knowledge retrieval,which can convert KG triples into natural language text via ChatGPT-aided linearization,leveraging large language models(LLMs)for context-aware reasoning.For a comprehensive comparison,MedicalGPT,HuatuoGPT,and BenTsao were selected as the baseline models.Performance was evaluated using bilingual evaluation understudy(BLEU),recall-oriented understudy for gisting evaluation(ROUGE),accuracy,and the domain-specific TCM-LCEval metrics,with validation from TCM oncology experts assessing answer accuracy,professionalism,and usability.Results The TCMLCM model achieved the optimal performance across all metrics,including a BLEU score of 32.15%,ROUGE-L of 59.08%,and an accuracy rate of 79.68%.Notably,in the TCM-LCEval assessment specific to the field of TCM,its performance was 3%−12%higher than that of the baseline model.Expert evaluations highlighted superior performance in accuracy and professionalism.Conclusion TCMLCM can provide an innovative solution for TCM lung cancer QA,demonstrating the feasibility of integrating structured KGs with LLMs.This work advances intelligent TCM healthcare tools and lays a foundation for future AI-driven applications in traditional medicine.展开更多
Question-answering(QA)models find answers to a given question.The necessity of automatically finding answers is increasing because it is very important and challenging from the large-scale QA data sets.In this paper,w...Question-answering(QA)models find answers to a given question.The necessity of automatically finding answers is increasing because it is very important and challenging from the large-scale QA data sets.In this paper,we deal with the QA pair matching approach in QA models,which finds the most relevant question and its recommended answer for a given question.Existing studies for the approach performed on the entire dataset or datasets within a category that the question writer manually specifies.In contrast,we aim to automatically find the category to which the question belongs by employing the text classification model and to find the answer corresponding to the question within the category.Due to the text classification model,we can effectively reduce the search space for finding the answers to a given question.Therefore,the proposed model improves the accuracy of the QA matching model and significantly reduces the model inference time.Furthermore,to improve the performance of finding similar sentences in each category,we present an ensemble embedding model for sentences,improving the performance compared to the individual embedding models.Using real-world QA data sets,we evaluate the performance of the proposed QA matching model.As a result,the accuracy of our final ensemble embedding model based on the text classification model is 81.18%,which outperforms the existing models by 9.81%∼14.16%point.Moreover,in terms of the model inference speed,our model is faster than the existing models by 2.61∼5.07 times due to the effective reduction of search spaces by the text classification model.展开更多
This paper compares 12 representative Chinese and English online questionanswering communities(Q&A communities) based on their basic functions, interactive modes, and customized services. An empirical experiment f...This paper compares 12 representative Chinese and English online questionanswering communities(Q&A communities) based on their basic functions, interactive modes, and customized services. An empirical experiment from a comparative perspective was also conducted on them by using 12 questions representing for four types of questions,which are assigned evenly to three different subject fields so as to examine the task performance of these 12 selected online Q&A communities. Our goal was to evaluate those online Q&A communities in terms of their quality and efficiency for answering questions posed to them. It was hoped that our empirical research would yield greater understanding and insights to the working intricacy of these online Q&A communities and hence their possible further improvement.展开更多
Traditional Chinese text retrieval systems return a ranked list of documentsin response to a user''s request. While a ranked list of documents may be an appropriate response forthe user, frequently it is not. ...Traditional Chinese text retrieval systems return a ranked list of documentsin response to a user''s request. While a ranked list of documents may be an appropriate response forthe user, frequently it is not. Usually it would be better for the system to provide the answeritself instead of requiring the user to search for the answer in a set of documents. Since Chinesetext retrieval has just been developed lately, and due to various specific characteristics ofChinese language, the approaches to its retrieval are quite different from those studies andresearches proposed to deal with Western language. Thus, an architecture that augments existingsearch engines is developed to support Chinese natural language question answering. In this paper anew approach to building Chinese question-answering system is described, which is thegeneral-purpose, fully-automated Chinese quest ion-answering system available on the web. In theapproach, we attempt to represent Chinese text by its characteristics, and try to convert theChinese text into ERE (E: entity, R: relation) relation data lists, and then to answer the questionthrough ERE relation model. The system performs quite well giving the simplicity of the techniquesbeing utilized. Experimental results show that question-answering accuracy can be greatly improvedby analyzing more and more matching ERE relation data lists. Simple ERE relation data extractiontechniques work well in our system making it efficient to use with many backend retrieval engines.展开更多
Question-answering systems provide short answers with the use of available information.The implementation mechanism for a question answering system is presented in this paper and is based on concepts and statistics.Th...Question-answering systems provide short answers with the use of available information.The implementation mechanism for a question answering system is presented in this paper and is based on concepts and statistics.The system determines the question and focuses on the answer types,making different conceptual expansions for different questions.It applies the latent semantic indexing(LSI)method to retrieve relevant passages.It uses matching algorithms to find a match between questions and sentences stored in a database.It also extracts answers from a frequently asked questions(FAQ)database by finding matching or similar sentences.The answering ability of the system has been improved with the use of LSI and FAQ.The question-answering system introduced in Chinese universities is a developed and proven system capable of precise results.展开更多
检索增强生成(retrieval-augmented generation,RAG)技术因具有为大语言模型(large language model,LLM)提供模型外知识的能力而受到人们的关注,然而绝大多数方法都难以同时兼顾局部的细节知识和原文中不连续的多跳知识。针对上述问题,...检索增强生成(retrieval-augmented generation,RAG)技术因具有为大语言模型(large language model,LLM)提供模型外知识的能力而受到人们的关注,然而绝大多数方法都难以同时兼顾局部的细节知识和原文中不连续的多跳知识。针对上述问题,提出基于聚类重组和预解析的检索增强生成方法。在索引阶段,首先通过聚类算法将不连续的相关知识组合成新分块,以提高多跳知识的检索能力;然后基于提示工程对各知识分块进行预解析生成更细粒度的新分块,以提高检索阶段的召回率。在检索阶段,将召回的所有新分块还原为原文分块,并连同查询语句输入给大语言模型以得到最终答案。在数据集QuALITY上对所提出的方法进行了评估,通过消融实验和开源基线对比实验验证了方法的有效性,并在公开的评测排行榜上取得了最佳效果。本文分析结果可为RAG的索引和检索技术提供参考。展开更多
基金Postgraduate Research&Practice Innovation Program of Jiangsu Province(KYCX24_2145).
文摘Objective To improve the accuracy and professionalism of question-answering(QA)model in traditional Chinese medicine(TCM)lung cancer by integrating large language models with structured knowledge graphs using the knowledge graph(KG)to text-enhanced retrievalaugmented generation(KG2TRAG)method.Methods The TCM lung cancer model(TCMLCM)was constructed by fine-tuning Chat-GLM2-6B on the specialized datasets Tianchi TCM,HuangDi,and ShenNong-TCM-Dataset,as well as a TCM lung cancer KG.The KG2TRAG method was applied to enhance the knowledge retrieval,which can convert KG triples into natural language text via ChatGPT-aided linearization,leveraging large language models(LLMs)for context-aware reasoning.For a comprehensive comparison,MedicalGPT,HuatuoGPT,and BenTsao were selected as the baseline models.Performance was evaluated using bilingual evaluation understudy(BLEU),recall-oriented understudy for gisting evaluation(ROUGE),accuracy,and the domain-specific TCM-LCEval metrics,with validation from TCM oncology experts assessing answer accuracy,professionalism,and usability.Results The TCMLCM model achieved the optimal performance across all metrics,including a BLEU score of 32.15%,ROUGE-L of 59.08%,and an accuracy rate of 79.68%.Notably,in the TCM-LCEval assessment specific to the field of TCM,its performance was 3%−12%higher than that of the baseline model.Expert evaluations highlighted superior performance in accuracy and professionalism.Conclusion TCMLCM can provide an innovative solution for TCM lung cancer QA,demonstrating the feasibility of integrating structured KGs with LLMs.This work advances intelligent TCM healthcare tools and lays a foundation for future AI-driven applications in traditional medicine.
基金This work was supported by the National Research Foundation of Korea(NRF)grant funded by the Korea government(MSIT)(No.2022R1F1A1067008)by the Basic Science Research Program through the National Research Foundation of Korea(NRF)funded by the Ministry of Education(No.2019R1A6A1A03032119).
文摘Question-answering(QA)models find answers to a given question.The necessity of automatically finding answers is increasing because it is very important and challenging from the large-scale QA data sets.In this paper,we deal with the QA pair matching approach in QA models,which finds the most relevant question and its recommended answer for a given question.Existing studies for the approach performed on the entire dataset or datasets within a category that the question writer manually specifies.In contrast,we aim to automatically find the category to which the question belongs by employing the text classification model and to find the answer corresponding to the question within the category.Due to the text classification model,we can effectively reduce the search space for finding the answers to a given question.Therefore,the proposed model improves the accuracy of the QA matching model and significantly reduces the model inference time.Furthermore,to improve the performance of finding similar sentences in each category,we present an ensemble embedding model for sentences,improving the performance compared to the individual embedding models.Using real-world QA data sets,we evaluate the performance of the proposed QA matching model.As a result,the accuracy of our final ensemble embedding model based on the text classification model is 81.18%,which outperforms the existing models by 9.81%∼14.16%point.Moreover,in terms of the model inference speed,our model is faster than the existing models by 2.61∼5.07 times due to the effective reduction of search spaces by the text classification model.
基金jointly supported by Wuhan International Science and Technology Cooperation Fund(Grant No.201070934337)the 3rd Special Award of China Postdoctoral Science Foundation(Grant No.201003497)National Science Foundation of USA(Grant No.NSF/IIS-1052773)
文摘This paper compares 12 representative Chinese and English online questionanswering communities(Q&A communities) based on their basic functions, interactive modes, and customized services. An empirical experiment from a comparative perspective was also conducted on them by using 12 questions representing for four types of questions,which are assigned evenly to three different subject fields so as to examine the task performance of these 12 selected online Q&A communities. Our goal was to evaluate those online Q&A communities in terms of their quality and efficiency for answering questions posed to them. It was hoped that our empirical research would yield greater understanding and insights to the working intricacy of these online Q&A communities and hence their possible further improvement.
文摘Traditional Chinese text retrieval systems return a ranked list of documentsin response to a user''s request. While a ranked list of documents may be an appropriate response forthe user, frequently it is not. Usually it would be better for the system to provide the answeritself instead of requiring the user to search for the answer in a set of documents. Since Chinesetext retrieval has just been developed lately, and due to various specific characteristics ofChinese language, the approaches to its retrieval are quite different from those studies andresearches proposed to deal with Western language. Thus, an architecture that augments existingsearch engines is developed to support Chinese natural language question answering. In this paper anew approach to building Chinese question-answering system is described, which is thegeneral-purpose, fully-automated Chinese quest ion-answering system available on the web. In theapproach, we attempt to represent Chinese text by its characteristics, and try to convert theChinese text into ERE (E: entity, R: relation) relation data lists, and then to answer the questionthrough ERE relation model. The system performs quite well giving the simplicity of the techniquesbeing utilized. Experimental results show that question-answering accuracy can be greatly improvedby analyzing more and more matching ERE relation data lists. Simple ERE relation data extractiontechniques work well in our system making it efficient to use with many backend retrieval engines.
基金supported by the National Natural Science Foundation of China(Grant No.60373095).
文摘Question-answering systems provide short answers with the use of available information.The implementation mechanism for a question answering system is presented in this paper and is based on concepts and statistics.The system determines the question and focuses on the answer types,making different conceptual expansions for different questions.It applies the latent semantic indexing(LSI)method to retrieve relevant passages.It uses matching algorithms to find a match between questions and sentences stored in a database.It also extracts answers from a frequently asked questions(FAQ)database by finding matching or similar sentences.The answering ability of the system has been improved with the use of LSI and FAQ.The question-answering system introduced in Chinese universities is a developed and proven system capable of precise results.
文摘检索增强生成(retrieval-augmented generation,RAG)技术因具有为大语言模型(large language model,LLM)提供模型外知识的能力而受到人们的关注,然而绝大多数方法都难以同时兼顾局部的细节知识和原文中不连续的多跳知识。针对上述问题,提出基于聚类重组和预解析的检索增强生成方法。在索引阶段,首先通过聚类算法将不连续的相关知识组合成新分块,以提高多跳知识的检索能力;然后基于提示工程对各知识分块进行预解析生成更细粒度的新分块,以提高检索阶段的召回率。在检索阶段,将召回的所有新分块还原为原文分块,并连同查询语句输入给大语言模型以得到最终答案。在数据集QuALITY上对所提出的方法进行了评估,通过消融实验和开源基线对比实验验证了方法的有效性,并在公开的评测排行榜上取得了最佳效果。本文分析结果可为RAG的索引和检索技术提供参考。
