Objective To improve the accuracy and professionalism of question-answering(QA)model in traditional Chinese medicine(TCM)lung cancer by integrating large language models with structured knowledge graphs using the know...Objective To improve the accuracy and professionalism of question-answering(QA)model in traditional Chinese medicine(TCM)lung cancer by integrating large language models with structured knowledge graphs using the knowledge graph(KG)to text-enhanced retrievalaugmented generation(KG2TRAG)method.Methods The TCM lung cancer model(TCMLCM)was constructed by fine-tuning Chat-GLM2-6B on the specialized datasets Tianchi TCM,HuangDi,and ShenNong-TCM-Dataset,as well as a TCM lung cancer KG.The KG2TRAG method was applied to enhance the knowledge retrieval,which can convert KG triples into natural language text via ChatGPT-aided linearization,leveraging large language models(LLMs)for context-aware reasoning.For a comprehensive comparison,MedicalGPT,HuatuoGPT,and BenTsao were selected as the baseline models.Performance was evaluated using bilingual evaluation understudy(BLEU),recall-oriented understudy for gisting evaluation(ROUGE),accuracy,and the domain-specific TCM-LCEval metrics,with validation from TCM oncology experts assessing answer accuracy,professionalism,and usability.Results The TCMLCM model achieved the optimal performance across all metrics,including a BLEU score of 32.15%,ROUGE-L of 59.08%,and an accuracy rate of 79.68%.Notably,in the TCM-LCEval assessment specific to the field of TCM,its performance was 3%−12%higher than that of the baseline model.Expert evaluations highlighted superior performance in accuracy and professionalism.Conclusion TCMLCM can provide an innovative solution for TCM lung cancer QA,demonstrating the feasibility of integrating structured KGs with LLMs.This work advances intelligent TCM healthcare tools and lays a foundation for future AI-driven applications in traditional medicine.展开更多
Question-answering(QA)models find answers to a given question.The necessity of automatically finding answers is increasing because it is very important and challenging from the large-scale QA data sets.In this paper,w...Question-answering(QA)models find answers to a given question.The necessity of automatically finding answers is increasing because it is very important and challenging from the large-scale QA data sets.In this paper,we deal with the QA pair matching approach in QA models,which finds the most relevant question and its recommended answer for a given question.Existing studies for the approach performed on the entire dataset or datasets within a category that the question writer manually specifies.In contrast,we aim to automatically find the category to which the question belongs by employing the text classification model and to find the answer corresponding to the question within the category.Due to the text classification model,we can effectively reduce the search space for finding the answers to a given question.Therefore,the proposed model improves the accuracy of the QA matching model and significantly reduces the model inference time.Furthermore,to improve the performance of finding similar sentences in each category,we present an ensemble embedding model for sentences,improving the performance compared to the individual embedding models.Using real-world QA data sets,we evaluate the performance of the proposed QA matching model.As a result,the accuracy of our final ensemble embedding model based on the text classification model is 81.18%,which outperforms the existing models by 9.81%∼14.16%point.Moreover,in terms of the model inference speed,our model is faster than the existing models by 2.61∼5.07 times due to the effective reduction of search spaces by the text classification model.展开更多
This paper compares 12 representative Chinese and English online questionanswering communities(Q&A communities) based on their basic functions, interactive modes, and customized services. An empirical experiment f...This paper compares 12 representative Chinese and English online questionanswering communities(Q&A communities) based on their basic functions, interactive modes, and customized services. An empirical experiment from a comparative perspective was also conducted on them by using 12 questions representing for four types of questions,which are assigned evenly to three different subject fields so as to examine the task performance of these 12 selected online Q&A communities. Our goal was to evaluate those online Q&A communities in terms of their quality and efficiency for answering questions posed to them. It was hoped that our empirical research would yield greater understanding and insights to the working intricacy of these online Q&A communities and hence their possible further improvement.展开更多
Traditional Chinese text retrieval systems return a ranked list of documentsin response to a user''s request. While a ranked list of documents may be an appropriate response forthe user, frequently it is not. ...Traditional Chinese text retrieval systems return a ranked list of documentsin response to a user''s request. While a ranked list of documents may be an appropriate response forthe user, frequently it is not. Usually it would be better for the system to provide the answeritself instead of requiring the user to search for the answer in a set of documents. Since Chinesetext retrieval has just been developed lately, and due to various specific characteristics ofChinese language, the approaches to its retrieval are quite different from those studies andresearches proposed to deal with Western language. Thus, an architecture that augments existingsearch engines is developed to support Chinese natural language question answering. In this paper anew approach to building Chinese question-answering system is described, which is thegeneral-purpose, fully-automated Chinese quest ion-answering system available on the web. In theapproach, we attempt to represent Chinese text by its characteristics, and try to convert theChinese text into ERE (E: entity, R: relation) relation data lists, and then to answer the questionthrough ERE relation model. The system performs quite well giving the simplicity of the techniquesbeing utilized. Experimental results show that question-answering accuracy can be greatly improvedby analyzing more and more matching ERE relation data lists. Simple ERE relation data extractiontechniques work well in our system making it efficient to use with many backend retrieval engines.展开更多
Question-answering systems provide short answers with the use of available information.The implementation mechanism for a question answering system is presented in this paper and is based on concepts and statistics.Th...Question-answering systems provide short answers with the use of available information.The implementation mechanism for a question answering system is presented in this paper and is based on concepts and statistics.The system determines the question and focuses on the answer types,making different conceptual expansions for different questions.It applies the latent semantic indexing(LSI)method to retrieve relevant passages.It uses matching algorithms to find a match between questions and sentences stored in a database.It also extracts answers from a frequently asked questions(FAQ)database by finding matching or similar sentences.The answering ability of the system has been improved with the use of LSI and FAQ.The question-answering system introduced in Chinese universities is a developed and proven system capable of precise results.展开更多
大模型时代,自动问答系统呈现出诸多新的特征。通过文献阅读和梳理,对自动问答系统特征和评测体系进行总结与归纳,从问答模型推理训练的训练数据、预训练框架、模型后处理、模型高效微调等阶段,对比大模型发展初期“追求数据和参数规模...大模型时代,自动问答系统呈现出诸多新的特征。通过文献阅读和梳理,对自动问答系统特征和评测体系进行总结与归纳,从问答模型推理训练的训练数据、预训练框架、模型后处理、模型高效微调等阶段,对比大模型发展初期“追求数据和参数规模”的训练方法和如今“注重数据和模型效率”之间的差异,系统分析基于大模型的自动问答系统新的特征。总结当前各种类型的自动问答大模型评测体系,并详细梳理自动化评价体系HELM(holistic evaluation of language model)在自动问答任务上的数据集、评价指标和量化计算方法。未来基于大模型的自动问答系统研究将会围绕多模态融合、高安全性、高可解释性、低资源消耗,以及结合大模型和自动化的综合评价体系这几个方面进一步拓展与深化。展开更多
基金Postgraduate Research&Practice Innovation Program of Jiangsu Province(KYCX24_2145).
文摘Objective To improve the accuracy and professionalism of question-answering(QA)model in traditional Chinese medicine(TCM)lung cancer by integrating large language models with structured knowledge graphs using the knowledge graph(KG)to text-enhanced retrievalaugmented generation(KG2TRAG)method.Methods The TCM lung cancer model(TCMLCM)was constructed by fine-tuning Chat-GLM2-6B on the specialized datasets Tianchi TCM,HuangDi,and ShenNong-TCM-Dataset,as well as a TCM lung cancer KG.The KG2TRAG method was applied to enhance the knowledge retrieval,which can convert KG triples into natural language text via ChatGPT-aided linearization,leveraging large language models(LLMs)for context-aware reasoning.For a comprehensive comparison,MedicalGPT,HuatuoGPT,and BenTsao were selected as the baseline models.Performance was evaluated using bilingual evaluation understudy(BLEU),recall-oriented understudy for gisting evaluation(ROUGE),accuracy,and the domain-specific TCM-LCEval metrics,with validation from TCM oncology experts assessing answer accuracy,professionalism,and usability.Results The TCMLCM model achieved the optimal performance across all metrics,including a BLEU score of 32.15%,ROUGE-L of 59.08%,and an accuracy rate of 79.68%.Notably,in the TCM-LCEval assessment specific to the field of TCM,its performance was 3%−12%higher than that of the baseline model.Expert evaluations highlighted superior performance in accuracy and professionalism.Conclusion TCMLCM can provide an innovative solution for TCM lung cancer QA,demonstrating the feasibility of integrating structured KGs with LLMs.This work advances intelligent TCM healthcare tools and lays a foundation for future AI-driven applications in traditional medicine.
基金This work was supported by the National Research Foundation of Korea(NRF)grant funded by the Korea government(MSIT)(No.2022R1F1A1067008)by the Basic Science Research Program through the National Research Foundation of Korea(NRF)funded by the Ministry of Education(No.2019R1A6A1A03032119).
文摘Question-answering(QA)models find answers to a given question.The necessity of automatically finding answers is increasing because it is very important and challenging from the large-scale QA data sets.In this paper,we deal with the QA pair matching approach in QA models,which finds the most relevant question and its recommended answer for a given question.Existing studies for the approach performed on the entire dataset or datasets within a category that the question writer manually specifies.In contrast,we aim to automatically find the category to which the question belongs by employing the text classification model and to find the answer corresponding to the question within the category.Due to the text classification model,we can effectively reduce the search space for finding the answers to a given question.Therefore,the proposed model improves the accuracy of the QA matching model and significantly reduces the model inference time.Furthermore,to improve the performance of finding similar sentences in each category,we present an ensemble embedding model for sentences,improving the performance compared to the individual embedding models.Using real-world QA data sets,we evaluate the performance of the proposed QA matching model.As a result,the accuracy of our final ensemble embedding model based on the text classification model is 81.18%,which outperforms the existing models by 9.81%∼14.16%point.Moreover,in terms of the model inference speed,our model is faster than the existing models by 2.61∼5.07 times due to the effective reduction of search spaces by the text classification model.
基金jointly supported by Wuhan International Science and Technology Cooperation Fund(Grant No.201070934337)the 3rd Special Award of China Postdoctoral Science Foundation(Grant No.201003497)National Science Foundation of USA(Grant No.NSF/IIS-1052773)
文摘This paper compares 12 representative Chinese and English online questionanswering communities(Q&A communities) based on their basic functions, interactive modes, and customized services. An empirical experiment from a comparative perspective was also conducted on them by using 12 questions representing for four types of questions,which are assigned evenly to three different subject fields so as to examine the task performance of these 12 selected online Q&A communities. Our goal was to evaluate those online Q&A communities in terms of their quality and efficiency for answering questions posed to them. It was hoped that our empirical research would yield greater understanding and insights to the working intricacy of these online Q&A communities and hence their possible further improvement.
文摘Traditional Chinese text retrieval systems return a ranked list of documentsin response to a user''s request. While a ranked list of documents may be an appropriate response forthe user, frequently it is not. Usually it would be better for the system to provide the answeritself instead of requiring the user to search for the answer in a set of documents. Since Chinesetext retrieval has just been developed lately, and due to various specific characteristics ofChinese language, the approaches to its retrieval are quite different from those studies andresearches proposed to deal with Western language. Thus, an architecture that augments existingsearch engines is developed to support Chinese natural language question answering. In this paper anew approach to building Chinese question-answering system is described, which is thegeneral-purpose, fully-automated Chinese quest ion-answering system available on the web. In theapproach, we attempt to represent Chinese text by its characteristics, and try to convert theChinese text into ERE (E: entity, R: relation) relation data lists, and then to answer the questionthrough ERE relation model. The system performs quite well giving the simplicity of the techniquesbeing utilized. Experimental results show that question-answering accuracy can be greatly improvedby analyzing more and more matching ERE relation data lists. Simple ERE relation data extractiontechniques work well in our system making it efficient to use with many backend retrieval engines.
基金supported by the National Natural Science Foundation of China(Grant No.60373095).
文摘Question-answering systems provide short answers with the use of available information.The implementation mechanism for a question answering system is presented in this paper and is based on concepts and statistics.The system determines the question and focuses on the answer types,making different conceptual expansions for different questions.It applies the latent semantic indexing(LSI)method to retrieve relevant passages.It uses matching algorithms to find a match between questions and sentences stored in a database.It also extracts answers from a frequently asked questions(FAQ)database by finding matching or similar sentences.The answering ability of the system has been improved with the use of LSI and FAQ.The question-answering system introduced in Chinese universities is a developed and proven system capable of precise results.
文摘表格作为一种重要的数据载体,能以紧凑的形式承载大量高价值信息,被广泛应用于经济、金融及科研等领域。表格问答(Table Question Answering,TableQA)旨在针对用自然语言描述的问题,从表格数据中自动进行推理并生成相应的答案,是自然语言处理与数据分析交叉领域的重要研究方向。与传统的文本问答和知识库问答相比,表格问答不仅需要理解自然语言,还须解析表格的二维结构,并处理数值计算与复杂逻辑推理,因此面临更大的挑战。近年来,随着多样化数据集的持续构建,表格问答技术不断取得进展。其研究范式经历了从基于规则与模板的方法,到统计学习与神经网络模型的应用,再到预训练语言模型的引入,整体性能不断提升。尤其是近年来大语言模型(Large Language Models,LLMs)的兴起,进一步推动了表格问答进入新的发展阶段。凭借卓越的跨任务泛化能力与推理能力,大语言模型加速了新型研究范式的形成与发展,为方法创新提供了有力支撑。文中系统梳理了表格问答技术的演进脉络与代表性方法,重点总结了大语言模型驱动下的最新研究进展,概述了当前研究面临的关键挑战,并对未来发展趋势进行了展望。
文摘大模型时代,自动问答系统呈现出诸多新的特征。通过文献阅读和梳理,对自动问答系统特征和评测体系进行总结与归纳,从问答模型推理训练的训练数据、预训练框架、模型后处理、模型高效微调等阶段,对比大模型发展初期“追求数据和参数规模”的训练方法和如今“注重数据和模型效率”之间的差异,系统分析基于大模型的自动问答系统新的特征。总结当前各种类型的自动问答大模型评测体系,并详细梳理自动化评价体系HELM(holistic evaluation of language model)在自动问答任务上的数据集、评价指标和量化计算方法。未来基于大模型的自动问答系统研究将会围绕多模态融合、高安全性、高可解释性、低资源消耗,以及结合大模型和自动化的综合评价体系这几个方面进一步拓展与深化。
