Diabetes poses a considerable global health challenge,with varying levels of diabetes knowledge among healthcare professionals,highlighting the importance of diabetes training.Large Language Models(LLMs)provide new in...Diabetes poses a considerable global health challenge,with varying levels of diabetes knowledge among healthcare professionals,highlighting the importance of diabetes training.Large Language Models(LLMs)provide new insights into diabetes training,but their performance in diabetes-related queries remains uncertain,especially outside the English language like Chinese.We first evaluated the performance of ten LLMs:ChatGPT-3.5,ChatGPT-4.0,Google Bard,LlaMA-7B,LlaMA2-7B,Baidu ERNIE Bot,Ali Tongyi Qianwen,MedGPT,HuatuoGPT,and Chinese LlaMA2-7B on diabetes-related queries,based on the Chinese National Certificate Examination for Primary Diabetes Care in China(NCE-CPDC)and the English Specialty Certificate Examination in Endocrinology and Diabetes of Membership of the Royal College of Physicians of the United Kingdom.Second,we assessed the training of primary care physicians(PCPs)without and with the assistance of ChatGPT-4.0 in the NCE-CPDC examination to ascertain the reliability of LLMs as medical assistants.We found that ChatGPT-4.0 outperformed other LLMs in the English examination,achieving a passing accuracy of 62.50%,which was significantly higher than that of Google Bard,LlaMA-7B,and LlaMA2-7B.For the NCE-CPFC examination,ChatGPT-4.0,Ali Tongyi Qianwen,Baidu ERNIE Bot,Google Bard,MedGPT,and ChatGPT-3.5 successfully passed,whereas LlaMA2-7B,HuatuoGPT,Chinese LLaMA2-7B,and LlaMA-7B failed.ChatGPT-4.0(84.82%)surpassed all PCPs and assisted most PCPs in the NCE-CPDC examination(improving by 1%–6.13%).In summary,LLMs demonstrated outstanding competence for diabetes-related questions in both the Chinese and English language,and hold great potential to assist future diabetes training for physicians globally.展开更多
The increasing prevalence of diabetes has become a global public health concern in the 21st century.In 2021,it was estimated that 537 million people had diabetes,and this number is projected to reach 643 million by 20...The increasing prevalence of diabetes has become a global public health concern in the 21st century.In 2021,it was estimated that 537 million people had diabetes,and this number is projected to reach 643 million by 2030,and 783 million by 2045[1].Such a huge burden of diabetes brings great challenges in its prevention and management,including early diagnosis,timely interventions,and regular monitoring of risk factor control and complications screening.Continuous self-care support and patient empowerment can enhance clinical and psychobehavioural outcomes[2],although these require additional resources including manpower,infrastructure(hard and technology),and finances.The emergence of digital health technologies(DHTs),especially artificial intelligence(AI),may help address these obstacles and alleviate the burden of diabetes[3].Large language models(LLMs),a generative AI that can accept image and text inputs and produce text outputs,have shown promise in various aspects of medical care.展开更多
基金supported by the Noncommunicable Chronic Diseases-National Science and Technology Major Project(2023ZD0509202 and 2023ZD0509201)National Natural Science Foundation of China(62077037,8238810007,82022012,81870598,62272298 and 82388101)+4 种基金the National Key Research and Development Program of China(2022YFC2502800 and 2022YFC2407000)the Shanghai Municipal Key Clinical Specialty,Shanghai Research Center for Endocrine and Metabolic Diseases(2022ZZ01002)the Chinese Academy of Engineering(2022-XY-08)the Innovative Research Team of High-level Local Universities in Shanghai(SHSMUZDCX20212700)Beijing Natural Science Foundation(IS23096).
文摘Diabetes poses a considerable global health challenge,with varying levels of diabetes knowledge among healthcare professionals,highlighting the importance of diabetes training.Large Language Models(LLMs)provide new insights into diabetes training,but their performance in diabetes-related queries remains uncertain,especially outside the English language like Chinese.We first evaluated the performance of ten LLMs:ChatGPT-3.5,ChatGPT-4.0,Google Bard,LlaMA-7B,LlaMA2-7B,Baidu ERNIE Bot,Ali Tongyi Qianwen,MedGPT,HuatuoGPT,and Chinese LlaMA2-7B on diabetes-related queries,based on the Chinese National Certificate Examination for Primary Diabetes Care in China(NCE-CPDC)and the English Specialty Certificate Examination in Endocrinology and Diabetes of Membership of the Royal College of Physicians of the United Kingdom.Second,we assessed the training of primary care physicians(PCPs)without and with the assistance of ChatGPT-4.0 in the NCE-CPDC examination to ascertain the reliability of LLMs as medical assistants.We found that ChatGPT-4.0 outperformed other LLMs in the English examination,achieving a passing accuracy of 62.50%,which was significantly higher than that of Google Bard,LlaMA-7B,and LlaMA2-7B.For the NCE-CPFC examination,ChatGPT-4.0,Ali Tongyi Qianwen,Baidu ERNIE Bot,Google Bard,MedGPT,and ChatGPT-3.5 successfully passed,whereas LlaMA2-7B,HuatuoGPT,Chinese LLaMA2-7B,and LlaMA-7B failed.ChatGPT-4.0(84.82%)surpassed all PCPs and assisted most PCPs in the NCE-CPDC examination(improving by 1%–6.13%).In summary,LLMs demonstrated outstanding competence for diabetes-related questions in both the Chinese and English language,and hold great potential to assist future diabetes training for physicians globally.
基金supported by the National Key R&D Program of China(2022YFC2502800 and 2022YFC2407000)the National Natural Science Foundation of China(8238810007,82022012,81870598 and 62272298)+3 种基金the Shanghai Municipal Key Clinical SpecialtyShanghai Research Center for Endocrine and Metabolic Diseases(2022ZZ01002)the Chinese Academy of Engineering(2022-XY-08)the Innovative Research Team of High-level Local Universities in Shanghai(SHSMU-ZDCX20212700)。
文摘The increasing prevalence of diabetes has become a global public health concern in the 21st century.In 2021,it was estimated that 537 million people had diabetes,and this number is projected to reach 643 million by 2030,and 783 million by 2045[1].Such a huge burden of diabetes brings great challenges in its prevention and management,including early diagnosis,timely interventions,and regular monitoring of risk factor control and complications screening.Continuous self-care support and patient empowerment can enhance clinical and psychobehavioural outcomes[2],although these require additional resources including manpower,infrastructure(hard and technology),and finances.The emergence of digital health technologies(DHTs),especially artificial intelligence(AI),may help address these obstacles and alleviate the burden of diabetes[3].Large language models(LLMs),a generative AI that can accept image and text inputs and produce text outputs,have shown promise in various aspects of medical care.
