For decades,the central dogma of oncology has been that a cancer’s identity is inextricably linked to its anatomical origin.This principle underpins the entire diagnostic and therapeutic framework,from histology-base...For decades,the central dogma of oncology has been that a cancer’s identity is inextricably linked to its anatomical origin.This principle underpins the entire diagnostic and therapeutic framework,from histology-based classification to site-specific treatment guidelines.Yet,this framework catastrophically fails for a substantial population of patients diagnosed with cancer of unknown primary(CUP).These patients present metastatic disease,yet their primary tumors remain elusive despite exhaustive clinical workup1.CUP,accounting for 1%-3%of all cancer diagnoses,is an enigma with devastating consequences;the median overall survival is only 2-12 months2-4.The inability to pinpoint an origin forces clinicians to rely on broad-spectrum empirical chemotherapy,such as taxane-carboplatin regimens,which have limited efficacy and exclude patients from the promise of targeted therapies and clinical trials5.CUP is not only a diagnostic challenge but also an indictment of the siloed approach to understanding malignancy:this cancer highlights the limitations of origin-based diagnostic frameworks.However,the confluence of high-dimensional biological data and advanced artificial intelligence(AI)is now poised to address this long-standing diagnostic limitation and to herald a new era for not only CUP but also oncology as a whole(Figure 1).展开更多
Business Process Modelling(BPM)is essential for analyzing,improving,and automating the flow of information within organizations,but traditional approaches based on manual interpretation are slow,error-prone,and requir...Business Process Modelling(BPM)is essential for analyzing,improving,and automating the flow of information within organizations,but traditional approaches based on manual interpretation are slow,error-prone,and require a high level of expertise.This article proposes an innovative alternative solution that overcomes these limitations by automatically generating comprehensive Business Process Modelling and Notation(BPMN)diagrams solely from verbal descriptions of the processes to be modeled,utilizing Large Language Models(LLMs)and multimodal Artificial Intelligence(AI).Experimental results,based on video recordings of process explanations provided by an expert from an organization(in this case,the Commercial Courts of a public justice administration),demonstrate that the proposed methodology successfully enables the automatic generation of complete and accurate BPMN diagrams,leading to significant improvements in the speed,accuracy,and accessibility of process modeling.This research makes a substantial contribution to the field of business process modeling,as its methodology is groundbreaking in its use of LLMs and multimodal AI capabilities to handle different types of source material(text and video),combining several tools to minimize the number of queries and reduce the complexity of the prompts required for the automatic generation of successful BPMN diagrams.展开更多
Artificial intelligence(AI)is transforming the diagnostic landscape of malignant tumors in the urinary system,including prostate cancer,bladder cancer,and renal cell carcinoma(RCC).By integrating imaging,pathology,and...Artificial intelligence(AI)is transforming the diagnostic landscape of malignant tumors in the urinary system,including prostate cancer,bladder cancer,and renal cell carcinoma(RCC).By integrating imaging,pathology,and molecular data,AI enhances the precision and reproducibility of tumor detection,grading,and risk stratification.In prostate cancer,AI-assisted multiparametric Magnetic resonance imaging(MRI)and digital pathology systems improve lesion localization and Gleason scoring.For bladder cancer,deep learning-based cystoscopy and radiomics models from Computed tomography/magnetic resonance imaging(CT/MRI)enable real-time lesion segmentation and non-invasive biomarker prediction,such as Programmed Cell Death-Ligand 1(PD-L1)expression.In RCC,AI,combined with CT/MRI and multi-omics data,aids in subtype classification and prognostic prediction,supporting personalized therapy.However,despite these promising advances,challenges such as data standardization,model generalizability,interpretability,and regulatory compliance hinder AI’s clinical translation.This review outlines the current state of AI in urological cancer diagnosis and prognosis,its technological innovations,and the clinical challenges and opportunities that lie ahead.展开更多
Background:The integration of 7 Tesla(7T)magnetic resonance imaging(MRI)with advanced multimodal artificial intelligence(AI)models represents a promising frontier in neuroimaging.The superior spatial resolution of 7TM...Background:The integration of 7 Tesla(7T)magnetic resonance imaging(MRI)with advanced multimodal artificial intelligence(AI)models represents a promising frontier in neuroimaging.The superior spatial resolution of 7TMRI provides detailed visualizations of brain structure,which are crucial forunderstanding complex central nervous system diseases and tumors.Concurrently,the application of multimodal AI to medical images enables interactive imaging-based diagnostic conversation.Methods:In this paper,we systematically investigate the capacity and feasibility of applying the existing advanced multimodal AI model ChatGPT-4V to 7T MRI under the context of brain tumors.First,we test whether ChatGPT-4V has knowledge about 7T MRI,and whether it can differentiate 7T MRI from 3T MRI.In addition,we explore whether ChatGPT-4V can recognize different 7T MRI modalities and whether it can correctly offer diagnosis of tumors based on single or multiple modality 7T MRI.Results:ChatGPT-4V exhibited accuracy of 84.4%in 3T-vs-7T differentiation and accuracy of 78.9%in 7T modality recognition.Meanwhile,in a human evaluation with three clinical experts,ChatGPT obtained average scores of 9.27/20 in single modality-based diagnosis and 21.25/25 in multiple modality-based diagnosis.Our study indicates that single-modality diagnosis and the interpretability of diagnostic decisions in clinical practice should be enhanced when ChatGPT-4V is applied to 7T data.Conclusions:In general,our analysis suggests that such integration has promise as a tool to improve the workflow of diagnostics in neurology,with a potentially transformative impact in the fields of medical image analysis and patient management.展开更多
In recent years,multimodal agent AI(MAA)has emerged as a pivotal area of research,holding promise for transforming human-machine interaction.Agent AI systems,capable of perceiving and responding to inputs from multipl...In recent years,multimodal agent AI(MAA)has emerged as a pivotal area of research,holding promise for transforming human-machine interaction.Agent AI systems,capable of perceiving and responding to inputs from multiple modalities(e.g.,language,vision,audio),have demonstrated remarkable progress in understanding complex environments and executing intricate tasks.This survey comprehensively reviews the state-of-the-art developments in MAA and examines its fundamental concepts,key techniques,and applications across diverse domains.We first introduce the basics of agent AI and its multimodal interaction capabilities.We then delve into the core technologies that enable agents to perform task planning,decision-making,and multi-sensory fusion.Furthermore,we focus on exploring various applications of MAA in robotics,healthcare,gaming,and beyond.Additionally,we mainly focus on analyzing the challenges and limitations of current systems and propose promising research directions for future improvements,including human-AI collaboration,online learning method improvement.By reviewing existing work and highlighting open questions,this survey aims to provide a comprehensive roadmap for researchers and practitioners in the field of MAA.展开更多
基金supported by the National Natural Science Foundation of China(Grant Nos.32270688,31801117,and 82430107 to X.L.,and 32500589 to H.S.)the China Postdoctoral Science Foundation(Grant Nos.BX20240253 and 2024M762384 to H.S.)+1 种基金the Natural Science Foundation of Tianjin(Grant No.24JCQNJC01280 to H.S.)Tianjin Key Medical Discipline(Specialty)Construction Project(Grant No.TJYXZDXK-3-003A).
文摘For decades,the central dogma of oncology has been that a cancer’s identity is inextricably linked to its anatomical origin.This principle underpins the entire diagnostic and therapeutic framework,from histology-based classification to site-specific treatment guidelines.Yet,this framework catastrophically fails for a substantial population of patients diagnosed with cancer of unknown primary(CUP).These patients present metastatic disease,yet their primary tumors remain elusive despite exhaustive clinical workup1.CUP,accounting for 1%-3%of all cancer diagnoses,is an enigma with devastating consequences;the median overall survival is only 2-12 months2-4.The inability to pinpoint an origin forces clinicians to rely on broad-spectrum empirical chemotherapy,such as taxane-carboplatin regimens,which have limited efficacy and exclude patients from the promise of targeted therapies and clinical trials5.CUP is not only a diagnostic challenge but also an indictment of the siloed approach to understanding malignancy:this cancer highlights the limitations of origin-based diagnostic frameworks.However,the confluence of high-dimensional biological data and advanced artificial intelligence(AI)is now poised to address this long-standing diagnostic limitation and to herald a new era for not only CUP but also oncology as a whole(Figure 1).
基金funded by Fundación CajaCanarias and Fundación Bancaria“la Caixa”,grant number 2023DIG11.
文摘Business Process Modelling(BPM)is essential for analyzing,improving,and automating the flow of information within organizations,but traditional approaches based on manual interpretation are slow,error-prone,and require a high level of expertise.This article proposes an innovative alternative solution that overcomes these limitations by automatically generating comprehensive Business Process Modelling and Notation(BPMN)diagrams solely from verbal descriptions of the processes to be modeled,utilizing Large Language Models(LLMs)and multimodal Artificial Intelligence(AI).Experimental results,based on video recordings of process explanations provided by an expert from an organization(in this case,the Commercial Courts of a public justice administration),demonstrate that the proposed methodology successfully enables the automatic generation of complete and accurate BPMN diagrams,leading to significant improvements in the speed,accuracy,and accessibility of process modeling.This research makes a substantial contribution to the field of business process modeling,as its methodology is groundbreaking in its use of LLMs and multimodal AI capabilities to handle different types of source material(text and video),combining several tools to minimize the number of queries and reduce the complexity of the prompts required for the automatic generation of successful BPMN diagrams.
基金supported by grants from the Hangzhou Key Project for Agricultural and Social Development under Grant No.20231203A12(JZ)the General Program of the Scientific Research Special Project for Post-Marketing Clinical Research of Innovative Drugs,Development Center for Medical Science&Technology,National Health Commission of the People’s Republic of China under Grant No.WKZX2024CX104202(JZ).
文摘Artificial intelligence(AI)is transforming the diagnostic landscape of malignant tumors in the urinary system,including prostate cancer,bladder cancer,and renal cell carcinoma(RCC).By integrating imaging,pathology,and molecular data,AI enhances the precision and reproducibility of tumor detection,grading,and risk stratification.In prostate cancer,AI-assisted multiparametric Magnetic resonance imaging(MRI)and digital pathology systems improve lesion localization and Gleason scoring.For bladder cancer,deep learning-based cystoscopy and radiomics models from Computed tomography/magnetic resonance imaging(CT/MRI)enable real-time lesion segmentation and non-invasive biomarker prediction,such as Programmed Cell Death-Ligand 1(PD-L1)expression.In RCC,AI,combined with CT/MRI and multi-omics data,aids in subtype classification and prognostic prediction,supporting personalized therapy.However,despite these promising advances,challenges such as data standardization,model generalizability,interpretability,and regulatory compliance hinder AI’s clinical translation.This review outlines the current state of AI in urological cancer diagnosis and prognosis,its technological innovations,and the clinical challenges and opportunities that lie ahead.
基金Science and Technology Innovation Plan Of Shanghai Science and Technology Commission,Grant/Award Number:21Y21900600Shanghai Zhou Liang Fu Medical Development Foundation,Grant/Award Number:XM00050-2024-3-8+1 种基金National Natural Science Foundation of China,Grant/Award Numbers:82127801,82227806,82272063Science and Technology Innovation 2030-Major Project,Grant/Award Number:2023ZD0511800。
文摘Background:The integration of 7 Tesla(7T)magnetic resonance imaging(MRI)with advanced multimodal artificial intelligence(AI)models represents a promising frontier in neuroimaging.The superior spatial resolution of 7TMRI provides detailed visualizations of brain structure,which are crucial forunderstanding complex central nervous system diseases and tumors.Concurrently,the application of multimodal AI to medical images enables interactive imaging-based diagnostic conversation.Methods:In this paper,we systematically investigate the capacity and feasibility of applying the existing advanced multimodal AI model ChatGPT-4V to 7T MRI under the context of brain tumors.First,we test whether ChatGPT-4V has knowledge about 7T MRI,and whether it can differentiate 7T MRI from 3T MRI.In addition,we explore whether ChatGPT-4V can recognize different 7T MRI modalities and whether it can correctly offer diagnosis of tumors based on single or multiple modality 7T MRI.Results:ChatGPT-4V exhibited accuracy of 84.4%in 3T-vs-7T differentiation and accuracy of 78.9%in 7T modality recognition.Meanwhile,in a human evaluation with three clinical experts,ChatGPT obtained average scores of 9.27/20 in single modality-based diagnosis and 21.25/25 in multiple modality-based diagnosis.Our study indicates that single-modality diagnosis and the interpretability of diagnostic decisions in clinical practice should be enhanced when ChatGPT-4V is applied to 7T data.Conclusions:In general,our analysis suggests that such integration has promise as a tool to improve the workflow of diagnostics in neurology,with a potentially transformative impact in the fields of medical image analysis and patient management.
基金supported in part by the National Natural Science Foundation of China under Grant Nos.62072365 and 62472348the Aviation Science Foundation of China under Grant No.2023M071070002+2 种基金the Key Research and Development Program of Shaanxi Province of China under Grant Nos.2022GY-332,2023-YBGY-230,and 2024GX-YBXM-533the Innovation Capability Support Plan of Shaanxi Province of China under Grant No.2022PT-33the Xi'an Science and Technology Plan Key Industrial Chain Technology Research Project under Grant No.23ZDCYJSGG0007.
文摘In recent years,multimodal agent AI(MAA)has emerged as a pivotal area of research,holding promise for transforming human-machine interaction.Agent AI systems,capable of perceiving and responding to inputs from multiple modalities(e.g.,language,vision,audio),have demonstrated remarkable progress in understanding complex environments and executing intricate tasks.This survey comprehensively reviews the state-of-the-art developments in MAA and examines its fundamental concepts,key techniques,and applications across diverse domains.We first introduce the basics of agent AI and its multimodal interaction capabilities.We then delve into the core technologies that enable agents to perform task planning,decision-making,and multi-sensory fusion.Furthermore,we focus on exploring various applications of MAA in robotics,healthcare,gaming,and beyond.Additionally,we mainly focus on analyzing the challenges and limitations of current systems and propose promising research directions for future improvements,including human-AI collaboration,online learning method improvement.By reviewing existing work and highlighting open questions,this survey aims to provide a comprehensive roadmap for researchers and practitioners in the field of MAA.
