摘要
随着生物多组学、医学影像和电子健康记录等多源医疗数据的爆炸式增长,单一模态难以刻画复杂疾病的生物学异质性。多模态医学数据融合技术通过在特征级、表示级和决策级整合异构信息,为疾病预测与治疗提供了新的可能。本研究系统梳理了近年来基于深度学习与统计建模的融合方法学进展,包括Transformer与图神经网络驱动的端到端框架,贝叶斯及潜在因子模型支撑的显式概率推断,以及信息瓶颈、共性-特异性分解等增强表示有效性的理论新视角。针对跨模态异质性和高维稀疏性,本文总结了早期、中期、晚期三类融合策略及协同训练、多视角对齐等训练范式,并讨论注意力机制在捕获互补信息中的作用。进一步结合癌症预后、生物标志物发现、药物反应预测和临床决策支持等应用案例,阐释融合模型在提高预测性能、增强可解释性和契合临床工作流方面的优势与挑战。本文提出面向临床可落地的未来研究方向:构建安全合规的联邦数据湖、发展因果可解释融合框架、加强与医护流程的深度耦合,以实现从多模态数据到精准诊疗的闭环转化。
With the explosive growth of multi-source medical data such as bio-multi-omics,medical imaging,and electronic health records,a single modality is unable to characterize the biological heterogeneity of complex diseases.Multimodal medical data fusion technology provides new possibilities for disease prediction and treatment by integrating heterogeneous information at the feature level,representation level,and decision level.This study systematically reviews the progress of fusion methodologies based on deep learning and statistical modeling in recent years,including end-toend frameworks driven by Transformer and graph neural networks,explicit probabilistic inference supported by Bayesian and latent factor models,and new theoretical perspectives such as information bottlenecks and commonality-specificity decomposition to enhance representation effectiveness.In view of cross-modal heterogeneity and high-dimensional sparsity,this paper summarizes three types of fusion strategies,namely early,mid-,and late-stage,as well as training paradigms such as collaborative training and multi-view alignment,and discusses the role of attention mechanisms in capturing complementary information.Further combined with application cases such as cancer prognosis,biomarker discovery,drug response prediction,and clinical decision support,this paper explains the advantages and challenges of fusion models in improving prediction performance,enhancing interpretability,and fitting clinical workflows.This paper proposes future research directions for clinical implementation:building a secure and compliant federal data lake,developing a causal explainable fusion framework,and strengthening deep coupling with medical care processes to achieve a closed-loop transformation from multimodal data to precision diagnosis and treatment.
作者
杨帆
YANG Fan(Department of Medical Dataology,School of Public Health,Cheeloo College of Medicine,Shandong University,Jinan 250012,Shandong,China;National Institute of Health and Medical Big Data,Jinan 250003,Shandong,China;Qilu Hospital of Shandong University,Jinan 250012,Shandong,China)
出处
《山东大学学报(医学版)》
北大核心
2025年第8期17-40,共24页
Journal of Shandong University(Health Sciences)
基金
国家自然科学基金(82273736)。
关键词
多模态融合
深度学习
信息瓶颈
可解释性
精准诊疗
Multimodal fusion
Deep learning
Information bottleneck
Explainability
Precision diagnosis and treatment
