With the improvement of multisource information sensing and data acquisition capabilities inside tunnels,the availability of multimodal data in tunnel engineering has significantly increased.However,due to structural ...With the improvement of multisource information sensing and data acquisition capabilities inside tunnels,the availability of multimodal data in tunnel engineering has significantly increased.However,due to structural differences in multimodal data,traditional intelligent advanced geological prediction models have limited capacity for data fusion.Furthermore,the lack of pre-trained models makes it difficult for neural networks trained from scratch to deeply explore the features of multimodal data.To address these challenges,we utilize the fusion capability of knowledge graph for multimodal data and the pre-trained knowledge of large language models(LLMs)to establish an intelligent advanced geological prediction model(GeoPredict-LLM).First,we develop an advanced geological prediction ontology model,forming a knowledge graph database.Using knowledge graph embeddings,multisource and multimodal data are transformed into low-dimensional vectors with a unified structure.Secondly,pre-trained LLMs,through reprogramming,reconstruct these low-dimensional vectors,imparting linguistic characteristics to the data.This transformation effectively reframes the complex task of advanced geological prediction as a"language-based"problem,enabling the model to approach the task from a linguistic perspective.Moreover,we propose the prompt-as-prefix method,which enables output generation,while freezing the core of the LLM,thereby significantly reduces the number of training parameters.Finally,evaluations show that compared to neural network models without pre-trained models,GeoPredict-LLM significantly improves prediction accuracy.It is worth noting that as long as a knowledge graph database can be established,GeoPredict-LLM can be adapted to multimodal data mining tasks with minimal modifications.展开更多
基金the National Natural Science Foundation of China(Grant Nos.52279103 and 52379103)。
文摘With the improvement of multisource information sensing and data acquisition capabilities inside tunnels,the availability of multimodal data in tunnel engineering has significantly increased.However,due to structural differences in multimodal data,traditional intelligent advanced geological prediction models have limited capacity for data fusion.Furthermore,the lack of pre-trained models makes it difficult for neural networks trained from scratch to deeply explore the features of multimodal data.To address these challenges,we utilize the fusion capability of knowledge graph for multimodal data and the pre-trained knowledge of large language models(LLMs)to establish an intelligent advanced geological prediction model(GeoPredict-LLM).First,we develop an advanced geological prediction ontology model,forming a knowledge graph database.Using knowledge graph embeddings,multisource and multimodal data are transformed into low-dimensional vectors with a unified structure.Secondly,pre-trained LLMs,through reprogramming,reconstruct these low-dimensional vectors,imparting linguistic characteristics to the data.This transformation effectively reframes the complex task of advanced geological prediction as a"language-based"problem,enabling the model to approach the task from a linguistic perspective.Moreover,we propose the prompt-as-prefix method,which enables output generation,while freezing the core of the LLM,thereby significantly reduces the number of training parameters.Finally,evaluations show that compared to neural network models without pre-trained models,GeoPredict-LLM significantly improves prediction accuracy.It is worth noting that as long as a knowledge graph database can be established,GeoPredict-LLM can be adapted to multimodal data mining tasks with minimal modifications.
