The contribution of this work is twofold： （1） a multimodality prediction method of chaotic time series with the Gaussian process mixture （GPM） model is proposed, which employs a divide and conquer strategy. It au...The contribution of this work is twofold： （1） a multimodality prediction method of chaotic time series with the Gaussian process mixture （GPM） model is proposed, which employs a divide and conquer strategy. It automatically divides the chaotic time series into multiple modalities with different extrinsic patterns and intrinsic characteristics, and thus can more precisely fit the chaotic time series. （2） An effective sparse hard-cut expec- tation maximization （SHC-EM） learning algorithm for the GPM model is proposed to improve the prediction performance. SHO-EM replaces a large learning sample set with fewer pseudo inputs, accelerating model learning based on these pseudo inputs. Experiments on Lorenz and Chua time series demonstrate that the proposed method yields not only accurate multimodality prediction, but also the prediction confidence interval SHC-EM outperforms the traditional variational 1earning in terms of both prediction accuracy and speed. In addition, SHC-EM is more robust and insusceptible to noise than variational learning.展开更多
Industrial fault diagnosis is crucial for ensuring the safety and efficiency of modern production systems.Industrial big data,particularly large-scale tabular data capturing multivariate time-series processes,offer va...Industrial fault diagnosis is crucial for ensuring the safety and efficiency of modern production systems.Industrial big data,particularly large-scale tabular data capturing multivariate time-series processes,offer valuable operational insights.Existing methods face significant challenges due to extreme label scarcity and massive unlabeled data volumes.Large Language Models(LLMs)hold great potential to address these issues due to their strong heterogeneous and few-shot learning capabilities.However,the application of LLMs to fault diagnosis with industrial big data,especially for tabular data,remains unexplored.In view of this,we propose a novel semi-supervised prefix tuning of LLMs for fault diagnosis with industrial big data.We first generate auxiliary prediction tasks based on the unlabeled data as the semi-supervised training materials for LLMs.Then we design a prefix-based soft embedding layer to fine-tune the LLMs,so that the model is able to learn the task-specific information in a parameter-efficient way.To make the model applicable to industrial big data,we also implement the Sparse Gaussian Processes(SGP)to filter the most informative samples to relieve the computational cost.Finally,we design a hybrid prompt template to effectively combine the hard and soft prompts and formulate the final prediction prompt for the industrial diagnosis tasks.The experiments have proven the superiority of the proposed method.展开更多
基金Supported by the National Natural Science Foundation of China under Grant No 60972106the China Postdoctoral Science Foundation under Grant No 2014M561053+1 种基金the Humanity and Social Science Foundation of Ministry of Education of China under Grant No 15YJA630108the Hebei Province Natural Science Foundation under Grant No E2016202341
文摘The contribution of this work is twofold： （1） a multimodality prediction method of chaotic time series with the Gaussian process mixture （GPM） model is proposed, which employs a divide and conquer strategy. It automatically divides the chaotic time series into multiple modalities with different extrinsic patterns and intrinsic characteristics, and thus can more precisely fit the chaotic time series. （2） An effective sparse hard-cut expec- tation maximization （SHC-EM） learning algorithm for the GPM model is proposed to improve the prediction performance. SHO-EM replaces a large learning sample set with fewer pseudo inputs, accelerating model learning based on these pseudo inputs. Experiments on Lorenz and Chua time series demonstrate that the proposed method yields not only accurate multimodality prediction, but also the prediction confidence interval SHC-EM outperforms the traditional variational 1earning in terms of both prediction accuracy and speed. In addition, SHC-EM is more robust and insusceptible to noise than variational learning.
基金supported by the National Key R&D Program of China(No.2023YFB4704900)the National Natural Science Foundation of China(Nos.62422312,62203134,and 62503337)+3 种基金the National Natural Science Funds for Distinguished Young Scholar(No.62325307)the Natural Science Foundation of Guangdong Province(No.023B1515120038)the Shenzhen Science and Technology Innovation Commission(Nos.20220809141216003 and KJZD20230923113801004)the Scientific Instrument Developing Project of Shenzhen University(No.2023YQ019).
文摘Industrial fault diagnosis is crucial for ensuring the safety and efficiency of modern production systems.Industrial big data,particularly large-scale tabular data capturing multivariate time-series processes,offer valuable operational insights.Existing methods face significant challenges due to extreme label scarcity and massive unlabeled data volumes.Large Language Models(LLMs)hold great potential to address these issues due to their strong heterogeneous and few-shot learning capabilities.However,the application of LLMs to fault diagnosis with industrial big data,especially for tabular data,remains unexplored.In view of this,we propose a novel semi-supervised prefix tuning of LLMs for fault diagnosis with industrial big data.We first generate auxiliary prediction tasks based on the unlabeled data as the semi-supervised training materials for LLMs.Then we design a prefix-based soft embedding layer to fine-tune the LLMs,so that the model is able to learn the task-specific information in a parameter-efficient way.To make the model applicable to industrial big data,we also implement the Sparse Gaussian Processes(SGP)to filter the most informative samples to relieve the computational cost.Finally,we design a hybrid prompt template to effectively combine the hard and soft prompts and formulate the final prediction prompt for the industrial diagnosis tasks.The experiments have proven the superiority of the proposed method.
