摘要
目前基于Transformer的时序预测算法主要是利用Transformer内的注意力机制来处理一连串的数据。与基于循环神经网络(Recurrent Neural Network, RNN)的方法不同,Transformer使模型可以访问历史数据的任意一个部分而不用考虑距离,这使得它有可能更适合于学习到具有长期依赖性的重复模式。而状态空间模型在学习时具有更强的先验信息,在先验信息正确时可以减少模型训练的数据量。文章根据两种方法的特点提出了状态空间模型与Transformer联合的时序预测算法SSMT,先用Transformer将数据特征映射为状态空间模型的参数,再使用状态空间模型预测时间序列在每一个时间点上的概率分布,通过概率分布进行详细预测。通过在真实数据集上的测试,该算法在数据量不大的小数据集上也能取得较好的效果。
The current Transformer-based temporal prediction algorithms mainly use the attention mechanism within the Transformer to process a sequence of data. Unlike RNN-based approaches, Transformer allows the model to access any part of the historical data without considering the distance, which makes it potentially more suitable for learning to recurrent patterns with long-term dependencies. In contrast, the state-space model has a stronger prior in learning, which can reduce the amount of data for model training when the prior is correct. A temporal prediction algorithm SSMT is proposed based on the characteristics of the two methods, where the data features are first mapped into parameters of the state space model using the Transformer, and then the probability distribution of the time series at each time point is predicted using the state space model, and then detailed predictions are made through the probability distribution. By testing on real datasets, the algorithm achieves better results on small datasets with small amount of data.
作者
刘立伟
余绍俊
Liu Liwei;Yu Shaojun(College of Information Science and Technology,Yunnan Normal University,Kunming 650500,China;School of Information Engineering,Kunming University,Kunming 650214,China)
出处
《无线互联科技》
2022年第24期131-137,共7页
Wireless Internet Technology
基金
国家自然科学基金,项目名称:协作D2D通信网络容量理论与方法研究,项目编号:61962033
云南省地方本科高校(部分)基础研究重点项目,项目名称:协作D2D通信网络性能评估与增强技术研究,项目编号:2018FH001-010。
