Multi-target tracking is facing the difficulties of modeling uncertain motion and observation noise.Traditional tracking algorithms are limited by specific models and priors that may mismatch a real-world scenario.In ...Multi-target tracking is facing the difficulties of modeling uncertain motion and observation noise.Traditional tracking algorithms are limited by specific models and priors that may mismatch a real-world scenario.In this paper,considering the model-free purpose,we present an online Multi-Target Intelligent Tracking(MTIT)algorithm based on a Deep Long-Short Term Memory(DLSTM)network for complex tracking requirements,named the MTIT-DLSTM algorithm.Firstly,to distinguish trajectories and concatenate the tracking task in a time sequence,we define a target tuple set that is the labeled Random Finite Set(RFS).Then,prediction and update blocks based on the DLSTM network are constructed to predict and estimate the state of targets,respectively.Further,the prediction block can learn the movement trend from the historical state sequence,while the update block can capture the noise characteristic from the historical measurement sequence.Finally,a data association scheme based on Hungarian algorithm and the heuristic track management strategy are employed to assign measurements to targets and adapt births and deaths.Experimental results manifest that,compared with the existing tracking algorithms,our proposed MTIT-DLSTM algorithm can improve effectively the accuracy and robustness in estimating the state of targets appearing at random positions,and be applied to linear and nonlinear multi-target tracking scenarios.展开更多
The unloading relaxation caused by excavation for construction of high arch dams is an important factor influencing the foundation’s integrity and strength.To evaluate the degree of unloading relaxation,the long-shor...The unloading relaxation caused by excavation for construction of high arch dams is an important factor influencing the foundation’s integrity and strength.To evaluate the degree of unloading relaxation,the long-short term memory(LSTM)network was used to estimate the depth of unloading relaxation zones on the left bank foundation of the Baihetan Arch Dam.Principal component analysis indicates that rock charac-teristics,the structural plane,the protection layer,lithology,and time are the main factors.The LSTM network results demonstrate the unloading relaxation characteristics of the left bank,and the relationships with the factors were also analyzed.The structural plane has the most significant influence on the distribution of unloading relaxation zones.Compared with massive basalt,the columnar jointed basalt experiences a more significant unloading relaxation phenomenon with a clear time effect,with the average unloading relaxation period being 50 d.The protection layer can effectively reduce the unloading relaxation depth by approximately 20%.展开更多
Speaker separation in complex acoustic environment is one of challenging tasks in speech separation.In practice,speakers are very often unmoving or moving slowly in normal communication.In this case,the spatial featur...Speaker separation in complex acoustic environment is one of challenging tasks in speech separation.In practice,speakers are very often unmoving or moving slowly in normal communication.In this case,the spatial features among the consecutive speech frames become highly correlated such that it is helpful for speaker separation by providing additional spatial information.To fully exploit this information,we design a separation system on Recurrent Neural Network(RNN)with long short-term memory(LSTM)which effectively learns the temporal dynamics of spatial features.In detail,a LSTM-based speaker separation algorithm is proposed to extract the spatial features in each time-frequency(TF)unit and form the corresponding feature vector.Then,we treat speaker separation as a supervised learning problem,where a modified ideal ratio mask(IRM)is defined as the training function during LSTM learning.Simulations show that the proposed system achieves attractive separation performance in noisy and reverberant environments.Specifically,during the untrained acoustic test with limited priors,e.g.,unmatched signal to noise ratio(SNR)and reverberation,the proposed LSTM based algorithm can still outperforms the existing DNN based method in the measures of PESQ and STOI.It indicates our method is more robust in untrained conditions.展开更多
Time series forecasting and analysis are widely used in many fields and application scenarios.Time series historical data reflects the change pattern and trend,which can serve the application and decision in each appl...Time series forecasting and analysis are widely used in many fields and application scenarios.Time series historical data reflects the change pattern and trend,which can serve the application and decision in each application scenario to a certain extent.In this paper,we select the time series prediction problem in the atmospheric environment scenario to start the application research.In terms of data support,we obtain the data of nearly 3500 vehicles in some cities in China fromRunwoda Research Institute,focusing on the major pollutant emission data of non-road mobile machinery and high emission vehicles in Beijing and Bozhou,Anhui Province to build the dataset and conduct the time series prediction analysis experiments on them.This paper proposes a P-gLSTNet model,and uses Autoregressive Integrated Moving Average model(ARIMA),long and short-term memory(LSTM),and Prophet to predict and compare the emissions in the future period.The experiments are validated on four public data sets and one self-collected data set,and the mean absolute error(MAE),root mean square error(RMSE),and mean absolute percentage error(MAPE)are selected as the evaluationmetrics.The experimental results show that the proposed P-gLSTNet fusion model predicts less error,outperforms the backbone method,and is more suitable for the prediction of time-series data in this scenario.展开更多
Some reconstruction-based anomaly detection models in multivariate time series have brought impressive performance advancements but suffer from weak generalization ability and a lack of anomaly identification.These li...Some reconstruction-based anomaly detection models in multivariate time series have brought impressive performance advancements but suffer from weak generalization ability and a lack of anomaly identification.These limitations can result in the misjudgment of models,leading to a degradation in overall detection performance.This paper proposes a novel transformer-like anomaly detection model adopting a contrastive learning module and a memory block(CLME)to overcome the above limitations.The contrastive learning module tailored for time series data can learn the contextual relationships to generate temporal fine-grained representations.The memory block can record normal patterns of these representations through the utilization of attention-based addressing and reintegration mechanisms.These two modules together effectively alleviate the problem of generalization.Furthermore,this paper introduces a fusion anomaly detection strategy that comprehensively takes into account the residual and feature spaces.Such a strategy can enlarge the discrepancies between normal and abnormal data,which is more conducive to anomaly identification.The proposed CLME model not only efficiently enhances the generalization performance but also improves the ability of anomaly detection.To validate the efficacy of the proposed approach,extensive experiments are conducted on well-established benchmark datasets,including SWaT,PSM,WADI,and MSL.The results demonstrate outstanding performance,with F1 scores of 90.58%,94.83%,91.58%,and 91.75%,respectively.These findings affirm the superiority of the CLME model over existing stateof-the-art anomaly detection methodologies in terms of its ability to detect anomalies within complex datasets accurately.展开更多
This paper is concerned with multidirectional associative memory neural network with distributed delays on almost-periodic time scales.Some sufficient conditions on the existence,uniqueness and the global exponential ...This paper is concerned with multidirectional associative memory neural network with distributed delays on almost-periodic time scales.Some sufficient conditions on the existence,uniqueness and the global exponential stability of almost-periodic solutions are established.An example is presented to illustrate the feasibility and effectiveness of the obtained results.展开更多
Haze-fog,which is an atmospheric aerosol caused by natural or man-made factors,seriously affects the physical and mental health of human beings.PM2.5(a particulate matter whose diameter is smaller than or equal to 2.5...Haze-fog,which is an atmospheric aerosol caused by natural or man-made factors,seriously affects the physical and mental health of human beings.PM2.5(a particulate matter whose diameter is smaller than or equal to 2.5 microns)is the chief culprit causing aerosol.To forecast the condition of PM2.5,this paper adopts the related the meteorological data and air pollutes data to predict the concentration of PM2.5.Since the meteorological data and air pollutes data are typical time series data,it is reasonable to adopt a machine learning method called Single Hidden-Layer Long Short-Term Memory Neural Network(SSHL-LSTMNN)containing memory capability to implement the prediction.However,the number of neurons in the hidden layer is difficult to decide unless manual testing is operated.In order to decide the best structure of the neural network and improve the accuracy of prediction,this paper employs a self-organizing algorithm,which uses Information Processing Capability(IPC)to adjust the number of the hidden neurons automatically during a learning phase.In a word,to predict PM2.5 concentration accurately,this paper proposes the SSHL-LSTMNN to predict PM2.5 concentration.In the experiment,not only the hourly precise prediction but also the daily longer-term prediction is taken into account.At last,the experimental results reflect that SSHL-LSTMNN performs the best.展开更多
Recurrent Neural Networks(RNNs)have been widely applied to deal with temporal problems,such as flood forecasting and financial data processing.On the one hand,traditional RNNs models amplify the gradient issue due to ...Recurrent Neural Networks(RNNs)have been widely applied to deal with temporal problems,such as flood forecasting and financial data processing.On the one hand,traditional RNNs models amplify the gradient issue due to the strict time serial dependency,making it difficult to realize a long-term memory function.On the other hand,RNNs cells are highly complex,which will signifi-cantly increase computational complexity and cause waste of computational resources during model training.In this paper,an improved Time Feedforward Connections Recurrent Neural Networks(TFC-RNNs)model was first proposed to address the gradient issue.A parallel branch was introduced for the hidden state at time t−2 to be directly transferred to time t without the nonlinear transforma-tion at time t−1.This is effective in improving the long-term dependence of RNNs.Then,a novel cell structure named Single Gate Recurrent Unit(SGRU)was presented.This cell structure can reduce the number of parameters for RNNs cell,consequently reducing the computational complexity.Next,applying SGRU to TFC-RNNs as a new TFC-SGRU model solves the above two difficulties.Finally,the performance of our proposed TFC-SGRU was verified through sev-eral experiments in terms of long-term memory and anti-interference capabilities.Experimental results demonstrated that our proposed TFC-SGRU model can cap-ture helpful information with time step 1500 and effectively filter out the noise.The TFC-SGRU model accuracy is better than the LSTM and GRU models regarding language processing ability.展开更多
Aimed at the problem that the traditional ART-2 neural network can not recognize a gradually changing course, an eternal term memory (ETM) vector is introduced into ART-2 to simulate the function of human brain, i.e. ...Aimed at the problem that the traditional ART-2 neural network can not recognize a gradually changing course, an eternal term memory (ETM) vector is introduced into ART-2 to simulate the function of human brain, i.e. the deep remembrance for the initial impression.. The eternal term memory vector is determined only by the initial vector that establishes category neuron node and is used to keep the remembrance for this vector for ever. Two times of vigilance algorithm are put forward, and the posterior input vector must first pass the first vigilance of this eternal term memory vector, only succeeded has it the qualification to begin the second vigilance of long term memory vector. The long term memory vector can be revised only when both of the vigilances are passed. Results of recognition examples show that the improved ART-2 overcomes the defect of traditional ART-2 and can recognize a gradually changing course effectively.展开更多
We propose a new approach for analyzing the global asymptotic stability of the extended discrete-time bidirectional associative memory （BAM） neural networks. By using the Euler rule, we discretize the continuous-tim...We propose a new approach for analyzing the global asymptotic stability of the extended discrete-time bidirectional associative memory （BAM） neural networks. By using the Euler rule, we discretize the continuous-time BAM neural networks as the extended discrete-time BAM neural networks with non-threshold activation functions. Here we present some conditions under which the neural networks have unique equilibrium points. To judge the global asymptotic stability of the equilibrium points, we introduce a new neural network model - standard neural network model （SNNM）. For the SNNMs, we derive the sufficient conditions for the global asymptotic stability of the equilibrium points, which are formulated as some linear matrix inequalities （LMIs）. We transform the discrete-time BAM into the SNNM and apply the general result about the SNNM to the determination of global asymptotic stability of the discrete-time BAM. The approach proposed extends the known stability results, has lower conservativeness, can be verified easily, and can also be applied to other forms of recurrent neural networks.展开更多
Considering the recent developments in deep learning, it has become increasingly important to verify what methods are valid for the prediction of multivariate time-series data. In this study, we propose a novel method...Considering the recent developments in deep learning, it has become increasingly important to verify what methods are valid for the prediction of multivariate time-series data. In this study, we propose a novel method of time-series prediction employing multiple deep learners combined with a Bayesian network where training data is divided into clusters using K-means clustering. We decided how many clusters are the best for K-means with the Bayesian information criteria. Depending on each cluster, the multiple deep learners are trained. We used three types of deep learners: deep neural network (DNN), recurrent neural network (RNN), and long short-term memory (LSTM). A naive Bayes classifier is used to determine which deep learner is in charge of predicting a particular time-series. Our proposed method will be applied to a set of financial time-series data, the Nikkei Average Stock price, to assess the accuracy of the predictions made. Compared with the conventional method of employing a single deep learner to acquire all the data, it is demonstrated by our proposed method that F-value and accuracy are improved.展开更多
针对部分场景下标签较少、样本不均衡的时序数据,为了更好的捕捉序列之间的逐步依赖关系,本文一方面使用具有因果关系属性的时域卷积网络构建生成对抗网络,另一方面使用长短期记忆网络构建嵌入网络和复现网络,以实现模型同时处理短期依...针对部分场景下标签较少、样本不均衡的时序数据,为了更好的捕捉序列之间的逐步依赖关系,本文一方面使用具有因果关系属性的时域卷积网络构建生成对抗网络,另一方面使用长短期记忆网络构建嵌入网络和复现网络,以实现模型同时处理短期依存项和长期依存项,从而提出一种基于时域卷积网络和长短期记忆网络的时间序列生成对抗网络(A Time-series Generative Adversarial Network based on Temporal convolutional network and Long-short term memory network, TL-TimeGAN)。采用覆盖性、有用性和相似度检验的综合分析方法作为合成数据质量的评价指标,进一步全面地评价合成数据的覆盖性、预测程度和相似性。最终,基于以太坊欺诈检测数据集,使用Tabnet网络对扩增数据进行异常检测并获得局部特征重要性以及全局特征重要性,以增强扩增数据应用于实际工作的实践指导价值。展开更多
By employing the Lyapunov stability theory and linear matrix inequality（LMI）technique,delay-dependent stability criterion is derived to ensure the exponential stability of bi-directional associative memory（BAM）neu...By employing the Lyapunov stability theory and linear matrix inequality（LMI）technique,delay-dependent stability criterion is derived to ensure the exponential stability of bi-directional associative memory（BAM）neural networks with time-varying delays.The proposed condition can be checked easily by LMI control toolbox in Matlab.A numerical example is given to demonstrate the effectiveness of our results.展开更多
Air pollution,specifically fine particulate matter(PM2.5),represents a critical environmental and public health concern due to its adverse effects on respiratory and cardiovascular systems.Accurate forecasting of PM2....Air pollution,specifically fine particulate matter(PM2.5),represents a critical environmental and public health concern due to its adverse effects on respiratory and cardiovascular systems.Accurate forecasting of PM2.5 concentrations is essential for mitigating health risks;however,the inherent nonlinearity and dynamic variability of air quality data present significant challenges.This study conducts a systematic evaluation of deep learning algorithms including Convolutional Neural Network(CNN),Long Short-Term Memory(LSTM),and the hybrid CNN-LSTM as well as statistical models,AutoRegressive Integrated Moving Average(ARIMA)and Maximum Likelihood Estimation(MLE)for hourly PM2.5 forecasting.Model performance is quantified using Root Mean Squared Error(RMSE),Mean Absolute Error(MAE),Mean Absolute Percentage Error(MAPE),and the Coefficient of Determination(R^(2))metrics.The comparative analysis identifies optimal predictive approaches for air quality modeling,emphasizing computational efficiency and accuracy.Additionally,CNN classification performance is evaluated using a confusion matrix,accuracy,precision,and F1-score.The results demonstrate that the Hybrid CNN-LSTM model outperforms standalone models,exhibiting lower error rates and higher R^(2) values,thereby highlighting the efficacy of deep learning-based hybrid architectures in achieving robust and precise PM2.5 forecasting.This study underscores the potential of advanced computational techniques in enhancing air quality prediction systems for environmental and public health applications.展开更多
滚动轴承是机械设备中的常见关键部件,准确预测其剩余使用寿命对机械设备的安全稳定运行至关重要。针对目前轴承寿命预测存在的轴承退化特征不明显、模型泛化能力差以及数据长期依赖关系难以捕捉的问题,提出基于时频域信号优化器(Time-F...滚动轴承是机械设备中的常见关键部件,准确预测其剩余使用寿命对机械设备的安全稳定运行至关重要。针对目前轴承寿命预测存在的轴承退化特征不明显、模型泛化能力差以及数据长期依赖关系难以捕捉的问题,提出基于时频域信号优化器(Time-Frequency domain signal Ratio Optimizer,TFRO)的多重膨胀多核时间卷积网络(Multi inflated Multi kernel Time Convolutional Network,Mi-MkTCN)模型。TFRO优化器为了精准记忆重要信息,在每一个时间节点上,将过去信息和当前信息重组,其中过去信息中的重要的时频域特征经过了有比例的分配。Mi-MkTCN利用多重膨胀确保重要特征不丢失,再利用多核时间卷积网络实现对不同尺度特征的提取。最终的消融对比实验验证了改进方法的有效性,模型的平均绝对误差、均方误差及均方根误差指标分别为0.00145、0.05069和0.12045。实验结果表明,所提方法显著提升了轴承剩余使用寿命的预测精度,为轴承剩余使用寿命预测提供了高精度、高鲁棒性的解决方案。展开更多
基金supported by the National Natural Science Foundation of China(No.62276204)Open Foundation of Science and Technology on Electronic Information Control Laboratory,Natural Science Basic Research Program of Shanxi,China(Nos.2022JM-340 and 2023-JC-QN-0710)China Postdoctoral Science Foundation(Nos.2020T130494 and 2018M633470).
文摘Multi-target tracking is facing the difficulties of modeling uncertain motion and observation noise.Traditional tracking algorithms are limited by specific models and priors that may mismatch a real-world scenario.In this paper,considering the model-free purpose,we present an online Multi-Target Intelligent Tracking(MTIT)algorithm based on a Deep Long-Short Term Memory(DLSTM)network for complex tracking requirements,named the MTIT-DLSTM algorithm.Firstly,to distinguish trajectories and concatenate the tracking task in a time sequence,we define a target tuple set that is the labeled Random Finite Set(RFS).Then,prediction and update blocks based on the DLSTM network are constructed to predict and estimate the state of targets,respectively.Further,the prediction block can learn the movement trend from the historical state sequence,while the update block can capture the noise characteristic from the historical measurement sequence.Finally,a data association scheme based on Hungarian algorithm and the heuristic track management strategy are employed to assign measurements to targets and adapt births and deaths.Experimental results manifest that,compared with the existing tracking algorithms,our proposed MTIT-DLSTM algorithm can improve effectively the accuracy and robustness in estimating the state of targets appearing at random positions,and be applied to linear and nonlinear multi-target tracking scenarios.
基金This work was supported by the National Key Research and Development Program of China(Grant No.2018YFC0407004)the Natural Science Foundation of China(Grants No.51939004 and 11772116).
文摘The unloading relaxation caused by excavation for construction of high arch dams is an important factor influencing the foundation’s integrity and strength.To evaluate the degree of unloading relaxation,the long-short term memory(LSTM)network was used to estimate the depth of unloading relaxation zones on the left bank foundation of the Baihetan Arch Dam.Principal component analysis indicates that rock charac-teristics,the structural plane,the protection layer,lithology,and time are the main factors.The LSTM network results demonstrate the unloading relaxation characteristics of the left bank,and the relationships with the factors were also analyzed.The structural plane has the most significant influence on the distribution of unloading relaxation zones.Compared with massive basalt,the columnar jointed basalt experiences a more significant unloading relaxation phenomenon with a clear time effect,with the average unloading relaxation period being 50 d.The protection layer can effectively reduce the unloading relaxation depth by approximately 20%.
基金This work is supported by the National Nature Science Foundation of China(NSFC)under Grant Nos.61571106,61501169,41706103the Fundamental Research Funds for the Central Universities under Grant No.2242013K30010.
文摘Speaker separation in complex acoustic environment is one of challenging tasks in speech separation.In practice,speakers are very often unmoving or moving slowly in normal communication.In this case,the spatial features among the consecutive speech frames become highly correlated such that it is helpful for speaker separation by providing additional spatial information.To fully exploit this information,we design a separation system on Recurrent Neural Network(RNN)with long short-term memory(LSTM)which effectively learns the temporal dynamics of spatial features.In detail,a LSTM-based speaker separation algorithm is proposed to extract the spatial features in each time-frequency(TF)unit and form the corresponding feature vector.Then,we treat speaker separation as a supervised learning problem,where a modified ideal ratio mask(IRM)is defined as the training function during LSTM learning.Simulations show that the proposed system achieves attractive separation performance in noisy and reverberant environments.Specifically,during the untrained acoustic test with limited priors,e.g.,unmatched signal to noise ratio(SNR)and reverberation,the proposed LSTM based algorithm can still outperforms the existing DNN based method in the measures of PESQ and STOI.It indicates our method is more robust in untrained conditions.
基金the Beijing Chaoyang District Collaborative Innovation Project(No.CYXT2013)the subject support of Beijing Municipal Science and Technology Key R&D Program-Capital Blue Sky Action Cultivation Project(Z19110900910000)+1 种基金“Research and Demonstration ofHigh Emission Vehicle Monitoring Equipment System Based on Sensor Integration Technology”(Z19110000911003)This work was supported by the Academic Research Projects of Beijing Union University(No.ZK80202103).
文摘Time series forecasting and analysis are widely used in many fields and application scenarios.Time series historical data reflects the change pattern and trend,which can serve the application and decision in each application scenario to a certain extent.In this paper,we select the time series prediction problem in the atmospheric environment scenario to start the application research.In terms of data support,we obtain the data of nearly 3500 vehicles in some cities in China fromRunwoda Research Institute,focusing on the major pollutant emission data of non-road mobile machinery and high emission vehicles in Beijing and Bozhou,Anhui Province to build the dataset and conduct the time series prediction analysis experiments on them.This paper proposes a P-gLSTNet model,and uses Autoregressive Integrated Moving Average model(ARIMA),long and short-term memory(LSTM),and Prophet to predict and compare the emissions in the future period.The experiments are validated on four public data sets and one self-collected data set,and the mean absolute error(MAE),root mean square error(RMSE),and mean absolute percentage error(MAPE)are selected as the evaluationmetrics.The experimental results show that the proposed P-gLSTNet fusion model predicts less error,outperforms the backbone method,and is more suitable for the prediction of time-series data in this scenario.
基金support from the Major National Science and Technology Special Projects(2016ZX02301003-004-007)the Natural Science Foundation of Hebei Province(F2020202067)。
文摘Some reconstruction-based anomaly detection models in multivariate time series have brought impressive performance advancements but suffer from weak generalization ability and a lack of anomaly identification.These limitations can result in the misjudgment of models,leading to a degradation in overall detection performance.This paper proposes a novel transformer-like anomaly detection model adopting a contrastive learning module and a memory block(CLME)to overcome the above limitations.The contrastive learning module tailored for time series data can learn the contextual relationships to generate temporal fine-grained representations.The memory block can record normal patterns of these representations through the utilization of attention-based addressing and reintegration mechanisms.These two modules together effectively alleviate the problem of generalization.Furthermore,this paper introduces a fusion anomaly detection strategy that comprehensively takes into account the residual and feature spaces.Such a strategy can enlarge the discrepancies between normal and abnormal data,which is more conducive to anomaly identification.The proposed CLME model not only efficiently enhances the generalization performance but also improves the ability of anomaly detection.To validate the efficacy of the proposed approach,extensive experiments are conducted on well-established benchmark datasets,including SWaT,PSM,WADI,and MSL.The results demonstrate outstanding performance,with F1 scores of 90.58%,94.83%,91.58%,and 91.75%,respectively.These findings affirm the superiority of the CLME model over existing stateof-the-art anomaly detection methodologies in terms of its ability to detect anomalies within complex datasets accurately.
基金the National Natural Science Foundation of China(11671406,12071491)the Research Fund of Shenzhen Institute of Information Technology(QN201703).
文摘This paper is concerned with multidirectional associative memory neural network with distributed delays on almost-periodic time scales.Some sufficient conditions on the existence,uniqueness and the global exponential stability of almost-periodic solutions are established.An example is presented to illustrate the feasibility and effectiveness of the obtained results.
文摘Haze-fog,which is an atmospheric aerosol caused by natural or man-made factors,seriously affects the physical and mental health of human beings.PM2.5(a particulate matter whose diameter is smaller than or equal to 2.5 microns)is the chief culprit causing aerosol.To forecast the condition of PM2.5,this paper adopts the related the meteorological data and air pollutes data to predict the concentration of PM2.5.Since the meteorological data and air pollutes data are typical time series data,it is reasonable to adopt a machine learning method called Single Hidden-Layer Long Short-Term Memory Neural Network(SSHL-LSTMNN)containing memory capability to implement the prediction.However,the number of neurons in the hidden layer is difficult to decide unless manual testing is operated.In order to decide the best structure of the neural network and improve the accuracy of prediction,this paper employs a self-organizing algorithm,which uses Information Processing Capability(IPC)to adjust the number of the hidden neurons automatically during a learning phase.In a word,to predict PM2.5 concentration accurately,this paper proposes the SSHL-LSTMNN to predict PM2.5 concentration.In the experiment,not only the hourly precise prediction but also the daily longer-term prediction is taken into account.At last,the experimental results reflect that SSHL-LSTMNN performs the best.
基金This work was funded by the National Science Foundation of Hunan Province(2020JJ2029)。
文摘Recurrent Neural Networks(RNNs)have been widely applied to deal with temporal problems,such as flood forecasting and financial data processing.On the one hand,traditional RNNs models amplify the gradient issue due to the strict time serial dependency,making it difficult to realize a long-term memory function.On the other hand,RNNs cells are highly complex,which will signifi-cantly increase computational complexity and cause waste of computational resources during model training.In this paper,an improved Time Feedforward Connections Recurrent Neural Networks(TFC-RNNs)model was first proposed to address the gradient issue.A parallel branch was introduced for the hidden state at time t−2 to be directly transferred to time t without the nonlinear transforma-tion at time t−1.This is effective in improving the long-term dependence of RNNs.Then,a novel cell structure named Single Gate Recurrent Unit(SGRU)was presented.This cell structure can reduce the number of parameters for RNNs cell,consequently reducing the computational complexity.Next,applying SGRU to TFC-RNNs as a new TFC-SGRU model solves the above two difficulties.Finally,the performance of our proposed TFC-SGRU was verified through sev-eral experiments in terms of long-term memory and anti-interference capabilities.Experimental results demonstrated that our proposed TFC-SGRU model can cap-ture helpful information with time step 1500 and effectively filter out the noise.The TFC-SGRU model accuracy is better than the LSTM and GRU models regarding language processing ability.
基金Sponsored by the National Natural Science Foundation of China (Grant No. 50305005)
文摘Aimed at the problem that the traditional ART-2 neural network can not recognize a gradually changing course, an eternal term memory (ETM) vector is introduced into ART-2 to simulate the function of human brain, i.e. the deep remembrance for the initial impression.. The eternal term memory vector is determined only by the initial vector that establishes category neuron node and is used to keep the remembrance for this vector for ever. Two times of vigilance algorithm are put forward, and the posterior input vector must first pass the first vigilance of this eternal term memory vector, only succeeded has it the qualification to begin the second vigilance of long term memory vector. The long term memory vector can be revised only when both of the vigilances are passed. Results of recognition examples show that the improved ART-2 overcomes the defect of traditional ART-2 and can recognize a gradually changing course effectively.
基金This project was supported by the National Natural Science Foundation of China (60074008) .
文摘We propose a new approach for analyzing the global asymptotic stability of the extended discrete-time bidirectional associative memory （BAM） neural networks. By using the Euler rule, we discretize the continuous-time BAM neural networks as the extended discrete-time BAM neural networks with non-threshold activation functions. Here we present some conditions under which the neural networks have unique equilibrium points. To judge the global asymptotic stability of the equilibrium points, we introduce a new neural network model - standard neural network model （SNNM）. For the SNNMs, we derive the sufficient conditions for the global asymptotic stability of the equilibrium points, which are formulated as some linear matrix inequalities （LMIs）. We transform the discrete-time BAM into the SNNM and apply the general result about the SNNM to the determination of global asymptotic stability of the discrete-time BAM. The approach proposed extends the known stability results, has lower conservativeness, can be verified easily, and can also be applied to other forms of recurrent neural networks.
文摘Considering the recent developments in deep learning, it has become increasingly important to verify what methods are valid for the prediction of multivariate time-series data. In this study, we propose a novel method of time-series prediction employing multiple deep learners combined with a Bayesian network where training data is divided into clusters using K-means clustering. We decided how many clusters are the best for K-means with the Bayesian information criteria. Depending on each cluster, the multiple deep learners are trained. We used three types of deep learners: deep neural network (DNN), recurrent neural network (RNN), and long short-term memory (LSTM). A naive Bayes classifier is used to determine which deep learner is in charge of predicting a particular time-series. Our proposed method will be applied to a set of financial time-series data, the Nikkei Average Stock price, to assess the accuracy of the predictions made. Compared with the conventional method of employing a single deep learner to acquire all the data, it is demonstrated by our proposed method that F-value and accuracy are improved.
文摘针对部分场景下标签较少、样本不均衡的时序数据,为了更好的捕捉序列之间的逐步依赖关系,本文一方面使用具有因果关系属性的时域卷积网络构建生成对抗网络,另一方面使用长短期记忆网络构建嵌入网络和复现网络,以实现模型同时处理短期依存项和长期依存项,从而提出一种基于时域卷积网络和长短期记忆网络的时间序列生成对抗网络(A Time-series Generative Adversarial Network based on Temporal convolutional network and Long-short term memory network, TL-TimeGAN)。采用覆盖性、有用性和相似度检验的综合分析方法作为合成数据质量的评价指标,进一步全面地评价合成数据的覆盖性、预测程度和相似性。最终,基于以太坊欺诈检测数据集,使用Tabnet网络对扩增数据进行异常检测并获得局部特征重要性以及全局特征重要性,以增强扩增数据应用于实际工作的实践指导价值。
基金supported by Natural Science Foundation of Hebei Province under Grant No.E2007000381
文摘By employing the Lyapunov stability theory and linear matrix inequality（LMI）technique,delay-dependent stability criterion is derived to ensure the exponential stability of bi-directional associative memory（BAM）neural networks with time-varying delays.The proposed condition can be checked easily by LMI control toolbox in Matlab.A numerical example is given to demonstrate the effectiveness of our results.
文摘Air pollution,specifically fine particulate matter(PM2.5),represents a critical environmental and public health concern due to its adverse effects on respiratory and cardiovascular systems.Accurate forecasting of PM2.5 concentrations is essential for mitigating health risks;however,the inherent nonlinearity and dynamic variability of air quality data present significant challenges.This study conducts a systematic evaluation of deep learning algorithms including Convolutional Neural Network(CNN),Long Short-Term Memory(LSTM),and the hybrid CNN-LSTM as well as statistical models,AutoRegressive Integrated Moving Average(ARIMA)and Maximum Likelihood Estimation(MLE)for hourly PM2.5 forecasting.Model performance is quantified using Root Mean Squared Error(RMSE),Mean Absolute Error(MAE),Mean Absolute Percentage Error(MAPE),and the Coefficient of Determination(R^(2))metrics.The comparative analysis identifies optimal predictive approaches for air quality modeling,emphasizing computational efficiency and accuracy.Additionally,CNN classification performance is evaluated using a confusion matrix,accuracy,precision,and F1-score.The results demonstrate that the Hybrid CNN-LSTM model outperforms standalone models,exhibiting lower error rates and higher R^(2) values,thereby highlighting the efficacy of deep learning-based hybrid architectures in achieving robust and precise PM2.5 forecasting.This study underscores the potential of advanced computational techniques in enhancing air quality prediction systems for environmental and public health applications.
文摘滚动轴承是机械设备中的常见关键部件,准确预测其剩余使用寿命对机械设备的安全稳定运行至关重要。针对目前轴承寿命预测存在的轴承退化特征不明显、模型泛化能力差以及数据长期依赖关系难以捕捉的问题,提出基于时频域信号优化器(Time-Frequency domain signal Ratio Optimizer,TFRO)的多重膨胀多核时间卷积网络(Multi inflated Multi kernel Time Convolutional Network,Mi-MkTCN)模型。TFRO优化器为了精准记忆重要信息,在每一个时间节点上,将过去信息和当前信息重组,其中过去信息中的重要的时频域特征经过了有比例的分配。Mi-MkTCN利用多重膨胀确保重要特征不丢失,再利用多核时间卷积网络实现对不同尺度特征的提取。最终的消融对比实验验证了改进方法的有效性,模型的平均绝对误差、均方误差及均方根误差指标分别为0.00145、0.05069和0.12045。实验结果表明,所提方法显著提升了轴承剩余使用寿命的预测精度,为轴承剩余使用寿命预测提供了高精度、高鲁棒性的解决方案。
