Tunnel boring machines(TBMs)have been widely utilised in tunnel construction due to their high efficiency and reliability.Accurately predicting TBM performance can improve project time management,cost control,and risk...Tunnel boring machines(TBMs)have been widely utilised in tunnel construction due to their high efficiency and reliability.Accurately predicting TBM performance can improve project time management,cost control,and risk management.This study aims to use deep learning to develop real-time models for predicting the penetration rate(PR).The models are built using data from the Changsha metro project,and their performances are evaluated using unseen data from the Zhengzhou Metro project.In one-step forecast,the predicted penetration rate follows the trend of the measured penetration rate in both training and testing.The autoregressive integrated moving average(ARIMA)model is compared with the recurrent neural network(RNN)model.The results show that univariate models,which only consider historical penetration rate itself,perform better than multivariate models that take into account multiple geological and operational parameters(GEO and OP).Next,an RNN variant combining time series of penetration rate with the last-step geological and operational parameters is developed,and it performs better than other models.A sensitivity analysis shows that the penetration rate is the most important parameter,while other parameters have a smaller impact on time series forecasting.It is also found that smoothed data are easier to predict with high accuracy.Nevertheless,over-simplified data can lose real characteristics in time series.In conclusion,the RNN variant can accurately predict the next-step penetration rate,and data smoothing is crucial in time series forecasting.This study provides practical guidance for TBM performance forecasting in practical engineering.展开更多
This research aims to enhance Clinical Decision Support Systems(CDSS)within Wireless Body Area Networks(WBANs)by leveraging advanced machine learning techniques.Specifically,we target the challenges of accurate diagno...This research aims to enhance Clinical Decision Support Systems(CDSS)within Wireless Body Area Networks(WBANs)by leveraging advanced machine learning techniques.Specifically,we target the challenges of accurate diagnosis in medical imaging and sequential data analysis using Recurrent Neural Networks(RNNs)with Long Short-Term Memory(LSTM)layers and echo state cells.These models are tailored to improve diagnostic precision,particularly for conditions like rotator cuff tears in osteoporosis patients and gastrointestinal diseases.Traditional diagnostic methods and existing CDSS frameworks often fall short in managing complex,sequential medical data,struggling with long-term dependencies and data imbalances,resulting in suboptimal accuracy and delayed decisions.Our goal is to develop Artificial Intelligence(AI)models that address these shortcomings,offering robust,real-time diagnostic support.We propose a hybrid RNN model that integrates SimpleRNN,LSTM layers,and echo state cells to manage long-term dependencies effectively.Additionally,we introduce CG-Net,a novel Convolutional Neural Network(CNN)framework for gastrointestinal disease classification,which outperforms traditional CNN models.We further enhance model performance through data augmentation and transfer learning,improving generalization and robustness against data scarcity and imbalance.Comprehensive validation,including 5-fold cross-validation and metrics such as accuracy,precision,recall,F1-score,and Area Under the Curve(AUC),confirms the models’reliability.Moreover,SHapley Additive exPlanations(SHAP)and Local Interpretable Model-agnostic Explanations(LIME)are employed to improve model interpretability.Our findings show that the proposed models significantly enhance diagnostic accuracy and efficiency,offering substantial advancements in WBANs and CDSS.展开更多
Accurate estimation of the remaining useful life(RUL)and health state for rollers is of great significance to hot rolling production.It can provide decision support for roller management so as to improve the productiv...Accurate estimation of the remaining useful life(RUL)and health state for rollers is of great significance to hot rolling production.It can provide decision support for roller management so as to improve the productivity of the hot rolling process.In addition,the RUL prediction for rollers is helpful in transitioning from the current regular maintenance strategy to conditional-based maintenance.Therefore,a new method that can extract coarse-grained and fine-grained features from batch data to predict the RUL of the rollers is proposed in this paper.Firstly,a new deep learning network architecture based on recurrent neural networks that can make full use of the extracted coarsegrained fine-grained features to estimate the heath indicator(HI)is developed,where the HI is able to indicate the health state of the roller.Following that,a state-space model is constructed to describe the HI,and the probabilistic distribution of RUL can be estimated by extrapolating the HI degradation model to a predefined failure threshold.Finally,application to a hot strip mill is given to verify the effectiveness of the proposed methods using data collected from an industrial site,and the relatively low RMSE and MAE values demonstrate its advantages compared with some other popular deep learning methods.展开更多
This paper deals with the stability of static recurrent neural networks （RNNs） with a time-varying delay. An augmented Lyapunov-Krasovskii functional is employed, in which some useful terms are included. Furthermore...This paper deals with the stability of static recurrent neural networks （RNNs） with a time-varying delay. An augmented Lyapunov-Krasovskii functional is employed, in which some useful terms are included. Furthermore, the relationship among the timevarying delay, its upper bound and their difierence, is taken into account, and novel bounding techniques for 1- τ（t） are employed. As a result, without ignoring any useful term in the derivative of the Lyapunov-Krasovskii functional, the resulting delay-dependent criteria show less conservative than the existing ones. Finally, a numerical example is given to demonstrate the effectiveness of the proposed methods.展开更多
The robust exponential stability of a larger class of discrete-time recurrent neural networks (RNNs) is explored in this paper. A novel neural network model, named standard neural network model (SNNM), is introduced t...The robust exponential stability of a larger class of discrete-time recurrent neural networks (RNNs) is explored in this paper. A novel neural network model, named standard neural network model (SNNM), is introduced to provide a general framework for stability analysis of RNNs. Most of the existing RNNs can be transformed into SNNMs to be analyzed in a unified way. Applying Lyapunov stability theory method and S-Procedure technique, two useful criteria of robust exponential stability for the discrete-time SNNMs are derived. The conditions presented are formulated as linear matrix inequalities (LMIs) to be easily solved using existing efficient convex optimization techniques. An example is presented to demonstrate the transformation procedure and the effectiveness of the results.展开更多
The robust global exponential stability of a class of interval recurrent neural networks（RNNs） is studied,and a new robust stability criterion is obtained in the form of linear matrix inequality.The problem of robus...The robust global exponential stability of a class of interval recurrent neural networks（RNNs） is studied,and a new robust stability criterion is obtained in the form of linear matrix inequality.The problem of robust stability of interval RNNs is transformed into a problem of solving a class of linear matrix inequalities.Thus,the robust stability of interval RNNs can be analyzed by directly using the linear matrix inequalities（LMI） toolbox of MATLAB.Numerical example is given to show the effectiveness of the obtained results.展开更多
Phishing attacks present a persistent and evolving threat in the cybersecurity land-scape,necessitating the development of more sophisticated detection methods.Traditional machine learning approaches to phishing detec...Phishing attacks present a persistent and evolving threat in the cybersecurity land-scape,necessitating the development of more sophisticated detection methods.Traditional machine learning approaches to phishing detection have relied heavily on feature engineering and have often fallen short in adapting to the dynamically changing patterns of phishingUniformResource Locator(URLs).Addressing these challenge,we introduce a framework that integrates the sequential data processing strengths of a Recurrent Neural Network(RNN)with the hyperparameter optimization prowess of theWhale Optimization Algorithm(WOA).Ourmodel capitalizes on an extensive Kaggle dataset,featuring over 11,000 URLs,each delineated by 30 attributes.The WOA’s hyperparameter optimization enhances the RNN’s performance,evidenced by a meticulous validation process.The results,encapsulated in precision,recall,and F1-score metrics,surpass baseline models,achieving an overall accuracy of 92%.This study not only demonstrates the RNN’s proficiency in learning complex patterns but also underscores the WOA’s effectiveness in refining machine learning models for the critical task of phishing detection.展开更多
Due to the increase in the types of business and equipment in telecommunications companies,the performance index data collected in the operation and maintenance process varies greatly.The diversity of index data makes...Due to the increase in the types of business and equipment in telecommunications companies,the performance index data collected in the operation and maintenance process varies greatly.The diversity of index data makes it very difficult to perform high-precision capacity prediction.In order to improve the forecasting efficiency of related indexes,this paper designs a classification method of capacity index data,which divides the capacity index data into trend type,periodic type and irregular type.Then for the prediction of trend data,it proposes a capacity index prediction model based on Recurrent Neural Network(RNN),denoted as RNN-LSTM-LSTM.This model includes a basic RNN,two Long Short-Term Memory(LSTM)networks and two Fully Connected layers.The experimental results show that,compared with the traditional Holt-Winters,Autoregressive Integrated Moving Average(ARIMA)and Back Propagation(BP)neural network prediction model,the mean square error(MSE)of the proposed RNN-LSTM-LSTM model are reduced by 11.82%and 20.34%on the order storage and data migration,which has greatly improved the efficiency of trend-type capacity index prediction.展开更多
In order to increase the accuracy rate of emotion recognition in voiceand video,the mixed convolutional neural network(CNN)and recurrent neural network(RNN)ae used to encode and integrate the two information sources.F...In order to increase the accuracy rate of emotion recognition in voiceand video,the mixed convolutional neural network(CNN)and recurrent neural network(RNN)ae used to encode and integrate the two information sources.For the audio signals,several frequency bands as well as some energy functions are extacted as low-level features by using a sophisticated audio technique,and then they are encoded w it a one-dimensional(I D)convolutional neural network to abstact high-level features.Finally,tiese are fed into a recurrent neural network for te sake of capturing dynamic tone changes in a temporal dimensionality.As a contrast,a two-dimensional(2D)convolutional neural network and a similar RNN are used to capture dynamic facial appearance changes of temporal sequences.The method was used in te Chinese Natral Audio-'Visual Emotion Database in te Chinese Conference on Pattern Recognition(CCPR)in2016.Experimental results demonstrate that te classification average precision of the proposed metiod is41.15%,which is increased by16.62%compaed with te baseline algorithm offered by the CCPR in2016.It is proved ta t te proposed method has higher accuracy in te identification of emotional information.展开更多
For training the present Neural Network(NN)models,the standard technique is to utilize decaying Learning Rates(LR).While the majority of these techniques commence with a large LR,they will decay multiple times over ti...For training the present Neural Network(NN)models,the standard technique is to utilize decaying Learning Rates(LR).While the majority of these techniques commence with a large LR,they will decay multiple times over time.Decaying has been proved to enhance generalization as well as optimization.Other parameters,such as the network’s size,the number of hidden layers,drop-outs to avoid overfitting,batch size,and so on,are solely based on heuristics.This work has proposed Adaptive Teaching Learning Based(ATLB)Heuristic to identify the optimal hyperparameters for diverse networks.Here we consider three architec-tures Recurrent Neural Networks(RNN),Long Short Term Memory(LSTM),Bidirectional Long Short Term Memory(BiLSTM)of Deep Neural Networks for classification.The evaluation of the proposed ATLB is done through the various learning rate schedulers Cyclical Learning Rate(CLR),Hyperbolic Tangent Decay(HTD),and Toggle between Hyperbolic Tangent Decay and Triangular mode with Restarts(T-HTR)techniques.Experimental results have shown the performance improvement on the 20Newsgroup,Reuters Newswire and IMDB dataset.展开更多
Kerr resonator is one of the most popular platforms to produce optical frequency comb and temporal cavity soliton.As an essential method for investigating the nonlinear dynamics of Kerr resonators,traditional numerica...Kerr resonator is one of the most popular platforms to produce optical frequency comb and temporal cavity soliton.As an essential method for investigating the nonlinear dynamics of Kerr resonators,traditional numerical simulations rely on solving the Lugiato-Lefever equation(LLE)using the split-step Fourier method(SSFM),which is computationally intensive and time-consuming.To address this challenge,this study proposes a recurrent neural network model with prior information feedback,enabling efficient and accurate prediction of soliton dynamics in Kerr resonator.With the acceleration of graphics processing unit(GPU),the computational efficiency improved by 20 times.We compared various recurrent neural networks and found that the gated recurrent unit(GRU)network demonstrated superior performance in this task.This work highlights the potential of artificial intelligence(AI)for modeling nonlinear optical dynamics in Kerr resonator,paving the way for designing optical frequency comb and generating ultrafast pulse.展开更多
Accurate pavement performance prediction plays a critical role in formulating maintenance and repair strategies for transportation departments,enabling the achievement of better pavement performance with limited finan...Accurate pavement performance prediction plays a critical role in formulating maintenance and repair strategies for transportation departments,enabling the achievement of better pavement performance with limited financial resources.However,due to the intricate influence of numerous factors on pavement performance deterioration,improving the accuracy of pavement performance prediction poses a challenge for conventional models.Therefore,the aim of this study is to establish a machine learning-based pavement performance prediction model.First,this study considers five factors that affect pavement performance,including pavement initial performance indicators,traffic loads,weather,pavement structure,and maintenance measures,and identifies 15 specific indicators that affect pavement performance based on these five factors.Then,based on the the long-term pavement performance(LTPP)database,the study screens and summarizes these indicators,obtaining 2464 high-quality pavement performance data for pavement conditions index(PCI)prediction and 3238 high-quality pavement performance data for international roughness index(IRI)prediction.Finally,three distinct prediction models are established,namely,the fully connected neural network(FCNN)model,the long short-term memory(LSTM)model,and the combined LSTM-attention model.The study shows that the LSTM-attention model performs significantly better than the FCNN and LSTM models,with an R2 coefficient of determination of 0.81 for PCI and 0.79 for IRI.The innovation of this paper is that the authors have introduced the attention mechanism on the basic of the LSTM model,which makes the fitting accuracy of the prediction model further improved.展开更多
With the popularity of smart handheld devices, mobile streaming video has multiplied the global network traffic in recent years. A huge concern of users' quality of experience(Qo E) has made rate adaptation method...With the popularity of smart handheld devices, mobile streaming video has multiplied the global network traffic in recent years. A huge concern of users' quality of experience(Qo E) has made rate adaptation methods very attractive. In this paper, we propose a two-phase rate adaptation strategy to improve users' real-time video Qo E. First, to measure and assess video Qo E, we provide a continuous Qo E prediction engine modeled by RNN recurrent neural network. Different from traditional Qo E models which consider the Qo E-aware factors separately or incompletely, our RNN-Qo E model accounts for three descriptive factors(video quality, rebuffering, and rate change) and reflects the impact of cognitive memory and recency. Besides, the video playing is separated into the initial startup phase and the steady playback phase, and we takes different optimization goals for each phase: the former aims at shortening the startup delay while the latter ameliorates the video quality and the rebufferings. Simulation results have shown that RNN-Qo E can follow the subjective Qo E quite well, and the proposed strategy can effectively reduce the occurrence of rebufferings caused by the mismatch between the requested video rates and the fluctuated throughput and attains standout performance on real-time Qo E compared with classical rate adaption methods.展开更多
Deep-Fake is an emerging technology used in synthetic media which manipulates individuals in existing images and videos with someone else’s likeness.This paper presents the comparative study of different deep neural ...Deep-Fake is an emerging technology used in synthetic media which manipulates individuals in existing images and videos with someone else’s likeness.This paper presents the comparative study of different deep neural networks employed for Deep-Fake video detection.In the model,the features from the training data are extracted with the intended Convolution Neural Network model to form feature vectors which are further analysed using a dense layer,a Long Short-Term Memoryand Gated Recurrent by adopting transfer learning with fine tuning for training the models.The model is evaluated to detect Artificial Intelligence based Deep fakes images and videos using benchmark datasets.Comparative analysis shows that the detections are majorly biased towards domain of the dataset but there is a noteworthy improvement in the model performance parameters by using Transfer Learning whereas Convolutional-Recurrent Neural Network has benefits in sequence detection.展开更多
In repeated zero-sum games,instead of constantly playing an equilibrium strategy of the stage game,learning to exploit the opponent given historical interactions could typically obtain a higher utility.However,when pl...In repeated zero-sum games,instead of constantly playing an equilibrium strategy of the stage game,learning to exploit the opponent given historical interactions could typically obtain a higher utility.However,when playing against a fully adaptive opponent,one would have dificulty identifying the opponent's adaptive dynamics and further exploiting its potential weakness.In this paper,we study the problem of optimizing against the adaptive opponent who uses no-regret learning.No-regret learning is a classic and widely-used branch of adaptive learning algorithms.We propose a general framework for online modeling no-regret opponents and exploiting their weakness.With this framework,one could approximate the opponent's no-regret learning dynamics and then develop a response plan to obtain a significant profit based on the inferences of the opponent's strategies.We employ two system identification architectures,including the recurrent neural network(RNN)and the nonlinear autoregressive exogenous model,and adopt an efficient greedy response plan within the framework.Theoretically,we prove the approximation capability of our RNN architecture at approximating specific no-regret dynamics.Empirically,we demonstrate that during interactions at a low level of non-stationarity,our architectures could approximate the dynamics with a low error,and the derived policies could exploit the no-regret opponent to obtain a decent utility.展开更多
Predicting player performance in sports is a critical challenge with significant implications for team success,fan engagement,and financial outcomes.Although,inMajor League Baseball(MLB),statistical methodologies such...Predicting player performance in sports is a critical challenge with significant implications for team success,fan engagement,and financial outcomes.Although,inMajor League Baseball(MLB),statistical methodologies such as sabermetrics have been widely used,the dynamic nature of sports makes accurate performance prediction a difficult task.Enhanced forecasts can provide immense value to team managers by aiding strategic player contract and acquisition decisions.This study addresses this challenge by employing the temporal fusion transformer(TFT),an advanced and cutting-edge deep learning model for complex data,to predict pitchers’earned run average(ERA),a key metric in baseball performance analysis.The performance of the TFT model is evaluated against recurrent neural network-based approaches and existing projection systems.In experimental results,the TFT based model consistently outperformed its counterparts,demonstrating superior accuracy in pitcher performance prediction.By leveraging the advanced capabilities of TFT,this study contributes to more precise player evaluations and improves strategic planning in baseball.展开更多
In this paper, an economic emission dispatch(EED) model is developed to reduce fuel cost and environmental pollution emissions. Considering the development of new energy sources in recent years, the EED problem involv...In this paper, an economic emission dispatch(EED) model is developed to reduce fuel cost and environmental pollution emissions. Considering the development of new energy sources in recent years, the EED problem involves thermal units with the valve point effect and WTs. Meanwhile, it complies with demand constraint and generator capacity constraints. A recurrent neural network(RNN) is proposed to search for local optimal solution of the introduced nonconvex EED problem. The optimality and convergence of the proposed dynamic model are given. The RNN algorithm is verified on a power generation system for the optimization of scheduling and minimization of total cost. Moreover, a particle swarm optimization(PSO) algorithm is compared with RNN under the same problematic frame. Numerical simulation results demonstrate that the optimal scheduling given by RNN is more precise and has lower total cost than PSO. In addition, the dynamic variation of power load demand is considered and the power distribution of eight generators during 12 time periods is depicted.展开更多
Tunnel boring machine(TBM) vibration induced by cutting complex ground contains essential information that can help engineers evaluate the interaction between a cutterhead and the ground itself.In this study,deep recu...Tunnel boring machine(TBM) vibration induced by cutting complex ground contains essential information that can help engineers evaluate the interaction between a cutterhead and the ground itself.In this study,deep recurrent neural networks(RNNs) and convolutional neural networks(CNNs) were used for vibration-based working face ground identification.First,field monitoring was conducted to obtain the TBM vibration data when tunneling in changing geological conditions,including mixed-face,homogeneous,and transmission ground.Next,RNNs and CNNs were utilized to develop vibration-based prediction models,which were then validated using the testing dataset.The accuracy of the long short-term memory(LSTM) and bidirectional LSTM(Bi-LSTM) models was approximately 70% with raw data;however,with instantaneous frequency transmission,the accuracy increased to approximately 80%.Two types of deep CNNs,GoogLeNet and ResNet,were trained and tested with time-frequency scalar diagrams from continuous wavelet transformation.The CNN models,with an accuracy greater than 96%,performed significantly better than the RNN models.The ResNet-18,with an accuracy of 98.28%,performed the best.When the sample length was set as the cutterhead rotation period,the deep CNN and RNN models achieved the highest accuracy while the proposed deep CNN model simultaneously achieved high prediction accuracy and feedback efficiency.The proposed model could promptly identify the ground conditions at the working face without stopping the normal tunneling process,and the TBM working parameters could be adjusted and optimized in a timely manner based on the predicted results.展开更多
This study presents a time series prediction model with output self feedback which is implemented based on online sequential extreme learning machine. The output variables derived from multilayer perception can feedba...This study presents a time series prediction model with output self feedback which is implemented based on online sequential extreme learning machine. The output variables derived from multilayer perception can feedback to the network input layer to create a temporal relation between the current node inputs and the lagged node outputs while overcoming the limitation of memory which is a vital port for any time-series prediction application. The model can overcome the static prediction problem with most time series prediction models and can effectively cope with the dynamic properties of time series data. A linear and a nonlinear forecasting algorithms based on online extreme learning machine are proposed to implement the output feedback forecasting model. They are both recursive estimator and have two distinct phases： Predict and Update. The proposed model was tested against different kinds of time series data and the results indicate that the model outperforms the original static model without feedback.展开更多
文摘Tunnel boring machines(TBMs)have been widely utilised in tunnel construction due to their high efficiency and reliability.Accurately predicting TBM performance can improve project time management,cost control,and risk management.This study aims to use deep learning to develop real-time models for predicting the penetration rate(PR).The models are built using data from the Changsha metro project,and their performances are evaluated using unseen data from the Zhengzhou Metro project.In one-step forecast,the predicted penetration rate follows the trend of the measured penetration rate in both training and testing.The autoregressive integrated moving average(ARIMA)model is compared with the recurrent neural network(RNN)model.The results show that univariate models,which only consider historical penetration rate itself,perform better than multivariate models that take into account multiple geological and operational parameters(GEO and OP).Next,an RNN variant combining time series of penetration rate with the last-step geological and operational parameters is developed,and it performs better than other models.A sensitivity analysis shows that the penetration rate is the most important parameter,while other parameters have a smaller impact on time series forecasting.It is also found that smoothed data are easier to predict with high accuracy.Nevertheless,over-simplified data can lose real characteristics in time series.In conclusion,the RNN variant can accurately predict the next-step penetration rate,and data smoothing is crucial in time series forecasting.This study provides practical guidance for TBM performance forecasting in practical engineering.
基金supported by the“Human Resources Program in Energy Technology”of the Korea Institute of Energy Technology Evaluation and Planning(KETEP)and granted financial resources from the Ministry of Trade,Industry,and Energy,Korea(No.20204010600090).
文摘This research aims to enhance Clinical Decision Support Systems(CDSS)within Wireless Body Area Networks(WBANs)by leveraging advanced machine learning techniques.Specifically,we target the challenges of accurate diagnosis in medical imaging and sequential data analysis using Recurrent Neural Networks(RNNs)with Long Short-Term Memory(LSTM)layers and echo state cells.These models are tailored to improve diagnostic precision,particularly for conditions like rotator cuff tears in osteoporosis patients and gastrointestinal diseases.Traditional diagnostic methods and existing CDSS frameworks often fall short in managing complex,sequential medical data,struggling with long-term dependencies and data imbalances,resulting in suboptimal accuracy and delayed decisions.Our goal is to develop Artificial Intelligence(AI)models that address these shortcomings,offering robust,real-time diagnostic support.We propose a hybrid RNN model that integrates SimpleRNN,LSTM layers,and echo state cells to manage long-term dependencies effectively.Additionally,we introduce CG-Net,a novel Convolutional Neural Network(CNN)framework for gastrointestinal disease classification,which outperforms traditional CNN models.We further enhance model performance through data augmentation and transfer learning,improving generalization and robustness against data scarcity and imbalance.Comprehensive validation,including 5-fold cross-validation and metrics such as accuracy,precision,recall,F1-score,and Area Under the Curve(AUC),confirms the models’reliability.Moreover,SHapley Additive exPlanations(SHAP)and Local Interpretable Model-agnostic Explanations(LIME)are employed to improve model interpretability.Our findings show that the proposed models significantly enhance diagnostic accuracy and efficiency,offering substantial advancements in WBANs and CDSS.
基金the Natural Science Foundation of China(NSFC)(61873024,61773053)the China Central Universities of USTB(FRF-TP-19-049A1Z)the National Key RD Program of China(2017YFB0306403)。
文摘Accurate estimation of the remaining useful life(RUL)and health state for rollers is of great significance to hot rolling production.It can provide decision support for roller management so as to improve the productivity of the hot rolling process.In addition,the RUL prediction for rollers is helpful in transitioning from the current regular maintenance strategy to conditional-based maintenance.Therefore,a new method that can extract coarse-grained and fine-grained features from batch data to predict the RUL of the rollers is proposed in this paper.Firstly,a new deep learning network architecture based on recurrent neural networks that can make full use of the extracted coarsegrained fine-grained features to estimate the heath indicator(HI)is developed,where the HI is able to indicate the health state of the roller.Following that,a state-space model is constructed to describe the HI,and the probabilistic distribution of RUL can be estimated by extrapolating the HI degradation model to a predefined failure threshold.Finally,application to a hot strip mill is given to verify the effectiveness of the proposed methods using data collected from an industrial site,and the relatively low RMSE and MAE values demonstrate its advantages compared with some other popular deep learning methods.
基金supported by National Natural Science Foundation of China (No. 60874025)Natural Science Foundation of Hunan Province of China (No. 10JJ6098)
文摘This paper deals with the stability of static recurrent neural networks （RNNs） with a time-varying delay. An augmented Lyapunov-Krasovskii functional is employed, in which some useful terms are included. Furthermore, the relationship among the timevarying delay, its upper bound and their difierence, is taken into account, and novel bounding techniques for 1- τ（t） are employed. As a result, without ignoring any useful term in the derivative of the Lyapunov-Krasovskii functional, the resulting delay-dependent criteria show less conservative than the existing ones. Finally, a numerical example is given to demonstrate the effectiveness of the proposed methods.
基金the National Natural Science Foundation of China (No. 60504024)the Research Project of Zhejiang Provin-cial Education Department (No. 20050905), China
文摘The robust exponential stability of a larger class of discrete-time recurrent neural networks (RNNs) is explored in this paper. A novel neural network model, named standard neural network model (SNNM), is introduced to provide a general framework for stability analysis of RNNs. Most of the existing RNNs can be transformed into SNNMs to be analyzed in a unified way. Applying Lyapunov stability theory method and S-Procedure technique, two useful criteria of robust exponential stability for the discrete-time SNNMs are derived. The conditions presented are formulated as linear matrix inequalities (LMIs) to be easily solved using existing efficient convex optimization techniques. An example is presented to demonstrate the transformation procedure and the effectiveness of the results.
基金Supported by the Natural Science Foundation of Shandong Province (ZR2010FM038,ZR2010FL017)
文摘The robust global exponential stability of a class of interval recurrent neural networks（RNNs） is studied,and a new robust stability criterion is obtained in the form of linear matrix inequality.The problem of robust stability of interval RNNs is transformed into a problem of solving a class of linear matrix inequalities.Thus,the robust stability of interval RNNs can be analyzed by directly using the linear matrix inequalities（LMI） toolbox of MATLAB.Numerical example is given to show the effectiveness of the obtained results.
基金Princess Nourah bint Abdulrahman University Researchers Supporting Project number(PNURSP2024R 343)PrincessNourah bint Abdulrahman University,Riyadh,Saudi ArabiaDeanship of Scientific Research at Northern Border University,Arar,Kingdom of Saudi Arabia,for funding this researchwork through the project number“NBU-FFR-2024-1092-02”.
文摘Phishing attacks present a persistent and evolving threat in the cybersecurity land-scape,necessitating the development of more sophisticated detection methods.Traditional machine learning approaches to phishing detection have relied heavily on feature engineering and have often fallen short in adapting to the dynamically changing patterns of phishingUniformResource Locator(URLs).Addressing these challenge,we introduce a framework that integrates the sequential data processing strengths of a Recurrent Neural Network(RNN)with the hyperparameter optimization prowess of theWhale Optimization Algorithm(WOA).Ourmodel capitalizes on an extensive Kaggle dataset,featuring over 11,000 URLs,each delineated by 30 attributes.The WOA’s hyperparameter optimization enhances the RNN’s performance,evidenced by a meticulous validation process.The results,encapsulated in precision,recall,and F1-score metrics,surpass baseline models,achieving an overall accuracy of 92%.This study not only demonstrates the RNN’s proficiency in learning complex patterns but also underscores the WOA’s effectiveness in refining machine learning models for the critical task of phishing detection.
基金supported by Research on Big Data Technology for New Generation Internet Operators(H04W180609)the second batch of Sichuan Science and Technology Service Industry Development Fund Projects in 2018(18KJFWSF0388).
文摘Due to the increase in the types of business and equipment in telecommunications companies,the performance index data collected in the operation and maintenance process varies greatly.The diversity of index data makes it very difficult to perform high-precision capacity prediction.In order to improve the forecasting efficiency of related indexes,this paper designs a classification method of capacity index data,which divides the capacity index data into trend type,periodic type and irregular type.Then for the prediction of trend data,it proposes a capacity index prediction model based on Recurrent Neural Network(RNN),denoted as RNN-LSTM-LSTM.This model includes a basic RNN,two Long Short-Term Memory(LSTM)networks and two Fully Connected layers.The experimental results show that,compared with the traditional Holt-Winters,Autoregressive Integrated Moving Average(ARIMA)and Back Propagation(BP)neural network prediction model,the mean square error(MSE)of the proposed RNN-LSTM-LSTM model are reduced by 11.82%and 20.34%on the order storage and data migration,which has greatly improved the efficiency of trend-type capacity index prediction.
文摘In order to increase the accuracy rate of emotion recognition in voiceand video,the mixed convolutional neural network(CNN)and recurrent neural network(RNN)ae used to encode and integrate the two information sources.For the audio signals,several frequency bands as well as some energy functions are extacted as low-level features by using a sophisticated audio technique,and then they are encoded w it a one-dimensional(I D)convolutional neural network to abstact high-level features.Finally,tiese are fed into a recurrent neural network for te sake of capturing dynamic tone changes in a temporal dimensionality.As a contrast,a two-dimensional(2D)convolutional neural network and a similar RNN are used to capture dynamic facial appearance changes of temporal sequences.The method was used in te Chinese Natral Audio-'Visual Emotion Database in te Chinese Conference on Pattern Recognition(CCPR)in2016.Experimental results demonstrate that te classification average precision of the proposed metiod is41.15%,which is increased by16.62%compaed with te baseline algorithm offered by the CCPR in2016.It is proved ta t te proposed method has higher accuracy in te identification of emotional information.
文摘For training the present Neural Network(NN)models,the standard technique is to utilize decaying Learning Rates(LR).While the majority of these techniques commence with a large LR,they will decay multiple times over time.Decaying has been proved to enhance generalization as well as optimization.Other parameters,such as the network’s size,the number of hidden layers,drop-outs to avoid overfitting,batch size,and so on,are solely based on heuristics.This work has proposed Adaptive Teaching Learning Based(ATLB)Heuristic to identify the optimal hyperparameters for diverse networks.Here we consider three architec-tures Recurrent Neural Networks(RNN),Long Short Term Memory(LSTM),Bidirectional Long Short Term Memory(BiLSTM)of Deep Neural Networks for classification.The evaluation of the proposed ATLB is done through the various learning rate schedulers Cyclical Learning Rate(CLR),Hyperbolic Tangent Decay(HTD),and Toggle between Hyperbolic Tangent Decay and Triangular mode with Restarts(T-HTR)techniques.Experimental results have shown the performance improvement on the 20Newsgroup,Reuters Newswire and IMDB dataset.
基金supported by the National Natural Science Foundation of China(Grant No.42327803)the Open Project Program of Wuhan National Laboratory for Optoelectronics(2023WNLOKF007)+4 种基金the Open Fund of the State Laboratory of Photonics and Communications(2025QZKF021)Technology Innovation Project of Hubei Province(2022BEC003)Key R&D Program of Hubei Province(2023BAB062)Major Science and Technology Projects of Wuhan(2023010302020030)Guangdong Basic and Applied Basic Research Foundation(2023A1515010965,2024A1515010017).
文摘Kerr resonator is one of the most popular platforms to produce optical frequency comb and temporal cavity soliton.As an essential method for investigating the nonlinear dynamics of Kerr resonators,traditional numerical simulations rely on solving the Lugiato-Lefever equation(LLE)using the split-step Fourier method(SSFM),which is computationally intensive and time-consuming.To address this challenge,this study proposes a recurrent neural network model with prior information feedback,enabling efficient and accurate prediction of soliton dynamics in Kerr resonator.With the acceleration of graphics processing unit(GPU),the computational efficiency improved by 20 times.We compared various recurrent neural networks and found that the gated recurrent unit(GRU)network demonstrated superior performance in this task.This work highlights the potential of artificial intelligence(AI)for modeling nonlinear optical dynamics in Kerr resonator,paving the way for designing optical frequency comb and generating ultrafast pulse.
基金supported by the Science and Technology Plan of Shandong Transportation Department(No.2021B47)the Key Research and Development Program of Ningxia Science and Technology Department(No.2022BEG02008)the Fundamental Research Funds for the Central Universities(No.22120210027).
文摘Accurate pavement performance prediction plays a critical role in formulating maintenance and repair strategies for transportation departments,enabling the achievement of better pavement performance with limited financial resources.However,due to the intricate influence of numerous factors on pavement performance deterioration,improving the accuracy of pavement performance prediction poses a challenge for conventional models.Therefore,the aim of this study is to establish a machine learning-based pavement performance prediction model.First,this study considers five factors that affect pavement performance,including pavement initial performance indicators,traffic loads,weather,pavement structure,and maintenance measures,and identifies 15 specific indicators that affect pavement performance based on these five factors.Then,based on the the long-term pavement performance(LTPP)database,the study screens and summarizes these indicators,obtaining 2464 high-quality pavement performance data for pavement conditions index(PCI)prediction and 3238 high-quality pavement performance data for international roughness index(IRI)prediction.Finally,three distinct prediction models are established,namely,the fully connected neural network(FCNN)model,the long short-term memory(LSTM)model,and the combined LSTM-attention model.The study shows that the LSTM-attention model performs significantly better than the FCNN and LSTM models,with an R2 coefficient of determination of 0.81 for PCI and 0.79 for IRI.The innovation of this paper is that the authors have introduced the attention mechanism on the basic of the LSTM model,which makes the fitting accuracy of the prediction model further improved.
基金supported by the National Nature Science Foundation of China(NSFC 60622110,61471220,91538107,91638205)National Basic Research Project of China(973,2013CB329006),GY22016058
文摘With the popularity of smart handheld devices, mobile streaming video has multiplied the global network traffic in recent years. A huge concern of users' quality of experience(Qo E) has made rate adaptation methods very attractive. In this paper, we propose a two-phase rate adaptation strategy to improve users' real-time video Qo E. First, to measure and assess video Qo E, we provide a continuous Qo E prediction engine modeled by RNN recurrent neural network. Different from traditional Qo E models which consider the Qo E-aware factors separately or incompletely, our RNN-Qo E model accounts for three descriptive factors(video quality, rebuffering, and rate change) and reflects the impact of cognitive memory and recency. Besides, the video playing is separated into the initial startup phase and the steady playback phase, and we takes different optimization goals for each phase: the former aims at shortening the startup delay while the latter ameliorates the video quality and the rebufferings. Simulation results have shown that RNN-Qo E can follow the subjective Qo E quite well, and the proposed strategy can effectively reduce the occurrence of rebufferings caused by the mismatch between the requested video rates and the fluctuated throughput and attains standout performance on real-time Qo E compared with classical rate adaption methods.
文摘Deep-Fake is an emerging technology used in synthetic media which manipulates individuals in existing images and videos with someone else’s likeness.This paper presents the comparative study of different deep neural networks employed for Deep-Fake video detection.In the model,the features from the training data are extracted with the intended Convolution Neural Network model to form feature vectors which are further analysed using a dense layer,a Long Short-Term Memoryand Gated Recurrent by adopting transfer learning with fine tuning for training the models.The model is evaluated to detect Artificial Intelligence based Deep fakes images and videos using benchmark datasets.Comparative analysis shows that the detections are majorly biased towards domain of the dataset but there is a noteworthy improvement in the model performance parameters by using Transfer Learning whereas Convolutional-Recurrent Neural Network has benefits in sequence detection.
基金the Science and Technology Innovation 2030-"New Generation Artificial Intelligence"Major Project(No.2018AAA0100901)。
文摘In repeated zero-sum games,instead of constantly playing an equilibrium strategy of the stage game,learning to exploit the opponent given historical interactions could typically obtain a higher utility.However,when playing against a fully adaptive opponent,one would have dificulty identifying the opponent's adaptive dynamics and further exploiting its potential weakness.In this paper,we study the problem of optimizing against the adaptive opponent who uses no-regret learning.No-regret learning is a classic and widely-used branch of adaptive learning algorithms.We propose a general framework for online modeling no-regret opponents and exploiting their weakness.With this framework,one could approximate the opponent's no-regret learning dynamics and then develop a response plan to obtain a significant profit based on the inferences of the opponent's strategies.We employ two system identification architectures,including the recurrent neural network(RNN)and the nonlinear autoregressive exogenous model,and adopt an efficient greedy response plan within the framework.Theoretically,we prove the approximation capability of our RNN architecture at approximating specific no-regret dynamics.Empirically,we demonstrate that during interactions at a low level of non-stationarity,our architectures could approximate the dynamics with a low error,and the derived policies could exploit the no-regret opponent to obtain a decent utility.
基金supported by SKKU Global Research Platform Research Fund,Sungkyunkwan University,2024-2025.
文摘Predicting player performance in sports is a critical challenge with significant implications for team success,fan engagement,and financial outcomes.Although,inMajor League Baseball(MLB),statistical methodologies such as sabermetrics have been widely used,the dynamic nature of sports makes accurate performance prediction a difficult task.Enhanced forecasts can provide immense value to team managers by aiding strategic player contract and acquisition decisions.This study addresses this challenge by employing the temporal fusion transformer(TFT),an advanced and cutting-edge deep learning model for complex data,to predict pitchers’earned run average(ERA),a key metric in baseball performance analysis.The performance of the TFT model is evaluated against recurrent neural network-based approaches and existing projection systems.In experimental results,the TFT based model consistently outperformed its counterparts,demonstrating superior accuracy in pitcher performance prediction.By leveraging the advanced capabilities of TFT,this study contributes to more precise player evaluations and improves strategic planning in baseball.
基金supported by the Fundamental Research Funds for the Central Universities (No. XDJK2019B010)the Natural Science Foundation of China(No. 61773320)+2 种基金the Natural Science of Chongqing Science and Technology Commission (CSTC)(No. cstc2018jcyj AX0583, No. cstc2018jcyj AX0810)the Research Foundation of Key Laboratory of Machine Perception and Children’s Intelligence Development funded by Chongqing University of Education (CQUE)(No. 16xjpt07)the Foundation of Chongqing University of Education (No. KY201702A)。
文摘In this paper, an economic emission dispatch(EED) model is developed to reduce fuel cost and environmental pollution emissions. Considering the development of new energy sources in recent years, the EED problem involves thermal units with the valve point effect and WTs. Meanwhile, it complies with demand constraint and generator capacity constraints. A recurrent neural network(RNN) is proposed to search for local optimal solution of the introduced nonconvex EED problem. The optimality and convergence of the proposed dynamic model are given. The RNN algorithm is verified on a power generation system for the optimization of scheduling and minimization of total cost. Moreover, a particle swarm optimization(PSO) algorithm is compared with RNN under the same problematic frame. Numerical simulation results demonstrate that the optimal scheduling given by RNN is more precise and has lower total cost than PSO. In addition, the dynamic variation of power load demand is considered and the power distribution of eight generators during 12 time periods is depicted.
基金supported by the National Natural Science Foundation of China(Grant No.52090082)the Natural Science Foundation of Shandong Province,China(Grant No.ZR2020ME243)the Shanghai Committee of Science and Technology(Grant No.19511100802)。
文摘Tunnel boring machine(TBM) vibration induced by cutting complex ground contains essential information that can help engineers evaluate the interaction between a cutterhead and the ground itself.In this study,deep recurrent neural networks(RNNs) and convolutional neural networks(CNNs) were used for vibration-based working face ground identification.First,field monitoring was conducted to obtain the TBM vibration data when tunneling in changing geological conditions,including mixed-face,homogeneous,and transmission ground.Next,RNNs and CNNs were utilized to develop vibration-based prediction models,which were then validated using the testing dataset.The accuracy of the long short-term memory(LSTM) and bidirectional LSTM(Bi-LSTM) models was approximately 70% with raw data;however,with instantaneous frequency transmission,the accuracy increased to approximately 80%.Two types of deep CNNs,GoogLeNet and ResNet,were trained and tested with time-frequency scalar diagrams from continuous wavelet transformation.The CNN models,with an accuracy greater than 96%,performed significantly better than the RNN models.The ResNet-18,with an accuracy of 98.28%,performed the best.When the sample length was set as the cutterhead rotation period,the deep CNN and RNN models achieved the highest accuracy while the proposed deep CNN model simultaneously achieved high prediction accuracy and feedback efficiency.The proposed model could promptly identify the ground conditions at the working face without stopping the normal tunneling process,and the TBM working parameters could be adjusted and optimized in a timely manner based on the predicted results.
基金Foundation item： the National Natural Science Foundation of China （No. 61203337）
文摘This study presents a time series prediction model with output self feedback which is implemented based on online sequential extreme learning machine. The output variables derived from multilayer perception can feedback to the network input layer to create a temporal relation between the current node inputs and the lagged node outputs while overcoming the limitation of memory which is a vital port for any time-series prediction application. The model can overcome the static prediction problem with most time series prediction models and can effectively cope with the dynamic properties of time series data. A linear and a nonlinear forecasting algorithms based on online extreme learning machine are proposed to implement the output feedback forecasting model. They are both recursive estimator and have two distinct phases： Predict and Update. The proposed model was tested against different kinds of time series data and the results indicate that the model outperforms the original static model without feedback.
