In this study, recurrent networks to downscale meteorological fields of the ERA-40 re-analysis dataset with focus on the meso-scale water balance were investigated. Therefore two types of recurrent neural networks wer...In this study, recurrent networks to downscale meteorological fields of the ERA-40 re-analysis dataset with focus on the meso-scale water balance were investigated. Therefore two types of recurrent neural networks were used. The first approach is a coupling between a recurrent neural network and a distributed watershed model and the second a nonlinear autoregressive with exogenous inputs (NARX) network, which directly predicted the component of the water balance. The approaches were deployed for a meso-scale catchment area in the Free State of Saxony, Germany. The results show that the coupled approach did not perform as well as the NARX network. But the meteorological output of the coupled approach already reaches an adequate quality. However the coupled model generates as input for the watershed model insufficient daily precipitation sums and not enough wet days were predicted. Hence the long-term annual cycle of the water balance could not be preserved with acceptable quality in contrary to the NARX approach. The residual storage change term indicates physical restrictions of the plausibility of the neural networks, whereas the physically based correlations among?the components of the water balance were preserved more accurately by the coupled approach.展开更多
Speech separation is an active research topic that plays an important role in numerous applications,such as speaker recognition,hearing pros-thesis,and autonomous robots.Many algorithms have been put forward to improv...Speech separation is an active research topic that plays an important role in numerous applications,such as speaker recognition,hearing pros-thesis,and autonomous robots.Many algorithms have been put forward to improve separation performance.However,speech separation in reverberant noisy environment is still a challenging task.To address this,a novel speech separation algorithm using gate recurrent unit(GRU)network based on microphone array has been proposed in this paper.The main aim of the proposed algorithm is to improve the separation performance and reduce the computational cost.The proposed algorithm extracts the sub-band steered response power-phase transform(SRP-PHAT)weighted by gammatone filter as the speech separation feature due to its discriminative and robust spatial position in formation.Since the GRU net work has the advantage of processing time series data with faster training speed and fewer training parameters,the GRU model is adopted to process the separation featuresof several sequential frames in the same sub-band to estimate the ideal Ratio Masking(IRM).The proposed algorithm decomposes the mixture signals into time-frequency(TF)units using gammatone filter bank in the frequency domain,and the target speech is reconstructed in the frequency domain by masking the mixture signal according to the estimated IRM.The operations of decomposing the mixture signal and reconstructing the target signal are completed in the frequency domain which can reduce the total computational cost.Experimental results demonstrate that the proposed algorithm realizes omnidirectional speech sep-aration in noisy and reverberant environments,provides good performance in terms of speech quality and intelligibility,and has the generalization capacity to reverberate.展开更多
The concept of synthetic lethality(SL)has been successfully used for targeted therapies.To further explore SL for cancer therapy,identifying more SL interactions with therapeutic potential are essential.Recently,graph...The concept of synthetic lethality(SL)has been successfully used for targeted therapies.To further explore SL for cancer therapy,identifying more SL interactions with therapeutic potential are essential.Recently,graph neural network-based deep learning methods have been proposed for SL prediction,which reduce the SL search space of wet-lab based methods.However,these methods ignore that most SL interactions depend strongly on genetic context,which limits the application of the predicted results.In this study,we proposed a graph recurrent network-based model for specific context-dependent SL prediction(SLGRN).In particular,we introduced a Graph Recurrent Network-based encoder to acquire a context-specific,low-dimensional feature representation for each node,facilitating the prediction of novel SL.SLGRN leveraged gate recurrent unit(GRU)and it incorporated a context-dependent-level state to effectively integrate information from all nodes.As a result,SLGRN outperforms the state-of-the-arts models for SL prediction.We subsequently validate novel SL interactions under different contexts based on combination therapy or patient survival analysis.Through in vitro experiments and retrospective clinical analysis,we emphasize the potential clinical significance of this context-specific SL prediction model.展开更多
In this paper,a new study concerning the usage of artificial neural networks in the control application is given.It is shown,that the data gathered during proper operation of a given control plant can be used in the l...In this paper,a new study concerning the usage of artificial neural networks in the control application is given.It is shown,that the data gathered during proper operation of a given control plant can be used in the learning process to fully embrace the control pattern.Interestingly,the instances driven by neural networks have the ability to outperform the original analytically driven scenarios.Three different control schemes,namely perfect,linear-quadratic,and generalized predictive controllers were used in the theoretical study.In addition,the nonlinear recurrent neural network-based generalized predictive controller with the radial basis function-originated predictor was obtained to exemplify the main results of the paper regarding the real-world application.展开更多
Electric vehicles(EVs)are gradually being deployed in the transportation sector.Although they have a high impact on reducing greenhouse gas emissions,their penetration is challenged by their random energy demand and d...Electric vehicles(EVs)are gradually being deployed in the transportation sector.Although they have a high impact on reducing greenhouse gas emissions,their penetration is challenged by their random energy demand and difficult scheduling of their optimal charging.To cope with these problems,this paper presents a novel approach for photovoltaic grid-connected microgrid EV charging station energy demand forecasting.The present study is part of a comprehensive framework involving emerging technologies such as drones and artificial intelligence designed to support the EVs’charging scheduling task.By using predictive algorithms for solar generation and load demand estimation,this approach aimed at ensuring dynamic and efficient energy flow between the solar energy source,the grid and the electric vehicles.The main contribution of this paper lies in developing an intelligent approach based on deep recurrent neural networks to forecast the energy demand using only its previous records.Therefore,various forecasters based on Long Short-term Memory,Gated Recurrent Unit,and their bi-directional and stacked variants were investigated using a real dataset collected from an EV charging station located at Trieste University(Italy).The developed forecasters have been evaluated and compared according to different metrics,including R,RMSE,MAE,and MAPE.We found that the obtained R values for both PV power generation and energy demand ranged between 97%and 98%.These study findings can be used for reliable and efficient decision-making on the management side of the optimal scheduling of the charging operations.展开更多
In the pharmaceutical industry,model-based prediction is a crucial stage in process development that allows pharmaceutical companies to simulate different scenarios toward improving process efficiency,reducing costs,a...In the pharmaceutical industry,model-based prediction is a crucial stage in process development that allows pharmaceutical companies to simulate different scenarios toward improving process efficiency,reducing costs,and enhancing product quality.Nevertheless,ensuring the quality of formulated pharmaceutical products through the management of raw material variations has always been a challenging task.In this work,data-driven chance-constrained recurrent neural networks(CCRNNs)are developed to address the issue arising from raw material uncertainty.Our goal is to explore how,by proactively incorporating uncertainty into the model training process,more accurate predictions and enhanced robustness can be realized.The proposed approach is tested on a fluid bed dryer(FBD)from a continuous pharmaceutical manufacturing pilot plant.The results demonstrate that CCRNN models offer more robust and accurate predictions for the critical quality attribute(CQA)-in this case,moisture content-when material variations occur,compared with conventional recurrent neural network-based models.展开更多
The converter steelmaking process represents a pivotal aspect of steel metallurgical production,with the characteristics of the flame at the furnace mouth serving as an indirect indicator of the internal smelting stag...The converter steelmaking process represents a pivotal aspect of steel metallurgical production,with the characteristics of the flame at the furnace mouth serving as an indirect indicator of the internal smelting stage.Effectively identifying and predicting the smelt-ing stage poses a significant challenge within industrial production.Traditional image-based methodologies,which rely on a single static flame image as input,demonstrate low recognition accuracy and inadequately extract the dynamic changes in smelting stage.To address this issue,the present study introduces an innovative recognition model that preprocesses flame video sequences from the furnace mouth and then employs a convolutional recurrent neural network(CRNN)to extract spatiotemporal features and derive recognition outputs.Ad-ditionally,we adopt feature layer visualization techniques to verify the model’s effectiveness and further enhance model performance by integrating the Bayesian optimization algorithm.The results indicate that the ResNet18 with convolutional block attention module(CBAM)in the convolutional layer demonstrates superior image feature extraction capabilities,achieving an accuracy of 90.70%and an area under the curve of 98.05%.The constructed Bayesian optimization-CRNN(BO-CRNN)model exhibits a significant improvement in comprehensive performance,with an accuracy of 97.01%and an area under the curve of 99.85%.Furthermore,statistics on the model’s average recognition time,computational complexity,and parameter quantity(Average recognition time:5.49 ms,floating-point opera-tions per second:18260.21 M(1 M=1×10^(6)),parameters:11.58 M)demonstrate superior performance.Through extensive repeated ex-periments on real-world datasets,the proposed CRNN model is capable of rapidly and accurately identifying smelting stages,offering a novel approach for converter smelting endpoint control.展开更多
Both the global exponential stability and the existence of periodic solutions for a class of recurrent neural networks with continuously distributed delays （RNNs） are studied. By employing the inequality α∏k=1^m ...Both the global exponential stability and the existence of periodic solutions for a class of recurrent neural networks with continuously distributed delays （RNNs） are studied. By employing the inequality α∏k=1^m bk^qk≤1/r ∑qkbk^r＋1/rα^r（α≥0,bk≥0,qk〉0,with ∑k=1^m qk=r-1,r≥1, constructing suitable Lyapunov r k=l k=l functions and applying the homeomorphism theory, a family of simple and new sufficient conditions are given ensuring the global exponential stability and the existence of periodic solutions of RNNs. The results extend and improve the results of earlier publications.展开更多
In social data analytics,Virtual Community(VC)detection is a primary challenge in discovering user relationships and enhancing social recommenda-tions.VC formation is used for personal interaction between communities....In social data analytics,Virtual Community(VC)detection is a primary challenge in discovering user relationships and enhancing social recommenda-tions.VC formation is used for personal interaction between communities.But the usual methods didn’t find the Suspicious Behaviour(SB)needed to make a VC.The Generalized Jaccard Suspicious Behavior Similarity-based Recurrent Deep Neural Network Classification and Ranking(GJSBS-RDNNCR)Model addresses these issues.The GJSBS-RDNNCR model comprises four layers for VC formation in Social Networks(SN).In the GJSBS-RDNNCR model,the SN is given as an input at the input layer.After that,the User’s Behaviors(UB)are extracted in the first Hidden Layer(HL),and the Generalized Jaccard Similarity coefficient calculates the similarity value at the second HL based on the SB.In the third HL,the similarity values are examined,and SB tendency is classified using the Activation Function(AF)in the Output Layer(OL).Finally,the ranking process is performed with classified users in SN and their SB.Results analysis is performed with metrics such as Classification Accuracy(CA),Time Complexity(TC),and False Positive Rate(FPR).The experimental setup consid-ers 250 tweet users from the dataset to identify the SBs of users.展开更多
Knowledge of pore-water pressure(PWP)variation is fundamental for slope stability.A precise prediction of PWP is difficult due to complex physical mechanisms and in situ natural variability.To explore the applicabilit...Knowledge of pore-water pressure(PWP)variation is fundamental for slope stability.A precise prediction of PWP is difficult due to complex physical mechanisms and in situ natural variability.To explore the applicability and advantages of recurrent neural networks(RNNs)on PWP prediction,three variants of RNNs,i.e.,standard RNN,long short-term memory(LSTM)and gated recurrent unit(GRU)are adopted and compared with a traditional static artificial neural network(ANN),i.e.,multi-layer perceptron(MLP).Measurements of rainfall and PWP of representative piezometers from a fully instrumented natural slope in Hong Kong are used to establish the prediction models.The coefficient of determination(R^2)and root mean square error(RMSE)are used for model evaluations.The influence of input time series length on the model performance is investigated.The results reveal that MLP can provide acceptable performance but is not robust.The uncertainty bounds of RMSE of the MLP model range from 0.24 kPa to 1.12 k Pa for the selected two piezometers.The standard RNN can perform better but the robustness is slightly affected when there are significant time lags between PWP changes and rainfall.The GRU and LSTM models can provide more precise and robust predictions than the standard RNN.The effects of the hidden layer structure and the dropout technique are investigated.The single-layer GRU is accurate enough for PWP prediction,whereas a double-layer GRU brings extra time cost with little accuracy improvement.The dropout technique is essential to overfitting prevention and improvement of accuracy.展开更多
Recurrent neural networks （RNN） have been very successful in handling sequence data. However, understanding RNN and finding the best practices for RNN learning is a difficult task, partly because there are many comp...Recurrent neural networks （RNN） have been very successful in handling sequence data. However, understanding RNN and finding the best practices for RNN learning is a difficult task, partly because there are many competing and complex hidden units, such as the long short-term memory （LSTM） and the gated recurrent unit （GRU）. We propose a gated unit for RNN, named as minimal gated unit （MCU）, since it only contains one gate, which is a minimal design among all gated hidden units. The design of MCU benefits from evaluation results on LSTM and GRU in the literature. Experiments on various sequence data show that MCU has comparable accuracy with GRU, but has a simpler structure, fewer parameters, and faster training. Hence, MGU is suitable in RNN＇s applications. Its simple architecture also means that it is easier to evaluate and tune, and in principle it is easier to study MGU＇s properties theoretically and empirically.展开更多
This paper proposes a co-evolutionary recurrent neural network （CERNN） for the multi-step-prediction of chaotic time series, it estimates the proper parameters of phase space reconstruction and optimizes the structu...This paper proposes a co-evolutionary recurrent neural network （CERNN） for the multi-step-prediction of chaotic time series, it estimates the proper parameters of phase space reconstruction and optimizes the structure of recurrent neural networks by coevolutionary strategy. The searching space was separated into two subspaces and the individuals are trained in a parallel computational procedure. It can dynamically combine the embedding method with the capability of recurrent neural network to incorporate past experience due to internal recurrence. The effectiveness of CERNN is evaluated by using three benchmark chaotic time series data sets： the Lorenz series, Mackey-Glass series and real-world sun spot series. The simulation results show that CERNN improves the performances of multi-step-prediction of chaotic time series.展开更多
The task of detecting fraud in credit card transactions is crucial to ensure the security and stability of a financial system,as well as to enforce customer confidence in digital payment systems.Historically,credit ca...The task of detecting fraud in credit card transactions is crucial to ensure the security and stability of a financial system,as well as to enforce customer confidence in digital payment systems.Historically,credit card companies have used rulebased approaches to detect fraudulent transactions,but these have proven inadequate due to the complexity of fraud strategies and have been replaced by much more powerful solutions based on machine learning or deep learning algorithms.Despite significant progress,the current approaches to fraud detection suffer from a number of limitations:for example,it is unclear whether some transaction features are more effective than others in discriminating fraudulent transactions,and they often neglect possible correlations among transactions,even though they could reveal illicit behaviour.In this paper,we propose a novel credit card fraud detection(CCFD)method based on a transaction behaviour-based hierarchical gated network.First,we introduce a feature-oriented extraction module capable of identifying key features from original transactions,and such analysis is effective in revealing the behavioural characteristics of fraudsters.Second,we design a transaction-oriented extraction module capable of capturing the correlation between users’historical and current transactional behaviour.Such information is crucial for revealing users’sequential behaviour patterns.Our approach,called transactional-behaviour-based hierarchical gated network model(TbHGN),extracts two types of new transactional features,which are then combined in a feature interaction module to learn the final transactional representations used for CCFD.We have conducted extensive experiments on a real-world credit card transaction dataset with an increase in average F1 between 1.42%and 6.53%and an improvement in average AUC between 0.63%and 2.78%over the state of the art.展开更多
Accurate estimation of the remaining useful life(RUL)and health state for rollers is of great significance to hot rolling production.It can provide decision support for roller management so as to improve the productiv...Accurate estimation of the remaining useful life(RUL)and health state for rollers is of great significance to hot rolling production.It can provide decision support for roller management so as to improve the productivity of the hot rolling process.In addition,the RUL prediction for rollers is helpful in transitioning from the current regular maintenance strategy to conditional-based maintenance.Therefore,a new method that can extract coarse-grained and fine-grained features from batch data to predict the RUL of the rollers is proposed in this paper.Firstly,a new deep learning network architecture based on recurrent neural networks that can make full use of the extracted coarsegrained fine-grained features to estimate the heath indicator(HI)is developed,where the HI is able to indicate the health state of the roller.Following that,a state-space model is constructed to describe the HI,and the probabilistic distribution of RUL can be estimated by extrapolating the HI degradation model to a predefined failure threshold.Finally,application to a hot strip mill is given to verify the effectiveness of the proposed methods using data collected from an industrial site,and the relatively low RMSE and MAE values demonstrate its advantages compared with some other popular deep learning methods.展开更多
Early fault warning for nuclear power machinery is conducive to timely troubleshooting and reductions in safety risks and unnecessary costs. This paper presents a novel intelligent fault prediction method, integrated ...Early fault warning for nuclear power machinery is conducive to timely troubleshooting and reductions in safety risks and unnecessary costs. This paper presents a novel intelligent fault prediction method, integrated probabilistic principal component analysis(PPCA), multi-resolution wavelet analysis, Bayesian inference, and RNN model for nuclear power machinery that consider data uncertainty and chaotic time series. After denoising the source data, the Bayesian PPCA method is employed for dimensional reduction to obtain a refined data group. A recurrent neural network(RNN) prediction model is constructed, and a Bayesian statistical inference approach is developed to quantitatively assess the prediction reliability of the model. By modeling and analyzing the data collected on the steam turbine and components of a nuclear power plant, the results of the goodness of fit, mean square error distribution, and Bayesian confidence indicate that the proposed RNN model can implement early warning in the fault creep period. The accuracy and reliability of the proposed model are quantitatively verified.展开更多
In order to accurately forecast the load of power system and enhance the stability of the power network, a novel unascertained mathematics based recurrent neural network (UMRNN) for power intelligence center (PIC) was...In order to accurately forecast the load of power system and enhance the stability of the power network, a novel unascertained mathematics based recurrent neural network (UMRNN) for power intelligence center (PIC) was created through three steps. First, by combining with the general project uncertain element transmission theory (GPUET), the basic definitions of stochastic, fuzzy, and grey uncertain elements were given based on the principal types of uncertain information. Second, a power dynamic alliance including four sectors: generation sector, transmission sector, distribution sector and customers was established. The key factors were amended according to the four transmission topologies of uncertain elements, thus the new factors entered the power intelligence center as the input elements. Finally, in the intelligence handing background of PIC, by performing uncertain and recursive process to the input values of network, and combining unascertained mathematics, the novel load forecasting model was built. Three different approaches were put forward to forecast an eastern regional power grid load in China. The root mean square error (ERMS) demonstrates that the forecasting accuracy of the proposed model UMRNN is 3% higher than that of BP neural network (BPNN), and 5% higher than that of autoregressive integrated moving average (ARIMA). Besides, an example also shows that the average relative error of the first quarter of 2008 forecasted by UMRNN is only 2.59%, which has high precision.展开更多
The Global Positioning System(GPS)has become a foundation for most location-based services and navigation systems,such as autonomous vehicles,drones,ships,and wearable devices.However,it is a challenge to verify if th...The Global Positioning System(GPS)has become a foundation for most location-based services and navigation systems,such as autonomous vehicles,drones,ships,and wearable devices.However,it is a challenge to verify if the reported geographic locations are valid due to various GPS spoofing tools.Pervasive tools,such as Fake GPS,Lockito,and software-defined radio,enable ordinary users to hijack and report fake GPS coordinates and cheat the monitoring server without being detected.Furthermore,it is also a challenge to get accurate sensor readings on mobile devices because of the high noise level introduced by commercial motion sensors.To this end,we propose DeepPOSE,a deep learning model,to address the noise introduced in sensor readings and detect GPS spoofing attacks on mobile platforms.Our design uses a convolutional and recurrent neural network to reduce the noise,to recover a vehicle's real-time trajectory from multiple sensor inputs.We further propose a novel scheme to map the constructed trajectory from sensor readings onto the Google map,to smartly eliminate the accumulation of errors on the trajectory estimation.The reconstructed trajectory from sensors is then used to detect the GPS spoofing attack.Compared with the existing method,the proposed approach demonstrates a significantly higher degree of accuracy for detecting GPS spoofing attacks.展开更多
A multilayer recurrent fuzzy neural network(MRFNN)is proposed for accurate dynamic system modeling.The proposed MRFNN has six layers combined with T-S fuzzy model.The recurrent structures are formed by local feedback ...A multilayer recurrent fuzzy neural network(MRFNN)is proposed for accurate dynamic system modeling.The proposed MRFNN has six layers combined with T-S fuzzy model.The recurrent structures are formed by local feedback connections in the membership layer and the rule layer.With these feedbacks,the fuzzy sets are time-varying and the temporal problem of dynamic system can be solved well.The parameters of MRFNN are learned by chaotic search(CS)and least square estimation(LSE)simultaneously,where CS is for tuning the premise parameters and LSE is for updating the consequent coefficients accordingly.Results of simulations show the proposed approach is effective for dynamic system modeling with high accuracy.展开更多
A vast amount of information has been produced in recent years,which brings a huge challenge to information management.The better usage of big data is of important theoretical and practical significance for effectivel...A vast amount of information has been produced in recent years,which brings a huge challenge to information management.The better usage of big data is of important theoretical and practical significance for effectively addressing and managing messages.In this paper,we propose a nine-rectangle-grid information model according to the information value and privacy,and then present information use policies based on the rough set theory.Recurrent neural networks were employed to classify OTT messages.The content of user interest is effectively incorporated into the classification process during the annotation of OTT messages,ending with a reliable trained classification model.Experimental results showed that the proposed method yielded an accurate classification performance and hence can be used for effective distribution and control of OTT messages.展开更多
The Gated Recurrent Unit(GRU) neural network has great potential in estimating and predicting a variable. In addition to radar reflectivity(Z), radar echo-top height(ET) is also a good indicator of rainfall rate(R). I...The Gated Recurrent Unit(GRU) neural network has great potential in estimating and predicting a variable. In addition to radar reflectivity(Z), radar echo-top height(ET) is also a good indicator of rainfall rate(R). In this study, we propose a new method, GRU_Z-ET, by introducing Z and ET as two independent variables into the GRU neural network to conduct the quantitative single-polarization radar precipitation estimation. The performance of GRU_Z-ET is compared with that of the other three methods in three heavy rainfall cases in China during 2018, namely, the traditional Z-R relationship(Z=300R1.4), the optimal Z-R relationship(Z=79R1.68) and the GRU neural network with only Z as the independent input variable(GRU_Z). The results indicate that the GRU_Z-ET performs the best, while the traditional Z-R relationship performs the worst. The performances of the rest two methods are similar.To further evaluate the performance of the GRU_Z-ET, 200 rainfall events with 21882 total samples during May–July of 2018 are used for statistical analysis. Results demonstrate that the spatial correlation coefficients, threat scores and probability of detection between the observed and estimated precipitation are the largest for the GRU_Z-ET and the smallest for the traditional Z-R relationship, and the root mean square error is just the opposite. In addition, these statistics of GRU_Z are similar to those of optimal Z-R relationship. Thus, it can be concluded that the performance of the GRU_ZET is the best in the four methods for the quantitative precipitation estimation.展开更多
基金supported by the Erasmus Mundus Action 2 Programme of the European Union and the German Weather Service(DWD)and the Czech Hydrological-Meteorological Service(CHMI).
文摘In this study, recurrent networks to downscale meteorological fields of the ERA-40 re-analysis dataset with focus on the meso-scale water balance were investigated. Therefore two types of recurrent neural networks were used. The first approach is a coupling between a recurrent neural network and a distributed watershed model and the second a nonlinear autoregressive with exogenous inputs (NARX) network, which directly predicted the component of the water balance. The approaches were deployed for a meso-scale catchment area in the Free State of Saxony, Germany. The results show that the coupled approach did not perform as well as the NARX network. But the meteorological output of the coupled approach already reaches an adequate quality. However the coupled model generates as input for the watershed model insufficient daily precipitation sums and not enough wet days were predicted. Hence the long-term annual cycle of the water balance could not be preserved with acceptable quality in contrary to the NARX approach. The residual storage change term indicates physical restrictions of the plausibility of the neural networks, whereas the physically based correlations among?the components of the water balance were preserved more accurately by the coupled approach.
基金This work is supported by Nanjing Institute of Technology(NIT)fund for Research Startup Projects of Introduced talents under Grant No.YKJ202019Nature Sci-ence Research Project of Higher Education Institutions in Jiangsu Province under Grant No.21KJB510018+1 种基金National Nature Science Foundation of China(NSFC)under Grant No.62001215NIT fund for Doctoral Research Projects under Grant No.ZKJ2020003.
文摘Speech separation is an active research topic that plays an important role in numerous applications,such as speaker recognition,hearing pros-thesis,and autonomous robots.Many algorithms have been put forward to improve separation performance.However,speech separation in reverberant noisy environment is still a challenging task.To address this,a novel speech separation algorithm using gate recurrent unit(GRU)network based on microphone array has been proposed in this paper.The main aim of the proposed algorithm is to improve the separation performance and reduce the computational cost.The proposed algorithm extracts the sub-band steered response power-phase transform(SRP-PHAT)weighted by gammatone filter as the speech separation feature due to its discriminative and robust spatial position in formation.Since the GRU net work has the advantage of processing time series data with faster training speed and fewer training parameters,the GRU model is adopted to process the separation featuresof several sequential frames in the same sub-band to estimate the ideal Ratio Masking(IRM).The proposed algorithm decomposes the mixture signals into time-frequency(TF)units using gammatone filter bank in the frequency domain,and the target speech is reconstructed in the frequency domain by masking the mixture signal according to the estimated IRM.The operations of decomposing the mixture signal and reconstructing the target signal are completed in the frequency domain which can reduce the total computational cost.Experimental results demonstrate that the proposed algorithm realizes omnidirectional speech sep-aration in noisy and reverberant environments,provides good performance in terms of speech quality and intelligibility,and has the generalization capacity to reverberate.
基金supported by the National Key Research and Development Program of China(2023YFC2604400)the National Natural Science Foundation of China(62103436)。
文摘The concept of synthetic lethality(SL)has been successfully used for targeted therapies.To further explore SL for cancer therapy,identifying more SL interactions with therapeutic potential are essential.Recently,graph neural network-based deep learning methods have been proposed for SL prediction,which reduce the SL search space of wet-lab based methods.However,these methods ignore that most SL interactions depend strongly on genetic context,which limits the application of the predicted results.In this study,we proposed a graph recurrent network-based model for specific context-dependent SL prediction(SLGRN).In particular,we introduced a Graph Recurrent Network-based encoder to acquire a context-specific,low-dimensional feature representation for each node,facilitating the prediction of novel SL.SLGRN leveraged gate recurrent unit(GRU)and it incorporated a context-dependent-level state to effectively integrate information from all nodes.As a result,SLGRN outperforms the state-of-the-arts models for SL prediction.We subsequently validate novel SL interactions under different contexts based on combination therapy or patient survival analysis.Through in vitro experiments and retrospective clinical analysis,we emphasize the potential clinical significance of this context-specific SL prediction model.
文摘In this paper,a new study concerning the usage of artificial neural networks in the control application is given.It is shown,that the data gathered during proper operation of a given control plant can be used in the learning process to fully embrace the control pattern.Interestingly,the instances driven by neural networks have the ability to outperform the original analytically driven scenarios.Three different control schemes,namely perfect,linear-quadratic,and generalized predictive controllers were used in the theoretical study.In addition,the nonlinear recurrent neural network-based generalized predictive controller with the radial basis function-originated predictor was obtained to exemplify the main results of the paper regarding the real-world application.
基金University of Jeddah,Jeddah,Saudi Arabia,grant No.(UJ-23-SRP-10).
文摘Electric vehicles(EVs)are gradually being deployed in the transportation sector.Although they have a high impact on reducing greenhouse gas emissions,their penetration is challenged by their random energy demand and difficult scheduling of their optimal charging.To cope with these problems,this paper presents a novel approach for photovoltaic grid-connected microgrid EV charging station energy demand forecasting.The present study is part of a comprehensive framework involving emerging technologies such as drones and artificial intelligence designed to support the EVs’charging scheduling task.By using predictive algorithms for solar generation and load demand estimation,this approach aimed at ensuring dynamic and efficient energy flow between the solar energy source,the grid and the electric vehicles.The main contribution of this paper lies in developing an intelligent approach based on deep recurrent neural networks to forecast the energy demand using only its previous records.Therefore,various forecasters based on Long Short-term Memory,Gated Recurrent Unit,and their bi-directional and stacked variants were investigated using a real dataset collected from an EV charging station located at Trieste University(Italy).The developed forecasters have been evaluated and compared according to different metrics,including R,RMSE,MAE,and MAPE.We found that the obtained R values for both PV power generation and energy demand ranged between 97%and 98%.These study findings can be used for reliable and efficient decision-making on the management side of the optimal scheduling of the charging operations.
基金Financial support from the Engineering and Physical Sciences Research Council grant EP/V034723/1(RiFTMaP)is gratefully acknowledged.
文摘In the pharmaceutical industry,model-based prediction is a crucial stage in process development that allows pharmaceutical companies to simulate different scenarios toward improving process efficiency,reducing costs,and enhancing product quality.Nevertheless,ensuring the quality of formulated pharmaceutical products through the management of raw material variations has always been a challenging task.In this work,data-driven chance-constrained recurrent neural networks(CCRNNs)are developed to address the issue arising from raw material uncertainty.Our goal is to explore how,by proactively incorporating uncertainty into the model training process,more accurate predictions and enhanced robustness can be realized.The proposed approach is tested on a fluid bed dryer(FBD)from a continuous pharmaceutical manufacturing pilot plant.The results demonstrate that CCRNN models offer more robust and accurate predictions for the critical quality attribute(CQA)-in this case,moisture content-when material variations occur,compared with conventional recurrent neural network-based models.
基金financially supported by the National Natural Science Foundation of China(No.52374320).
文摘The converter steelmaking process represents a pivotal aspect of steel metallurgical production,with the characteristics of the flame at the furnace mouth serving as an indirect indicator of the internal smelting stage.Effectively identifying and predicting the smelt-ing stage poses a significant challenge within industrial production.Traditional image-based methodologies,which rely on a single static flame image as input,demonstrate low recognition accuracy and inadequately extract the dynamic changes in smelting stage.To address this issue,the present study introduces an innovative recognition model that preprocesses flame video sequences from the furnace mouth and then employs a convolutional recurrent neural network(CRNN)to extract spatiotemporal features and derive recognition outputs.Ad-ditionally,we adopt feature layer visualization techniques to verify the model’s effectiveness and further enhance model performance by integrating the Bayesian optimization algorithm.The results indicate that the ResNet18 with convolutional block attention module(CBAM)in the convolutional layer demonstrates superior image feature extraction capabilities,achieving an accuracy of 90.70%and an area under the curve of 98.05%.The constructed Bayesian optimization-CRNN(BO-CRNN)model exhibits a significant improvement in comprehensive performance,with an accuracy of 97.01%and an area under the curve of 99.85%.Furthermore,statistics on the model’s average recognition time,computational complexity,and parameter quantity(Average recognition time:5.49 ms,floating-point opera-tions per second:18260.21 M(1 M=1×10^(6)),parameters:11.58 M)demonstrate superior performance.Through extensive repeated ex-periments on real-world datasets,the proposed CRNN model is capable of rapidly and accurately identifying smelting stages,offering a novel approach for converter smelting endpoint control.
文摘Both the global exponential stability and the existence of periodic solutions for a class of recurrent neural networks with continuously distributed delays （RNNs） are studied. By employing the inequality α∏k=1^m bk^qk≤1/r ∑qkbk^r＋1/rα^r（α≥0,bk≥0,qk〉0,with ∑k=1^m qk=r-1,r≥1, constructing suitable Lyapunov r k=l k=l functions and applying the homeomorphism theory, a family of simple and new sufficient conditions are given ensuring the global exponential stability and the existence of periodic solutions of RNNs. The results extend and improve the results of earlier publications.
文摘In social data analytics,Virtual Community(VC)detection is a primary challenge in discovering user relationships and enhancing social recommenda-tions.VC formation is used for personal interaction between communities.But the usual methods didn’t find the Suspicious Behaviour(SB)needed to make a VC.The Generalized Jaccard Suspicious Behavior Similarity-based Recurrent Deep Neural Network Classification and Ranking(GJSBS-RDNNCR)Model addresses these issues.The GJSBS-RDNNCR model comprises four layers for VC formation in Social Networks(SN).In the GJSBS-RDNNCR model,the SN is given as an input at the input layer.After that,the User’s Behaviors(UB)are extracted in the first Hidden Layer(HL),and the Generalized Jaccard Similarity coefficient calculates the similarity value at the second HL based on the SB.In the third HL,the similarity values are examined,and SB tendency is classified using the Activation Function(AF)in the Output Layer(OL).Finally,the ranking process is performed with classified users in SN and their SB.Results analysis is performed with metrics such as Classification Accuracy(CA),Time Complexity(TC),and False Positive Rate(FPR).The experimental setup consid-ers 250 tweet users from the dataset to identify the SBs of users.
基金supported by the Natural Science Foundation of China(Grant Nos.51979158,51639008,51679135,and 51422905)the Program of Shanghai Academic Research Leader by Science and Technology Commission of Shanghai Municipality(Project No.19XD1421900)。
文摘Knowledge of pore-water pressure(PWP)variation is fundamental for slope stability.A precise prediction of PWP is difficult due to complex physical mechanisms and in situ natural variability.To explore the applicability and advantages of recurrent neural networks(RNNs)on PWP prediction,three variants of RNNs,i.e.,standard RNN,long short-term memory(LSTM)and gated recurrent unit(GRU)are adopted and compared with a traditional static artificial neural network(ANN),i.e.,multi-layer perceptron(MLP).Measurements of rainfall and PWP of representative piezometers from a fully instrumented natural slope in Hong Kong are used to establish the prediction models.The coefficient of determination(R^2)and root mean square error(RMSE)are used for model evaluations.The influence of input time series length on the model performance is investigated.The results reveal that MLP can provide acceptable performance but is not robust.The uncertainty bounds of RMSE of the MLP model range from 0.24 kPa to 1.12 k Pa for the selected two piezometers.The standard RNN can perform better but the robustness is slightly affected when there are significant time lags between PWP changes and rainfall.The GRU and LSTM models can provide more precise and robust predictions than the standard RNN.The effects of the hidden layer structure and the dropout technique are investigated.The single-layer GRU is accurate enough for PWP prediction,whereas a double-layer GRU brings extra time cost with little accuracy improvement.The dropout technique is essential to overfitting prevention and improvement of accuracy.
基金supported by National Natural Science Foundation of China(Nos.61422203 and 61333014)National Key Basic Research Program of China(No.2014CB340501)
文摘Recurrent neural networks （RNN） have been very successful in handling sequence data. However, understanding RNN and finding the best practices for RNN learning is a difficult task, partly because there are many competing and complex hidden units, such as the long short-term memory （LSTM） and the gated recurrent unit （GRU）. We propose a gated unit for RNN, named as minimal gated unit （MCU）, since it only contains one gate, which is a minimal design among all gated hidden units. The design of MCU benefits from evaluation results on LSTM and GRU in the literature. Experiments on various sequence data show that MCU has comparable accuracy with GRU, but has a simpler structure, fewer parameters, and faster training. Hence, MGU is suitable in RNN＇s applications. Its simple architecture also means that it is easier to evaluate and tune, and in principle it is easier to study MGU＇s properties theoretically and empirically.
基金Project supported by the State Key Program of National Natural Science of China (Grant No 30230350)the Natural Science Foundation of Guangdong Province,China (Grant No 07006474)
文摘This paper proposes a co-evolutionary recurrent neural network （CERNN） for the multi-step-prediction of chaotic time series, it estimates the proper parameters of phase space reconstruction and optimizes the structure of recurrent neural networks by coevolutionary strategy. The searching space was separated into two subspaces and the individuals are trained in a parallel computational procedure. It can dynamically combine the embedding method with the capability of recurrent neural network to incorporate past experience due to internal recurrence. The effectiveness of CERNN is evaluated by using three benchmark chaotic time series data sets： the Lorenz series, Mackey-Glass series and real-world sun spot series. The simulation results show that CERNN improves the performances of multi-step-prediction of chaotic time series.
基金supported in part by the National Natural Science Foundation of China(61972241)the Natural Science Foundation of Shanghai(24ZR1427500,22ZR1427100)+1 种基金the Key Projects of Natural Science Research in Anhui Higher Education Institutions(2022AH051909)Bengbu University 2021 High-Level Scientific Research and Cultivation Project(2021pyxm04).
文摘The task of detecting fraud in credit card transactions is crucial to ensure the security and stability of a financial system,as well as to enforce customer confidence in digital payment systems.Historically,credit card companies have used rulebased approaches to detect fraudulent transactions,but these have proven inadequate due to the complexity of fraud strategies and have been replaced by much more powerful solutions based on machine learning or deep learning algorithms.Despite significant progress,the current approaches to fraud detection suffer from a number of limitations:for example,it is unclear whether some transaction features are more effective than others in discriminating fraudulent transactions,and they often neglect possible correlations among transactions,even though they could reveal illicit behaviour.In this paper,we propose a novel credit card fraud detection(CCFD)method based on a transaction behaviour-based hierarchical gated network.First,we introduce a feature-oriented extraction module capable of identifying key features from original transactions,and such analysis is effective in revealing the behavioural characteristics of fraudsters.Second,we design a transaction-oriented extraction module capable of capturing the correlation between users’historical and current transactional behaviour.Such information is crucial for revealing users’sequential behaviour patterns.Our approach,called transactional-behaviour-based hierarchical gated network model(TbHGN),extracts two types of new transactional features,which are then combined in a feature interaction module to learn the final transactional representations used for CCFD.We have conducted extensive experiments on a real-world credit card transaction dataset with an increase in average F1 between 1.42%and 6.53%and an improvement in average AUC between 0.63%and 2.78%over the state of the art.
基金the Natural Science Foundation of China(NSFC)(61873024,61773053)the China Central Universities of USTB(FRF-TP-19-049A1Z)the National Key RD Program of China(2017YFB0306403)。
文摘Accurate estimation of the remaining useful life(RUL)and health state for rollers is of great significance to hot rolling production.It can provide decision support for roller management so as to improve the productivity of the hot rolling process.In addition,the RUL prediction for rollers is helpful in transitioning from the current regular maintenance strategy to conditional-based maintenance.Therefore,a new method that can extract coarse-grained and fine-grained features from batch data to predict the RUL of the rollers is proposed in this paper.Firstly,a new deep learning network architecture based on recurrent neural networks that can make full use of the extracted coarsegrained fine-grained features to estimate the heath indicator(HI)is developed,where the HI is able to indicate the health state of the roller.Following that,a state-space model is constructed to describe the HI,and the probabilistic distribution of RUL can be estimated by extrapolating the HI degradation model to a predefined failure threshold.Finally,application to a hot strip mill is given to verify the effectiveness of the proposed methods using data collected from an industrial site,and the relatively low RMSE and MAE values demonstrate its advantages compared with some other popular deep learning methods.
基金the National Natural Science Foundation of China(No.51875209)the Guangdong Basic and Applied Basic Research Foundation(No.2019B1515120060)the Open Funds of State Key Laboratory of Nuclear Power Safety Monitoring Technology and Equipment。
文摘Early fault warning for nuclear power machinery is conducive to timely troubleshooting and reductions in safety risks and unnecessary costs. This paper presents a novel intelligent fault prediction method, integrated probabilistic principal component analysis(PPCA), multi-resolution wavelet analysis, Bayesian inference, and RNN model for nuclear power machinery that consider data uncertainty and chaotic time series. After denoising the source data, the Bayesian PPCA method is employed for dimensional reduction to obtain a refined data group. A recurrent neural network(RNN) prediction model is constructed, and a Bayesian statistical inference approach is developed to quantitatively assess the prediction reliability of the model. By modeling and analyzing the data collected on the steam turbine and components of a nuclear power plant, the results of the goodness of fit, mean square error distribution, and Bayesian confidence indicate that the proposed RNN model can implement early warning in the fault creep period. The accuracy and reliability of the proposed model are quantitatively verified.
基金Projects(70572090, 70373017) supported by the National Natural Science Foundation of China
文摘In order to accurately forecast the load of power system and enhance the stability of the power network, a novel unascertained mathematics based recurrent neural network (UMRNN) for power intelligence center (PIC) was created through three steps. First, by combining with the general project uncertain element transmission theory (GPUET), the basic definitions of stochastic, fuzzy, and grey uncertain elements were given based on the principal types of uncertain information. Second, a power dynamic alliance including four sectors: generation sector, transmission sector, distribution sector and customers was established. The key factors were amended according to the four transmission topologies of uncertain elements, thus the new factors entered the power intelligence center as the input elements. Finally, in the intelligence handing background of PIC, by performing uncertain and recursive process to the input values of network, and combining unascertained mathematics, the novel load forecasting model was built. Three different approaches were put forward to forecast an eastern regional power grid load in China. The root mean square error (ERMS) demonstrates that the forecasting accuracy of the proposed model UMRNN is 3% higher than that of BP neural network (BPNN), and 5% higher than that of autoregressive integrated moving average (ARIMA). Besides, an example also shows that the average relative error of the first quarter of 2008 forecasted by UMRNN is only 2.59%, which has high precision.
基金This work was supported in part by NSF under Grants CNS-1950704,CNS-1828593,and OAC-1829771,ONR under Grant N00014-20-1-2065,NSA under Grant H98230-21-1-0278,and the Commonwealth Cyber Initiative.
文摘The Global Positioning System(GPS)has become a foundation for most location-based services and navigation systems,such as autonomous vehicles,drones,ships,and wearable devices.However,it is a challenge to verify if the reported geographic locations are valid due to various GPS spoofing tools.Pervasive tools,such as Fake GPS,Lockito,and software-defined radio,enable ordinary users to hijack and report fake GPS coordinates and cheat the monitoring server without being detected.Furthermore,it is also a challenge to get accurate sensor readings on mobile devices because of the high noise level introduced by commercial motion sensors.To this end,we propose DeepPOSE,a deep learning model,to address the noise introduced in sensor readings and detect GPS spoofing attacks on mobile platforms.Our design uses a convolutional and recurrent neural network to reduce the noise,to recover a vehicle's real-time trajectory from multiple sensor inputs.We further propose a novel scheme to map the constructed trajectory from sensor readings onto the Google map,to smartly eliminate the accumulation of errors on the trajectory estimation.The reconstructed trajectory from sensors is then used to detect the GPS spoofing attack.Compared with the existing method,the proposed approach demonstrates a significantly higher degree of accuracy for detecting GPS spoofing attacks.
文摘A multilayer recurrent fuzzy neural network(MRFNN)is proposed for accurate dynamic system modeling.The proposed MRFNN has six layers combined with T-S fuzzy model.The recurrent structures are formed by local feedback connections in the membership layer and the rule layer.With these feedbacks,the fuzzy sets are time-varying and the temporal problem of dynamic system can be solved well.The parameters of MRFNN are learned by chaotic search(CS)and least square estimation(LSE)simultaneously,where CS is for tuning the premise parameters and LSE is for updating the consequent coefficients accordingly.Results of simulations show the proposed approach is effective for dynamic system modeling with high accuracy.
基金This work is supported by the Research on Big Data in Application for Education of BUPT(No.2018Y0403)Fundamental Research Funds of BUPT(No.2018XKJC07,2018RC27)the National Natural Science Foundation of China(No.61571059).
文摘A vast amount of information has been produced in recent years,which brings a huge challenge to information management.The better usage of big data is of important theoretical and practical significance for effectively addressing and managing messages.In this paper,we propose a nine-rectangle-grid information model according to the information value and privacy,and then present information use policies based on the rough set theory.Recurrent neural networks were employed to classify OTT messages.The content of user interest is effectively incorporated into the classification process during the annotation of OTT messages,ending with a reliable trained classification model.Experimental results showed that the proposed method yielded an accurate classification performance and hence can be used for effective distribution and control of OTT messages.
基金jointly supported by the National Science Foundation of China (Grant Nos. 42275007 and 41865003)Jiangxi Provincial Department of science and technology project (Grant No. 20171BBG70004)。
文摘The Gated Recurrent Unit(GRU) neural network has great potential in estimating and predicting a variable. In addition to radar reflectivity(Z), radar echo-top height(ET) is also a good indicator of rainfall rate(R). In this study, we propose a new method, GRU_Z-ET, by introducing Z and ET as two independent variables into the GRU neural network to conduct the quantitative single-polarization radar precipitation estimation. The performance of GRU_Z-ET is compared with that of the other three methods in three heavy rainfall cases in China during 2018, namely, the traditional Z-R relationship(Z=300R1.4), the optimal Z-R relationship(Z=79R1.68) and the GRU neural network with only Z as the independent input variable(GRU_Z). The results indicate that the GRU_Z-ET performs the best, while the traditional Z-R relationship performs the worst. The performances of the rest two methods are similar.To further evaluate the performance of the GRU_Z-ET, 200 rainfall events with 21882 total samples during May–July of 2018 are used for statistical analysis. Results demonstrate that the spatial correlation coefficients, threat scores and probability of detection between the observed and estimated precipitation are the largest for the GRU_Z-ET and the smallest for the traditional Z-R relationship, and the root mean square error is just the opposite. In addition, these statistics of GRU_Z are similar to those of optimal Z-R relationship. Thus, it can be concluded that the performance of the GRU_ZET is the best in the four methods for the quantitative precipitation estimation.
