Deep neural networks have achieved tremendous success in various fields,and the structure of these networks is a key factor in their success.In this paper,we focus on the research of ensemble learning based on deep ne...Deep neural networks have achieved tremendous success in various fields,and the structure of these networks is a key factor in their success.In this paper,we focus on the research of ensemble learning based on deep network structure and propose a new deep network ensemble framework(DNEF).Unlike other ensemble learning models,DNEF is an ensemble learning architecture of network structures,with serial iteration between the hidden layers,while base classifiers are trained in parallel within these hidden layers.Specifically,DNEF uses randomly sampled data as input and implements serial iteration based on the weighting strategy between hidden layers.In the hidden layers,each node represents a base classifier,and multiple nodes generate training data for the next hidden layer according to the transfer strategy.The DNEF operates based on two strategies:(1)The weighting strategy calculates the training instance weights of the nodes according to their weaknesses in the previous layer.(2)The transfer strategy adaptively selects each node’s instances with weights as transfer instances and transfer weights,which are combined with the training data of nodes as input for the next hidden layer.These two strategies improve the accuracy and generalization of DNEF.This research integrates the ensemble of all nodes as the final output of DNEF.The experimental results reveal that the DNEF framework surpasses the traditional ensemble models and functions with high accuracy and innovative deep ensemble methods.展开更多
The collaborative filtering(CF) technique has been widely used recently in recommendation systems. It needs historical data to give predictions. However, the data sparsity problem still exists. We propose a new item-b...The collaborative filtering(CF) technique has been widely used recently in recommendation systems. It needs historical data to give predictions. However, the data sparsity problem still exists. We propose a new item-based restricted Boltzmann machine(RBM) approach for CF and use the deep multilayer RBM network structure, which alleviates the data sparsity problem and has excellent ability to extract features. Each item is treated as a single RBM, and different items share the same weights and biases. The parameters are learned layer by layer in the deep network. The batch gradient descent algorithm with minibatch is used to increase the convergence speed. The new feature vector discovered by the multilayer RBM network structure is very effective in predicting a rating and achieves a better result. Experimental results on the data set of MovieL ens show that the item-based multilayer RBM approach achieves the best performance, with a mean absolute error of 0.6424 and a root-mean-square error of 0.7843.展开更多
As a crucial storage and buffering apparatus for balancing the production and consumption of byproduct gases in industrial processes, accurate prediction of gas tank levels is essential for optimizing energy system sc...As a crucial storage and buffering apparatus for balancing the production and consumption of byproduct gases in industrial processes, accurate prediction of gas tank levels is essential for optimizing energy system scheduling. Considering that the continuous switching of the pressure and valve status(mechanism knowledge) would bring about multiple working conditions of the equipment, a multi-condition time sequential network ensembled method is proposed. In order to especially consider the time dependence of different conditions, a centralwise condition sequential network is developed, where the network branches are specially designed based on the condition switching sequences. A branch combination transfer learning strategy is developed to tackle the sample imbalance problem of different condition data. Since the condition or status data are real-time information that cannot be recognized during the prediction process, a pre-trained and ensemble learning approach is further proposed to fuse the outputs of the multi-condition networks and realize a transient-state involved prediction. The performance of the proposed method is validated on practical energy data coming from a domestic steel plant, comparing with the state-of-the-art algorithms. The results show that the proposed method can maintain a high prediction accuracy under different condition switching cases, which would provide effective guidance for the optimal scheduling of the industrial energy systems.展开更多
Stack Overflow provides a platform for developers to seek suitable solutions by asking questions and receiving answers on various topics.However,many questions are usually not answered quickly enough.Since the questio...Stack Overflow provides a platform for developers to seek suitable solutions by asking questions and receiving answers on various topics.However,many questions are usually not answered quickly enough.Since the questioners are eager to know the specific time interval at which a question can be answered,it becomes an important task for Stack Overflow to feedback the answer time to the question.To address this issue,we propose a model for predicting the answer time of questions,named Predicting Answer Time(i.e.,PAT model),which consists of two parts:a feature acquisition and fusion model,and a deep neural network model.The framework uses a variety of features mined from questions in Stack Overflow,including the question description,question title,question tags,the creation time of the question,and other temporal features.These features are fused and fed into the deep neural network to predict the answer time of the question.As a case study,post data from Stack Overflow are used to assess the model.We use traditional regression algorithms as the baselines,such as Linear Regression,K-Nearest Neighbors Regression,Support Vector Regression,Multilayer Perceptron Regression,and Random Forest Regression.Experimental results show that the PAT model can predict the answer time of questions more accurately than traditional regression algorithms,and shorten the error of the predicted answer time by nearly 10 hours.展开更多
基金supported by the National Natural Science Foundation of China under Grant 62002122Guangzhou Municipal Science and Technology Bureau under Grant 202102080492Key Scientific and Technological Research and Department of Education of Guangdong Province under Grant 2019KTSCX014.
文摘Deep neural networks have achieved tremendous success in various fields,and the structure of these networks is a key factor in their success.In this paper,we focus on the research of ensemble learning based on deep network structure and propose a new deep network ensemble framework(DNEF).Unlike other ensemble learning models,DNEF is an ensemble learning architecture of network structures,with serial iteration between the hidden layers,while base classifiers are trained in parallel within these hidden layers.Specifically,DNEF uses randomly sampled data as input and implements serial iteration based on the weighting strategy between hidden layers.In the hidden layers,each node represents a base classifier,and multiple nodes generate training data for the next hidden layer according to the transfer strategy.The DNEF operates based on two strategies:(1)The weighting strategy calculates the training instance weights of the nodes according to their weaknesses in the previous layer.(2)The transfer strategy adaptively selects each node’s instances with weights as transfer instances and transfer weights,which are combined with the training data of nodes as input for the next hidden layer.These two strategies improve the accuracy and generalization of DNEF.This research integrates the ensemble of all nodes as the final output of DNEF.The experimental results reveal that the DNEF framework surpasses the traditional ensemble models and functions with high accuracy and innovative deep ensemble methods.
基金Project supported by the National Science and Technology Suppor Plan(No.2013BAH21B02-01)the Beijing Natural Science Foundation(No.4153058)
文摘The collaborative filtering(CF) technique has been widely used recently in recommendation systems. It needs historical data to give predictions. However, the data sparsity problem still exists. We propose a new item-based restricted Boltzmann machine(RBM) approach for CF and use the deep multilayer RBM network structure, which alleviates the data sparsity problem and has excellent ability to extract features. Each item is treated as a single RBM, and different items share the same weights and biases. The parameters are learned layer by layer in the deep network. The batch gradient descent algorithm with minibatch is used to increase the convergence speed. The new feature vector discovered by the multilayer RBM network structure is very effective in predicting a rating and achieves a better result. Experimental results on the data set of MovieL ens show that the item-based multilayer RBM approach achieves the best performance, with a mean absolute error of 0.6424 and a root-mean-square error of 0.7843.
基金supported by the National Natural Sciences Foundation of China(62125302,62203087)Liaoning Revitalization Talents Program(XLYC2002087)+1 种基金Sci-Tech Talent Innovation Support Program of Dalian(2022RG03)Young Elite Scientist Sponsorship Program by China Association for Science and Technology(YESS20220018)
文摘As a crucial storage and buffering apparatus for balancing the production and consumption of byproduct gases in industrial processes, accurate prediction of gas tank levels is essential for optimizing energy system scheduling. Considering that the continuous switching of the pressure and valve status(mechanism knowledge) would bring about multiple working conditions of the equipment, a multi-condition time sequential network ensembled method is proposed. In order to especially consider the time dependence of different conditions, a centralwise condition sequential network is developed, where the network branches are specially designed based on the condition switching sequences. A branch combination transfer learning strategy is developed to tackle the sample imbalance problem of different condition data. Since the condition or status data are real-time information that cannot be recognized during the prediction process, a pre-trained and ensemble learning approach is further proposed to fuse the outputs of the multi-condition networks and realize a transient-state involved prediction. The performance of the proposed method is validated on practical energy data coming from a domestic steel plant, comparing with the state-of-the-art algorithms. The results show that the proposed method can maintain a high prediction accuracy under different condition switching cases, which would provide effective guidance for the optimal scheduling of the industrial energy systems.
基金supported by the National Natural Science Foundation of China under Grant Nos.61902050,61602077 and 61672122the China Postdoctoral Science Foundation under Grant No.2020M670736+1 种基金the Fundamental Research Funds for the Central Universities of China under Grant Nos.3132019355 and 2020cxxmss14the High Education Science and Technology Planning Program of Shandong Provincial Education Department of China under Grant Nos.J18KA340 and J18KA385.
文摘Stack Overflow provides a platform for developers to seek suitable solutions by asking questions and receiving answers on various topics.However,many questions are usually not answered quickly enough.Since the questioners are eager to know the specific time interval at which a question can be answered,it becomes an important task for Stack Overflow to feedback the answer time to the question.To address this issue,we propose a model for predicting the answer time of questions,named Predicting Answer Time(i.e.,PAT model),which consists of two parts:a feature acquisition and fusion model,and a deep neural network model.The framework uses a variety of features mined from questions in Stack Overflow,including the question description,question title,question tags,the creation time of the question,and other temporal features.These features are fused and fed into the deep neural network to predict the answer time of the question.As a case study,post data from Stack Overflow are used to assess the model.We use traditional regression algorithms as the baselines,such as Linear Regression,K-Nearest Neighbors Regression,Support Vector Regression,Multilayer Perceptron Regression,and Random Forest Regression.Experimental results show that the PAT model can predict the answer time of questions more accurately than traditional regression algorithms,and shorten the error of the predicted answer time by nearly 10 hours.
