In order to improve the efficiency of speech emotion recognition across corpora,a speech emotion transfer learning method based on the deep sparse auto-encoder is proposed.The algorithm first reconstructs a small amou...In order to improve the efficiency of speech emotion recognition across corpora,a speech emotion transfer learning method based on the deep sparse auto-encoder is proposed.The algorithm first reconstructs a small amount of data in the target domain by training the deep sparse auto-encoder,so that the encoder can learn the low-dimensional structural representation of the target domain data.Then,the source domain data and the target domain data are coded by the trained deep sparse auto-encoder to obtain the reconstruction data of the low-dimensional structural representation close to the target domain.Finally,a part of the reconstructed tagged target domain data is mixed with the reconstructed source domain data to jointly train the classifier.This part of the target domain data is used to guide the source domain data.Experiments on the CASIA,SoutheastLab corpus show that the model recognition rate after a small amount of data transferred reached 89.2%and 72.4%on the DNN.Compared to the training results of the complete original corpus,it only decreased by 2%in the CASIA corpus,and only 3.4%in the SoutheastLab corpus.Experiments show that the algorithm can achieve the effect of labeling all data in the extreme case that the data set has only a small amount of data tagged.展开更多
With the rapid development of mechanical equipment,mechanical health monitoring field has entered the era of big data.Deep learning has made a great achievement in the processing of large data of image and speech due ...With the rapid development of mechanical equipment,mechanical health monitoring field has entered the era of big data.Deep learning has made a great achievement in the processing of large data of image and speech due to the powerful modeling capabilities,this also brings influence to the mechanical fault diagnosis field.Therefore,according to the characteristics of motor vibration signals(nonstationary and difficult to deal with)and mechanical‘big data’,combined with deep learning,a motor fault diagnosis method based on stacked de-noising auto-encoder is proposed.The frequency domain signals obtained by the Fourier transform are used as input to the network.This method can extract features adaptively and unsupervised,and get rid of the dependence of traditional machine learning methods on human extraction features.A supervised fine tuning of the model is then carried out by backpropagation.The Asynchronous motor in Drivetrain Dynamics Simulator system was taken as the research object,the effectiveness of the proposed method was verified by a large number of data,and research on visualization of network output,the results shown that the SDAE method is more efficient and more intelligent.展开更多
(Aim)COVID-19 is an ongoing infectious disease.It has caused more than 107.45 m confirmed cases and 2.35 m deaths till 11/Feb/2021.Traditional computer vision methods have achieved promising results on the automatic s...(Aim)COVID-19 is an ongoing infectious disease.It has caused more than 107.45 m confirmed cases and 2.35 m deaths till 11/Feb/2021.Traditional computer vision methods have achieved promising results on the automatic smart diagnosis.(Method)This study aims to propose a novel deep learning method that can obtain better performance.We use the pseudo-Zernike moment(PZM),derived from Zernike moment,as the extracted features.Two settings are introducing:(i)image plane over unit circle;and(ii)image plane inside the unit circle.Afterward,we use a deep-stacked sparse autoencoder(DSSAE)as the classifier.Besides,multiple-way data augmentation is chosen to overcome overfitting.The multiple-way data augmentation is based on Gaussian noise,salt-and-pepper noise,speckle noise,horizontal and vertical shear,rotation,Gamma correction,random translation and scaling.(Results)10 runs of 10-fold cross validation shows that our PZM-DSSAE method achieves a sensitivity of 92.06%±1.54%,a specificity of 92.56%±1.06%,a precision of 92.53%±1.03%,and an accuracy of 92.31%±1.08%.Its F1 score,MCC,and FMI arrive at 92.29%±1.10%,84.64%±2.15%,and 92.29%±1.10%,respectively.The AUC of our model is 0.9576.(Conclusion)We demonstrate“image plane over unit circle”can get better results than“image plane inside a unit circle.”Besides,this proposed PZM-DSSAE model is better than eight state-of-the-art approaches.展开更多
Rolling bearings are important central components in rotating machines, whose fault diagnosis is crucial in condition-based maintenance to reduce the complexity of different kinds of faults. To classify various rollin...Rolling bearings are important central components in rotating machines, whose fault diagnosis is crucial in condition-based maintenance to reduce the complexity of different kinds of faults. To classify various rolling bearing faults, a prognostic algorithm consisting of four phases was proposed. Since stacked denoising auto-encoder can be filtered, noise of large numbers of mechanical vibration signals was used for deep learning structure to extract the characteristics of the noise. Unsupervised pre-training method, which can greatly simplify the traditional manual extraction approach, was utilized to process the depth of the data automatically. Furthermore, the aggregation layer of stacked denoising auto-encoder(SDA) was proposed to get rid of gradient disappearance in deeper layers of network, mix superficial nodes’ expression with deeper layers, and avoid the insufficient express ability in deeper layers. Principal component analysis(PCA) was adopted to extract different features for classification. According to the experimental data of this method and from the comparison results, the proposed method of rolling bearing fault classification reached 97.02% of correct rate, suggesting a better performance than other algorithms.展开更多
Intelligent diagnosis approaches with shallow architectural models play an essential role in healthcare.Deep Learning(DL)models with unsupervised learning concepts have been proposed because high-quality feature extra...Intelligent diagnosis approaches with shallow architectural models play an essential role in healthcare.Deep Learning(DL)models with unsupervised learning concepts have been proposed because high-quality feature extraction and adequate labelled details significantly influence shallow models.On the other hand,skin lesionbased segregation and disintegration procedures play an essential role in earlier skin cancer detection.However,artefacts,an unclear boundary,poor contrast,and different lesion sizes make detection difficult.To address the issues in skin lesion diagnosis,this study creates the UDLS-DDOA model,an intelligent Unsupervised Deep Learning-based Stacked Auto-encoder(UDLS)optimized by Dynamic Differential Annealed Optimization(DDOA).Pre-processing,segregation,feature removal or separation,and disintegration are part of the proposed skin lesion diagnosis model.Pre-processing of skin lesion images occurs at the initial level for noise removal in the image using the Top hat filter and painting methodology.Following that,a Fuzzy C-Means(FCM)segregation procedure is performed using a Quasi-Oppositional Elephant Herd Optimization(QOEHO)algorithm.Besides,a novel feature extraction technique using the UDLS technique is applied where the parameter tuning takes place using DDOA.In the end,the disintegration procedure would be accomplished using a SoftMax(SM)classifier.The UDLS-DDOA model is tested against the International Skin Imaging Collaboration(ISIC)dataset,and the experimental results are examined using various computational attributes.The simulation results demonstrated that the UDLS-DDOA model outperformed the compared methods significantly.展开更多
The influenza virus changes its antigenicity frequently due to rapid mutations, leading to immune escape and failure of vaccination. Rapid determination of the influenza antigenicity could help identify the antigenic ...The influenza virus changes its antigenicity frequently due to rapid mutations, leading to immune escape and failure of vaccination. Rapid determination of the influenza antigenicity could help identify the antigenic variants in time. Here, we built a stacked auto-encoder (SAE) model for predicting the antigenic variant of human influenza A(H3N2) viruses based on the hemagglutinin (HA) protein sequences. The model achieved an accuracy of 0.95 in five-fold cross-validations, better than the logistic regression model did. Further analysis of the model shows that most of the active nodes in the hidden layer reflected the combined contribution of multiple residues to antigenic variation. Besides, some features (residues on HA protein) in the input layer were observed to take part in multiple active nodes, such as residue 189, 145 and 156, which were also reported to mostly determine the antigenic variation of influenza A(H3N2) viruses. Overall,this work is not only useful for rapidly identifying antigenic variants in influenza prevention, but also an interesting attempt in inferring the mechanisms of biological process through analysis of SAE model, which may give some insights into interpretation of the deep learning展开更多
为了提升入侵检测的准确率,鉴于自编码器在学习特征方面的优势以及残差网络在构建深层模型方面的成熟应用,提出一种基于特征降维的改进残差网络入侵检测模型(improved residual network intrusion detection model based on feature dim...为了提升入侵检测的准确率,鉴于自编码器在学习特征方面的优势以及残差网络在构建深层模型方面的成熟应用,提出一种基于特征降维的改进残差网络入侵检测模型(improved residual network intrusion detection model based on feature dimensionality reduction,IRFD),进而缓解传统机器学习入侵检测模型的低准确率问题。IRFD采用堆叠降噪稀疏自编码器策略对数据进行降维,从而提取有效特征。利用卷积注意力机制对残差网络进行改进,构建能提取关键特征的分类网络,并利用两个典型的入侵检测数据集验证IRFD的检测性能。实验结果表明,IRFD在数据集UNSW-NB15和CICIDS 2017上的准确率均达到99%以上,且F1-score分别为99.5%和99.7%。与基线模型相比,提出的IRFD在准确率、精确率和F1-score性能上均有较大提升。展开更多
The increasingly complex and interconnected train control information network is vulnerable to a variety of malicious traffic attacks,and the existing malicious traffic detection methods mainly rely on machine learnin...The increasingly complex and interconnected train control information network is vulnerable to a variety of malicious traffic attacks,and the existing malicious traffic detection methods mainly rely on machine learning,such as poor robustness,weak generalization,and a lack of ability to learn common features.Therefore,this paper proposes a malicious traffic identification method based on stacked sparse denoising autoencoders combined with a regularized extreme learning machine through particle swarm optimization.Firstly,the simulation environment of the Chinese train control system-3,was constructed for data acquisition.Then Pearson coefficient and other methods are used for pre-processing,then a stacked sparse denoising autoencoder is used to achieve nonlinear dimensionality reduction of features,and finally regularization extreme learning machine optimized by particle swarm optimization is used to achieve classification.Experimental data show that the proposed method has good training performance,with an average accuracy of 97.57%and a false negative rate of 2.43%,which is better than other alternative methods.In addition,ablation experiments were performed to evaluate the contribution of each component,and the results showed that the combination of methods was superior to individual methods.To further evaluate the generalization ability of the model in different scenarios,publicly available data sets of industrial control system networks were used.The results show that the model has robust detection capability in various types of network attacks.展开更多
基金The National Natural Science Foundation of China(No.61871213,61673108,61571106)Six Talent Peaks Project in Jiangsu Province(No.2016-DZXX-023)
文摘In order to improve the efficiency of speech emotion recognition across corpora,a speech emotion transfer learning method based on the deep sparse auto-encoder is proposed.The algorithm first reconstructs a small amount of data in the target domain by training the deep sparse auto-encoder,so that the encoder can learn the low-dimensional structural representation of the target domain data.Then,the source domain data and the target domain data are coded by the trained deep sparse auto-encoder to obtain the reconstruction data of the low-dimensional structural representation close to the target domain.Finally,a part of the reconstructed tagged target domain data is mixed with the reconstructed source domain data to jointly train the classifier.This part of the target domain data is used to guide the source domain data.Experiments on the CASIA,SoutheastLab corpus show that the model recognition rate after a small amount of data transferred reached 89.2%and 72.4%on the DNN.Compared to the training results of the complete original corpus,it only decreased by 2%in the CASIA corpus,and only 3.4%in the SoutheastLab corpus.Experiments show that the algorithm can achieve the effect of labeling all data in the extreme case that the data set has only a small amount of data tagged.
基金This research is supported financially by Natural Science Foundation of China(Grant No.51505234,51405241,51575283).
文摘With the rapid development of mechanical equipment,mechanical health monitoring field has entered the era of big data.Deep learning has made a great achievement in the processing of large data of image and speech due to the powerful modeling capabilities,this also brings influence to the mechanical fault diagnosis field.Therefore,according to the characteristics of motor vibration signals(nonstationary and difficult to deal with)and mechanical‘big data’,combined with deep learning,a motor fault diagnosis method based on stacked de-noising auto-encoder is proposed.The frequency domain signals obtained by the Fourier transform are used as input to the network.This method can extract features adaptively and unsupervised,and get rid of the dependence of traditional machine learning methods on human extraction features.A supervised fine tuning of the model is then carried out by backpropagation.The Asynchronous motor in Drivetrain Dynamics Simulator system was taken as the research object,the effectiveness of the proposed method was verified by a large number of data,and research on visualization of network output,the results shown that the SDAE method is more efficient and more intelligent.
基金This study was supported by Royal Society International Exchanges Cost Share Award,UK(RP202G0230)Medical Research Council Confidence in Concept Award,UK(MC_PC_17171)+1 种基金Hope Foundation for Cancer Research,UK(RM60G0680)Global Challenges Research Fund(GCRF),UK(P202PF11)。
文摘(Aim)COVID-19 is an ongoing infectious disease.It has caused more than 107.45 m confirmed cases and 2.35 m deaths till 11/Feb/2021.Traditional computer vision methods have achieved promising results on the automatic smart diagnosis.(Method)This study aims to propose a novel deep learning method that can obtain better performance.We use the pseudo-Zernike moment(PZM),derived from Zernike moment,as the extracted features.Two settings are introducing:(i)image plane over unit circle;and(ii)image plane inside the unit circle.Afterward,we use a deep-stacked sparse autoencoder(DSSAE)as the classifier.Besides,multiple-way data augmentation is chosen to overcome overfitting.The multiple-way data augmentation is based on Gaussian noise,salt-and-pepper noise,speckle noise,horizontal and vertical shear,rotation,Gamma correction,random translation and scaling.(Results)10 runs of 10-fold cross validation shows that our PZM-DSSAE method achieves a sensitivity of 92.06%±1.54%,a specificity of 92.56%±1.06%,a precision of 92.53%±1.03%,and an accuracy of 92.31%±1.08%.Its F1 score,MCC,and FMI arrive at 92.29%±1.10%,84.64%±2.15%,and 92.29%±1.10%,respectively.The AUC of our model is 0.9576.(Conclusion)We demonstrate“image plane over unit circle”can get better results than“image plane inside a unit circle.”Besides,this proposed PZM-DSSAE model is better than eight state-of-the-art approaches.
基金Sponsored by the National Natural Science Foundation of China(Grant No.51704138)
文摘Rolling bearings are important central components in rotating machines, whose fault diagnosis is crucial in condition-based maintenance to reduce the complexity of different kinds of faults. To classify various rolling bearing faults, a prognostic algorithm consisting of four phases was proposed. Since stacked denoising auto-encoder can be filtered, noise of large numbers of mechanical vibration signals was used for deep learning structure to extract the characteristics of the noise. Unsupervised pre-training method, which can greatly simplify the traditional manual extraction approach, was utilized to process the depth of the data automatically. Furthermore, the aggregation layer of stacked denoising auto-encoder(SDA) was proposed to get rid of gradient disappearance in deeper layers of network, mix superficial nodes’ expression with deeper layers, and avoid the insufficient express ability in deeper layers. Principal component analysis(PCA) was adopted to extract different features for classification. According to the experimental data of this method and from the comparison results, the proposed method of rolling bearing fault classification reached 97.02% of correct rate, suggesting a better performance than other algorithms.
基金deputyship for Research&Innovation,Ministry of Education in Saudi Arabia,for funding this research work through Project Number (IFP-2020-133).
文摘Intelligent diagnosis approaches with shallow architectural models play an essential role in healthcare.Deep Learning(DL)models with unsupervised learning concepts have been proposed because high-quality feature extraction and adequate labelled details significantly influence shallow models.On the other hand,skin lesionbased segregation and disintegration procedures play an essential role in earlier skin cancer detection.However,artefacts,an unclear boundary,poor contrast,and different lesion sizes make detection difficult.To address the issues in skin lesion diagnosis,this study creates the UDLS-DDOA model,an intelligent Unsupervised Deep Learning-based Stacked Auto-encoder(UDLS)optimized by Dynamic Differential Annealed Optimization(DDOA).Pre-processing,segregation,feature removal or separation,and disintegration are part of the proposed skin lesion diagnosis model.Pre-processing of skin lesion images occurs at the initial level for noise removal in the image using the Top hat filter and painting methodology.Following that,a Fuzzy C-Means(FCM)segregation procedure is performed using a Quasi-Oppositional Elephant Herd Optimization(QOEHO)algorithm.Besides,a novel feature extraction technique using the UDLS technique is applied where the parameter tuning takes place using DDOA.In the end,the disintegration procedure would be accomplished using a SoftMax(SM)classifier.The UDLS-DDOA model is tested against the International Skin Imaging Collaboration(ISIC)dataset,and the experimental results are examined using various computational attributes.The simulation results demonstrated that the UDLS-DDOA model outperformed the compared methods significantly.
文摘The influenza virus changes its antigenicity frequently due to rapid mutations, leading to immune escape and failure of vaccination. Rapid determination of the influenza antigenicity could help identify the antigenic variants in time. Here, we built a stacked auto-encoder (SAE) model for predicting the antigenic variant of human influenza A(H3N2) viruses based on the hemagglutinin (HA) protein sequences. The model achieved an accuracy of 0.95 in five-fold cross-validations, better than the logistic regression model did. Further analysis of the model shows that most of the active nodes in the hidden layer reflected the combined contribution of multiple residues to antigenic variation. Besides, some features (residues on HA protein) in the input layer were observed to take part in multiple active nodes, such as residue 189, 145 and 156, which were also reported to mostly determine the antigenic variation of influenza A(H3N2) viruses. Overall,this work is not only useful for rapidly identifying antigenic variants in influenza prevention, but also an interesting attempt in inferring the mechanisms of biological process through analysis of SAE model, which may give some insights into interpretation of the deep learning
文摘为了提升入侵检测的准确率,鉴于自编码器在学习特征方面的优势以及残差网络在构建深层模型方面的成熟应用,提出一种基于特征降维的改进残差网络入侵检测模型(improved residual network intrusion detection model based on feature dimensionality reduction,IRFD),进而缓解传统机器学习入侵检测模型的低准确率问题。IRFD采用堆叠降噪稀疏自编码器策略对数据进行降维,从而提取有效特征。利用卷积注意力机制对残差网络进行改进,构建能提取关键特征的分类网络,并利用两个典型的入侵检测数据集验证IRFD的检测性能。实验结果表明,IRFD在数据集UNSW-NB15和CICIDS 2017上的准确率均达到99%以上,且F1-score分别为99.5%和99.7%。与基线模型相比,提出的IRFD在准确率、精确率和F1-score性能上均有较大提升。
文摘The increasingly complex and interconnected train control information network is vulnerable to a variety of malicious traffic attacks,and the existing malicious traffic detection methods mainly rely on machine learning,such as poor robustness,weak generalization,and a lack of ability to learn common features.Therefore,this paper proposes a malicious traffic identification method based on stacked sparse denoising autoencoders combined with a regularized extreme learning machine through particle swarm optimization.Firstly,the simulation environment of the Chinese train control system-3,was constructed for data acquisition.Then Pearson coefficient and other methods are used for pre-processing,then a stacked sparse denoising autoencoder is used to achieve nonlinear dimensionality reduction of features,and finally regularization extreme learning machine optimized by particle swarm optimization is used to achieve classification.Experimental data show that the proposed method has good training performance,with an average accuracy of 97.57%and a false negative rate of 2.43%,which is better than other alternative methods.In addition,ablation experiments were performed to evaluate the contribution of each component,and the results showed that the combination of methods was superior to individual methods.To further evaluate the generalization ability of the model in different scenarios,publicly available data sets of industrial control system networks were used.The results show that the model has robust detection capability in various types of network attacks.
