In bilingual translation,attention-based Neural Machine Translation(NMT)models are used to achieve synchrony between input and output sequences and the notion of alignment.NMT model has obtained state-of-the-art perfo...In bilingual translation,attention-based Neural Machine Translation(NMT)models are used to achieve synchrony between input and output sequences and the notion of alignment.NMT model has obtained state-of-the-art performance for several language pairs.However,there has been little work exploring useful architectures for Urdu-to-English machine translation.We conducted extensive Urdu-to-English translation experiments using Long short-term memory(LSTM)/Bidirectional recurrent neural networks(Bi-RNN)/Statistical recurrent unit(SRU)/Gated recurrent unit(GRU)/Convolutional neural network(CNN)and Transformer.Experimental results show that Bi-RNN and LSTM with attention mechanism trained iteratively,with a scalable data set,make precise predictions on unseen data.The trained models yielded competitive results by achieving 62.6%and 61%accuracy and 49.67 and 47.14 BLEU scores,respectively.From a qualitative perspective,the translation of the test sets was examined manually,and it was observed that trained models tend to produce repetitive output more frequently.The attention score produced by Bi-RNN and LSTM produced clear alignment,while GRU showed incorrect translation for words,poor alignment and lack of a clear structure.Therefore,we considered refining the attention-based models by defining an additional attention-based dropout layer.Attention dropout fixes alignment errors and minimizes translation errors at the word level.After empirical demonstration and comparison with their counterparts,we found improvement in the quality of the resulting translation system and a decrease in the perplexity and over-translation score.The ability of the proposed model was evaluated using Arabic-English and Persian-English datasets as well.We empirically concluded that adding an attention-based dropout layer helps improve GRU,SRU,and Transformer translation and is considerably more efficient in translation quality and speed.展开更多
Flipover,an enhanced dropout technique,is introduced to improve the robustness of artificial neural networks.In contrast to dropout,which involves randomly removing certain neurons and their connections,flipover rando...Flipover,an enhanced dropout technique,is introduced to improve the robustness of artificial neural networks.In contrast to dropout,which involves randomly removing certain neurons and their connections,flipover randomly selects neurons and reverts their outputs using a negative multiplier during training.This approach offers stronger regularization than conventional dropout,refining model performance by(1)mitigating overfitting,matching or even exceeding the efficacy of dropout;(2)amplifying robustness to noise;and(3)enhancing resilience against adversarial attacks.Extensive experiments across various neural networks affirm the effectiveness of flipover in deep learning.展开更多
In today's modern electric vehicles,enhancing the safety-critical cyber-physical system(CPS)'s performance is necessary for the safe maneuverability of the vehicle.As a typical CPS,the braking system is crucia...In today's modern electric vehicles,enhancing the safety-critical cyber-physical system(CPS)'s performance is necessary for the safe maneuverability of the vehicle.As a typical CPS,the braking system is crucial for the vehicle design and safe control.However,precise state estimation of the brake pressure is desired to perform safe driving with a high degree of autonomy.In this paper,a sensorless state estimation technique of the vehicle's brake pressure is developed using a deep-learning approach.A deep neural network(DNN)is structured and trained using deep-learning training techniques,such as,dropout and rectified units.These techniques are utilized to obtain more accurate model for brake pressure state estimation applications.The proposed model is trained using real experimental training data which were collected via conducting real vehicle testing.The vehicle was attached to a chassis dynamometer while the brake pressure data were collected under random driving cycles.Based on these experimental data,the DNN is trained and the performance of the proposed state estimation approach is validated accordingly.The results demonstrate high-accuracy brake pressure state estimation with RMSE of 0.048 MPa.展开更多
基金This work was supported by the Institute for Big Data Analytics and Artificial Intelligence(IBDAAI),Universiti Teknologi Mara,Shah Alam,Selangor.Malaysia.
文摘In bilingual translation,attention-based Neural Machine Translation(NMT)models are used to achieve synchrony between input and output sequences and the notion of alignment.NMT model has obtained state-of-the-art performance for several language pairs.However,there has been little work exploring useful architectures for Urdu-to-English machine translation.We conducted extensive Urdu-to-English translation experiments using Long short-term memory(LSTM)/Bidirectional recurrent neural networks(Bi-RNN)/Statistical recurrent unit(SRU)/Gated recurrent unit(GRU)/Convolutional neural network(CNN)and Transformer.Experimental results show that Bi-RNN and LSTM with attention mechanism trained iteratively,with a scalable data set,make precise predictions on unseen data.The trained models yielded competitive results by achieving 62.6%and 61%accuracy and 49.67 and 47.14 BLEU scores,respectively.From a qualitative perspective,the translation of the test sets was examined manually,and it was observed that trained models tend to produce repetitive output more frequently.The attention score produced by Bi-RNN and LSTM produced clear alignment,while GRU showed incorrect translation for words,poor alignment and lack of a clear structure.Therefore,we considered refining the attention-based models by defining an additional attention-based dropout layer.Attention dropout fixes alignment errors and minimizes translation errors at the word level.After empirical demonstration and comparison with their counterparts,we found improvement in the quality of the resulting translation system and a decrease in the perplexity and over-translation score.The ability of the proposed model was evaluated using Arabic-English and Persian-English datasets as well.We empirically concluded that adding an attention-based dropout layer helps improve GRU,SRU,and Transformer translation and is considerably more efficient in translation quality and speed.
基金supported in part by the National Institutes of Health,Nos.R01CA237267,R01HL151561,R01EB031102,and R01EB032716.
文摘Flipover,an enhanced dropout technique,is introduced to improve the robustness of artificial neural networks.In contrast to dropout,which involves randomly removing certain neurons and their connections,flipover randomly selects neurons and reverts their outputs using a negative multiplier during training.This approach offers stronger regularization than conventional dropout,refining model performance by(1)mitigating overfitting,matching or even exceeding the efficacy of dropout;(2)amplifying robustness to noise;and(3)enhancing resilience against adversarial attacks.Extensive experiments across various neural networks affirm the effectiveness of flipover in deep learning.
文摘In today's modern electric vehicles,enhancing the safety-critical cyber-physical system(CPS)'s performance is necessary for the safe maneuverability of the vehicle.As a typical CPS,the braking system is crucial for the vehicle design and safe control.However,precise state estimation of the brake pressure is desired to perform safe driving with a high degree of autonomy.In this paper,a sensorless state estimation technique of the vehicle's brake pressure is developed using a deep-learning approach.A deep neural network(DNN)is structured and trained using deep-learning training techniques,such as,dropout and rectified units.These techniques are utilized to obtain more accurate model for brake pressure state estimation applications.The proposed model is trained using real experimental training data which were collected via conducting real vehicle testing.The vehicle was attached to a chassis dynamometer while the brake pressure data were collected under random driving cycles.Based on these experimental data,the DNN is trained and the performance of the proposed state estimation approach is validated accordingly.The results demonstrate high-accuracy brake pressure state estimation with RMSE of 0.048 MPa.
