Deep neural networks(DNNs)have found extensive applications in safety-critical artificial intelligence systems,such as autonomous driving and facial recognition systems.However,recent research has revealed their susce...Deep neural networks(DNNs)have found extensive applications in safety-critical artificial intelligence systems,such as autonomous driving and facial recognition systems.However,recent research has revealed their susceptibility to backdoors maliciously injected by adversaries.This vulnerability arises due to the intricate architecture and opacity of DNNs,resulting in numerous redundant neurons embedded within the models.Adversaries exploit these vulnerabilities to conceal malicious backdoor information within DNNs,thereby causing erroneous outputs and posing substantial threats to the efficacy of DNN-based applications.This article presents a comprehensive survey of backdoor attacks against DNNs and the countermeasure methods employed to mitigate them.Initially,we trace the evolution of the concept from traditional backdoor attacks to backdoor attacks against DNNs,highlighting the feasibility and practicality of generating backdoor attacks against DNNs.Subsequently,we provide an overview of notable works encompassing various attack and defense strategies,facilitating a comparative analysis of their approaches.Through these discussions,we offer constructive insights aimed at refining these techniques.Finally,we extend our research perspective to the domain of large language models(LLMs)and synthesize the characteristics and developmental trends of backdoor attacks and defense methods targeting LLMs.Through a systematic review of existing studies on backdoor vulnerabilities in LLMs,we identify critical open challenges in this field and propose actionable directions for future research.展开更多
Deep neural networks(DNNs)are effective in solving both forward and inverse problems for nonlinear partial differential equations(PDEs).However,conventional DNNs are not effective in handling problems such as delay di...Deep neural networks(DNNs)are effective in solving both forward and inverse problems for nonlinear partial differential equations(PDEs).However,conventional DNNs are not effective in handling problems such as delay differential equations(DDEs)and delay integrodifferential equations(DIDEs)with constant delays,primarily due to their low regularity at delayinduced breaking points.In this paper,a DNN method that combines multi-task learning(MTL)which is proposed to solve both the forward and inverse problems of DIDEs.The core idea of this approach is to divide the original equation into multiple tasks based on the delay,using auxiliary outputs to represent the integral terms,followed by the use of MTL to seamlessly incorporate the properties at the breaking points into the loss function.Furthermore,given the increased training dificulty associated with multiple tasks and outputs,we employ a sequential training scheme to reduce training complexity and provide reference solutions for subsequent tasks.This approach significantly enhances the approximation accuracy of solving DIDEs with DNNs,as demonstrated by comparisons with traditional DNN methods.We validate the effectiveness of this method through several numerical experiments,test various parameter sharing structures in MTL and compare the testing results of these structures.Finally,this method is implemented to solve the inverse problem of nonlinear DIDE and the results show that the unknown parameters of DIDE can be discovered with sparse or noisy data.展开更多
Fundoscopic diagnosis involves assessing the proper functioning of the eye’s nerves,blood vessels,retinal health,and the impact of diabetes on the optic nerves.Fundus disorders are a major global health concern,affec...Fundoscopic diagnosis involves assessing the proper functioning of the eye’s nerves,blood vessels,retinal health,and the impact of diabetes on the optic nerves.Fundus disorders are a major global health concern,affecting millions of people worldwide due to their widespread occurrence.Fundus photography generates machine-based eye images that assist in diagnosing and treating ocular diseases such as diabetic retinopathy.As a result,accurate fundus detection is essential for early diagnosis and effective treatment,helping to prevent severe complications and improve patient outcomes.To address this need,this article introduces a Derivative Model for Fundus Detection using Deep NeuralNetworks(DMFD-DNN)to enhance diagnostic precision.Thismethod selects key features for fundus detection using the least derivative,which identifies features correlating with stored fundus images.Feature filtering relies on the minimum derivative,determined by extracting both similar and varying textures.In this research,the DNN model was integrated with the derivative model.Fundus images were segmented,features were extracted,and the DNN was iteratively trained to identify fundus regions reliably.The goal was to improve the precision of fundoscopic diagnosis by training the DNN incrementally,taking into account the least possible derivative across iterations,and using outputs from previous cycles.The hidden layer of the neural network operates on the most significant derivative,which may reduce precision across iterations.These derivatives are treated as inaccurate,and the model is subsequently trained using selective features and their corresponding extractions.The proposed model outperforms previous techniques in detecting fundus regions,achieving 94.98%accuracy and 91.57%sensitivity,with a minimal error rate of 5.43%.It significantly reduces feature extraction time to 1.462 s and minimizes computational overhead,thereby improving operational efficiency and scalability.Ultimately,the proposed model enhances diagnostic precision and reduces errors,leading to more effective fundus dysfunction diagnosis and treatment.展开更多
Deep neural networks provide accurate results for most applications.However,they need a big dataset to train properly.Providing a big dataset is a significant challenge in most applications.Image augmentation refers t...Deep neural networks provide accurate results for most applications.However,they need a big dataset to train properly.Providing a big dataset is a significant challenge in most applications.Image augmentation refers to techniques that increase the amount of image data.Common operations for image augmentation include changes in illumination,rotation,contrast,size,viewing angle,and others.Recently,Generative Adversarial Networks(GANs)have been employed for image generation.However,like image augmentation methods,GAN approaches can only generate images that are similar to the original images.Therefore,they also cannot generate new classes of data.Texture images presentmore challenges than general images,and generating textures is more complex than creating other types of images.This study proposes a gradient-based deep neural network method that generates a new class of texture.It is possible to rapidly generate new classes of textures using different kernels from pre-trained deep networks.After generating new textures for each class,the number of textures increases through image augmentation.During this process,several techniques are proposed to automatically remove incomplete and similar textures that are created.The proposed method is faster than some well-known generative networks by around 4 to 10 times.In addition,the quality of the generated textures surpasses that of these networks.The proposed method can generate textures that surpass those of someGANs and parametric models in certain image qualitymetrics.It can provide a big texture dataset to train deep networks.A new big texture dataset is created artificially using the proposed method.This dataset is approximately 2 GB in size and comprises 30,000 textures,each 150×150 pixels in size,organized into 600 classes.It is uploaded to the Kaggle site and Google Drive.This dataset is called BigTex.Compared to other texture datasets,the proposed dataset is the largest and can serve as a comprehensive texture dataset for training more powerful deep neural networks and mitigating overfitting.展开更多
Optical deep learning based on diffractive optical elements offers unique advantages for parallel processing,computational speed,and power efficiency.One landmark method is the diffractive deep neural network(D^(2) NN...Optical deep learning based on diffractive optical elements offers unique advantages for parallel processing,computational speed,and power efficiency.One landmark method is the diffractive deep neural network(D^(2) NN)based on three-dimensional printing technology operated in the terahertz spectral range.Since the terahertz bandwidth involves limited interparticle coupling and material losses,this paper extends D^(2) NN to visible wavelengths.A general theory including a revised formula is proposed to solve any contradictions between wavelength,neuron size,and fabrication limitations.A novel visible light D^(2) NN classifier is used to recognize unchanged targets(handwritten digits ranging from 0 to 9)and targets that have been changed(i.e.,targets that have been covered or altered)at a visible wavelength of 632.8 nm.The obtained experimental classification accuracy(84%)and numerical classification accuracy(91.57%)quantify the match between the theoretical design and fabricated system performance.The presented framework can be used to apply a D^(2) NN to various practical applications and design other new applications.展开更多
Controlling multiple multi-joint fish-like robots has long captivated the attention of engineers and biologists,for which a fundamental but challenging topic is to robustly track the postures of the individuals in rea...Controlling multiple multi-joint fish-like robots has long captivated the attention of engineers and biologists,for which a fundamental but challenging topic is to robustly track the postures of the individuals in real time.This requires detecting multiple robots,estimating multi-joint postures,and tracking identities,as well as processing fast in real time.To the best of our knowledge,this challenge has not been tackled in the previous studies.In this paper,to precisely track the planar postures of multiple swimming multi-joint fish-like robots in real time,we propose a novel deep neural network-based method,named TAB-IOL.Its TAB part fuses the top-down and bottom-up approaches for vision-based pose estimation,while the IOL part with long short-term memory considers the motion constraints among joints for precise pose tracking.The satisfying performance of our TAB-IOL is verified by testing on a group of freely swimming fish-like robots in various scenarios with strong disturbances and by a deed comparison of accuracy,speed,and robustness with most state-of-the-art algorithms.Further,based on the precise pose estimation and tracking realized by our TAB-IOL,several formation control experiments are conducted for the group of fish-like robots.The results clearly demonstrate that our TAB-IOL lays a solid foundation for the coordination control of multiple fish-like robots in a real working environment.We believe our proposed method will facilitate the growth and development of related fields.展开更多
Blind image quality assessment(BIQA)is of fundamental importance in low-level computer vision community.Increasing interest has been drawn in exploiting deep neural networks for BIQA.Despite of the notable success ach...Blind image quality assessment(BIQA)is of fundamental importance in low-level computer vision community.Increasing interest has been drawn in exploiting deep neural networks for BIQA.Despite of the notable success achieved,there is a broad consensus that training deep convolutional neural networks(DCNN)heavily relies on massive annotated data.Unfortunately,BIQA is typically a small sample problem,resulting the generalization ability of BIQA severely restricted.In order to improve the accuracy and generalization ability of BIQA metrics,this work proposed a totally opinion-unaware BIQA in which no subjective annotations are involved in the training stage.Multiple full-reference image quality assessment(FR-IQA)metrics are employed to label the distorted image as a substitution of subjective quality annotation.A deep neural network(DNN)is trained to blindly predict the multiple FR-IQA score in absence of corresponding pristine image.In the end,a selfsupervised FR-IQA score aggregator implemented by adversarial auto-encoder pools the predictions of multiple FR-IQA scores into the final quality predicting score.Even though none of subjective scores are involved in the training stage,experimental results indicate that our proposed full reference induced BIQA framework is as competitive as state-of-the-art BIQA metrics.展开更多
Geochemical maps are of great value in mineral exploration.Integrated geochemical anomaly maps provide comprehensive information about mapping assemblages of element concentrations to possible types of mineralization/...Geochemical maps are of great value in mineral exploration.Integrated geochemical anomaly maps provide comprehensive information about mapping assemblages of element concentrations to possible types of mineralization/ore,but vary depending on expert's knowledge and experience.This paper aims to test the capability of deep neural networks to delineate integrated anomaly based on a case study of the Zhaojikou Pb-Zn deposit,Southeast China.Three hundred fifty two samples were collected,and each sample consisted of 26 variables covering elemental composition,geological,and tectonic information.At first,generative adversarial networks were adopted for data augmentation.Then,DNN was trained on sets of synthetic and real data to identify an integrated anomaly.Finally,the results of DNN analyses were visualized in probability maps and compared with traditional anomaly maps to check its performance.Results showed that the average accuracy of the validation set was 94.76%.The probability maps showed that newly-identified integrated anomalous areas had a probability of above 75%in the northeast zones.It also showed that DNN models that used big data not only successfully recognized the anomalous areas identified on traditional geochemical element maps,but also discovered new anomalous areas,not picked up by the elemental anomaly maps previously.展开更多
Currently,Bitcoin is the world’s most popular cryptocurrency.The price of Bitcoin is extremely volatile,which can be described as high-benefit and high-risk.To minimize the risk involved,a means of more accurately pr...Currently,Bitcoin is the world’s most popular cryptocurrency.The price of Bitcoin is extremely volatile,which can be described as high-benefit and high-risk.To minimize the risk involved,a means of more accurately predicting the Bitcoin price is required.Most of the existing studies of Bitcoin prediction are based on historical(i.e.,benchmark)data,without considering the real-time(i.e.,live)data.To mitigate the issue of price volatility and achieve more precise outcomes,this study suggests using historical and real-time data to predict the Bitcoin candlestick—or open,high,low,and close(OHLC)—prices.Seeking a better prediction model,the present study proposes time series-based deep learning models.In particular,two deep learning algorithms were applied,namely,long short-term memory(LSTM)and gated recurrent unit(GRU).Using real-time data,the Bitcoin candlesticks were predicted for three intervals:the next 4 h,the next 12 h,and the next 24 h.The results showed that the best-performing model was the LSTM-based model with the 4-h interval.In particular,this model achieved a stellar performance with a mean absolute percentage error(MAPE)of 0.63,a root mean square error(RMSE)of 0.0009,a mean square error(MSE)of 9e-07,a mean absolute error(MAE)of 0.0005,and an R-squared coefficient(R2)of 0.994.With these results,the proposed prediction model has demonstrated its efficiency over the models proposed in previous studies.The findings of this study have considerable implications in the business field,as the proposed model can assist investors and traders in precisely identifying Bitcoin sales and buying opportunities.展开更多
The medical community has more concern on lung cancer analysis.Medical experts’physical segmentation of lung cancers is time-consuming and needs to be automated.The research study’s objective is to diagnose lung tum...The medical community has more concern on lung cancer analysis.Medical experts’physical segmentation of lung cancers is time-consuming and needs to be automated.The research study’s objective is to diagnose lung tumors at an early stage to extend the life of humans using deep learning techniques.Computer-Aided Diagnostic(CAD)system aids in the diagnosis and shortens the time necessary to detect the tumor detected.The application of Deep Neural Networks(DNN)has also been exhibited as an excellent and effective method in classification and segmentation tasks.This research aims to separate lung cancers from images of Magnetic Resonance Imaging(MRI)with threshold segmentation.The Honey hook process categorizes lung cancer based on characteristics retrieved using several classifiers.Considering this principle,the work presents a solution for image compression utilizing a Deep Wave Auto-Encoder(DWAE).The combination of the two approaches significantly reduces the overall size of the feature set required for any future classification process performed using DNN.The proposed DWAE-DNN image classifier is applied to a lung imaging dataset with Radial Basis Function(RBF)classifier.The study reported promising results with an accuracy of 97.34%,whereas using the Decision Tree(DT)classifier has an accuracy of 94.24%.The proposed approach(DWAE-DNN)is found to classify the images with an accuracy of 98.67%,either as malignant or normal patients.In contrast to the accuracy requirements,the work also uses the benchmark standards like specificity,sensitivity,and precision to evaluate the efficiency of the network.It is found from an investigation that the DT classifier provides the maximum performance in the DWAE-DNN depending on the network’s performance on image testing,as shown by the data acquired by the categorizers themselves.展开更多
Parkinson’s disease is a serious disease that causes death.Recently,a new dataset has been introduced on this disease.The aim of this study is to improve the predictive performance of the model designed for Parkinson...Parkinson’s disease is a serious disease that causes death.Recently,a new dataset has been introduced on this disease.The aim of this study is to improve the predictive performance of the model designed for Parkinson’s disease diagnosis.By and large,original DNN models were designed by using specific or random number of neurons and layers.This study analyzed the effects of parameters,i.e.,neuron number and activation function on the model performance based on growing and pruning approach.In other words,this study addressed the optimum hidden layer and neuron numbers and ideal activation and optimization functions in order to find out the best Deep Neural Networks model.In this context of this study,several models were designed and evaluated.The overall results revealed that the Deep Neural Networks were significantly successful with 99.34%accuracy value on test data.Also,it presents the highest prediction performance reported so far.Therefore,this study presents a model promising with respect to more accurate Parkinson’s disease diagnosis.展开更多
Objective:The process of manually recognize the lesion tissue in pathological images is a key,laborious and subjective step in tumor diagnosis.An automatic segmentation method is proposed to segment lesion tissue in p...Objective:The process of manually recognize the lesion tissue in pathological images is a key,laborious and subjective step in tumor diagnosis.An automatic segmentation method is proposed to segment lesion tissue in pathological images.Methods:We present a region of interest(ROI)method to generate a new pre-training dataset for training initial weights on DNNs to solve the overfitting problem.To improve the segmentation performance,a multiscale and multi-resolution ensemble strategy is proposed.Our methods are validated on a public segmentation dataset of colonoscopy images.Results:By using the ROI pre-training method,the Dice score of DeepLabV3 and ResUNet increases from 0.607 to 0.739 and from 0.572 to 0.741,respectively.The ensemble method is used in the testing phase,the Dice score of DeepLabV3 and ResUNet increased to 0.760 and 0.786.Conclusion:The ROI pre-training method and ensemble strategy can be applied to DeepLabV3 and ResUNet to improve the segmentation performance of colonoscopy images.展开更多
Recently,due to the availability of big data and the rapid growth of computing power,artificial intelligence(AI)has regained tremendous attention and investment.Machine learning(ML)approaches have been successfully ap...Recently,due to the availability of big data and the rapid growth of computing power,artificial intelligence(AI)has regained tremendous attention and investment.Machine learning(ML)approaches have been successfully applied to solve many problems in academia and in industry.Although the explosion of big data applications is driving the development of ML,it also imposes severe challenges of data processing speed and scalability on conventional computer systems.Computing platforms that are dedicatedly designed for AI applications have been considered,ranging from a complement to von Neumann platforms to a“must-have”and stand-alone technical solution.These platforms,which belong to a larger category named“domain-specific computing,”focus on specific customization for AI.In this article,we focus on summarizing the recent advances in accelerator designs for deep neural networks(DNNs)-that is,DNN accelerators.We discuss various architectures that support DNN executions in terms of computing units,dataflow optimization,targeted network topologies,architectures on emerging technologies,and accelerators for emerging applications.We also provide our visions on the future trend of AI chip designs.展开更多
Supervised fault diagnosis typically assumes that all the types of machinery failures are known.However,in practice unknown types of defect,i.e.,novelties,may occur,whose detection is a challenging task.In this paper,...Supervised fault diagnosis typically assumes that all the types of machinery failures are known.However,in practice unknown types of defect,i.e.,novelties,may occur,whose detection is a challenging task.In this paper,a novel fault diagnostic method is developed for both diagnostics and detection of novelties.To this end,a sparse autoencoder-based multi-head Deep Neural Network(DNN)is presented to jointly learn a shared encoding representation for both unsupervised reconstruction and supervised classification of the monitoring data.The detection of novelties is based on the reconstruction error.Moreover,the computational burden is reduced by directly training the multi-head DNN with rectified linear unit activation function,instead of performing the pre-training and fine-tuning phases required for classical DNNs.The addressed method is applied to a benchmark bearing case study and to experimental data acquired from a delta 3D printer.The results show that its performance is satisfactory both in detection of novelties and fault diagnosis,outperforming other state-of-the-art methods.This research proposes a novel fault diagnostics method which can not only diagnose the known type of defect,but also detect unknown types of defects.展开更多
Oil leakage between the slipper and swash plate of an axial piston pump has a significant effect on the efficiency of the pump.Therefore,it is extremely important that any leakage can be predicted.This study investiga...Oil leakage between the slipper and swash plate of an axial piston pump has a significant effect on the efficiency of the pump.Therefore,it is extremely important that any leakage can be predicted.This study investigates the leakage,oil film thickness,and pocket pressure values of a slipper with circular dimples under different working conditions.The results reveal that flat slippers suffer less leakage than those with textured surfaces.Also,a deep learning-based framework is proposed for modeling the slipper behavior.This framework is a long short-term memory-based deep neural network,which has been extremely successful in predicting time series.The model is compared with four conventional machine learning methods.In addition,statistical analyses and comparisons confirm the superiority of the proposed model.展开更多
For training the present Neural Network(NN)models,the standard technique is to utilize decaying Learning Rates(LR).While the majority of these techniques commence with a large LR,they will decay multiple times over ti...For training the present Neural Network(NN)models,the standard technique is to utilize decaying Learning Rates(LR).While the majority of these techniques commence with a large LR,they will decay multiple times over time.Decaying has been proved to enhance generalization as well as optimization.Other parameters,such as the network’s size,the number of hidden layers,drop-outs to avoid overfitting,batch size,and so on,are solely based on heuristics.This work has proposed Adaptive Teaching Learning Based(ATLB)Heuristic to identify the optimal hyperparameters for diverse networks.Here we consider three architec-tures Recurrent Neural Networks(RNN),Long Short Term Memory(LSTM),Bidirectional Long Short Term Memory(BiLSTM)of Deep Neural Networks for classification.The evaluation of the proposed ATLB is done through the various learning rate schedulers Cyclical Learning Rate(CLR),Hyperbolic Tangent Decay(HTD),and Toggle between Hyperbolic Tangent Decay and Triangular mode with Restarts(T-HTR)techniques.Experimental results have shown the performance improvement on the 20Newsgroup,Reuters Newswire and IMDB dataset.展开更多
With the increasing of data size and model size,deep neural networks(DNNs)show outstanding performance in many artificial intelligence(AI)applications.But the big model size makes it a challenge for high-performance a...With the increasing of data size and model size,deep neural networks(DNNs)show outstanding performance in many artificial intelligence(AI)applications.But the big model size makes it a challenge for high-performance and low-power running DNN on processors,such as central processing unit(CPU),graphics processing unit(GPU),and tensor processing unit(TPU).This paper proposes a LOGNN data representation of 8 bits and a hardware and software co-design deep neural network accelerator LACC to meet the challenge.LOGNN data representation replaces multiply operations to add and shift operations in running DNN.LACC accelerator achieves higher efficiency than the state-of-the-art DNN accelerators by domain specific arithmetic computing units.Finally,LACC speeds up the performance per watt by 1.5 times,compared to the state-of-the-art DNN accelerators on average.展开更多
Accurate estimation of the state of charge(SoC)of lithium-ion batteries in electric vehicles(EVs)is crucial for optimizing performance,ensuring safety,and extending battery life.However,traditional estimation methods ...Accurate estimation of the state of charge(SoC)of lithium-ion batteries in electric vehicles(EVs)is crucial for optimizing performance,ensuring safety,and extending battery life.However,traditional estimation methods often struggle with the nonlinear and dynamic behavior of battery systems,leading to inaccuracies that compromise the efficiency and reliability of electric vehicles.This study proposes a novel approach for SoC estimation in BMW EVs by integrating a metaheuristic algorithm with deep neural networks.Specifically,teaching-learning based optimization(TLBO)is employed to optimize the weights and biases of the deep neural networks model,enhancing estimation accuracy.The proposed TLBO-deep neural networks(TLBO-DNNs)method was evaluated on a dataset of 1,064,000 samples,with performance assessed using mean absolute error(MAE),root mean square error(RMSE),and convergence value.The TLBO-DNNs model achieved an MAE of 3.4480,an RMSE of 4.6487,and a convergence value of 0.0328,outperforming other hybrid approaches.These include the barnacle mating optimizer-deep neural networks(BMO-DNNs)with an MAE of 5.3848,an RMSE of 7.0395,and a convergence value of 0.0492;the evolutionary mating algorithm-deep neural networks(EMA-DNNs)with an MAE of 7.6127,an RMSE of 11.2287,and a convergence value of 0.0536;and the particle swarm optimization-deep neural networks(PSO-DNNs)with an MAE of 4.3089,an RMSE of 5.9672,and a convergence value of 0.0345.Additionally,the TLBO-DNNs approach outperformed standalone models,including the autoregressive integrated moving average(ARIMA)model(MAE:14.3301,RMSE:7.0697)and support vector machines(MAE:6.0065,RMSE:8.0360).This hybrid TLBO-DNNs technique demonstrates significant potential for enhancing battery management systems in electric vehicles,contributing to improved efficiency and reliability in electric vehicle operations.展开更多
Hardware transient faults are proven to have a significant impact on deep neural networks (DNNs), whose safety-critical misclassification (SCM) in autonomous vehicles, healthcare, and space applications is increased u...Hardware transient faults are proven to have a significant impact on deep neural networks (DNNs), whose safety-critical misclassification (SCM) in autonomous vehicles, healthcare, and space applications is increased up to four times. However, the inaccuracy evaluation using accurate fault injection is time-consuming and requires several hours and even a couple of days on a complete simulation platform. To accelerate the evaluation of hardware transient faults on DNNs, we design a unified and end-to-end automatic methodology, A-Mean, using the silent data corruption (SDC) rate of basic operations (such as convolution, addition, multiply, ReLU, and max-pooling) and a static two-level mean calculation mechanism to rapidly compute the overall SDC rate, for estimating the general classification metric accuracy and application-specific metric SCM. More importantly, a max-policy is used to determine the SDC boundary of non-sequential structures in DNNs. Then, the worst-case scheme is used to further calculate the enlarged SCM and halved accuracy under transient faults, via merging the static results of SDC with the original data from one-time dynamic fault-free execution. Furthermore, all of the steps mentioned above have been implemented automatically, so that this easy-to-use automatic tool can be employed for prompt evaluation of transient faults on diverse DNNs. Meanwhile, a novel metric “fault sensitivity” is defined to characterize the variation of transient fault-induced higher SCM and lower accuracy. The comparative results with a state-of-the-art fault injection method TensorFI+ on five DNN models and four datasets show that our proposed estimation method A-Mean achieves up to 922.80 times speedup, with just 4.20% SCM loss and 0.77% accuracy loss on average. The artifact of A-Mean is publicly available at https://github.com/breatrice321/A-Mean.展开更多
Model discovery based on existing data has been one of the major focuses of mathematical modelers for decades.Despite tremendous achievements in model identification from adequate data,how to unravel the models from l...Model discovery based on existing data has been one of the major focuses of mathematical modelers for decades.Despite tremendous achievements in model identification from adequate data,how to unravel the models from limited data is less resolved.This paper focuses on the model discovery problem when the data is not efficiently sampled in time,which is common due to limited experimental accessibility and labor/resource constraints.Specifically,we introduce a recursive deep neural network(RDNN)for data-driven model discovery.This recursive approach can retrieve the governing equation efficiently and significantly improve the approximation accuracy by increasing the recursive stages.In particular,our proposed approach shows superior power when the existing data are sampled with a large time lag,from which the traditional approach might not be able to recover the model well.Several examples of dynamical systems are used to benchmark this newly proposed recursive approach.Numerical comparisons confirm the effectiveness of this recursive neural network for model discovery.The accompanying codes are available at https://github.com/c2fd/RDNNs.展开更多
基金supported in part by the National Natural Science Foundation of China under Grants No.62372087 and No.62072076the Research Fund of State Key Laboratory of Processors under Grant No.CLQ202310the CSC scholarship.
文摘Deep neural networks(DNNs)have found extensive applications in safety-critical artificial intelligence systems,such as autonomous driving and facial recognition systems.However,recent research has revealed their susceptibility to backdoors maliciously injected by adversaries.This vulnerability arises due to the intricate architecture and opacity of DNNs,resulting in numerous redundant neurons embedded within the models.Adversaries exploit these vulnerabilities to conceal malicious backdoor information within DNNs,thereby causing erroneous outputs and posing substantial threats to the efficacy of DNN-based applications.This article presents a comprehensive survey of backdoor attacks against DNNs and the countermeasure methods employed to mitigate them.Initially,we trace the evolution of the concept from traditional backdoor attacks to backdoor attacks against DNNs,highlighting the feasibility and practicality of generating backdoor attacks against DNNs.Subsequently,we provide an overview of notable works encompassing various attack and defense strategies,facilitating a comparative analysis of their approaches.Through these discussions,we offer constructive insights aimed at refining these techniques.Finally,we extend our research perspective to the domain of large language models(LLMs)and synthesize the characteristics and developmental trends of backdoor attacks and defense methods targeting LLMs.Through a systematic review of existing studies on backdoor vulnerabilities in LLMs,we identify critical open challenges in this field and propose actionable directions for future research.
文摘Deep neural networks(DNNs)are effective in solving both forward and inverse problems for nonlinear partial differential equations(PDEs).However,conventional DNNs are not effective in handling problems such as delay differential equations(DDEs)and delay integrodifferential equations(DIDEs)with constant delays,primarily due to their low regularity at delayinduced breaking points.In this paper,a DNN method that combines multi-task learning(MTL)which is proposed to solve both the forward and inverse problems of DIDEs.The core idea of this approach is to divide the original equation into multiple tasks based on the delay,using auxiliary outputs to represent the integral terms,followed by the use of MTL to seamlessly incorporate the properties at the breaking points into the loss function.Furthermore,given the increased training dificulty associated with multiple tasks and outputs,we employ a sequential training scheme to reduce training complexity and provide reference solutions for subsequent tasks.This approach significantly enhances the approximation accuracy of solving DIDEs with DNNs,as demonstrated by comparisons with traditional DNN methods.We validate the effectiveness of this method through several numerical experiments,test various parameter sharing structures in MTL and compare the testing results of these structures.Finally,this method is implemented to solve the inverse problem of nonlinear DIDE and the results show that the unknown parameters of DIDE can be discovered with sparse or noisy data.
基金supported by the Basic Science Research Program through the National Research Foundation of Korea(NRF)funded by the Ministry of Education(No.2021R1F1A1055408)supported by the Researchers Supporting Project Number(MHIRSP2024005)Almaarefa University,Riyadh,Saudi Arabia.
文摘Fundoscopic diagnosis involves assessing the proper functioning of the eye’s nerves,blood vessels,retinal health,and the impact of diabetes on the optic nerves.Fundus disorders are a major global health concern,affecting millions of people worldwide due to their widespread occurrence.Fundus photography generates machine-based eye images that assist in diagnosing and treating ocular diseases such as diabetic retinopathy.As a result,accurate fundus detection is essential for early diagnosis and effective treatment,helping to prevent severe complications and improve patient outcomes.To address this need,this article introduces a Derivative Model for Fundus Detection using Deep NeuralNetworks(DMFD-DNN)to enhance diagnostic precision.Thismethod selects key features for fundus detection using the least derivative,which identifies features correlating with stored fundus images.Feature filtering relies on the minimum derivative,determined by extracting both similar and varying textures.In this research,the DNN model was integrated with the derivative model.Fundus images were segmented,features were extracted,and the DNN was iteratively trained to identify fundus regions reliably.The goal was to improve the precision of fundoscopic diagnosis by training the DNN incrementally,taking into account the least possible derivative across iterations,and using outputs from previous cycles.The hidden layer of the neural network operates on the most significant derivative,which may reduce precision across iterations.These derivatives are treated as inaccurate,and the model is subsequently trained using selective features and their corresponding extractions.The proposed model outperforms previous techniques in detecting fundus regions,achieving 94.98%accuracy and 91.57%sensitivity,with a minimal error rate of 5.43%.It significantly reduces feature extraction time to 1.462 s and minimizes computational overhead,thereby improving operational efficiency and scalability.Ultimately,the proposed model enhances diagnostic precision and reduces errors,leading to more effective fundus dysfunction diagnosis and treatment.
基金supported via funding from Prince Sattam bin Abdulaziz University(PSAU/2025/R/1446)Princess Nourah bint Abdulrahman University(PNURSP2025R300)Prince Sultan University.
文摘Deep neural networks provide accurate results for most applications.However,they need a big dataset to train properly.Providing a big dataset is a significant challenge in most applications.Image augmentation refers to techniques that increase the amount of image data.Common operations for image augmentation include changes in illumination,rotation,contrast,size,viewing angle,and others.Recently,Generative Adversarial Networks(GANs)have been employed for image generation.However,like image augmentation methods,GAN approaches can only generate images that are similar to the original images.Therefore,they also cannot generate new classes of data.Texture images presentmore challenges than general images,and generating textures is more complex than creating other types of images.This study proposes a gradient-based deep neural network method that generates a new class of texture.It is possible to rapidly generate new classes of textures using different kernels from pre-trained deep networks.After generating new textures for each class,the number of textures increases through image augmentation.During this process,several techniques are proposed to automatically remove incomplete and similar textures that are created.The proposed method is faster than some well-known generative networks by around 4 to 10 times.In addition,the quality of the generated textures surpasses that of these networks.The proposed method can generate textures that surpass those of someGANs and parametric models in certain image qualitymetrics.It can provide a big texture dataset to train deep networks.A new big texture dataset is created artificially using the proposed method.This dataset is approximately 2 GB in size and comprises 30,000 textures,each 150×150 pixels in size,organized into 600 classes.It is uploaded to the Kaggle site and Google Drive.This dataset is called BigTex.Compared to other texture datasets,the proposed dataset is the largest and can serve as a comprehensive texture dataset for training more powerful deep neural networks and mitigating overfitting.
基金This research was supported in part by National Natural Science Foundation of China(61675056 and 61875048).
文摘Optical deep learning based on diffractive optical elements offers unique advantages for parallel processing,computational speed,and power efficiency.One landmark method is the diffractive deep neural network(D^(2) NN)based on three-dimensional printing technology operated in the terahertz spectral range.Since the terahertz bandwidth involves limited interparticle coupling and material losses,this paper extends D^(2) NN to visible wavelengths.A general theory including a revised formula is proposed to solve any contradictions between wavelength,neuron size,and fabrication limitations.A novel visible light D^(2) NN classifier is used to recognize unchanged targets(handwritten digits ranging from 0 to 9)and targets that have been changed(i.e.,targets that have been covered or altered)at a visible wavelength of 632.8 nm.The obtained experimental classification accuracy(84%)and numerical classification accuracy(91.57%)quantify the match between the theoretical design and fabricated system performance.The presented framework can be used to apply a D^(2) NN to various practical applications and design other new applications.
基金This work was supported in part by the National Natural Science Foundation of China(61973007,61633002).
文摘Controlling multiple multi-joint fish-like robots has long captivated the attention of engineers and biologists,for which a fundamental but challenging topic is to robustly track the postures of the individuals in real time.This requires detecting multiple robots,estimating multi-joint postures,and tracking identities,as well as processing fast in real time.To the best of our knowledge,this challenge has not been tackled in the previous studies.In this paper,to precisely track the planar postures of multiple swimming multi-joint fish-like robots in real time,we propose a novel deep neural network-based method,named TAB-IOL.Its TAB part fuses the top-down and bottom-up approaches for vision-based pose estimation,while the IOL part with long short-term memory considers the motion constraints among joints for precise pose tracking.The satisfying performance of our TAB-IOL is verified by testing on a group of freely swimming fish-like robots in various scenarios with strong disturbances and by a deed comparison of accuracy,speed,and robustness with most state-of-the-art algorithms.Further,based on the precise pose estimation and tracking realized by our TAB-IOL,several formation control experiments are conducted for the group of fish-like robots.The results clearly demonstrate that our TAB-IOL lays a solid foundation for the coordination control of multiple fish-like robots in a real working environment.We believe our proposed method will facilitate the growth and development of related fields.
基金supported by the Public Welfare Technology Application Research Project of Zhejiang Province,China(No.LGF21F010001)the Key Research and Development Program of Zhejiang Province,China(Grant No.2019C01002)the Key Research and Development Program of Zhejiang Province,China(Grant No.2021C03138)。
文摘Blind image quality assessment(BIQA)is of fundamental importance in low-level computer vision community.Increasing interest has been drawn in exploiting deep neural networks for BIQA.Despite of the notable success achieved,there is a broad consensus that training deep convolutional neural networks(DCNN)heavily relies on massive annotated data.Unfortunately,BIQA is typically a small sample problem,resulting the generalization ability of BIQA severely restricted.In order to improve the accuracy and generalization ability of BIQA metrics,this work proposed a totally opinion-unaware BIQA in which no subjective annotations are involved in the training stage.Multiple full-reference image quality assessment(FR-IQA)metrics are employed to label the distorted image as a substitution of subjective quality annotation.A deep neural network(DNN)is trained to blindly predict the multiple FR-IQA score in absence of corresponding pristine image.In the end,a selfsupervised FR-IQA score aggregator implemented by adversarial auto-encoder pools the predictions of multiple FR-IQA scores into the final quality predicting score.Even though none of subjective scores are involved in the training stage,experimental results indicate that our proposed full reference induced BIQA framework is as competitive as state-of-the-art BIQA metrics.
基金supported by NFSC Funds(Grant Nos.41902071 and 42011530173)the Doctoral Research Start-up Fund,East China University of Technology(DHBK2019313)。
文摘Geochemical maps are of great value in mineral exploration.Integrated geochemical anomaly maps provide comprehensive information about mapping assemblages of element concentrations to possible types of mineralization/ore,but vary depending on expert's knowledge and experience.This paper aims to test the capability of deep neural networks to delineate integrated anomaly based on a case study of the Zhaojikou Pb-Zn deposit,Southeast China.Three hundred fifty two samples were collected,and each sample consisted of 26 variables covering elemental composition,geological,and tectonic information.At first,generative adversarial networks were adopted for data augmentation.Then,DNN was trained on sets of synthetic and real data to identify an integrated anomaly.Finally,the results of DNN analyses were visualized in probability maps and compared with traditional anomaly maps to check its performance.Results showed that the average accuracy of the validation set was 94.76%.The probability maps showed that newly-identified integrated anomalous areas had a probability of above 75%in the northeast zones.It also showed that DNN models that used big data not only successfully recognized the anomalous areas identified on traditional geochemical element maps,but also discovered new anomalous areas,not picked up by the elemental anomaly maps previously.
文摘Currently,Bitcoin is the world’s most popular cryptocurrency.The price of Bitcoin is extremely volatile,which can be described as high-benefit and high-risk.To minimize the risk involved,a means of more accurately predicting the Bitcoin price is required.Most of the existing studies of Bitcoin prediction are based on historical(i.e.,benchmark)data,without considering the real-time(i.e.,live)data.To mitigate the issue of price volatility and achieve more precise outcomes,this study suggests using historical and real-time data to predict the Bitcoin candlestick—or open,high,low,and close(OHLC)—prices.Seeking a better prediction model,the present study proposes time series-based deep learning models.In particular,two deep learning algorithms were applied,namely,long short-term memory(LSTM)and gated recurrent unit(GRU).Using real-time data,the Bitcoin candlesticks were predicted for three intervals:the next 4 h,the next 12 h,and the next 24 h.The results showed that the best-performing model was the LSTM-based model with the 4-h interval.In particular,this model achieved a stellar performance with a mean absolute percentage error(MAPE)of 0.63,a root mean square error(RMSE)of 0.0009,a mean square error(MSE)of 9e-07,a mean absolute error(MAE)of 0.0005,and an R-squared coefficient(R2)of 0.994.With these results,the proposed prediction model has demonstrated its efficiency over the models proposed in previous studies.The findings of this study have considerable implications in the business field,as the proposed model can assist investors and traders in precisely identifying Bitcoin sales and buying opportunities.
基金the Researchers Supporting Project Number(RSP2023R 509)King Saud University,Riyadh,Saudi ArabiaThis work was supported in part by the Higher Education Sprout Project from the Ministry of Education(MOE)and National Science and Technology Council,Taiwan,(109-2628-E-224-001-MY3)in part by Isuzu Optics Corporation.Dr.Shih-Yu Chen is the corresponding author.
文摘The medical community has more concern on lung cancer analysis.Medical experts’physical segmentation of lung cancers is time-consuming and needs to be automated.The research study’s objective is to diagnose lung tumors at an early stage to extend the life of humans using deep learning techniques.Computer-Aided Diagnostic(CAD)system aids in the diagnosis and shortens the time necessary to detect the tumor detected.The application of Deep Neural Networks(DNN)has also been exhibited as an excellent and effective method in classification and segmentation tasks.This research aims to separate lung cancers from images of Magnetic Resonance Imaging(MRI)with threshold segmentation.The Honey hook process categorizes lung cancer based on characteristics retrieved using several classifiers.Considering this principle,the work presents a solution for image compression utilizing a Deep Wave Auto-Encoder(DWAE).The combination of the two approaches significantly reduces the overall size of the feature set required for any future classification process performed using DNN.The proposed DWAE-DNN image classifier is applied to a lung imaging dataset with Radial Basis Function(RBF)classifier.The study reported promising results with an accuracy of 97.34%,whereas using the Decision Tree(DT)classifier has an accuracy of 94.24%.The proposed approach(DWAE-DNN)is found to classify the images with an accuracy of 98.67%,either as malignant or normal patients.In contrast to the accuracy requirements,the work also uses the benchmark standards like specificity,sensitivity,and precision to evaluate the efficiency of the network.It is found from an investigation that the DT classifier provides the maximum performance in the DWAE-DNN depending on the network’s performance on image testing,as shown by the data acquired by the categorizers themselves.
文摘Parkinson’s disease is a serious disease that causes death.Recently,a new dataset has been introduced on this disease.The aim of this study is to improve the predictive performance of the model designed for Parkinson’s disease diagnosis.By and large,original DNN models were designed by using specific or random number of neurons and layers.This study analyzed the effects of parameters,i.e.,neuron number and activation function on the model performance based on growing and pruning approach.In other words,this study addressed the optimum hidden layer and neuron numbers and ideal activation and optimization functions in order to find out the best Deep Neural Networks model.In this context of this study,several models were designed and evaluated.The overall results revealed that the Deep Neural Networks were significantly successful with 99.34%accuracy value on test data.Also,it presents the highest prediction performance reported so far.Therefore,this study presents a model promising with respect to more accurate Parkinson’s disease diagnosis.
基金the National Major Science and Technology Projects(grant no.2018AAA0100201)the National Natural Science Foundation of China(grant no.61906127).
文摘Objective:The process of manually recognize the lesion tissue in pathological images is a key,laborious and subjective step in tumor diagnosis.An automatic segmentation method is proposed to segment lesion tissue in pathological images.Methods:We present a region of interest(ROI)method to generate a new pre-training dataset for training initial weights on DNNs to solve the overfitting problem.To improve the segmentation performance,a multiscale and multi-resolution ensemble strategy is proposed.Our methods are validated on a public segmentation dataset of colonoscopy images.Results:By using the ROI pre-training method,the Dice score of DeepLabV3 and ResUNet increases from 0.607 to 0.739 and from 0.572 to 0.741,respectively.The ensemble method is used in the testing phase,the Dice score of DeepLabV3 and ResUNet increased to 0.760 and 0.786.Conclusion:The ROI pre-training method and ensemble strategy can be applied to DeepLabV3 and ResUNet to improve the segmentation performance of colonoscopy images.
基金the National Science Foundations(NSFs)(1822085,1725456,1816833,1500848,1719160,and 1725447)the NSF Computing and Communication Foundations(1740352)+1 种基金the Nanoelectronics COmputing REsearch Program in the Semiconductor Research Corporation(NC-2766-A)the Center for Research in Intelligent Storage and Processing-in-Memory,one of six centers in the Joint University Microelectronics Program,a SRC program sponsored by Defense Advanced Research Projects Agency.
文摘Recently,due to the availability of big data and the rapid growth of computing power,artificial intelligence(AI)has regained tremendous attention and investment.Machine learning(ML)approaches have been successfully applied to solve many problems in academia and in industry.Although the explosion of big data applications is driving the development of ML,it also imposes severe challenges of data processing speed and scalability on conventional computer systems.Computing platforms that are dedicatedly designed for AI applications have been considered,ranging from a complement to von Neumann platforms to a“must-have”and stand-alone technical solution.These platforms,which belong to a larger category named“domain-specific computing,”focus on specific customization for AI.In this article,we focus on summarizing the recent advances in accelerator designs for deep neural networks(DNNs)-that is,DNN accelerators.We discuss various architectures that support DNN executions in terms of computing units,dataflow optimization,targeted network topologies,architectures on emerging technologies,and accelerators for emerging applications.We also provide our visions on the future trend of AI chip designs.
基金Supported by National Natural Science Foundation of China(Grant Nos.52005103,71801046,51775112,51975121)Guangdong Province Basic and Applied Basic Research Foundation of China(Grant No.2019B1515120095)+1 种基金Intelligent Manufacturing PHM Innovation Team Program(Grant Nos.2018KCXTD029,TDYB2019010)MoST International Cooperation Program(6-14).
文摘Supervised fault diagnosis typically assumes that all the types of machinery failures are known.However,in practice unknown types of defect,i.e.,novelties,may occur,whose detection is a challenging task.In this paper,a novel fault diagnostic method is developed for both diagnostics and detection of novelties.To this end,a sparse autoencoder-based multi-head Deep Neural Network(DNN)is presented to jointly learn a shared encoding representation for both unsupervised reconstruction and supervised classification of the monitoring data.The detection of novelties is based on the reconstruction error.Moreover,the computational burden is reduced by directly training the multi-head DNN with rectified linear unit activation function,instead of performing the pre-training and fine-tuning phases required for classical DNNs.The addressed method is applied to a benchmark bearing case study and to experimental data acquired from a delta 3D printer.The results show that its performance is satisfactory both in detection of novelties and fault diagnosis,outperforming other state-of-the-art methods.This research proposes a novel fault diagnostics method which can not only diagnose the known type of defect,but also detect unknown types of defects.
基金Supported by Erciyes University Scientific Research Projects Coordination Unit(Grant No.FDK-2016-6986).
文摘Oil leakage between the slipper and swash plate of an axial piston pump has a significant effect on the efficiency of the pump.Therefore,it is extremely important that any leakage can be predicted.This study investigates the leakage,oil film thickness,and pocket pressure values of a slipper with circular dimples under different working conditions.The results reveal that flat slippers suffer less leakage than those with textured surfaces.Also,a deep learning-based framework is proposed for modeling the slipper behavior.This framework is a long short-term memory-based deep neural network,which has been extremely successful in predicting time series.The model is compared with four conventional machine learning methods.In addition,statistical analyses and comparisons confirm the superiority of the proposed model.
文摘For training the present Neural Network(NN)models,the standard technique is to utilize decaying Learning Rates(LR).While the majority of these techniques commence with a large LR,they will decay multiple times over time.Decaying has been proved to enhance generalization as well as optimization.Other parameters,such as the network’s size,the number of hidden layers,drop-outs to avoid overfitting,batch size,and so on,are solely based on heuristics.This work has proposed Adaptive Teaching Learning Based(ATLB)Heuristic to identify the optimal hyperparameters for diverse networks.Here we consider three architec-tures Recurrent Neural Networks(RNN),Long Short Term Memory(LSTM),Bidirectional Long Short Term Memory(BiLSTM)of Deep Neural Networks for classification.The evaluation of the proposed ATLB is done through the various learning rate schedulers Cyclical Learning Rate(CLR),Hyperbolic Tangent Decay(HTD),and Toggle between Hyperbolic Tangent Decay and Triangular mode with Restarts(T-HTR)techniques.Experimental results have shown the performance improvement on the 20Newsgroup,Reuters Newswire and IMDB dataset.
基金Supported by the National Key Research and Development Program of China(No.2018AAA0103300,2017YFA0700900,2017YFA0700902,2017YFA0700901,2019AAA0103802,2020AAA0103802)。
文摘With the increasing of data size and model size,deep neural networks(DNNs)show outstanding performance in many artificial intelligence(AI)applications.But the big model size makes it a challenge for high-performance and low-power running DNN on processors,such as central processing unit(CPU),graphics processing unit(GPU),and tensor processing unit(TPU).This paper proposes a LOGNN data representation of 8 bits and a hardware and software co-design deep neural network accelerator LACC to meet the challenge.LOGNN data representation replaces multiply operations to add and shift operations in running DNN.LACC accelerator achieves higher efficiency than the state-of-the-art DNN accelerators by domain specific arithmetic computing units.Finally,LACC speeds up the performance per watt by 1.5 times,compared to the state-of-the-art DNN accelerators on average.
基金supported by Ministry of Higher Education Malaysia(MOHE)under Fundamental Research Grant Scheme(Grant No.:FRGS/1/2024/ICT02/UMP/02/).
文摘Accurate estimation of the state of charge(SoC)of lithium-ion batteries in electric vehicles(EVs)is crucial for optimizing performance,ensuring safety,and extending battery life.However,traditional estimation methods often struggle with the nonlinear and dynamic behavior of battery systems,leading to inaccuracies that compromise the efficiency and reliability of electric vehicles.This study proposes a novel approach for SoC estimation in BMW EVs by integrating a metaheuristic algorithm with deep neural networks.Specifically,teaching-learning based optimization(TLBO)is employed to optimize the weights and biases of the deep neural networks model,enhancing estimation accuracy.The proposed TLBO-deep neural networks(TLBO-DNNs)method was evaluated on a dataset of 1,064,000 samples,with performance assessed using mean absolute error(MAE),root mean square error(RMSE),and convergence value.The TLBO-DNNs model achieved an MAE of 3.4480,an RMSE of 4.6487,and a convergence value of 0.0328,outperforming other hybrid approaches.These include the barnacle mating optimizer-deep neural networks(BMO-DNNs)with an MAE of 5.3848,an RMSE of 7.0395,and a convergence value of 0.0492;the evolutionary mating algorithm-deep neural networks(EMA-DNNs)with an MAE of 7.6127,an RMSE of 11.2287,and a convergence value of 0.0536;and the particle swarm optimization-deep neural networks(PSO-DNNs)with an MAE of 4.3089,an RMSE of 5.9672,and a convergence value of 0.0345.Additionally,the TLBO-DNNs approach outperformed standalone models,including the autoregressive integrated moving average(ARIMA)model(MAE:14.3301,RMSE:7.0697)and support vector machines(MAE:6.0065,RMSE:8.0360).This hybrid TLBO-DNNs technique demonstrates significant potential for enhancing battery management systems in electric vehicles,contributing to improved efficiency and reliability in electric vehicle operations.
基金Project supported by the Shanghai Pujiang Talent Program,China(No.21PJD026)。
文摘Hardware transient faults are proven to have a significant impact on deep neural networks (DNNs), whose safety-critical misclassification (SCM) in autonomous vehicles, healthcare, and space applications is increased up to four times. However, the inaccuracy evaluation using accurate fault injection is time-consuming and requires several hours and even a couple of days on a complete simulation platform. To accelerate the evaluation of hardware transient faults on DNNs, we design a unified and end-to-end automatic methodology, A-Mean, using the silent data corruption (SDC) rate of basic operations (such as convolution, addition, multiply, ReLU, and max-pooling) and a static two-level mean calculation mechanism to rapidly compute the overall SDC rate, for estimating the general classification metric accuracy and application-specific metric SCM. More importantly, a max-policy is used to determine the SDC boundary of non-sequential structures in DNNs. Then, the worst-case scheme is used to further calculate the enlarged SCM and halved accuracy under transient faults, via merging the static results of SDC with the original data from one-time dynamic fault-free execution. Furthermore, all of the steps mentioned above have been implemented automatically, so that this easy-to-use automatic tool can be employed for prompt evaluation of transient faults on diverse DNNs. Meanwhile, a novel metric “fault sensitivity” is defined to characterize the variation of transient fault-induced higher SCM and lower accuracy. The comparative results with a state-of-the-art fault injection method TensorFI+ on five DNN models and four datasets show that our proposed estimation method A-Mean achieves up to 922.80 times speedup, with just 4.20% SCM loss and 0.77% accuracy loss on average. The artifact of A-Mean is publicly available at https://github.com/breatrice321/A-Mean.
基金support from the NSF-DMS-1816783 and NVIDIA Corporation for their GPU donation for conducting some of the numerical simulations in this paper.
文摘Model discovery based on existing data has been one of the major focuses of mathematical modelers for decades.Despite tremendous achievements in model identification from adequate data,how to unravel the models from limited data is less resolved.This paper focuses on the model discovery problem when the data is not efficiently sampled in time,which is common due to limited experimental accessibility and labor/resource constraints.Specifically,we introduce a recursive deep neural network(RDNN)for data-driven model discovery.This recursive approach can retrieve the governing equation efficiently and significantly improve the approximation accuracy by increasing the recursive stages.In particular,our proposed approach shows superior power when the existing data are sampled with a large time lag,from which the traditional approach might not be able to recover the model well.Several examples of dynamical systems are used to benchmark this newly proposed recursive approach.Numerical comparisons confirm the effectiveness of this recursive neural network for model discovery.The accompanying codes are available at https://github.com/c2fd/RDNNs.
