The purpose of this study was to address the challenges in predicting and classifying accuracy in modeling Container Dwell Time (CDT) using Artificial Neural Networks (ANN). This objective was driven by the suboptimal...The purpose of this study was to address the challenges in predicting and classifying accuracy in modeling Container Dwell Time (CDT) using Artificial Neural Networks (ANN). This objective was driven by the suboptimal outcomes reported in previous studies and sought to apply an innovative approach to improve these results. To achieve this, the study applied the Fusion of Activation Functions (FAFs) to a substantial dataset. This dataset included 307,594 container records from the Port of Tema from 2014 to 2022, encompassing both import and transit containers. The RandomizedSearchCV algorithm from Python’s Scikit-learn library was utilized in the methodological approach to yield the optimal activation function for prediction accuracy. The results indicated that “ajaLT”, a fusion of the Logistic and Hyperbolic Tangent Activation Functions, provided the best prediction accuracy, reaching a high of 82%. Despite these encouraging findings, it’s crucial to recognize the study’s limitations. While Fusion of Activation Functions is a promising method, further evaluation is necessary across different container types and port operations to ascertain the broader applicability and generalizability of these findings. The original value of this study lies in its innovative application of FAFs to CDT. Unlike previous studies, this research evaluates the method based on prediction accuracy rather than training time. It opens new avenues for machine learning engineers and researchers in applying FAFs to enhance prediction accuracy in CDT modeling, contributing to a previously underexplored area.展开更多
A vehicle engine cooling system is of utmost importance to ensure that the engine operates in a safe temperature range.In most radiators that are used to cool an engine,water serves as a cooling fluid.The performance ...A vehicle engine cooling system is of utmost importance to ensure that the engine operates in a safe temperature range.In most radiators that are used to cool an engine,water serves as a cooling fluid.The performance of a radiator in terms of heat transmission is significantly influenced by the incorporation of nanoparticles into the cooling water.Concentration and uniformity of nanoparticle distribution are the two major factors for the practical use of nanofluids.The shape and size of nanoparticles also have a great impact on the performance of heat transfer.Many researchers are investigating the impact of nanoparticles on heat transfer.This study aims to develop an artificial neural network(ANN)model for predicting the thermal conductivity of an ethylene glycol(EG)/waterbased crystalline nanocellulose(CNC)nanofluid for cooling internal combustion engine.The implementation of an artificial neural network considering different activation functions in the hidden layer is made to find the bestmodel for the cooling of an engine using the nanofluid.Accuracies of the model with different activation functions in artificial neural networks are analyzed for different nanofluid concentrations and temperatures.In artificial neural networks,Levenberg–Marquardt is an optimization approach used with activation functions,including Tansig and Logsig functions in the training phase.The findings of each training,testing,and validation phase are presented to demonstrate the network that provides the highest level of accuracy.The best result was obtained with Tansig,which has a correlation of 0.99903 and an error of 3.7959×10^(–8).It has also been noticed that the Logsig function can also be a good model due to its correlation of 0.99890 and an error of 4.9218×10^(–8).Thus ourANNwith Tansig and Logsig functions demonstrates a high correlation between the actual output and the predicted output.展开更多
The stability of a periodic oscillation and the global exponential class of recurrent neural networks with non-monotone activation functions and time-varying delays are analyzed. For two sets of activation functions, ...The stability of a periodic oscillation and the global exponential class of recurrent neural networks with non-monotone activation functions and time-varying delays are analyzed. For two sets of activation functions, some algebraic criteria for ascertaining global exponential periodicity and global exponential stability of the class of recurrent neural networks are derived by using the comparison principle and the theory of monotone operator. These conditions are easy to check in terms of system parameters. In addition, we provide a new and efficacious method for the qualitative analysis of various neural networks.展开更多
In this paper, the multistability issue is discussed for delayed complex-valued recurrent neural networks with discontinuous real-imaginary-type activation functions. Based on a fixed theorem and stability definition,...In this paper, the multistability issue is discussed for delayed complex-valued recurrent neural networks with discontinuous real-imaginary-type activation functions. Based on a fixed theorem and stability definition, sufficient criteria are established for the existence and stability of multiple equilibria of complex-valued recurrent neural networks. The number of stable equilibria is larger than that of real-valued recurrent neural networks, which can be used to achieve high-capacity associative memories. One numerical example is provided to show the effectiveness and superiority of the presented results.展开更多
Activation functions play an essential role in converting the output of the artificial neural network into nonlinear results,since without this nonlinearity,the results of the network will be less accurate.Nonlinearity...Activation functions play an essential role in converting the output of the artificial neural network into nonlinear results,since without this nonlinearity,the results of the network will be less accurate.Nonlinearity is the mission of all nonlinear functions,except for polynomials.The activation function must be dif-ferentiable for backpropagation learning.This study’s objective is to determine the best activation functions for the approximation of each fractal image.Different results have been attained using Matlab and Visual Basic programs,which indi-cate that the bounded function is more helpful than other functions.The non-lin-earity of the activation function is important when using neural networks for coding fractal images because the coefficients of the Iterated Function System are different according to the different types of fractals.The most commonly cho-sen activation function is the sigmoidal function,which produces a positive value.Other functions,such as tansh or arctan,whose values can be positive or negative depending on the network input,tend to train neural networks faster.The coding speed of the fractal image is different depending on the appropriate activation function chosen for each fractal shape.In this paper,we have provided the appro-priate activation functions for each type of system of iterated functions that help the network to identify the transactions of the system.展开更多
This paper describes our implementation of several neural networks built on a field programmable gate array (FPGA) and used to recognize a handwritten digit dataset—the Modified National Institute of Standards and Te...This paper describes our implementation of several neural networks built on a field programmable gate array (FPGA) and used to recognize a handwritten digit dataset—the Modified National Institute of Standards and Technology (MNIST) database. We also propose a novel hardware-friendly activation function called the dynamic Rectifid Linear Unit (ReLU)—D-ReLU function that achieves higher performance than traditional activation functions at no cost to accuracy. We built a 2-layer online training multilayer perceptron (MLP) neural network on an FPGA with varying data width. Reducing the data width from 8 to 4 bits only reduces prediction accuracy by 11%, but the FPGA area decreases by 41%. Compared to networks that use the sigmoid functions, our proposed D-ReLU function uses 24% - 41% less area with no loss to prediction accuracy. Further reducing the data width of the 3-layer networks from 8 to 4 bits, the prediction accuracies only decrease by 3% - 5%, with area being reduced by 9% - 28%. Moreover, FPGA solutions have 29 times faster execution time, even despite running at a 60× lower clock rate. Thus, FPGA implementations of neural networks offer a high-performance, low power alternative to traditional software methods, and our novel D-ReLU activation function offers additional improvements to performance and power saving.展开更多
Deep learning(DL),derived from the domain of Artificial Neural Networks(ANN),forms one of the most essential components of modern deep learning algorithms.DL segmentation models rely on layer-by-layer convolution-base...Deep learning(DL),derived from the domain of Artificial Neural Networks(ANN),forms one of the most essential components of modern deep learning algorithms.DL segmentation models rely on layer-by-layer convolution-based feature representation,guided by forward and backward propagation.Acritical aspect of this process is the selection of an appropriate activation function(AF)to ensure robustmodel learning.However,existing activation functions often fail to effectively address the vanishing gradient problem or are complicated by the need for manual parameter tuning.Most current research on activation function design focuses on classification tasks using natural image datasets such asMNIST,CIFAR-10,and CIFAR-100.To address this gap,this study proposesMed-ReLU,a novel activation function specifically designed for medical image segmentation.Med-ReLU prevents deep learning models fromsuffering dead neurons or vanishing gradient issues.It is a hybrid activation function that combines the properties of ReLU and Softsign.For positive inputs,Med-ReLU adopts the linear behavior of ReLU to avoid vanishing gradients,while for negative inputs,it exhibits the Softsign’s polynomial convergence,ensuring robust training and avoiding inactive neurons across the training set.The training performance and segmentation accuracy ofMed-ReLU have been thoroughly evaluated,demonstrating stable learning behavior and resistance to overfitting.It consistently outperforms state-of-the-art activation functions inmedical image segmentation tasks.Designed as a parameter-free function,Med-ReLU is simple to implement in complex deep learning architectures,and its effectiveness spans various neural network models and anomaly detection scenarios.展开更多
Physics-informed neural networks(PINNs)are known to suffer from optimization difficulty.In this work,we reveal the connection between the optimization difficulty of PINNs and activation functions.Specifically,we show ...Physics-informed neural networks(PINNs)are known to suffer from optimization difficulty.In this work,we reveal the connection between the optimization difficulty of PINNs and activation functions.Specifically,we show that PINNs exhibit high sensitivity to activation functions when solving PDEs with distinct properties.Existing works usually choose activation functions by inefficient trial-and-error.To avoid the inefficient manual selection and to alleviate the optimization difficulty of PINNs,we introduce adaptive activation functions to search for the optimal function when solving different problems.We compare different adaptive activation functions and discuss their limitations in the context of PINNs.Furthermore,we propose to tailor the idea of learning combinations of candidate activation functions to the PINNs optimization,which has a higher requirement for the smoothness and diversity on learned functions.This is achieved by removing activation functions which cannot provide higher-order derivatives from the candidate set and incorporating elementary functions with different properties according to our prior knowledge about the PDE at hand.We further enhance the search space with adaptive slopes.The proposed adaptive activation function can be used to solve different PDE systems in an interpretable way.Its effectiveness is demonstrated on a series of benchmarks.Code is available at https://github.com/LeapLabTHU/AdaAFforPINNs.展开更多
Purpose–The purpose of this paper is to develop a method for the existence,uniqueness and globally robust stability of the equilibrium point for Cohen–Grossberg neural networks with time-varying delays,continuous di...Purpose–The purpose of this paper is to develop a method for the existence,uniqueness and globally robust stability of the equilibrium point for Cohen–Grossberg neural networks with time-varying delays,continuous distributed delays and a kind of discontinuous activation functions.Design/methodology/approach–Basedonthe Leray–Schauderalternativetheoremand chainrule,by using a novel integral inequality dealing with monotone non-decreasing function,the authors obtain a delay-dependent sufficient condition with less conservativeness for robust stability of considered neural networks.Findings–Itturns out thattheauthors’delay-dependent sufficientcondition canbeformed intermsof linear matrix inequalities conditions.Two examples show the effectiveness of the obtained results.Originality/value–The novelty of the proposed approach lies in dealing with a new kind of discontinuous activation functions by using the Leray–Schauder alternative theorem,chain rule and a novel integral inequality on monotone non-decreasing function.展开更多
Binary neural networks(BNNs)show promising utilization in cost and power-restricted domains such as edge devices and mobile systems.This is due to its significantly less computation and storage demand,but at the cost ...Binary neural networks(BNNs)show promising utilization in cost and power-restricted domains such as edge devices and mobile systems.This is due to its significantly less computation and storage demand,but at the cost of degraded performance.To close the accuracy gap,in this paper we propose to add a complementary activation function(AF)ahead of the sign based binarization,and rely on the genetic algorithm(GA)to automatically search for the ideal AFs.These AFs can help extract extra information from the input data in the forward pass,while allowing improved gradient approximation in the backward pass.Fifteen novel AFs are identified through our GA-based search,while most of them show improved performance(up to 2.54%on ImageNet)when testing on different datasets and network models.Interestingly,periodic functions are identified as a key component for most of the discovered AFs,which rarely exist in human designed AFs.Our method offers a novel approach for designing general and application-specific BNN architecture.GAAF will be released on GitHub.展开更多
This paper discusses the general decay synchronization problem for a class of fuzzy competitive neural networks with time-varying delays and discontinuous activation functions. Firstly, based on the concept of Filippo...This paper discusses the general decay synchronization problem for a class of fuzzy competitive neural networks with time-varying delays and discontinuous activation functions. Firstly, based on the concept of Filippov solutions for right-hand discontinuous systems, some sufficient conditions for general decay synchronization of the considered system are obtained via designing a nonlinear feedback controller and applying discontinuous differential equation theory, Lyapunov functional methods and some inequality techniques. Finally, one numerical example is given to verify the effectiveness of the proposed theoretical results. The general decay synchronization considered in this article can better estimate the convergence rate of the system, and the exponential synchronization and polynomial synchronization can be seen as its special cases.展开更多
In the field of image processing,the analysis of Synthetic Aperture Radar(SAR)images is crucial due to its broad range of applications.However,SAR images are often affected by coherent speckle noise,which significantl...In the field of image processing,the analysis of Synthetic Aperture Radar(SAR)images is crucial due to its broad range of applications.However,SAR images are often affected by coherent speckle noise,which significantly degrades image quality.Traditional denoising methods,typically based on filter techniques,often face challenges related to inefficiency and limited adaptability.To address these limitations,this study proposes a novel SAR image denoising algorithm based on an enhanced residual network architecture,with the objective of enhancing the utility of SAR imagery in complex electromagnetic environments.The proposed algorithm integrates residual network modules,which directly process the noisy input images to generate denoised outputs.This approach not only reduces computational complexity but also mitigates the difficulties associated with model training.By combining the Transformer module with the residual block,the algorithm enhances the network's ability to extract global features,offering superior feature extraction capabilities compared to CNN-based residual modules.Additionally,the algorithm employs the adaptive activation function Meta-ACON,which dynamically adjusts the activation patterns of neurons,thereby improving the network's feature extraction efficiency.The effectiveness of the proposed denoising method is empirically validated using real SAR images from the RSOD dataset.The proposed algorithm exhibits remarkable performance in terms of EPI,SSIM,and ENL,while achieving a substantial enhancement in PSNR when compared to traditional and deep learning-based algorithms.The PSNR performance is enhanced by over twofold.Moreover,the evaluation of the MSTAR SAR dataset substantiates the algorithm's robustness and applicability in SAR denoising tasks,with a PSNR of 25.2021 being attained.These findings underscore the efficacy of the proposed algorithm in mitigating speckle noise while preserving critical features in SAR imagery,thereby enhancing its quality and usability in practical scenarios.展开更多
Objective:To conduct a scoping review on the application status of the Functional Activity Score(FAS)in postoperative active pain management in China,providing a reference for its standardized and normative promotion....Objective:To conduct a scoping review on the application status of the Functional Activity Score(FAS)in postoperative active pain management in China,providing a reference for its standardized and normative promotion.Methods:Computerized searches of Chinese and English databases were performed to collect studies published by Chinese scholars from 2005 to July 2025 on the application of FAS in postoperative active pain management.After strict screening,the basic characteristics,application fields,assessment models,evaluation timing,types of functional activities,and clinical outcomes of the included literature were systematically analyzed.Results:A total of 18 studies were included,involving surgical types such as thoracic surgery,general surgery,and orthopedics.All studies adopted FAS combined with the Numeric Rating Scale(NRS)for assessment,with evaluation timing mostly concentrated within 72 hours postoperatively.The selected functional activities primarily included respiration-related and limb movements.Evaluation indicators covered pain control,functional recovery,complications,adverse events,patient experience,and tool assessment,with most studies reporting positive outcomes.Conclusion:FAS can effectively enhance pain control and promote functional recovery in postoperative active pain management in China,demonstrating high clinical value.However,existing studies exhibit inconsistencies in assessment criteria,selection of activity types,and research quality.展开更多
Bilateral arm raising movements have been used in brain rehabilitation for a long time. However, no study has been reported on the effect of these movements on the cerebral cortex. In this study, using functional near...Bilateral arm raising movements have been used in brain rehabilitation for a long time. However, no study has been reported on the effect of these movements on the cerebral cortex. In this study, using functional near infrared spectroscopy（f NIRS）, we attempted to investigate cortical activation generated during bilateral arm raising movements. Ten normal subjects were recruited for this study. f NIRS was performed using an f NIRS system with 49 channels. Bilateral arm raising movements were performed in sitting position at the rate of 0.5 Hz. We measured values of oxyhemoglobin and total hemoglobin in five regions of interest： the primary sensorimotor cortex, premotor cortex, supplementary motor area, prefrontal cortex, and posterior parietal cortex. During performance of bilateral arm raising movements, oxyhemoglobin and total hemoglobin values in the primary sensorimotor cortex, premotor cortex, supplementary motor area, and prefrontal cortex were similar, but higher in these regions than those in the prefrontal cortex. We observed activation of the arm somatotopic areas of the primary sensorimotor cortex and premotor cortex in both hemispheres during bilateral arm raising movements. According to this result, bilateral arm raising movements appeared to induce large-scale neuronal activation and therefore arm raising movements would be good exercise for recovery of brain functions.展开更多
Recently,deep learning has achieved remarkable results in fields that require human cognitive ability,learning ability,and reasoning ability.Activation functions are very important because they provide the ability of ...Recently,deep learning has achieved remarkable results in fields that require human cognitive ability,learning ability,and reasoning ability.Activation functions are very important because they provide the ability of artificial neural networks to learn complex patterns through nonlinearity.Various activation functions are being studied to solve problems such as vanishing gradients and dying nodes that may occur in the deep learning process.However,it takes a lot of time and effort for researchers to use the existing activation function in their research.Therefore,in this paper,we propose a universal activation function(UA)so that researchers can easily create and apply various activation functions and improve the performance of neural networks.UA can generate new types of activation functions as well as functions like traditional activation functions by properly adjusting three hyperparameters.The famous Convolutional Neural Network(CNN)and benchmark datasetwere used to evaluate the experimental performance of the UA proposed in this study.We compared the performance of the artificial neural network to which the traditional activation function is applied and the artificial neural network to which theUA is applied.In addition,we evaluated the performance of the new activation function generated by adjusting the hyperparameters of theUA.The experimental performance evaluation results showed that the classification performance of CNNs improved by up to 5%through the UA,although most of them showed similar performance to the traditional activation function.展开更多
Machine Learning(ML)and Deep Learning(DL)technologies are revolutionizing the medical domain,especially with Electrocardiogram(ECG),by providing new tools and techniques for diagnosing,treating,and preventing diseases...Machine Learning(ML)and Deep Learning(DL)technologies are revolutionizing the medical domain,especially with Electrocardiogram(ECG),by providing new tools and techniques for diagnosing,treating,and preventing diseases.However,DL architectures are computationally more demanding.In recent years,researchers have focused on combining the computationally less intensive portion of the DL architectures with ML approaches,say for example,combining the convolutional layer blocks of Convolution Neural Networks(CNNs)into ML algorithms such as Extreme Gradient Boosting(XGBoost)and K-Nearest Neighbor(KNN)resulting in CNN-XGBoost and CNN-KNN,respectively.However,these approaches are homogenous in the sense that they use a fixed Activation Function(AFs)in the sequence of convolution and pooling layers,thereby limiting the ability to capture unique features.Since various AFs are readily available and each could capture unique features,we propose a Convolutionbased Heterogeneous Activation Facility(CHAF)which uses multiple AFs in the convolution layer blocks,one for each block,with a motivation of extracting features in a better manner to improve the accuracy.The proposed CHAF approach is validated on PTB and shown to outperform the homogeneous approaches such as CNN-KNN and CNN-XGBoost.For PTB dataset,proposed CHAF-KNN has an accuracy of 99.55%and an F1 score of 99.68%in just 0.008 s,outperforming the state-of-the-art CNN-XGBoost which has an accuracy of 99.38%and an F1 score of 99.32%in 1.23 s.To validate the generality of the proposed CHAF,experiments were repeated on MIT-BIH dataset,and the proposed CHAF-KNN is shown to outperform CNN-KNN and CNN-XGBoost.展开更多
Density functional theory calculations were carried out to explore the potential energy surface（PES） associated with the gas-phase reaction of Ni L2（L=SO3CH3） with acetone. The geometries and energies of the react...Density functional theory calculations were carried out to explore the potential energy surface（PES） associated with the gas-phase reaction of Ni L2（L=SO3CH3） with acetone. The geometries and energies of the reactants, intermediates, products and transition states of the triplet ground potential energy surfaces of [Ni, O, C2, H4] were obtained at the B3LYP/6-311＋＋G（d,p） levels in C,H,O atoms and B3LYP/ Lanl2 dz in Ni atom. It was found through our calculations that the decabonylation of acetaldehyde contains four steps including encounter complexation, C-C activation, aldehyde H-shift and nonreactive dissociation. The results revealed that C-C activation induced by Ni L2（L=SO3CH3） led to the decarbonylation of acetaldehyde.展开更多
Accurate prediction of the rate of penetration(ROP)is significant for drilling optimization.While the intelligent ROP prediction model based on fully connected neural networks(FNN)outperforms traditional ROP equations...Accurate prediction of the rate of penetration(ROP)is significant for drilling optimization.While the intelligent ROP prediction model based on fully connected neural networks(FNN)outperforms traditional ROP equations and machine learning algorithms,its lack of interpretability undermines its credibility.This study proposes a novel interpretation and characterization method for the FNN ROP prediction model using the Rectified Linear Unit(ReLU)activation function.By leveraging the derivative of the ReLU function,the FNN function calculation process is transformed into vector operations.The FNN model is linearly characterized through further simplification,enabling its interpretation and analysis.The proposed method is applied in ROP prediction scenarios using drilling data from three vertical wells in the Tarim Oilfield.The results demonstrate that the FNN ROP prediction model with ReLU as the activation function performs exceptionally well.The relative activation frequency curve of hidden layer neurons aids in analyzing the overfitting of the FNN ROP model and determining drilling data similarity.In the well sections with similar drilling data,averaging the weight parameters enables linear characterization of the FNN ROP prediction model,leading to the establishment of a corresponding linear representation equation.Furthermore,the quantitative analysis of each feature's influence on ROP facilitates the proposal of drilling parameter optimization schemes for the current well section.The established linear characterization equation exhibits high precision,strong stability,and adaptability through the application and validation across multiple well sections.展开更多
This paper presents a novel framework aimed at quantifying uncertainties associated with the 3D reconstruction of smoke from2Dimages.This approach reconstructs color and density fields from 2D images using Neural Radi...This paper presents a novel framework aimed at quantifying uncertainties associated with the 3D reconstruction of smoke from2Dimages.This approach reconstructs color and density fields from 2D images using Neural Radiance Field(NeRF)and improves image quality using frequency regularization.The NeRF model is obtained via joint training ofmultiple artificial neural networks,whereby the expectation and standard deviation of density fields and RGB values can be evaluated for each pixel.In addition,customized physics-informed neural network(PINN)with residual blocks and two-layer activation functions are utilized to input the density fields of the NeRF into Navier-Stokes equations and convection-diffusion equations to reconstruct the velocity field.The velocity uncertainties are also evaluated through ensemble learning.The effectiveness of the proposed algorithm is demonstrated through numerical examples.The presentmethod is an important step towards downstream tasks such as reliability analysis and robust optimization in engineering design.展开更多
The development of defect prediction plays a significant role in improving software quality. Such predictions are used to identify defective modules before the testing and to minimize the time and cost. The software w...The development of defect prediction plays a significant role in improving software quality. Such predictions are used to identify defective modules before the testing and to minimize the time and cost. The software with defects negatively impacts operational costs and finally affects customer satisfaction. Numerous approaches exist to predict software defects. However, the timely and accurate software bugs are the major challenging issues. To improve the timely and accurate software defect prediction, a novel technique called Nonparametric Statistical feature scaled QuAdratic regressive convolution Deep nEural Network (SQADEN) is introduced. The proposed SQADEN technique mainly includes two major processes namely metric or feature selection and classification. First, the SQADEN uses the nonparametric statistical Torgerson–Gower scaling technique for identifying the relevant software metrics by measuring the similarity using the dice coefficient. The feature selection process is used to minimize the time complexity of software fault prediction. With the selected metrics, software fault perdition with the help of the Quadratic Censored regressive convolution deep neural network-based classification. The deep learning classifier analyzes the training and testing samples using the contingency correlation coefficient. The softstep activation function is used to provide the final fault prediction results. To minimize the error, the Nelder–Mead method is applied to solve non-linear least-squares problems. Finally, accurate classification results with a minimum error are obtained at the output layer. Experimental evaluation is carried out with different quantitative metrics such as accuracy, precision, recall, F-measure, and time complexity. The analyzed results demonstrate the superior performance of our proposed SQADEN technique with maximum accuracy, sensitivity and specificity by 3%, 3%, 2% and 3% and minimum time and space by 13% and 15% when compared with the two state-of-the-art methods.展开更多
文摘The purpose of this study was to address the challenges in predicting and classifying accuracy in modeling Container Dwell Time (CDT) using Artificial Neural Networks (ANN). This objective was driven by the suboptimal outcomes reported in previous studies and sought to apply an innovative approach to improve these results. To achieve this, the study applied the Fusion of Activation Functions (FAFs) to a substantial dataset. This dataset included 307,594 container records from the Port of Tema from 2014 to 2022, encompassing both import and transit containers. The RandomizedSearchCV algorithm from Python’s Scikit-learn library was utilized in the methodological approach to yield the optimal activation function for prediction accuracy. The results indicated that “ajaLT”, a fusion of the Logistic and Hyperbolic Tangent Activation Functions, provided the best prediction accuracy, reaching a high of 82%. Despite these encouraging findings, it’s crucial to recognize the study’s limitations. While Fusion of Activation Functions is a promising method, further evaluation is necessary across different container types and port operations to ascertain the broader applicability and generalizability of these findings. The original value of this study lies in its innovative application of FAFs to CDT. Unlike previous studies, this research evaluates the method based on prediction accuracy rather than training time. It opens new avenues for machine learning engineers and researchers in applying FAFs to enhance prediction accuracy in CDT modeling, contributing to a previously underexplored area.
基金supported by the International Publication Research Grant No.RDU223301.
文摘A vehicle engine cooling system is of utmost importance to ensure that the engine operates in a safe temperature range.In most radiators that are used to cool an engine,water serves as a cooling fluid.The performance of a radiator in terms of heat transmission is significantly influenced by the incorporation of nanoparticles into the cooling water.Concentration and uniformity of nanoparticle distribution are the two major factors for the practical use of nanofluids.The shape and size of nanoparticles also have a great impact on the performance of heat transfer.Many researchers are investigating the impact of nanoparticles on heat transfer.This study aims to develop an artificial neural network(ANN)model for predicting the thermal conductivity of an ethylene glycol(EG)/waterbased crystalline nanocellulose(CNC)nanofluid for cooling internal combustion engine.The implementation of an artificial neural network considering different activation functions in the hidden layer is made to find the bestmodel for the cooling of an engine using the nanofluid.Accuracies of the model with different activation functions in artificial neural networks are analyzed for different nanofluid concentrations and temperatures.In artificial neural networks,Levenberg–Marquardt is an optimization approach used with activation functions,including Tansig and Logsig functions in the training phase.The findings of each training,testing,and validation phase are presented to demonstrate the network that provides the highest level of accuracy.The best result was obtained with Tansig,which has a correlation of 0.99903 and an error of 3.7959×10^(–8).It has also been noticed that the Logsig function can also be a good model due to its correlation of 0.99890 and an error of 4.9218×10^(–8).Thus ourANNwith Tansig and Logsig functions demonstrates a high correlation between the actual output and the predicted output.
基金Supported by the Natural Science Foundation of Hubei Province (2007ABA124)the Youth Project Foundation of Hubei Province Education Department (Q200722001)the Major Foundation of Hubei Province Education Department (D200722002)
文摘The stability of a periodic oscillation and the global exponential class of recurrent neural networks with non-monotone activation functions and time-varying delays are analyzed. For two sets of activation functions, some algebraic criteria for ascertaining global exponential periodicity and global exponential stability of the class of recurrent neural networks are derived by using the comparison principle and the theory of monotone operator. These conditions are easy to check in terms of system parameters. In addition, we provide a new and efficacious method for the qualitative analysis of various neural networks.
基金Project supported by the National Natural Science Foundation of China(Grant Nos.61374094 and 61503338)the Natural Science Foundation of Zhejiang Province,China(Grant No.LQ15F030005)
文摘In this paper, the multistability issue is discussed for delayed complex-valued recurrent neural networks with discontinuous real-imaginary-type activation functions. Based on a fixed theorem and stability definition, sufficient criteria are established for the existence and stability of multiple equilibria of complex-valued recurrent neural networks. The number of stable equilibria is larger than that of real-valued recurrent neural networks, which can be used to achieve high-capacity associative memories. One numerical example is provided to show the effectiveness and superiority of the presented results.
文摘Activation functions play an essential role in converting the output of the artificial neural network into nonlinear results,since without this nonlinearity,the results of the network will be less accurate.Nonlinearity is the mission of all nonlinear functions,except for polynomials.The activation function must be dif-ferentiable for backpropagation learning.This study’s objective is to determine the best activation functions for the approximation of each fractal image.Different results have been attained using Matlab and Visual Basic programs,which indi-cate that the bounded function is more helpful than other functions.The non-lin-earity of the activation function is important when using neural networks for coding fractal images because the coefficients of the Iterated Function System are different according to the different types of fractals.The most commonly cho-sen activation function is the sigmoidal function,which produces a positive value.Other functions,such as tansh or arctan,whose values can be positive or negative depending on the network input,tend to train neural networks faster.The coding speed of the fractal image is different depending on the appropriate activation function chosen for each fractal shape.In this paper,we have provided the appro-priate activation functions for each type of system of iterated functions that help the network to identify the transactions of the system.
文摘This paper describes our implementation of several neural networks built on a field programmable gate array (FPGA) and used to recognize a handwritten digit dataset—the Modified National Institute of Standards and Technology (MNIST) database. We also propose a novel hardware-friendly activation function called the dynamic Rectifid Linear Unit (ReLU)—D-ReLU function that achieves higher performance than traditional activation functions at no cost to accuracy. We built a 2-layer online training multilayer perceptron (MLP) neural network on an FPGA with varying data width. Reducing the data width from 8 to 4 bits only reduces prediction accuracy by 11%, but the FPGA area decreases by 41%. Compared to networks that use the sigmoid functions, our proposed D-ReLU function uses 24% - 41% less area with no loss to prediction accuracy. Further reducing the data width of the 3-layer networks from 8 to 4 bits, the prediction accuracies only decrease by 3% - 5%, with area being reduced by 9% - 28%. Moreover, FPGA solutions have 29 times faster execution time, even despite running at a 60× lower clock rate. Thus, FPGA implementations of neural networks offer a high-performance, low power alternative to traditional software methods, and our novel D-ReLU activation function offers additional improvements to performance and power saving.
基金The researchers would like to thank the Deanship of Graduate Studies and Scientific Research at Qassim University for financial support(QU-APC-2025).
文摘Deep learning(DL),derived from the domain of Artificial Neural Networks(ANN),forms one of the most essential components of modern deep learning algorithms.DL segmentation models rely on layer-by-layer convolution-based feature representation,guided by forward and backward propagation.Acritical aspect of this process is the selection of an appropriate activation function(AF)to ensure robustmodel learning.However,existing activation functions often fail to effectively address the vanishing gradient problem or are complicated by the need for manual parameter tuning.Most current research on activation function design focuses on classification tasks using natural image datasets such asMNIST,CIFAR-10,and CIFAR-100.To address this gap,this study proposesMed-ReLU,a novel activation function specifically designed for medical image segmentation.Med-ReLU prevents deep learning models fromsuffering dead neurons or vanishing gradient issues.It is a hybrid activation function that combines the properties of ReLU and Softsign.For positive inputs,Med-ReLU adopts the linear behavior of ReLU to avoid vanishing gradients,while for negative inputs,it exhibits the Softsign’s polynomial convergence,ensuring robust training and avoiding inactive neurons across the training set.The training performance and segmentation accuracy ofMed-ReLU have been thoroughly evaluated,demonstrating stable learning behavior and resistance to overfitting.It consistently outperforms state-of-the-art activation functions inmedical image segmentation tasks.Designed as a parameter-free function,Med-ReLU is simple to implement in complex deep learning architectures,and its effectiveness spans various neural network models and anomaly detection scenarios.
基金supported in part by the National Natural Science Foundation of China under Grants 62276150the Guoqiang Institute of Tsinghua University.
文摘Physics-informed neural networks(PINNs)are known to suffer from optimization difficulty.In this work,we reveal the connection between the optimization difficulty of PINNs and activation functions.Specifically,we show that PINNs exhibit high sensitivity to activation functions when solving PDEs with distinct properties.Existing works usually choose activation functions by inefficient trial-and-error.To avoid the inefficient manual selection and to alleviate the optimization difficulty of PINNs,we introduce adaptive activation functions to search for the optimal function when solving different problems.We compare different adaptive activation functions and discuss their limitations in the context of PINNs.Furthermore,we propose to tailor the idea of learning combinations of candidate activation functions to the PINNs optimization,which has a higher requirement for the smoothness and diversity on learned functions.This is achieved by removing activation functions which cannot provide higher-order derivatives from the candidate set and incorporating elementary functions with different properties according to our prior knowledge about the PDE at hand.We further enhance the search space with adaptive slopes.The proposed adaptive activation function can be used to solve different PDE systems in an interpretable way.Its effectiveness is demonstrated on a series of benchmarks.Code is available at https://github.com/LeapLabTHU/AdaAFforPINNs.
基金supported by the National Natural Science Foundation of China No.61273022the Research Foundation of Department of Education of Liaoning Province No.JDL2017031.
文摘Purpose–The purpose of this paper is to develop a method for the existence,uniqueness and globally robust stability of the equilibrium point for Cohen–Grossberg neural networks with time-varying delays,continuous distributed delays and a kind of discontinuous activation functions.Design/methodology/approach–Basedonthe Leray–Schauderalternativetheoremand chainrule,by using a novel integral inequality dealing with monotone non-decreasing function,the authors obtain a delay-dependent sufficient condition with less conservativeness for robust stability of considered neural networks.Findings–Itturns out thattheauthors’delay-dependent sufficientcondition canbeformed intermsof linear matrix inequalities conditions.Two examples show the effectiveness of the obtained results.Originality/value–The novelty of the proposed approach lies in dealing with a new kind of discontinuous activation functions by using the Leray–Schauder alternative theorem,chain rule and a novel integral inequality on monotone non-decreasing function.
文摘Binary neural networks(BNNs)show promising utilization in cost and power-restricted domains such as edge devices and mobile systems.This is due to its significantly less computation and storage demand,but at the cost of degraded performance.To close the accuracy gap,in this paper we propose to add a complementary activation function(AF)ahead of the sign based binarization,and rely on the genetic algorithm(GA)to automatically search for the ideal AFs.These AFs can help extract extra information from the input data in the forward pass,while allowing improved gradient approximation in the backward pass.Fifteen novel AFs are identified through our GA-based search,while most of them show improved performance(up to 2.54%on ImageNet)when testing on different datasets and network models.Interestingly,periodic functions are identified as a key component for most of the discovered AFs,which rarely exist in human designed AFs.Our method offers a novel approach for designing general and application-specific BNN architecture.GAAF will be released on GitHub.
文摘This paper discusses the general decay synchronization problem for a class of fuzzy competitive neural networks with time-varying delays and discontinuous activation functions. Firstly, based on the concept of Filippov solutions for right-hand discontinuous systems, some sufficient conditions for general decay synchronization of the considered system are obtained via designing a nonlinear feedback controller and applying discontinuous differential equation theory, Lyapunov functional methods and some inequality techniques. Finally, one numerical example is given to verify the effectiveness of the proposed theoretical results. The general decay synchronization considered in this article can better estimate the convergence rate of the system, and the exponential synchronization and polynomial synchronization can be seen as its special cases.
文摘In the field of image processing,the analysis of Synthetic Aperture Radar(SAR)images is crucial due to its broad range of applications.However,SAR images are often affected by coherent speckle noise,which significantly degrades image quality.Traditional denoising methods,typically based on filter techniques,often face challenges related to inefficiency and limited adaptability.To address these limitations,this study proposes a novel SAR image denoising algorithm based on an enhanced residual network architecture,with the objective of enhancing the utility of SAR imagery in complex electromagnetic environments.The proposed algorithm integrates residual network modules,which directly process the noisy input images to generate denoised outputs.This approach not only reduces computational complexity but also mitigates the difficulties associated with model training.By combining the Transformer module with the residual block,the algorithm enhances the network's ability to extract global features,offering superior feature extraction capabilities compared to CNN-based residual modules.Additionally,the algorithm employs the adaptive activation function Meta-ACON,which dynamically adjusts the activation patterns of neurons,thereby improving the network's feature extraction efficiency.The effectiveness of the proposed denoising method is empirically validated using real SAR images from the RSOD dataset.The proposed algorithm exhibits remarkable performance in terms of EPI,SSIM,and ENL,while achieving a substantial enhancement in PSNR when compared to traditional and deep learning-based algorithms.The PSNR performance is enhanced by over twofold.Moreover,the evaluation of the MSTAR SAR dataset substantiates the algorithm's robustness and applicability in SAR denoising tasks,with a PSNR of 25.2021 being attained.These findings underscore the efficacy of the proposed algorithm in mitigating speckle noise while preserving critical features in SAR imagery,thereby enhancing its quality and usability in practical scenarios.
基金School-level Educational Reform Project of Hangzhou Normal University(Project No.:HLXYJG202304)。
文摘Objective:To conduct a scoping review on the application status of the Functional Activity Score(FAS)in postoperative active pain management in China,providing a reference for its standardized and normative promotion.Methods:Computerized searches of Chinese and English databases were performed to collect studies published by Chinese scholars from 2005 to July 2025 on the application of FAS in postoperative active pain management.After strict screening,the basic characteristics,application fields,assessment models,evaluation timing,types of functional activities,and clinical outcomes of the included literature were systematically analyzed.Results:A total of 18 studies were included,involving surgical types such as thoracic surgery,general surgery,and orthopedics.All studies adopted FAS combined with the Numeric Rating Scale(NRS)for assessment,with evaluation timing mostly concentrated within 72 hours postoperatively.The selected functional activities primarily included respiration-related and limb movements.Evaluation indicators covered pain control,functional recovery,complications,adverse events,patient experience,and tool assessment,with most studies reporting positive outcomes.Conclusion:FAS can effectively enhance pain control and promote functional recovery in postoperative active pain management in China,demonstrating high clinical value.However,existing studies exhibit inconsistencies in assessment criteria,selection of activity types,and research quality.
基金supported by the DGIST R&D Program of the Ministry of Science,ICT and Future Planning,No.16-BD-0401
文摘Bilateral arm raising movements have been used in brain rehabilitation for a long time. However, no study has been reported on the effect of these movements on the cerebral cortex. In this study, using functional near infrared spectroscopy（f NIRS）, we attempted to investigate cortical activation generated during bilateral arm raising movements. Ten normal subjects were recruited for this study. f NIRS was performed using an f NIRS system with 49 channels. Bilateral arm raising movements were performed in sitting position at the rate of 0.5 Hz. We measured values of oxyhemoglobin and total hemoglobin in five regions of interest： the primary sensorimotor cortex, premotor cortex, supplementary motor area, prefrontal cortex, and posterior parietal cortex. During performance of bilateral arm raising movements, oxyhemoglobin and total hemoglobin values in the primary sensorimotor cortex, premotor cortex, supplementary motor area, and prefrontal cortex were similar, but higher in these regions than those in the prefrontal cortex. We observed activation of the arm somatotopic areas of the primary sensorimotor cortex and premotor cortex in both hemispheres during bilateral arm raising movements. According to this result, bilateral arm raising movements appeared to induce large-scale neuronal activation and therefore arm raising movements would be good exercise for recovery of brain functions.
基金This work was supported by the National Research Foundation of Korea(NRF)grant funded by the Korea government(MSIT)(No.2022R1F1A1062953).
文摘Recently,deep learning has achieved remarkable results in fields that require human cognitive ability,learning ability,and reasoning ability.Activation functions are very important because they provide the ability of artificial neural networks to learn complex patterns through nonlinearity.Various activation functions are being studied to solve problems such as vanishing gradients and dying nodes that may occur in the deep learning process.However,it takes a lot of time and effort for researchers to use the existing activation function in their research.Therefore,in this paper,we propose a universal activation function(UA)so that researchers can easily create and apply various activation functions and improve the performance of neural networks.UA can generate new types of activation functions as well as functions like traditional activation functions by properly adjusting three hyperparameters.The famous Convolutional Neural Network(CNN)and benchmark datasetwere used to evaluate the experimental performance of the UA proposed in this study.We compared the performance of the artificial neural network to which the traditional activation function is applied and the artificial neural network to which theUA is applied.In addition,we evaluated the performance of the new activation function generated by adjusting the hyperparameters of theUA.The experimental performance evaluation results showed that the classification performance of CNNs improved by up to 5%through the UA,although most of them showed similar performance to the traditional activation function.
文摘Machine Learning(ML)and Deep Learning(DL)technologies are revolutionizing the medical domain,especially with Electrocardiogram(ECG),by providing new tools and techniques for diagnosing,treating,and preventing diseases.However,DL architectures are computationally more demanding.In recent years,researchers have focused on combining the computationally less intensive portion of the DL architectures with ML approaches,say for example,combining the convolutional layer blocks of Convolution Neural Networks(CNNs)into ML algorithms such as Extreme Gradient Boosting(XGBoost)and K-Nearest Neighbor(KNN)resulting in CNN-XGBoost and CNN-KNN,respectively.However,these approaches are homogenous in the sense that they use a fixed Activation Function(AFs)in the sequence of convolution and pooling layers,thereby limiting the ability to capture unique features.Since various AFs are readily available and each could capture unique features,we propose a Convolutionbased Heterogeneous Activation Facility(CHAF)which uses multiple AFs in the convolution layer blocks,one for each block,with a motivation of extracting features in a better manner to improve the accuracy.The proposed CHAF approach is validated on PTB and shown to outperform the homogeneous approaches such as CNN-KNN and CNN-XGBoost.For PTB dataset,proposed CHAF-KNN has an accuracy of 99.55%and an F1 score of 99.68%in just 0.008 s,outperforming the state-of-the-art CNN-XGBoost which has an accuracy of 99.38%and an F1 score of 99.32%in 1.23 s.To validate the generality of the proposed CHAF,experiments were repeated on MIT-BIH dataset,and the proposed CHAF-KNN is shown to outperform CNN-KNN and CNN-XGBoost.
基金Funded by the National Natural Science Foundation of China(No.51174179)
文摘Density functional theory calculations were carried out to explore the potential energy surface（PES） associated with the gas-phase reaction of Ni L2（L=SO3CH3） with acetone. The geometries and energies of the reactants, intermediates, products and transition states of the triplet ground potential energy surfaces of [Ni, O, C2, H4] were obtained at the B3LYP/6-311＋＋G（d,p） levels in C,H,O atoms and B3LYP/ Lanl2 dz in Ni atom. It was found through our calculations that the decabonylation of acetaldehyde contains four steps including encounter complexation, C-C activation, aldehyde H-shift and nonreactive dissociation. The results revealed that C-C activation induced by Ni L2（L=SO3CH3） led to the decarbonylation of acetaldehyde.
基金The authors greatly thanked the financial support from the National Key Research and Development Program of China(funded by National Natural Science Foundation of China,No.2019YFA0708300)the Strategic Cooperation Technology Projects of CNPC and CUPB(funded by China National Petroleum Corporation,No.ZLZX2020-03)+1 种基金the National Science Fund for Distinguished Young Scholars(funded by National Natural Science Foundation of China,No.52125401)Science Foundation of China University of Petroleum,Beijing(funded by China University of petroleum,Beijing,No.2462022SZBH002).
文摘Accurate prediction of the rate of penetration(ROP)is significant for drilling optimization.While the intelligent ROP prediction model based on fully connected neural networks(FNN)outperforms traditional ROP equations and machine learning algorithms,its lack of interpretability undermines its credibility.This study proposes a novel interpretation and characterization method for the FNN ROP prediction model using the Rectified Linear Unit(ReLU)activation function.By leveraging the derivative of the ReLU function,the FNN function calculation process is transformed into vector operations.The FNN model is linearly characterized through further simplification,enabling its interpretation and analysis.The proposed method is applied in ROP prediction scenarios using drilling data from three vertical wells in the Tarim Oilfield.The results demonstrate that the FNN ROP prediction model with ReLU as the activation function performs exceptionally well.The relative activation frequency curve of hidden layer neurons aids in analyzing the overfitting of the FNN ROP model and determining drilling data similarity.In the well sections with similar drilling data,averaging the weight parameters enables linear characterization of the FNN ROP prediction model,leading to the establishment of a corresponding linear representation equation.Furthermore,the quantitative analysis of each feature's influence on ROP facilitates the proposal of drilling parameter optimization schemes for the current well section.The established linear characterization equation exhibits high precision,strong stability,and adaptability through the application and validation across multiple well sections.
基金funded by the National Natural Science Foundation of China(NSFC)(No.52274222)research project supported by Shanxi Scholarship Council of China(No.2023-036).
文摘This paper presents a novel framework aimed at quantifying uncertainties associated with the 3D reconstruction of smoke from2Dimages.This approach reconstructs color and density fields from 2D images using Neural Radiance Field(NeRF)and improves image quality using frequency regularization.The NeRF model is obtained via joint training ofmultiple artificial neural networks,whereby the expectation and standard deviation of density fields and RGB values can be evaluated for each pixel.In addition,customized physics-informed neural network(PINN)with residual blocks and two-layer activation functions are utilized to input the density fields of the NeRF into Navier-Stokes equations and convection-diffusion equations to reconstruct the velocity field.The velocity uncertainties are also evaluated through ensemble learning.The effectiveness of the proposed algorithm is demonstrated through numerical examples.The presentmethod is an important step towards downstream tasks such as reliability analysis and robust optimization in engineering design.
文摘The development of defect prediction plays a significant role in improving software quality. Such predictions are used to identify defective modules before the testing and to minimize the time and cost. The software with defects negatively impacts operational costs and finally affects customer satisfaction. Numerous approaches exist to predict software defects. However, the timely and accurate software bugs are the major challenging issues. To improve the timely and accurate software defect prediction, a novel technique called Nonparametric Statistical feature scaled QuAdratic regressive convolution Deep nEural Network (SQADEN) is introduced. The proposed SQADEN technique mainly includes two major processes namely metric or feature selection and classification. First, the SQADEN uses the nonparametric statistical Torgerson–Gower scaling technique for identifying the relevant software metrics by measuring the similarity using the dice coefficient. The feature selection process is used to minimize the time complexity of software fault prediction. With the selected metrics, software fault perdition with the help of the Quadratic Censored regressive convolution deep neural network-based classification. The deep learning classifier analyzes the training and testing samples using the contingency correlation coefficient. The softstep activation function is used to provide the final fault prediction results. To minimize the error, the Nelder–Mead method is applied to solve non-linear least-squares problems. Finally, accurate classification results with a minimum error are obtained at the output layer. Experimental evaluation is carried out with different quantitative metrics such as accuracy, precision, recall, F-measure, and time complexity. The analyzed results demonstrate the superior performance of our proposed SQADEN technique with maximum accuracy, sensitivity and specificity by 3%, 3%, 2% and 3% and minimum time and space by 13% and 15% when compared with the two state-of-the-art methods.
