Purpose: This study aimed to enhance the prediction of container dwell time, a crucial factor for optimizing port operations, resource allocation, and supply chain efficiency. Determining an optimal learning rate for ...Purpose: This study aimed to enhance the prediction of container dwell time, a crucial factor for optimizing port operations, resource allocation, and supply chain efficiency. Determining an optimal learning rate for training Artificial Neural Networks (ANNs) has remained a challenging task due to the diverse sizes, complexity, and types of data involved. Design/Method/Approach: This research used a RandomizedSearchCV algorithm, a random search approach, to bridge this knowledge gap. The algorithm was applied to container dwell time data from the TOS system of the Port of Tema, which included 307,594 container records from 2014 to 2022. Findings: The RandomizedSearchCV method outperformed standard training methods both in terms of reducing training time and improving prediction accuracy, highlighting the significant role of the constant learning rate as a hyperparameter. Research Limitations and Implications: Although the study provides promising outcomes, the results are limited to the data extracted from the Port of Tema and may differ in other contexts. Further research is needed to generalize these findings across various port systems. Originality/Value: This research underscores the potential of RandomizedSearchCV as a valuable tool for optimizing ANN training in container dwell time prediction. It also accentuates the significance of automated learning rate selection, offering novel insights into the optimization of container dwell time prediction, with implications for improving port efficiency and supply chain operations.展开更多
For accelerating the supervised learning by the SpikeProp algorithm with the temporal coding paradigm in spiking neural networks (SNNs), three learning rate adaptation methods (heuristic rule, delta-delta rule, and de...For accelerating the supervised learning by the SpikeProp algorithm with the temporal coding paradigm in spiking neural networks (SNNs), three learning rate adaptation methods (heuristic rule, delta-delta rule, and delta-bar-delta rule), which are used to speed up training in artificial neural networks, are used to develop the training algorithms for feedforward SNN. The performance of these algorithms is investigated by four experiments: classical XOR (exclusive or) problem, Iris dataset, fault diagnosis in the Tennessee Eastman process, and Poisson trains of discrete spikes. The results demonstrate that all the three learning rate adaptation methods are able to speed up convergence of SNN compared with the original SpikeProp algorithm. Furthermore, if the adaptive learning rate is used in combination with the momentum term, the two modifications will balance each other in a beneficial way to accomplish rapid and steady convergence. In the three learning rate adaptation methods, delta-bar-delta rule performs the best. The delta-bar-delta method with momentum has the fastest convergence rate, the greatest stability of training process, and the maximum accuracy of network learning. The proposed algorithms in this paper are simple and efficient, and consequently valuable for practical applications of SNN.展开更多
This paper considers a robust kernel regularized classification algorithm with a non-convex loss function which is proposed to alleviate the performance deterioration caused by the outliers.A comparison relationship b...This paper considers a robust kernel regularized classification algorithm with a non-convex loss function which is proposed to alleviate the performance deterioration caused by the outliers.A comparison relationship between the excess misclassification error and the excess generalization error is provided;from this,along with the convex analysis theory,a kind of learning rate is derived.The results show that the performance of the classifier is effected by the outliers,and the extent of impact can be controlled by choosing the homotopy parameters properly.展开更多
A dynamic learning rate Gaussian mixture model（GMM）algorithm is proposed to deal with the problem of slow adaption of GMM in the case of moving object detection in the outdoor surveillance,especially in the presence...A dynamic learning rate Gaussian mixture model（GMM）algorithm is proposed to deal with the problem of slow adaption of GMM in the case of moving object detection in the outdoor surveillance,especially in the presence of sudden illumination changes.The GMM is mostly used for detecting objects in complex scenes for intelligent monitoring systems.To solve this problem,a mixture Gaussian model has been built for each pixel in the video frame,and according to the scene change from the frame difference,the learning rate of GMM can be dynamically adjusted.The experiments show that the proposed method gives good results with an adaptive GMM learning rate when we compare it with GMM method with a fixed learning rate.The method was tested on a certain dataset,and tests in the case of sudden natural light changes show that our method has a better accuracy and lower false alarm rate.展开更多
A new algorithm to exploit the learning rates of gradient descent method is presented, based on the second-order Taylor expansion of the error energy function with respect to learning rate, at some values decided by &...A new algorithm to exploit the learning rates of gradient descent method is presented, based on the second-order Taylor expansion of the error energy function with respect to learning rate, at some values decided by "award-punish" strategy. Detailed deduction of the algorithm applied to RBF networks is given. Simulation studies show that this algorithm can increase the rate of convergence and improve the performance of the gradient descent method.展开更多
The rapid spread of the novel Coronavirus (COVID-19) has emphasized the necessity for advanced diagnostic tools to enhance the detection and management of the virus. This study investigates the effectiveness of Convol...The rapid spread of the novel Coronavirus (COVID-19) has emphasized the necessity for advanced diagnostic tools to enhance the detection and management of the virus. This study investigates the effectiveness of Convolutional Neural Networks (CNNs) in the diagnosis of COVID-19 from chest X-ray and CT images, focusing on the impact of varying learning rates and optimization strategies. Despite the abundance of chest X-ray datasets from various institutions, the lack of a dedicated COVID-19 dataset for computational analysis presents a significant challenge. Our work introduces an empirical analysis across four distinct learning rate policies—Cyclic, Step Based, Time-Based, and Epoch Based—each tested with four different optimizers: Adam, Adagrad, RMSprop, and Stochastic Gradient Descent (SGD). The performance of these configurations was evaluated in terms of training and validation accuracy over 100 epochs. Our results demonstrate significant differences in model performance, with the Cyclic learning rate policy combined with SGD optimizer achieving the highest validation accuracy of 83.33%. This study contributes to the existing body of knowledge by outlining effective CNN configurations for COVID-19 image dataset analysis, offering insights into the optimization of machine learning models for the diagnosis of infectious diseases. Our findings underscore the potential of CNNs in supplementing traditional PCR tests, providing a computational approach to identify patterns in chest X-rays and CT scans indicative of COVID-19, thereby aiding in the swift and accurate diagnosis of the virus.展开更多
Complex road conditions without signalized intersections when the traffic flow is nearly saturated result in high traffic congestion and accidents,reducing the traffic efficiency of intelligent vehicles.The complex ro...Complex road conditions without signalized intersections when the traffic flow is nearly saturated result in high traffic congestion and accidents,reducing the traffic efficiency of intelligent vehicles.The complex road traffic environment of smart vehicles and other vehicles frequently experiences conflicting start and stop motion.The fine-grained scheduling of autonomous vehicles(AVs)at non-signalized intersections,which is a promising technique for exploring optimal driving paths for both assisted driving nowadays and driverless cars in the near future,has attracted significant attention owing to its high potential for improving road safety and traffic efficiency.Fine-grained scheduling primarily focuses on signalized intersection scenarios,as applying it directly to non-signalized intersections is challenging because each AV can move freely without traffic signal control.This may cause frequent driving collisions and low road traffic efficiency.Therefore,this study proposes a novel algorithm to address this issue.Our work focuses on the fine-grained scheduling of automated vehicles at non-signal intersections via dual reinforced training(FS-DRL).For FS-DRL,we first use a grid to describe the non-signalized intersection and propose a convolutional neural network(CNN)-based fast decision model that can rapidly yield a coarse-grained scheduling decision for each AV in a distributed manner.We then load these coarse-grained scheduling decisions onto a deep Q-learning network(DQN)for further evaluation.We use an adaptive learning rate to maximize the reward function and employ parameterεto tradeoff the fast speed of coarse-grained scheduling in the CNN and optimal fine-grained scheduling in the DQN.In addition,we prove that using this adaptive learning rate leads to a converged loss rate with an extremely small number of training loops.The simulation results show that compared with Dijkstra,RNN,and ant colony-based scheduling,FS-DRL yields a high accuracy of 96.5%on the sample,with improved performance of approximately 61.54%-85.37%in terms of the average conflict and traffic efficiency.展开更多
Skin cancer is the abnormal development of cells on the surface of the skin and is one of the most fatal diseases in humans.It usually appears in locations that are exposed to the sun,but can also appear in areas that...Skin cancer is the abnormal development of cells on the surface of the skin and is one of the most fatal diseases in humans.It usually appears in locations that are exposed to the sun,but can also appear in areas that are not regularly exposed to the sun.Due to the striking similarities between benign and malignant lesions,skin cancer detection remains a problem,even for expert dermatologists.Considering the inability of dermatologists to di-agnose skin cancer accurately,a convolutional neural network(CNN)approach was used for skin cancer diag-nosis.However,the CNN model requires a significant number of image datasets for better performance;thus,image augmentation and transfer learning techniques have been used in this study to boost the number of images and the performance of the model,because there are a limited number of medical images.This study proposes an ensemble transfer-learning-based model that can efficiently classify skin lesions into one of seven categories to aid dermatologists in skin cancer detection:(i)actinic keratoses,(ii)basal cell carcinoma,(iii)benign keratosis,(iv)dermatofibroma,(v)melanocytic nevi,(vi)melanoma,and(vii)vascular skin lesions.Five transfer learning models were used as the basis of the ensemble:MobileNet,EfficientNetV2B2,Xception,ResNeXt101,and Den-seNet201.In addition to the stratified 10-fold cross-validation,the results of each individual model were fused to achieve greater classification accuracy.An annealing learning rate scheduler and test time augmentation(TTA)were also used to increase the performance of the model during the training and testing stages.A total of 10,015 publicly available dermoscopy images from the HAM10000(Human Against Machine)dataset,which contained samples from the seven common skin lesion categories,were used to train and evaluate the models.The proposed technique attained 94.49%accuracy on the dataset.These results suggest that this strategy can be useful for improving the accuracy of skin cancer classification.However,the weighted average of F1-score,recall,and precision were obtained to be 94.68%,94.49%,and 95.07%,respectively.展开更多
To overcome the shortcomings of traditional dairy cow feed intake assessment model andBP neural network, this paper proposes a method of optimizing BP neural network usingpolynomial decay learning rate, taking the cow...To overcome the shortcomings of traditional dairy cow feed intake assessment model andBP neural network, this paper proposes a method of optimizing BP neural network usingpolynomial decay learning rate, taking the cow’s body weight, lying duration, lying times,walking steps, foraging duration and concentrate-roughage ratio as input variables andtaking the actual feed intake is the output variable to establish a dairy cow feed intakeassessment model, and the model is trained and verified by experimental data collectedon site. For the sake of comparative study, feed intake is simultaneously assessed by SVRmodel, KNN logistic regression model, traditional BP neural network model, and multilayerBP neural network model. The results show that the established BP model using the polynomial decay learning rate has the highest assessment accuracy, the MSPE, RMSE, MAE,MAPE and R2 are 0.043 kg2/d and 0.208 kg/d, 0.173 kg/d, 1.37% and 0.94 respectively. Compared with SVR model and KNN mode, the RMSE value reduced by 43.9% and 26.5%, it isalso found that the model designed in this paper has many advantages in comparison withthe BP model and multilayer BP model in terms of precision and generalization. Therefore,this method is ready to be applied for accurately evaluating the dairy cow feed intake, andit can provide theoretical guidance and technical support for the precise-feeding and canalso be of high significance in the improvement of dairy precise-breeding.展开更多
In the present paper, we give an investigation on the learning rate of l2-coefficient regularized classification with strong loss and the data dependent kernel functional spaces. The results show that the learning rat...In the present paper, we give an investigation on the learning rate of l2-coefficient regularized classification with strong loss and the data dependent kernel functional spaces. The results show that the learning rate is influenced by the strong convexity.展开更多
Stochastic variational inference （SVI） can learn topic models with very big corpora. It optimizes the variational objective by using the stochastic natural gradient algorithm with a decreasing learning rate. This ra...Stochastic variational inference （SVI） can learn topic models with very big corpora. It optimizes the variational objective by using the stochastic natural gradient algorithm with a decreasing learning rate. This rate is crucial for SVI; however, it is often tuned by hand in real applications. To address this, we develop a novel algorithm, which tunes the learning rate of each iteration adaptively. The proposed algorithm uses the Kullback-Leibler （KL） divergence to measure the similarity between the variational distribution with noisy update and that with batch update, and then optimizes the learning rates by minimizing the KL divergence. We apply our algorithm to two representative topic models： latent Dirichlet allocation and hierarchical Dirichlet process. Experimental results indicate that our algorithm performs better and converges faster than commonly used learning rates.展开更多
Van der Pauw＇s function is often used in the measurement of a semiconductor＇s resistivity. However, it is difficult to obtain its value from voltage measurements because it has an implicit form. If it can be express...Van der Pauw＇s function is often used in the measurement of a semiconductor＇s resistivity. However, it is difficult to obtain its value from voltage measurements because it has an implicit form. If it can be expressed as a polynomial, a semiconductor＇s resistivity can be obtained from such measurements. Normally, five orders of the abscissa can provide sufficient precision during the expression of any non-linear function. Therefore, the key is to determine the coefficients of the polynomial. By taking five coefficients as weights to construct a neuronetwork, neurocomputing has been used to solve this problem. Finally, the polynomial expression for van der Pauw＇s function is obtained.展开更多
Nowadays,the amount of wed data is increasing at a rapid speed,which presents a serious challenge to the web monitoring.Text sentiment analysis,an important research topic in the area of natural language processing,is...Nowadays,the amount of wed data is increasing at a rapid speed,which presents a serious challenge to the web monitoring.Text sentiment analysis,an important research topic in the area of natural language processing,is a crucial task in the web monitoring area.The accuracy of traditional text sentiment analysis methods might be degraded in dealing with mass data.Deep learning is a hot research topic of the artificial intelligence in the recent years.By now,several research groups have studied the sentiment analysis of English texts using deep learning methods.In contrary,relatively few works have so far considered the Chinese text sentiment analysis toward this direction.In this paper,a method for analyzing the Chinese text sentiment is proposed based on the convolutional neural network(CNN)in deep learning in order to improve the analysis accuracy.The feature values of the CNN after the training process are nonuniformly distributed.In order to overcome this problem,a method for normalizing the feature values is proposed.Moreover,the dimensions of the text features are optimized through simulations.Finally,a method for updating the learning rate in the training process of the CNN is presented in order to achieve better performances.Experiment results on the typical datasets indicate that the accuracy of the proposed method can be improved compared with that of the traditional supervised machine learning methods,e.g.,the support vector machine method.展开更多
Breast cancer(BC)is the most widespread tumor in females worldwide and is a severe public health issue.BC is the leading reason of death affecting females between the ages of 20 to 59 around the world.Early detection ...Breast cancer(BC)is the most widespread tumor in females worldwide and is a severe public health issue.BC is the leading reason of death affecting females between the ages of 20 to 59 around the world.Early detection and therapy can help women receive effective treatment and,as a result,decrease the rate of breast cancer disease.The cancer tumor develops when cells grow improperly and attack the healthy tissue in the human body.Tumors are classified as benign or malignant,and the absence of cancer in the breast is considered normal.Deep learning,machine learning,and transfer learning models are applied to detect and identify cancerous tissue like BC.This research assists in the identification and classification of BC.We implemented the pre-trained model AlexNet and proposed model Breast cancer identification and classification(BCIC),which are machine learning-based models,by evaluating them in the form of comparative research.We used 3 datasets,A,B,and C.We fuzzed these datasets and got 2 datasets,A2C and B3C.Dataset A2C is the fusion of A,B,and C with 2 classes categorized as benign and malignant.Dataset B3C is the fusion of datasets A,B,and C with 3 classes classified as benign,malignant,and normal.We used customized AlexNet according to our datasets and BCIC in our proposed model.We achieved an accuracy of 86.5%on Dataset B3C and 76.8%on Dataset A2C by using AlexNet,and we achieved the optimum accuracy of 94.5%on Dataset B3C and 94.9%on Dataset A2C by using proposed model BCIC at 40 epochs with 0.00008 learning rate.We proposed fuzzed dataset model using transfer learning.We fuzzed three datasets to get more accurate results and the proposed model achieved the highest prediction accuracy using fuzzed dataset transfer learning technique.展开更多
Evaluation for the performance of learning algorithm has been the main thread of theoretical research of machine learning. The performance of the regularized regression algorithm based on independent and identically d...Evaluation for the performance of learning algorithm has been the main thread of theoretical research of machine learning. The performance of the regularized regression algorithm based on independent and identically distributed（i.i.d.） samples has been researched by a large number of references. In the present paper we provide the convergence rates for the performance of regularized regression based on the inputs of p-order Markov chains.展开更多
To pursue the outperformance of deep nets in learning,we construct a deep net with three hidden layers and prove that,implementing the empirical risk minimization(ERM)on this deep net,the estimator can theoretically r...To pursue the outperformance of deep nets in learning,we construct a deep net with three hidden layers and prove that,implementing the empirical risk minimization(ERM)on this deep net,the estimator can theoretically realize the optimal learning rates without the classical saturation problem.In other words,deepening the networks with only three hidden layers can overcome the saturation and not degrade the optimal learning rates.The obtained results underlie the success of deep nets and provide a theoretical guidance for deep learning.展开更多
This paper investigates some approximation properties and learning rates of Lipschitz kernel on the sphere. A perfect convergence rate on the shifts of Lipschitz kernel on the sphere, which is faster than O（n-1/2）, ...This paper investigates some approximation properties and learning rates of Lipschitz kernel on the sphere. A perfect convergence rate on the shifts of Lipschitz kernel on the sphere, which is faster than O（n-1/2）, is obtained, where n is the number of parameters needed in the approximation. By means of the approximation, a learning rate of regularized least square algorithm with the Lipschitz kernel on the sphere is also deduced.展开更多
Age has always been an important factor in studying second language acquisition.Marinova‐Todd points out three misconceptions about research in support of CPH and disputes the existence of a Critical Period in L2 lea...Age has always been an important factor in studying second language acquisition.Marinova‐Todd points out three misconceptions about research in support of CPH and disputes the existence of a Critical Period in L2 learning.Hyltenstam&Abrahamsson refutes Marinova‐Todd’s“three misconceptions”.By contrasting and comparing the views and evidence from the two papers,the present author analyzes the differences among various claims on the rate of learning and different brain organization,as well as the possible causes for the disagreements.展开更多
Children's English learning in China attracts more and more people's attention and is on the tendency of starting at an early age. Under the trend of "learning English from childhood", the author has...Children's English learning in China attracts more and more people's attention and is on the tendency of starting at an early age. Under the trend of "learning English from childhood", the author has explored the Critical Period Hypothesis and discussed the younger learners' disadvantages and older learners' advantages when learning English. and concludes that early-age English learning is not feasible.展开更多
For training the present Neural Network(NN)models,the standard technique is to utilize decaying Learning Rates(LR).While the majority of these techniques commence with a large LR,they will decay multiple times over ti...For training the present Neural Network(NN)models,the standard technique is to utilize decaying Learning Rates(LR).While the majority of these techniques commence with a large LR,they will decay multiple times over time.Decaying has been proved to enhance generalization as well as optimization.Other parameters,such as the network’s size,the number of hidden layers,drop-outs to avoid overfitting,batch size,and so on,are solely based on heuristics.This work has proposed Adaptive Teaching Learning Based(ATLB)Heuristic to identify the optimal hyperparameters for diverse networks.Here we consider three architec-tures Recurrent Neural Networks(RNN),Long Short Term Memory(LSTM),Bidirectional Long Short Term Memory(BiLSTM)of Deep Neural Networks for classification.The evaluation of the proposed ATLB is done through the various learning rate schedulers Cyclical Learning Rate(CLR),Hyperbolic Tangent Decay(HTD),and Toggle between Hyperbolic Tangent Decay and Triangular mode with Restarts(T-HTR)techniques.Experimental results have shown the performance improvement on the 20Newsgroup,Reuters Newswire and IMDB dataset.展开更多
文摘Purpose: This study aimed to enhance the prediction of container dwell time, a crucial factor for optimizing port operations, resource allocation, and supply chain efficiency. Determining an optimal learning rate for training Artificial Neural Networks (ANNs) has remained a challenging task due to the diverse sizes, complexity, and types of data involved. Design/Method/Approach: This research used a RandomizedSearchCV algorithm, a random search approach, to bridge this knowledge gap. The algorithm was applied to container dwell time data from the TOS system of the Port of Tema, which included 307,594 container records from 2014 to 2022. Findings: The RandomizedSearchCV method outperformed standard training methods both in terms of reducing training time and improving prediction accuracy, highlighting the significant role of the constant learning rate as a hyperparameter. Research Limitations and Implications: Although the study provides promising outcomes, the results are limited to the data extracted from the Port of Tema and may differ in other contexts. Further research is needed to generalize these findings across various port systems. Originality/Value: This research underscores the potential of RandomizedSearchCV as a valuable tool for optimizing ANN training in container dwell time prediction. It also accentuates the significance of automated learning rate selection, offering novel insights into the optimization of container dwell time prediction, with implications for improving port efficiency and supply chain operations.
基金Supported by the National Natural Science Foundation of China (60904018, 61203040)the Natural Science Foundation of Fujian Province of China (2009J05147, 2011J01352)+1 种基金the Foundation for Distinguished Young Scholars of Higher Education of Fujian Province of China (JA10004)the Science Research Foundation of Huaqiao University (09BS617)
文摘For accelerating the supervised learning by the SpikeProp algorithm with the temporal coding paradigm in spiking neural networks (SNNs), three learning rate adaptation methods (heuristic rule, delta-delta rule, and delta-bar-delta rule), which are used to speed up training in artificial neural networks, are used to develop the training algorithms for feedforward SNN. The performance of these algorithms is investigated by four experiments: classical XOR (exclusive or) problem, Iris dataset, fault diagnosis in the Tennessee Eastman process, and Poisson trains of discrete spikes. The results demonstrate that all the three learning rate adaptation methods are able to speed up convergence of SNN compared with the original SpikeProp algorithm. Furthermore, if the adaptive learning rate is used in combination with the momentum term, the two modifications will balance each other in a beneficial way to accomplish rapid and steady convergence. In the three learning rate adaptation methods, delta-bar-delta rule performs the best. The delta-bar-delta method with momentum has the fastest convergence rate, the greatest stability of training process, and the maximum accuracy of network learning. The proposed algorithms in this paper are simple and efficient, and consequently valuable for practical applications of SNN.
基金supported by the NSF(61877039)the NSFC/RGC Joint Research Scheme(12061160462 and N City U 102/20)of China+2 种基金the NSF(LY19F020013)of Zhejiang Provincethe Special Project for Scientific and Technological Cooperation(20212BDH80021)of Jiangxi Provincethe Science and Technology Project in Jiangxi Province Department of Education(GJJ211334)。
文摘This paper considers a robust kernel regularized classification algorithm with a non-convex loss function which is proposed to alleviate the performance deterioration caused by the outliers.A comparison relationship between the excess misclassification error and the excess generalization error is provided;from this,along with the convex analysis theory,a kind of learning rate is derived.The results show that the performance of the classifier is effected by the outliers,and the extent of impact can be controlled by choosing the homotopy parameters properly.
文摘A dynamic learning rate Gaussian mixture model（GMM）algorithm is proposed to deal with the problem of slow adaption of GMM in the case of moving object detection in the outdoor surveillance,especially in the presence of sudden illumination changes.The GMM is mostly used for detecting objects in complex scenes for intelligent monitoring systems.To solve this problem,a mixture Gaussian model has been built for each pixel in the video frame,and according to the scene change from the frame difference,the learning rate of GMM can be dynamically adjusted.The experiments show that the proposed method gives good results with an adaptive GMM learning rate when we compare it with GMM method with a fixed learning rate.The method was tested on a certain dataset,and tests in the case of sudden natural light changes show that our method has a better accuracy and lower false alarm rate.
基金Open Foundation of State Key Lab of Transmission of Wide-Band FiberTechnologies of Communication Systems
文摘A new algorithm to exploit the learning rates of gradient descent method is presented, based on the second-order Taylor expansion of the error energy function with respect to learning rate, at some values decided by "award-punish" strategy. Detailed deduction of the algorithm applied to RBF networks is given. Simulation studies show that this algorithm can increase the rate of convergence and improve the performance of the gradient descent method.
文摘The rapid spread of the novel Coronavirus (COVID-19) has emphasized the necessity for advanced diagnostic tools to enhance the detection and management of the virus. This study investigates the effectiveness of Convolutional Neural Networks (CNNs) in the diagnosis of COVID-19 from chest X-ray and CT images, focusing on the impact of varying learning rates and optimization strategies. Despite the abundance of chest X-ray datasets from various institutions, the lack of a dedicated COVID-19 dataset for computational analysis presents a significant challenge. Our work introduces an empirical analysis across four distinct learning rate policies—Cyclic, Step Based, Time-Based, and Epoch Based—each tested with four different optimizers: Adam, Adagrad, RMSprop, and Stochastic Gradient Descent (SGD). The performance of these configurations was evaluated in terms of training and validation accuracy over 100 epochs. Our results demonstrate significant differences in model performance, with the Cyclic learning rate policy combined with SGD optimizer achieving the highest validation accuracy of 83.33%. This study contributes to the existing body of knowledge by outlining effective CNN configurations for COVID-19 image dataset analysis, offering insights into the optimization of machine learning models for the diagnosis of infectious diseases. Our findings underscore the potential of CNNs in supplementing traditional PCR tests, providing a computational approach to identify patterns in chest X-rays and CT scans indicative of COVID-19, thereby aiding in the swift and accurate diagnosis of the virus.
基金Supported by National Natural Science Foundation of China(Grant No.61803206)Jiangsu Provincial Natural Science Foundation(Grant No.222300420468)Jiangsu Provincial key R&D Program(Grant No.BE2017008-2).
文摘Complex road conditions without signalized intersections when the traffic flow is nearly saturated result in high traffic congestion and accidents,reducing the traffic efficiency of intelligent vehicles.The complex road traffic environment of smart vehicles and other vehicles frequently experiences conflicting start and stop motion.The fine-grained scheduling of autonomous vehicles(AVs)at non-signalized intersections,which is a promising technique for exploring optimal driving paths for both assisted driving nowadays and driverless cars in the near future,has attracted significant attention owing to its high potential for improving road safety and traffic efficiency.Fine-grained scheduling primarily focuses on signalized intersection scenarios,as applying it directly to non-signalized intersections is challenging because each AV can move freely without traffic signal control.This may cause frequent driving collisions and low road traffic efficiency.Therefore,this study proposes a novel algorithm to address this issue.Our work focuses on the fine-grained scheduling of automated vehicles at non-signal intersections via dual reinforced training(FS-DRL).For FS-DRL,we first use a grid to describe the non-signalized intersection and propose a convolutional neural network(CNN)-based fast decision model that can rapidly yield a coarse-grained scheduling decision for each AV in a distributed manner.We then load these coarse-grained scheduling decisions onto a deep Q-learning network(DQN)for further evaluation.We use an adaptive learning rate to maximize the reward function and employ parameterεto tradeoff the fast speed of coarse-grained scheduling in the CNN and optimal fine-grained scheduling in the DQN.In addition,we prove that using this adaptive learning rate leads to a converged loss rate with an extremely small number of training loops.The simulation results show that compared with Dijkstra,RNN,and ant colony-based scheduling,FS-DRL yields a high accuracy of 96.5%on the sample,with improved performance of approximately 61.54%-85.37%in terms of the average conflict and traffic efficiency.
文摘Skin cancer is the abnormal development of cells on the surface of the skin and is one of the most fatal diseases in humans.It usually appears in locations that are exposed to the sun,but can also appear in areas that are not regularly exposed to the sun.Due to the striking similarities between benign and malignant lesions,skin cancer detection remains a problem,even for expert dermatologists.Considering the inability of dermatologists to di-agnose skin cancer accurately,a convolutional neural network(CNN)approach was used for skin cancer diag-nosis.However,the CNN model requires a significant number of image datasets for better performance;thus,image augmentation and transfer learning techniques have been used in this study to boost the number of images and the performance of the model,because there are a limited number of medical images.This study proposes an ensemble transfer-learning-based model that can efficiently classify skin lesions into one of seven categories to aid dermatologists in skin cancer detection:(i)actinic keratoses,(ii)basal cell carcinoma,(iii)benign keratosis,(iv)dermatofibroma,(v)melanocytic nevi,(vi)melanoma,and(vii)vascular skin lesions.Five transfer learning models were used as the basis of the ensemble:MobileNet,EfficientNetV2B2,Xception,ResNeXt101,and Den-seNet201.In addition to the stratified 10-fold cross-validation,the results of each individual model were fused to achieve greater classification accuracy.An annealing learning rate scheduler and test time augmentation(TTA)were also used to increase the performance of the model during the training and testing stages.A total of 10,015 publicly available dermoscopy images from the HAM10000(Human Against Machine)dataset,which contained samples from the seven common skin lesion categories,were used to train and evaluate the models.The proposed technique attained 94.49%accuracy on the dataset.These results suggest that this strategy can be useful for improving the accuracy of skin cancer classification.However,the weighted average of F1-score,recall,and precision were obtained to be 94.68%,94.49%,and 95.07%,respectively.
基金This research is financially supported by National Thirteenth Five-Year National Key R&D Plan(2016YFD0700204)China Postdoctoral Science Foundation(2017M611346)+3 种基金the China Agriculture Research System(CARS-36)the Natural Science Foundation of Heilongjiang Province of China(C2018018)Postdoctoral Science Foundation of Heilongjiang(LBHZ12040)the University Nursing Program for Young Scholars with Creative Talents in Heilongjiang Province under Grant(UNPYSCT-2018143).
文摘To overcome the shortcomings of traditional dairy cow feed intake assessment model andBP neural network, this paper proposes a method of optimizing BP neural network usingpolynomial decay learning rate, taking the cow’s body weight, lying duration, lying times,walking steps, foraging duration and concentrate-roughage ratio as input variables andtaking the actual feed intake is the output variable to establish a dairy cow feed intakeassessment model, and the model is trained and verified by experimental data collectedon site. For the sake of comparative study, feed intake is simultaneously assessed by SVRmodel, KNN logistic regression model, traditional BP neural network model, and multilayerBP neural network model. The results show that the established BP model using the polynomial decay learning rate has the highest assessment accuracy, the MSPE, RMSE, MAE,MAPE and R2 are 0.043 kg2/d and 0.208 kg/d, 0.173 kg/d, 1.37% and 0.94 respectively. Compared with SVR model and KNN mode, the RMSE value reduced by 43.9% and 26.5%, it isalso found that the model designed in this paper has many advantages in comparison withthe BP model and multilayer BP model in terms of precision and generalization. Therefore,this method is ready to be applied for accurately evaluating the dairy cow feed intake, andit can provide theoretical guidance and technical support for the precise-feeding and canalso be of high significance in the improvement of dairy precise-breeding.
基金Supported by National Natural Science Foundation of China(Grant Nos.10871226,11001247 and 61179041)Natural Science Foundation of Zhejiang Province(Grant No.Y6100096)
文摘In the present paper, we give an investigation on the learning rate of l2-coefficient regularized classification with strong loss and the data dependent kernel functional spaces. The results show that the learning rate is influenced by the strong convexity.
基金This work was supported by the National Natural Science Foundation of China under Grant Nos. 61170092, 61133011 and 61103091.
文摘Stochastic variational inference （SVI） can learn topic models with very big corpora. It optimizes the variational objective by using the stochastic natural gradient algorithm with a decreasing learning rate. This rate is crucial for SVI; however, it is often tuned by hand in real applications. To address this, we develop a novel algorithm, which tunes the learning rate of each iteration adaptively. The proposed algorithm uses the Kullback-Leibler （KL） divergence to measure the similarity between the variational distribution with noisy update and that with batch update, and then optimizes the learning rates by minimizing the KL divergence. We apply our algorithm to two representative topic models： latent Dirichlet allocation and hierarchical Dirichlet process. Experimental results indicate that our algorithm performs better and converges faster than commonly used learning rates.
文摘Van der Pauw＇s function is often used in the measurement of a semiconductor＇s resistivity. However, it is difficult to obtain its value from voltage measurements because it has an implicit form. If it can be expressed as a polynomial, a semiconductor＇s resistivity can be obtained from such measurements. Normally, five orders of the abscissa can provide sufficient precision during the expression of any non-linear function. Therefore, the key is to determine the coefficients of the polynomial. By taking five coefficients as weights to construct a neuronetwork, neurocomputing has been used to solve this problem. Finally, the polynomial expression for van der Pauw＇s function is obtained.
文摘Nowadays,the amount of wed data is increasing at a rapid speed,which presents a serious challenge to the web monitoring.Text sentiment analysis,an important research topic in the area of natural language processing,is a crucial task in the web monitoring area.The accuracy of traditional text sentiment analysis methods might be degraded in dealing with mass data.Deep learning is a hot research topic of the artificial intelligence in the recent years.By now,several research groups have studied the sentiment analysis of English texts using deep learning methods.In contrary,relatively few works have so far considered the Chinese text sentiment analysis toward this direction.In this paper,a method for analyzing the Chinese text sentiment is proposed based on the convolutional neural network(CNN)in deep learning in order to improve the analysis accuracy.The feature values of the CNN after the training process are nonuniformly distributed.In order to overcome this problem,a method for normalizing the feature values is proposed.Moreover,the dimensions of the text features are optimized through simulations.Finally,a method for updating the learning rate in the training process of the CNN is presented in order to achieve better performances.Experiment results on the typical datasets indicate that the accuracy of the proposed method can be improved compared with that of the traditional supervised machine learning methods,e.g.,the support vector machine method.
基金supported by Research Fund from University of Johannes-burg,Johannesburg City,South Africa.
文摘Breast cancer(BC)is the most widespread tumor in females worldwide and is a severe public health issue.BC is the leading reason of death affecting females between the ages of 20 to 59 around the world.Early detection and therapy can help women receive effective treatment and,as a result,decrease the rate of breast cancer disease.The cancer tumor develops when cells grow improperly and attack the healthy tissue in the human body.Tumors are classified as benign or malignant,and the absence of cancer in the breast is considered normal.Deep learning,machine learning,and transfer learning models are applied to detect and identify cancerous tissue like BC.This research assists in the identification and classification of BC.We implemented the pre-trained model AlexNet and proposed model Breast cancer identification and classification(BCIC),which are machine learning-based models,by evaluating them in the form of comparative research.We used 3 datasets,A,B,and C.We fuzzed these datasets and got 2 datasets,A2C and B3C.Dataset A2C is the fusion of A,B,and C with 2 classes categorized as benign and malignant.Dataset B3C is the fusion of datasets A,B,and C with 3 classes classified as benign,malignant,and normal.We used customized AlexNet according to our datasets and BCIC in our proposed model.We achieved an accuracy of 86.5%on Dataset B3C and 76.8%on Dataset A2C by using AlexNet,and we achieved the optimum accuracy of 94.5%on Dataset B3C and 94.9%on Dataset A2C by using proposed model BCIC at 40 epochs with 0.00008 learning rate.We proposed fuzzed dataset model using transfer learning.We fuzzed three datasets to get more accurate results and the proposed model achieved the highest prediction accuracy using fuzzed dataset transfer learning technique.
基金Supported by the National Natural Science Foundation of China (10871226)the Natural Science Foundation of Zhejiang Province (Y6100096)
文摘Evaluation for the performance of learning algorithm has been the main thread of theoretical research of machine learning. The performance of the regularized regression algorithm based on independent and identically distributed（i.i.d.） samples has been researched by a large number of references. In the present paper we provide the convergence rates for the performance of regularized regression based on the inputs of p-order Markov chains.
基金Supported by the National Natural Science Foundation of China(61806162,12271431,12171388)Shaanxi Mathematical Basic Science Research Project 22JSQ023。
文摘To pursue the outperformance of deep nets in learning,we construct a deep net with three hidden layers and prove that,implementing the empirical risk minimization(ERM)on this deep net,the estimator can theoretically realize the optimal learning rates without the classical saturation problem.In other words,deepening the networks with only three hidden layers can overcome the saturation and not degrade the optimal learning rates.The obtained results underlie the success of deep nets and provide a theoretical guidance for deep learning.
基金Supported by the National Natural Science Foundation of China(61272023,91330118)
文摘This paper investigates some approximation properties and learning rates of Lipschitz kernel on the sphere. A perfect convergence rate on the shifts of Lipschitz kernel on the sphere, which is faster than O（n-1/2）, is obtained, where n is the number of parameters needed in the approximation. By means of the approximation, a learning rate of regularized least square algorithm with the Lipschitz kernel on the sphere is also deduced.
文摘Age has always been an important factor in studying second language acquisition.Marinova‐Todd points out three misconceptions about research in support of CPH and disputes the existence of a Critical Period in L2 learning.Hyltenstam&Abrahamsson refutes Marinova‐Todd’s“three misconceptions”.By contrasting and comparing the views and evidence from the two papers,the present author analyzes the differences among various claims on the rate of learning and different brain organization,as well as the possible causes for the disagreements.
文摘Children's English learning in China attracts more and more people's attention and is on the tendency of starting at an early age. Under the trend of "learning English from childhood", the author has explored the Critical Period Hypothesis and discussed the younger learners' disadvantages and older learners' advantages when learning English. and concludes that early-age English learning is not feasible.
文摘For training the present Neural Network(NN)models,the standard technique is to utilize decaying Learning Rates(LR).While the majority of these techniques commence with a large LR,they will decay multiple times over time.Decaying has been proved to enhance generalization as well as optimization.Other parameters,such as the network’s size,the number of hidden layers,drop-outs to avoid overfitting,batch size,and so on,are solely based on heuristics.This work has proposed Adaptive Teaching Learning Based(ATLB)Heuristic to identify the optimal hyperparameters for diverse networks.Here we consider three architec-tures Recurrent Neural Networks(RNN),Long Short Term Memory(LSTM),Bidirectional Long Short Term Memory(BiLSTM)of Deep Neural Networks for classification.The evaluation of the proposed ATLB is done through the various learning rate schedulers Cyclical Learning Rate(CLR),Hyperbolic Tangent Decay(HTD),and Toggle between Hyperbolic Tangent Decay and Triangular mode with Restarts(T-HTR)techniques.Experimental results have shown the performance improvement on the 20Newsgroup,Reuters Newswire and IMDB dataset.
