This paper considers a robust kernel regularized classification algorithm with a non-convex loss function which is proposed to alleviate the performance deterioration caused by the outliers.A comparison relationship b...This paper considers a robust kernel regularized classification algorithm with a non-convex loss function which is proposed to alleviate the performance deterioration caused by the outliers.A comparison relationship between the excess misclassification error and the excess generalization error is provided;from this,along with the convex analysis theory,a kind of learning rate is derived.The results show that the performance of the classifier is effected by the outliers,and the extent of impact can be controlled by choosing the homotopy parameters properly.展开更多
The rapid spread of the novel Coronavirus (COVID-19) has emphasized the necessity for advanced diagnostic tools to enhance the detection and management of the virus. This study investigates the effectiveness of Convol...The rapid spread of the novel Coronavirus (COVID-19) has emphasized the necessity for advanced diagnostic tools to enhance the detection and management of the virus. This study investigates the effectiveness of Convolutional Neural Networks (CNNs) in the diagnosis of COVID-19 from chest X-ray and CT images, focusing on the impact of varying learning rates and optimization strategies. Despite the abundance of chest X-ray datasets from various institutions, the lack of a dedicated COVID-19 dataset for computational analysis presents a significant challenge. Our work introduces an empirical analysis across four distinct learning rate policies—Cyclic, Step Based, Time-Based, and Epoch Based—each tested with four different optimizers: Adam, Adagrad, RMSprop, and Stochastic Gradient Descent (SGD). The performance of these configurations was evaluated in terms of training and validation accuracy over 100 epochs. Our results demonstrate significant differences in model performance, with the Cyclic learning rate policy combined with SGD optimizer achieving the highest validation accuracy of 83.33%. This study contributes to the existing body of knowledge by outlining effective CNN configurations for COVID-19 image dataset analysis, offering insights into the optimization of machine learning models for the diagnosis of infectious diseases. Our findings underscore the potential of CNNs in supplementing traditional PCR tests, providing a computational approach to identify patterns in chest X-rays and CT scans indicative of COVID-19, thereby aiding in the swift and accurate diagnosis of the virus.展开更多
Complex road conditions without signalized intersections when the traffic flow is nearly saturated result in high traffic congestion and accidents,reducing the traffic efficiency of intelligent vehicles.The complex ro...Complex road conditions without signalized intersections when the traffic flow is nearly saturated result in high traffic congestion and accidents,reducing the traffic efficiency of intelligent vehicles.The complex road traffic environment of smart vehicles and other vehicles frequently experiences conflicting start and stop motion.The fine-grained scheduling of autonomous vehicles(AVs)at non-signalized intersections,which is a promising technique for exploring optimal driving paths for both assisted driving nowadays and driverless cars in the near future,has attracted significant attention owing to its high potential for improving road safety and traffic efficiency.Fine-grained scheduling primarily focuses on signalized intersection scenarios,as applying it directly to non-signalized intersections is challenging because each AV can move freely without traffic signal control.This may cause frequent driving collisions and low road traffic efficiency.Therefore,this study proposes a novel algorithm to address this issue.Our work focuses on the fine-grained scheduling of automated vehicles at non-signal intersections via dual reinforced training(FS-DRL).For FS-DRL,we first use a grid to describe the non-signalized intersection and propose a convolutional neural network(CNN)-based fast decision model that can rapidly yield a coarse-grained scheduling decision for each AV in a distributed manner.We then load these coarse-grained scheduling decisions onto a deep Q-learning network(DQN)for further evaluation.We use an adaptive learning rate to maximize the reward function and employ parameterεto tradeoff the fast speed of coarse-grained scheduling in the CNN and optimal fine-grained scheduling in the DQN.In addition,we prove that using this adaptive learning rate leads to a converged loss rate with an extremely small number of training loops.The simulation results show that compared with Dijkstra,RNN,and ant colony-based scheduling,FS-DRL yields a high accuracy of 96.5%on the sample,with improved performance of approximately 61.54%-85.37%in terms of the average conflict and traffic efficiency.展开更多
Skin cancer is the abnormal development of cells on the surface of the skin and is one of the most fatal diseases in humans.It usually appears in locations that are exposed to the sun,but can also appear in areas that...Skin cancer is the abnormal development of cells on the surface of the skin and is one of the most fatal diseases in humans.It usually appears in locations that are exposed to the sun,but can also appear in areas that are not regularly exposed to the sun.Due to the striking similarities between benign and malignant lesions,skin cancer detection remains a problem,even for expert dermatologists.Considering the inability of dermatologists to di-agnose skin cancer accurately,a convolutional neural network(CNN)approach was used for skin cancer diag-nosis.However,the CNN model requires a significant number of image datasets for better performance;thus,image augmentation and transfer learning techniques have been used in this study to boost the number of images and the performance of the model,because there are a limited number of medical images.This study proposes an ensemble transfer-learning-based model that can efficiently classify skin lesions into one of seven categories to aid dermatologists in skin cancer detection:(i)actinic keratoses,(ii)basal cell carcinoma,(iii)benign keratosis,(iv)dermatofibroma,(v)melanocytic nevi,(vi)melanoma,and(vii)vascular skin lesions.Five transfer learning models were used as the basis of the ensemble:MobileNet,EfficientNetV2B2,Xception,ResNeXt101,and Den-seNet201.In addition to the stratified 10-fold cross-validation,the results of each individual model were fused to achieve greater classification accuracy.An annealing learning rate scheduler and test time augmentation(TTA)were also used to increase the performance of the model during the training and testing stages.A total of 10,015 publicly available dermoscopy images from the HAM10000(Human Against Machine)dataset,which contained samples from the seven common skin lesion categories,were used to train and evaluate the models.The proposed technique attained 94.49%accuracy on the dataset.These results suggest that this strategy can be useful for improving the accuracy of skin cancer classification.However,the weighted average of F1-score,recall,and precision were obtained to be 94.68%,94.49%,and 95.07%,respectively.展开更多
Purpose: This study aimed to enhance the prediction of container dwell time, a crucial factor for optimizing port operations, resource allocation, and supply chain efficiency. Determining an optimal learning rate for ...Purpose: This study aimed to enhance the prediction of container dwell time, a crucial factor for optimizing port operations, resource allocation, and supply chain efficiency. Determining an optimal learning rate for training Artificial Neural Networks (ANNs) has remained a challenging task due to the diverse sizes, complexity, and types of data involved. Design/Method/Approach: This research used a RandomizedSearchCV algorithm, a random search approach, to bridge this knowledge gap. The algorithm was applied to container dwell time data from the TOS system of the Port of Tema, which included 307,594 container records from 2014 to 2022. Findings: The RandomizedSearchCV method outperformed standard training methods both in terms of reducing training time and improving prediction accuracy, highlighting the significant role of the constant learning rate as a hyperparameter. Research Limitations and Implications: Although the study provides promising outcomes, the results are limited to the data extracted from the Port of Tema and may differ in other contexts. Further research is needed to generalize these findings across various port systems. Originality/Value: This research underscores the potential of RandomizedSearchCV as a valuable tool for optimizing ANN training in container dwell time prediction. It also accentuates the significance of automated learning rate selection, offering novel insights into the optimization of container dwell time prediction, with implications for improving port efficiency and supply chain operations.展开更多
Age has always been an important factor in studying second language acquisition.Marinova‐Todd points out three misconceptions about research in support of CPH and disputes the existence of a Critical Period in L2 lea...Age has always been an important factor in studying second language acquisition.Marinova‐Todd points out three misconceptions about research in support of CPH and disputes the existence of a Critical Period in L2 learning.Hyltenstam&Abrahamsson refutes Marinova‐Todd’s“three misconceptions”.By contrasting and comparing the views and evidence from the two papers,the present author analyzes the differences among various claims on the rate of learning and different brain organization,as well as the possible causes for the disagreements.展开更多
Children's English learning in China attracts more and more people's attention and is on the tendency of starting at an early age. Under the trend of "learning English from childhood", the author has...Children's English learning in China attracts more and more people's attention and is on the tendency of starting at an early age. Under the trend of "learning English from childhood", the author has explored the Critical Period Hypothesis and discussed the younger learners' disadvantages and older learners' advantages when learning English. and concludes that early-age English learning is not feasible.展开更多
Nowadays,the amount of wed data is increasing at a rapid speed,which presents a serious challenge to the web monitoring.Text sentiment analysis,an important research topic in the area of natural language processing,is...Nowadays,the amount of wed data is increasing at a rapid speed,which presents a serious challenge to the web monitoring.Text sentiment analysis,an important research topic in the area of natural language processing,is a crucial task in the web monitoring area.The accuracy of traditional text sentiment analysis methods might be degraded in dealing with mass data.Deep learning is a hot research topic of the artificial intelligence in the recent years.By now,several research groups have studied the sentiment analysis of English texts using deep learning methods.In contrary,relatively few works have so far considered the Chinese text sentiment analysis toward this direction.In this paper,a method for analyzing the Chinese text sentiment is proposed based on the convolutional neural network(CNN)in deep learning in order to improve the analysis accuracy.The feature values of the CNN after the training process are nonuniformly distributed.In order to overcome this problem,a method for normalizing the feature values is proposed.Moreover,the dimensions of the text features are optimized through simulations.Finally,a method for updating the learning rate in the training process of the CNN is presented in order to achieve better performances.Experiment results on the typical datasets indicate that the accuracy of the proposed method can be improved compared with that of the traditional supervised machine learning methods,e.g.,the support vector machine method.展开更多
For accelerating the supervised learning by the SpikeProp algorithm with the temporal coding paradigm in spiking neural networks (SNNs), three learning rate adaptation methods (heuristic rule, delta-delta rule, and de...For accelerating the supervised learning by the SpikeProp algorithm with the temporal coding paradigm in spiking neural networks (SNNs), three learning rate adaptation methods (heuristic rule, delta-delta rule, and delta-bar-delta rule), which are used to speed up training in artificial neural networks, are used to develop the training algorithms for feedforward SNN. The performance of these algorithms is investigated by four experiments: classical XOR (exclusive or) problem, Iris dataset, fault diagnosis in the Tennessee Eastman process, and Poisson trains of discrete spikes. The results demonstrate that all the three learning rate adaptation methods are able to speed up convergence of SNN compared with the original SpikeProp algorithm. Furthermore, if the adaptive learning rate is used in combination with the momentum term, the two modifications will balance each other in a beneficial way to accomplish rapid and steady convergence. In the three learning rate adaptation methods, delta-bar-delta rule performs the best. The delta-bar-delta method with momentum has the fastest convergence rate, the greatest stability of training process, and the maximum accuracy of network learning. The proposed algorithms in this paper are simple and efficient, and consequently valuable for practical applications of SNN.展开更多
Breast cancer(BC)is the most widespread tumor in females worldwide and is a severe public health issue.BC is the leading reason of death affecting females between the ages of 20 to 59 around the world.Early detection ...Breast cancer(BC)is the most widespread tumor in females worldwide and is a severe public health issue.BC is the leading reason of death affecting females between the ages of 20 to 59 around the world.Early detection and therapy can help women receive effective treatment and,as a result,decrease the rate of breast cancer disease.The cancer tumor develops when cells grow improperly and attack the healthy tissue in the human body.Tumors are classified as benign or malignant,and the absence of cancer in the breast is considered normal.Deep learning,machine learning,and transfer learning models are applied to detect and identify cancerous tissue like BC.This research assists in the identification and classification of BC.We implemented the pre-trained model AlexNet and proposed model Breast cancer identification and classification(BCIC),which are machine learning-based models,by evaluating them in the form of comparative research.We used 3 datasets,A,B,and C.We fuzzed these datasets and got 2 datasets,A2C and B3C.Dataset A2C is the fusion of A,B,and C with 2 classes categorized as benign and malignant.Dataset B3C is the fusion of datasets A,B,and C with 3 classes classified as benign,malignant,and normal.We used customized AlexNet according to our datasets and BCIC in our proposed model.We achieved an accuracy of 86.5%on Dataset B3C and 76.8%on Dataset A2C by using AlexNet,and we achieved the optimum accuracy of 94.5%on Dataset B3C and 94.9%on Dataset A2C by using proposed model BCIC at 40 epochs with 0.00008 learning rate.We proposed fuzzed dataset model using transfer learning.We fuzzed three datasets to get more accurate results and the proposed model achieved the highest prediction accuracy using fuzzed dataset transfer learning technique.展开更多
Evaluation for the performance of learning algorithm has been the main thread of theoretical research of machine learning. The performance of the regularized regression algorithm based on independent and identically d...Evaluation for the performance of learning algorithm has been the main thread of theoretical research of machine learning. The performance of the regularized regression algorithm based on independent and identically distributed（i.i.d.） samples has been researched by a large number of references. In the present paper we provide the convergence rates for the performance of regularized regression based on the inputs of p-order Markov chains.展开更多
A dynamic learning rate Gaussian mixture model（GMM）algorithm is proposed to deal with the problem of slow adaption of GMM in the case of moving object detection in the outdoor surveillance,especially in the presence...A dynamic learning rate Gaussian mixture model（GMM）algorithm is proposed to deal with the problem of slow adaption of GMM in the case of moving object detection in the outdoor surveillance,especially in the presence of sudden illumination changes.The GMM is mostly used for detecting objects in complex scenes for intelligent monitoring systems.To solve this problem,a mixture Gaussian model has been built for each pixel in the video frame,and according to the scene change from the frame difference,the learning rate of GMM can be dynamically adjusted.The experiments show that the proposed method gives good results with an adaptive GMM learning rate when we compare it with GMM method with a fixed learning rate.The method was tested on a certain dataset,and tests in the case of sudden natural light changes show that our method has a better accuracy and lower false alarm rate.展开更多
To pursue the outperformance of deep nets in learning,we construct a deep net with three hidden layers and prove that,implementing the empirical risk minimization(ERM)on this deep net,the estimator can theoretically r...To pursue the outperformance of deep nets in learning,we construct a deep net with three hidden layers and prove that,implementing the empirical risk minimization(ERM)on this deep net,the estimator can theoretically realize the optimal learning rates without the classical saturation problem.In other words,deepening the networks with only three hidden layers can overcome the saturation and not degrade the optimal learning rates.The obtained results underlie the success of deep nets and provide a theoretical guidance for deep learning.展开更多
This paper investigates some approximation properties and learning rates of Lipschitz kernel on the sphere. A perfect convergence rate on the shifts of Lipschitz kernel on the sphere, which is faster than O（n-1/2）, ...This paper investigates some approximation properties and learning rates of Lipschitz kernel on the sphere. A perfect convergence rate on the shifts of Lipschitz kernel on the sphere, which is faster than O（n-1/2）, is obtained, where n is the number of parameters needed in the approximation. By means of the approximation, a learning rate of regularized least square algorithm with the Lipschitz kernel on the sphere is also deduced.展开更多
A new algorithm to exploit the learning rates of gradient descent method is presented, based on the second-order Taylor expansion of the error energy function with respect to learning rate, at some values decided by &...A new algorithm to exploit the learning rates of gradient descent method is presented, based on the second-order Taylor expansion of the error energy function with respect to learning rate, at some values decided by "award-punish" strategy. Detailed deduction of the algorithm applied to RBF networks is given. Simulation studies show that this algorithm can increase the rate of convergence and improve the performance of the gradient descent method.展开更多
For training the present Neural Network(NN)models,the standard technique is to utilize decaying Learning Rates(LR).While the majority of these techniques commence with a large LR,they will decay multiple times over ti...For training the present Neural Network(NN)models,the standard technique is to utilize decaying Learning Rates(LR).While the majority of these techniques commence with a large LR,they will decay multiple times over time.Decaying has been proved to enhance generalization as well as optimization.Other parameters,such as the network’s size,the number of hidden layers,drop-outs to avoid overfitting,batch size,and so on,are solely based on heuristics.This work has proposed Adaptive Teaching Learning Based(ATLB)Heuristic to identify the optimal hyperparameters for diverse networks.Here we consider three architec-tures Recurrent Neural Networks(RNN),Long Short Term Memory(LSTM),Bidirectional Long Short Term Memory(BiLSTM)of Deep Neural Networks for classification.The evaluation of the proposed ATLB is done through the various learning rate schedulers Cyclical Learning Rate(CLR),Hyperbolic Tangent Decay(HTD),and Toggle between Hyperbolic Tangent Decay and Triangular mode with Restarts(T-HTR)techniques.Experimental results have shown the performance improvement on the 20Newsgroup,Reuters Newswire and IMDB dataset.展开更多
Aim To build an adaptive fuzzy neural controller and simulate it. Methods\ Fuzzy logic and back propagation(BP) algorithm are combined to utilize their advantages while avoiding the disadvantages. Results and Conclus...Aim To build an adaptive fuzzy neural controller and simulate it. Methods\ Fuzzy logic and back propagation(BP) algorithm are combined to utilize their advantages while avoiding the disadvantages. Results and Conclusion\ Simulation results of the third order plant with disturbances and dead times show the validity of the presented controller. The presented controller can control cases that preceding controllers were unable to control.展开更多
Background:Health-related fitness knowledge(HRFK)has been an essential concept for many health and physical education programs.There has been limited understanding and longitudinal investigation on HRFK growth.This lo...Background:Health-related fitness knowledge(HRFK)has been an essential concept for many health and physical education programs.There has been limited understanding and longitudinal investigation on HRFK growth.This longitudinal study examined HRFK growth and its individual-and school-level correlates in middle school years under 1 curriculum condition:Five for Life.Methods:Participants were 12,044 students from 47 middle schools.Data were collected at both individual/participant and school/institution levels.Individual-level variables included gender,grade,and HRFK test scores.School-level variables included percentage of students receiving free and reduced meals(FARM),student-to-faculty ratio for physical education,and school academic performance(SAP).We used hierarchical linear modeling to examine HRFK 3-year growth in relation to individual-and school-level correlates.Results:The average HRFK score at 6th grade for females was 42.81%±1.32%.The predicted HRFK growth was 17.06%±1.02%per year,holding other factors constant.A 1-standard deviation increase in FARM correlated with a 14.68%-point decrease in predicted test score(p=0.02).A 1-standard deviation increase in SAP was associated with an 11.90%-point increase in HRFK score.Males had a significantly lower growth rate than females during the middle school years(0.78%/year,p=0.02).Conclusion:The result showed that both individual-and school-level variables such as gender,FARM,and SAP influenced HRFK growth.Educators should heed gender differences in growth curves and recognize the correlates of school-level variables.展开更多
Even though much advancements have been achieved with regards to the recognition of handwritten characters,researchers still face difficulties with the handwritten character recognition problem,especially with the adv...Even though much advancements have been achieved with regards to the recognition of handwritten characters,researchers still face difficulties with the handwritten character recognition problem,especially with the advent of new datasets like the Extended Modified National Institute of Standards and Technology dataset(EMNIST).The EMNIST dataset represents a challenge for both machine-learning and deep-learning techniques due to inter-class similarity and intra-class variability.Inter-class similarity exists because of the similarity between the shapes of certain characters in the dataset.The presence of intra-class variability is mainly due to different shapes written by different writers for the same character.In this research,we have optimized a deep residual network to achieve higher accuracy vs.the published state-of-the-art results.This approach is mainly based on the prebuilt deep residual network model ResNet18,whose architecture has been enhanced by using the optimal number of residual blocks and the optimal size of the receptive field of the first convolutional filter,the replacement of the first max-pooling filter by an average pooling filter,and the addition of a drop-out layer before the fully connected layer.A distinctive modification has been introduced by replacing the final addition layer with a depth concatenation layer,which resulted in a novel deep architecture having higher accuracy vs.the pure residual architecture.Moreover,the dataset images’sizes have been adjusted to optimize their visibility in the network.Finally,by tuning the training hyperparameters and using rotation and shear augmentations,the proposed model outperformed the state-of-the-art models by achieving average accuracies of 95.91%and 90.90%for the Letters and Balanced dataset sections,respectively.Furthermore,the average accuracies were improved to 95.9%and 91.06%for the Letters and Balanced sections,respectively,by using a group of 5 instances of the trained models and averaging the output class probabilities.展开更多
The servo-motor possesses a strongly nonlinear property due to the effect of the stimulating input voltage, load-torque and environmental operating conditions. So it is rather difficult to derive a traditional mathema...The servo-motor possesses a strongly nonlinear property due to the effect of the stimulating input voltage, load-torque and environmental operating conditions. So it is rather difficult to derive a traditional mathematical model which is capable of expressing both its dynamics and steady-state characteristics. A neural network-based adaptive control strategy is proposed in this paper. In this method, two neural networks have been adopted for system identification （NNI） and control （NNC）, respectively. Then, the commonly-used specialized learning has been modified, by taking the NNI output as the approximation output of the servo-motor during the weights training to get sensitivity information. Moreover, the rule for choosing the learning rate is given on the basis of the analysis of Lyapunov stability. Finally, an example of applying the proposed control strategy on a servo-motor is presented to show its effectiveness.展开更多
基金supported by the NSF(61877039)the NSFC/RGC Joint Research Scheme(12061160462 and N City U 102/20)of China+2 种基金the NSF(LY19F020013)of Zhejiang Provincethe Special Project for Scientific and Technological Cooperation(20212BDH80021)of Jiangxi Provincethe Science and Technology Project in Jiangxi Province Department of Education(GJJ211334)。
文摘This paper considers a robust kernel regularized classification algorithm with a non-convex loss function which is proposed to alleviate the performance deterioration caused by the outliers.A comparison relationship between the excess misclassification error and the excess generalization error is provided;from this,along with the convex analysis theory,a kind of learning rate is derived.The results show that the performance of the classifier is effected by the outliers,and the extent of impact can be controlled by choosing the homotopy parameters properly.
文摘The rapid spread of the novel Coronavirus (COVID-19) has emphasized the necessity for advanced diagnostic tools to enhance the detection and management of the virus. This study investigates the effectiveness of Convolutional Neural Networks (CNNs) in the diagnosis of COVID-19 from chest X-ray and CT images, focusing on the impact of varying learning rates and optimization strategies. Despite the abundance of chest X-ray datasets from various institutions, the lack of a dedicated COVID-19 dataset for computational analysis presents a significant challenge. Our work introduces an empirical analysis across four distinct learning rate policies—Cyclic, Step Based, Time-Based, and Epoch Based—each tested with four different optimizers: Adam, Adagrad, RMSprop, and Stochastic Gradient Descent (SGD). The performance of these configurations was evaluated in terms of training and validation accuracy over 100 epochs. Our results demonstrate significant differences in model performance, with the Cyclic learning rate policy combined with SGD optimizer achieving the highest validation accuracy of 83.33%. This study contributes to the existing body of knowledge by outlining effective CNN configurations for COVID-19 image dataset analysis, offering insights into the optimization of machine learning models for the diagnosis of infectious diseases. Our findings underscore the potential of CNNs in supplementing traditional PCR tests, providing a computational approach to identify patterns in chest X-rays and CT scans indicative of COVID-19, thereby aiding in the swift and accurate diagnosis of the virus.
基金Supported by National Natural Science Foundation of China(Grant No.61803206)Jiangsu Provincial Natural Science Foundation(Grant No.222300420468)Jiangsu Provincial key R&D Program(Grant No.BE2017008-2).
文摘Complex road conditions without signalized intersections when the traffic flow is nearly saturated result in high traffic congestion and accidents,reducing the traffic efficiency of intelligent vehicles.The complex road traffic environment of smart vehicles and other vehicles frequently experiences conflicting start and stop motion.The fine-grained scheduling of autonomous vehicles(AVs)at non-signalized intersections,which is a promising technique for exploring optimal driving paths for both assisted driving nowadays and driverless cars in the near future,has attracted significant attention owing to its high potential for improving road safety and traffic efficiency.Fine-grained scheduling primarily focuses on signalized intersection scenarios,as applying it directly to non-signalized intersections is challenging because each AV can move freely without traffic signal control.This may cause frequent driving collisions and low road traffic efficiency.Therefore,this study proposes a novel algorithm to address this issue.Our work focuses on the fine-grained scheduling of automated vehicles at non-signal intersections via dual reinforced training(FS-DRL).For FS-DRL,we first use a grid to describe the non-signalized intersection and propose a convolutional neural network(CNN)-based fast decision model that can rapidly yield a coarse-grained scheduling decision for each AV in a distributed manner.We then load these coarse-grained scheduling decisions onto a deep Q-learning network(DQN)for further evaluation.We use an adaptive learning rate to maximize the reward function and employ parameterεto tradeoff the fast speed of coarse-grained scheduling in the CNN and optimal fine-grained scheduling in the DQN.In addition,we prove that using this adaptive learning rate leads to a converged loss rate with an extremely small number of training loops.The simulation results show that compared with Dijkstra,RNN,and ant colony-based scheduling,FS-DRL yields a high accuracy of 96.5%on the sample,with improved performance of approximately 61.54%-85.37%in terms of the average conflict and traffic efficiency.
文摘Skin cancer is the abnormal development of cells on the surface of the skin and is one of the most fatal diseases in humans.It usually appears in locations that are exposed to the sun,but can also appear in areas that are not regularly exposed to the sun.Due to the striking similarities between benign and malignant lesions,skin cancer detection remains a problem,even for expert dermatologists.Considering the inability of dermatologists to di-agnose skin cancer accurately,a convolutional neural network(CNN)approach was used for skin cancer diag-nosis.However,the CNN model requires a significant number of image datasets for better performance;thus,image augmentation and transfer learning techniques have been used in this study to boost the number of images and the performance of the model,because there are a limited number of medical images.This study proposes an ensemble transfer-learning-based model that can efficiently classify skin lesions into one of seven categories to aid dermatologists in skin cancer detection:(i)actinic keratoses,(ii)basal cell carcinoma,(iii)benign keratosis,(iv)dermatofibroma,(v)melanocytic nevi,(vi)melanoma,and(vii)vascular skin lesions.Five transfer learning models were used as the basis of the ensemble:MobileNet,EfficientNetV2B2,Xception,ResNeXt101,and Den-seNet201.In addition to the stratified 10-fold cross-validation,the results of each individual model were fused to achieve greater classification accuracy.An annealing learning rate scheduler and test time augmentation(TTA)were also used to increase the performance of the model during the training and testing stages.A total of 10,015 publicly available dermoscopy images from the HAM10000(Human Against Machine)dataset,which contained samples from the seven common skin lesion categories,were used to train and evaluate the models.The proposed technique attained 94.49%accuracy on the dataset.These results suggest that this strategy can be useful for improving the accuracy of skin cancer classification.However,the weighted average of F1-score,recall,and precision were obtained to be 94.68%,94.49%,and 95.07%,respectively.
文摘Purpose: This study aimed to enhance the prediction of container dwell time, a crucial factor for optimizing port operations, resource allocation, and supply chain efficiency. Determining an optimal learning rate for training Artificial Neural Networks (ANNs) has remained a challenging task due to the diverse sizes, complexity, and types of data involved. Design/Method/Approach: This research used a RandomizedSearchCV algorithm, a random search approach, to bridge this knowledge gap. The algorithm was applied to container dwell time data from the TOS system of the Port of Tema, which included 307,594 container records from 2014 to 2022. Findings: The RandomizedSearchCV method outperformed standard training methods both in terms of reducing training time and improving prediction accuracy, highlighting the significant role of the constant learning rate as a hyperparameter. Research Limitations and Implications: Although the study provides promising outcomes, the results are limited to the data extracted from the Port of Tema and may differ in other contexts. Further research is needed to generalize these findings across various port systems. Originality/Value: This research underscores the potential of RandomizedSearchCV as a valuable tool for optimizing ANN training in container dwell time prediction. It also accentuates the significance of automated learning rate selection, offering novel insights into the optimization of container dwell time prediction, with implications for improving port efficiency and supply chain operations.
文摘Age has always been an important factor in studying second language acquisition.Marinova‐Todd points out three misconceptions about research in support of CPH and disputes the existence of a Critical Period in L2 learning.Hyltenstam&Abrahamsson refutes Marinova‐Todd’s“three misconceptions”.By contrasting and comparing the views and evidence from the two papers,the present author analyzes the differences among various claims on the rate of learning and different brain organization,as well as the possible causes for the disagreements.
文摘Children's English learning in China attracts more and more people's attention and is on the tendency of starting at an early age. Under the trend of "learning English from childhood", the author has explored the Critical Period Hypothesis and discussed the younger learners' disadvantages and older learners' advantages when learning English. and concludes that early-age English learning is not feasible.
文摘Nowadays,the amount of wed data is increasing at a rapid speed,which presents a serious challenge to the web monitoring.Text sentiment analysis,an important research topic in the area of natural language processing,is a crucial task in the web monitoring area.The accuracy of traditional text sentiment analysis methods might be degraded in dealing with mass data.Deep learning is a hot research topic of the artificial intelligence in the recent years.By now,several research groups have studied the sentiment analysis of English texts using deep learning methods.In contrary,relatively few works have so far considered the Chinese text sentiment analysis toward this direction.In this paper,a method for analyzing the Chinese text sentiment is proposed based on the convolutional neural network(CNN)in deep learning in order to improve the analysis accuracy.The feature values of the CNN after the training process are nonuniformly distributed.In order to overcome this problem,a method for normalizing the feature values is proposed.Moreover,the dimensions of the text features are optimized through simulations.Finally,a method for updating the learning rate in the training process of the CNN is presented in order to achieve better performances.Experiment results on the typical datasets indicate that the accuracy of the proposed method can be improved compared with that of the traditional supervised machine learning methods,e.g.,the support vector machine method.
基金Supported by the National Natural Science Foundation of China (60904018, 61203040)the Natural Science Foundation of Fujian Province of China (2009J05147, 2011J01352)+1 种基金the Foundation for Distinguished Young Scholars of Higher Education of Fujian Province of China (JA10004)the Science Research Foundation of Huaqiao University (09BS617)
文摘For accelerating the supervised learning by the SpikeProp algorithm with the temporal coding paradigm in spiking neural networks (SNNs), three learning rate adaptation methods (heuristic rule, delta-delta rule, and delta-bar-delta rule), which are used to speed up training in artificial neural networks, are used to develop the training algorithms for feedforward SNN. The performance of these algorithms is investigated by four experiments: classical XOR (exclusive or) problem, Iris dataset, fault diagnosis in the Tennessee Eastman process, and Poisson trains of discrete spikes. The results demonstrate that all the three learning rate adaptation methods are able to speed up convergence of SNN compared with the original SpikeProp algorithm. Furthermore, if the adaptive learning rate is used in combination with the momentum term, the two modifications will balance each other in a beneficial way to accomplish rapid and steady convergence. In the three learning rate adaptation methods, delta-bar-delta rule performs the best. The delta-bar-delta method with momentum has the fastest convergence rate, the greatest stability of training process, and the maximum accuracy of network learning. The proposed algorithms in this paper are simple and efficient, and consequently valuable for practical applications of SNN.
基金supported by Research Fund from University of Johannes-burg,Johannesburg City,South Africa.
文摘Breast cancer(BC)is the most widespread tumor in females worldwide and is a severe public health issue.BC is the leading reason of death affecting females between the ages of 20 to 59 around the world.Early detection and therapy can help women receive effective treatment and,as a result,decrease the rate of breast cancer disease.The cancer tumor develops when cells grow improperly and attack the healthy tissue in the human body.Tumors are classified as benign or malignant,and the absence of cancer in the breast is considered normal.Deep learning,machine learning,and transfer learning models are applied to detect and identify cancerous tissue like BC.This research assists in the identification and classification of BC.We implemented the pre-trained model AlexNet and proposed model Breast cancer identification and classification(BCIC),which are machine learning-based models,by evaluating them in the form of comparative research.We used 3 datasets,A,B,and C.We fuzzed these datasets and got 2 datasets,A2C and B3C.Dataset A2C is the fusion of A,B,and C with 2 classes categorized as benign and malignant.Dataset B3C is the fusion of datasets A,B,and C with 3 classes classified as benign,malignant,and normal.We used customized AlexNet according to our datasets and BCIC in our proposed model.We achieved an accuracy of 86.5%on Dataset B3C and 76.8%on Dataset A2C by using AlexNet,and we achieved the optimum accuracy of 94.5%on Dataset B3C and 94.9%on Dataset A2C by using proposed model BCIC at 40 epochs with 0.00008 learning rate.We proposed fuzzed dataset model using transfer learning.We fuzzed three datasets to get more accurate results and the proposed model achieved the highest prediction accuracy using fuzzed dataset transfer learning technique.
基金Supported by the National Natural Science Foundation of China (10871226)the Natural Science Foundation of Zhejiang Province (Y6100096)
文摘Evaluation for the performance of learning algorithm has been the main thread of theoretical research of machine learning. The performance of the regularized regression algorithm based on independent and identically distributed（i.i.d.） samples has been researched by a large number of references. In the present paper we provide the convergence rates for the performance of regularized regression based on the inputs of p-order Markov chains.
文摘A dynamic learning rate Gaussian mixture model（GMM）algorithm is proposed to deal with the problem of slow adaption of GMM in the case of moving object detection in the outdoor surveillance,especially in the presence of sudden illumination changes.The GMM is mostly used for detecting objects in complex scenes for intelligent monitoring systems.To solve this problem,a mixture Gaussian model has been built for each pixel in the video frame,and according to the scene change from the frame difference,the learning rate of GMM can be dynamically adjusted.The experiments show that the proposed method gives good results with an adaptive GMM learning rate when we compare it with GMM method with a fixed learning rate.The method was tested on a certain dataset,and tests in the case of sudden natural light changes show that our method has a better accuracy and lower false alarm rate.
基金Supported by the National Natural Science Foundation of China(61806162,12271431,12171388)Shaanxi Mathematical Basic Science Research Project 22JSQ023。
文摘To pursue the outperformance of deep nets in learning,we construct a deep net with three hidden layers and prove that,implementing the empirical risk minimization(ERM)on this deep net,the estimator can theoretically realize the optimal learning rates without the classical saturation problem.In other words,deepening the networks with only three hidden layers can overcome the saturation and not degrade the optimal learning rates.The obtained results underlie the success of deep nets and provide a theoretical guidance for deep learning.
基金Supported by the National Natural Science Foundation of China(61272023,91330118)
文摘This paper investigates some approximation properties and learning rates of Lipschitz kernel on the sphere. A perfect convergence rate on the shifts of Lipschitz kernel on the sphere, which is faster than O（n-1/2）, is obtained, where n is the number of parameters needed in the approximation. By means of the approximation, a learning rate of regularized least square algorithm with the Lipschitz kernel on the sphere is also deduced.
基金Open Foundation of State Key Lab of Transmission of Wide-Band FiberTechnologies of Communication Systems
文摘A new algorithm to exploit the learning rates of gradient descent method is presented, based on the second-order Taylor expansion of the error energy function with respect to learning rate, at some values decided by "award-punish" strategy. Detailed deduction of the algorithm applied to RBF networks is given. Simulation studies show that this algorithm can increase the rate of convergence and improve the performance of the gradient descent method.
文摘For training the present Neural Network(NN)models,the standard technique is to utilize decaying Learning Rates(LR).While the majority of these techniques commence with a large LR,they will decay multiple times over time.Decaying has been proved to enhance generalization as well as optimization.Other parameters,such as the network’s size,the number of hidden layers,drop-outs to avoid overfitting,batch size,and so on,are solely based on heuristics.This work has proposed Adaptive Teaching Learning Based(ATLB)Heuristic to identify the optimal hyperparameters for diverse networks.Here we consider three architec-tures Recurrent Neural Networks(RNN),Long Short Term Memory(LSTM),Bidirectional Long Short Term Memory(BiLSTM)of Deep Neural Networks for classification.The evaluation of the proposed ATLB is done through the various learning rate schedulers Cyclical Learning Rate(CLR),Hyperbolic Tangent Decay(HTD),and Toggle between Hyperbolic Tangent Decay and Triangular mode with Restarts(T-HTR)techniques.Experimental results have shown the performance improvement on the 20Newsgroup,Reuters Newswire and IMDB dataset.
文摘Aim To build an adaptive fuzzy neural controller and simulate it. Methods\ Fuzzy logic and back propagation(BP) algorithm are combined to utilize their advantages while avoiding the disadvantages. Results and Conclusion\ Simulation results of the third order plant with disturbances and dead times show the validity of the presented controller. The presented controller can control cases that preceding controllers were unable to control.
基金supported by the Focused on Community Health Foundation(#16-353)。
文摘Background:Health-related fitness knowledge(HRFK)has been an essential concept for many health and physical education programs.There has been limited understanding and longitudinal investigation on HRFK growth.This longitudinal study examined HRFK growth and its individual-and school-level correlates in middle school years under 1 curriculum condition:Five for Life.Methods:Participants were 12,044 students from 47 middle schools.Data were collected at both individual/participant and school/institution levels.Individual-level variables included gender,grade,and HRFK test scores.School-level variables included percentage of students receiving free and reduced meals(FARM),student-to-faculty ratio for physical education,and school academic performance(SAP).We used hierarchical linear modeling to examine HRFK 3-year growth in relation to individual-and school-level correlates.Results:The average HRFK score at 6th grade for females was 42.81%±1.32%.The predicted HRFK growth was 17.06%±1.02%per year,holding other factors constant.A 1-standard deviation increase in FARM correlated with a 14.68%-point decrease in predicted test score(p=0.02).A 1-standard deviation increase in SAP was associated with an 11.90%-point increase in HRFK score.Males had a significantly lower growth rate than females during the middle school years(0.78%/year,p=0.02).Conclusion:The result showed that both individual-and school-level variables such as gender,FARM,and SAP influenced HRFK growth.Educators should heed gender differences in growth curves and recognize the correlates of school-level variables.
文摘Even though much advancements have been achieved with regards to the recognition of handwritten characters,researchers still face difficulties with the handwritten character recognition problem,especially with the advent of new datasets like the Extended Modified National Institute of Standards and Technology dataset(EMNIST).The EMNIST dataset represents a challenge for both machine-learning and deep-learning techniques due to inter-class similarity and intra-class variability.Inter-class similarity exists because of the similarity between the shapes of certain characters in the dataset.The presence of intra-class variability is mainly due to different shapes written by different writers for the same character.In this research,we have optimized a deep residual network to achieve higher accuracy vs.the published state-of-the-art results.This approach is mainly based on the prebuilt deep residual network model ResNet18,whose architecture has been enhanced by using the optimal number of residual blocks and the optimal size of the receptive field of the first convolutional filter,the replacement of the first max-pooling filter by an average pooling filter,and the addition of a drop-out layer before the fully connected layer.A distinctive modification has been introduced by replacing the final addition layer with a depth concatenation layer,which resulted in a novel deep architecture having higher accuracy vs.the pure residual architecture.Moreover,the dataset images’sizes have been adjusted to optimize their visibility in the network.Finally,by tuning the training hyperparameters and using rotation and shear augmentations,the proposed model outperformed the state-of-the-art models by achieving average accuracies of 95.91%and 90.90%for the Letters and Balanced dataset sections,respectively.Furthermore,the average accuracies were improved to 95.9%and 91.06%for the Letters and Balanced sections,respectively,by using a group of 5 instances of the trained models and averaging the output class probabilities.
基金National Science Foundation of China (No.60572055)Advanced Research Grant of Shanghai Normal University (No.DYL200809)Guangxi Science Foundation (No.0339068).
文摘The servo-motor possesses a strongly nonlinear property due to the effect of the stimulating input voltage, load-torque and environmental operating conditions. So it is rather difficult to derive a traditional mathematical model which is capable of expressing both its dynamics and steady-state characteristics. A neural network-based adaptive control strategy is proposed in this paper. In this method, two neural networks have been adopted for system identification （NNI） and control （NNC）, respectively. Then, the commonly-used specialized learning has been modified, by taking the NNI output as the approximation output of the servo-motor during the weights training to get sensitivity information. Moreover, the rule for choosing the learning rate is given on the basis of the analysis of Lyapunov stability. Finally, an example of applying the proposed control strategy on a servo-motor is presented to show its effectiveness.
