The distribution of sampling data influences completeness of rule base so that extrapolating missing rules is very difficult. Based on data mining, a self-learning method is developed for identifying fuzzy model and e...The distribution of sampling data influences completeness of rule base so that extrapolating missing rules is very difficult. Based on data mining, a self-learning method is developed for identifying fuzzy model and extrapolating missing rules, by means of confidence measure and the improved gradient descent method. The proposed approach can not only identify fuzzy model, update its parameters and determine optimal output fuzzy sets simultaneously, but also resolve the uncontrollable problem led by the regions that data do not cover. The simulation results show the effectiveness and accuracy of the proposed approach with the classical truck backer-upper control problem verifying.展开更多
A new algorithm to exploit the learning rates of gradient descent method is presented, based on the second-order Taylor expansion of the error energy function with respect to learning rate, at some values decided by &...A new algorithm to exploit the learning rates of gradient descent method is presented, based on the second-order Taylor expansion of the error energy function with respect to learning rate, at some values decided by "award-punish" strategy. Detailed deduction of the algorithm applied to RBF networks is given. Simulation studies show that this algorithm can increase the rate of convergence and improve the performance of the gradient descent method.展开更多
Diabetic Retinopathy(DR)is a type of disease in eyes as a result of a diabetic condition that ends up damaging the retina,leading to blindness or loss of vision.Morphological and physiological retinal variations invol...Diabetic Retinopathy(DR)is a type of disease in eyes as a result of a diabetic condition that ends up damaging the retina,leading to blindness or loss of vision.Morphological and physiological retinal variations involving slowdown of blood flow in the retina,elevation of leukocyte cohesion,basement membrane dystrophy,and decline of pericyte cells,develop.As DR in its initial stage has no symptoms,early detection and automated diagnosis can prevent further visual damage.In this research,using a Deep Neural Network(DNN),segmentation methods are proposed to detect the retinal defects such as exudates,hemorrhages,microaneurysms from digital fundus images and then the conditions are classified accurately to identify the grades as mild,moderate,severe,no PDR,PDR in DR.Initially,saliency detection is applied on color images to detect maximum salient foreground objects from the background.Next,structure tensor is applied powerfully to enhance the local patterns of edge elements and intensity changes that occur on edges of the object.Finally,active contours approximation is performed using gradient descent to segment the lesions from the images.Afterwards,the output images from the proposed segmentation process are subjected to evaluate the ratio between the total contour area and the total true contour arc length to label the classes as mild,moderate,severe,No PDR and PDR.Based on the computed ratio obtained from segmented images,the severity levels were identified.Meanwhile,statistical parameters like the mean and the standard deviation of pixel intensities,mean of hue,saturation and deviation clustering,are estimated through K-means,which are computed as features from the output images of the proposed segmentation process.Using these derived feature sets as input to the classifier,the classification of DR was performed.Finally,a VGG-19 deep neural network was trained and tested using the derived feature sets from the KAGGLE fundus image dataset containing 35,126 images in total.The VGG-19 is trained with features extracted from 20,000 images and tested with features extracted from 5,000 images to achieve a sensitivity of 82%and an accuracy of 96%.The proposed system was able to label and classify DR grades automatically.展开更多
The gradient descent(GD)method is used to fit the measured data(i.e.,the laser grain-size distribution of the sediments)with a sum of four weighted lognormal functions.The method is calibrated by a series of ideal num...The gradient descent(GD)method is used to fit the measured data(i.e.,the laser grain-size distribution of the sediments)with a sum of four weighted lognormal functions.The method is calibrated by a series of ideal numerical experiments.The numerical results indicate that the GD method not only is easy to operate but also could effectively optimize the parameters of the fitting function with the error decreasing steadily.The method is applied to numerical partitioning of laser grain-size components of a series of Garzêloess samples and three bottom sedimentary samples of submarine turbidity currents modeled in an open channel laboratory flume.The overall fitting results are satisfactory.As a new approach of data fitting,the GD method could also be adapted to solve other optimization problems.展开更多
In this work,we develop a stochastic gradient descent method for the computational optimal design of random rough surfaces in thin-film solar cells.We formulate the design problems as random PDE-constrained optimizati...In this work,we develop a stochastic gradient descent method for the computational optimal design of random rough surfaces in thin-film solar cells.We formulate the design problems as random PDE-constrained optimization problems and seek the optimal statistical parameters for the random surfaces.The optimizations at fixed frequency as well as at multiple frequencies and multiple incident angles are investigated.To evaluate the gradient of the objective function,we derive the shape derivatives for the interfaces and apply the adjoint state method to perform the computation.The stochastic gradient descent method evaluates the gradient of the objective function only at a few samples for each iteration,which reduces the computational cost significantly.Various numerical experiments are conducted to illustrate the efficiency of the method and significant increases of the absorptance for the optimal random structures.We also examine the convergence of the stochastic gradient descent algorithm theoretically and prove that the numerical method is convergent under certain assumptions for the random interfaces.展开更多
In this paper,we propose a gradient descent method to estimate the parameters in a Markov chain choice model.Particularly,we derive closed-form formula for the gradient of the log-likelihood function and show the conv...In this paper,we propose a gradient descent method to estimate the parameters in a Markov chain choice model.Particularly,we derive closed-form formula for the gradient of the log-likelihood function and show the convergence of the algorithm.Numerical experiments verify the efficiency of our approach by comparing with the expectation-maximization algorithm.We show that the similar result can be extended to a more general case that one does not have observation of the no-purchase data.展开更多
This paper is concerned with convergence of stochastic gradient algorithms with momentum terms in the nonconvex setting.A class of stochastic momentum methods,including stochastic gradient descent,heavy ball and Neste...This paper is concerned with convergence of stochastic gradient algorithms with momentum terms in the nonconvex setting.A class of stochastic momentum methods,including stochastic gradient descent,heavy ball and Nesterov’s accelerated gradient,is analyzed in a general framework under mild assumptions.Based on the convergence result of expected gradients,the authors prove the almost sure convergence by a detailed discussion of the effects of momentum and the number of upcrossings.It is worth noting that there are not additional restrictions imposed on the objective function and stepsize.Another improvement over previous results is that the existing Lipschitz condition of the gradient is relaxed into the condition of H?lder continuity.As a byproduct,the authors apply a localization procedure to extend the results to stochastic stepsizes.展开更多
Purpose-The purpose of this paper is to use the internal model control to deal with nonlinear stable systems affected by parametric uncertainties.Design/methodology/approach-The dynamics of a considered system are app...Purpose-The purpose of this paper is to use the internal model control to deal with nonlinear stable systems affected by parametric uncertainties.Design/methodology/approach-The dynamics of a considered system are approximated by a Takagi-Sugeno fuzzy model.The parameters of the fuzzy rules premises are determined manually.However,the parameters of the fuzzy rules conclusions are updated using the descent gradient method under inequality constraints in order to ensure the stability of each local model.In fact,without making these constraints the training algorithm can procure one or several unstable local models even if the desired accuracy in the training step is achieved.The considered robust control approach is the internal model.It is synthesized based on the Takagi-Sugeno fuzzy model.Two control strategies are considered.The first one is based on the parallel distribution compensation principle.It consists in associating an internal model control for each local model.However,for the second strategy,the control law is computed based on the global Takagi-Sugeno fuzzy model.Findings-According to the simulation results,the stability of all local models is obtained and the proposed fuzzy internal model control approaches ensure robustness against parametric uncertainties.Originality/value-This paper introduces a method for the identification of fuzzy model parameters ensuring the stability of all local models.Using the resulting fuzzy model,two fuzzy internal model control designs are presented.展开更多
This paper investigates the cooperative adaptive optimal output regulation problem of continuous-time linear multi-agent systems.As the multi-agent system dynamics are uncertain,solving regulator equations and the cor...This paper investigates the cooperative adaptive optimal output regulation problem of continuous-time linear multi-agent systems.As the multi-agent system dynamics are uncertain,solving regulator equations and the corresponding algebraic Riccati equations is challenging,especially for high-order systems.In this paper,a novel method is proposed to approximate the solution of regulator equations,i.e.,gradient descent method.It is worth noting that this method obtains gradients through online data rather than model information.A data-driven distributed adaptive suboptimal controller is developed by adaptive dynamic programming,so that each follower can achieve asymptotic tracking and disturbance rejection.Finally,the effectiveness of the proposed control method is validated by simulations.展开更多
Purpose–Since many global path planning algorithms cannot achieve the planned path with both safety and economy,this study aims to propose a path planning method for unmanned vehicles with a controllable distance fro...Purpose–Since many global path planning algorithms cannot achieve the planned path with both safety and economy,this study aims to propose a path planning method for unmanned vehicles with a controllable distance from obstacles.Design/methodology/approach–First,combining satellite image and the Voronoi field algorithm(VFA)generates rasterized environmental information and establishes navigation area boundary.Second,establishing a hazard function associated with navigation area boundary improves the evaluation function of the A*algorithm and uses the improved A*algorithm for global path planning.Finally,to reduce the number of redundant nodes in the planned path and smooth the path,node optimization and gradient descent method(GDM)are used.Then,a continuous smooth path that meets the actual navigation requirements of unmanned vehicle is obtained.Findings–The simulation experiment proved that the proposed global path planning method can realize the control of the distance between the planned path and the obstacle by setting different navigation area boundaries.The node reduction rate is between 33.52%and 73.15%,and the smoothness meets the navigation requirements.This method is reasonable and effective in the global path planning process of unmanned vehicle and can provide reference to unmanned vehicles’autonomous obstacle avoidance decision-making.Originality/value–This study establishes navigation area boundary for the environment based on the VFA and uses the improved Aalgorithm to generate a navigation path that takes into account both safety and economy.This study also proposes a method to solve the redundancy of grid environment path nodes and large-angle steering and to smooth the path to improve the applicability of the proposed global path planning method.The proposed global path planning method solves the requirements of path safety and smoothness.展开更多
In this paper,we propose an accelerated stochastic variance reduction gradient method with a trust-region-like framework,referred as the NMSVRG-TR method.Based on NMSVRG,we incorporate a Katyusha-like acceleration ste...In this paper,we propose an accelerated stochastic variance reduction gradient method with a trust-region-like framework,referred as the NMSVRG-TR method.Based on NMSVRG,we incorporate a Katyusha-like acceleration step into the stochastic trust region scheme,which improves the convergence rate of the SVRG methods.Under appropriate assumptions,the linear convergence of the algorithm is provided for strongly convex objective functions.Numerical experiment results show that our algorithm is generally superior to some existing stochastic gradient methods.展开更多
基金This project was supported by State Science &Technology Pursuing Project (2001BA204B01) of China and Foundation forUniversity Key Teacher by the Ministry of Education of China.
文摘The distribution of sampling data influences completeness of rule base so that extrapolating missing rules is very difficult. Based on data mining, a self-learning method is developed for identifying fuzzy model and extrapolating missing rules, by means of confidence measure and the improved gradient descent method. The proposed approach can not only identify fuzzy model, update its parameters and determine optimal output fuzzy sets simultaneously, but also resolve the uncontrollable problem led by the regions that data do not cover. The simulation results show the effectiveness and accuracy of the proposed approach with the classical truck backer-upper control problem verifying.
基金Open Foundation of State Key Lab of Transmission of Wide-Band FiberTechnologies of Communication Systems
文摘A new algorithm to exploit the learning rates of gradient descent method is presented, based on the second-order Taylor expansion of the error energy function with respect to learning rate, at some values decided by "award-punish" strategy. Detailed deduction of the algorithm applied to RBF networks is given. Simulation studies show that this algorithm can increase the rate of convergence and improve the performance of the gradient descent method.
文摘Diabetic Retinopathy(DR)is a type of disease in eyes as a result of a diabetic condition that ends up damaging the retina,leading to blindness or loss of vision.Morphological and physiological retinal variations involving slowdown of blood flow in the retina,elevation of leukocyte cohesion,basement membrane dystrophy,and decline of pericyte cells,develop.As DR in its initial stage has no symptoms,early detection and automated diagnosis can prevent further visual damage.In this research,using a Deep Neural Network(DNN),segmentation methods are proposed to detect the retinal defects such as exudates,hemorrhages,microaneurysms from digital fundus images and then the conditions are classified accurately to identify the grades as mild,moderate,severe,no PDR,PDR in DR.Initially,saliency detection is applied on color images to detect maximum salient foreground objects from the background.Next,structure tensor is applied powerfully to enhance the local patterns of edge elements and intensity changes that occur on edges of the object.Finally,active contours approximation is performed using gradient descent to segment the lesions from the images.Afterwards,the output images from the proposed segmentation process are subjected to evaluate the ratio between the total contour area and the total true contour arc length to label the classes as mild,moderate,severe,No PDR and PDR.Based on the computed ratio obtained from segmented images,the severity levels were identified.Meanwhile,statistical parameters like the mean and the standard deviation of pixel intensities,mean of hue,saturation and deviation clustering,are estimated through K-means,which are computed as features from the output images of the proposed segmentation process.Using these derived feature sets as input to the classifier,the classification of DR was performed.Finally,a VGG-19 deep neural network was trained and tested using the derived feature sets from the KAGGLE fundus image dataset containing 35,126 images in total.The VGG-19 is trained with features extracted from 20,000 images and tested with features extracted from 5,000 images to achieve a sensitivity of 82%and an accuracy of 96%.The proposed system was able to label and classify DR grades automatically.
基金supported by the National Natural Science Foundation of China(Grant Nos.41072176,41371496)the National Science and Technology Supporting Program of China(Grant No.2013BAK05B04)the Fundamental Research Funds for the Central Universities(Grant No.201261006)
文摘The gradient descent(GD)method is used to fit the measured data(i.e.,the laser grain-size distribution of the sediments)with a sum of four weighted lognormal functions.The method is calibrated by a series of ideal numerical experiments.The numerical results indicate that the GD method not only is easy to operate but also could effectively optimize the parameters of the fitting function with the error decreasing steadily.The method is applied to numerical partitioning of laser grain-size components of a series of Garzêloess samples and three bottom sedimentary samples of submarine turbidity currents modeled in an open channel laboratory flume.The overall fitting results are satisfactory.As a new approach of data fitting,the GD method could also be adapted to solve other optimization problems.
基金partially supported by the DOE grant DE-SC0022253the work of JL was partially supported by the NSF grant DMS-1719851 and DMS-2011148.
文摘In this work,we develop a stochastic gradient descent method for the computational optimal design of random rough surfaces in thin-film solar cells.We formulate the design problems as random PDE-constrained optimization problems and seek the optimal statistical parameters for the random surfaces.The optimizations at fixed frequency as well as at multiple frequencies and multiple incident angles are investigated.To evaluate the gradient of the objective function,we derive the shape derivatives for the interfaces and apply the adjoint state method to perform the computation.The stochastic gradient descent method evaluates the gradient of the objective function only at a few samples for each iteration,which reduces the computational cost significantly.Various numerical experiments are conducted to illustrate the efficiency of the method and significant increases of the absorptance for the optimal random structures.We also examine the convergence of the stochastic gradient descent algorithm theoretically and prove that the numerical method is convergent under certain assumptions for the random interfaces.
文摘In this paper,we propose a gradient descent method to estimate the parameters in a Markov chain choice model.Particularly,we derive closed-form formula for the gradient of the log-likelihood function and show the convergence of the algorithm.Numerical experiments verify the efficiency of our approach by comparing with the expectation-maximization algorithm.We show that the similar result can be extended to a more general case that one does not have observation of the no-purchase data.
基金supported by the National Natural Science Foundation of China (Nos. 11631004,12031009)the National Key R&D Program of China (No. 2018YFA0703900)。
文摘This paper is concerned with convergence of stochastic gradient algorithms with momentum terms in the nonconvex setting.A class of stochastic momentum methods,including stochastic gradient descent,heavy ball and Nesterov’s accelerated gradient,is analyzed in a general framework under mild assumptions.Based on the convergence result of expected gradients,the authors prove the almost sure convergence by a detailed discussion of the effects of momentum and the number of upcrossings.It is worth noting that there are not additional restrictions imposed on the objective function and stepsize.Another improvement over previous results is that the existing Lipschitz condition of the gradient is relaxed into the condition of H?lder continuity.As a byproduct,the authors apply a localization procedure to extend the results to stochastic stepsizes.
文摘Purpose-The purpose of this paper is to use the internal model control to deal with nonlinear stable systems affected by parametric uncertainties.Design/methodology/approach-The dynamics of a considered system are approximated by a Takagi-Sugeno fuzzy model.The parameters of the fuzzy rules premises are determined manually.However,the parameters of the fuzzy rules conclusions are updated using the descent gradient method under inequality constraints in order to ensure the stability of each local model.In fact,without making these constraints the training algorithm can procure one or several unstable local models even if the desired accuracy in the training step is achieved.The considered robust control approach is the internal model.It is synthesized based on the Takagi-Sugeno fuzzy model.Two control strategies are considered.The first one is based on the parallel distribution compensation principle.It consists in associating an internal model control for each local model.However,for the second strategy,the control law is computed based on the global Takagi-Sugeno fuzzy model.Findings-According to the simulation results,the stability of all local models is obtained and the proposed fuzzy internal model control approaches ensure robustness against parametric uncertainties.Originality/value-This paper introduces a method for the identification of fuzzy model parameters ensuring the stability of all local models.Using the resulting fuzzy model,two fuzzy internal model control designs are presented.
基金supported in part by the National Natural Science Foundation of China under Grant No.62373090the U.S.National Science Foundation under Grant No.CNS-2227153.
文摘This paper investigates the cooperative adaptive optimal output regulation problem of continuous-time linear multi-agent systems.As the multi-agent system dynamics are uncertain,solving regulator equations and the corresponding algebraic Riccati equations is challenging,especially for high-order systems.In this paper,a novel method is proposed to approximate the solution of regulator equations,i.e.,gradient descent method.It is worth noting that this method obtains gradients through online data rather than model information.A data-driven distributed adaptive suboptimal controller is developed by adaptive dynamic programming,so that each follower can achieve asymptotic tracking and disturbance rejection.Finally,the effectiveness of the proposed control method is validated by simulations.
文摘Purpose–Since many global path planning algorithms cannot achieve the planned path with both safety and economy,this study aims to propose a path planning method for unmanned vehicles with a controllable distance from obstacles.Design/methodology/approach–First,combining satellite image and the Voronoi field algorithm(VFA)generates rasterized environmental information and establishes navigation area boundary.Second,establishing a hazard function associated with navigation area boundary improves the evaluation function of the A*algorithm and uses the improved A*algorithm for global path planning.Finally,to reduce the number of redundant nodes in the planned path and smooth the path,node optimization and gradient descent method(GDM)are used.Then,a continuous smooth path that meets the actual navigation requirements of unmanned vehicle is obtained.Findings–The simulation experiment proved that the proposed global path planning method can realize the control of the distance between the planned path and the obstacle by setting different navigation area boundaries.The node reduction rate is between 33.52%and 73.15%,and the smoothness meets the navigation requirements.This method is reasonable and effective in the global path planning process of unmanned vehicle and can provide reference to unmanned vehicles’autonomous obstacle avoidance decision-making.Originality/value–This study establishes navigation area boundary for the environment based on the VFA and uses the improved Aalgorithm to generate a navigation path that takes into account both safety and economy.This study also proposes a method to solve the redundancy of grid environment path nodes and large-angle steering and to smooth the path to improve the applicability of the proposed global path planning method.The proposed global path planning method solves the requirements of path safety and smoothness.
文摘In this paper,we propose an accelerated stochastic variance reduction gradient method with a trust-region-like framework,referred as the NMSVRG-TR method.Based on NMSVRG,we incorporate a Katyusha-like acceleration step into the stochastic trust region scheme,which improves the convergence rate of the SVRG methods.Under appropriate assumptions,the linear convergence of the algorithm is provided for strongly convex objective functions.Numerical experiment results show that our algorithm is generally superior to some existing stochastic gradient methods.
