Some properties of a class of quasi-differentiable functions（the difference of two finite convex functions） are considered in this paper. And the convergence of the steepest descent algorithm for unconstrained and c...Some properties of a class of quasi-differentiable functions（the difference of two finite convex functions） are considered in this paper. And the convergence of the steepest descent algorithm for unconstrained and constrained quasi-differentiable programming is proved.展开更多
Under some assumptions, the solution set of a nonlinear complementarity problem coincides with the set of local minima of the corresponding minimization problem. This paper uses a family of new merit functions to deal...Under some assumptions, the solution set of a nonlinear complementarity problem coincides with the set of local minima of the corresponding minimization problem. This paper uses a family of new merit functions to deal with nonlinear complementarity problem where the underlying function is assumed to be a continuous but not necessarily locally Lipschitzian map and gives a descent algorithm for solving the nonsmooth continuous complementarity problems. In addition, the global convergence of the derivative free descent algorithm is also proved.展开更多
A gradient descent algorithm with adjustable parameter for attitude estimation is developed,aiming at the attitude measurement for small unmanned aerial vehicle(UAV)in real-time flight conditions.The accelerometer and...A gradient descent algorithm with adjustable parameter for attitude estimation is developed,aiming at the attitude measurement for small unmanned aerial vehicle(UAV)in real-time flight conditions.The accelerometer and magnetometer are introduced to construct an error equation with the gyros,thus the drifting characteristics of gyroscope can be compensated by solving the error equation utilized by the gradient descent algorithm.Performance of the presented algorithm is evaluated using a self-proposed micro-electro-mechanical system(MEMS)based attitude heading reference system which is mounted on a tri-axis turntable.The on-ground,turntable and flight experiments indicate that the estimation attitude has a good accuracy.Also,the presented system is compared with an open-source flight control system which runs extended Kalman filter(EKF),and the results show that the attitude control system using the gradient descent method can estimate the attitudes for UAV effectively.展开更多
In this research work,the localized generation from renewable resources and the distribution of energy to agricultural loads,which is a local microgrid concept,have been considered,and its feasibility has been assesse...In this research work,the localized generation from renewable resources and the distribution of energy to agricultural loads,which is a local microgrid concept,have been considered,and its feasibility has been assessed.Two dispatch algorithms,named Cycle Charging and Load Following,are implemented to find the optimal solution(i.e.,net cost,operation cost,carbon emission.energy cost,component sizing,etc.)of the hybrid system.The microgrid is also modeled in the DIgSILENT Power Factory platform,and the respective power system responses are then evaluated.The development of dispatch algorithms specifically tailored for agricultural applications has enabled to dynamically manage energy flows,responding to fluctuating demands and resource availability in real-time.Through careful consideration of factors such as seasonal variations and irrigation requirements,these algorithms have enhanced the resilience and adaptability of the microgrid to dynamic operational conditions.However,it is revealed that both approaches have produced the same techno-economic results showing no significant difference.This illustrates the fact that the considered microgrid can be implemented with either strategy without significant fluctuation in performance.The study has shown that the harmful gas emission has also been limited to only 17,928 kg/year of CO_(2),and 77.7 kg/year of Sulfur Dioxide.For the proposed microgrid and load profile of 165.29 kWh/day,the net present cost is USD 718,279,and the cost of energy is USD 0.0463 with a renewable fraction of 97.6%.The optimal sizes for PV,Bio,Grid,Electrolyzer,and Converter are 1494,500,999,999,500,and 495 kW,respectively.For a hydrogen tank(HTank),the optimal size is found to be 350 kg.This research work provides critical insights into the techno-economic feasibility and environmental impact of integrating biomass-PV-hydrogen storage-Grid hybrid renewable microgrids into agricultural settings.展开更多
In the field of calculating the attack area of air-to-air missiles in modern air combat scenarios,the limitations of existing research,including real-time calculation,accuracy efficiency trade-off,and the absence of t...In the field of calculating the attack area of air-to-air missiles in modern air combat scenarios,the limitations of existing research,including real-time calculation,accuracy efficiency trade-off,and the absence of the three-dimensional attack area model,restrict their practical applications.To address these issues,an improved backtracking algorithm is proposed to improve calculation efficiency.A significant reduction in solution time and maintenance of accuracy in the three-dimensional attack area are achieved by using the proposed algorithm.Furthermore,the age-layered population structure genetic programming(ALPS-GP)algorithm is introduced to determine an analytical polynomial model of the three-dimensional attack area,considering real-time requirements.The accuracy of the polynomial model is enhanced through the coefficient correction using an improved gradient descent algorithm.The study reveals a remarkable combination of high accuracy and efficient real-time computation,with a mean error of 91.89 m using the analytical polynomial model of the three-dimensional attack area solved in just 10^(-4)s,thus meeting the requirements of real-time combat scenarios.展开更多
The distributed nonconvex optimization problem of minimizing a global cost function formed by a sum of n local cost functions by using local information exchange is considered.This problem is an important component of...The distributed nonconvex optimization problem of minimizing a global cost function formed by a sum of n local cost functions by using local information exchange is considered.This problem is an important component of many machine learning techniques with data parallelism,such as deep learning and federated learning.We propose a distributed primal-dual stochastic gradient descent(SGD)algorithm,suitable for arbitrarily connected communication networks and any smooth(possibly nonconvex)cost functions.We show that the proposed algorithm achieves the linear speedup convergence rate O(1/(√nT))for general nonconvex cost functions and the linear speedup convergence rate O(1/(nT)) when the global cost function satisfies the Polyak-Lojasiewicz(P-L)condition,where T is the total number of iterations.We also show that the output of the proposed algorithm with constant parameters linearly converges to a neighborhood of a global optimum.We demonstrate through numerical experiments the efficiency of our algorithm in comparison with the baseline centralized SGD and recently proposed distributed SGD algorithms.展开更多
A recommender system(RS)relying on latent factor analysis usually adopts stochastic gradient descent(SGD)as its learning algorithm.However,owing to its serial mechanism,an SGD algorithm suffers from low efficiency and...A recommender system(RS)relying on latent factor analysis usually adopts stochastic gradient descent(SGD)as its learning algorithm.However,owing to its serial mechanism,an SGD algorithm suffers from low efficiency and scalability when handling large-scale industrial problems.Aiming at addressing this issue,this study proposes a momentum-incorporated parallel stochastic gradient descent(MPSGD)algorithm,whose main idea is two-fold:a)implementing parallelization via a novel datasplitting strategy,and b)accelerating convergence rate by integrating momentum effects into its training process.With it,an MPSGD-based latent factor(MLF)model is achieved,which is capable of performing efficient and high-quality recommendations.Experimental results on four high-dimensional and sparse matrices generated by industrial RS indicate that owing to an MPSGD algorithm,an MLF model outperforms the existing state-of-the-art ones in both computational efficiency and scalability.展开更多
A large number of sparse signal reconstruction algorithms have been continuously proposed, but almost all greedy algorithms add a fixed number of indices to the support set in each iteration. Although the mechanism of...A large number of sparse signal reconstruction algorithms have been continuously proposed, but almost all greedy algorithms add a fixed number of indices to the support set in each iteration. Although the mechanism of selecting the fixed number of indexes improves the reconstruction efficiency, it also brings the problem of low index selection accuracy. Based on the full study of the theory of compressed sensing, we propose a dynamic indexes selection strategy based on residual update to improve the performance of the compressed sampling matching pursuit algorithm (CoSaMP). As an extension of CoSaMP algorithm, the proposed algorithm adopts a residual comparison strategy to improve the accuracy of backtracking selected indexes. This backtracking strategy can efficiently select backtracking indexes. And without increasing the computational complexity, the proposed improvement algorithm has a higher exact reconstruction rate and peak signal to noise ratio (PSNR). Simulation results demonstrate the proposed algorithm significantly outperforms the CoSaMP for image recovery and one-dimensional signal.展开更多
In this paper, a new method named as the gradually descent method was proposed to solve the discrete global optimization problem. With the aid of an auxiliary function, this method enables to convert the problem of fi...In this paper, a new method named as the gradually descent method was proposed to solve the discrete global optimization problem. With the aid of an auxiliary function, this method enables to convert the problem of finding one discrete minimizer of the objective function f to that of finding another at each cycle. The auxiliary function can ensure that a point, except a prescribed point, is not its integer stationary point if the value of objective function at the point is greater than the scalar which is chosen properly. This property leads to a better minimizer of f found more easily by some classical local search methods. The computational results show that this algorithm is quite efficient and reliable for solving nonlinear integer programming problems.展开更多
Coordinate descent method is a unconstrained optimization technique. When it is applied to support vector machine （SVM）, at each step the method updates one component of w by solving a one-variable sub-problem while...Coordinate descent method is a unconstrained optimization technique. When it is applied to support vector machine （SVM）, at each step the method updates one component of w by solving a one-variable sub-problem while fixing other components. All components of w update after one iteration. Then go to next iteration. Though the method converges and converges fast in the beginning, it converges slow for final convergence. To improve the speed of final convergence of coordinate descent method, Hooke and Jeeves algorithm which adds pattern search after every iteration in coordinate descent method was applied to SVM and a global Newton algorithm was used to solve one-variable subproblems. We proved the convergence of the algorithm. Experimental results show Hooke and Jeeves＇ method does accelerate convergence specially for final convergence and achieves higher testing accuracy more quickly in classification.展开更多
基金Supported by the State Foundations of Ph.D.Units(20020141013)Supported by the NSF of China(10001007)
文摘Some properties of a class of quasi-differentiable functions（the difference of two finite convex functions） are considered in this paper. And the convergence of the steepest descent algorithm for unconstrained and constrained quasi-differentiable programming is proved.
基金Acknowledgment： This work was partly supported by the National Natural Science Foundation of China（60672150） and Science and Technology Planning Project of Shenzhen, China （szkj0706）.
基金Supported by the National Science foundation of China(10671126, 40771095)the Key Project for Fundamental Research of STCSM(06JC14057)+1 种基金Shanghai Leading Academic Discipline Project(S30501)the Innovation Fund Project for Graduate Students of Shanghai(JWCXSL0801)
文摘Under some assumptions, the solution set of a nonlinear complementarity problem coincides with the set of local minima of the corresponding minimization problem. This paper uses a family of new merit functions to deal with nonlinear complementarity problem where the underlying function is assumed to be a continuous but not necessarily locally Lipschitzian map and gives a descent algorithm for solving the nonsmooth continuous complementarity problems. In addition, the global convergence of the derivative free descent algorithm is also proved.
基金supported by the Fundamental Research Funds for the Central Universities(No.56XAA17075)
文摘A gradient descent algorithm with adjustable parameter for attitude estimation is developed,aiming at the attitude measurement for small unmanned aerial vehicle(UAV)in real-time flight conditions.The accelerometer and magnetometer are introduced to construct an error equation with the gyros,thus the drifting characteristics of gyroscope can be compensated by solving the error equation utilized by the gradient descent algorithm.Performance of the presented algorithm is evaluated using a self-proposed micro-electro-mechanical system(MEMS)based attitude heading reference system which is mounted on a tri-axis turntable.The on-ground,turntable and flight experiments indicate that the estimation attitude has a good accuracy.Also,the presented system is compared with an open-source flight control system which runs extended Kalman filter(EKF),and the results show that the attitude control system using the gradient descent method can estimate the attitudes for UAV effectively.
基金financed by the Ministry of Science and Technology(MOST)Bangladesh under Special Research grant for the FY 2023-24(SRG 232410)Further,the authors extend their appreciation to the Deanship of Scientific Research at Northern Border University,Arar,Saudi Arabi for funding this research work through the project number“NBU-FFR-2025-3623-05”。
文摘In this research work,the localized generation from renewable resources and the distribution of energy to agricultural loads,which is a local microgrid concept,have been considered,and its feasibility has been assessed.Two dispatch algorithms,named Cycle Charging and Load Following,are implemented to find the optimal solution(i.e.,net cost,operation cost,carbon emission.energy cost,component sizing,etc.)of the hybrid system.The microgrid is also modeled in the DIgSILENT Power Factory platform,and the respective power system responses are then evaluated.The development of dispatch algorithms specifically tailored for agricultural applications has enabled to dynamically manage energy flows,responding to fluctuating demands and resource availability in real-time.Through careful consideration of factors such as seasonal variations and irrigation requirements,these algorithms have enhanced the resilience and adaptability of the microgrid to dynamic operational conditions.However,it is revealed that both approaches have produced the same techno-economic results showing no significant difference.This illustrates the fact that the considered microgrid can be implemented with either strategy without significant fluctuation in performance.The study has shown that the harmful gas emission has also been limited to only 17,928 kg/year of CO_(2),and 77.7 kg/year of Sulfur Dioxide.For the proposed microgrid and load profile of 165.29 kWh/day,the net present cost is USD 718,279,and the cost of energy is USD 0.0463 with a renewable fraction of 97.6%.The optimal sizes for PV,Bio,Grid,Electrolyzer,and Converter are 1494,500,999,999,500,and 495 kW,respectively.For a hydrogen tank(HTank),the optimal size is found to be 350 kg.This research work provides critical insights into the techno-economic feasibility and environmental impact of integrating biomass-PV-hydrogen storage-Grid hybrid renewable microgrids into agricultural settings.
基金National Natural Science Foundation of China(62373187)Forward-looking Layout Special Projects(ILA220591A22)。
文摘In the field of calculating the attack area of air-to-air missiles in modern air combat scenarios,the limitations of existing research,including real-time calculation,accuracy efficiency trade-off,and the absence of the three-dimensional attack area model,restrict their practical applications.To address these issues,an improved backtracking algorithm is proposed to improve calculation efficiency.A significant reduction in solution time and maintenance of accuracy in the three-dimensional attack area are achieved by using the proposed algorithm.Furthermore,the age-layered population structure genetic programming(ALPS-GP)algorithm is introduced to determine an analytical polynomial model of the three-dimensional attack area,considering real-time requirements.The accuracy of the polynomial model is enhanced through the coefficient correction using an improved gradient descent algorithm.The study reveals a remarkable combination of high accuracy and efficient real-time computation,with a mean error of 91.89 m using the analytical polynomial model of the three-dimensional attack area solved in just 10^(-4)s,thus meeting the requirements of real-time combat scenarios.
基金supported by the Knut and Alice Wallenberg Foundationthe Swedish Foundation for Strategic Research+1 种基金the Swedish Research Councilthe National Natural Science Foundation of China(62133003,61991403,61991404,61991400)。
文摘The distributed nonconvex optimization problem of minimizing a global cost function formed by a sum of n local cost functions by using local information exchange is considered.This problem is an important component of many machine learning techniques with data parallelism,such as deep learning and federated learning.We propose a distributed primal-dual stochastic gradient descent(SGD)algorithm,suitable for arbitrarily connected communication networks and any smooth(possibly nonconvex)cost functions.We show that the proposed algorithm achieves the linear speedup convergence rate O(1/(√nT))for general nonconvex cost functions and the linear speedup convergence rate O(1/(nT)) when the global cost function satisfies the Polyak-Lojasiewicz(P-L)condition,where T is the total number of iterations.We also show that the output of the proposed algorithm with constant parameters linearly converges to a neighborhood of a global optimum.We demonstrate through numerical experiments the efficiency of our algorithm in comparison with the baseline centralized SGD and recently proposed distributed SGD algorithms.
基金supported in part by the National Natural Science Foundation of China(61772493)the Deanship of Scientific Research(DSR)at King Abdulaziz University(RG-48-135-40)+1 种基金Guangdong Province Universities and College Pearl River Scholar Funded Scheme(2019)the Natural Science Foundation of Chongqing(cstc2019jcyjjqX0013)。
文摘A recommender system(RS)relying on latent factor analysis usually adopts stochastic gradient descent(SGD)as its learning algorithm.However,owing to its serial mechanism,an SGD algorithm suffers from low efficiency and scalability when handling large-scale industrial problems.Aiming at addressing this issue,this study proposes a momentum-incorporated parallel stochastic gradient descent(MPSGD)algorithm,whose main idea is two-fold:a)implementing parallelization via a novel datasplitting strategy,and b)accelerating convergence rate by integrating momentum effects into its training process.With it,an MPSGD-based latent factor(MLF)model is achieved,which is capable of performing efficient and high-quality recommendations.Experimental results on four high-dimensional and sparse matrices generated by industrial RS indicate that owing to an MPSGD algorithm,an MLF model outperforms the existing state-of-the-art ones in both computational efficiency and scalability.
文摘A large number of sparse signal reconstruction algorithms have been continuously proposed, but almost all greedy algorithms add a fixed number of indices to the support set in each iteration. Although the mechanism of selecting the fixed number of indexes improves the reconstruction efficiency, it also brings the problem of low index selection accuracy. Based on the full study of the theory of compressed sensing, we propose a dynamic indexes selection strategy based on residual update to improve the performance of the compressed sampling matching pursuit algorithm (CoSaMP). As an extension of CoSaMP algorithm, the proposed algorithm adopts a residual comparison strategy to improve the accuracy of backtracking selected indexes. This backtracking strategy can efficiently select backtracking indexes. And without increasing the computational complexity, the proposed improvement algorithm has a higher exact reconstruction rate and peak signal to noise ratio (PSNR). Simulation results demonstrate the proposed algorithm significantly outperforms the CoSaMP for image recovery and one-dimensional signal.
基金Project supported by the National Natural Science Foundation of China(Grant No.10271073)
文摘In this paper, a new method named as the gradually descent method was proposed to solve the discrete global optimization problem. With the aid of an auxiliary function, this method enables to convert the problem of finding one discrete minimizer of the objective function f to that of finding another at each cycle. The auxiliary function can ensure that a point, except a prescribed point, is not its integer stationary point if the value of objective function at the point is greater than the scalar which is chosen properly. This property leads to a better minimizer of f found more easily by some classical local search methods. The computational results show that this algorithm is quite efficient and reliable for solving nonlinear integer programming problems.
基金supported by the National Natural Science Foundation of China (6057407560705004)
文摘Coordinate descent method is a unconstrained optimization technique. When it is applied to support vector machine （SVM）, at each step the method updates one component of w by solving a one-variable sub-problem while fixing other components. All components of w update after one iteration. Then go to next iteration. Though the method converges and converges fast in the beginning, it converges slow for final convergence. To improve the speed of final convergence of coordinate descent method, Hooke and Jeeves algorithm which adds pattern search after every iteration in coordinate descent method was applied to SVM and a global Newton algorithm was used to solve one-variable subproblems. We proved the convergence of the algorithm. Experimental results show Hooke and Jeeves＇ method does accelerate convergence specially for final convergence and achieves higher testing accuracy more quickly in classification.
