An improved genetic algorithm（IGA） based on a novel selection strategy to handle nonlinear programming problems is proposed.Each individual in selection process is represented as a three-dimensional feature vector w...An improved genetic algorithm（IGA） based on a novel selection strategy to handle nonlinear programming problems is proposed.Each individual in selection process is represented as a three-dimensional feature vector which is composed of objective function value,the degree of constraints violations and the number of constraints violations.It is easy to distinguish excellent individuals from general individuals by using an individuals＇ feature vector.Additionally,a local search（LS） process is incorporated into selection operation so as to find feasible solutions located in the neighboring areas of some infeasible solutions.The combination of IGA and LS should offer the advantage of both the quality of solutions and diversity of solutions.Experimental results over a set of benchmark problems demonstrate that IGA has better performance than other algorithms.展开更多
In this paper, we focus on a class of nonlinear bilevel programming problems where the follower’s objective is a function of the linear expression of all variables, and the follower’s constraint functions are convex...In this paper, we focus on a class of nonlinear bilevel programming problems where the follower’s objective is a function of the linear expression of all variables, and the follower’s constraint functions are convex with respect to the follower’s variables. First, based on the features of the follower’s problem, we give a new decomposition scheme by which the follower’s optimal solution can be obtained easily. Then, to solve efficiently this class of problems by using evolutionary algorithm, novel evolutionary operators are designed by considering the best individuals and the diversity of individuals in the populations. Finally, based on these techniques, a new evolutionary algorithm is proposed. The numerical results on 20 test problems illustrate that the proposed algorithm is efficient and stable.展开更多
Refineries often need to find similar crude oil to replace the scarce crude oil for stabilizing the feedstock property. We introduced the method for calculation of crude blended properties firstly, and then created a ...Refineries often need to find similar crude oil to replace the scarce crude oil for stabilizing the feedstock property. We introduced the method for calculation of crude blended properties firstly, and then created a crude oil selection and blending optimization model based on the data of crude oil property. The model is a mixed-integer nonlinear programming(MINLP) with constraints, and the target is to maximize the similarity between the blended crude oil and the objective crude oil. Furthermore, the model takes into account the selection of crude oils and their blending ratios simultaneously, and transforms the problem of looking for similar crude oil into the crude oil selection and blending optimization problem. We applied the Improved Cuckoo Search(ICS) algorithm to solving the model. Through the simulations, ICS was compared with the genetic algorithm, the particle swarm optimization algorithm and the CPLEX solver. The results show that ICS has very good optimization efficiency. The blending solution can provide a reference for refineries to find the similar crude oil. And the method proposed can also give some references to selection and blending optimization of other materials.展开更多
The distributed nonconvex optimization problem of minimizing a global cost function formed by a sum of n local cost functions by using local information exchange is considered.This problem is an important component of...The distributed nonconvex optimization problem of minimizing a global cost function formed by a sum of n local cost functions by using local information exchange is considered.This problem is an important component of many machine learning techniques with data parallelism,such as deep learning and federated learning.We propose a distributed primal-dual stochastic gradient descent(SGD)algorithm,suitable for arbitrarily connected communication networks and any smooth(possibly nonconvex)cost functions.We show that the proposed algorithm achieves the linear speedup convergence rate O(1/(√nT))for general nonconvex cost functions and the linear speedup convergence rate O(1/(nT)) when the global cost function satisfies the Polyak-Lojasiewicz(P-L)condition,where T is the total number of iterations.We also show that the output of the proposed algorithm with constant parameters linearly converges to a neighborhood of a global optimum.We demonstrate through numerical experiments the efficiency of our algorithm in comparison with the baseline centralized SGD and recently proposed distributed SGD algorithms.展开更多
In this paper, a new method named as the gradually descent method was proposed to solve the discrete global optimization problem. With the aid of an auxiliary function, this method enables to convert the problem of fi...In this paper, a new method named as the gradually descent method was proposed to solve the discrete global optimization problem. With the aid of an auxiliary function, this method enables to convert the problem of finding one discrete minimizer of the objective function f to that of finding another at each cycle. The auxiliary function can ensure that a point, except a prescribed point, is not its integer stationary point if the value of objective function at the point is greater than the scalar which is chosen properly. This property leads to a better minimizer of f found more easily by some classical local search methods. The computational results show that this algorithm is quite efficient and reliable for solving nonlinear integer programming problems.展开更多
Based on the improved particle swarm optimization(PSO) algorithm,an optimization approach for the cargo oil tank design(COTD) is presented in this paper.The purpose is to design an optimal overall dimension of the car...Based on the improved particle swarm optimization(PSO) algorithm,an optimization approach for the cargo oil tank design(COTD) is presented in this paper.The purpose is to design an optimal overall dimension of the cargo oil tank(COT) under various kinds of constraints in the preliminary design stage.A non-linear programming model is built to simulate the optimization design,in which the requirements and rules for COTD are used as the constraints.Considering the distance between the inner shell and hull,a fuzzy constraint is used to express the feasibility degree of the double-hull configuration.In terms of the characteristic of COTD,the PSO algorithm is improved to solve this problem.A bivariate extremum strategy is presented to deal with the fuzzy constraint,by which the maximum and minimum cargo capacities are obtained simultaneously.Finally,the simulation demonstrates the feasibility and effectiveness of the proposed approach.展开更多
In the field of calculating the attack area of air-to-air missiles in modern air combat scenarios,the limitations of existing research,including real-time calculation,accuracy efficiency trade-off,and the absence of t...In the field of calculating the attack area of air-to-air missiles in modern air combat scenarios,the limitations of existing research,including real-time calculation,accuracy efficiency trade-off,and the absence of the three-dimensional attack area model,restrict their practical applications.To address these issues,an improved backtracking algorithm is proposed to improve calculation efficiency.A significant reduction in solution time and maintenance of accuracy in the three-dimensional attack area are achieved by using the proposed algorithm.Furthermore,the age-layered population structure genetic programming(ALPS-GP)algorithm is introduced to determine an analytical polynomial model of the three-dimensional attack area,considering real-time requirements.The accuracy of the polynomial model is enhanced through the coefficient correction using an improved gradient descent algorithm.The study reveals a remarkable combination of high accuracy and efficient real-time computation,with a mean error of 91.89 m using the analytical polynomial model of the three-dimensional attack area solved in just 10^(-4)s,thus meeting the requirements of real-time combat scenarios.展开更多
Aim To determine the global optimal solution for a mine ventilation network under given network topology and airway characteristics. Methods\ The genetic algorithm was used to find the global optimal solution of the ...Aim To determine the global optimal solution for a mine ventilation network under given network topology and airway characteristics. Methods\ The genetic algorithm was used to find the global optimal solution of the network. Results\ A modified genetic algorithm is presented with its characteristics and principle. Instead of working on the conventional bit by bit operation, both the crossover and mutation operators are handled in real values by the proposed algorithms. To prevent the system from turning into a premature problem, the elitists from two groups of possible solutions are selected to reproduce the new populations. Conclusion\ The simulation results show that the method outperforms the conventional nonlinear programming approach whether from the viewpoint of the number of iterations required to find the optimum solutions or from the final solutions obtained.展开更多
In this exposition paper we present the optimal transport problem of Monge-Ampère-Kantorovitch(MAK in short)and its approximative entropical regularization.Contrary to the MAK optimal transport problem,the soluti...In this exposition paper we present the optimal transport problem of Monge-Ampère-Kantorovitch(MAK in short)and its approximative entropical regularization.Contrary to the MAK optimal transport problem,the solution of the entropical optimal transport problem is always unique,and is characterized by the Schrödinger system.The relationship between the Schrödinger system,the associated Bernstein process and the optimal transport was developed by Léonard[32,33](and by Mikami[39]earlier via an h-process).We present Sinkhorn’s algorithm for solving the Schrödinger system and the recent results on its convergence rate.We study the gradient descent algorithm based on the dual optimal question and prove its exponential convergence,whose rate might be independent of the regularization constant.This exposition is motivated by recent applications of optimal transport to different domains such as machine learning,image processing,econometrics,astrophysics etc..展开更多
To solve the constraints of multi-objective optimization of the driver system and high nonlinear problems, according to the relevant dimensions of a car, we build a simulation model with Hybrid Ⅲ 50th dummy driver co...To solve the constraints of multi-objective optimization of the driver system and high nonlinear problems, according to the relevant dimensions of a car, we build a simulation model with Hybrid Ⅲ 50th dummy driver constraint system. The comparison of the driver mechanics index of the experimental data with the simulation data in the frontal crash shows that the accuracy of simulation model meets the requirements. The optimal Latin test design is adopted, and the global sensitivity analysis of the design parameters is carried out based on the Kriging model. The four most sensitive parameters are selected, and the parameters are solved by a multi-island genetic algorithm.And then the nonlinear programming quadratic line(NLPQL) algorithm is used to search for accurate optimization. The optimal parameters of the occupant restraint system are determined: the limiting force value of force limiter 2 985.603 N, belt extension 12.684%, airbag point explosion time 27.585 ms, and airbag vent diameter 27.338 mm, with the weighted injury criterion(WIC) decreased by 12.97%, the head injury decreased by 22.60%, and the chest compression decreased by 7.29%. The results show that the system integration of passive safety devices such as seat belts and airbags can effectively protect the driver.展开更多
This paper proposes the design and a comparative study of two nonlinear systems modeling techniques. These two approaches are developed to address a class of nonlinear systems with time-varying parameter. The first is...This paper proposes the design and a comparative study of two nonlinear systems modeling techniques. These two approaches are developed to address a class of nonlinear systems with time-varying parameter. The first is a Radial Basis Function (RBF) neural networks and the second is a Multi Layer Perceptron (MLP). The MLP model consists of an input layer, an output layer and usually one or more hidden layers. However, training MLP network based on back propagation learning is computationally expensive. In this paper, an RBF network is called. The parameters of the RBF model are optimized by two methods: the Gradient Descent (GD) method and Genetic Algorithms (GA). However, the MLP model is optimized by the Gradient Descent method. The performance of both models are evaluated first by using a numerical simulation and second by handling a chemical process known as the Continuous Stirred Tank Reactor CSTR. It has been shown that in both validation operations the results were successful. The optimized RBF model by Genetic Algorithms gave the best results.展开更多
In the evolving landscape of artificial intelligence and machine learning, the choice of optimization algorithm can significantly impact the success of model training and the accuracy of predictions. This paper embark...In the evolving landscape of artificial intelligence and machine learning, the choice of optimization algorithm can significantly impact the success of model training and the accuracy of predictions. This paper embarks on a rigorous and comprehensive exploration of widely adopted optimization techniques, specifically focusing on their performance when applied to the notoriously challenging Rosenbrock function. As a benchmark problem known for its deceptive curvature and narrow valleys, the Rosenbrock function provides a fertile ground for examining the nuances and intricacies of algorithmic behavior. The study delves into a diverse array of optimization methods, including traditional Gradient Descent, its stochastic variant (SGD), and the more sophisticated Gradient Descent with Momentum. The investigation further extends to adaptive methods like RMSprop, AdaGrad, and the highly regarded Adam optimizer. By meticulously analyzing and visualizing the optimization paths, convergence rates, and gradient norms, this paper uncovers critical insights into the strengths and limitations of each technique. Our findings not only illuminate the intricate dynamics of these algorithms but also offer actionable guidance for their deployment in complex, real-world optimization problems. This comparative analysis promises to intrigue and inspire researchers and practitioners alike, as it reveals the subtle yet profound impacts of algorithmic choices in the quest for optimization excellence.展开更多
Dear Editor,Through distributed machine learning,multi-UAV systems can achieve global optimization goals without a centralized server,such as optimal target tracking,by leveraging local calculation and communication w...Dear Editor,Through distributed machine learning,multi-UAV systems can achieve global optimization goals without a centralized server,such as optimal target tracking,by leveraging local calculation and communication with neighbors.In this work,we implement the stochastic gradient descent algorithm(SGD)distributedly to optimize tracking errors based on local state and aggregation of the neighbors'estimation.However,Byzantine agents can mislead neighbors,causing deviations from optimal tracking.We prove that the swarm achieves resilient convergence if aggregated results lie within the normal neighbors'convex hull,which can be guaranteed by the introduced centerpoint-based aggregation rule.In the given simulated scenarios,distributed learning using average,geometric median(GM),and coordinate-wise median(CM)based aggregation rules fail to track the target.Compared to solely using the centerpoint aggregation method,our approach,which combines a pre-filter with the centroid aggregation rule,significantly enhances resilience against Byzantine attacks,achieving faster convergence and smaller tracking errors.展开更多
基金supported by the National Natural Science Foundation of China (60632050)National Basic Research Program of Jiangsu Province University (08KJB520003)
文摘An improved genetic algorithm（IGA） based on a novel selection strategy to handle nonlinear programming problems is proposed.Each individual in selection process is represented as a three-dimensional feature vector which is composed of objective function value,the degree of constraints violations and the number of constraints violations.It is easy to distinguish excellent individuals from general individuals by using an individuals＇ feature vector.Additionally,a local search（LS） process is incorporated into selection operation so as to find feasible solutions located in the neighboring areas of some infeasible solutions.The combination of IGA and LS should offer the advantage of both the quality of solutions and diversity of solutions.Experimental results over a set of benchmark problems demonstrate that IGA has better performance than other algorithms.
文摘In this paper, we focus on a class of nonlinear bilevel programming problems where the follower’s objective is a function of the linear expression of all variables, and the follower’s constraint functions are convex with respect to the follower’s variables. First, based on the features of the follower’s problem, we give a new decomposition scheme by which the follower’s optimal solution can be obtained easily. Then, to solve efficiently this class of problems by using evolutionary algorithm, novel evolutionary operators are designed by considering the best individuals and the diversity of individuals in the populations. Finally, based on these techniques, a new evolutionary algorithm is proposed. The numerical results on 20 test problems illustrate that the proposed algorithm is efficient and stable.
基金supported by the National Natural Science Foundation of China(No.21365008)the Science Foundation of Guangxi province of China(No.2012GXNSFAA053230)
文摘Refineries often need to find similar crude oil to replace the scarce crude oil for stabilizing the feedstock property. We introduced the method for calculation of crude blended properties firstly, and then created a crude oil selection and blending optimization model based on the data of crude oil property. The model is a mixed-integer nonlinear programming(MINLP) with constraints, and the target is to maximize the similarity between the blended crude oil and the objective crude oil. Furthermore, the model takes into account the selection of crude oils and their blending ratios simultaneously, and transforms the problem of looking for similar crude oil into the crude oil selection and blending optimization problem. We applied the Improved Cuckoo Search(ICS) algorithm to solving the model. Through the simulations, ICS was compared with the genetic algorithm, the particle swarm optimization algorithm and the CPLEX solver. The results show that ICS has very good optimization efficiency. The blending solution can provide a reference for refineries to find the similar crude oil. And the method proposed can also give some references to selection and blending optimization of other materials.
基金supported by the Knut and Alice Wallenberg Foundationthe Swedish Foundation for Strategic Research+1 种基金the Swedish Research Councilthe National Natural Science Foundation of China(62133003,61991403,61991404,61991400)。
文摘The distributed nonconvex optimization problem of minimizing a global cost function formed by a sum of n local cost functions by using local information exchange is considered.This problem is an important component of many machine learning techniques with data parallelism,such as deep learning and federated learning.We propose a distributed primal-dual stochastic gradient descent(SGD)algorithm,suitable for arbitrarily connected communication networks and any smooth(possibly nonconvex)cost functions.We show that the proposed algorithm achieves the linear speedup convergence rate O(1/(√nT))for general nonconvex cost functions and the linear speedup convergence rate O(1/(nT)) when the global cost function satisfies the Polyak-Lojasiewicz(P-L)condition,where T is the total number of iterations.We also show that the output of the proposed algorithm with constant parameters linearly converges to a neighborhood of a global optimum.We demonstrate through numerical experiments the efficiency of our algorithm in comparison with the baseline centralized SGD and recently proposed distributed SGD algorithms.
基金Project supported by the National Natural Science Foundation of China(Grant No.10271073)
文摘In this paper, a new method named as the gradually descent method was proposed to solve the discrete global optimization problem. With the aid of an auxiliary function, this method enables to convert the problem of finding one discrete minimizer of the objective function f to that of finding another at each cycle. The auxiliary function can ensure that a point, except a prescribed point, is not its integer stationary point if the value of objective function at the point is greater than the scalar which is chosen properly. This property leads to a better minimizer of f found more easily by some classical local search methods. The computational results show that this algorithm is quite efficient and reliable for solving nonlinear integer programming problems.
基金the National Special Fund for Agro-scientific Research in the Public Interest(No.201003024)
文摘Based on the improved particle swarm optimization(PSO) algorithm,an optimization approach for the cargo oil tank design(COTD) is presented in this paper.The purpose is to design an optimal overall dimension of the cargo oil tank(COT) under various kinds of constraints in the preliminary design stage.A non-linear programming model is built to simulate the optimization design,in which the requirements and rules for COTD are used as the constraints.Considering the distance between the inner shell and hull,a fuzzy constraint is used to express the feasibility degree of the double-hull configuration.In terms of the characteristic of COTD,the PSO algorithm is improved to solve this problem.A bivariate extremum strategy is presented to deal with the fuzzy constraint,by which the maximum and minimum cargo capacities are obtained simultaneously.Finally,the simulation demonstrates the feasibility and effectiveness of the proposed approach.
基金National Natural Science Foundation of China(62373187)Forward-looking Layout Special Projects(ILA220591A22)。
文摘In the field of calculating the attack area of air-to-air missiles in modern air combat scenarios,the limitations of existing research,including real-time calculation,accuracy efficiency trade-off,and the absence of the three-dimensional attack area model,restrict their practical applications.To address these issues,an improved backtracking algorithm is proposed to improve calculation efficiency.A significant reduction in solution time and maintenance of accuracy in the three-dimensional attack area are achieved by using the proposed algorithm.Furthermore,the age-layered population structure genetic programming(ALPS-GP)algorithm is introduced to determine an analytical polynomial model of the three-dimensional attack area,considering real-time requirements.The accuracy of the polynomial model is enhanced through the coefficient correction using an improved gradient descent algorithm.The study reveals a remarkable combination of high accuracy and efficient real-time computation,with a mean error of 91.89 m using the analytical polynomial model of the three-dimensional attack area solved in just 10^(-4)s,thus meeting the requirements of real-time combat scenarios.
文摘Aim To determine the global optimal solution for a mine ventilation network under given network topology and airway characteristics. Methods\ The genetic algorithm was used to find the global optimal solution of the network. Results\ A modified genetic algorithm is presented with its characteristics and principle. Instead of working on the conventional bit by bit operation, both the crossover and mutation operators are handled in real values by the proposed algorithms. To prevent the system from turning into a premature problem, the elitists from two groups of possible solutions are selected to reproduce the new populations. Conclusion\ The simulation results show that the method outperforms the conventional nonlinear programming approach whether from the viewpoint of the number of iterations required to find the optimum solutions or from the final solutions obtained.
文摘In this exposition paper we present the optimal transport problem of Monge-Ampère-Kantorovitch(MAK in short)and its approximative entropical regularization.Contrary to the MAK optimal transport problem,the solution of the entropical optimal transport problem is always unique,and is characterized by the Schrödinger system.The relationship between the Schrödinger system,the associated Bernstein process and the optimal transport was developed by Léonard[32,33](and by Mikami[39]earlier via an h-process).We present Sinkhorn’s algorithm for solving the Schrödinger system and the recent results on its convergence rate.We study the gradient descent algorithm based on the dual optimal question and prove its exponential convergence,whose rate might be independent of the regularization constant.This exposition is motivated by recent applications of optimal transport to different domains such as machine learning,image processing,econometrics,astrophysics etc..
基金Supported by Natural Science and Technology Research Project of the Jiangxi Education Department(GJJ202002, GJJ2202620)。
文摘To solve the constraints of multi-objective optimization of the driver system and high nonlinear problems, according to the relevant dimensions of a car, we build a simulation model with Hybrid Ⅲ 50th dummy driver constraint system. The comparison of the driver mechanics index of the experimental data with the simulation data in the frontal crash shows that the accuracy of simulation model meets the requirements. The optimal Latin test design is adopted, and the global sensitivity analysis of the design parameters is carried out based on the Kriging model. The four most sensitive parameters are selected, and the parameters are solved by a multi-island genetic algorithm.And then the nonlinear programming quadratic line(NLPQL) algorithm is used to search for accurate optimization. The optimal parameters of the occupant restraint system are determined: the limiting force value of force limiter 2 985.603 N, belt extension 12.684%, airbag point explosion time 27.585 ms, and airbag vent diameter 27.338 mm, with the weighted injury criterion(WIC) decreased by 12.97%, the head injury decreased by 22.60%, and the chest compression decreased by 7.29%. The results show that the system integration of passive safety devices such as seat belts and airbags can effectively protect the driver.
文摘This paper proposes the design and a comparative study of two nonlinear systems modeling techniques. These two approaches are developed to address a class of nonlinear systems with time-varying parameter. The first is a Radial Basis Function (RBF) neural networks and the second is a Multi Layer Perceptron (MLP). The MLP model consists of an input layer, an output layer and usually one or more hidden layers. However, training MLP network based on back propagation learning is computationally expensive. In this paper, an RBF network is called. The parameters of the RBF model are optimized by two methods: the Gradient Descent (GD) method and Genetic Algorithms (GA). However, the MLP model is optimized by the Gradient Descent method. The performance of both models are evaluated first by using a numerical simulation and second by handling a chemical process known as the Continuous Stirred Tank Reactor CSTR. It has been shown that in both validation operations the results were successful. The optimized RBF model by Genetic Algorithms gave the best results.
文摘In the evolving landscape of artificial intelligence and machine learning, the choice of optimization algorithm can significantly impact the success of model training and the accuracy of predictions. This paper embarks on a rigorous and comprehensive exploration of widely adopted optimization techniques, specifically focusing on their performance when applied to the notoriously challenging Rosenbrock function. As a benchmark problem known for its deceptive curvature and narrow valleys, the Rosenbrock function provides a fertile ground for examining the nuances and intricacies of algorithmic behavior. The study delves into a diverse array of optimization methods, including traditional Gradient Descent, its stochastic variant (SGD), and the more sophisticated Gradient Descent with Momentum. The investigation further extends to adaptive methods like RMSprop, AdaGrad, and the highly regarded Adam optimizer. By meticulously analyzing and visualizing the optimization paths, convergence rates, and gradient norms, this paper uncovers critical insights into the strengths and limitations of each technique. Our findings not only illuminate the intricate dynamics of these algorithms but also offer actionable guidance for their deployment in complex, real-world optimization problems. This comparative analysis promises to intrigue and inspire researchers and practitioners alike, as it reveals the subtle yet profound impacts of algorithmic choices in the quest for optimization excellence.
基金supported By Guangdong Major Project of Basic and Applied Basic Research(2023B0303000009)Guangdong Basic and Applied Basic Research Foundation(2024A1515030153,2025A1515011587)+1 种基金Project of Department of Education of Guangdong Province(2023ZDZX4046)Shen-zhen Natural Science Fund(Stable Support Plan Program 20231122121608001),Ningbo Municipal Science and Technology Bureau(ZX2024000604).
文摘Dear Editor,Through distributed machine learning,multi-UAV systems can achieve global optimization goals without a centralized server,such as optimal target tracking,by leveraging local calculation and communication with neighbors.In this work,we implement the stochastic gradient descent algorithm(SGD)distributedly to optimize tracking errors based on local state and aggregation of the neighbors'estimation.However,Byzantine agents can mislead neighbors,causing deviations from optimal tracking.We prove that the swarm achieves resilient convergence if aggregated results lie within the normal neighbors'convex hull,which can be guaranteed by the introduced centerpoint-based aggregation rule.In the given simulated scenarios,distributed learning using average,geometric median(GM),and coordinate-wise median(CM)based aggregation rules fail to track the target.Compared to solely using the centerpoint aggregation method,our approach,which combines a pre-filter with the centroid aggregation rule,significantly enhances resilience against Byzantine attacks,achieving faster convergence and smaller tracking errors.
