Trust-region methods are popular for nonlinear optimization problems. How to determine the predicted reduction of the trust-region subproblem is a key issue for trust-region methods. Powell gave an estimation of the l...Trust-region methods are popular for nonlinear optimization problems. How to determine the predicted reduction of the trust-region subproblem is a key issue for trust-region methods. Powell gave an estimation of the lower bound of the trust-region subproblem by considering the negative gradient direction. In this article, we give an alternate way to estimate the same lower bound of the trust-region subproblem.展开更多
In this paper we present a filter-trust-region algorithm for solving LC1 unconstrained optimization problems which uses the second Dini upper directional derivative. We establish the global convergence of the algorith...In this paper we present a filter-trust-region algorithm for solving LC1 unconstrained optimization problems which uses the second Dini upper directional derivative. We establish the global convergence of the algorithm under reasonable assumptions.展开更多
In this paper, a projected gradient trust region algorithm for solving nonlinear equality systems with convex constraints is considered. The global convergence results are developed in a very general setting of comput...In this paper, a projected gradient trust region algorithm for solving nonlinear equality systems with convex constraints is considered. The global convergence results are developed in a very general setting of computing trial directions by this method combining with the line search technique. Close to the solution set this method is locally Q-superlinearly convergent under an error bound assumption which is much weaker than the standard nonsingularity condition.展开更多
In this paper, we propose a nonmonotone adap-tive trust-region method for solving symmetric nonlinear equations problems. The convergent result of the presented method will be estab-lished under favorable conditions. ...In this paper, we propose a nonmonotone adap-tive trust-region method for solving symmetric nonlinear equations problems. The convergent result of the presented method will be estab-lished under favorable conditions. Numerical results are reported.展开更多
We present a stochastic trust-region model-based framework in which its radius is related to the probabilistic models.Especially,we propose a specific algorithm termed STRME,in which the trust-region radius depends li...We present a stochastic trust-region model-based framework in which its radius is related to the probabilistic models.Especially,we propose a specific algorithm termed STRME,in which the trust-region radius depends linearly on the gradient used to define the latest model.The complexity results of the STRME method in nonconvex,convex and strongly convex settings are presented,which match those of the existing algorithms based on probabilistic properties.In addition,several numerical experiments are carried out to reveal the benefits of the proposed methods compared to the existing stochastic trust-region methods and other relevant stochastic gradient methods.展开更多
This paper presents a new trust-region algorithm for general nonlinear constrained optimization problems. Certain equivalent KKT conditions of the problems are derived. Global convergence of the algorithm to a first-o...This paper presents a new trust-region algorithm for general nonlinear constrained optimization problems. Certain equivalent KKT conditions of the problems are derived. Global convergence of the algorithm to a first-order KKT point is established under mild conditions on the trial steps. Numerical example is also reported.展开更多
Because of its vital role of the trust-region subproblem （TRS） in various applications, for example, in optimization and in ill-posed problems, there are several factorization-free algorithms for solving the large-s...Because of its vital role of the trust-region subproblem （TRS） in various applications, for example, in optimization and in ill-posed problems, there are several factorization-free algorithms for solving the large-scale sparse TRS. The truncated Lanczos approach proposed by N. I. M. Gould, S. Lucidi, M. Roma, and P. L. Toint [SIAM J. Optim., 1999, 9： 504-525] is a natural extension of the classical Lanczos method for the symmetric linear system and eigenvalue problem and, indeed follows the classical Rayleigh-Ritz procedure for eigenvalue computations. It consists of 1） projecting the original TRS to the Krylov subspa^es to yield smaller size TRS＇s and then 2） solving the resulted TRS＇s to get the approximates of the original TRS. This paper presents a posterior error bounds for both the global optimal value and the optimal solution between the original TRS and their projected counterparts. Our error bounds mainly rely on the factors from the Lanczos process as well as the data of the original TRS and, could be helpful in designing certain stopping criteria for the truncated Lanczos approach.展开更多
In this paper, a new trust region algorithm for minimax optimization problems is proposed, which solves only one quadratic subproblem based on a new approximation model at each iteration. The approach is different wit...In this paper, a new trust region algorithm for minimax optimization problems is proposed, which solves only one quadratic subproblem based on a new approximation model at each iteration. The approach is different with the traditional algorithms that usually require to solve two quadratic subproblems. Moreover, to avoid Maratos effect, the nonmonotone strategy is employed. The analysis shows that, under standard conditions, the algorithm has global and superlinear convergence. Preliminary numerical experiments are conducted to show the effiency of the new method.展开更多
We consider the extended trust-region subproblem with two linear inequalities. In the "nonintersecting" case of this problem, Burer and Yang(2015) have proved that its semi-definite programming relaxation wi...We consider the extended trust-region subproblem with two linear inequalities. In the "nonintersecting" case of this problem, Burer and Yang(2015) have proved that its semi-definite programming relaxation with second-order-cone reformulation(SDPR-SOCR) is a tight relaxation. In the more complicated "intersecting" case, which is discussed in this paper, so far there is no result except for a counterexample for the SDPR-SOCR. We present a necessary and sufficient condition for the SDPR-SOCR to be a tight relaxation in both the "nonintersecting" and "intersecting" cases. As an application of this condition, it is verified easily that the "nonintersecting" SDPR-SOCR is a tight relaxation indeed. Furthermore, as another application of the condition, we prove that there exist at least three regions among the four regions in the trust-region ball divided by the two intersecting linear cuts, on which the SDPR-SOCR must be a tight relaxation. Finally, the results of numerical experiments show that the SDPR-SOCR can work efficiently in decreasing or even eliminating the duality gap of the nonconvex extended trust-region subproblem with two intersecting linear inequalities indeed.展开更多
This paper studied subspace properties of the Celis–Dennis–Tapia(CDT)subproblem that arises in some trust-region algorithms for equality constrained optimization.The analysis is an extension of that presented by Wa...This paper studied subspace properties of the Celis–Dennis–Tapia(CDT)subproblem that arises in some trust-region algorithms for equality constrained optimization.The analysis is an extension of that presented by Wang and Yuan(Numer.Math.104:241–269,2006)for the standard trust-region subproblem.Under suitable conditions,it is shown that the trial step obtained from the CDT subproblem is in the subspace spanned by all the gradient vectors of the objective function and of the constraints computed until the current iteration.Based on this observation,a subspace version of the Powell–Yuan trust-region algorithm is proposed for equality constrained optimization problems where the number of constraints is much lower than the number of variables. The convergence analysis is given and numerical results arealso reported.展开更多
A new trust-region and affine scaling algorithm for linearly constrained optimization is presentedin this paper. Under no nondegenerate assumption, we prove that any limit point of the sequence generatedby the new alg...A new trust-region and affine scaling algorithm for linearly constrained optimization is presentedin this paper. Under no nondegenerate assumption, we prove that any limit point of the sequence generatedby the new algorithm satisfies the first order necessary condition and there exists at least one limit point ofthe sequence which satisfies the second order necessary condition. Some preliminary numerical experiments are reported.展开更多
This paper presents a new trust-region algorithm for n-dimension nonlinear optimization subject to m nonlinear inequality constraints. Equivalent KKT conditions are derived, which is the basis for constructing the new...This paper presents a new trust-region algorithm for n-dimension nonlinear optimization subject to m nonlinear inequality constraints. Equivalent KKT conditions are derived, which is the basis for constructing the new algorithm. Global convergence of the algorithm to a first-order KKT point is established under mild conditions on the trial steps, local quadratic convergence theorem is proved for nondegenerate minimizer point. Numerical experiment is presented to show the effectiveness of our approach.展开更多
In this article,we consider the primal-dual path-following method and the trust-region updating strategy for the standard linear programming problem.For the rank-deficient problem with the small noisy data,we also giv...In this article,we consider the primal-dual path-following method and the trust-region updating strategy for the standard linear programming problem.For the rank-deficient problem with the small noisy data,we also give the preprocessing method based on the QR decomposition with column pivoting.Then,we prove the global convergence of the new method when the initial point is strictly primal-dual feasible.Finally,for some rankdeficient problems with or without the small noisy data from the NETLIB collection,we compare it with other two popular interior-point methods,i.e.the subroutine pathfollow.m and the built-in subroutine linprog.m of the MATLAB environment.Numerical results show that the new method is more robust than the other two methods for the rank-deficient problem with the small noise data.展开更多
In this paper,we present a new adaptive trust-region method for solving nonlinear unconstrained optimization problems.More precisely,a trust-region radius based on a nonmonotone technique uses an approximation of Hes...In this paper,we present a new adaptive trust-region method for solving nonlinear unconstrained optimization problems.More precisely,a trust-region radius based on a nonmonotone technique uses an approximation of Hessian which is adaptively chosen.We produce a suitable trust-region radius;preserve the global convergence under classical assumptions to the first-order critical points;improve the practical performance of the new algorithm compared to other exiting variants.Moreover,the quadratic convergence rate is established under suitable conditions.Computational results on the CUTEst test collection of unconstrained problems are presented to show the effectiveness of the proposed algorithm compared with some exiting methods.展开更多
In this paper, we propose an algorithm for solving inequality constrained mini-max optimization problem. In this algorithm, an active set strategy is used together with mul- tiplier method to convert the inequality co...In this paper, we propose an algorithm for solving inequality constrained mini-max optimization problem. In this algorithm, an active set strategy is used together with mul- tiplier method to convert the inequality constrained mini-max optimization problem into unconstrained optimization problem. A trust-region method is a well-accepted technique in constrained optimization to assure global convergence and is more robust when they deal with rounding errors. One of the advantages of trust-region method is that it does not require the objective function of the model to be convex. A global convergence analysis for the proposed algorithm is presented under some conditions. To show the efficiency of the algorithm numerical results for a number of test problems are reported.展开更多
The speeding-up and slowing-down(SUSD)direction is a novel direction,which is proved to converge to the gradient descent direction under some conditions.The authors propose the derivative-free optimization algorithm S...The speeding-up and slowing-down(SUSD)direction is a novel direction,which is proved to converge to the gradient descent direction under some conditions.The authors propose the derivative-free optimization algorithm SUSD-TR,which combines the SUSD direction based on the covariance matrix of interpolation points and the solution of the trust-region subproblem of the interpolation model function at the current iteration step.They analyze the optimization dynamics and convergence of the algorithm SUSD-TR.Details of the trial step and structure step are given.Numerical results show their algorithm’s efficiency,and the comparison indicates that SUSD-TR greatly improves the method’s performance based on the method that only goes along the SUSD direction.Their algorithm is competitive with state-of-the-art mathematical derivative-free optimization algorithms.展开更多
文摘Trust-region methods are popular for nonlinear optimization problems. How to determine the predicted reduction of the trust-region subproblem is a key issue for trust-region methods. Powell gave an estimation of the lower bound of the trust-region subproblem by considering the negative gradient direction. In this article, we give an alternate way to estimate the same lower bound of the trust-region subproblem.
基金Supported by the National Natural Science Foundation of China （10231060）, the Special Research Found of Doctoral Program of Higher Education of China（200d0319003 ）, the Research Project of Xuzhou Institute of Technology（ XKY200622）.
基金Supported by CERG: CityU 101005 of the Government of Hong Kong SAR, Chinathe National Natural ScienceFoundation of China, the Specialized Research Fund of Doctoral Program of Higher Education of China (Grant No.20040319003)the Natural Science Fund of Jiangsu Province of China (Grant No. BK2006214)
文摘In this paper we present a filter-trust-region algorithm for solving LC1 unconstrained optimization problems which uses the second Dini upper directional derivative. We establish the global convergence of the algorithm under reasonable assumptions.
基金Supported by the National Natural Science Foundation of China (10871130)the Research Fund for the Doctoral Program of Higher Education of China (20093127110005)the Scientific Computing Key Laboratory of Shanghai Universities
文摘In this paper, a projected gradient trust region algorithm for solving nonlinear equality systems with convex constraints is considered. The global convergence results are developed in a very general setting of computing trial directions by this method combining with the line search technique. Close to the solution set this method is locally Q-superlinearly convergent under an error bound assumption which is much weaker than the standard nonsingularity condition.
文摘In this paper, we propose a nonmonotone adap-tive trust-region method for solving symmetric nonlinear equations problems. The convergent result of the presented method will be estab-lished under favorable conditions. Numerical results are reported.
基金This research is partially supported by the National Natural Science Foundation of China 11331012 and 11688101.
文摘We present a stochastic trust-region model-based framework in which its radius is related to the probabilistic models.Especially,we propose a specific algorithm termed STRME,in which the trust-region radius depends linearly on the gradient used to define the latest model.The complexity results of the STRME method in nonconvex,convex and strongly convex settings are presented,which match those of the existing algorithms based on probabilistic properties.In addition,several numerical experiments are carried out to reveal the benefits of the proposed methods compared to the existing stochastic trust-region methods and other relevant stochastic gradient methods.
基金Supported by the Scientific Research Foundation of Hunan Provincial Education Department(02B021) Hunan Provincial Natural Science Foundation,China(03JJY6002)
文摘This paper presents a new trust-region algorithm for general nonlinear constrained optimization problems. Certain equivalent KKT conditions of the problems are derived. Global convergence of the algorithm to a first-order KKT point is established under mild conditions on the trial steps. Numerical example is also reported.
基金The authors would like to thank the anonymous referees for their careful reading and comments. This work of the first author was supported in part by the National Natural Science Foundation of China （Grant Nos. 11671246, 91730303, 11371102） and the work of the second author was supported in part by the National Natural Science Foundation of China （Grant Nos. 91730304, 11371102, 91330201）.
文摘Because of its vital role of the trust-region subproblem （TRS） in various applications, for example, in optimization and in ill-posed problems, there are several factorization-free algorithms for solving the large-scale sparse TRS. The truncated Lanczos approach proposed by N. I. M. Gould, S. Lucidi, M. Roma, and P. L. Toint [SIAM J. Optim., 1999, 9： 504-525] is a natural extension of the classical Lanczos method for the symmetric linear system and eigenvalue problem and, indeed follows the classical Rayleigh-Ritz procedure for eigenvalue computations. It consists of 1） projecting the original TRS to the Krylov subspa^es to yield smaller size TRS＇s and then 2） solving the resulted TRS＇s to get the approximates of the original TRS. This paper presents a posterior error bounds for both the global optimal value and the optimal solution between the original TRS and their projected counterparts. Our error bounds mainly rely on the factors from the Lanczos process as well as the data of the original TRS and, could be helpful in designing certain stopping criteria for the truncated Lanczos approach.
文摘In this paper, a new trust region algorithm for minimax optimization problems is proposed, which solves only one quadratic subproblem based on a new approximation model at each iteration. The approach is different with the traditional algorithms that usually require to solve two quadratic subproblems. Moreover, to avoid Maratos effect, the nonmonotone strategy is employed. The analysis shows that, under standard conditions, the algorithm has global and superlinear convergence. Preliminary numerical experiments are conducted to show the effiency of the new method.
基金supported by National Natural Science Foundation of China(Grant Nos.11471052,11171040,11001030 and 61375066)the Grant of China Scholarship Council
文摘We consider the extended trust-region subproblem with two linear inequalities. In the "nonintersecting" case of this problem, Burer and Yang(2015) have proved that its semi-definite programming relaxation with second-order-cone reformulation(SDPR-SOCR) is a tight relaxation. In the more complicated "intersecting" case, which is discussed in this paper, so far there is no result except for a counterexample for the SDPR-SOCR. We present a necessary and sufficient condition for the SDPR-SOCR to be a tight relaxation in both the "nonintersecting" and "intersecting" cases. As an application of this condition, it is verified easily that the "nonintersecting" SDPR-SOCR is a tight relaxation indeed. Furthermore, as another application of the condition, we prove that there exist at least three regions among the four regions in the trust-region ball divided by the two intersecting linear cuts, on which the SDPR-SOCR must be a tight relaxation. Finally, the results of numerical experiments show that the SDPR-SOCR can work efficiently in decreasing or even eliminating the duality gap of the nonconvex extended trust-region subproblem with two intersecting linear inequalities indeed.
基金G.N.Grapiglia was supported by Coordination for the Improvement of Higher Education Personnel(CAPES),Brazil(Grant PGCI No.12347/12-4).J.Yuan was partially supported by Coordination for the Improvement of Higher Education Personnel(CAPES)and by the National Council for Scientific and Technological Development(CNPq),Brazil.Y.-x.Yuan was partially supported by Natural Science Foundation of China,China(Grant No.11331012)This work was carried out while the first author was visiting Institute of Computational Mathematics and Scientific/Engineering Computing of the Chinese Academy of Sciences.He would like to thank Professor Ya-xiang Yuan,Professor Yu-hong Dai,Dr.Xin Liu and Dr.Ya-feng Liu for their warm hospitality.The authors also are grateful to Dr.Wei Leng for his help in installing and configuring the CUTEr.Finally,the authors would like to thank the two referees for their helpful comments.
文摘This paper studied subspace properties of the Celis–Dennis–Tapia(CDT)subproblem that arises in some trust-region algorithms for equality constrained optimization.The analysis is an extension of that presented by Wang and Yuan(Numer.Math.104:241–269,2006)for the standard trust-region subproblem.Under suitable conditions,it is shown that the trial step obtained from the CDT subproblem is in the subspace spanned by all the gradient vectors of the objective function and of the constraints computed until the current iteration.Based on this observation,a subspace version of the Powell–Yuan trust-region algorithm is proposed for equality constrained optimization problems where the number of constraints is much lower than the number of variables. The convergence analysis is given and numerical results arealso reported.
基金This work was supported by the National Natural Science Foundation of China(Crant No.39830070).
文摘A new trust-region and affine scaling algorithm for linearly constrained optimization is presentedin this paper. Under no nondegenerate assumption, we prove that any limit point of the sequence generatedby the new algorithm satisfies the first order necessary condition and there exists at least one limit point ofthe sequence which satisfies the second order necessary condition. Some preliminary numerical experiments are reported.
基金the Scientific Research Foundation of Hunan Provincial Education Department, No. 02B021, and the National Natural Science Foundation of China, No. 10171008.
文摘This paper presents a new trust-region algorithm for n-dimension nonlinear optimization subject to m nonlinear inequality constraints. Equivalent KKT conditions are derived, which is the basis for constructing the new algorithm. Global convergence of the algorithm to a first-order KKT point is established under mild conditions on the trial steps, local quadratic convergence theorem is proved for nondegenerate minimizer point. Numerical experiment is presented to show the effectiveness of our approach.
基金This work was supported in part by Grant 61876199 from National Natural Science Foundation of China,Grant YBWL2011085 from Huawei Technologies Co.,Ltd.,and Grant YJCB2011003HIInnovation Research Program of Huawei Technologies Co.,Ltd..The first author is grateful to professor Li-Zhi Liao for introducing him the interiorpoint methods when he visited Hong Kong Baptist University in July,2012.
文摘In this article,we consider the primal-dual path-following method and the trust-region updating strategy for the standard linear programming problem.For the rank-deficient problem with the small noisy data,we also give the preprocessing method based on the QR decomposition with column pivoting.Then,we prove the global convergence of the new method when the initial point is strictly primal-dual feasible.Finally,for some rankdeficient problems with or without the small noisy data from the NETLIB collection,we compare it with other two popular interior-point methods,i.e.the subroutine pathfollow.m and the built-in subroutine linprog.m of the MATLAB environment.Numerical results show that the new method is more robust than the other two methods for the rank-deficient problem with the small noise data.
文摘In this paper,we present a new adaptive trust-region method for solving nonlinear unconstrained optimization problems.More precisely,a trust-region radius based on a nonmonotone technique uses an approximation of Hessian which is adaptively chosen.We produce a suitable trust-region radius;preserve the global convergence under classical assumptions to the first-order critical points;improve the practical performance of the new algorithm compared to other exiting variants.Moreover,the quadratic convergence rate is established under suitable conditions.Computational results on the CUTEst test collection of unconstrained problems are presented to show the effectiveness of the proposed algorithm compared with some exiting methods.
文摘In this paper, we propose an algorithm for solving inequality constrained mini-max optimization problem. In this algorithm, an active set strategy is used together with mul- tiplier method to convert the inequality constrained mini-max optimization problem into unconstrained optimization problem. A trust-region method is a well-accepted technique in constrained optimization to assure global convergence and is more robust when they deal with rounding errors. One of the advantages of trust-region method is that it does not require the objective function of the model to be convex. A global convergence analysis for the proposed algorithm is presented under some conditions. To show the efficiency of the algorithm numerical results for a number of test problems are reported.
基金supported by the National Natural Science Foundation of China(No.12288201)。
文摘The speeding-up and slowing-down(SUSD)direction is a novel direction,which is proved to converge to the gradient descent direction under some conditions.The authors propose the derivative-free optimization algorithm SUSD-TR,which combines the SUSD direction based on the covariance matrix of interpolation points and the solution of the trust-region subproblem of the interpolation model function at the current iteration step.They analyze the optimization dynamics and convergence of the algorithm SUSD-TR.Details of the trial step and structure step are given.Numerical results show their algorithm’s efficiency,and the comparison indicates that SUSD-TR greatly improves the method’s performance based on the method that only goes along the SUSD direction.Their algorithm is competitive with state-of-the-art mathematical derivative-free optimization algorithms.
