Trust-region methods are popular for nonlinear optimization problems. How to determine the predicted reduction of the trust-region subproblem is a key issue for trust-region methods. Powell gave an estimation of the l...Trust-region methods are popular for nonlinear optimization problems. How to determine the predicted reduction of the trust-region subproblem is a key issue for trust-region methods. Powell gave an estimation of the lower bound of the trust-region subproblem by considering the negative gradient direction. In this article, we give an alternate way to estimate the same lower bound of the trust-region subproblem.展开更多
In this paper we present a filter-trust-region algorithm for solving LC1 unconstrained optimization problems which uses the second Dini upper directional derivative. We establish the global convergence of the algorith...In this paper we present a filter-trust-region algorithm for solving LC1 unconstrained optimization problems which uses the second Dini upper directional derivative. We establish the global convergence of the algorithm under reasonable assumptions.展开更多
In this paper, a projected gradient trust region algorithm for solving nonlinear equality systems with convex constraints is considered. The global convergence results are developed in a very general setting of comput...In this paper, a projected gradient trust region algorithm for solving nonlinear equality systems with convex constraints is considered. The global convergence results are developed in a very general setting of computing trial directions by this method combining with the line search technique. Close to the solution set this method is locally Q-superlinearly convergent under an error bound assumption which is much weaker than the standard nonsingularity condition.展开更多
In this paper, we propose a nonmonotone adap-tive trust-region method for solving symmetric nonlinear equations problems. The convergent result of the presented method will be estab-lished under favorable conditions. ...In this paper, we propose a nonmonotone adap-tive trust-region method for solving symmetric nonlinear equations problems. The convergent result of the presented method will be estab-lished under favorable conditions. Numerical results are reported.展开更多
The truncated singular value decomposition has been widely used in many areas of science including engineering,and statistics,etc.In this paper,the original truncated complex singular value decomposition problem is fo...The truncated singular value decomposition has been widely used in many areas of science including engineering,and statistics,etc.In this paper,the original truncated complex singular value decomposition problem is formulated as a Riemannian optimiza-tion problem on a product of two complex Stiefel manifolds,a practical algorithm based on the generic Riemannian trust-region method of Absil et al.is presented to solve the underlying problem,which enjoys the global convergence and local superlinear conver-gence rate.Numerical experiments are provided to illustrate the efficiency of the proposed method.Comparisons with some classical Riemannian gradient-type methods,the existing Riemannian version of limited-memory BFGS algorithms in the MATLAB toolbox Manopt and the Riemannian manifold optimization library ROPTLIB,and some latest infeasible methods for solving manifold optimization problems,are also provided to show the merits of the proposed approach.展开更多
We present a stochastic trust-region model-based framework in which its radius is related to the probabilistic models.Especially,we propose a specific algorithm termed STRME,in which the trust-region radius depends li...We present a stochastic trust-region model-based framework in which its radius is related to the probabilistic models.Especially,we propose a specific algorithm termed STRME,in which the trust-region radius depends linearly on the gradient used to define the latest model.The complexity results of the STRME method in nonconvex,convex and strongly convex settings are presented,which match those of the existing algorithms based on probabilistic properties.In addition,several numerical experiments are carried out to reveal the benefits of the proposed methods compared to the existing stochastic trust-region methods and other relevant stochastic gradient methods.展开更多
This paper presents a new trust-region algorithm for general nonlinear constrained optimization problems. Certain equivalent KKT conditions of the problems are derived. Global convergence of the algorithm to a first-o...This paper presents a new trust-region algorithm for general nonlinear constrained optimization problems. Certain equivalent KKT conditions of the problems are derived. Global convergence of the algorithm to a first-order KKT point is established under mild conditions on the trial steps. Numerical example is also reported.展开更多
Because of its vital role of the trust-region subproblem （TRS） in various applications, for example, in optimization and in ill-posed problems, there are several factorization-free algorithms for solving the large-s...Because of its vital role of the trust-region subproblem （TRS） in various applications, for example, in optimization and in ill-posed problems, there are several factorization-free algorithms for solving the large-scale sparse TRS. The truncated Lanczos approach proposed by N. I. M. Gould, S. Lucidi, M. Roma, and P. L. Toint [SIAM J. Optim., 1999, 9： 504-525] is a natural extension of the classical Lanczos method for the symmetric linear system and eigenvalue problem and, indeed follows the classical Rayleigh-Ritz procedure for eigenvalue computations. It consists of 1） projecting the original TRS to the Krylov subspa^es to yield smaller size TRS＇s and then 2） solving the resulted TRS＇s to get the approximates of the original TRS. This paper presents a posterior error bounds for both the global optimal value and the optimal solution between the original TRS and their projected counterparts. Our error bounds mainly rely on the factors from the Lanczos process as well as the data of the original TRS and, could be helpful in designing certain stopping criteria for the truncated Lanczos approach.展开更多
In this paper, a new trust region algorithm for minimax optimization problems is proposed, which solves only one quadratic subproblem based on a new approximation model at each iteration. The approach is different wit...In this paper, a new trust region algorithm for minimax optimization problems is proposed, which solves only one quadratic subproblem based on a new approximation model at each iteration. The approach is different with the traditional algorithms that usually require to solve two quadratic subproblems. Moreover, to avoid Maratos effect, the nonmonotone strategy is employed. The analysis shows that, under standard conditions, the algorithm has global and superlinear convergence. Preliminary numerical experiments are conducted to show the effiency of the new method.展开更多
We consider the extended trust-region subproblem with two linear inequalities. In the "nonintersecting" case of this problem, Burer and Yang(2015) have proved that its semi-definite programming relaxation wi...We consider the extended trust-region subproblem with two linear inequalities. In the "nonintersecting" case of this problem, Burer and Yang(2015) have proved that its semi-definite programming relaxation with second-order-cone reformulation(SDPR-SOCR) is a tight relaxation. In the more complicated "intersecting" case, which is discussed in this paper, so far there is no result except for a counterexample for the SDPR-SOCR. We present a necessary and sufficient condition for the SDPR-SOCR to be a tight relaxation in both the "nonintersecting" and "intersecting" cases. As an application of this condition, it is verified easily that the "nonintersecting" SDPR-SOCR is a tight relaxation indeed. Furthermore, as another application of the condition, we prove that there exist at least three regions among the four regions in the trust-region ball divided by the two intersecting linear cuts, on which the SDPR-SOCR must be a tight relaxation. Finally, the results of numerical experiments show that the SDPR-SOCR can work efficiently in decreasing or even eliminating the duality gap of the nonconvex extended trust-region subproblem with two intersecting linear inequalities indeed.展开更多
This paper studied subspace properties of the Celis–Dennis–Tapia(CDT)subproblem that arises in some trust-region algorithms for equality constrained optimization.The analysis is an extension of that presented by Wa...This paper studied subspace properties of the Celis–Dennis–Tapia(CDT)subproblem that arises in some trust-region algorithms for equality constrained optimization.The analysis is an extension of that presented by Wang and Yuan(Numer.Math.104:241–269,2006)for the standard trust-region subproblem.Under suitable conditions,it is shown that the trial step obtained from the CDT subproblem is in the subspace spanned by all the gradient vectors of the objective function and of the constraints computed until the current iteration.Based on this observation,a subspace version of the Powell–Yuan trust-region algorithm is proposed for equality constrained optimization problems where the number of constraints is much lower than the number of variables. The convergence analysis is given and numerical results arealso reported.展开更多
A new trust-region and affine scaling algorithm for linearly constrained optimization is presentedin this paper. Under no nondegenerate assumption, we prove that any limit point of the sequence generatedby the new alg...A new trust-region and affine scaling algorithm for linearly constrained optimization is presentedin this paper. Under no nondegenerate assumption, we prove that any limit point of the sequence generatedby the new algorithm satisfies the first order necessary condition and there exists at least one limit point ofthe sequence which satisfies the second order necessary condition. Some preliminary numerical experiments are reported.展开更多
文摘Trust-region methods are popular for nonlinear optimization problems. How to determine the predicted reduction of the trust-region subproblem is a key issue for trust-region methods. Powell gave an estimation of the lower bound of the trust-region subproblem by considering the negative gradient direction. In this article, we give an alternate way to estimate the same lower bound of the trust-region subproblem.
基金Supported by the National Natural Science Foundation of China （10231060）, the Special Research Found of Doctoral Program of Higher Education of China（200d0319003 ）, the Research Project of Xuzhou Institute of Technology（ XKY200622）.
基金Supported by CERG: CityU 101005 of the Government of Hong Kong SAR, Chinathe National Natural ScienceFoundation of China, the Specialized Research Fund of Doctoral Program of Higher Education of China (Grant No.20040319003)the Natural Science Fund of Jiangsu Province of China (Grant No. BK2006214)
文摘In this paper we present a filter-trust-region algorithm for solving LC1 unconstrained optimization problems which uses the second Dini upper directional derivative. We establish the global convergence of the algorithm under reasonable assumptions.
基金Supported by the National Natural Science Foundation of China (10871130)the Research Fund for the Doctoral Program of Higher Education of China (20093127110005)the Scientific Computing Key Laboratory of Shanghai Universities
文摘In this paper, a projected gradient trust region algorithm for solving nonlinear equality systems with convex constraints is considered. The global convergence results are developed in a very general setting of computing trial directions by this method combining with the line search technique. Close to the solution set this method is locally Q-superlinearly convergent under an error bound assumption which is much weaker than the standard nonsingularity condition.
文摘In this paper, we propose a nonmonotone adap-tive trust-region method for solving symmetric nonlinear equations problems. The convergent result of the presented method will be estab-lished under favorable conditions. Numerical results are reported.
基金supported by the National Natural Science Foundation of China(Grant Nos.12261026,11961012,12201149)by the Natural Science Foundation of Guangxi Province(Grant Nos.2016GXNSFAA380074,2023GXNSFAA026067)+4 种基金by the Innovation Project of GUET Graduate Education(Grant No.2022YXW01)by the GUET Graduate Innovation Project(Grant No.2022YCXS142)by the Guangxi Key Laboratory of Automatic Detecting Technology and Instruments(Grant Nos.YQ23103,YQ21103,YQ22106)by the Special Fund for Science and Technological Bases and Talents of Guangxi(Grant No.2021AC06001)by the Guizhou Science and Technology Program of Projects(Grant No.ZK2021G339)。
文摘The truncated singular value decomposition has been widely used in many areas of science including engineering,and statistics,etc.In this paper,the original truncated complex singular value decomposition problem is formulated as a Riemannian optimiza-tion problem on a product of two complex Stiefel manifolds,a practical algorithm based on the generic Riemannian trust-region method of Absil et al.is presented to solve the underlying problem,which enjoys the global convergence and local superlinear conver-gence rate.Numerical experiments are provided to illustrate the efficiency of the proposed method.Comparisons with some classical Riemannian gradient-type methods,the existing Riemannian version of limited-memory BFGS algorithms in the MATLAB toolbox Manopt and the Riemannian manifold optimization library ROPTLIB,and some latest infeasible methods for solving manifold optimization problems,are also provided to show the merits of the proposed approach.
基金This research is partially supported by the National Natural Science Foundation of China 11331012 and 11688101.
文摘We present a stochastic trust-region model-based framework in which its radius is related to the probabilistic models.Especially,we propose a specific algorithm termed STRME,in which the trust-region radius depends linearly on the gradient used to define the latest model.The complexity results of the STRME method in nonconvex,convex and strongly convex settings are presented,which match those of the existing algorithms based on probabilistic properties.In addition,several numerical experiments are carried out to reveal the benefits of the proposed methods compared to the existing stochastic trust-region methods and other relevant stochastic gradient methods.
基金Supported by the Scientific Research Foundation of Hunan Provincial Education Department(02B021) Hunan Provincial Natural Science Foundation,China(03JJY6002)
文摘This paper presents a new trust-region algorithm for general nonlinear constrained optimization problems. Certain equivalent KKT conditions of the problems are derived. Global convergence of the algorithm to a first-order KKT point is established under mild conditions on the trial steps. Numerical example is also reported.
基金The authors would like to thank the anonymous referees for their careful reading and comments. This work of the first author was supported in part by the National Natural Science Foundation of China （Grant Nos. 11671246, 91730303, 11371102） and the work of the second author was supported in part by the National Natural Science Foundation of China （Grant Nos. 91730304, 11371102, 91330201）.
文摘Because of its vital role of the trust-region subproblem （TRS） in various applications, for example, in optimization and in ill-posed problems, there are several factorization-free algorithms for solving the large-scale sparse TRS. The truncated Lanczos approach proposed by N. I. M. Gould, S. Lucidi, M. Roma, and P. L. Toint [SIAM J. Optim., 1999, 9： 504-525] is a natural extension of the classical Lanczos method for the symmetric linear system and eigenvalue problem and, indeed follows the classical Rayleigh-Ritz procedure for eigenvalue computations. It consists of 1） projecting the original TRS to the Krylov subspa^es to yield smaller size TRS＇s and then 2） solving the resulted TRS＇s to get the approximates of the original TRS. This paper presents a posterior error bounds for both the global optimal value and the optimal solution between the original TRS and their projected counterparts. Our error bounds mainly rely on the factors from the Lanczos process as well as the data of the original TRS and, could be helpful in designing certain stopping criteria for the truncated Lanczos approach.
文摘In this paper, a new trust region algorithm for minimax optimization problems is proposed, which solves only one quadratic subproblem based on a new approximation model at each iteration. The approach is different with the traditional algorithms that usually require to solve two quadratic subproblems. Moreover, to avoid Maratos effect, the nonmonotone strategy is employed. The analysis shows that, under standard conditions, the algorithm has global and superlinear convergence. Preliminary numerical experiments are conducted to show the effiency of the new method.
基金supported by National Natural Science Foundation of China(Grant Nos.11471052,11171040,11001030 and 61375066)the Grant of China Scholarship Council
文摘We consider the extended trust-region subproblem with two linear inequalities. In the "nonintersecting" case of this problem, Burer and Yang(2015) have proved that its semi-definite programming relaxation with second-order-cone reformulation(SDPR-SOCR) is a tight relaxation. In the more complicated "intersecting" case, which is discussed in this paper, so far there is no result except for a counterexample for the SDPR-SOCR. We present a necessary and sufficient condition for the SDPR-SOCR to be a tight relaxation in both the "nonintersecting" and "intersecting" cases. As an application of this condition, it is verified easily that the "nonintersecting" SDPR-SOCR is a tight relaxation indeed. Furthermore, as another application of the condition, we prove that there exist at least three regions among the four regions in the trust-region ball divided by the two intersecting linear cuts, on which the SDPR-SOCR must be a tight relaxation. Finally, the results of numerical experiments show that the SDPR-SOCR can work efficiently in decreasing or even eliminating the duality gap of the nonconvex extended trust-region subproblem with two intersecting linear inequalities indeed.
基金G.N.Grapiglia was supported by Coordination for the Improvement of Higher Education Personnel(CAPES),Brazil(Grant PGCI No.12347/12-4).J.Yuan was partially supported by Coordination for the Improvement of Higher Education Personnel(CAPES)and by the National Council for Scientific and Technological Development(CNPq),Brazil.Y.-x.Yuan was partially supported by Natural Science Foundation of China,China(Grant No.11331012)This work was carried out while the first author was visiting Institute of Computational Mathematics and Scientific/Engineering Computing of the Chinese Academy of Sciences.He would like to thank Professor Ya-xiang Yuan,Professor Yu-hong Dai,Dr.Xin Liu and Dr.Ya-feng Liu for their warm hospitality.The authors also are grateful to Dr.Wei Leng for his help in installing and configuring the CUTEr.Finally,the authors would like to thank the two referees for their helpful comments.
文摘This paper studied subspace properties of the Celis–Dennis–Tapia(CDT)subproblem that arises in some trust-region algorithms for equality constrained optimization.The analysis is an extension of that presented by Wang and Yuan(Numer.Math.104:241–269,2006)for the standard trust-region subproblem.Under suitable conditions,it is shown that the trial step obtained from the CDT subproblem is in the subspace spanned by all the gradient vectors of the objective function and of the constraints computed until the current iteration.Based on this observation,a subspace version of the Powell–Yuan trust-region algorithm is proposed for equality constrained optimization problems where the number of constraints is much lower than the number of variables. The convergence analysis is given and numerical results arealso reported.
基金This work was supported by the National Natural Science Foundation of China(Crant No.39830070).
文摘A new trust-region and affine scaling algorithm for linearly constrained optimization is presentedin this paper. Under no nondegenerate assumption, we prove that any limit point of the sequence generatedby the new algorithm satisfies the first order necessary condition and there exists at least one limit point ofthe sequence which satisfies the second order necessary condition. Some preliminary numerical experiments are reported.
