Recently, Gilbert and Nocedal([3]) investigated global convergence of conjugate gradient methods related to Polak-Ribiere formular, they restricted beta(k) to non-negative value. [5] discussed the same problem as that...Recently, Gilbert and Nocedal([3]) investigated global convergence of conjugate gradient methods related to Polak-Ribiere formular, they restricted beta(k) to non-negative value. [5] discussed the same problem as that in [3] and relaxed beta(k) to be negative with the objective function being convex. This paper allows beta(k) to be selected in a wider range than [5]. Especially, the global convergence of the corresponding algorithm without sufficient decrease condition is proved.展开更多
Nonlinear conjugate gradient methods have played an important role in solving large scale unconstrained optimi-zation problems,it is characterized by the simplicity of their iteration and their low memory requirements...Nonlinear conjugate gradient methods have played an important role in solving large scale unconstrained optimi-zation problems,it is characterized by the simplicity of their iteration and their low memory requirements.It is well-known that the direction generated by a conjugate gradient method may be not a descent direction.In this paper,a new class of nonlinear conjugate gradient method is presented,its search direction is a descent direction for the objective function.If the objective function is differentiable and its gradient is Lipschitz continuous,the line sbarch satisfies strong Wolfe condition,the global convergence result is established.展开更多
In this paper, a new steplength formula is proposed for unconstrained optimization,which can determine the step-size only by one step and avoids the line search step. Global convergence of the five well-known conjugat...In this paper, a new steplength formula is proposed for unconstrained optimization,which can determine the step-size only by one step and avoids the line search step. Global convergence of the five well-known conjugate gradient methods with this formula is analyzed,and the corresponding results are as follows：（1） The DY method globally converges for a strongly convex LC^1 objective function;（2） The CD method, the FR method, the PRP method and the LS method globally converge for a general, not necessarily convex, LC^1 objective function.展开更多
This paper discusses the global convergence of a class of nonmonotone conjugate gra- dient methods(NM methods) for nonconvex object functions.This class of methods includes the nonmonotone counterpart of modified Po...This paper discusses the global convergence of a class of nonmonotone conjugate gra- dient methods(NM methods) for nonconvex object functions.This class of methods includes the nonmonotone counterpart of modified Polak- Ribière method and modified Hestenes- Stiefel method as special cases展开更多
Nonlinear conjugate gradient methods have played an important role in solving large scale uncon-srined optimization problems,it is careterired by the simplicity of their iteration and their low memory requirements.It ...Nonlinear conjugate gradient methods have played an important role in solving large scale uncon-srined optimization problems,it is careterired by the simplicity of their iteration and their low memory requirements.It is wll-known that the dreetion generted by a cojugate gradient method may be not a de-scent direction.In this paper,a new class of nonlinear conjugate gradient method is presented,its serch di-rection is a descent direction for the oiective funetion.If the obiective function is dfrentiable and its gradi ent is Lipschitz continuous,the line search satisfies strong Wolle condition,the global convergence result is established.展开更多
In this paper we test different conjugate gradient (CG) methods for solving large-scale unconstrained optimization problems. The methods are divided in two groups: the first group includes five basic CG methods and th...In this paper we test different conjugate gradient (CG) methods for solving large-scale unconstrained optimization problems. The methods are divided in two groups: the first group includes five basic CG methods and the second five hybrid CG methods. A collection of medium-scale and large-scale test problems are drawn from a standard code of test problems, CUTE. The conjugate gradient methods are ranked according to the numerical results. Some remarks are given.展开更多
We consider solving integral equations of the second kind defined on the half-line [0, infinity) by the preconditioned conjugate gradient method. Convergence is known to be slow due to the non-compactness of the assoc...We consider solving integral equations of the second kind defined on the half-line [0, infinity) by the preconditioned conjugate gradient method. Convergence is known to be slow due to the non-compactness of the associated integral operator. In this paper, we construct two different circulant integral operators to be used as preconditioners for the method to speed up its convergence rate. We prove that if the given integral operator is close to a convolution-type integral operator, then the preconditioned systems will have spectrum clustered around 1 and hence the preconditioned conjugate gradient method will converge superlinearly. Numerical examples are given to illustrate the fast convergence.展开更多
In this paper,we propose a three-term conjugate gradient method for solving unconstrained optimization problems based on the Hestenes-Stiefel(HS)conjugate gradient method and Polak-Ribiere-Polyak(PRP)conjugate gradien...In this paper,we propose a three-term conjugate gradient method for solving unconstrained optimization problems based on the Hestenes-Stiefel(HS)conjugate gradient method and Polak-Ribiere-Polyak(PRP)conjugate gradient method.Under the condition of standard Wolfe line search,the proposed search direction is the descent direction.For general nonlinear functions,the method is globally convergent.Finally,numerical results show that the proposed method is efficient.展开更多
The restrictively preconditioned conjugate gradient （RPCG） method is further developed to solve large sparse system of linear equations of a block two-by-two structure. The basic idea of this new approach is that we...The restrictively preconditioned conjugate gradient （RPCG） method is further developed to solve large sparse system of linear equations of a block two-by-two structure. The basic idea of this new approach is that we apply the RPCG method to the normal-residual equation of the block two-by-two linear system and construct each required approximate matrix by making use of the incomplete orthogonal factorization of the involved matrix blocks. Numerical experiments show that the new method, called the restrictively preconditioned conjugate gradient on normal residual （RPCGNR）, is more robust and effective than either the known RPCG method or the standard conjugate gradient on normal residual （CGNR） method when being used for solving the large sparse saddle point problems.展开更多
Two Armijo-type line searches are proposed in this paper for nonlinear conjugate gradient methods. Under these line searches, global convergence results are established for several famous conjugate gradient methods, i...Two Armijo-type line searches are proposed in this paper for nonlinear conjugate gradient methods. Under these line searches, global convergence results are established for several famous conjugate gradient methods, including the Fletcher-Reeves method, the Polak-Ribiere-Polyak method, and the conjugate descent method.展开更多
Two fundamental convergence theorems are given for nonlinear conjugate gradient methods only under the descent condition. As a result, methods related to the Fletcher-Reeves algorithm still converge for parameters in ...Two fundamental convergence theorems are given for nonlinear conjugate gradient methods only under the descent condition. As a result, methods related to the Fletcher-Reeves algorithm still converge for parameters in a slightly wider range, in particular, for a parameter in its upper bound. For methods related to the Polak-Ribiere algorithm, it is shown that some negative values of the conjugate parameter do not prevent convergence. If the objective function is convex, some convergence results hold for the Hestenes-Stiefel algorithm.展开更多
We study the conjugate gradient method for solving a system of linear equations with coefficients which are measurable functions and establish the rate of convergence of this method.
With the development of computational power, there has been an increased focus on data-fitting related seismic inversion techniques for high fidelity seismic velocity model and image, such as full-waveform inversion a...With the development of computational power, there has been an increased focus on data-fitting related seismic inversion techniques for high fidelity seismic velocity model and image, such as full-waveform inversion and least squares migration. However, though more advanced than conventional methods, these data fitting methods can be very expensive in terms of computational cost. Recently, various techniques to optimize these data-fitting seismic inversion problems have been implemented to cater for the industrial need for much improved efficiency. In this study, we propose a general stochastic conjugate gradient method for these data-fitting related inverse problems. We first prescribe the basic theory of our method and then give synthetic examples. Our numerical experiments illustrate the potential of this method for large-size seismic inversion application.展开更多
A hybridization of the three–term conjugate gradient method proposed by Zhang et al. and the nonlinear conjugate gradient method proposed by Polak and Ribi`ere, and Polyak is suggested. Based on an eigenvalue analysi...A hybridization of the three–term conjugate gradient method proposed by Zhang et al. and the nonlinear conjugate gradient method proposed by Polak and Ribi`ere, and Polyak is suggested. Based on an eigenvalue analysis, it is shown that search directions of the proposed method satisfy the sufficient descent condition, independent of the line search and the objective function convexity. Global convergence of the method is established under an Armijo–type line search condition. Numerical experiments show practical efficiency of the proposed method.展开更多
Fast solving large-scale linear equations in the finite element analysis is a classical subject in computational mechanics. It is a key technique in computer aided engineering （CAE） and computer aided manufacturing ...Fast solving large-scale linear equations in the finite element analysis is a classical subject in computational mechanics. It is a key technique in computer aided engineering （CAE） and computer aided manufacturing （CAM）. This paper presents a high-efficiency improved symmetric successive over-relaxation （ISSOR） preconditioned conjugate gradient （PCG） method, which maintains lelism consistent with the original form. Ideally, the by 50% as compared with the original algorithm. the convergence and inherent paralcomputation can It is suitable for be reduced nearly high-performance computing with its inherent basic high-efficiency operations. By comparing with the numerical results, it is shown that the proposed method has the best performance.展开更多
In this paper,an efficient conjugate gradient method is given to solve the general unconstrained optimization problems,which can guarantee the sufficient descent property and the global convergence with the strong Wol...In this paper,an efficient conjugate gradient method is given to solve the general unconstrained optimization problems,which can guarantee the sufficient descent property and the global convergence with the strong Wolfe line search conditions.Numerical results show that the new method is efficient and stationary by comparing with PRP＋ method,so it can be widely used in scientific computation.展开更多
Conjugate gradient optimization algorithms depend on the search directions with different choices for the parameters in the search directions. In this note, by combining the nice numerical performance of PR and HS met...Conjugate gradient optimization algorithms depend on the search directions with different choices for the parameters in the search directions. In this note, by combining the nice numerical performance of PR and HS methods with the global convergence property of the class of conjugate gradient methods presented by HU and STOREY(1991), a class of new restarting conjugate gradient methods is presented. Global convergences of the new method with two kinds of common line searches, are proved. Firstly, it is shown that, using reverse modulus of continuity function and forcing function, the new method for solving unconstrained optimization can work for a continously dif ferentiable function with Curry-Altman's step size rule and a bounded level set. Secondly, by using comparing technique, some general convergence properties of the new method with other kind of step size rule are established. Numerical experiments show that the new method is efficient by comparing with FR conjugate gradient method.展开更多
In this note,by combining the nice numerical performance of PR and HS methods with the global convergence property of FR method,a class of new restarting three terms conjugate gradient methods is presented.Global conv...In this note,by combining the nice numerical performance of PR and HS methods with the global convergence property of FR method,a class of new restarting three terms conjugate gradient methods is presented.Global convergence properties of the new method with two kinds of common line searches are proved.展开更多
Many methods have been put forward to solve unconstrained optimization problems,among which conjugate gradient method(CG)is very important.With the increasing emergence of large⁃scale problems,the subspace technology ...Many methods have been put forward to solve unconstrained optimization problems,among which conjugate gradient method(CG)is very important.With the increasing emergence of large⁃scale problems,the subspace technology has become particularly important and widely used in the field of optimization.In this study,a new CG method was put forward,which combined subspace technology and a cubic regularization model.Besides,a special scaled norm in a cubic regularization model was analyzed.Under certain conditions,some significant characteristics of the search direction were given and the convergence of the algorithm was built.Numerical comparisons show that for the 145 test functions under the CUTEr library,the proposed method is better than two classical CG methods and two new subspaces conjugate gradient methods.展开更多
文摘Recently, Gilbert and Nocedal([3]) investigated global convergence of conjugate gradient methods related to Polak-Ribiere formular, they restricted beta(k) to non-negative value. [5] discussed the same problem as that in [3] and relaxed beta(k) to be negative with the objective function being convex. This paper allows beta(k) to be selected in a wider range than [5]. Especially, the global convergence of the corresponding algorithm without sufficient decrease condition is proved.
文摘Nonlinear conjugate gradient methods have played an important role in solving large scale unconstrained optimi-zation problems,it is characterized by the simplicity of their iteration and their low memory requirements.It is well-known that the direction generated by a conjugate gradient method may be not a descent direction.In this paper,a new class of nonlinear conjugate gradient method is presented,its search direction is a descent direction for the objective function.If the objective function is differentiable and its gradient is Lipschitz continuous,the line sbarch satisfies strong Wolfe condition,the global convergence result is established.
基金Supported by the National Natural Science Foundation of China(Grant No.11761014)the Natural Science Foundation of Guangxi Zhuang Autonomous Region(Grant No.2017GXNSFAA198243)+2 种基金Guangxi Basic Ability Improvement Project for the Middle-Aged and Young Teachers of Colleges and Universities(Grant Nos.2017KY0068KY2016YB069)Guangxi Higher Education Undergraduate Course Teaching Reform Project(Grant No.2017JGB147)
文摘In this paper, a new steplength formula is proposed for unconstrained optimization,which can determine the step-size only by one step and avoids the line search step. Global convergence of the five well-known conjugate gradient methods with this formula is analyzed,and the corresponding results are as follows：（1） The DY method globally converges for a strongly convex LC^1 objective function;（2） The CD method, the FR method, the PRP method and the LS method globally converge for a general, not necessarily convex, LC^1 objective function.
基金Supported by the National Natural Science Foundation of China(1 0 1 6 1 0 0 2 ) and Guangxi Natural Sci-ence Foundation (0 1 3 5 0 0 4 )
文摘This paper discusses the global convergence of a class of nonmonotone conjugate gra- dient methods(NM methods) for nonconvex object functions.This class of methods includes the nonmonotone counterpart of modified Polak- Ribière method and modified Hestenes- Stiefel method as special cases
基金NSF Project of China Grant(61273179,11201039)Hubei Provincial Department of Education Grant(D20101304)
文摘Nonlinear conjugate gradient methods have played an important role in solving large scale uncon-srined optimization problems,it is careterired by the simplicity of their iteration and their low memory requirements.It is wll-known that the dreetion generted by a cojugate gradient method may be not a de-scent direction.In this paper,a new class of nonlinear conjugate gradient method is presented,its serch di-rection is a descent direction for the oiective funetion.If the obiective function is dfrentiable and its gradi ent is Lipschitz continuous,the line search satisfies strong Wolle condition,the global convergence result is established.
基金Research partially supported by Chinese NSF grants 19801033,19771047 and 10171104
文摘In this paper we test different conjugate gradient (CG) methods for solving large-scale unconstrained optimization problems. The methods are divided in two groups: the first group includes five basic CG methods and the second five hybrid CG methods. A collection of medium-scale and large-scale test problems are drawn from a standard code of test problems, CUTE. The conjugate gradient methods are ranked according to the numerical results. Some remarks are given.
文摘We consider solving integral equations of the second kind defined on the half-line [0, infinity) by the preconditioned conjugate gradient method. Convergence is known to be slow due to the non-compactness of the associated integral operator. In this paper, we construct two different circulant integral operators to be used as preconditioners for the method to speed up its convergence rate. We prove that if the given integral operator is close to a convolution-type integral operator, then the preconditioned systems will have spectrum clustered around 1 and hence the preconditioned conjugate gradient method will converge superlinearly. Numerical examples are given to illustrate the fast convergence.
基金Supported by the Science and Technology Project of Guangxi(Guike AD23023002)。
文摘In this paper,we propose a three-term conjugate gradient method for solving unconstrained optimization problems based on the Hestenes-Stiefel(HS)conjugate gradient method and Polak-Ribiere-Polyak(PRP)conjugate gradient method.Under the condition of standard Wolfe line search,the proposed search direction is the descent direction.For general nonlinear functions,the method is globally convergent.Finally,numerical results show that the proposed method is efficient.
基金supported by the National Basic Research Program (No.2005CB321702)the China NNSF Outstanding Young Scientist Foundation (No.10525102)the National Natural Science Foundation (No.10471146),P.R.China
文摘The restrictively preconditioned conjugate gradient （RPCG） method is further developed to solve large sparse system of linear equations of a block two-by-two structure. The basic idea of this new approach is that we apply the RPCG method to the normal-residual equation of the block two-by-two linear system and construct each required approximate matrix by making use of the incomplete orthogonal factorization of the involved matrix blocks. Numerical experiments show that the new method, called the restrictively preconditioned conjugate gradient on normal residual （RPCGNR）, is more robust and effective than either the known RPCG method or the standard conjugate gradient on normal residual （CGNR） method when being used for solving the large sparse saddle point problems.
基金Supported by the National Natural Science Foundation of China (No.19801033 and 10171104).
文摘Two Armijo-type line searches are proposed in this paper for nonlinear conjugate gradient methods. Under these line searches, global convergence results are established for several famous conjugate gradient methods, including the Fletcher-Reeves method, the Polak-Ribiere-Polyak method, and the conjugate descent method.
基金This Project is supported Supported by the National Natural Science Foundation of China (No.19731001).
文摘Two fundamental convergence theorems are given for nonlinear conjugate gradient methods only under the descent condition. As a result, methods related to the Fletcher-Reeves algorithm still converge for parameters in a slightly wider range, in particular, for a parameter in its upper bound. For methods related to the Polak-Ribiere algorithm, it is shown that some negative values of the conjugate parameter do not prevent convergence. If the objective function is convex, some convergence results hold for the Hestenes-Stiefel algorithm.
文摘We study the conjugate gradient method for solving a system of linear equations with coefficients which are measurable functions and establish the rate of convergence of this method.
基金partially supported by the National Natural Science Foundation of China (No.41230318)
文摘With the development of computational power, there has been an increased focus on data-fitting related seismic inversion techniques for high fidelity seismic velocity model and image, such as full-waveform inversion and least squares migration. However, though more advanced than conventional methods, these data fitting methods can be very expensive in terms of computational cost. Recently, various techniques to optimize these data-fitting seismic inversion problems have been implemented to cater for the industrial need for much improved efficiency. In this study, we propose a general stochastic conjugate gradient method for these data-fitting related inverse problems. We first prescribe the basic theory of our method and then give synthetic examples. Our numerical experiments illustrate the potential of this method for large-size seismic inversion application.
基金Supported by Research Council of Semnan University
文摘A hybridization of the three–term conjugate gradient method proposed by Zhang et al. and the nonlinear conjugate gradient method proposed by Polak and Ribi`ere, and Polyak is suggested. Based on an eigenvalue analysis, it is shown that search directions of the proposed method satisfy the sufficient descent condition, independent of the line search and the objective function convexity. Global convergence of the method is established under an Armijo–type line search condition. Numerical experiments show practical efficiency of the proposed method.
基金Project supported by the National Natural Science Foundation of China(Nos.5130926141030747+3 种基金41102181and 51121005)the National Basic Research Program of China(973 Program)(No.2011CB013503)the Young Teachers’ Initial Funding Scheme of Sun Yat-sen University(No.39000-1188140)
文摘Fast solving large-scale linear equations in the finite element analysis is a classical subject in computational mechanics. It is a key technique in computer aided engineering （CAE） and computer aided manufacturing （CAM）. This paper presents a high-efficiency improved symmetric successive over-relaxation （ISSOR） preconditioned conjugate gradient （PCG） method, which maintains lelism consistent with the original form. Ideally, the by 50% as compared with the original algorithm. the convergence and inherent paralcomputation can It is suitable for be reduced nearly high-performance computing with its inherent basic high-efficiency operations. By comparing with the numerical results, it is shown that the proposed method has the best performance.
基金Supported by the Fund of Chongqing Education Committee(KJ091104)
文摘In this paper,an efficient conjugate gradient method is given to solve the general unconstrained optimization problems,which can guarantee the sufficient descent property and the global convergence with the strong Wolfe line search conditions.Numerical results show that the new method is efficient and stationary by comparing with PRP＋ method,so it can be widely used in scientific computation.
文摘Conjugate gradient optimization algorithms depend on the search directions with different choices for the parameters in the search directions. In this note, by combining the nice numerical performance of PR and HS methods with the global convergence property of the class of conjugate gradient methods presented by HU and STOREY(1991), a class of new restarting conjugate gradient methods is presented. Global convergences of the new method with two kinds of common line searches, are proved. Firstly, it is shown that, using reverse modulus of continuity function and forcing function, the new method for solving unconstrained optimization can work for a continously dif ferentiable function with Curry-Altman's step size rule and a bounded level set. Secondly, by using comparing technique, some general convergence properties of the new method with other kind of step size rule are established. Numerical experiments show that the new method is efficient by comparing with FR conjugate gradient method.
基金Supported by the National Natural Science Foundation of China（10571106） Supported by the Fundamental Research Funds for the Central Universities（10CX04044A）
文摘In this note,by combining the nice numerical performance of PR and HS methods with the global convergence property of FR method,a class of new restarting three terms conjugate gradient methods is presented.Global convergence properties of the new method with two kinds of common line searches are proved.
基金Sponsored by the National Natural Science Foundation of China(Grant No.11901561).
文摘Many methods have been put forward to solve unconstrained optimization problems,among which conjugate gradient method(CG)is very important.With the increasing emergence of large⁃scale problems,the subspace technology has become particularly important and widely used in the field of optimization.In this study,a new CG method was put forward,which combined subspace technology and a cubic regularization model.Besides,a special scaled norm in a cubic regularization model was analyzed.Under certain conditions,some significant characteristics of the search direction were given and the convergence of the algorithm was built.Numerical comparisons show that for the 145 test functions under the CUTEr library,the proposed method is better than two classical CG methods and two new subspaces conjugate gradient methods.
