Nonmonotone gradient methods generally perform better than their monotone counterparts especially on unconstrained quadratic optimization.However,the known convergence rate of the monotone method is often much better ...Nonmonotone gradient methods generally perform better than their monotone counterparts especially on unconstrained quadratic optimization.However,the known convergence rate of the monotone method is often much better than its nonmonotone variant.With the aim of shrinking the gap between theory and practice of nonmonotone gradient methods,we introduce a property for convergence analysis of a large collection of gradient methods.We prove that any gradient method using stepsizes satisfying the property will converge R-linearly at a rate of 1-λ_(1)/M_(1),whereλ_(1)is the smallest eigenvalue of Hessian matrix and M_(1)is the upper bound of the inverse stepsize.Our results indicate that the existing convergence rates of many nonmonotone methods can be improved to 1-1/κwithκbeing the associated condition number.展开更多
基金supported by the National Natural Science Foundation of China(No.11701137)the Natural Science Foundation of Hebei Province(No.A2021202010).
文摘Nonmonotone gradient methods generally perform better than their monotone counterparts especially on unconstrained quadratic optimization.However,the known convergence rate of the monotone method is often much better than its nonmonotone variant.With the aim of shrinking the gap between theory and practice of nonmonotone gradient methods,we introduce a property for convergence analysis of a large collection of gradient methods.We prove that any gradient method using stepsizes satisfying the property will converge R-linearly at a rate of 1-λ_(1)/M_(1),whereλ_(1)is the smallest eigenvalue of Hessian matrix and M_(1)is the upper bound of the inverse stepsize.Our results indicate that the existing convergence rates of many nonmonotone methods can be improved to 1-1/κwithκbeing the associated condition number.
