We propose two nonmonotone retraction-based proximal gradient methods for solving a class of nonconvex nonsmooth optimization problems over the Stiefel manifold.The proposed methods are equipped with the descent direc...We propose two nonmonotone retraction-based proximal gradient methods for solving a class of nonconvex nonsmooth optimization problems over the Stiefel manifold.The proposed methods are equipped with the descent direction obtained by a proximal mapping restricted in tangent space of the manifold and the BarzilaiBorwein stepsizes determined by two recent iteration points and the corresponding descent directions.By employing,respectively,the Grippo-Lampariello-Lucidi nonmonotone line search strategy and the Dai-Fletcher nonmonotone line search strategy,our proposed methods are proved to be globally convergent.Analysis on the iteration complexity for obtaining an?-stationary solution is provided.Numerical results on the sparse principle component analysis problems demonstrate the efficiency of our methods.展开更多
The problem of nonconvex and nonsmooth optimization(NNO)has been extensively studied in the machine learning community,leading to the development of numerous fast and convergent numerical algorithms.Existing algorithm...The problem of nonconvex and nonsmooth optimization(NNO)has been extensively studied in the machine learning community,leading to the development of numerous fast and convergent numerical algorithms.Existing algorithms typically employ unified iteration schemes and require explicit solutions to subproblems for ensuring convergence.However,these inflexible iteration schemes overlook task-specific details and may encounter difficulties in providing explicit solutions to subproblems.In contrast,there is evidence suggesting that practical applications can benefit from approximately solving subproblems;however,many existing works fail to establish the theoretical validity of such approximations.In this paper,the authors propose a hybrid inexact proximal alternating method(hiPAM),which addresses a general NNO problem with coupled terms while overcoming all aforementioned challenges.The proposed hiPAM algorithm offers a flexible yet highly efficient approach by seamlessly integrating any efficient methods for approximate subproblem solving that cater to specificities.Additionally,the authors have devised a simple yet implementable stopping criterion that generates a Cauchy sequence and ultimately converges to a critical point of the original NNO problem.The proposed numerical experiments using both simulated and real data have demonstrated that hiPAM represents an exceedingly efficient and robust approach to NNO problems.展开更多
In this paper,the authors propose a novel smoothing descent type algorithm with extrapolation for solving a class of constrained nonsmooth and nonconvex problems,where the nonconvex term is possibly nonsmooth.Their al...In this paper,the authors propose a novel smoothing descent type algorithm with extrapolation for solving a class of constrained nonsmooth and nonconvex problems,where the nonconvex term is possibly nonsmooth.Their algorithm adopts the proximal gradient algorithm with extrapolation and a safe-guarding policy to minimize the smoothed objective function for better practical and theoretical performance.Moreover,the algorithm uses a easily checking rule to update the smoothing parameter to ensure that any accumulation point of the generated sequence is an(afne-scaled)Clarke stationary point of the original nonsmooth and nonconvex problem.Their experimental results indicate the effectiveness of the proposed algorithm.展开更多
基金supported by the National Natural Science Foundation of China(Grant Nos.11671116,12071108 and 11701137)Natural Science Foundation of Hebei Province(Grant No.A2021202010)。
文摘We propose two nonmonotone retraction-based proximal gradient methods for solving a class of nonconvex nonsmooth optimization problems over the Stiefel manifold.The proposed methods are equipped with the descent direction obtained by a proximal mapping restricted in tangent space of the manifold and the BarzilaiBorwein stepsizes determined by two recent iteration points and the corresponding descent directions.By employing,respectively,the Grippo-Lampariello-Lucidi nonmonotone line search strategy and the Dai-Fletcher nonmonotone line search strategy,our proposed methods are proved to be globally convergent.Analysis on the iteration complexity for obtaining an?-stationary solution is provided.Numerical results on the sparse principle component analysis problems demonstrate the efficiency of our methods.
基金supported by the National Key R&D Program of China under Grant No.2023YFA1011303the National Natural Science Foundation of China under Grant Nos.61806057 and 12301479+1 种基金the China Postdoctoral Science Foundation under Grant No.2018M632018the Natural Science Foundation of Liaoning Province under Grant No.2023-MS-126。
文摘The problem of nonconvex and nonsmooth optimization(NNO)has been extensively studied in the machine learning community,leading to the development of numerous fast and convergent numerical algorithms.Existing algorithms typically employ unified iteration schemes and require explicit solutions to subproblems for ensuring convergence.However,these inflexible iteration schemes overlook task-specific details and may encounter difficulties in providing explicit solutions to subproblems.In contrast,there is evidence suggesting that practical applications can benefit from approximately solving subproblems;however,many existing works fail to establish the theoretical validity of such approximations.In this paper,the authors propose a hybrid inexact proximal alternating method(hiPAM),which addresses a general NNO problem with coupled terms while overcoming all aforementioned challenges.The proposed hiPAM algorithm offers a flexible yet highly efficient approach by seamlessly integrating any efficient methods for approximate subproblem solving that cater to specificities.Additionally,the authors have devised a simple yet implementable stopping criterion that generates a Cauchy sequence and ultimately converges to a critical point of the original NNO problem.The proposed numerical experiments using both simulated and real data have demonstrated that hiPAM represents an exceedingly efficient and robust approach to NNO problems.
基金supported by the National Natural Science Foundation of China(No.12001144)Zhejiang Provincial Natural Science Foundation of China(No.LQ20A010007)NSF/DMS-2152961。
文摘In this paper,the authors propose a novel smoothing descent type algorithm with extrapolation for solving a class of constrained nonsmooth and nonconvex problems,where the nonconvex term is possibly nonsmooth.Their algorithm adopts the proximal gradient algorithm with extrapolation and a safe-guarding policy to minimize the smoothed objective function for better practical and theoretical performance.Moreover,the algorithm uses a easily checking rule to update the smoothing parameter to ensure that any accumulation point of the generated sequence is an(afne-scaled)Clarke stationary point of the original nonsmooth and nonconvex problem.Their experimental results indicate the effectiveness of the proposed algorithm.
