This paper deals with the stability of static recurrent neural networks （RNNs） with a time-varying delay. An augmented Lyapunov-Krasovskii functional is employed, in which some useful terms are included. Furthermore...This paper deals with the stability of static recurrent neural networks （RNNs） with a time-varying delay. An augmented Lyapunov-Krasovskii functional is employed, in which some useful terms are included. Furthermore, the relationship among the timevarying delay, its upper bound and their difierence, is taken into account, and novel bounding techniques for 1- τ（t） are employed. As a result, without ignoring any useful term in the derivative of the Lyapunov-Krasovskii functional, the resulting delay-dependent criteria show less conservative than the existing ones. Finally, a numerical example is given to demonstrate the effectiveness of the proposed methods.展开更多
Tensor robust principal component analysis（TRPCA） problem aims to separate a low-rank tensor and a sparse tensor from their sum. This problem has recently attracted considerable research attention due to its wide ra...Tensor robust principal component analysis（TRPCA） problem aims to separate a low-rank tensor and a sparse tensor from their sum. This problem has recently attracted considerable research attention due to its wide range of potential applications in computer vision and pattern recognition. In this paper, we propose a new model to deal with the TRPCA problem by an alternation minimization algorithm along with two adaptive rankadjusting strategies. For the underlying low-rank tensor, we simultaneously perform low-rank matrix factorizations to its all-mode matricizations; while for the underlying sparse tensor,a soft-threshold shrinkage scheme is applied. Our method can be used to deal with the separation between either an exact or an approximate low-rank tensor and a sparse one. We established the subsequence convergence of our algorithm in the sense that any limit point of the iterates satisfies the KKT conditions. When the iteration stops, the output will be modified by applying a high-order SVD approach to achieve an exactly low-rank final result as the accurate rank has been calculated. The numerical experiments demonstrate that our method could achieve better results than the compared methods.展开更多
An exact augmented Lagrangian function for the nonlinear nonconvex programming problems with inequality constraints was discussed. Under suitable hypotheses, the relationship was established between the local unconstr...An exact augmented Lagrangian function for the nonlinear nonconvex programming problems with inequality constraints was discussed. Under suitable hypotheses, the relationship was established between the local unconstrained minimizers of the augmented Lagrangian function on the space of problem variables and the local minimizers of the original constrained problem. Furthermore, under some assumptions, the relationship was also established between the global solutions of the augmented Lagrangian function on some compact subset of the space of problem variables and the global solutions of the constrained problem. Therefore, f^om the theoretical point of view, a solution of the inequality constrained problem and the corresponding values of the Lagrange multipliers can be found by the well-known method of multipliers which resort to the unconstrained minimization of the augmented Lagrangian function presented.展开更多
This paper focuses on the problem of delay-dependent stability of linear systems with time-varying delay.A new delay-product-type augmented Lyapunov-Krasovskii functional(LKF)is constructed.Based on the LKF and by emp...This paper focuses on the problem of delay-dependent stability of linear systems with time-varying delay.A new delay-product-type augmented Lyapunov-Krasovskii functional(LKF)is constructed.Based on the LKF and by employing a generalized free-matrix-based integral inequality,less conservative delay-dependent stability criteria are obtained.Finally,two well-known numerical examples are used to confirm the effectiveness and the superiority of the presented stability criteria.展开更多
Non-intrusive load monitoring is a technique for monitoring the operating conditions of electrical appliances by collecting the aggregated electrical information at the household power inlet.Despite several studies on...Non-intrusive load monitoring is a technique for monitoring the operating conditions of electrical appliances by collecting the aggregated electrical information at the household power inlet.Despite several studies on the mining of unique load characteristics,few studies have extensively considered the high computational burden and sample training.Based on lowfrequency sampling data,a non-intrusive load monitoring algorithm utilizing the graph total variation(GTV)is proposed in this study.The algorithm can effectively depict the load state without the need for prior training.First,the combined Kmeans clustering algorithm and graph signals are used to build concise and accurate graph structures as load models.The GTV representing the internal structure of the graph signal is introduced as the optimization model and solved using the augmented Lagrangian iterative algorithm.The introduction of the difference operator reduces the computing cost and addresses the inaccurate reconstruction of the graph signal.With low-frequency sampling data,the algorithm only requires a little prior data and no training,thereby reducing the computing cost.Experiments conducted using the reference energy disaggregation dataset and almanac of minutely power dataset demonstrated the stable superiority of the algorithm and its low computational burden.展开更多
We propose a new trust region algorithm for nonlinear constrained optimization problems. In each iteration of our algorithm, the trial step is computed by minimizing a quadratic approximation to the augmented Lagrange...We propose a new trust region algorithm for nonlinear constrained optimization problems. In each iteration of our algorithm, the trial step is computed by minimizing a quadratic approximation to the augmented Lagrange function in the trust region. The augmented Lagrange function is also used as a merit function to decide whether the trial step should be accepted. Our method extends the traditional trust region approach by combining a filter technique into the rules for accepting trial steps so that a trial step could still be accepted even when it is rejected by the traditional rule based on merit function reduction. An estimate of the Lagrange multiplier is updated at each iteration, and the penalty parameter is updated to force sufficient reduction in the norm of the constraint violations. Active set technique is used to handle the inequality constraints. Numerical results for a set of constrained problems from the CUTEr collection are also reported.展开更多
An augmented Lagrangian trust region method with a bi=object strategy is proposed for solving nonlinear equality constrained optimization, which falls in between penalty-type methods and penalty-free ones. At each ite...An augmented Lagrangian trust region method with a bi=object strategy is proposed for solving nonlinear equality constrained optimization, which falls in between penalty-type methods and penalty-free ones. At each iteration, a trial step is computed by minimizing a quadratic approximation model to the augmented Lagrangian function within a trust region. The model is a standard trust region subproblem for unconstrained optimization and hence can efficiently be solved by many existing methods. To choose the penalty parameter, an auxiliary trust region subproblem is introduced related to the constraint violation. It turns out that the penalty parameter need not be monotonically increasing and will not tend to infinity. A bi-object strategy, which is related to the objective function and the measure of constraint violation, is utilized to decide whether the trial step will be accepted or not. Global convergence of the method is established under mild assumptions. Numerical experiments are made, which illustrate the efficiency of the algorithm on various difficult situations.展开更多
Nonlinear convex cone programming(NCCP)models have found many practical applications.In this paper,we introduce a flexible first-order primal-dual algorithm,called the variant auxiliary problem principle(VAPP),for sol...Nonlinear convex cone programming(NCCP)models have found many practical applications.In this paper,we introduce a flexible first-order primal-dual algorithm,called the variant auxiliary problem principle(VAPP),for solving NCCP problems when the objective function and constraints are convex but may be nonsmooth.At each iteration,VAPP generates a nonlinear approximation of the primal augmented Lagrangian model.The approximation incorporates both linearization and a distance-like proximal term,and then the iterations of VAPP are shown to possess a decomposition property for NCCP.Motivated by recent applications in big data analytics,there has been a growing interest in the convergence rate analysis of algorithms with parallel computing capabilities for large scale optimization problems.We establish O(1/t)convergence rate towards primal optimality,feasibility and dual optimality.By adaptively setting parameters at different iterations,we show an O(1/t2)rate for the strongly convex case.Finally,we discuss some issues in the implementation of VAPP.展开更多
The augmented Lagrangian method is a classical method for solving constrained optimization.Recently,the augmented Lagrangian method attracts much attention due to its applications to sparse optimization in compressive...The augmented Lagrangian method is a classical method for solving constrained optimization.Recently,the augmented Lagrangian method attracts much attention due to its applications to sparse optimization in compressive sensing and low rank matrix optimization problems.However,most Lagrangian methods use first order information to update the Lagrange multipliers,which lead to only linear convergence.In this paper,we study an update technique based on second order information and prove that superlinear convergence can be obtained.Theoretical properties of the update formula are given and some implementation issues regarding the new update are also discussed.展开更多
A continuation algorithm for the solution of max-cut problems is proposed in this paper. Unlike the available semi-definite relaxation, a max-cut problem is converted into a continuous nonlinear programming by employi...A continuation algorithm for the solution of max-cut problems is proposed in this paper. Unlike the available semi-definite relaxation, a max-cut problem is converted into a continuous nonlinear programming by employing NCP functions, and the resulting nonlinear programming problem is then solved by using the augmented Lagrange penalty function method. The convergence property of the proposed algorithm is studied. Numerical experiments and comparisons with the Geomeans and Williamson randomized algorithm made on some max-cut test problems show that the algorithm generates satisfactory solutions for all the test problems with much less computation costs.展开更多
Firstly,this paper proposes a generalized log-determinant optimization model with the purpose of estimating the high-dimensional sparse inverse covariance matrices.Under the normality assumption,the zero components in...Firstly,this paper proposes a generalized log-determinant optimization model with the purpose of estimating the high-dimensional sparse inverse covariance matrices.Under the normality assumption,the zero components in the inverse covariance matrices represent the conditional independence between pairs of variables given all the other variables.The generalized model considered in this study,because of the setting of the eigenvalue bounded constraints,covers a large number of existing estimators as special cases.Secondly,rather than directly tracking the challenging optimization problem,this paper uses a couple of alternating direction methods of multipliers(ADMM)to solve its dual model where 5 separable structures are contained.The first implemented algorithm is based on a single Gauss–Seidel iteration,but it does not necessarily converge theoretically.In contrast,the second algorithm employs the symmetric Gauss–Seidel(sGS)based ADMM which is equivalent to the 2-block iterative scheme from the latest sGS decomposition theorem.Finally,we do numerical simulations using the synthetic data and the real data set which show that both algorithms are very effective in estimating high-dimensional sparse inverse covariance matrix.展开更多
基金supported by National Natural Science Foundation of China (No. 60874025)Natural Science Foundation of Hunan Province of China (No. 10JJ6098)
文摘This paper deals with the stability of static recurrent neural networks （RNNs） with a time-varying delay. An augmented Lyapunov-Krasovskii functional is employed, in which some useful terms are included. Furthermore, the relationship among the timevarying delay, its upper bound and their difierence, is taken into account, and novel bounding techniques for 1- τ（t） are employed. As a result, without ignoring any useful term in the derivative of the Lyapunov-Krasovskii functional, the resulting delay-dependent criteria show less conservative than the existing ones. Finally, a numerical example is given to demonstrate the effectiveness of the proposed methods.
基金Supported by the National Natural Science Foundation of China(Grant Nos.6157209961320106008+2 种基金91230103)National Science and Technology Major Project(Grant Nos.2013ZX040050212014ZX04001011)
文摘Tensor robust principal component analysis（TRPCA） problem aims to separate a low-rank tensor and a sparse tensor from their sum. This problem has recently attracted considerable research attention due to its wide range of potential applications in computer vision and pattern recognition. In this paper, we propose a new model to deal with the TRPCA problem by an alternation minimization algorithm along with two adaptive rankadjusting strategies. For the underlying low-rank tensor, we simultaneously perform low-rank matrix factorizations to its all-mode matricizations; while for the underlying sparse tensor,a soft-threshold shrinkage scheme is applied. Our method can be used to deal with the separation between either an exact or an approximate low-rank tensor and a sparse one. We established the subsequence convergence of our algorithm in the sense that any limit point of the iterates satisfies the KKT conditions. When the iteration stops, the output will be modified by applying a high-order SVD approach to achieve an exactly low-rank final result as the accurate rank has been calculated. The numerical experiments demonstrate that our method could achieve better results than the compared methods.
文摘An exact augmented Lagrangian function for the nonlinear nonconvex programming problems with inequality constraints was discussed. Under suitable hypotheses, the relationship was established between the local unconstrained minimizers of the augmented Lagrangian function on the space of problem variables and the local minimizers of the original constrained problem. Furthermore, under some assumptions, the relationship was also established between the global solutions of the augmented Lagrangian function on some compact subset of the space of problem variables and the global solutions of the constrained problem. Therefore, f^om the theoretical point of view, a solution of the inequality constrained problem and the corresponding values of the Lagrange multipliers can be found by the well-known method of multipliers which resort to the unconstrained minimization of the augmented Lagrangian function presented.
基金the National Natural Science Fund of China under Grant Nos.61741308,61703153,61672225the Natural Science Fund of Hunan Province under Grant Nos.2018JJ2096 and 2018JJ4075。
文摘This paper focuses on the problem of delay-dependent stability of linear systems with time-varying delay.A new delay-product-type augmented Lyapunov-Krasovskii functional(LKF)is constructed.Based on the LKF and by employing a generalized free-matrix-based integral inequality,less conservative delay-dependent stability criteria are obtained.Finally,two well-known numerical examples are used to confirm the effectiveness and the superiority of the presented stability criteria.
基金supported by National Natural Science Foundation of China(No.52107117)。
文摘Non-intrusive load monitoring is a technique for monitoring the operating conditions of electrical appliances by collecting the aggregated electrical information at the household power inlet.Despite several studies on the mining of unique load characteristics,few studies have extensively considered the high computational burden and sample training.Based on lowfrequency sampling data,a non-intrusive load monitoring algorithm utilizing the graph total variation(GTV)is proposed in this study.The algorithm can effectively depict the load state without the need for prior training.First,the combined Kmeans clustering algorithm and graph signals are used to build concise and accurate graph structures as load models.The GTV representing the internal structure of the graph signal is introduced as the optimization model and solved using the augmented Lagrangian iterative algorithm.The introduction of the difference operator reduces the computing cost and addresses the inaccurate reconstruction of the graph signal.With low-frequency sampling data,the algorithm only requires a little prior data and no training,thereby reducing the computing cost.Experiments conducted using the reference energy disaggregation dataset and almanac of minutely power dataset demonstrated the stable superiority of the algorithm and its low computational burden.
基金supported by NSFC Grant 10831006CAS grant kjcx-yw-s7
文摘We propose a new trust region algorithm for nonlinear constrained optimization problems. In each iteration of our algorithm, the trial step is computed by minimizing a quadratic approximation to the augmented Lagrange function in the trust region. The augmented Lagrange function is also used as a merit function to decide whether the trial step should be accepted. Our method extends the traditional trust region approach by combining a filter technique into the rules for accepting trial steps so that a trial step could still be accepted even when it is rejected by the traditional rule based on merit function reduction. An estimate of the Lagrange multiplier is updated at each iteration, and the penalty parameter is updated to force sufficient reduction in the norm of the constraint violations. Active set technique is used to handle the inequality constraints. Numerical results for a set of constrained problems from the CUTEr collection are also reported.
文摘An augmented Lagrangian trust region method with a bi=object strategy is proposed for solving nonlinear equality constrained optimization, which falls in between penalty-type methods and penalty-free ones. At each iteration, a trial step is computed by minimizing a quadratic approximation model to the augmented Lagrangian function within a trust region. The model is a standard trust region subproblem for unconstrained optimization and hence can efficiently be solved by many existing methods. To choose the penalty parameter, an auxiliary trust region subproblem is introduced related to the constraint violation. It turns out that the penalty parameter need not be monotonically increasing and will not tend to infinity. A bi-object strategy, which is related to the objective function and the measure of constraint violation, is utilized to decide whether the trial step will be accepted or not. Global convergence of the method is established under mild assumptions. Numerical experiments are made, which illustrate the efficiency of the algorithm on various difficult situations.
基金This research was supported by the National Natural Science Foundation of China(Nos.71471112 and 71871140).
文摘Nonlinear convex cone programming(NCCP)models have found many practical applications.In this paper,we introduce a flexible first-order primal-dual algorithm,called the variant auxiliary problem principle(VAPP),for solving NCCP problems when the objective function and constraints are convex but may be nonsmooth.At each iteration,VAPP generates a nonlinear approximation of the primal augmented Lagrangian model.The approximation incorporates both linearization and a distance-like proximal term,and then the iterations of VAPP are shown to possess a decomposition property for NCCP.Motivated by recent applications in big data analytics,there has been a growing interest in the convergence rate analysis of algorithms with parallel computing capabilities for large scale optimization problems.We establish O(1/t)convergence rate towards primal optimality,feasibility and dual optimality.By adaptively setting parameters at different iterations,we show an O(1/t2)rate for the strongly convex case.Finally,we discuss some issues in the implementation of VAPP.
基金Supported by National Natural Science Foundation of China(Grant Nos.10831006,11021101)by CAS(Grant No.kjcx-yw-s7)
文摘The augmented Lagrangian method is a classical method for solving constrained optimization.Recently,the augmented Lagrangian method attracts much attention due to its applications to sparse optimization in compressive sensing and low rank matrix optimization problems.However,most Lagrangian methods use first order information to update the Lagrange multipliers,which lead to only linear convergence.In this paper,we study an update technique based on second order information and prove that superlinear convergence can be obtained.Theoretical properties of the update formula are given and some implementation issues regarding the new update are also discussed.
基金Key Project supported by National Natural Science Foundation of China,10231060
文摘A continuation algorithm for the solution of max-cut problems is proposed in this paper. Unlike the available semi-definite relaxation, a max-cut problem is converted into a continuous nonlinear programming by employing NCP functions, and the resulting nonlinear programming problem is then solved by using the augmented Lagrange penalty function method. The convergence property of the proposed algorithm is studied. Numerical experiments and comparisons with the Geomeans and Williamson randomized algorithm made on some max-cut test problems show that the algorithm generates satisfactory solutions for all the test problems with much less computation costs.
基金the National Natural Science Foundation of China(No.11971149).
文摘Firstly,this paper proposes a generalized log-determinant optimization model with the purpose of estimating the high-dimensional sparse inverse covariance matrices.Under the normality assumption,the zero components in the inverse covariance matrices represent the conditional independence between pairs of variables given all the other variables.The generalized model considered in this study,because of the setting of the eigenvalue bounded constraints,covers a large number of existing estimators as special cases.Secondly,rather than directly tracking the challenging optimization problem,this paper uses a couple of alternating direction methods of multipliers(ADMM)to solve its dual model where 5 separable structures are contained.The first implemented algorithm is based on a single Gauss–Seidel iteration,but it does not necessarily converge theoretically.In contrast,the second algorithm employs the symmetric Gauss–Seidel(sGS)based ADMM which is equivalent to the 2-block iterative scheme from the latest sGS decomposition theorem.Finally,we do numerical simulations using the synthetic data and the real data set which show that both algorithms are very effective in estimating high-dimensional sparse inverse covariance matrix.
