This paper addresses the learning algorithm on the unit sphere.The main purpose is to present an error analysis for regression generated by regularized least square algorithms with spherical harmonics kernel.The exces...This paper addresses the learning algorithm on the unit sphere.The main purpose is to present an error analysis for regression generated by regularized least square algorithms with spherical harmonics kernel.The excess error can be estimated by the sum of sample errors and regularization errors.Our study shows that by introducing a suitable spherical harmonics kernel,the regularization parameter can decrease arbitrarily fast with the sample size.展开更多
In view of the composition analysis and identification of ancient glass products, L1 regularization, K-Means cluster analysis, elbow rule and other methods were comprehensively used to build logical regression, cluste...In view of the composition analysis and identification of ancient glass products, L1 regularization, K-Means cluster analysis, elbow rule and other methods were comprehensively used to build logical regression, cluster analysis, hyper-parameter test and other models, and SPSS, Python and other tools were used to obtain the classification rules of glass products under different fluxes, sub classification under different chemical compositions, hyper-parameter K value test and rationality analysis. Research can provide theoretical support for the protection and restoration of ancient glass relics.展开更多
This study examines the relationship between macroeconomic variables and stock price indices of four prominent OPEC oil-exporting members.Bayesian model averaging(BMA)and regularized linear regression(RLR)are employed...This study examines the relationship between macroeconomic variables and stock price indices of four prominent OPEC oil-exporting members.Bayesian model averaging(BMA)and regularized linear regression(RLR)are employed to address uncertainties arising from different estimation models and variable selection.Jointness is utilized to determine the nature of relationships among variable pairs.The case study spans macroeconomic variables and stock prices from 1996 to 2018.BMA findings reveal a strong positive association between stock price indices and both consumer price index(CPI)and broad money growth in each analyzed OPEC country.Additionally,the study suggests a weak negative correlation between OPEC oil prices and the stock price index.RLR results align with BMA analysis,offering insights valuable for policymakers and international wealth managers.展开更多
A novel near infrared （NIR） modeling method-Laplacian regularized least squares regression （LapRLSR） was presented,which can take the advantage of many unlabeled spectra to promote the prediction performance of th...A novel near infrared （NIR） modeling method-Laplacian regularized least squares regression （LapRLSR） was presented,which can take the advantage of many unlabeled spectra to promote the prediction performance of the model even if there are only few calibration samples. Using LapRLSR modeling, NIR spectral analysis was applied to the online monitoring of the concentration of salvia acid B in the column separation of Salvianolate. The results demonstrated that LapRLSR outperformed partial least squares （PLS） significantly, and NIR online analysis was applicable.展开更多
We consider a wide range of non-convex regularized minimization problems, where the non-convex regularization term is composite with a linear function engaged in sparse learning. Recent theoretical investigations have...We consider a wide range of non-convex regularized minimization problems, where the non-convex regularization term is composite with a linear function engaged in sparse learning. Recent theoretical investigations have demonstrated their superiority over their convex counterparts. The computational challenge lies in the fact that the proximal mapping associated with non-convex regularization is not easily obtained due to the imposed linear composition. Fortunately, the problem structure allows one to introduce an auxiliary variable and reformulate it as an optimization problem with linear constraints, which can be solved using the Linearized Alternating Direction Method of Multipliers （LADMM）. Despite the success of LADMM in practice, it remains unknown whether LADMM is convergent in solving such non-convex compositely regularized optimizations. In this research, we first present a detailed convergence analysis of the LADMM algorithm for solving a non-convex compositely regularized optimization problem with a large class of non-convex penalties. Furthermore, we propose an Adaptive LADMM （AdaLADMM） algorithm with a line-search criterion. Experimental results on different genres of datasets validate the efficacy of the proposed algorithm.展开更多
In this study, we propose and compare stochastic variants of the extra-gradient alternating direction method, named the stochastic extra-gradient alternating direction method with Lagrangian function（SEGL） and the s...In this study, we propose and compare stochastic variants of the extra-gradient alternating direction method, named the stochastic extra-gradient alternating direction method with Lagrangian function（SEGL） and the stochastic extra-gradient alternating direction method with augmented Lagrangian function（SEGAL）, to minimize the graph-guided optimization problems, which are composited with two convex objective functions in large scale.A number of important applications in machine learning follow the graph-guided optimization formulation, such as linear regression, logistic regression, Lasso, structured extensions of Lasso, and structured regularized logistic regression. We conduct experiments on fused logistic regression and graph-guided regularized regression. Experimental results on several genres of datasets demonstrate that the proposed algorithm outperforms other competing algorithms, and SEGAL has better performance than SEGL in practical use.展开更多
We propose a new technique for reconstructing surfaces from a large set of unorganized 3D data points and their associated normal vectors. The surface is represented as the zero level set of an implicit volume model w...We propose a new technique for reconstructing surfaces from a large set of unorganized 3D data points and their associated normal vectors. The surface is represented as the zero level set of an implicit volume model which fits the data points and normal constraints. Compared with variational implicit surfaces, we make use of surface normal vectors at data points directly in the implicit model and avoid of introducing manufactured off-surface points. Given n surface point/normal pairs, the proposed method only needs to solve an n×n positive definite linear system. It allows fitting large datasets effectively and robustly. We demonstrate the performance of the proposed method with both globally supported and compactly supported radial basis functions on several datasets.展开更多
We consider the problem of minimizing the average of a large number of smooth component functions over one smooth inequality constraint.We propose and analyze a stochastic Moving Balls Approximation(SMBA)method.Like s...We consider the problem of minimizing the average of a large number of smooth component functions over one smooth inequality constraint.We propose and analyze a stochastic Moving Balls Approximation(SMBA)method.Like stochastic gradient(SG)met hods,the SMBA method's iteration cost is independent of the number of component functions and by exploiting the smoothness of the constraint function,our method can be easily implemented.Theoretical and computational properties of SMBA are studied,and convergence results are established.Numerical experiments indicate that our algorithm dramatically outperforms the existing Moving Balls Approximation algorithm(MBA)for the structure of our problem.展开更多
基金supported by National Natural Science Foundation of China (Grant Nos. 61272023 and 61075054)
文摘This paper addresses the learning algorithm on the unit sphere.The main purpose is to present an error analysis for regression generated by regularized least square algorithms with spherical harmonics kernel.The excess error can be estimated by the sum of sample errors and regularization errors.Our study shows that by introducing a suitable spherical harmonics kernel,the regularization parameter can decrease arbitrarily fast with the sample size.
文摘In view of the composition analysis and identification of ancient glass products, L1 regularization, K-Means cluster analysis, elbow rule and other methods were comprehensively used to build logical regression, cluster analysis, hyper-parameter test and other models, and SPSS, Python and other tools were used to obtain the classification rules of glass products under different fluxes, sub classification under different chemical compositions, hyper-parameter K value test and rationality analysis. Research can provide theoretical support for the protection and restoration of ancient glass relics.
文摘This study examines the relationship between macroeconomic variables and stock price indices of four prominent OPEC oil-exporting members.Bayesian model averaging(BMA)and regularized linear regression(RLR)are employed to address uncertainties arising from different estimation models and variable selection.Jointness is utilized to determine the nature of relationships among variable pairs.The case study spans macroeconomic variables and stock prices from 1996 to 2018.BMA findings reveal a strong positive association between stock price indices and both consumer price index(CPI)and broad money growth in each analyzed OPEC country.Additionally,the study suggests a weak negative correlation between OPEC oil prices and the stock price index.RLR results align with BMA analysis,offering insights valuable for policymakers and international wealth managers.
文摘A novel near infrared （NIR） modeling method-Laplacian regularized least squares regression （LapRLSR） was presented,which can take the advantage of many unlabeled spectra to promote the prediction performance of the model even if there are only few calibration samples. Using LapRLSR modeling, NIR spectral analysis was applied to the online monitoring of the concentration of salvia acid B in the column separation of Salvianolate. The results demonstrated that LapRLSR outperformed partial least squares （PLS） significantly, and NIR online analysis was applicable.
基金supported by the National Natural Science Foundation of China(Nos.61303264,61202482,and 61202488)Guangxi Cooperative Innovation Center of Cloud Computing and Big Data(No.YD16505)Distinguished Young Scientist Promotion of National University of Defense Technology
文摘We consider a wide range of non-convex regularized minimization problems, where the non-convex regularization term is composite with a linear function engaged in sparse learning. Recent theoretical investigations have demonstrated their superiority over their convex counterparts. The computational challenge lies in the fact that the proximal mapping associated with non-convex regularization is not easily obtained due to the imposed linear composition. Fortunately, the problem structure allows one to introduce an auxiliary variable and reformulate it as an optimization problem with linear constraints, which can be solved using the Linearized Alternating Direction Method of Multipliers （LADMM）. Despite the success of LADMM in practice, it remains unknown whether LADMM is convergent in solving such non-convex compositely regularized optimizations. In this research, we first present a detailed convergence analysis of the LADMM algorithm for solving a non-convex compositely regularized optimization problem with a large class of non-convex penalties. Furthermore, we propose an Adaptive LADMM （AdaLADMM） algorithm with a line-search criterion. Experimental results on different genres of datasets validate the efficacy of the proposed algorithm.
基金supported by the National Natural Science Foundation of China(No.61303264)the National Key Research and Development Program of China(No.2016YFB1000401)
文摘In this study, we propose and compare stochastic variants of the extra-gradient alternating direction method, named the stochastic extra-gradient alternating direction method with Lagrangian function（SEGL） and the stochastic extra-gradient alternating direction method with augmented Lagrangian function（SEGAL）, to minimize the graph-guided optimization problems, which are composited with two convex objective functions in large scale.A number of important applications in machine learning follow the graph-guided optimization formulation, such as linear regression, logistic regression, Lasso, structured extensions of Lasso, and structured regularized logistic regression. We conduct experiments on fused logistic regression and graph-guided regularized regression. Experimental results on several genres of datasets demonstrate that the proposed algorithm outperforms other competing algorithms, and SEGAL has better performance than SEGL in practical use.
基金Supported by the National Basic Research Program of China (Grant No.2006CB303102)the National Natural Science Foundation of China (Grant No.60703028)
文摘We propose a new technique for reconstructing surfaces from a large set of unorganized 3D data points and their associated normal vectors. The surface is represented as the zero level set of an implicit volume model which fits the data points and normal constraints. Compared with variational implicit surfaces, we make use of surface normal vectors at data points directly in the implicit model and avoid of introducing manufactured off-surface points. Given n surface point/normal pairs, the proposed method only needs to solve an n×n positive definite linear system. It allows fitting large datasets effectively and robustly. We demonstrate the performance of the proposed method with both globally supported and compactly supported radial basis functions on several datasets.
文摘We consider the problem of minimizing the average of a large number of smooth component functions over one smooth inequality constraint.We propose and analyze a stochastic Moving Balls Approximation(SMBA)method.Like stochastic gradient(SG)met hods,the SMBA method's iteration cost is independent of the number of component functions and by exploiting the smoothness of the constraint function,our method can be easily implemented.Theoretical and computational properties of SMBA are studied,and convergence results are established.Numerical experiments indicate that our algorithm dramatically outperforms the existing Moving Balls Approximation algorithm(MBA)for the structure of our problem.
