Kernel-based methods work by embedding the data into a feature space and then searching linear hypothesis among the embedding data points. The performance is mostly affected by which kernel is used. A promising way is...Kernel-based methods work by embedding the data into a feature space and then searching linear hypothesis among the embedding data points. The performance is mostly affected by which kernel is used. A promising way is to learn the kernel from the data automatically. A general regularized risk functional （RRF） criterion for kernel matrix learning is proposed. Compared with the RRF criterion, general RRF criterion takes into account the geometric distributions of the embedding data points. It is proven that the distance between different geometric distdbutions can be estimated by their centroid distance in the reproducing kernel Hilbert space. Using this criterion for kernel matrix learning leads to a convex quadratically constrained quadratic programming （QCQP） problem. For several commonly used loss functions, their mathematical formulations are given. Experiment results on a collection of benchmark data sets demonstrate the effectiveness of the proposed method.展开更多
Latent factor(LF)models are highly effective in extracting useful knowledge from High-Dimensional and Sparse(HiDS)matrices which are commonly seen in various industrial applications.An LF model usually adopts iterativ...Latent factor(LF)models are highly effective in extracting useful knowledge from High-Dimensional and Sparse(HiDS)matrices which are commonly seen in various industrial applications.An LF model usually adopts iterative optimizers,which may consume many iterations to achieve a local optima,resulting in considerable time cost.Hence,determining how to accelerate the training process for LF models has become a significant issue.To address this,this work proposes a randomized latent factor(RLF)model.It incorporates the principle of randomized learning techniques from neural networks into the LF analysis of HiDS matrices,thereby greatly alleviating computational burden.It also extends a standard learning process for randomized neural networks in context of LF analysis to make the resulting model represent an HiDS matrix correctly.Experimental results on three HiDS matrices from industrial applications demonstrate that compared with state-of-the-art LF models,RLF is able to achieve significantly higher computational efficiency and comparable prediction accuracy for missing data.I provides an important alternative approach to LF analysis of HiDS matrices,which is especially desired for industrial applications demanding highly efficient models.展开更多
High-dimensional and sparse(HiDS)matrices commonly arise in various industrial applications,e.g.,recommender systems(RSs),social networks,and wireless sensor networks.Since they contain rich information,how to accurat...High-dimensional and sparse(HiDS)matrices commonly arise in various industrial applications,e.g.,recommender systems(RSs),social networks,and wireless sensor networks.Since they contain rich information,how to accurately represent them is of great significance.A latent factor(LF)model is one of the most popular and successful ways to address this issue.Current LF models mostly adopt L2-norm-oriented Loss to represent an HiDS matrix,i.e.,they sum the errors between observed data and predicted ones with L2-norm.Yet L2-norm is sensitive to outlier data.Unfortunately,outlier data usually exist in such matrices.For example,an HiDS matrix from RSs commonly contains many outlier ratings due to some heedless/malicious users.To address this issue,this work proposes a smooth L1-norm-oriented latent factor(SL-LF)model.Its main idea is to adopt smooth L1-norm rather than L2-norm to form its Loss,making it have both strong robustness and high accuracy in predicting the missing data of an HiDS matrix.Experimental results on eight HiDS matrices generated by industrial applications verify that the proposed SL-LF model not only is robust to the outlier data but also has significantly higher prediction accuracy than state-of-the-art models when they are used to predict the missing data of HiDS matrices.展开更多
The kernel energy method(KEM) has been shown to provide fast and accurate molecular energy calculations for molecules at their equilibrium geometries.KEM breaks a molecule into smaller subsets,called kernels,for the p...The kernel energy method(KEM) has been shown to provide fast and accurate molecular energy calculations for molecules at their equilibrium geometries.KEM breaks a molecule into smaller subsets,called kernels,for the purposes of calculation.The results from the kernels are summed according to an expression characteristic of KEM to obtain the full molecule energy.A generalization of the kernel expansion to density matrices provides the full molecule density matrix and orbitals.In this study,the kernel expansion for the density matrix is examined in the context of density functional theory(DFT) Kohn-Sham(KS) calculations.A kernel expansion for the one-body density matrix analogous to the kernel expansion for energy is defined,and is then converted into a normalizedprojector by using the Clinton algorithm.Such normalized projectors are factorizable into linear combination of atomic orbitals(LCAO) matrices that deliver full-molecule Kohn-Sham molecular orbitals in the atomic orbital basis.Both straightforward KEM energies and energies from a normalized,idempotent density matrix obtained from a density matrix kernel expansion to which the Clinton algorithm has been applied are compared to reference energies obtained from calculations on the full system without any kernel expansion.Calculations were performed both for a simple proof-of-concept system consisting of three atoms in a linear configuration and for a water cluster consisting of twelve water molecules.In the case of the proof-of-concept system,calculations were performed using the STO-3 G and6-31 G(d,p) bases over a range of atomic separations,some very far from equilibrium.The water cluster was calculated in the 6-31 G(d,p) basis at an equilibrium geometry.The normalized projector density energies are more accurate than the straightforward KEM energy results in nearly all cases.In the case of the water cluster,the energy of the normalized projector is approximately four times more accurate than the straightforward KEM energy result.The KS density matrices of this study are applicable to quantum crystallography.展开更多
The polar codes defined by the kernel matrix are a class of codes with low coding-decoding complexity and can achieve the Shannon limit. In this paper, a novel method to construct the 2<sup>n</sup>-dimensi...The polar codes defined by the kernel matrix are a class of codes with low coding-decoding complexity and can achieve the Shannon limit. In this paper, a novel method to construct the 2<sup>n</sup>-dimensional kernel matrix is proposed, that is based on primitive BCH codes that make use of the interception, the direct sum and adding a row and a column. For ensuring polarization of the kernel matrix, a solution is also put forward when the partial distances of the constructed kernel matrix exceed their upper bound. And the lower bound of exponent of the 2<sup>n</sup>-dimensional kernel matrix is obtained. The lower bound of exponent of our constructed kernel matrix is tighter than Gilbert-Varshamov (G-V) type, and the scaling exponent is better in the case of 16-dimensional.展开更多
The estimation of covariance matrices is very important in many fields, such as statistics. In real applications, data are frequently influenced by high dimensions and noise. However, most relevant studies are based o...The estimation of covariance matrices is very important in many fields, such as statistics. In real applications, data are frequently influenced by high dimensions and noise. However, most relevant studies are based on complete data. This paper studies the optimal estimation of high-dimensional covariance matrices based on missing and noisy sample under the norm. First, the model with sub-Gaussian additive noise is presented. The generalized sample covariance is then modified to define a hard thresholding estimator , and the minimax upper bound is derived. After that, the minimax lower bound is derived, and it is concluded that the estimator presented in this article is rate-optimal. Finally, numerical simulation analysis is performed. The result shows that for missing samples with sub-Gaussian noise, if the true covariance matrix is sparse, the hard thresholding estimator outperforms the traditional estimate method.展开更多
The paper is related to the norm estimate of Mercer kernel matrices. The lower and upper bound estimates of Rayleigh entropy numbers for some Mercer kernel matrices on [0, 1] × [0, 1] based on the Bernstein-Durrm...The paper is related to the norm estimate of Mercer kernel matrices. The lower and upper bound estimates of Rayleigh entropy numbers for some Mercer kernel matrices on [0, 1] × [0, 1] based on the Bernstein-Durrmeyer operator kernel are obtained, with which and the approximation property of the Bernstein-Durrmeyer operator the lower and upper bounds of the Rayleigh entropy number and the l2 -norm for general Mercer kernel matrices on [0, 1] x [0, 1] are provided.展开更多
We investigate the structure of a large precision matrix in Gaussian graphical models by decomposing it into a low rank component and a remainder part with sparse precision matrix.Based on the decomposition,we propose...We investigate the structure of a large precision matrix in Gaussian graphical models by decomposing it into a low rank component and a remainder part with sparse precision matrix.Based on the decomposition,we propose to estimate the large precision matrix by inverting a principal orthogonal decomposition(IPOD).The IPOD approach has appealing practical interpretations in conditional graphical models given the low rank component,and it connects to Gaussian graphical models with latent variables.Specifically,we show that the low rank component in the decomposition of the large precision matrix can be viewed as the contribution from the latent variables in a Gaussian graphical model.Compared with existing approaches for latent variable graphical models,the IPOD is conveniently feasible in practice where only inverting a low-dimensional matrix is required.To identify the number of latent variables,which is an objective of its own interest,we investigate and justify an approach by examining the ratios of adjacent eigenvalues of the sample covariance matrix?Theoretical properties,numerical examples,and a real data application demonstrate the merits of the IPOD approach in its convenience,performance,and interpretability.展开更多
Most financial signals show time dependency that,combined with noisy and extreme events,poses serious problems in the parameter estimations of statistical models.Moreover,when addressing asset pricing,portfolio select...Most financial signals show time dependency that,combined with noisy and extreme events,poses serious problems in the parameter estimations of statistical models.Moreover,when addressing asset pricing,portfolio selection,and investment strategies,accurate estimates of the relationship among assets are as necessary as are delicate in a time-dependent context.In this regard,fundamental tools that increasingly attract research interests are precision matrix and graphical models,which are able to obtain insights into the joint evolution of financial quantities.In this paper,we present a robust divergence estimator for a time-varying precision matrix that can manage both the extreme events and time-dependency that affect financial time series.Furthermore,we provide an algorithm to handle parameter estimations that uses the“maximization–minimization”approach.We apply the methodology to synthetic data to test its performances.Then,we consider the cryptocurrency market as a real data application,given its remarkable suitability for the proposed method because of its volatile and unregulated nature.展开更多
This paper investigates the nonlinear boundary value problem resulting from the exact reduction of the Navier-Stokes equations for unsteady magnetohydrodynamic boundary layer flow over the stretching/shrinking permeab...This paper investigates the nonlinear boundary value problem resulting from the exact reduction of the Navier-Stokes equations for unsteady magnetohydrodynamic boundary layer flow over the stretching/shrinking permeable sheet submerged in a moving fluid.To solve this equation,a numerical method is proposed based on a Laguerre functions with reproducing kernel Hilbert space method.Using the operational matrices of derivative,we reduced the problem to a set of algebraic equations.We also compare this work with some other numerical results and present a solution that proves to be highly accurate.展开更多
Driven by the challenge of integrating large amount of experimental data, classification technique emerges as one of the major and popular tools in computational biology and bioinformatics research. Machine learning m...Driven by the challenge of integrating large amount of experimental data, classification technique emerges as one of the major and popular tools in computational biology and bioinformatics research. Machine learning methods, especially kernel methods with Support Vector Machines （SVMs） are very popular and effective tools. In the perspective of kernel matrix, a technique namely Eigen- matrix translation has been introduced for protein data classification. The Eigen-matrix translation strategy has a lot of nice properties which deserve more exploration. This paper investigates the major role of Eigen-matrix translation in classification. The authors propose that its importance lies in the dimension reduction of predictor attributes within the data set. This is very important when the dimension of features is huge. The authors show by numerical experiments on real biological data sets that the proposed framework is crucial and effective in improving classification accuracy. This can therefore serve as a novel perspective for future research in dimension reduction problems.展开更多
稀疏线性方程组求解等高性能计算应用常常涉及稀疏矩阵向量乘(SpMV)序列Ax,A2x,…,Asx的计算.上述SpMV序列操作又称为稀疏矩阵幂函数(matrix power kernel,MPK).由于MPK执行多次SpMV且稀疏矩阵保持不变,在缓存(cache)中重用稀疏矩阵,可...稀疏线性方程组求解等高性能计算应用常常涉及稀疏矩阵向量乘(SpMV)序列Ax,A2x,…,Asx的计算.上述SpMV序列操作又称为稀疏矩阵幂函数(matrix power kernel,MPK).由于MPK执行多次SpMV且稀疏矩阵保持不变,在缓存(cache)中重用稀疏矩阵,可避免每次执行SpMV均从主存加载A,从而缓解SpMV访存受限问题,提升MPK性能.但缓存数据重用会导致相邻SpMV操作之间的数据依赖,现有MPK优化多针对单次SpMV调用,或在实现数据重用时引入过多额外开销.提出了缓存感知的MPK(cache-awareMPK,Ca-MPK),基于稀疏矩阵的依赖图,设计了体系结构感知的递归划分方法,将依赖图划分为适合缓存大小的子图/子矩阵,通过构建分割子图解耦数据依赖,根据特定顺序在子矩阵上调度执行SpMV,实现缓存数据重用.测试结果表明,Ca-MPK相对于Intel OneMKL库和最新MPK实现,平均性能提升分别多达约1.57倍和1.40倍.展开更多
基金supported by the National Natural Science Fundation of China (60736021)the Joint Funds of NSFC-Guangdong Province(U0735003)
文摘Kernel-based methods work by embedding the data into a feature space and then searching linear hypothesis among the embedding data points. The performance is mostly affected by which kernel is used. A promising way is to learn the kernel from the data automatically. A general regularized risk functional （RRF） criterion for kernel matrix learning is proposed. Compared with the RRF criterion, general RRF criterion takes into account the geometric distributions of the embedding data points. It is proven that the distance between different geometric distdbutions can be estimated by their centroid distance in the reproducing kernel Hilbert space. Using this criterion for kernel matrix learning leads to a convex quadratically constrained quadratic programming （QCQP） problem. For several commonly used loss functions, their mathematical formulations are given. Experiment results on a collection of benchmark data sets demonstrate the effectiveness of the proposed method.
基金supported in part by the National Natural Science Foundation of China (6177249391646114)+1 种基金Chongqing research program of technology innovation and application (cstc2017rgzn-zdyfX0020)in part by the Pioneer Hundred Talents Program of Chinese Academy of Sciences
文摘Latent factor(LF)models are highly effective in extracting useful knowledge from High-Dimensional and Sparse(HiDS)matrices which are commonly seen in various industrial applications.An LF model usually adopts iterative optimizers,which may consume many iterations to achieve a local optima,resulting in considerable time cost.Hence,determining how to accelerate the training process for LF models has become a significant issue.To address this,this work proposes a randomized latent factor(RLF)model.It incorporates the principle of randomized learning techniques from neural networks into the LF analysis of HiDS matrices,thereby greatly alleviating computational burden.It also extends a standard learning process for randomized neural networks in context of LF analysis to make the resulting model represent an HiDS matrix correctly.Experimental results on three HiDS matrices from industrial applications demonstrate that compared with state-of-the-art LF models,RLF is able to achieve significantly higher computational efficiency and comparable prediction accuracy for missing data.I provides an important alternative approach to LF analysis of HiDS matrices,which is especially desired for industrial applications demanding highly efficient models.
基金supported in part by the National Natural Science Foundation of China(61702475,61772493,61902370,62002337)in part by the Natural Science Foundation of Chongqing,China(cstc2019jcyj-msxmX0578,cstc2019jcyjjqX0013)+1 种基金in part by the Chinese Academy of Sciences“Light of West China”Program,in part by the Pioneer Hundred Talents Program of Chinese Academy of Sciencesby Technology Innovation and Application Development Project of Chongqing,China(cstc2019jscx-fxydX0027)。
文摘High-dimensional and sparse(HiDS)matrices commonly arise in various industrial applications,e.g.,recommender systems(RSs),social networks,and wireless sensor networks.Since they contain rich information,how to accurately represent them is of great significance.A latent factor(LF)model is one of the most popular and successful ways to address this issue.Current LF models mostly adopt L2-norm-oriented Loss to represent an HiDS matrix,i.e.,they sum the errors between observed data and predicted ones with L2-norm.Yet L2-norm is sensitive to outlier data.Unfortunately,outlier data usually exist in such matrices.For example,an HiDS matrix from RSs commonly contains many outlier ratings due to some heedless/malicious users.To address this issue,this work proposes a smooth L1-norm-oriented latent factor(SL-LF)model.Its main idea is to adopt smooth L1-norm rather than L2-norm to form its Loss,making it have both strong robustness and high accuracy in predicting the missing data of an HiDS matrix.Experimental results on eight HiDS matrices generated by industrial applications verify that the proposed SL-LF model not only is robust to the outlier data but also has significantly higher prediction accuracy than state-of-the-art models when they are used to predict the missing data of HiDS matrices.
文摘The kernel energy method(KEM) has been shown to provide fast and accurate molecular energy calculations for molecules at their equilibrium geometries.KEM breaks a molecule into smaller subsets,called kernels,for the purposes of calculation.The results from the kernels are summed according to an expression characteristic of KEM to obtain the full molecule energy.A generalization of the kernel expansion to density matrices provides the full molecule density matrix and orbitals.In this study,the kernel expansion for the density matrix is examined in the context of density functional theory(DFT) Kohn-Sham(KS) calculations.A kernel expansion for the one-body density matrix analogous to the kernel expansion for energy is defined,and is then converted into a normalizedprojector by using the Clinton algorithm.Such normalized projectors are factorizable into linear combination of atomic orbitals(LCAO) matrices that deliver full-molecule Kohn-Sham molecular orbitals in the atomic orbital basis.Both straightforward KEM energies and energies from a normalized,idempotent density matrix obtained from a density matrix kernel expansion to which the Clinton algorithm has been applied are compared to reference energies obtained from calculations on the full system without any kernel expansion.Calculations were performed both for a simple proof-of-concept system consisting of three atoms in a linear configuration and for a water cluster consisting of twelve water molecules.In the case of the proof-of-concept system,calculations were performed using the STO-3 G and6-31 G(d,p) bases over a range of atomic separations,some very far from equilibrium.The water cluster was calculated in the 6-31 G(d,p) basis at an equilibrium geometry.The normalized projector density energies are more accurate than the straightforward KEM energy results in nearly all cases.In the case of the water cluster,the energy of the normalized projector is approximately four times more accurate than the straightforward KEM energy result.The KS density matrices of this study are applicable to quantum crystallography.
文摘The polar codes defined by the kernel matrix are a class of codes with low coding-decoding complexity and can achieve the Shannon limit. In this paper, a novel method to construct the 2<sup>n</sup>-dimensional kernel matrix is proposed, that is based on primitive BCH codes that make use of the interception, the direct sum and adding a row and a column. For ensuring polarization of the kernel matrix, a solution is also put forward when the partial distances of the constructed kernel matrix exceed their upper bound. And the lower bound of exponent of the 2<sup>n</sup>-dimensional kernel matrix is obtained. The lower bound of exponent of our constructed kernel matrix is tighter than Gilbert-Varshamov (G-V) type, and the scaling exponent is better in the case of 16-dimensional.
文摘The estimation of covariance matrices is very important in many fields, such as statistics. In real applications, data are frequently influenced by high dimensions and noise. However, most relevant studies are based on complete data. This paper studies the optimal estimation of high-dimensional covariance matrices based on missing and noisy sample under the norm. First, the model with sub-Gaussian additive noise is presented. The generalized sample covariance is then modified to define a hard thresholding estimator , and the minimax upper bound is derived. After that, the minimax lower bound is derived, and it is concluded that the estimator presented in this article is rate-optimal. Finally, numerical simulation analysis is performed. The result shows that for missing samples with sub-Gaussian noise, if the true covariance matrix is sparse, the hard thresholding estimator outperforms the traditional estimate method.
基金Supported by the Science Foundation of Zhejiang Province(Y604003)
文摘The paper is related to the norm estimate of Mercer kernel matrices. The lower and upper bound estimates of Rayleigh entropy numbers for some Mercer kernel matrices on [0, 1] × [0, 1] based on the Bernstein-Durrmeyer operator kernel are obtained, with which and the approximation property of the Bernstein-Durrmeyer operator the lower and upper bounds of the Rayleigh entropy number and the l2 -norm for general Mercer kernel matrices on [0, 1] x [0, 1] are provided.
文摘We investigate the structure of a large precision matrix in Gaussian graphical models by decomposing it into a low rank component and a remainder part with sparse precision matrix.Based on the decomposition,we propose to estimate the large precision matrix by inverting a principal orthogonal decomposition(IPOD).The IPOD approach has appealing practical interpretations in conditional graphical models given the low rank component,and it connects to Gaussian graphical models with latent variables.Specifically,we show that the low rank component in the decomposition of the large precision matrix can be viewed as the contribution from the latent variables in a Gaussian graphical model.Compared with existing approaches for latent variable graphical models,the IPOD is conveniently feasible in practice where only inverting a low-dimensional matrix is required.To identify the number of latent variables,which is an objective of its own interest,we investigate and justify an approach by examining the ratios of adjacent eigenvalues of the sample covariance matrix?Theoretical properties,numerical examples,and a real data application demonstrate the merits of the IPOD approach in its convenience,performance,and interpretability.
文摘Most financial signals show time dependency that,combined with noisy and extreme events,poses serious problems in the parameter estimations of statistical models.Moreover,when addressing asset pricing,portfolio selection,and investment strategies,accurate estimates of the relationship among assets are as necessary as are delicate in a time-dependent context.In this regard,fundamental tools that increasingly attract research interests are precision matrix and graphical models,which are able to obtain insights into the joint evolution of financial quantities.In this paper,we present a robust divergence estimator for a time-varying precision matrix that can manage both the extreme events and time-dependency that affect financial time series.Furthermore,we provide an algorithm to handle parameter estimations that uses the“maximization–minimization”approach.We apply the methodology to synthetic data to test its performances.Then,we consider the cryptocurrency market as a real data application,given its remarkable suitability for the proposed method because of its volatile and unregulated nature.
文摘This paper investigates the nonlinear boundary value problem resulting from the exact reduction of the Navier-Stokes equations for unsteady magnetohydrodynamic boundary layer flow over the stretching/shrinking permeable sheet submerged in a moving fluid.To solve this equation,a numerical method is proposed based on a Laguerre functions with reproducing kernel Hilbert space method.Using the operational matrices of derivative,we reduced the problem to a set of algebraic equations.We also compare this work with some other numerical results and present a solution that proves to be highly accurate.
文摘We exploit the theory of reproducing kernels to deduce a matrix inequality for the inverse of the restriction of a positive definite Hermitian matrix.
基金supported by Research Grants Council of Hong Kong under Grant No.17301214HKU CERG Grants,Fundamental Research Funds for the Central Universities+2 种基金the Research Funds of Renmin University of ChinaHung Hing Ying Physical Research Grantthe Natural Science Foundation of China under Grant No.11271144
文摘Driven by the challenge of integrating large amount of experimental data, classification technique emerges as one of the major and popular tools in computational biology and bioinformatics research. Machine learning methods, especially kernel methods with Support Vector Machines （SVMs） are very popular and effective tools. In the perspective of kernel matrix, a technique namely Eigen- matrix translation has been introduced for protein data classification. The Eigen-matrix translation strategy has a lot of nice properties which deserve more exploration. This paper investigates the major role of Eigen-matrix translation in classification. The authors propose that its importance lies in the dimension reduction of predictor attributes within the data set. This is very important when the dimension of features is huge. The authors show by numerical experiments on real biological data sets that the proposed framework is crucial and effective in improving classification accuracy. This can therefore serve as a novel perspective for future research in dimension reduction problems.
文摘稀疏线性方程组求解等高性能计算应用常常涉及稀疏矩阵向量乘(SpMV)序列Ax,A2x,…,Asx的计算.上述SpMV序列操作又称为稀疏矩阵幂函数(matrix power kernel,MPK).由于MPK执行多次SpMV且稀疏矩阵保持不变,在缓存(cache)中重用稀疏矩阵,可避免每次执行SpMV均从主存加载A,从而缓解SpMV访存受限问题,提升MPK性能.但缓存数据重用会导致相邻SpMV操作之间的数据依赖,现有MPK优化多针对单次SpMV调用,或在实现数据重用时引入过多额外开销.提出了缓存感知的MPK(cache-awareMPK,Ca-MPK),基于稀疏矩阵的依赖图,设计了体系结构感知的递归划分方法,将依赖图划分为适合缓存大小的子图/子矩阵,通过构建分割子图解耦数据依赖,根据特定顺序在子矩阵上调度执行SpMV,实现缓存数据重用.测试结果表明,Ca-MPK相对于Intel OneMKL库和最新MPK实现,平均性能提升分别多达约1.57倍和1.40倍.
