The purpose of variable selection is to identify important predictors for response variables. Although there are many varieties of variable selection methods, almost all of them have a problem of not accounting for th...The purpose of variable selection is to identify important predictors for response variables. Although there are many varieties of variable selection methods, almost all of them have a problem of not accounting for the relationship between predictors. Therefore it would well happen that the selected subset of identified predictors leads to hard-to-interpret model consisted of only interaction terms. In design of experiments, the analysis is driven by the effect heredity principle which governs the relationship between an interaction and its corresponding main effects. In this paper, the authors extend the variable selection method the Lasso with effect heredity principle to its Bayesian version. In the example, the authors analyze the data obtained from typical screening design Plackett-Bunnan design and compare the result from the ordinary Bayesian Lasso and proposed method.展开更多
In this paper, we consider the problem of estimating a high dimensional precision matrix of Gaussian graphical model. Taking advantage of the connection between multivariate linear regression and entries of the precis...In this paper, we consider the problem of estimating a high dimensional precision matrix of Gaussian graphical model. Taking advantage of the connection between multivariate linear regression and entries of the precision matrix, we propose Bayesian Lasso together with neighborhood regression estimate for Gaussian graphical model. This method can obtain parameter estimation and model selection simultaneously. Moreover, the proposed method can provide symmetric confidence intervals of all entries of the precision matrix.展开更多
Nowadays a common problem when processing data sets with the large number of covariates compared to small sample sizes (fat data sets) is to estimate the parameters associated with each covariate. When the number of c...Nowadays a common problem when processing data sets with the large number of covariates compared to small sample sizes (fat data sets) is to estimate the parameters associated with each covariate. When the number of covariates far exceeds the number of samples, the parameter estimation becomes very difficult. Researchers in many fields such as text categorization deal with the burden of finding and estimating important covariates without overfitting the model. In this study, we developed a Sparse Probit Bayesian Model (SPBM) based on Gibbs sampling which utilizes double exponentials prior to induce shrinkage and reduce the number of covariates in the model. The method was evaluated using ten domains such as mathematics, the corpuses of which were downloaded from Wikipedia. From the downloaded corpuses, we created the TFIDF matrix corresponding to all domains and divided the whole data set randomly into training and testing groups of size 300. To make the model more robust we performed 50 re-samplings on selection of training and test groups. The model was implemented in R and the Gibbs sampler ran for 60 k iterations and the first 20 k was discarded as burn in. We performed classification on training and test groups by calculating P (yi = 1) and according to [1] [2] the threshold of 0.5 was used as decision rule. Our model’s performance was compared to Support Vector Machines (SVM) using average sensitivity and specificity across 50 runs. The SPBM achieved high classification accuracy and outperformed SVM in almost all domains analyzed.展开更多
In recent years, variable selection based on penalty likelihood methods has aroused great concern. Based on the Gibbs sampling algorithm of asymmetric Laplace distribution, this paper considers the quantile regression...In recent years, variable selection based on penalty likelihood methods has aroused great concern. Based on the Gibbs sampling algorithm of asymmetric Laplace distribution, this paper considers the quantile regression with adaptive Lasso and Lasso penalty from a Bayesian point of view. Under the non-Bayesian and Bayesian framework, several regularization quantile regression methods are systematically compared for error terms with different distributions and heteroscedasticity. Under the error term of asymmetric Laplace distribution, statistical simulation results show that the Bayesian regularized quantile regression is superior to other distributions in all quantiles. And based on the asymmetric Laplace distribution, the Bayesian regularized quantile regression approach performs better than the non-Bayesian approach in parameter estimation and prediction. Through real data analyses, we also confirm the above conclusions.展开更多
文摘The purpose of variable selection is to identify important predictors for response variables. Although there are many varieties of variable selection methods, almost all of them have a problem of not accounting for the relationship between predictors. Therefore it would well happen that the selected subset of identified predictors leads to hard-to-interpret model consisted of only interaction terms. In design of experiments, the analysis is driven by the effect heredity principle which governs the relationship between an interaction and its corresponding main effects. In this paper, the authors extend the variable selection method the Lasso with effect heredity principle to its Bayesian version. In the example, the authors analyze the data obtained from typical screening design Plackett-Bunnan design and compare the result from the ordinary Bayesian Lasso and proposed method.
基金Supported by the National Natural Science Foundation of China(No.11571080)
文摘In this paper, we consider the problem of estimating a high dimensional precision matrix of Gaussian graphical model. Taking advantage of the connection between multivariate linear regression and entries of the precision matrix, we propose Bayesian Lasso together with neighborhood regression estimate for Gaussian graphical model. This method can obtain parameter estimation and model selection simultaneously. Moreover, the proposed method can provide symmetric confidence intervals of all entries of the precision matrix.
文摘为了提高稀疏信号恢复的准确性,开展了基于自适应套索算子(Least absolute shrinkage and selection operator,LASSO)先验的稀疏贝叶斯学习(Sparse Bayesian learning,SBL)算法研究.1)在稀疏贝叶斯模型构建阶段,构造了一种新的多层贝叶斯框架,赋予信号中元素独立的LASSO先验.该先验比现有稀疏先验更有效地鼓励稀疏并且该模型中所有参数更新存在闭合解.然后在该多层贝叶斯框架的基础上提出了一种基于自适应LASSO先验的SBL算法.2)为降低提出的算法的计算复杂度,在贝叶斯推断阶段利用空间轮换变元方法对提出的算法进行改进,避免了矩阵求逆运算,使参数更新快速高效,从而提出了一种基于自适应LASSO先验的快速SBL算法.本文提出的算法的稀疏恢复性能通过实验进行了验证,分别针对不同大小测量矩阵的稀疏信号恢复以及单快拍波达方向(Direction of arrival,DOA)估计开展了实验.实验结果表明:提出基于自适应LASSO先验的SBL算法比现有算法具有更高的稀疏恢复准确度;提出的快速算法的准确度略低于提出的基于自适应LASSO先验的SBL算法,但计算复杂度明显降低.
文摘Nowadays a common problem when processing data sets with the large number of covariates compared to small sample sizes (fat data sets) is to estimate the parameters associated with each covariate. When the number of covariates far exceeds the number of samples, the parameter estimation becomes very difficult. Researchers in many fields such as text categorization deal with the burden of finding and estimating important covariates without overfitting the model. In this study, we developed a Sparse Probit Bayesian Model (SPBM) based on Gibbs sampling which utilizes double exponentials prior to induce shrinkage and reduce the number of covariates in the model. The method was evaluated using ten domains such as mathematics, the corpuses of which were downloaded from Wikipedia. From the downloaded corpuses, we created the TFIDF matrix corresponding to all domains and divided the whole data set randomly into training and testing groups of size 300. To make the model more robust we performed 50 re-samplings on selection of training and test groups. The model was implemented in R and the Gibbs sampler ran for 60 k iterations and the first 20 k was discarded as burn in. We performed classification on training and test groups by calculating P (yi = 1) and according to [1] [2] the threshold of 0.5 was used as decision rule. Our model’s performance was compared to Support Vector Machines (SVM) using average sensitivity and specificity across 50 runs. The SPBM achieved high classification accuracy and outperformed SVM in almost all domains analyzed.
文摘In recent years, variable selection based on penalty likelihood methods has aroused great concern. Based on the Gibbs sampling algorithm of asymmetric Laplace distribution, this paper considers the quantile regression with adaptive Lasso and Lasso penalty from a Bayesian point of view. Under the non-Bayesian and Bayesian framework, several regularization quantile regression methods are systematically compared for error terms with different distributions and heteroscedasticity. Under the error term of asymmetric Laplace distribution, statistical simulation results show that the Bayesian regularized quantile regression is superior to other distributions in all quantiles. And based on the asymmetric Laplace distribution, the Bayesian regularized quantile regression approach performs better than the non-Bayesian approach in parameter estimation and prediction. Through real data analyses, we also confirm the above conclusions.
