This paper deals with the statistical modeling of latent topic hierarchies in text corpora. The height of the topic tree is assumed as fixed, while the number of topics on each level as unknown a priori and to be infe...This paper deals with the statistical modeling of latent topic hierarchies in text corpora. The height of the topic tree is assumed as fixed, while the number of topics on each level as unknown a priori and to be inferred from data. Taking a nonpara-metric Bayesian approach to this problem, we propose a new probabilistic generative model based on the nested hierarchical Dirichlet process (nHDP) and present a Markov chain Monte Carlo sampling algorithm for the inference of the topic tree structure as well as the word distribution of each topic and topic distribution of each document. Our theoretical analysis and experiment results show that this model can produce a more compact hierarchical topic structure and captures more fine-grained topic rela-tionships compared to the hierarchical latent Dirichlet allocation model.展开更多
Accurate classification and prediction of future traffic conditions are essential for developing effective strategies for congestion mitigation on the highway systems. Speed distribution is one of the traffic stream p...Accurate classification and prediction of future traffic conditions are essential for developing effective strategies for congestion mitigation on the highway systems. Speed distribution is one of the traffic stream parameters, which has been used to quantify the traffic conditions. Previous studies have shown that multi-modal probability distribution of speeds gives excellent results when simultaneously evaluating congested and free-flow traffic conditions. However, most of these previous analytical studies do not incorporate the influencing factors in characterizing these conditions. This study evaluates the impact of traffic occupancy on the multi-state speed distribution using the Bayesian Dirichlet Process Mixtures of Generalized Linear Models (DPM-GLM). Further, the study estimates the speed cut-point values of traffic states, which separate them into homogeneous groups using Bayesian change-point detection (BCD) technique. The study used 2015 archived one-year traffic data collected on Florida’s Interstate 295 freeway corridor. Information criteria results revealed three traffic states, which were identified as free-flow, transitional flow condition (congestion onset/offset), and the congested condition. The findings of the DPM-GLM indicated that in all estimated states, the traffic speed decreases when traffic occupancy increases. Comparison of the influence of traffic occupancy between traffic states showed that traffic occupancy has more impact on the free-flow and the congested state than on the transitional flow condition. With respect to estimating the threshold speed value, the results of the BCD model revealed promising findings in characterizing levels of traffic congestion.展开更多
In this paper, Spike-and-Slab Dirichlet Process (SS-DP) priors are introduced and discussed for non-parametric Bayesian modeling and inference, especially in the mixture models context. Specifying a spike-and-slab bas...In this paper, Spike-and-Slab Dirichlet Process (SS-DP) priors are introduced and discussed for non-parametric Bayesian modeling and inference, especially in the mixture models context. Specifying a spike-and-slab base measure for DP priors combines the merits of Dirichlet process and spike-and-slab priors and serves as a flexible approach in Bayesian model selection and averaging. Computationally, Bayesian Expectation-Maximization (BEM) is utilized to obtain MAP estimates. Two simulated examples in mixture modeling and time series analysis contexts demonstrate the models and computational methodology.展开更多
When estimating the direction of arrival (DOA) of wideband signals from multiple sources, the performance of sparse Bayesian methods is influenced by the frequency bands occupied by signals in different directions. Th...When estimating the direction of arrival (DOA) of wideband signals from multiple sources, the performance of sparse Bayesian methods is influenced by the frequency bands occupied by signals in different directions. This is particularly true when multiple signal frequency bands overlap. Message passing algorithms (MPA) with Dirichlet process (DP) prior can be employed in a sparse Bayesian learning (SBL) framework with high precision. However, existing methods suffer from either high complexity or low precision. To address this, we propose a low-complexity DOA estimation algorithm based on a factor graph. This approach introduces two strong constraints via a stretching transformation of the factor graph. The first constraint separates the observation from the DP prior, enabling the application of the unitary approximate message passing (UAMP) algorithm for simplified inference and mitigation of divergence issues. The second constraint compensates for the deviation in estimation angle caused by the grid mismatch problem. Compared to state-of-the-art algorithms, our proposed method offers higher estimation accuracy and lower complexity.展开更多
The core of the nonparametric/semiparametric Bayesian analysis is to relax the particular parametric assumptions on the distributions of interest to be unknown and random,and assign them a prior.Selecting a suitable p...The core of the nonparametric/semiparametric Bayesian analysis is to relax the particular parametric assumptions on the distributions of interest to be unknown and random,and assign them a prior.Selecting a suitable prior therefore is especially critical in the nonparametric Bayesian fitting.As the distribution of distribution,Dirichlet process(DP)is the most appreciated nonparametric prior due to its nice theoretical proprieties,modeling flexibility and computational feasibility.In this paper,we review and summarize some developments of DP during the past decades.Our focus is mainly concentrated upon its theoretical properties,various extensions,statistical modeling and applications to the latent variable models.展开更多
In the Bayesian mixture modeling framework it is possible to infer the necessary number of components to model the data and therefore it is unnecessary to explicitly restrict the number of components. Nonparametric mi...In the Bayesian mixture modeling framework it is possible to infer the necessary number of components to model the data and therefore it is unnecessary to explicitly restrict the number of components. Nonparametric mixture models sidestep the problem of finding the ＂correct＂ number of mixture components by assuming infinitely many components. In this paper Dirichlet process mixture （DPM） models are cast as infinite mixture models and inference using Markov chain Monte Carlo is described. The specification of the priors on the model parameters is often guided by mathematical and practical convenience. The primary goal of this paper is to compare the choice of conjugate and non-conjugate base distributions on a particular class of DPM models which is widely used in applications, the Dirichlet process Gaussian mixture model （DPGMM）. We compare computational efficiency and modeling performance of DPGMM defined using a conjugate and a conditionally conjugate base distribution. We show that better density models can result from using a wider class of priors with no or only a modest increase in computational effort.展开更多
In this paper,a nonparametric Bayesian graph topic model(GTM)based on hierarchical Dirichlet process(HDP)is proposed.The HDP makes the number of topics selected flexibly,which breaks the limitation that the number of ...In this paper,a nonparametric Bayesian graph topic model(GTM)based on hierarchical Dirichlet process(HDP)is proposed.The HDP makes the number of topics selected flexibly,which breaks the limitation that the number of topics need to be given in advance.Moreover,theGTMreleases the assumption of‘bag of words’and considers the graph structure of the text.The combination of HDP and GTM takes advantage of both which is named as HDP–GTM.The variational inference algorithm is used for the posterior inference and the convergence of the algorithm is analysed.We apply the proposed model in text categorisation,comparing to three related topic models,latent Dirichlet allocation(LDA),GTM and HDP.展开更多
In order to meet the real-time performance requirements,intelligent decisions in Internet of things applications must take place right here right now at the network edge.Pushing the artificial intelligence frontier to...In order to meet the real-time performance requirements,intelligent decisions in Internet of things applications must take place right here right now at the network edge.Pushing the artificial intelligence frontier to achieve edge intelligence is nontrivial due to the constrained computing resources and limited training data at the network edge.To tackle these challenges,we develop a distributionally robust optimization(DRO)-based edge learning algorithm,where the uncertainty model is constructed to foster the synergy of cloud knowledge and local training.Specifically,the cloud transferred knowledge is in the form of a Dirichlet process prior distribution for the edge model parameters,and the edge device further constructs an uncertainty set centered around the empirical distribution of its local samples.The edge learning DRO problem,subject to these two distributional uncertainty constraints,is recast as a single-layer optimization problem using a duality approach.We then use an Expectation-Maximization algorithm-inspired method to derive a convex relaxation,based on which we devise algorithms to learn the edge model.Furthermore,we illustrate that the meta-learning fast adaptation procedure is equivalent to our proposed Dirichlet process prior-based approach.Finally,extensive experiments are implemented to showcase the performance gain over standard approaches using edge data only.展开更多
针对高铁列车运行数据中异常样本难以实时识别和聚类结构随数据演化动态变化等问题,本文提出一种基于狄利克雷过程混合模型的后验归类式增量聚类与异常检测方法(Posterior Classification-based Incremental Dirichlet Process Mixture ...针对高铁列车运行数据中异常样本难以实时识别和聚类结构随数据演化动态变化等问题,本文提出一种基于狄利克雷过程混合模型的后验归类式增量聚类与异常检测方法(Posterior Classification-based Incremental Dirichlet Process Mixture Model,PC-IDPMM)。该方法在离线阶段构建聚类模型并识别异常样本,在线阶段结合后验概率快速归类新样本,并通过密度聚类提取新结构,实现模型的结构扩展与参数更新。为验证模型性能,本文基于广深高铁实测数据开展实验。结果表明:PC-IDPMM在保持聚类结构一致性的同时,实现主簇统计特征的稳定更新,AUC(Area Under the Curve)达90.55%,优于多种离线方法;计算效率方面,训练时间与内存消耗较离线模型分别减少约85%和80%。此外,PC-IDPMM可基于列车前序站点数据实现实时异常预警,辅助调度系统在延误初期干预,将累计晚点由572 min降至320 min,实验结果验证了该方法在高频数据环境下的实时性与应用价值。展开更多
基金Project (No. 60773180) supported by the National Natural Science Foundation of China
文摘This paper deals with the statistical modeling of latent topic hierarchies in text corpora. The height of the topic tree is assumed as fixed, while the number of topics on each level as unknown a priori and to be inferred from data. Taking a nonpara-metric Bayesian approach to this problem, we propose a new probabilistic generative model based on the nested hierarchical Dirichlet process (nHDP) and present a Markov chain Monte Carlo sampling algorithm for the inference of the topic tree structure as well as the word distribution of each topic and topic distribution of each document. Our theoretical analysis and experiment results show that this model can produce a more compact hierarchical topic structure and captures more fine-grained topic rela-tionships compared to the hierarchical latent Dirichlet allocation model.
文摘Accurate classification and prediction of future traffic conditions are essential for developing effective strategies for congestion mitigation on the highway systems. Speed distribution is one of the traffic stream parameters, which has been used to quantify the traffic conditions. Previous studies have shown that multi-modal probability distribution of speeds gives excellent results when simultaneously evaluating congested and free-flow traffic conditions. However, most of these previous analytical studies do not incorporate the influencing factors in characterizing these conditions. This study evaluates the impact of traffic occupancy on the multi-state speed distribution using the Bayesian Dirichlet Process Mixtures of Generalized Linear Models (DPM-GLM). Further, the study estimates the speed cut-point values of traffic states, which separate them into homogeneous groups using Bayesian change-point detection (BCD) technique. The study used 2015 archived one-year traffic data collected on Florida’s Interstate 295 freeway corridor. Information criteria results revealed three traffic states, which were identified as free-flow, transitional flow condition (congestion onset/offset), and the congested condition. The findings of the DPM-GLM indicated that in all estimated states, the traffic speed decreases when traffic occupancy increases. Comparison of the influence of traffic occupancy between traffic states showed that traffic occupancy has more impact on the free-flow and the congested state than on the transitional flow condition. With respect to estimating the threshold speed value, the results of the BCD model revealed promising findings in characterizing levels of traffic congestion.
文摘In this paper, Spike-and-Slab Dirichlet Process (SS-DP) priors are introduced and discussed for non-parametric Bayesian modeling and inference, especially in the mixture models context. Specifying a spike-and-slab base measure for DP priors combines the merits of Dirichlet process and spike-and-slab priors and serves as a flexible approach in Bayesian model selection and averaging. Computationally, Bayesian Expectation-Maximization (BEM) is utilized to obtain MAP estimates. Two simulated examples in mixture modeling and time series analysis contexts demonstrate the models and computational methodology.
基金supported in part by the National Natural Science Foundation of China(Nos.6202780103 and 62033001)the Innovation Key Project of Guangxi Province(No.AA22068059)+2 种基金the Key Research and Development Program of Guilin(No.2020010332)the Natural Science Foundation of Henan Province(No.222300420504)Academic Degrees and Graduate Education Reform Project of Henan Province(No.2021SJGLX262Y).
文摘When estimating the direction of arrival (DOA) of wideband signals from multiple sources, the performance of sparse Bayesian methods is influenced by the frequency bands occupied by signals in different directions. This is particularly true when multiple signal frequency bands overlap. Message passing algorithms (MPA) with Dirichlet process (DP) prior can be employed in a sparse Bayesian learning (SBL) framework with high precision. However, existing methods suffer from either high complexity or low precision. To address this, we propose a low-complexity DOA estimation algorithm based on a factor graph. This approach introduces two strong constraints via a stretching transformation of the factor graph. The first constraint separates the observation from the DP prior, enabling the application of the unitary approximate message passing (UAMP) algorithm for simplified inference and mitigation of divergence issues. The second constraint compensates for the deviation in estimation angle caused by the grid mismatch problem. Compared to state-of-the-art algorithms, our proposed method offers higher estimation accuracy and lower complexity.
基金supported in part by the National Natural Science Foundation of China(Grant No.11471161)the Technological Innovation Item in Jiangsu Province(No.BK2008156).
文摘The core of the nonparametric/semiparametric Bayesian analysis is to relax the particular parametric assumptions on the distributions of interest to be unknown and random,and assign them a prior.Selecting a suitable prior therefore is especially critical in the nonparametric Bayesian fitting.As the distribution of distribution,Dirichlet process(DP)is the most appreciated nonparametric prior due to its nice theoretical proprieties,modeling flexibility and computational feasibility.In this paper,we review and summarize some developments of DP during the past decades.Our focus is mainly concentrated upon its theoretical properties,various extensions,statistical modeling and applications to the latent variable models.
基金supported by Gatsby Charitable Foundation and PASCAL2
文摘In the Bayesian mixture modeling framework it is possible to infer the necessary number of components to model the data and therefore it is unnecessary to explicitly restrict the number of components. Nonparametric mixture models sidestep the problem of finding the ＂correct＂ number of mixture components by assuming infinitely many components. In this paper Dirichlet process mixture （DPM） models are cast as infinite mixture models and inference using Markov chain Monte Carlo is described. The specification of the priors on the model parameters is often guided by mathematical and practical convenience. The primary goal of this paper is to compare the choice of conjugate and non-conjugate base distributions on a particular class of DPM models which is widely used in applications, the Dirichlet process Gaussian mixture model （DPGMM）. We compare computational efficiency and modeling performance of DPGMM defined using a conjugate and a conditionally conjugate base distribution. We show that better density models can result from using a wider class of priors with no or only a modest increase in computational effort.
基金supported by NSFC under grant No.71371074the 111 Project under No.B14019.
文摘In this paper,a nonparametric Bayesian graph topic model(GTM)based on hierarchical Dirichlet process(HDP)is proposed.The HDP makes the number of topics selected flexibly,which breaks the limitation that the number of topics need to be given in advance.Moreover,theGTMreleases the assumption of‘bag of words’and considers the graph structure of the text.The combination of HDP and GTM takes advantage of both which is named as HDP–GTM.The variational inference algorithm is used for the posterior inference and the convergence of the algorithm is analysed.We apply the proposed model in text categorisation,comparing to three related topic models,latent Dirichlet allocation(LDA),GTM and HDP.
基金This work was supported in part by NSF under Grant CPS-1739344,ARO under grant W911NF-16-1-0448the DTRA under Grant HDTRA1-13-1-0029Part of this work will appear in the Proceedings of 40th IEEE International Conference on Distributed Computing Systems(ICDCS),Singapore,July 8-10,2020。
文摘In order to meet the real-time performance requirements,intelligent decisions in Internet of things applications must take place right here right now at the network edge.Pushing the artificial intelligence frontier to achieve edge intelligence is nontrivial due to the constrained computing resources and limited training data at the network edge.To tackle these challenges,we develop a distributionally robust optimization(DRO)-based edge learning algorithm,where the uncertainty model is constructed to foster the synergy of cloud knowledge and local training.Specifically,the cloud transferred knowledge is in the form of a Dirichlet process prior distribution for the edge model parameters,and the edge device further constructs an uncertainty set centered around the empirical distribution of its local samples.The edge learning DRO problem,subject to these two distributional uncertainty constraints,is recast as a single-layer optimization problem using a duality approach.We then use an Expectation-Maximization algorithm-inspired method to derive a convex relaxation,based on which we devise algorithms to learn the edge model.Furthermore,we illustrate that the meta-learning fast adaptation procedure is equivalent to our proposed Dirichlet process prior-based approach.Finally,extensive experiments are implemented to showcase the performance gain over standard approaches using edge data only.
文摘针对高铁列车运行数据中异常样本难以实时识别和聚类结构随数据演化动态变化等问题,本文提出一种基于狄利克雷过程混合模型的后验归类式增量聚类与异常检测方法(Posterior Classification-based Incremental Dirichlet Process Mixture Model,PC-IDPMM)。该方法在离线阶段构建聚类模型并识别异常样本,在线阶段结合后验概率快速归类新样本,并通过密度聚类提取新结构,实现模型的结构扩展与参数更新。为验证模型性能,本文基于广深高铁实测数据开展实验。结果表明:PC-IDPMM在保持聚类结构一致性的同时,实现主簇统计特征的稳定更新,AUC(Area Under the Curve)达90.55%,优于多种离线方法;计算效率方面,训练时间与内存消耗较离线模型分别减少约85%和80%。此外,PC-IDPMM可基于列车前序站点数据实现实时异常预警,辅助调度系统在延误初期干预,将累计晚点由572 min降至320 min,实验结果验证了该方法在高频数据环境下的实时性与应用价值。
