Prediction intervals(PIs)for industrial time series can provide useful guidance for workers.Given that the failure of industrial sensors may cause the missing point in inputs,the existing kernel dynamic Bayesian netwo...Prediction intervals(PIs)for industrial time series can provide useful guidance for workers.Given that the failure of industrial sensors may cause the missing point in inputs,the existing kernel dynamic Bayesian networks(KDBN),serving as an effective method for PIs construction,suffer from high computational load using the stochastic algorithm for inference.This study proposes a variational inference method for the KDBN for the purpose of fast inference,which avoids the timeconsuming stochastic sampling.The proposed algorithm contains two stages.The first stage involves the inference of the missing inputs by using a local linearization based variational inference,and based on the computed posterior distributions over the missing inputs the second stage sees a Gaussian approximation for probability over the nodes in future time slices.To verify the effectiveness of the proposed method,a synthetic dataset and a practical dataset of generation flow of blast furnace gas(BFG)are employed with different ratios of missing inputs.The experimental results indicate that the proposed method can provide reliable PIs for the generation flow of BFG and it exhibits shorter computing time than the stochastic based one.展开更多
A distributed bearing-only target tracking algorithm based on variational Bayesian inference(VBI)under random measurement anomalies is proposed for the problem of adverse effect of random measurement anomalies on the ...A distributed bearing-only target tracking algorithm based on variational Bayesian inference(VBI)under random measurement anomalies is proposed for the problem of adverse effect of random measurement anomalies on the state estimation accuracy of moving targets in bearing-only tracking scenarios.Firstly,the measurement information of each sensor is complemented by using triangulation under the distributed framework.Secondly,the Student-t distribution is selected to model the measurement likelihood probability density function,and the joint posteriori probability density function of the estimated variables is approximately decoupled by VBI.Finally,the estimation results of each local filter are sent to the fusion center and fed back to each local filter.The simulation results show that the proposed distributed bearing-only target tracking algorithm based on VBI in the presence of abnormal measurement noise comprehensively considers the influence of system nonlinearity and random anomaly of measurement noise,and has higher estimation accuracy and robustness than other existing algorithms in the above scenarios.展开更多
Recently,deep learning-based semantic communication has garnered widespread attention,with numerous systems designed for transmitting diverse data sources,including text,image,and speech,etc.While efforts have been di...Recently,deep learning-based semantic communication has garnered widespread attention,with numerous systems designed for transmitting diverse data sources,including text,image,and speech,etc.While efforts have been directed toward improving system performance,many studies have concentrated on enhancing the structure of the encoder and decoder.However,this often overlooks the resulting increase in model complexity,imposing additional storage and computational burdens on smart devices.Furthermore,existing work tends to prioritize explicit semantics,neglecting the potential of implicit semantics.This paper aims to easily and effectively enhance the receiver's decoding capability without modifying the encoder and decoder structures.We propose a novel semantic communication system with variational neural inference for text transmission.Specifically,we introduce a simple but effective variational neural inferer at the receiver to infer the latent semantic information within the received text.This information is then utilized to assist in the decoding process.The simulation results show a significant enhancement in system performance and improved robustness.展开更多
Due to the increasing demand for goods movement,externalities from freight mobility have attracted much concern among local citizens and policymakers.Freight truck-related crash is one of these externalities and impac...Due to the increasing demand for goods movement,externalities from freight mobility have attracted much concern among local citizens and policymakers.Freight truck-related crash is one of these externalities and impacts urban freight transportation most drastically.Previous studies have mainly focused on correlation analyses of influencing factors based on crash density/count data,but have paid little attention to the inherent uncertainties of freight truck-related crashes(FTCs)from a spatial perspective.While establishing an interpretable analysis model for freight truck-related accidents that consid-ers uncertainties is of great significance for promoting the robust development of urban freight transportation systems.Hence,this study proposes the concept of FTC hazard(FTCH),and employs the Bayesian neural network(BNN)model based on stochastic varia-tional inference to model uncertainty.Considering the difficulty in interpreting deep learning-based models,this study introduces the local interpretable modelagnostic expla-nation(LIME)model into the analysis framework to explain the results of the neural net-work model.This study then verifies the feasibility of the proposed analysis framework using data from California from 2011 to 2020.Results show that FTCHs can be effectively modeled by predicting confidence intervals for effects of built environment factors,in par-ticular demographics,land use,and road network structure.Results based on LIME values indicate the spatial heterogeneity in influence mechanisms on FTCHs between areas within the metropolitan regions and alongside the freeways.These findings may help transport planners and logistic managers develop more effective measures to avoid potential nega-tive effects brought by FTCHs in local communities.展开更多
Topic modeling is a mainstream and effective technology to deal with text data, with wide applications in text analysis, natural language, personalized recommendation, computer vision, etc. Among all the known topic m...Topic modeling is a mainstream and effective technology to deal with text data, with wide applications in text analysis, natural language, personalized recommendation, computer vision, etc. Among all the known topic models, supervised Latent Dirichlet Allocation （sLDA） is acknowledged as a popular and competitive supervised topic model. How- ever, the gradual increase of the scale of datasets makes sLDA more and more inefficient and time-consuming, and limits its applications in a very narrow range. To solve it, a parallel online sLDA, named PO-sLDA （Parallel and Online sLDA）, is proposed in this study. It uses the stochastic variational inference as the learning method to make the training procedure more rapid and efficient, and a parallel computing mechanism implemented via the MapReduce framework is proposed to promote the capacity of cloud computing and big data processing. The online training capacity supported by PO-sLDA expands the application scope of this approach, making it instrumental for real-life applications with high real-time demand. The validation using two datasets with different sizes shows that the proposed approach has the comparative accuracy as the sLDA and can efficiently accelerate the training procedure. Moreover, its good convergence and online training capacity make it lucrative for the large-scale text data analyzing and processing.展开更多
Stochastic variational inference is an efficient Bayesian inference technology for massive datasets,which approximates posteriors by using noisy gradient estimates.Traditional stochastic variational inference can only...Stochastic variational inference is an efficient Bayesian inference technology for massive datasets,which approximates posteriors by using noisy gradient estimates.Traditional stochastic variational inference can only be performed in a centralized manner,which limits its applications in a wide range of situations where data is possessed by multiple nodes.Therefore,this paper develops a novel trust-region based stochastic variational inference algorithm for a general class of conjugate-exponential models over distributed and asynchronous networks,where the global parameters are diffused over the network by using the Metropolis rule and the local parameters are updated by using the trust-region method.Besides,a simple rule is introduced to balance the transmission frequencies between neighboring nodes such that the proposed distributed algorithm can be performed in an asynchronous manner.The utility of the proposed algorithm is tested by fitting the Bernoulli model and the Gaussian model to different datasets on a synthetic network,and experimental results demonstrate its effectiveness and advantages over existing works.展开更多
Stochastic variational inference （SVI） can learn topic models with very big corpora. It optimizes the variational objective by using the stochastic natural gradient algorithm with a decreasing learning rate. This ra...Stochastic variational inference （SVI） can learn topic models with very big corpora. It optimizes the variational objective by using the stochastic natural gradient algorithm with a decreasing learning rate. This rate is crucial for SVI; however, it is often tuned by hand in real applications. To address this, we develop a novel algorithm, which tunes the learning rate of each iteration adaptively. The proposed algorithm uses the Kullback-Leibler （KL） divergence to measure the similarity between the variational distribution with noisy update and that with batch update, and then optimizes the learning rates by minimizing the KL divergence. We apply our algorithm to two representative topic models： latent Dirichlet allocation and hierarchical Dirichlet process. Experimental results indicate that our algorithm performs better and converges faster than commonly used learning rates.展开更多
This paper is focused on the state estimation problem for nonlinear systems with unknown statistics of measurement noise.Based on the cubature Kalman filter,we propose a new nonlinear filtering algorithm that employs ...This paper is focused on the state estimation problem for nonlinear systems with unknown statistics of measurement noise.Based on the cubature Kalman filter,we propose a new nonlinear filtering algorithm that employs a skew t distribution to characterize the asymmetry of the measurement noise.The system states and the statistics of skew t noise distribution,including the shape matrix,the scale matrix,and the degree of freedom(DOF)are estimated jointly by employing variational Bayesian(VB)inference.The proposed method is validated in a target tracking example.Results of the simulation indicate that the proposed nonlinear filter can perform satisfactorily in the presence of unknown statistics of measurement noise and outperform than the existing state-of-the-art nonlinear filters.展开更多
Efficient estimation of line spectral from quantized samples is of significant importance in information theory and signal processing,e.g.,channel estimation in energy efficient massive MIMO systems and direction of a...Efficient estimation of line spectral from quantized samples is of significant importance in information theory and signal processing,e.g.,channel estimation in energy efficient massive MIMO systems and direction of arrival estimation.The goal of this paper is to recover the line spectral as well as its corresponding parameters including the model order,frequencies and amplitudes from heavily quantized samples.To this end,we propose an efficient gridless Bayesian algorithm named VALSE-EP,which is a combination of the high resolution and low complexity gridless variational line spectral estimation(VALSE)and expectation propagation(EP).The basic idea of VALSE-EP is to iteratively approximate the challenging quantized model of line spectral estimation as a sequence of simple pseudo unquantized models,where VALSE is applied.Moreover,to obtain a benchmark of the performance of the proposed algorithm,the Cram′er Rao bound(CRB)is derived.Finally,numerical experiments on both synthetic and real data are performed,demonstrating the near CRB performance of the proposed VALSE-EP for line spectral estimation from quantized samples.展开更多
A novel variational Bayesian inference based on adaptive cubature Kalman filter(VBACKF)algorithm is proposed for the problem of state estimation in a target tracking system with time-varying measurement noise and rand...A novel variational Bayesian inference based on adaptive cubature Kalman filter(VBACKF)algorithm is proposed for the problem of state estimation in a target tracking system with time-varying measurement noise and random measurement losses.Firstly,the Inverse-Wishart(IW)distribution is chosen to model the covariance matrix of time-varying measurement noise in the cubature Kalman filter framework.Secondly,the Bernoulli random variable is introduced as the judgement factor of the measurement losses,and the Beta distribution is selected as the conjugate prior distribution of measurement loss probability to ensure that the posterior distribution and prior distribution have the same function form.Finally,the joint posterior probability density function of the estimated variables is approximately decoupled by the variational Bayesian inference,and the fixed-point iteration approach is used to update the estimated variables.The simulation results show that the proposed VBACKF algorithm considers the comprehensive effects of system nonlinearity,time-varying measurement noise and unknown measurement loss probability,moreover,effectively improves the accuracy of target state estimation in complex scene.展开更多
Aiming at the problem of filtering precision degradation caused by the random outliers of process noise and measurement noise in multi-target tracking(MTT) system,a new Gaussian-Student’s t mixture distribution proba...Aiming at the problem of filtering precision degradation caused by the random outliers of process noise and measurement noise in multi-target tracking(MTT) system,a new Gaussian-Student’s t mixture distribution probability hypothesis density(PHD) robust filtering algorithm based on variational Bayesian inference(GST-vbPHD) is proposed.Firstly,since it can accurately describe the heavy-tailed characteristics of noise with outliers,Gaussian-Student’s t mixture distribution is employed to model process noise and measurement noise respectively.Then Bernoulli random variable is introduced to correct the likelihood distribution of the mixture probability,leading hierarchical Gaussian distribution constructed by the Gaussian-Student’s t mixture distribution suitable to model non-stationary noise.Finally,the approximate solutions including target weights,measurement noise covariance and state estimation error covariance are obtained according to variational Bayesian inference approach.The simulation results show that,in the heavy-tailed noise environment,the proposed algorithm leads to strong improvements over the traditional PHD filter and the Student’s t distribution PHD filter.展开更多
A measurement system for the scattering characteristics of warhead fragments based on high-speed imaging systems offers advantages such as simple deployment,flexible maneuverability,and high spatiotemporal resolution,...A measurement system for the scattering characteristics of warhead fragments based on high-speed imaging systems offers advantages such as simple deployment,flexible maneuverability,and high spatiotemporal resolution,enabling the acquisition of full-process data of the fragment scattering process.However,mismatches between camera frame rates and target velocities can lead to long motion blur tails of high-speed fragment targets,resulting in low signal-to-noise ratios and rendering conventional detection algorithms ineffective in dynamic strong interference testing environments.In this study,we propose a detection framework centered on dynamic strong interference disturbance signal separation and suppression.We introduce a mixture Gaussian model constrained under a joint spatialtemporal-transform domain Dirichlet process,combined with total variation regularization to achieve disturbance signal suppression.Experimental results demonstrate that the proposed disturbance suppression method can be integrated with certain conventional motion target detection tasks,enabling adaptation to real-world data to a certain extent.Moreover,we provide a specific implementation of this process,which achieves a detection rate close to 100%with an approximate 0%false alarm rate in multiple sets of real target field test data.This research effectively advances the development of the field of damage parameter testing.展开更多
The estimation of sparse underwater acoustic(UWA)channels can be regarded as an inference problem involving hidden variables within the Bayesian framework.While the classical sparse Bayesian learning(SBL),derived thro...The estimation of sparse underwater acoustic(UWA)channels can be regarded as an inference problem involving hidden variables within the Bayesian framework.While the classical sparse Bayesian learning(SBL),derived through the expectation maximization(EM)algorithm,has been widely employed for UWA channel estimation,it still differs from the real posterior expectation of channels.In this paper,we propose an approach that combines variational inference(VI)and Markov chain Monte Carlo(MCMC)methods to provide a more accurate posterior estimation.Specifically,the SBL is first re-derived with VI,allowing us to replace the posterior distribution of the hidden variables with a variational distribution.Then,we determine the full conditional probability distribution for each variable in the variational distribution and then iteratively perform random Gibbs sampling in MCMC to converge the Markov chain.The results of simulation and experiment indicate that our estimation method achieves lower mean square error and bit error rate compared to the classic SBL approach.Additionally,it demonstrates an acceptable convergence speed.展开更多
In the tag recommendation task on academic platforms,existing methods disregard users’customized preferences in favor of extracting tags based just on the content of the articles.Besides,it uses co-occurrence techniq...In the tag recommendation task on academic platforms,existing methods disregard users’customized preferences in favor of extracting tags based just on the content of the articles.Besides,it uses co-occurrence techniques and tries to combine nodes’textual content for modelling.They still do not,however,directly simulate many interactions in network learning.In order to address these issues,we present a novel system that more thoroughly integrates user preferences and citation networks into article labelling recommendations.Specifically,we first employ path similarity to quantify the degree of similarity between user labelling preferences and articles in the citation network.Then,the Commuting Matrix for massive node pair paths is used to improve computational performance.Finally,the two commonalities mentioned above are combined with the interaction paper labels based on the additivity of Poisson distribution.In addition,we also consider solving the model’s parameters by applying variational inference.Experimental results demonstrate that our suggested framework agrees and significantly outperforms the state-of-the-art baseline on two real datasets by efficiently merging the three relational data.Based on the Area Under Curve(AUC)and Mean Average Precision(MAP)analysis,the performance of the suggested task is evaluated,and it is demonstrated to have a greater solving efficiency than current techniques.展开更多
Background:Single-cell RNA-sequencing(scRNA-seq)has emerged as a powerful tool for cancer research,enabling in-depth characterization of tumor heterogeneity at the single-cell level.Recently,several scRNA-seq copy num...Background:Single-cell RNA-sequencing(scRNA-seq)has emerged as a powerful tool for cancer research,enabling in-depth characterization of tumor heterogeneity at the single-cell level.Recently,several scRNA-seq copy number variation(scCNV)inference methods have been developed,expanding the application of scRNA-seq to study genetic heterogeneity in cancer using transcriptomic data.However,the fidelity of these methods has not been investigated systematically.Methods:We benchmarked five commonly used scCNV inference methods:HoneyBADGER,CopyKAT,CaSpER,inferCNV,and sciCNV.We evaluated their performance across four different scRNA-seq platforms using data from our previous multicenter study.We evaluated scCNV performance further using scRNA-seq datasets derived from mixed samples consisting of five human lung adenocarcinoma cell lines and also sequenced tissues from a small cell lung cancer patient and used the data to validate our findings with a clinical scRNA-seq dataset.Results:We found that the sensitivity and specificity of the five scCNV inference methods varied,depending on the selection of reference data,sequencing depth,and read length.CopyKAT and CaSpER outperformed other methods overall,while inferCNV,sciCNV,and CopyKAT performed better than other methods in subclone identification.We found that batch effects significantly affected the performance of subclone identification in mixed datasets in most methods we tested.Conclusion:Our benchmarking study revealed the strengths and weaknesses of each of these scCNV inference methods and provided guidance for selecting the optimal CNV inference method using scRNA-seq data.展开更多
Transformers designed for natural language processing have originally been explored for computer vision in recent research. Various Vision Transformers(ViTs) play an increasingly important role in the field of image t...Transformers designed for natural language processing have originally been explored for computer vision in recent research. Various Vision Transformers(ViTs) play an increasingly important role in the field of image tasks such as computer vision, multimodal fusion and multimedia analysis. However, to obtain promising performance, most existing ViTs usually rely on artificially filtered high-quality images, which may suffer from inherent noise risk.Generally, such well-constructed images are not always available in every situation. To this end,we propose a Robust ViT(RViT) to focus on the relevant and robust representation learning for image classification tasks. Specifically, we first develop a novel Denoising VTUnet module,where we conceptualize the nonrobust noise as the uncertainty under the variational conditions.Furthermore, we design a fusion transformer backbone with a tailored fusion attention mechanism to perform image classification based on the extracted robust representations effectively. To demonstrate the superiority of our model, the compared experiments are conducted on several popular datasets. Benefiting from the sequence regularity of the Transformer and captured robust feature,the proposed method exceeds compared Transformer-based models with superior performance in visual tasks.展开更多
Geological model compression is crucial for making large and complex models more manageable.By reducing the size of these models,compression techniques enable efficient storage,enhance computational efficiency,making ...Geological model compression is crucial for making large and complex models more manageable.By reducing the size of these models,compression techniques enable efficient storage,enhance computational efficiency,making it feasible to perform complex simulations and analyses in a shorter time.This is particularly important in applications such as reservoir management,groundwater hydrology,and geological carbon storage,where large geomodels with millions of grid cells are common.This study presents a comprehensive overview of previous work on geomodel compression and introduces several autoencoder-based deep-learning architectures for low-dimensional representation of modified Bruggefield geomodels.The compression and reconstruction efficiencies of autoencoders(AE),variational autoencoders(VAE),vector-quantized variational autoencoders(VQ-VAE),and vector-quantized variational autoencoders 2(VQ-VAE2)were tested and compared to the traditional singular value decomposition(SVD)method.Results show that the deep-learning-based approaches significantly outperform SVD,achieving higher compression ratios while maintaining or even exceeding the reconstruction quality.Notably,VQ-VAE2 achieves the highest compression ratio of 667:1 with a structural similarity index metric(SSIM)of 0.92,far surpassing the 10:1 compression ratio of SVD with a SSIM of 0.9.The result of this work shows that,unlike traditional approaches,which often rely on linear transformations and can struggle to capture complex,non-linear relationships within geological data,VQ-VAE's use of vector quantization helps in preserving high-resolution details and enhances the model's ability to generalize across varying geological complexities.展开更多
In the article, an improved variational inference （VI） framework for learning finite Beta-Liouville mixture models （BLM） is proposed for proportional data classification and clustering. Within the VI framework, so...In the article, an improved variational inference （VI） framework for learning finite Beta-Liouville mixture models （BLM） is proposed for proportional data classification and clustering. Within the VI framework, some non-linear approximation techniques are adopted to obtain the approximated variational object functions. Analytical solutions are obtained for the variational posterior distributions. Compared to the expectation maximization （EM） algorithm which is commonly used for learning mixture models, underfitting and overfitting events can be prevented. Furthermore, parameters and complexity of the mixture model （model order） can be estimated simultaneously. Experiment shows that both synthetic and real-world data sets are to demonstrate the feasibility and advantages of the proposed method.展开更多
Bayesian-based methods have emerged as an effective approach in continual learning(CL) to solve catastrophic forgetting. One prominent example is Variational Continual Learning(VCL), which demonstrates remarkable perf...Bayesian-based methods have emerged as an effective approach in continual learning(CL) to solve catastrophic forgetting. One prominent example is Variational Continual Learning(VCL), which demonstrates remarkable performance in task-incremental learning(task-IL).However, class-incremental learning(class-IL) is still challenging for VCL, and the reasons behind this limitation remain unclear. Relying on the sophisticated neural mechanisms, particularly the mechanism of memory consolidation during sleep, the human brain possesses inherent advantages for both task-IL and class-IL scenarios, which provides insight for a braininspired VCL. To identify the reasons for the inadequacy of VCL in class-IL, we first conduct a comprehensive theoretical analysis of VCL. On this basis, we propose a novel Bayesian framework named as Learning within Sleeping(Lw S) by leveraging the memory consolidation.By simulating the distribution integration and generalization observed during memory consolidation in sleep, Lw S achieves the idea of prior knowledge guiding posterior knowledge learning as in VCL. In addition, with emulating the process of memory reactivation of the brain,Lw S imposes a constraint on feature invariance to mitigate forgetting learned knowledge. Experimental results demonstrate that Lw S outperforms both Bayesian and non-Bayesian methods in task-IL and class-IL scenarios, which further indicates the effectiveness of incorporating brain mechanisms on designing novel approaches for CL.展开更多
Gene regulatory network(GRN)refers to the complex network formed by regulatory interactions between genes in living cells.In this paper,we consider inferring GRNs in single cells based on single-cell RNA sequencing(sc...Gene regulatory network(GRN)refers to the complex network formed by regulatory interactions between genes in living cells.In this paper,we consider inferring GRNs in single cells based on single-cell RNA sequencing(scRNA-seq)data.In scRNA-seq,single cells are often profiled from mixed populations,and their cell identities are unknown.A common practice for single-cell GRN analysis is to first cluster the cells and infer GRNs for every cluster separately.However,this two-step procedure ignores uncertainty in the clustering step and thus could lead to inaccurate estimation of the networks.Here,we consider the mixture Poisson lognormal model(MPLN)for network inference of count data from mixed populations.The precision matrices of the MPLN are the GRNs of different cell types.To avoid the intractable optimization of the MPLN’s log-likelihood,we develop an algorithm called variational mixture Poisson log-normal(VMPLN)to jointly estimate the GRNs of different cell types based on the variational inference method.We compare VMPLN with state-of-the-art single-cell regulatory network inference methods.Comprehensive simulation shows that VMPLN achieves better performance,especially in scenarios where different cell types have a high mixing degree.Benchmarking on real scRNA-seq data also demonstrates that VMPLN can provide more accurate network estimation in most cases.Finally,we apply VMPLN to a large scRNA-seq dataset from patients infected with severe acute respiratory syndrome coronavirus 2(SARS-CoV-2)and find that VMPLN identifies critical differences in regulatory networks in immune cells between patients with moderate and severe symptoms.The source codes are available on the GitHub website(github.com/XiDsLab/SCVMPLN).展开更多
基金supported by the National Key Research andDevelopment Program of China(2017YFA0700300)the National Natural Sciences Foundation of China(61533005,61703071,61603069)。
文摘Prediction intervals(PIs)for industrial time series can provide useful guidance for workers.Given that the failure of industrial sensors may cause the missing point in inputs,the existing kernel dynamic Bayesian networks(KDBN),serving as an effective method for PIs construction,suffer from high computational load using the stochastic algorithm for inference.This study proposes a variational inference method for the KDBN for the purpose of fast inference,which avoids the timeconsuming stochastic sampling.The proposed algorithm contains two stages.The first stage involves the inference of the missing inputs by using a local linearization based variational inference,and based on the computed posterior distributions over the missing inputs the second stage sees a Gaussian approximation for probability over the nodes in future time slices.To verify the effectiveness of the proposed method,a synthetic dataset and a practical dataset of generation flow of blast furnace gas(BFG)are employed with different ratios of missing inputs.The experimental results indicate that the proposed method can provide reliable PIs for the generation flow of BFG and it exhibits shorter computing time than the stochastic based one.
基金Supported by the Science and Technology Key Project of Science and Technology Department of Henan Province(No.252102211041)the Key Research and Development Projects of Henan Province(No.231111212500).
文摘A distributed bearing-only target tracking algorithm based on variational Bayesian inference(VBI)under random measurement anomalies is proposed for the problem of adverse effect of random measurement anomalies on the state estimation accuracy of moving targets in bearing-only tracking scenarios.Firstly,the measurement information of each sensor is complemented by using triangulation under the distributed framework.Secondly,the Student-t distribution is selected to model the measurement likelihood probability density function,and the joint posteriori probability density function of the estimated variables is approximately decoupled by VBI.Finally,the estimation results of each local filter are sent to the fusion center and fed back to each local filter.The simulation results show that the proposed distributed bearing-only target tracking algorithm based on VBI in the presence of abnormal measurement noise comprehensively considers the influence of system nonlinearity and random anomaly of measurement noise,and has higher estimation accuracy and robustness than other existing algorithms in the above scenarios.
基金supported in part by the National Science Foundation of China(NSFC)with grant no.62271514in part by the Science,Technology and Innovation Commission of Shenzhen Municipality with grant no.JCYJ20210324120002007 and ZDSYS20210623091807023in part by the State Key Laboratory of Public Big Data with grant no.PBD2023-01。
文摘Recently,deep learning-based semantic communication has garnered widespread attention,with numerous systems designed for transmitting diverse data sources,including text,image,and speech,etc.While efforts have been directed toward improving system performance,many studies have concentrated on enhancing the structure of the encoder and decoder.However,this often overlooks the resulting increase in model complexity,imposing additional storage and computational burdens on smart devices.Furthermore,existing work tends to prioritize explicit semantics,neglecting the potential of implicit semantics.This paper aims to easily and effectively enhance the receiver's decoding capability without modifying the encoder and decoder structures.We propose a novel semantic communication system with variational neural inference for text transmission.Specifically,we introduce a simple but effective variational neural inferer at the receiver to infer the latent semantic information within the received text.This information is then utilized to assist in the decoding process.The simulation results show a significant enhancement in system performance and improved robustness.
基金supported by the Shanghai Sailing Program of China(ID:20YF1451700)the Science and Technology Commission of Shanghai Municipality of China(Nos.23692119000&21692112203)the Fundamental Research Funds for the Central Universities of China(No.2023-4-YB-01).
文摘Due to the increasing demand for goods movement,externalities from freight mobility have attracted much concern among local citizens and policymakers.Freight truck-related crash is one of these externalities and impacts urban freight transportation most drastically.Previous studies have mainly focused on correlation analyses of influencing factors based on crash density/count data,but have paid little attention to the inherent uncertainties of freight truck-related crashes(FTCs)from a spatial perspective.While establishing an interpretable analysis model for freight truck-related accidents that consid-ers uncertainties is of great significance for promoting the robust development of urban freight transportation systems.Hence,this study proposes the concept of FTC hazard(FTCH),and employs the Bayesian neural network(BNN)model based on stochastic varia-tional inference to model uncertainty.Considering the difficulty in interpreting deep learning-based models,this study introduces the local interpretable modelagnostic expla-nation(LIME)model into the analysis framework to explain the results of the neural net-work model.This study then verifies the feasibility of the proposed analysis framework using data from California from 2011 to 2020.Results show that FTCHs can be effectively modeled by predicting confidence intervals for effects of built environment factors,in par-ticular demographics,land use,and road network structure.Results based on LIME values indicate the spatial heterogeneity in influence mechanisms on FTCHs between areas within the metropolitan regions and alongside the freeways.These findings may help transport planners and logistic managers develop more effective measures to avoid potential nega-tive effects brought by FTCHs in local communities.
基金This work was supported in part by the National Natural Science Foundation of China under Grant Nos. 61572226 and 61876069, and the Key Scientific and Technological Research and Development Project of Jilin Province of China under Grant Nos. 20180201067GX and 20180201044GX.
文摘Topic modeling is a mainstream and effective technology to deal with text data, with wide applications in text analysis, natural language, personalized recommendation, computer vision, etc. Among all the known topic models, supervised Latent Dirichlet Allocation （sLDA） is acknowledged as a popular and competitive supervised topic model. How- ever, the gradual increase of the scale of datasets makes sLDA more and more inefficient and time-consuming, and limits its applications in a very narrow range. To solve it, a parallel online sLDA, named PO-sLDA （Parallel and Online sLDA）, is proposed in this study. It uses the stochastic variational inference as the learning method to make the training procedure more rapid and efficient, and a parallel computing mechanism implemented via the MapReduce framework is proposed to promote the capacity of cloud computing and big data processing. The online training capacity supported by PO-sLDA expands the application scope of this approach, making it instrumental for real-life applications with high real-time demand. The validation using two datasets with different sizes shows that the proposed approach has the comparative accuracy as the sLDA and can efficiently accelerate the training procedure. Moreover, its good convergence and online training capacity make it lucrative for the large-scale text data analyzing and processing.
基金the National Natural Science Foundation of China under Grant Nos.61922076,61873252,61725304,and 61973324in part by Guangdong Basic and Applied Basic Research Foundation under Grant No.2021B1515020094in part by the Guangdong Provincial Key Laboratory of Computational Science under Grant No.2020B1212060032。
文摘Stochastic variational inference is an efficient Bayesian inference technology for massive datasets,which approximates posteriors by using noisy gradient estimates.Traditional stochastic variational inference can only be performed in a centralized manner,which limits its applications in a wide range of situations where data is possessed by multiple nodes.Therefore,this paper develops a novel trust-region based stochastic variational inference algorithm for a general class of conjugate-exponential models over distributed and asynchronous networks,where the global parameters are diffused over the network by using the Metropolis rule and the local parameters are updated by using the trust-region method.Besides,a simple rule is introduced to balance the transmission frequencies between neighboring nodes such that the proposed distributed algorithm can be performed in an asynchronous manner.The utility of the proposed algorithm is tested by fitting the Bernoulli model and the Gaussian model to different datasets on a synthetic network,and experimental results demonstrate its effectiveness and advantages over existing works.
基金This work was supported by the National Natural Science Foundation of China under Grant Nos. 61170092, 61133011 and 61103091.
文摘Stochastic variational inference （SVI） can learn topic models with very big corpora. It optimizes the variational objective by using the stochastic natural gradient algorithm with a decreasing learning rate. This rate is crucial for SVI; however, it is often tuned by hand in real applications. To address this, we develop a novel algorithm, which tunes the learning rate of each iteration adaptively. The proposed algorithm uses the Kullback-Leibler （KL） divergence to measure the similarity between the variational distribution with noisy update and that with batch update, and then optimizes the learning rates by minimizing the KL divergence. We apply our algorithm to two representative topic models： latent Dirichlet allocation and hierarchical Dirichlet process. Experimental results indicate that our algorithm performs better and converges faster than commonly used learning rates.
基金This work was supported in part by National Natural Science Foundation of China under Grants 62103167 and 61833007in part by the Natural Science Foundation of Jiangsu Province under Grant BK20210451.
文摘This paper is focused on the state estimation problem for nonlinear systems with unknown statistics of measurement noise.Based on the cubature Kalman filter,we propose a new nonlinear filtering algorithm that employs a skew t distribution to characterize the asymmetry of the measurement noise.The system states and the statistics of skew t noise distribution,including the shape matrix,the scale matrix,and the degree of freedom(DOF)are estimated jointly by employing variational Bayesian(VB)inference.The proposed method is validated in a target tracking example.Results of the simulation indicate that the proposed nonlinear filter can perform satisfactorily in the presence of unknown statistics of measurement noise and outperform than the existing state-of-the-art nonlinear filters.
基金supported by National Natural Science Foundation of China(No.61901415)。
文摘Efficient estimation of line spectral from quantized samples is of significant importance in information theory and signal processing,e.g.,channel estimation in energy efficient massive MIMO systems and direction of arrival estimation.The goal of this paper is to recover the line spectral as well as its corresponding parameters including the model order,frequencies and amplitudes from heavily quantized samples.To this end,we propose an efficient gridless Bayesian algorithm named VALSE-EP,which is a combination of the high resolution and low complexity gridless variational line spectral estimation(VALSE)and expectation propagation(EP).The basic idea of VALSE-EP is to iteratively approximate the challenging quantized model of line spectral estimation as a sequence of simple pseudo unquantized models,where VALSE is applied.Moreover,to obtain a benchmark of the performance of the proposed algorithm,the Cram′er Rao bound(CRB)is derived.Finally,numerical experiments on both synthetic and real data are performed,demonstrating the near CRB performance of the proposed VALSE-EP for line spectral estimation from quantized samples.
基金Supported by the National Natural Science Foundation of China(No.61976080)the Science and Technology Key Project of Science and TechnologyDepartment of Henan Province(No.212102310298)+1 种基金the Academic Degrees&Graduate Education Reform Project of Henan Province(No.2021SJGLX195Y)the Innovation and Quality Improvement Project for Graduate Education of Henan University(No.SYL20010101)。
文摘A novel variational Bayesian inference based on adaptive cubature Kalman filter(VBACKF)algorithm is proposed for the problem of state estimation in a target tracking system with time-varying measurement noise and random measurement losses.Firstly,the Inverse-Wishart(IW)distribution is chosen to model the covariance matrix of time-varying measurement noise in the cubature Kalman filter framework.Secondly,the Bernoulli random variable is introduced as the judgement factor of the measurement losses,and the Beta distribution is selected as the conjugate prior distribution of measurement loss probability to ensure that the posterior distribution and prior distribution have the same function form.Finally,the joint posterior probability density function of the estimated variables is approximately decoupled by the variational Bayesian inference,and the fixed-point iteration approach is used to update the estimated variables.The simulation results show that the proposed VBACKF algorithm considers the comprehensive effects of system nonlinearity,time-varying measurement noise and unknown measurement loss probability,moreover,effectively improves the accuracy of target state estimation in complex scene.
基金Supported by the National Natural Science Foundation of China(No.61976080)the Science and Technology Key Project of Science and Technology Department of Henan Province(No.212102310298)the Innovation and Quality Improvement Project for Graduate Education of Henan University(No.SYL20010101)。
文摘Aiming at the problem of filtering precision degradation caused by the random outliers of process noise and measurement noise in multi-target tracking(MTT) system,a new Gaussian-Student’s t mixture distribution probability hypothesis density(PHD) robust filtering algorithm based on variational Bayesian inference(GST-vbPHD) is proposed.Firstly,since it can accurately describe the heavy-tailed characteristics of noise with outliers,Gaussian-Student’s t mixture distribution is employed to model process noise and measurement noise respectively.Then Bernoulli random variable is introduced to correct the likelihood distribution of the mixture probability,leading hierarchical Gaussian distribution constructed by the Gaussian-Student’s t mixture distribution suitable to model non-stationary noise.Finally,the approximate solutions including target weights,measurement noise covariance and state estimation error covariance are obtained according to variational Bayesian inference approach.The simulation results show that,in the heavy-tailed noise environment,the proposed algorithm leads to strong improvements over the traditional PHD filter and the Student’s t distribution PHD filter.
文摘A measurement system for the scattering characteristics of warhead fragments based on high-speed imaging systems offers advantages such as simple deployment,flexible maneuverability,and high spatiotemporal resolution,enabling the acquisition of full-process data of the fragment scattering process.However,mismatches between camera frame rates and target velocities can lead to long motion blur tails of high-speed fragment targets,resulting in low signal-to-noise ratios and rendering conventional detection algorithms ineffective in dynamic strong interference testing environments.In this study,we propose a detection framework centered on dynamic strong interference disturbance signal separation and suppression.We introduce a mixture Gaussian model constrained under a joint spatialtemporal-transform domain Dirichlet process,combined with total variation regularization to achieve disturbance signal suppression.Experimental results demonstrate that the proposed disturbance suppression method can be integrated with certain conventional motion target detection tasks,enabling adaptation to real-world data to a certain extent.Moreover,we provide a specific implementation of this process,which achieves a detection rate close to 100%with an approximate 0%false alarm rate in multiple sets of real target field test data.This research effectively advances the development of the field of damage parameter testing.
基金funded by the Excellent Youth Science Fund of Heilongjiang Province(Grant No.YQ2022F001).
文摘The estimation of sparse underwater acoustic(UWA)channels can be regarded as an inference problem involving hidden variables within the Bayesian framework.While the classical sparse Bayesian learning(SBL),derived through the expectation maximization(EM)algorithm,has been widely employed for UWA channel estimation,it still differs from the real posterior expectation of channels.In this paper,we propose an approach that combines variational inference(VI)and Markov chain Monte Carlo(MCMC)methods to provide a more accurate posterior estimation.Specifically,the SBL is first re-derived with VI,allowing us to replace the posterior distribution of the hidden variables with a variational distribution.Then,we determine the full conditional probability distribution for each variable in the variational distribution and then iteratively perform random Gibbs sampling in MCMC to converge the Markov chain.The results of simulation and experiment indicate that our estimation method achieves lower mean square error and bit error rate compared to the classic SBL approach.Additionally,it demonstrates an acceptable convergence speed.
基金supported by the National Natural Science Foundation of China(No.62271274).
文摘In the tag recommendation task on academic platforms,existing methods disregard users’customized preferences in favor of extracting tags based just on the content of the articles.Besides,it uses co-occurrence techniques and tries to combine nodes’textual content for modelling.They still do not,however,directly simulate many interactions in network learning.In order to address these issues,we present a novel system that more thoroughly integrates user preferences and citation networks into article labelling recommendations.Specifically,we first employ path similarity to quantify the degree of similarity between user labelling preferences and articles in the citation network.Then,the Commuting Matrix for massive node pair paths is used to improve computational performance.Finally,the two commonalities mentioned above are combined with the interaction paper labels based on the additivity of Poisson distribution.In addition,we also consider solving the model’s parameters by applying variational inference.Experimental results demonstrate that our suggested framework agrees and significantly outperforms the state-of-the-art baseline on two real datasets by efficiently merging the three relational data.Based on the Area Under Curve(AUC)and Mean Average Precision(MAP)analysis,the performance of the suggested task is evaluated,and it is demonstrated to have a greater solving efficiency than current techniques.
基金carried out at the LLU Center for Genomics was funded in part by the Ardmore Institute of Health grant 2150141(CW)and Dr.Charles A.Sims’gift to LLU Center for Genomicssupported by the National Center for Biotechnology Information of the National Library of Medicine(NLM),National Institutes of Healt。
文摘Background:Single-cell RNA-sequencing(scRNA-seq)has emerged as a powerful tool for cancer research,enabling in-depth characterization of tumor heterogeneity at the single-cell level.Recently,several scRNA-seq copy number variation(scCNV)inference methods have been developed,expanding the application of scRNA-seq to study genetic heterogeneity in cancer using transcriptomic data.However,the fidelity of these methods has not been investigated systematically.Methods:We benchmarked five commonly used scCNV inference methods:HoneyBADGER,CopyKAT,CaSpER,inferCNV,and sciCNV.We evaluated their performance across four different scRNA-seq platforms using data from our previous multicenter study.We evaluated scCNV performance further using scRNA-seq datasets derived from mixed samples consisting of five human lung adenocarcinoma cell lines and also sequenced tissues from a small cell lung cancer patient and used the data to validate our findings with a clinical scRNA-seq dataset.Results:We found that the sensitivity and specificity of the five scCNV inference methods varied,depending on the selection of reference data,sequencing depth,and read length.CopyKAT and CaSpER outperformed other methods overall,while inferCNV,sciCNV,and CopyKAT performed better than other methods in subclone identification.We found that batch effects significantly affected the performance of subclone identification in mixed datasets in most methods we tested.Conclusion:Our benchmarking study revealed the strengths and weaknesses of each of these scCNV inference methods and provided guidance for selecting the optimal CNV inference method using scRNA-seq data.
文摘Transformers designed for natural language processing have originally been explored for computer vision in recent research. Various Vision Transformers(ViTs) play an increasingly important role in the field of image tasks such as computer vision, multimodal fusion and multimedia analysis. However, to obtain promising performance, most existing ViTs usually rely on artificially filtered high-quality images, which may suffer from inherent noise risk.Generally, such well-constructed images are not always available in every situation. To this end,we propose a Robust ViT(RViT) to focus on the relevant and robust representation learning for image classification tasks. Specifically, we first develop a novel Denoising VTUnet module,where we conceptualize the nonrobust noise as the uncertainty under the variational conditions.Furthermore, we design a fusion transformer backbone with a tailored fusion attention mechanism to perform image classification based on the extracted robust representations effectively. To demonstrate the superiority of our model, the compared experiments are conducted on several popular datasets. Benefiting from the sequence regularity of the Transformer and captured robust feature,the proposed method exceeds compared Transformer-based models with superior performance in visual tasks.
基金supported by ConocoPhillips through the DataDriven Intelligent Engineering and Characterization(DICE)research group.
文摘Geological model compression is crucial for making large and complex models more manageable.By reducing the size of these models,compression techniques enable efficient storage,enhance computational efficiency,making it feasible to perform complex simulations and analyses in a shorter time.This is particularly important in applications such as reservoir management,groundwater hydrology,and geological carbon storage,where large geomodels with millions of grid cells are common.This study presents a comprehensive overview of previous work on geomodel compression and introduces several autoencoder-based deep-learning architectures for low-dimensional representation of modified Bruggefield geomodels.The compression and reconstruction efficiencies of autoencoders(AE),variational autoencoders(VAE),vector-quantized variational autoencoders(VQ-VAE),and vector-quantized variational autoencoders 2(VQ-VAE2)were tested and compared to the traditional singular value decomposition(SVD)method.Results show that the deep-learning-based approaches significantly outperform SVD,achieving higher compression ratios while maintaining or even exceeding the reconstruction quality.Notably,VQ-VAE2 achieves the highest compression ratio of 667:1 with a structural similarity index metric(SSIM)of 0.92,far surpassing the 10:1 compression ratio of SVD with a SSIM of 0.9.The result of this work shows that,unlike traditional approaches,which often rely on linear transformations and can struggle to capture complex,non-linear relationships within geological data,VQ-VAE's use of vector quantization helps in preserving high-resolution details and enhances the model's ability to generalize across varying geological complexities.
基金supported by the National Natural Science Foundation of China(61303232,61363085,61121061,60972077)the Hi-Tech Research and Development Program of China(2009AA01Z430)
文摘In the article, an improved variational inference （VI） framework for learning finite Beta-Liouville mixture models （BLM） is proposed for proportional data classification and clustering. Within the VI framework, some non-linear approximation techniques are adopted to obtain the approximated variational object functions. Analytical solutions are obtained for the variational posterior distributions. Compared to the expectation maximization （EM） algorithm which is commonly used for learning mixture models, underfitting and overfitting events can be prevented. Furthermore, parameters and complexity of the mixture model （model order） can be estimated simultaneously. Experiment shows that both synthetic and real-world data sets are to demonstrate the feasibility and advantages of the proposed method.
基金supported by the National Natural Science Foundation of China under Grant 62236001 and Grant 62325601
文摘Bayesian-based methods have emerged as an effective approach in continual learning(CL) to solve catastrophic forgetting. One prominent example is Variational Continual Learning(VCL), which demonstrates remarkable performance in task-incremental learning(task-IL).However, class-incremental learning(class-IL) is still challenging for VCL, and the reasons behind this limitation remain unclear. Relying on the sophisticated neural mechanisms, particularly the mechanism of memory consolidation during sleep, the human brain possesses inherent advantages for both task-IL and class-IL scenarios, which provides insight for a braininspired VCL. To identify the reasons for the inadequacy of VCL in class-IL, we first conduct a comprehensive theoretical analysis of VCL. On this basis, we propose a novel Bayesian framework named as Learning within Sleeping(Lw S) by leveraging the memory consolidation.By simulating the distribution integration and generalization observed during memory consolidation in sleep, Lw S achieves the idea of prior knowledge guiding posterior knowledge learning as in VCL. In addition, with emulating the process of memory reactivation of the brain,Lw S imposes a constraint on feature invariance to mitigate forgetting learned knowledge. Experimental results demonstrate that Lw S outperforms both Bayesian and non-Bayesian methods in task-IL and class-IL scenarios, which further indicates the effectiveness of incorporating brain mechanisms on designing novel approaches for CL.
基金National Key Research and Development Program of China,Grant/Award Numbers:2020YFE0204200,2020YFE0204000National Natural Science Foundation of China,Grant/Award Numbers:11971039,12371286Foundation of Shuanghu Laboratory,Grant/Award Number:SH‐2024JK01。
文摘Gene regulatory network(GRN)refers to the complex network formed by regulatory interactions between genes in living cells.In this paper,we consider inferring GRNs in single cells based on single-cell RNA sequencing(scRNA-seq)data.In scRNA-seq,single cells are often profiled from mixed populations,and their cell identities are unknown.A common practice for single-cell GRN analysis is to first cluster the cells and infer GRNs for every cluster separately.However,this two-step procedure ignores uncertainty in the clustering step and thus could lead to inaccurate estimation of the networks.Here,we consider the mixture Poisson lognormal model(MPLN)for network inference of count data from mixed populations.The precision matrices of the MPLN are the GRNs of different cell types.To avoid the intractable optimization of the MPLN’s log-likelihood,we develop an algorithm called variational mixture Poisson log-normal(VMPLN)to jointly estimate the GRNs of different cell types based on the variational inference method.We compare VMPLN with state-of-the-art single-cell regulatory network inference methods.Comprehensive simulation shows that VMPLN achieves better performance,especially in scenarios where different cell types have a high mixing degree.Benchmarking on real scRNA-seq data also demonstrates that VMPLN can provide more accurate network estimation in most cases.Finally,we apply VMPLN to a large scRNA-seq dataset from patients infected with severe acute respiratory syndrome coronavirus 2(SARS-CoV-2)and find that VMPLN identifies critical differences in regulatory networks in immune cells between patients with moderate and severe symptoms.The source codes are available on the GitHub website(github.com/XiDsLab/SCVMPLN).
