We investigate integral-type functionals of the first hitting times for continuous-time Markov chains. Recursive formulas and drift conditions for calculating or bounding integral-type functionals are obtained. The co...We investigate integral-type functionals of the first hitting times for continuous-time Markov chains. Recursive formulas and drift conditions for calculating or bounding integral-type functionals are obtained. The connection between the subexponential integral-type functionals and the subexponential ergodicity is established. Moreover, these results are applied to the birth-death processes. Polynomial integral-type functionals and polynomial ergodicity are studied, and a sufficient criterion for a central limit theorem is also presented.展开更多
This paper studies the existence of the higher orders deviation matrices for continuous time Markov chains by the moments for the hitting times. An estimate of the polynomial convergence rates for the transition matri...This paper studies the existence of the higher orders deviation matrices for continuous time Markov chains by the moments for the hitting times. An estimate of the polynomial convergence rates for the transition matrix to the stationary measure is obtained. Finally, the explicit formulas for birth-death processes are presented.展开更多
We introduce the notion of the contraction integrated semigroups and give the Lumber-Phillips characterization of the generator, and also the charaterazied generators of isometric integrated semigroups. For their appl...We introduce the notion of the contraction integrated semigroups and give the Lumber-Phillips characterization of the generator, and also the charaterazied generators of isometric integrated semigroups. For their application, a necessary and sufficient condition for q-matrices Q generating a contraction integrated semigroup is given, and a necessary and sufficient condition for a transition function to be a Feller-Reuter-Riley transition function is also given in terms of its q-matrix.展开更多
We investigate perturbation for continuous-time Markov chains(CTMCs) on a countable state space. Explicit bounds on ?D and D are derived in terms of a drift condition, where ? and D represent the perturbation of the i...We investigate perturbation for continuous-time Markov chains(CTMCs) on a countable state space. Explicit bounds on ?D and D are derived in terms of a drift condition, where ? and D represent the perturbation of the intensity matrices and the deviation matrix, respectively. Moreover, we obtain perturbation bounds on the stationary distributions, which extends the results by Liu(2012) for uniformly bounded CTMCs to general(possibly unbounded) CTMCs. Our arguments are mainly based on the technique of augmented truncations.展开更多
This paper studies the limit average variance criterion for continuous-time Markov decision processes in Polish spaces. Based on two approaches, this paper proves not only the existence of solutions to the variance mi...This paper studies the limit average variance criterion for continuous-time Markov decision processes in Polish spaces. Based on two approaches, this paper proves not only the existence of solutions to the variance minimization optimality equation and the existence of a variance minimal policy that is canonical, but also the existence of solutions to the two variance minimization optimality inequalities and the existence of a variance minimal policy which may not be canonical. An example is given to illustrate all of our conditions.展开更多
Gearbox in offshore wind turbines is a component with the highest failure rates during operation. Analysis of gearbox repair policy that includes economic considerations is important for the effective operation of off...Gearbox in offshore wind turbines is a component with the highest failure rates during operation. Analysis of gearbox repair policy that includes economic considerations is important for the effective operation of offshore wind farms. From their initial perfect working states, gearboxes degrade with time, which leads to decreased working efficiency. Thus, offshore wind turbine gearboxes can be considered to be multi-state systems with the various levels of productivity for different working states. To efficiently compute the time-dependent distribution of this multi-state system and analyze its reliability, application of the nonhomogeneous continuous-time Markov process(NHCTMP) is appropriate for this type of object. To determine the relationship between operation time and maintenance cost, many factors must be taken into account, including maintenance processes and vessel requirements. Finally, an optimal repair policy can be formulated based on this relationship.展开更多
This geo-historical case study analyses Vistelius’ingenious idea of conceptual stochastic models and their application as Markov chain analysis in the geosciences.Vistelius(1915–1995)is regarded as one of the founde...This geo-historical case study analyses Vistelius’ingenious idea of conceptual stochastic models and their application as Markov chain analysis in the geosciences.Vistelius(1915–1995)is regarded as one of the founders of mathematical geology.He was the fi rst to defi ne mathematical geology as“a scientifi c discipline concerned with the construction,analysis and use of conceptual mathematical models of geological events to solve concrete problems”(Vistelius in Principles of mathematical geology,Nauka,Leningrad,1980;Principles of mathematical geology,Kluwer Academic Publishers,Dordrecht,1992).Mathematical models in this context should be primarily probabilistic because of the large number of infl uencing natural factors.They must be conceptual to avoid fundamental errors in application.Vistelius devoted his seminal book to geological random sequences and their description and analysis using Markov models as stochastic tools.He applied this approach to grain sequences in granitic intrusive rocks and to sedimentary rock layers.Among other things,Vistelius has used Markov chain analysis in mineral resource exploration to distinguish between“ideal”granites,which are not subsequently mineralized,and mainly hydrothermally mineralized,sometimes ore-bearing granites which contain at least two generations of main minerals.The application of this special conceptual stochastic model is demonstrated on Lusatian granite(Saxony,Germany).展开更多
Permanent magnet synchronous motors(PMSMs), owing to the features of low maintenance costs, great efficiency, and high power density, are extensively utilized in applications such as rail transportation, industrial ro...Permanent magnet synchronous motors(PMSMs), owing to the features of low maintenance costs, great efficiency, and high power density, are extensively utilized in applications such as rail transportation, industrial robots, and new energy electric vehicles. However, the application of space vector pulse width modulation(SVPWM) results in the motor phase current exhibiting clustered harmonic distributions at the integer multiples of the switching frequency, leading to motor noise and vibration issues. To address the issues, this paper proposes a three-random SVPWM(TRPWM) strategy based on a threestate Markov chain, integrating random pulse position, random switching frequency, and random small vector dwell time. By adhering to the principle of voltage-second balance, this strategy spreads the concentrated high-frequency harmonics over a wider frequency range, significantly reducing the magnitude of the concentrated harmonics in the phase current. Comparative experiments with conventional SVPWM, conventional dual-random SVPWM, and conventional three-random SVPWM strategies demonstrate that the proposed approach achieves the expansion of harmonics at integer multiples of the switching frequency in the phase current, effectively suppressing high-frequency vibrations in PMSMs.展开更多
This paper considers the variance optimization problem of average reward in continuous-time Markov decision process (MDP). It is assumed that the state space is countable and the action space is Borel measurable space...This paper considers the variance optimization problem of average reward in continuous-time Markov decision process (MDP). It is assumed that the state space is countable and the action space is Borel measurable space. The main purpose of this paper is to find the policy with the minimal variance in the deterministic stationary policy space. Unlike the traditional Markov decision process, the cost function in the variance criterion will be affected by future actions. To this end, we convert the variance minimization problem into a standard (MDP) by introducing a concept called pseudo-variance. Further, by giving the policy iterative algorithm of pseudo-variance optimization problem, the optimal policy of the original variance optimization problem is derived, and a sufficient condition for the variance optimal policy is given. Finally, we use an example to illustrate the conclusion of this paper.展开更多
在多种合理的无信息先验分布下,基于Markov Chain Monte Carlo方法,提出了一种简单且易于抽样的幂律过程的Bayesian分析方法.所提方法将失效、时间截尾数据统一分析,能快捷地获取幂律过程模型参数的Markov Chain Monte Carlo样本,利用...在多种合理的无信息先验分布下,基于Markov Chain Monte Carlo方法,提出了一种简单且易于抽样的幂律过程的Bayesian分析方法.所提方法将失效、时间截尾数据统一分析,能快捷地获取幂律过程模型参数的Markov Chain Monte Carlo样本,利用该样本不但能直接给出模型参数函数的后验分布,还能给出单样预测和双样预测的分析.一个经典工程数值算例说明了所提方法的可行性、合理性与有效性.该方法具有一定的优越性,可为小子样可靠性增长分析提供一种值得参考的方法.展开更多
目的 针对妇幼卫生纵向数据的任意缺失模式,采用多重填补方法进行填补,探求最佳填补结果,以便对数据作进一步分析与研究。方法 运用SAS9.0 ,采用多重填补方法Markov China Monte Carlo(MCMC)模型对缺失数据进行多次填补并综合分析。...目的 针对妇幼卫生纵向数据的任意缺失模式,采用多重填补方法进行填补,探求最佳填补结果,以便对数据作进一步分析与研究。方法 运用SAS9.0 ,采用多重填补方法Markov China Monte Carlo(MCMC)模型对缺失数据进行多次填补并综合分析。结果 填补5次所得结果最优。结论 多重填补方法可以处理有缺失数据资料中的许多普遍问题,可提高统计效率,尤其是MCMC模型在处理复杂的缺失数据上,优势明显。展开更多
目的对医院出院病人调查表普遍存在的数据缺失进行填补与分析,以保证统计调查表的质量,为医院以及上级卫生部门了解现状,进行预策和决策提供技术支持和质量保证。方法运用SAS9.1,采用多重填补方法Markov Chain Monte Carlo(MCMC)模型对...目的对医院出院病人调查表普遍存在的数据缺失进行填补与分析,以保证统计调查表的质量,为医院以及上级卫生部门了解现状,进行预策和决策提供技术支持和质量保证。方法运用SAS9.1,采用多重填补方法Markov Chain Monte Carlo(MCMC)模型对缺失数据进行多次填补并综合分析。结果MCMC填补10次的结果最优。结论(Multiple Imputation)MI方法在解决医院出院病人调查表数据缺失时有优势,发挥空间较大,且填补效率较高。展开更多
Modeling non coding background sequences appropriately is important for the detection of regulatory elements from DNA sequences. Based on the chi square statistic test, some explanations about why to choose higher ...Modeling non coding background sequences appropriately is important for the detection of regulatory elements from DNA sequences. Based on the chi square statistic test, some explanations about why to choose higher order Markov chain model and how to automatically select the proper order are given in this paper. The chi square test is first run on synthetic data sets to show that it can efficiently find the proper order of Markov chain. Using chi square test, distinct higher order context dependences inherent in ten sets of sequences of yeast S.cerevisiae from other literature have been found. So the Markov chain with higher order would be more suitable for modeling the non coding background sequences than an independent model.展开更多
A Markov chain-based stochastic model （MCM） is developed to simulate the movement of particles in a 2D bubbling fluidized bed （BFB）. The state spaces are determined by the discretized physical cells of the bed, an...A Markov chain-based stochastic model （MCM） is developed to simulate the movement of particles in a 2D bubbling fluidized bed （BFB）. The state spaces are determined by the discretized physical cells of the bed, and the transition probability matrix is directly calculated by the results of a discrete element method （DEM） simulation. The Markov property of the BFB is discussed by the comparison results calculated from both static and dynamic transition probability matrices. The static matrix is calculated based on the Markov chain while the dynamic matrix is calculated based on the memory property of the particle movement. Results show that the difference in the trends of particle movement between the static and dynamic matrix calculation is very small. Besides, the particle mixing curves of the MCM and DEM have the same trend and similar numerical values, and the details show the time averaged characteristic of the MCM and also expose its shortcoming in describing the instantaneous particle dynamics in the BFB.展开更多
目的探讨基于Markov Chain Monte Carlo(MCMC)模型的多重估算法在处理医院调查资料缺失数据中的应用。方法运用SAS9.2编写程序,在分析数据的分布类型和缺失机制的基础上,采用MCMC法对缺失数据进行多次填补和联合统计推断,分析多重估算...目的探讨基于Markov Chain Monte Carlo(MCMC)模型的多重估算法在处理医院调查资料缺失数据中的应用。方法运用SAS9.2编写程序,在分析数据的分布类型和缺失机制的基础上,采用MCMC法对缺失数据进行多次填补和联合统计推断,分析多重估算法的优势。结果数据服从多元正态分布与随机缺失,采用MCMC法填补10次所得的结果最佳。结论多重估算既可反映缺失数据的不确定性,又可充分利用现有资料的信息、提高统计效率、对模型的估计结果更加可信,是处理缺失数据的有效方法。展开更多
基金Acknowledgements The authors would like to thank Professor Yong-Hua Mao for useful discussion. This work was supported in part by the National Natural Science Foundation of China （Grant Nos. 11571372, 11501576, 11771452） and the Excellent Young Scientific Research Fund of Hunan Provincial Education Department （Grant No. 15B252）.
文摘We investigate integral-type functionals of the first hitting times for continuous-time Markov chains. Recursive formulas and drift conditions for calculating or bounding integral-type functionals are obtained. The connection between the subexponential integral-type functionals and the subexponential ergodicity is established. Moreover, these results are applied to the birth-death processes. Polynomial integral-type functionals and polynomial ergodicity are studied, and a sufficient criterion for a central limit theorem is also presented.
基金This work was supported in part by the'973'Projectthe Research Fund for the Doctoral Program of Higher Education(Grant No.20010027007)+1 种基金the National Natural Science Foundation of China(Grant Nos.10121101 and 10301007)the National Natural Science Foundation of China for Distinguished Young Scholars(Grant No.10025105).
文摘This paper studies the existence of the higher orders deviation matrices for continuous time Markov chains by the moments for the hitting times. An estimate of the polynomial convergence rates for the transition matrix to the stationary measure is obtained. Finally, the explicit formulas for birth-death processes are presented.
文摘We introduce the notion of the contraction integrated semigroups and give the Lumber-Phillips characterization of the generator, and also the charaterazied generators of isometric integrated semigroups. For their application, a necessary and sufficient condition for q-matrices Q generating a contraction integrated semigroup is given, and a necessary and sufficient condition for a transition function to be a Feller-Reuter-Riley transition function is also given in terms of its q-matrix.
基金supported by National Natural Science Foundation of China(Grant No.11211120144)the Fundamental Research Funds for the Central Universities(Grant No.2010QYZD001)
文摘We investigate perturbation for continuous-time Markov chains(CTMCs) on a countable state space. Explicit bounds on ?D and D are derived in terms of a drift condition, where ? and D represent the perturbation of the intensity matrices and the deviation matrix, respectively. Moreover, we obtain perturbation bounds on the stationary distributions, which extends the results by Liu(2012) for uniformly bounded CTMCs to general(possibly unbounded) CTMCs. Our arguments are mainly based on the technique of augmented truncations.
基金supported by the National Natural Science Foundation of China(10801056)the Natural Science Foundation of Ningbo(2010A610094)
文摘This paper studies the limit average variance criterion for continuous-time Markov decision processes in Polish spaces. Based on two approaches, this paper proves not only the existence of solutions to the variance minimization optimality equation and the existence of a variance minimal policy that is canonical, but also the existence of solutions to the two variance minimization optimality inequalities and the existence of a variance minimal policy which may not be canonical. An example is given to illustrate all of our conditions.
文摘Gearbox in offshore wind turbines is a component with the highest failure rates during operation. Analysis of gearbox repair policy that includes economic considerations is important for the effective operation of offshore wind farms. From their initial perfect working states, gearboxes degrade with time, which leads to decreased working efficiency. Thus, offshore wind turbine gearboxes can be considered to be multi-state systems with the various levels of productivity for different working states. To efficiently compute the time-dependent distribution of this multi-state system and analyze its reliability, application of the nonhomogeneous continuous-time Markov process(NHCTMP) is appropriate for this type of object. To determine the relationship between operation time and maintenance cost, many factors must be taken into account, including maintenance processes and vessel requirements. Finally, an optimal repair policy can be formulated based on this relationship.
基金Open Access funding enabled and organized by Projekt DEAL.
文摘This geo-historical case study analyses Vistelius’ingenious idea of conceptual stochastic models and their application as Markov chain analysis in the geosciences.Vistelius(1915–1995)is regarded as one of the founders of mathematical geology.He was the fi rst to defi ne mathematical geology as“a scientifi c discipline concerned with the construction,analysis and use of conceptual mathematical models of geological events to solve concrete problems”(Vistelius in Principles of mathematical geology,Nauka,Leningrad,1980;Principles of mathematical geology,Kluwer Academic Publishers,Dordrecht,1992).Mathematical models in this context should be primarily probabilistic because of the large number of infl uencing natural factors.They must be conceptual to avoid fundamental errors in application.Vistelius devoted his seminal book to geological random sequences and their description and analysis using Markov models as stochastic tools.He applied this approach to grain sequences in granitic intrusive rocks and to sedimentary rock layers.Among other things,Vistelius has used Markov chain analysis in mineral resource exploration to distinguish between“ideal”granites,which are not subsequently mineralized,and mainly hydrothermally mineralized,sometimes ore-bearing granites which contain at least two generations of main minerals.The application of this special conceptual stochastic model is demonstrated on Lusatian granite(Saxony,Germany).
基金supported by the Pioneer Project of Zhejiang Province under Grant 2024C01014the National Natural Science Foundation of China under Grants 52177055 and 52277064。
文摘Permanent magnet synchronous motors(PMSMs), owing to the features of low maintenance costs, great efficiency, and high power density, are extensively utilized in applications such as rail transportation, industrial robots, and new energy electric vehicles. However, the application of space vector pulse width modulation(SVPWM) results in the motor phase current exhibiting clustered harmonic distributions at the integer multiples of the switching frequency, leading to motor noise and vibration issues. To address the issues, this paper proposes a three-random SVPWM(TRPWM) strategy based on a threestate Markov chain, integrating random pulse position, random switching frequency, and random small vector dwell time. By adhering to the principle of voltage-second balance, this strategy spreads the concentrated high-frequency harmonics over a wider frequency range, significantly reducing the magnitude of the concentrated harmonics in the phase current. Comparative experiments with conventional SVPWM, conventional dual-random SVPWM, and conventional three-random SVPWM strategies demonstrate that the proposed approach achieves the expansion of harmonics at integer multiples of the switching frequency in the phase current, effectively suppressing high-frequency vibrations in PMSMs.
文摘This paper considers the variance optimization problem of average reward in continuous-time Markov decision process (MDP). It is assumed that the state space is countable and the action space is Borel measurable space. The main purpose of this paper is to find the policy with the minimal variance in the deterministic stationary policy space. Unlike the traditional Markov decision process, the cost function in the variance criterion will be affected by future actions. To this end, we convert the variance minimization problem into a standard (MDP) by introducing a concept called pseudo-variance. Further, by giving the policy iterative algorithm of pseudo-variance optimization problem, the optimal policy of the original variance optimization problem is derived, and a sufficient condition for the variance optimal policy is given. Finally, we use an example to illustrate the conclusion of this paper.
文摘在多种合理的无信息先验分布下,基于Markov Chain Monte Carlo方法,提出了一种简单且易于抽样的幂律过程的Bayesian分析方法.所提方法将失效、时间截尾数据统一分析,能快捷地获取幂律过程模型参数的Markov Chain Monte Carlo样本,利用该样本不但能直接给出模型参数函数的后验分布,还能给出单样预测和双样预测的分析.一个经典工程数值算例说明了所提方法的可行性、合理性与有效性.该方法具有一定的优越性,可为小子样可靠性增长分析提供一种值得参考的方法.
文摘目的 针对妇幼卫生纵向数据的任意缺失模式,采用多重填补方法进行填补,探求最佳填补结果,以便对数据作进一步分析与研究。方法 运用SAS9.0 ,采用多重填补方法Markov China Monte Carlo(MCMC)模型对缺失数据进行多次填补并综合分析。结果 填补5次所得结果最优。结论 多重填补方法可以处理有缺失数据资料中的许多普遍问题,可提高统计效率,尤其是MCMC模型在处理复杂的缺失数据上,优势明显。
文摘目的对医院出院病人调查表普遍存在的数据缺失进行填补与分析,以保证统计调查表的质量,为医院以及上级卫生部门了解现状,进行预策和决策提供技术支持和质量保证。方法运用SAS9.1,采用多重填补方法Markov Chain Monte Carlo(MCMC)模型对缺失数据进行多次填补并综合分析。结果MCMC填补10次的结果最优。结论(Multiple Imputation)MI方法在解决医院出院病人调查表数据缺失时有优势,发挥空间较大,且填补效率较高。
文摘Modeling non coding background sequences appropriately is important for the detection of regulatory elements from DNA sequences. Based on the chi square statistic test, some explanations about why to choose higher order Markov chain model and how to automatically select the proper order are given in this paper. The chi square test is first run on synthetic data sets to show that it can efficiently find the proper order of Markov chain. Using chi square test, distinct higher order context dependences inherent in ten sets of sequences of yeast S.cerevisiae from other literature have been found. So the Markov chain with higher order would be more suitable for modeling the non coding background sequences than an independent model.
基金The National Science Foundation of China(No.51276036,51306035)the Fundamental Research Funds for the Central Universities(No.KYLX_0114)
文摘A Markov chain-based stochastic model （MCM） is developed to simulate the movement of particles in a 2D bubbling fluidized bed （BFB）. The state spaces are determined by the discretized physical cells of the bed, and the transition probability matrix is directly calculated by the results of a discrete element method （DEM） simulation. The Markov property of the BFB is discussed by the comparison results calculated from both static and dynamic transition probability matrices. The static matrix is calculated based on the Markov chain while the dynamic matrix is calculated based on the memory property of the particle movement. Results show that the difference in the trends of particle movement between the static and dynamic matrix calculation is very small. Besides, the particle mixing curves of the MCM and DEM have the same trend and similar numerical values, and the details show the time averaged characteristic of the MCM and also expose its shortcoming in describing the instantaneous particle dynamics in the BFB.
文摘目的探讨基于Markov Chain Monte Carlo(MCMC)模型的多重估算法在处理医院调查资料缺失数据中的应用。方法运用SAS9.2编写程序,在分析数据的分布类型和缺失机制的基础上,采用MCMC法对缺失数据进行多次填补和联合统计推断,分析多重估算法的优势。结果数据服从多元正态分布与随机缺失,采用MCMC法填补10次所得的结果最佳。结论多重估算既可反映缺失数据的不确定性,又可充分利用现有资料的信息、提高统计效率、对模型的估计结果更加可信,是处理缺失数据的有效方法。
