This paper is concerned with a novel integrated multi-step heuristic dynamic programming(MsHDP)algorithm for solving optimal control problems.It is shown that,initialized by the zero cost function,MsHDP can converge t...This paper is concerned with a novel integrated multi-step heuristic dynamic programming(MsHDP)algorithm for solving optimal control problems.It is shown that,initialized by the zero cost function,MsHDP can converge to the optimal solution of the Hamilton-Jacobi-Bellman(HJB)equation.Then,the stability of the system is analyzed using control policies generated by MsHDP.Also,a general stability criterion is designed to determine the admissibility of the current control policy.That is,the criterion is applicable not only to traditional value iteration and policy iteration but also to MsHDP.Further,based on the convergence and the stability criterion,the integrated MsHDP algorithm using immature control policies is developed to accelerate learning efficiency greatly.Besides,actor-critic is utilized to implement the integrated MsHDP scheme,where neural networks are used to evaluate and improve the iterative policy as the parameter architecture.Finally,two simulation examples are given to demonstrate that the learning effectiveness of the integrated MsHDP scheme surpasses those of other fixed or integrated methods.展开更多
Speciation research during the last several decades has confirmed that natural selection frequently drives the genera- tion of new species. But how does this process generally unfold in nature？ We argue that answerin...Speciation research during the last several decades has confirmed that natural selection frequently drives the genera- tion of new species. But how does this process generally unfold in nature？ We argue that answering this question requires a clearer conceptual framework for understanding selection＇s role in speciation. We present a unified framework of speciation, pro- viding mechanistic descriptions of fundamentally distinct routes to speciation, and how these may interact during lineage splitting Two major categories are recognized： reproductive isolation resulting from （1） responses to selection, ＂speciation by selection,＂ or （2） non-selective processes, ＂speciation without selection.＂ Speciation by selection can occur via three mechanisms： （1） similar selection, （2） divergent selection, and （3） reinforcement selection. Understanding ecology＇s role in speciation requires uncovering how these three mechanisms contribute to reproductive isolation, and their relative importance compared to non-selective proce- sses, because all three mechanisms can occur side-by-side during speciation. To accomplish this, we highlight examination of groups of organisms inhabiting replicated environmental gradients. This scenario is common in nature, and a large literature illus- trates that both parallel and non-parallel responses to similar environments are widespread, and each can result in speciation. This recognition reveals four general pathways of speciation by similar or divergent selection--parallel and nonparallel responses to similar and divergent selection. Altogether, we present a more precise framework for speciation research, draw attention to some under-recognized features of speciation, emphasize the multidimensionality of speciation, reveal limitations of some previous tests and descriptions of speciation mechanisms, and point to a number of directions for future investigation [Current Zoology 59 （1）： 31-52, 2013].展开更多
基金the National Key Research and Development Program of China(2021ZD0112302)the National Natural Science Foundation of China(62222301,61890930-5,62021003)the Beijing Natural Science Foundation(JQ19013).
文摘This paper is concerned with a novel integrated multi-step heuristic dynamic programming(MsHDP)algorithm for solving optimal control problems.It is shown that,initialized by the zero cost function,MsHDP can converge to the optimal solution of the Hamilton-Jacobi-Bellman(HJB)equation.Then,the stability of the system is analyzed using control policies generated by MsHDP.Also,a general stability criterion is designed to determine the admissibility of the current control policy.That is,the criterion is applicable not only to traditional value iteration and policy iteration but also to MsHDP.Further,based on the convergence and the stability criterion,the integrated MsHDP algorithm using immature control policies is developed to accelerate learning efficiency greatly.Besides,actor-critic is utilized to implement the integrated MsHDP scheme,where neural networks are used to evaluate and improve the iterative policy as the parameter architecture.Finally,two simulation examples are given to demonstrate that the learning effectiveness of the integrated MsHDP scheme surpasses those of other fixed or integrated methods.
文摘Speciation research during the last several decades has confirmed that natural selection frequently drives the genera- tion of new species. But how does this process generally unfold in nature？ We argue that answering this question requires a clearer conceptual framework for understanding selection＇s role in speciation. We present a unified framework of speciation, pro- viding mechanistic descriptions of fundamentally distinct routes to speciation, and how these may interact during lineage splitting Two major categories are recognized： reproductive isolation resulting from （1） responses to selection, ＂speciation by selection,＂ or （2） non-selective processes, ＂speciation without selection.＂ Speciation by selection can occur via three mechanisms： （1） similar selection, （2） divergent selection, and （3） reinforcement selection. Understanding ecology＇s role in speciation requires uncovering how these three mechanisms contribute to reproductive isolation, and their relative importance compared to non-selective proce- sses, because all three mechanisms can occur side-by-side during speciation. To accomplish this, we highlight examination of groups of organisms inhabiting replicated environmental gradients. This scenario is common in nature, and a large literature illus- trates that both parallel and non-parallel responses to similar environments are widespread, and each can result in speciation. This recognition reveals four general pathways of speciation by similar or divergent selection--parallel and nonparallel responses to similar and divergent selection. Altogether, we present a more precise framework for speciation research, draw attention to some under-recognized features of speciation, emphasize the multidimensionality of speciation, reveal limitations of some previous tests and descriptions of speciation mechanisms, and point to a number of directions for future investigation [Current Zoology 59 （1）： 31-52, 2013].
