In this paper we will see the model of Universe according to Dynamic Universe Model of Cosmology by visualizing various processes that are happening in the Universe as per experimental evidences. For simplifying the m...In this paper we will see the model of Universe according to Dynamic Universe Model of Cosmology by visualizing various processes that are happening in the Universe as per experimental evidences. For simplifying the matter here, we will see in part 1: about the Galaxy life cycle, where the birth and death of Galaxies discussed. Probably Universe gives guidance for the movement of Galaxies. We call this Part 1: Thinking and Reproducing Universe or Mindless Universe? (Galaxy life cycle). We see every day Sun, Stars, Galaxies etc., dissipating enormous energy in the form of radiation by the way of fusion of Hydrogen to helium. So after sometime all the Hydrogen is spent and Universe will die, is it not? … Dynamic Universe Model says that the energy in the form of electromagnetic radiation passing grazingly near any gravitating mass changes in frequency and finally will convert into neutrinos (mass). Hence Dynamic Universe Model proposes another process where energy will be converted back into matter and the cycle energy to mass to energy continues, sustaining the Universe to maintain this present status for ever in this form something like a Steady state model without any expansion. This we will see in Part 2: Energy - Mass - Energy Cycle. After converting energy into mass “how various elements are formed and where they are formed?” will be next logical question. Dynamic Universe Model says that these various particles change into higher massive particles or may get bombarded into stars or planets and various elements are formed. Here we bifurcate the formation of elements into 6 processes. They are for Elementary particles and elements generated in frequency changing process, By Cosmic rays, By Small stars, By Large Stars, By Super Novae and Manmade elements By Neutron Stars. This we will discuss in Part 3: Nucleosynthesis.展开更多
This paper introduces a practical solving scheme of gradetransition trajectory optimization(GTTO) problems under typical certificate-checking–updating framework. Due to complicated kinetics of polymerization,differen...This paper introduces a practical solving scheme of gradetransition trajectory optimization(GTTO) problems under typical certificate-checking–updating framework. Due to complicated kinetics of polymerization,differential/algebraic equations(DAEs) always cause great computational burden and system non-linearity usually makes GTTO non-convex bearing multiple optima. Therefore, coupled with the three-stage decomposition model, a three-section algorithm of dynamic programming(TSDP) is proposed based on the general iteration mechanism of iterative programming(IDP) and incorporated with adaptivegrid allocation scheme and heuristic modifications. The algorithm iteratively performs dynamic programming with heuristic modifications under constant calculation loads and adaptively allocates the valued computational resources to the regions that can further improve the optimality under the guidance of local error estimates. TSDP is finally compared with IDP and interior point method(IP) to verify its efficiency of computation.展开更多
This paper studies data-driven learning-based methods for the finite-horizon optimal control of linear time-varying discretetime systems. First, a novel finite-horizon Policy Iteration (PI) method for linear time-vary...This paper studies data-driven learning-based methods for the finite-horizon optimal control of linear time-varying discretetime systems. First, a novel finite-horizon Policy Iteration (PI) method for linear time-varying discrete-time systems is presented. Its connections with existing in finite-horizon PI methods are discussed. Then, both data-drive n off-policy PI and Value Iteration (VI) algorithms are derived to find approximate optimal controllers when the system dynamics is completely unknown. Under mild conditions, the proposed data-driven off-policy algorithms converge to the optimal solution. Finally, the effectiveness and feasibility of the developed methods are validated by a practical example of spacecraft attitude control.展开更多
This paper is concerned with the relationship between maximum principle and dynamic programming in zero-sum stochastic differential games. Under the assumption that the value function is enough smooth, relations among...This paper is concerned with the relationship between maximum principle and dynamic programming in zero-sum stochastic differential games. Under the assumption that the value function is enough smooth, relations among the adjoint processes, the generalized Hamiltonian function and the value function are given. A portfolio optimization problem under model uncertainty in the financial market is discussed to show the applications of our result.展开更多
Considering the economics and securities for the operation of a power system, this paper presents a new adaptive dynamic programming approach for security-constrained unit commitment (SCUC) problems. In response to t...Considering the economics and securities for the operation of a power system, this paper presents a new adaptive dynamic programming approach for security-constrained unit commitment (SCUC) problems. In response to the “curse of dimension” problem of dynamic programming, the approach solves the Bellman’s equation of SCUC approximately by solving a sequence of simplified single stage optimization problems. An extended sequential truncation technique is proposed to explore the state space of the approach, which is superior to traditional sequential truncation in daily cost for unit commitment. Different test cases from 30 to 300 buses over a 24 h horizon are analyzed. Extensive numerical comparisons show that the proposed approach is capable of obtaining the optimal unit commitment schedules without any network and bus voltage violations, and minimizing the operation cost as well.展开更多
This paper presents a new design approach to achieve decentralized optimal control of high-dimension complex singular systems with dynamic uncertainties. Based on robust adaptive dynamic programming(robust ADP) method...This paper presents a new design approach to achieve decentralized optimal control of high-dimension complex singular systems with dynamic uncertainties. Based on robust adaptive dynamic programming(robust ADP) method, controllers for solving the singular systems optimal control problem are designed. The proposed algorithm can work well when the system model is not exactly known but the input and output data can be measured. The policy iteration of each controller only uses their own states and input information for learning,and do not need to know the whole system dynamics. Simulation results on the New England 10-machine 39-bus test system show the effectiveness of the designed controller.展开更多
Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and ...Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.展开更多
In this article we derive a general differential equation that describes long-term economic growth in terms of cyclical and trend components. Equation is based on the model of non-linear accelerator of induced investm...In this article we derive a general differential equation that describes long-term economic growth in terms of cyclical and trend components. Equation is based on the model of non-linear accelerator of induced investment. A scheme is proposed for obtaining approximate solutions of nonlinear differential equation by splitting solution into the rapidly oscillating business cycles and slowly varying trend using Krylov-Bogoliubov-Mitropolsky averaging. Simplest modes of the economic system are described. Characteristics of the bifurcation point are found and bifurcation phenomenon is interpreted as loss of stability making the economic system available to structural change and accepting innovations. System being in a nonequilibrium state has a dynamics with self-sustained undamped oscillations. The model is verified with economic development of the US during the fifth Kondratieff cycle (1982-2010). Model adequately describes real process of economic growth in both quantitative and qualitative aspects. It is one of major results that the model gives a rough estimation of critical points of system stability loss and falling into a crisis recession. The model is used to forecast the macroeconomic dynamics of the US during the sixth Kondratieff cycle (2018-2050). For this forecast we use fixed production capital functional dependence on a long-term Kondratieff cycle and medium-term Juglar and Kuznets cycles. More accurate estimations of the time of crisis and recession are based on the model of accelerating log-periodic oscillations. The explosive growth of the prices of highly liquid commodities such as gold and oil is taken as real predictors of the global financial crisis. The second wave of crisis is expected to come in June 2011.展开更多
Dynamic Programming (DP) algorithm is used to find the optimal trajectories under Beijing cycle for the power management of synergic electric system (SES) which is composed of battery and super capacitor. Feasible rul...Dynamic Programming (DP) algorithm is used to find the optimal trajectories under Beijing cycle for the power management of synergic electric system (SES) which is composed of battery and super capacitor. Feasible rules are derived from analyzing the optimal trajectories, and it has the highest contribution to Hybrid Electric Vehicle (HEV). The methods of how to get the best performance is also educed. Using the new Rule-based power management strat-egy adopted from the optimal results, it is easy to demonstrate the effectiveness of the new strategy in further improvement of the fuel economy by the synergic hybrid system.展开更多
Learning-based methods have become mainstream for solving residential energy scheduling problems. In order to improve the learning efficiency of existing methods and increase the utilization of renewable energy, we pr...Learning-based methods have become mainstream for solving residential energy scheduling problems. In order to improve the learning efficiency of existing methods and increase the utilization of renewable energy, we propose the Dyna actiondependent heuristic dynamic programming(Dyna-ADHDP)method, which incorporates the ideas of learning and planning from the Dyna framework in action-dependent heuristic dynamic programming. This method defines a continuous action space for precise control of an energy storage system and allows online optimization of algorithm performance during the real-time operation of the residential energy model. Meanwhile, the target network is introduced during the training process to make the training smoother and more efficient. We conducted experimental comparisons with the benchmark method using simulated and real data to verify its applicability and performance. The results confirm the method's excellent performance and generalization capabilities, as well as its excellence in increasing renewable energy utilization and extending equipment life.展开更多
In this paper,a distributed adaptive dynamic programming(ADP)framework based on value iteration is proposed for multi-player differential games.In the game setting,players have no access to the information of others&#...In this paper,a distributed adaptive dynamic programming(ADP)framework based on value iteration is proposed for multi-player differential games.In the game setting,players have no access to the information of others'system parameters or control laws.Each player adopts an on-policy value iteration algorithm as the basic learning framework.To deal with the incomplete information structure,players collect a period of system trajectory data to compensate for the lack of information.The policy updating step is implemented by a nonlinear optimization problem aiming to search for the proximal admissible policy.Theoretical analysis shows that by adopting proximal policy searching rules,the approximated policies can converge to a neighborhood of equilibrium policies.The efficacy of our method is illustrated by three examples,which also demonstrate that the proposed method can accelerate the learning process compared with the centralized learning framework.展开更多
This paper studies the problem of optimal parallel tracking control for continuous-time general nonlinear systems.Unlike existing optimal state feedback control,the control input of the optimal parallel control is int...This paper studies the problem of optimal parallel tracking control for continuous-time general nonlinear systems.Unlike existing optimal state feedback control,the control input of the optimal parallel control is introduced into the feedback system.However,due to the introduction of control input into the feedback system,the optimal state feedback control methods can not be applied directly.To address this problem,an augmented system and an augmented performance index function are proposed firstly.Thus,the general nonlinear system is transformed into an affine nonlinear system.The difference between the optimal parallel control and the optimal state feedback control is analyzed theoretically.It is proven that the optimal parallel control with the augmented performance index function can be seen as the suboptimal state feedback control with the traditional performance index function.Moreover,an adaptive dynamic programming(ADP)technique is utilized to implement the optimal parallel tracking control using a critic neural network(NN)to approximate the value function online.The stability analysis of the closed-loop system is performed using the Lyapunov theory,and the tracking error and NN weights errors are uniformly ultimately bounded(UUB).Also,the optimal parallel controller guarantees the continuity of the control input under the circumstance that there are finite jump discontinuities in the reference signals.Finally,the effectiveness of the developed optimal parallel control method is verified in two cases.展开更多
The aim of this work is to develop an improved region based active contour and dynamic programming based method for accurate segmentation of left ventricle （LV） from multi-slice cine short axis cardiac magnetic reso...The aim of this work is to develop an improved region based active contour and dynamic programming based method for accurate segmentation of left ventricle （LV） from multi-slice cine short axis cardiac magnetic resonance （MR） images. Intensity inhomogeneity and weak object boundaries present in MR images hinder the segmentation accuracy. The proposed active contour model driven by a local Gaussian distribution fitting （LGDF） energy and an auxiliary global intensity fitting energy improves the accuracy of endocardial boundary detection. The weightage of the global energy fitting term is dynamically adjusted using a spatially varying weight function. Dynamic programming scheme proposed for the segmentation of epicardium considers the myocardium probability map and a distance weighted edge map in the cost matrix. Radial distance weighted technique and conical geometry are employed for segmenting the basal slices with left ventricle outflow tract （LVOT） and most apical slices. The proposed method is validated on a public dataset comprising 45 subjects from medical image computing and computer assisted interventions （MICCAI） 2009 segmentation challenge. The average percentage of good endocardial and epicardial contours detected is about 99%, average perpendicular distance of the detected good contours from the manual reference contours is 1.95 mm, and the dice similarity coefficient between the detected contours and the reference contours is 0.91. Correlation coefficient and the coefficient of determination between the ejection fraction measurements from manual segmentation and the automated method are respectively 0.9781 and 0.9567, for LV mass these values are 0.9249 and 0.8554. Statistical analysis of the results reveals a good agreement between the clinical parameters determined manually and those estimated using the automated method.展开更多
In the Cellular Long-Term Evolution (LTE) downlink, the smallest radio resource unit a Scheduler can assign to a user is a Resource Block (RB). Each RB consists of twelve (12) adjacent Orthogonal Frequency Division Mu...In the Cellular Long-Term Evolution (LTE) downlink, the smallest radio resource unit a Scheduler can assign to a user is a Resource Block (RB). Each RB consists of twelve (12) adjacent Orthogonal Frequency Division Multiplexing (OFDM) sub-carriers with inter-subcarrier spacing of 15 kHz. Over the years, researchers have investigated the problem of radio resource allocation in cellular LTE downlink and have made useful contributions. In an earlier paper for example, we proposed a deterministic dynamic programming based technique for optimal allocation of RBs in the downlink of multiuser Cellular LTE System. We found that this proposed methodology optimally allocates RBs to users at every transmission instant, but the computational time associated with the allocation policy was high. In the current work, we propose a truncated dynamic programming based technique for efficient and optimal allocation of radio resource. This paper also addresses uncertainty emanating from users’ mobility within a Cell coverage area. The objective is to significantly reduce the computational time and dynamically select applicable modulation scheme (i.e., QPSK, 16QAM, or 64QAM) in response to users’ mobility. We compare the proposed scheme with the Fair allocation and the earlier proposed dynamic programming based techniques. It is shown that the proposed methodology is more efficient in allocating radio resource and has better performance than both the Fair Allocation and the deterministic dynamic programming based techniques.展开更多
The residential energy scheduling of solar energy is an important research area of smart grid. On the demand side, factors such as household loads, storage batteries, the outside public utility grid and renewable ener...The residential energy scheduling of solar energy is an important research area of smart grid. On the demand side, factors such as household loads, storage batteries, the outside public utility grid and renewable energy resources, are combined together as a nonlinear, time-varying, indefinite and complex system, which is difficult to manage or optimize. Many nations have already applied the residential real-time pricing to balance the burden on their grid. In order to enhance electricity efficiency of the residential micro grid, this paper presents an action dependent heuristic dynamic programming(ADHDP) method to solve the residential energy scheduling problem. The highlights of this paper are listed below. First,the weather-type classification is adopted to establish three types of programming models based on the features of the solar energy. In addition, the priorities of different energy resources are set to reduce the loss of electrical energy transmissions.Second, three ADHDP-based neural networks, which can update themselves during applications, are designed to manage the flows of electricity. Third, simulation results show that the proposed scheduling method has effectively reduced the total electricity cost and improved load balancing process. The comparison with the particle swarm optimization algorithm further proves that the present method has a promising effect on energy management to save cost.展开更多
This paper researches the adaptive scheduling problem of multiple electronic support measures(multi-ESM) in a ground moving radar targets tracking application. It is a sequential decision-making problem in uncertain e...This paper researches the adaptive scheduling problem of multiple electronic support measures(multi-ESM) in a ground moving radar targets tracking application. It is a sequential decision-making problem in uncertain environment. For adaptive selection of appropriate ESMs, we generalize an approximate dynamic programming(ADP) framework to the dynamic case. We define the environment model and agent model, respectively. To handle the partially observable challenge, we apply the unsented Kalman filter(UKF) algorithm for belief state estimation. To reduce the computational burden, a simulation-based approach rollout with a redesigned base policy is proposed to approximate the long-term cumulative reward. Meanwhile, Monte Carlo sampling is combined into the rollout to estimate the expectation of the rewards. The experiments indicate that our method outperforms other strategies due to its better performance in larger-scale problems.展开更多
A simplified model is proposed for an easy understanding of the coarse-grained technique and for achieving a first approximation to the behavior of gases. A mole of a gas substance, within a cubic container, is repres...A simplified model is proposed for an easy understanding of the coarse-grained technique and for achieving a first approximation to the behavior of gases. A mole of a gas substance, within a cubic container, is represented by six particles symmetrically moving. The impacts of particles on container walls, the inter-particle collisions, as well as the volume of particles and the inter-particle attractive forces, obeying a Lennard-Jones curve, are taken into account. Thanks to the symmetry, the problem is reduced to the nonlinear dynamic analysis of a SDOF oscillator, which is numerically solved by a step-by-step time integration algorithm. Five applications of proposed model, on Carbon Dioxide, are presented: 1) Ideal gas in STP conditions. 2) Real gas in STP conditions. 3) Condensation for small molar volume. 4) Critical point. 5) Iso-kinetic energy curves and iso-therms in the critical point region. Results of the proposed model are compared with test data and results of the Van der Waals model for real gases.展开更多
In this paper,a novel adaptive Fault-Tolerant Control(FTC)strategy is proposed for non-minimum phase Hypersonic Vehicles(HSVs)that are affected by actuator faults and parameter uncertainties.The strategy is based on t...In this paper,a novel adaptive Fault-Tolerant Control(FTC)strategy is proposed for non-minimum phase Hypersonic Vehicles(HSVs)that are affected by actuator faults and parameter uncertainties.The strategy is based on the output redefinition method and Adaptive Dynamic Programming(ADP).The intelligent FTC scheme consists of two main parts:a basic fault-tolerant and stable controller and an ADP-based supplementary controller.In the basic FTC part,an output redefinition approach is designed to make zero-dynamics stable with respect to the new output.Then,Ideal Internal Dynamic(IID)is obtained using an optimal bounded inversion approach,and a tracking controller is designed for the new output to realize output tracking of the nonminimum phase HSV system.For the ADP-based compensation control part,an ActionDependent Heuristic Dynamic Programming(ADHDP)adopting an actor-critic learning structure is utilized to further optimize the tracking performance of the HSV control system.Finally,simulation results are provided to verify the effectiveness and efficiency of the proposed FTC algorithm.展开更多
文摘In this paper we will see the model of Universe according to Dynamic Universe Model of Cosmology by visualizing various processes that are happening in the Universe as per experimental evidences. For simplifying the matter here, we will see in part 1: about the Galaxy life cycle, where the birth and death of Galaxies discussed. Probably Universe gives guidance for the movement of Galaxies. We call this Part 1: Thinking and Reproducing Universe or Mindless Universe? (Galaxy life cycle). We see every day Sun, Stars, Galaxies etc., dissipating enormous energy in the form of radiation by the way of fusion of Hydrogen to helium. So after sometime all the Hydrogen is spent and Universe will die, is it not? … Dynamic Universe Model says that the energy in the form of electromagnetic radiation passing grazingly near any gravitating mass changes in frequency and finally will convert into neutrinos (mass). Hence Dynamic Universe Model proposes another process where energy will be converted back into matter and the cycle energy to mass to energy continues, sustaining the Universe to maintain this present status for ever in this form something like a Steady state model without any expansion. This we will see in Part 2: Energy - Mass - Energy Cycle. After converting energy into mass “how various elements are formed and where they are formed?” will be next logical question. Dynamic Universe Model says that these various particles change into higher massive particles or may get bombarded into stars or planets and various elements are formed. Here we bifurcate the formation of elements into 6 processes. They are for Elementary particles and elements generated in frequency changing process, By Cosmic rays, By Small stars, By Large Stars, By Super Novae and Manmade elements By Neutron Stars. This we will discuss in Part 3: Nucleosynthesis.
基金Supported by the National Basic Research Program of China(2012CB720500)the National High Technology Research and Development Program of China(2013AA040702)
文摘This paper introduces a practical solving scheme of gradetransition trajectory optimization(GTTO) problems under typical certificate-checking–updating framework. Due to complicated kinetics of polymerization,differential/algebraic equations(DAEs) always cause great computational burden and system non-linearity usually makes GTTO non-convex bearing multiple optima. Therefore, coupled with the three-stage decomposition model, a three-section algorithm of dynamic programming(TSDP) is proposed based on the general iteration mechanism of iterative programming(IDP) and incorporated with adaptivegrid allocation scheme and heuristic modifications. The algorithm iteratively performs dynamic programming with heuristic modifications under constant calculation loads and adaptively allocates the valued computational resources to the regions that can further improve the optimality under the guidance of local error estimates. TSDP is finally compared with IDP and interior point method(IP) to verify its efficiency of computation.
基金The work of B. Pang and Z.-P. Jiang has been supported in part by the National Science Foundation (No. ECCS-1501044).
文摘This paper studies data-driven learning-based methods for the finite-horizon optimal control of linear time-varying discretetime systems. First, a novel finite-horizon Policy Iteration (PI) method for linear time-varying discrete-time systems is presented. Its connections with existing in finite-horizon PI methods are discussed. Then, both data-drive n off-policy PI and Value Iteration (VI) algorithms are derived to find approximate optimal controllers when the system dynamics is completely unknown. Under mild conditions, the proposed data-driven off-policy algorithms converge to the optimal solution. Finally, the effectiveness and feasibility of the developed methods are validated by a practical example of spacecraft attitude control.
文摘This paper is concerned with the relationship between maximum principle and dynamic programming in zero-sum stochastic differential games. Under the assumption that the value function is enough smooth, relations among the adjoint processes, the generalized Hamiltonian function and the value function are given. A portfolio optimization problem under model uncertainty in the financial market is discussed to show the applications of our result.
文摘Considering the economics and securities for the operation of a power system, this paper presents a new adaptive dynamic programming approach for security-constrained unit commitment (SCUC) problems. In response to the “curse of dimension” problem of dynamic programming, the approach solves the Bellman’s equation of SCUC approximately by solving a sequence of simplified single stage optimization problems. An extended sequential truncation technique is proposed to explore the state space of the approach, which is superior to traditional sequential truncation in daily cost for unit commitment. Different test cases from 30 to 300 buses over a 24 h horizon are analyzed. Extensive numerical comparisons show that the proposed approach is capable of obtaining the optimal unit commitment schedules without any network and bus voltage violations, and minimizing the operation cost as well.
基金supported in part by the National Natural Science Foundation of China(61473070,61433004,61627809)SAPI Fundamental Research Funds(2018ZCX22)
文摘This paper presents a new design approach to achieve decentralized optimal control of high-dimension complex singular systems with dynamic uncertainties. Based on robust adaptive dynamic programming(robust ADP) method, controllers for solving the singular systems optimal control problem are designed. The proposed algorithm can work well when the system model is not exactly known but the input and output data can be measured. The policy iteration of each controller only uses their own states and input information for learning,and do not need to know the whole system dynamics. Simulation results on the New England 10-machine 39-bus test system show the effectiveness of the designed controller.
基金supported in part by the National Natural Science Foundation of China(62222301, 62073085, 62073158, 61890930-5, 62021003)the National Key Research and Development Program of China (2021ZD0112302, 2021ZD0112301, 2018YFC1900800-5)Beijing Natural Science Foundation (JQ19013)。
文摘Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.
基金supported in part by National Natural Science Foundation of China(61533017,61273140,61304079,61374105,61379099,61233001)Fundamental Research Funds for the Central Universities(FRF-TP-15-056A3)the Open Research Project from SKLMCCS(20150104)
文摘In this article we derive a general differential equation that describes long-term economic growth in terms of cyclical and trend components. Equation is based on the model of non-linear accelerator of induced investment. A scheme is proposed for obtaining approximate solutions of nonlinear differential equation by splitting solution into the rapidly oscillating business cycles and slowly varying trend using Krylov-Bogoliubov-Mitropolsky averaging. Simplest modes of the economic system are described. Characteristics of the bifurcation point are found and bifurcation phenomenon is interpreted as loss of stability making the economic system available to structural change and accepting innovations. System being in a nonequilibrium state has a dynamics with self-sustained undamped oscillations. The model is verified with economic development of the US during the fifth Kondratieff cycle (1982-2010). Model adequately describes real process of economic growth in both quantitative and qualitative aspects. It is one of major results that the model gives a rough estimation of critical points of system stability loss and falling into a crisis recession. The model is used to forecast the macroeconomic dynamics of the US during the sixth Kondratieff cycle (2018-2050). For this forecast we use fixed production capital functional dependence on a long-term Kondratieff cycle and medium-term Juglar and Kuznets cycles. More accurate estimations of the time of crisis and recession are based on the model of accelerating log-periodic oscillations. The explosive growth of the prices of highly liquid commodities such as gold and oil is taken as real predictors of the global financial crisis. The second wave of crisis is expected to come in June 2011.
文摘Dynamic Programming (DP) algorithm is used to find the optimal trajectories under Beijing cycle for the power management of synergic electric system (SES) which is composed of battery and super capacitor. Feasible rules are derived from analyzing the optimal trajectories, and it has the highest contribution to Hybrid Electric Vehicle (HEV). The methods of how to get the best performance is also educed. Using the new Rule-based power management strat-egy adopted from the optimal results, it is easy to demonstrate the effectiveness of the new strategy in further improvement of the fuel economy by the synergic hybrid system.
基金supported in part by the National Key Research and Development Program of China(2024YFB4709100,2021YFE0206100)the National Natural Science Foundation of China(62073321)+1 种基金the National Defense Basic Scientific Research Program(JCKY2019203C029)the Science and Technology Development Fund,Macao SAR,China(0015/2020/AMJ)
文摘Learning-based methods have become mainstream for solving residential energy scheduling problems. In order to improve the learning efficiency of existing methods and increase the utilization of renewable energy, we propose the Dyna actiondependent heuristic dynamic programming(Dyna-ADHDP)method, which incorporates the ideas of learning and planning from the Dyna framework in action-dependent heuristic dynamic programming. This method defines a continuous action space for precise control of an energy storage system and allows online optimization of algorithm performance during the real-time operation of the residential energy model. Meanwhile, the target network is introduced during the training process to make the training smoother and more efficient. We conducted experimental comparisons with the benchmark method using simulated and real data to verify its applicability and performance. The results confirm the method's excellent performance and generalization capabilities, as well as its excellence in increasing renewable energy utilization and extending equipment life.
基金supported by the Aeronautical Science Foundation of China(20220001057001)an Open Project of the National Key Laboratory of Air-based Information Perception and Fusion(202437)
文摘In this paper,a distributed adaptive dynamic programming(ADP)framework based on value iteration is proposed for multi-player differential games.In the game setting,players have no access to the information of others'system parameters or control laws.Each player adopts an on-policy value iteration algorithm as the basic learning framework.To deal with the incomplete information structure,players collect a period of system trajectory data to compensate for the lack of information.The policy updating step is implemented by a nonlinear optimization problem aiming to search for the proximal admissible policy.Theoretical analysis shows that by adopting proximal policy searching rules,the approximated policies can converge to a neighborhood of equilibrium policies.The efficacy of our method is illustrated by three examples,which also demonstrate that the proposed method can accelerate the learning process compared with the centralized learning framework.
基金supported in part by the National Key Reseanch and Development Program of China(2018AAA0101502,2018YFB1702300)in part by the National Natural Science Foundation of China(61722312,61533019,U1811463,61533017)in part by the Intel Collaborative Research Institute for Intelligent and Automated Connected Vehicles。
文摘This paper studies the problem of optimal parallel tracking control for continuous-time general nonlinear systems.Unlike existing optimal state feedback control,the control input of the optimal parallel control is introduced into the feedback system.However,due to the introduction of control input into the feedback system,the optimal state feedback control methods can not be applied directly.To address this problem,an augmented system and an augmented performance index function are proposed firstly.Thus,the general nonlinear system is transformed into an affine nonlinear system.The difference between the optimal parallel control and the optimal state feedback control is analyzed theoretically.It is proven that the optimal parallel control with the augmented performance index function can be seen as the suboptimal state feedback control with the traditional performance index function.Moreover,an adaptive dynamic programming(ADP)technique is utilized to implement the optimal parallel tracking control using a critic neural network(NN)to approximate the value function online.The stability analysis of the closed-loop system is performed using the Lyapunov theory,and the tracking error and NN weights errors are uniformly ultimately bounded(UUB).Also,the optimal parallel controller guarantees the continuity of the control input under the circumstance that there are finite jump discontinuities in the reference signals.Finally,the effectiveness of the developed optimal parallel control method is verified in two cases.
基金supported by Department of Science and Technology, Ministry of Science and Technology, India (No. DST/TSG/ICT/2010/08)
文摘The aim of this work is to develop an improved region based active contour and dynamic programming based method for accurate segmentation of left ventricle （LV） from multi-slice cine short axis cardiac magnetic resonance （MR） images. Intensity inhomogeneity and weak object boundaries present in MR images hinder the segmentation accuracy. The proposed active contour model driven by a local Gaussian distribution fitting （LGDF） energy and an auxiliary global intensity fitting energy improves the accuracy of endocardial boundary detection. The weightage of the global energy fitting term is dynamically adjusted using a spatially varying weight function. Dynamic programming scheme proposed for the segmentation of epicardium considers the myocardium probability map and a distance weighted edge map in the cost matrix. Radial distance weighted technique and conical geometry are employed for segmenting the basal slices with left ventricle outflow tract （LVOT） and most apical slices. The proposed method is validated on a public dataset comprising 45 subjects from medical image computing and computer assisted interventions （MICCAI） 2009 segmentation challenge. The average percentage of good endocardial and epicardial contours detected is about 99%, average perpendicular distance of the detected good contours from the manual reference contours is 1.95 mm, and the dice similarity coefficient between the detected contours and the reference contours is 0.91. Correlation coefficient and the coefficient of determination between the ejection fraction measurements from manual segmentation and the automated method are respectively 0.9781 and 0.9567, for LV mass these values are 0.9249 and 0.8554. Statistical analysis of the results reveals a good agreement between the clinical parameters determined manually and those estimated using the automated method.
文摘In the Cellular Long-Term Evolution (LTE) downlink, the smallest radio resource unit a Scheduler can assign to a user is a Resource Block (RB). Each RB consists of twelve (12) adjacent Orthogonal Frequency Division Multiplexing (OFDM) sub-carriers with inter-subcarrier spacing of 15 kHz. Over the years, researchers have investigated the problem of radio resource allocation in cellular LTE downlink and have made useful contributions. In an earlier paper for example, we proposed a deterministic dynamic programming based technique for optimal allocation of RBs in the downlink of multiuser Cellular LTE System. We found that this proposed methodology optimally allocates RBs to users at every transmission instant, but the computational time associated with the allocation policy was high. In the current work, we propose a truncated dynamic programming based technique for efficient and optimal allocation of radio resource. This paper also addresses uncertainty emanating from users’ mobility within a Cell coverage area. The objective is to significantly reduce the computational time and dynamically select applicable modulation scheme (i.e., QPSK, 16QAM, or 64QAM) in response to users’ mobility. We compare the proposed scheme with the Fair allocation and the earlier proposed dynamic programming based techniques. It is shown that the proposed methodology is more efficient in allocating radio resource and has better performance than both the Fair Allocation and the deterministic dynamic programming based techniques.
基金supported in part by the National Natural Science Foundation of China(61533017,U1501251,61374105,61722312)
文摘The residential energy scheduling of solar energy is an important research area of smart grid. On the demand side, factors such as household loads, storage batteries, the outside public utility grid and renewable energy resources, are combined together as a nonlinear, time-varying, indefinite and complex system, which is difficult to manage or optimize. Many nations have already applied the residential real-time pricing to balance the burden on their grid. In order to enhance electricity efficiency of the residential micro grid, this paper presents an action dependent heuristic dynamic programming(ADHDP) method to solve the residential energy scheduling problem. The highlights of this paper are listed below. First,the weather-type classification is adopted to establish three types of programming models based on the features of the solar energy. In addition, the priorities of different energy resources are set to reduce the loss of electrical energy transmissions.Second, three ADHDP-based neural networks, which can update themselves during applications, are designed to manage the flows of electricity. Third, simulation results show that the proposed scheduling method has effectively reduced the total electricity cost and improved load balancing process. The comparison with the particle swarm optimization algorithm further proves that the present method has a promising effect on energy management to save cost.
基金supported by the National Natural Science Foundation of China(6157328561305133)
文摘This paper researches the adaptive scheduling problem of multiple electronic support measures(multi-ESM) in a ground moving radar targets tracking application. It is a sequential decision-making problem in uncertain environment. For adaptive selection of appropriate ESMs, we generalize an approximate dynamic programming(ADP) framework to the dynamic case. We define the environment model and agent model, respectively. To handle the partially observable challenge, we apply the unsented Kalman filter(UKF) algorithm for belief state estimation. To reduce the computational burden, a simulation-based approach rollout with a redesigned base policy is proposed to approximate the long-term cumulative reward. Meanwhile, Monte Carlo sampling is combined into the rollout to estimate the expectation of the rewards. The experiments indicate that our method outperforms other strategies due to its better performance in larger-scale problems.
文摘A simplified model is proposed for an easy understanding of the coarse-grained technique and for achieving a first approximation to the behavior of gases. A mole of a gas substance, within a cubic container, is represented by six particles symmetrically moving. The impacts of particles on container walls, the inter-particle collisions, as well as the volume of particles and the inter-particle attractive forces, obeying a Lennard-Jones curve, are taken into account. Thanks to the symmetry, the problem is reduced to the nonlinear dynamic analysis of a SDOF oscillator, which is numerically solved by a step-by-step time integration algorithm. Five applications of proposed model, on Carbon Dioxide, are presented: 1) Ideal gas in STP conditions. 2) Real gas in STP conditions. 3) Condensation for small molar volume. 4) Critical point. 5) Iso-kinetic energy curves and iso-therms in the critical point region. Results of the proposed model are compared with test data and results of the Van der Waals model for real gases.
基金supported in part by the Science Center Program of National Natural Science Foundation of China(62373189,62188101,62020106003)the Research Fund of State Key Laboratory of Mechanics and Control for Aerospace Structures,China。
文摘In this paper,a novel adaptive Fault-Tolerant Control(FTC)strategy is proposed for non-minimum phase Hypersonic Vehicles(HSVs)that are affected by actuator faults and parameter uncertainties.The strategy is based on the output redefinition method and Adaptive Dynamic Programming(ADP).The intelligent FTC scheme consists of two main parts:a basic fault-tolerant and stable controller and an ADP-based supplementary controller.In the basic FTC part,an output redefinition approach is designed to make zero-dynamics stable with respect to the new output.Then,Ideal Internal Dynamic(IID)is obtained using an optimal bounded inversion approach,and a tracking controller is designed for the new output to realize output tracking of the nonminimum phase HSV system.For the ADP-based compensation control part,an ActionDependent Heuristic Dynamic Programming(ADHDP)adopting an actor-critic learning structure is utilized to further optimize the tracking performance of the HSV control system.Finally,simulation results are provided to verify the effectiveness and efficiency of the proposed FTC algorithm.
