Reliable and efficient communication is essential for Unmanned Aerial Vehicle(UAV)networks,especially in dynamic and resource-constrained environments such as disaster management,surveillance,and environmental monitor...Reliable and efficient communication is essential for Unmanned Aerial Vehicle(UAV)networks,especially in dynamic and resource-constrained environments such as disaster management,surveillance,and environmental monitoring.Frequent topology changes,high mobility,and limited energy availability pose significant challenges to maintaining stable and high-performance routing.Traditional routing protocols,such as Ad hoc On-Demand Distance Vector(AODV),Load-Balanced Optimized Predictive Ad hoc Routing(LB-OPAR),and Destination-Sequenced Distance Vector(DSDV),often experience performance degradation under such conditions.To address these limitations,this study evaluates the effectiveness of Dynamic Adaptive Routing(DAR),a protocol designed to adapt routing decisions in real time based on network dynamics and resource constraints.The research utilizes the Network Simulator 3(NS-3)platform to conduct controlled simulations,measuring key performance indicators such as latency,Packet Delivery Ratio(PDR),energy consumption,and throughput.Comparative analysis reveals that DAR consistently outperforms conventional protocols,achieving a 20%-30% reduction in latency,a 25% decrease in energy consumption,and marked improvements in throughput and PDR.These results highlight DAR’s ability to maintain high communication reliability while optimizing resource usage in challenging operational scenarios.By providing empirical evidence of DAR’s advantages in highly dynamic UAV network environments,this study contributes to advancing adaptive routing strategies.The findings not only validate DAR’s robustness and scalability but also lay the groundwork for integrating artificial intelligence-driven decision-making and real-world UAV deployment.Future work will explore cross-layer optimization,multi-UAV coordination,and experimental validation in field trials,aiming to further enhance communication resilience and energy efficiency in next-generation aerial networks.展开更多
Dynamic adaptability is a key feature in biological macromolecules,enabling selective binding and catalysis[1].From DNA supercoiling to enzyme conformational changes,biological systems have evolved intricate ways to d...Dynamic adaptability is a key feature in biological macromolecules,enabling selective binding and catalysis[1].From DNA supercoiling to enzyme conformational changes,biological systems have evolved intricate ways to dynamically adjust their structures to accommodate functional needs.Mimicking this adaptability in synthetic systems is an ongoing challenge in supramolecular chemistry.展开更多
In this paper,a distributed adaptive dynamic programming(ADP)framework based on value iteration is proposed for multi-player differential games.In the game setting,players have no access to the information of others&#...In this paper,a distributed adaptive dynamic programming(ADP)framework based on value iteration is proposed for multi-player differential games.In the game setting,players have no access to the information of others'system parameters or control laws.Each player adopts an on-policy value iteration algorithm as the basic learning framework.To deal with the incomplete information structure,players collect a period of system trajectory data to compensate for the lack of information.The policy updating step is implemented by a nonlinear optimization problem aiming to search for the proximal admissible policy.Theoretical analysis shows that by adopting proximal policy searching rules,the approximated policies can converge to a neighborhood of equilibrium policies.The efficacy of our method is illustrated by three examples,which also demonstrate that the proposed method can accelerate the learning process compared with the centralized learning framework.展开更多
Learning-based methods have become mainstream for solving residential energy scheduling problems. In order to improve the learning efficiency of existing methods and increase the utilization of renewable energy, we pr...Learning-based methods have become mainstream for solving residential energy scheduling problems. In order to improve the learning efficiency of existing methods and increase the utilization of renewable energy, we propose the Dyna actiondependent heuristic dynamic programming(Dyna-ADHDP)method, which incorporates the ideas of learning and planning from the Dyna framework in action-dependent heuristic dynamic programming. This method defines a continuous action space for precise control of an energy storage system and allows online optimization of algorithm performance during the real-time operation of the residential energy model. Meanwhile, the target network is introduced during the training process to make the training smoother and more efficient. We conducted experimental comparisons with the benchmark method using simulated and real data to verify its applicability and performance. The results confirm the method's excellent performance and generalization capabilities, as well as its excellence in increasing renewable energy utilization and extending equipment life.展开更多
Gaze estimation,a crucial non-verbal communication cue,has achieved remarkable progress through convolutional neural networks.However,accurate gaze prediction in uncon-strained environments,particularly in extreme hea...Gaze estimation,a crucial non-verbal communication cue,has achieved remarkable progress through convolutional neural networks.However,accurate gaze prediction in uncon-strained environments,particularly in extreme head poses,partial occlusions,and abnormal lighting,remains challenging.Existing models often struggle to effectively focus on discriminative ocular features,leading to suboptimal performance.To address these limitations,this paper proposes dual-branch gaze estimation with Gaussian mixture distribution heatmaps and dynamic adaptive loss function(DMGDL),a novel dual-branch gaze estimation algorithm.By introducing Gaussian mixture distribution heatmaps centered on pupil positions as spatial attention guides,the model is enabled to prioritize ocular regions.Additionally,a dual-branch network architecture is designed to separately extract features for yaw and pitch angles,enhancing flexibility and mitigating cross-angle interference.A dynamic adaptive loss function is further formulated to address discontinuities in angle estimation,improving robustness and convergence stability.Experimental evaluations on three benchmark datasets demonstrate that DMGDL outperforms state-of-the-art methods,achiev-ing a mean angular error of 3.98°on the Max-Planck institute for informatics face gaze(MPI-IFaceGaze)dataset,10.21°on the physically unconstrained gaze estimation in the wild(Gaze360)dataset and 6.14°on the real-time eye gaze estimation in natural environments(RT-Gene)dataset,exhibiting superior generalization and robustness.展开更多
Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and ...Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.展开更多
In this paper,a novel adaptive Fault-Tolerant Control(FTC)strategy is proposed for non-minimum phase Hypersonic Vehicles(HSVs)that are affected by actuator faults and parameter uncertainties.The strategy is based on t...In this paper,a novel adaptive Fault-Tolerant Control(FTC)strategy is proposed for non-minimum phase Hypersonic Vehicles(HSVs)that are affected by actuator faults and parameter uncertainties.The strategy is based on the output redefinition method and Adaptive Dynamic Programming(ADP).The intelligent FTC scheme consists of two main parts:a basic fault-tolerant and stable controller and an ADP-based supplementary controller.In the basic FTC part,an output redefinition approach is designed to make zero-dynamics stable with respect to the new output.Then,Ideal Internal Dynamic(IID)is obtained using an optimal bounded inversion approach,and a tracking controller is designed for the new output to realize output tracking of the nonminimum phase HSV system.For the ADP-based compensation control part,an ActionDependent Heuristic Dynamic Programming(ADHDP)adopting an actor-critic learning structure is utilized to further optimize the tracking performance of the HSV control system.Finally,simulation results are provided to verify the effectiveness and efficiency of the proposed FTC algorithm.展开更多
In order to address the output feedback issue for linear discrete-time systems, this work suggests a brand-new adaptive dynamic programming(ADP) technique based on the internal model principle(IMP). The proposed metho...In order to address the output feedback issue for linear discrete-time systems, this work suggests a brand-new adaptive dynamic programming(ADP) technique based on the internal model principle(IMP). The proposed method, termed as IMP-ADP, does not require complete state feedback-merely the measurement of input and output data. More specifically, based on the IMP, the output control problem can first be converted into a stabilization problem. We then design an observer to reproduce the full state of the system by measuring the inputs and outputs. Moreover, this technique includes both a policy iteration algorithm and a value iteration algorithm to determine the optimal feedback gain without using a dynamic system model. It is important that with this concept one does not need to solve the regulator equation. Finally, this control method was tested on an inverter system of grid-connected LCLs to demonstrate that the proposed method provides the desired performance in terms of both tracking and disturbance rejection.展开更多
The helicopter Trailing-Edge Flaps(TEFs)technology is one of the recent hot topics in morphing wing research.By employing controlled deflection,TEFs can effectively reduce the vibration level of helicopters.Thus,desig...The helicopter Trailing-Edge Flaps(TEFs)technology is one of the recent hot topics in morphing wing research.By employing controlled deflection,TEFs can effectively reduce the vibration level of helicopters.Thus,designing specific vibration reduction control methods for the helicopters equipped with trailing-edge flaps is of significant practical value.This paper studies the optimal control problem for helicopter-vibration systems with TEFs under the framework of adaptive dynamic programming combined with Reinforcement Learning(RL).Time-delay and disturbances,caused by complexity of helicopter dynamics,inevitably deteriorate the control performance of vibration reduction.To solve this problem,a zero-sum game formulation with a linear quadratic form for reducing vibration of helicopter systems is presented with a virtual predictor.In this context,an off-policy reinforcement learning algorithm is developed to determine the optimal control policy.The algorithm utilizes only vertical vibration load data to achieve a policy that reduces vibration,attains Nash equilibrium,and addresses disturbances while compensating for time-delay without knowledge of the dynamics of the helicopter system.The effectiveness of the proposed method is demonstrated in a virtual platform.展开更多
We have succeeded in 2-slit interference simulation by assuming that a travelling particle interacts with its environment, getting information on the environmental condition according to the adaptive dynamics by Ohya,...We have succeeded in 2-slit interference simulation by assuming that a travelling particle interacts with its environment, getting information on the environmental condition according to the adaptive dynamics by Ohya, thus proposed the possibility that the entanglement comes from the interaction with the environment (Ando et al., 2023). This concept means that there should be no isolated or inertial system other than our unique universe space. Taking this message into account and assuming that the signal velocity is constant against our unique universe space, we reconsidered the inertial system and relativity theory by Galilei and Einstein and found several misunderstandings and errors. Time delay and Lorentz shrinkage were derived similarly to the prediction by special relativity theory, but Lorentz transformation and 4-dimensional time/space view were not. They must have implicitly and unconsciously assumed that any signals transfer information without giving any influences to any systems different from our adaptive dynamical view. We propose that their relativity theories should be reinterpreted in view of adaptive dynamics.展开更多
We applied adaptive dynamics to double slit interference phenomenon using particle model and obtained partial successful results in our previous report. The patterns qualitatively corresponded well with experiments. S...We applied adaptive dynamics to double slit interference phenomenon using particle model and obtained partial successful results in our previous report. The patterns qualitatively corresponded well with experiments. Several properties such as concave single slit pattern and large influence of slight displacement of the emission position were different from the experimental results. In this study we tried other slit conditions and obtained consistent patterns with experiments. We do not claim that the adaptive dynamics is the principle of quantum mechanics, but the present results support the probability of adaptive dynamics as the candidate of the basis of quantum mechanics. We discuss the advantages of the adaptive dynamical view for foundations of quantum mechanics.展开更多
This paper studies motor joint control of a 4-degree-of-freedom(DoF)robotic manipulator using learning-based Adaptive Dynamic Programming(ADP)approach.The manipulator’s dynamics are modelled as an open-loop 4-link se...This paper studies motor joint control of a 4-degree-of-freedom(DoF)robotic manipulator using learning-based Adaptive Dynamic Programming(ADP)approach.The manipulator’s dynamics are modelled as an open-loop 4-link serial kinematic chain with 4 Degrees of Freedom(DoF).Decentralised optimal controllers are designed for each link using ADP approach based on a set of cost matrices and data collected from exploration trajectories.The proposed control strategy employs an off-line,off-policy iterative approach to derive four optimal control policies,one for each joint,under exploration strategies.The objective of the controller is to control the position of each joint.Simulation and experimental results show that four independent optimal controllers are found,each under similar exploration strategies,and the proposed ADP approach successfully yields optimal linear control policies despite the presence of these complexities.The experimental results conducted on the Quanser Qarm robotic platform demonstrate the effectiveness of the proposed ADP controllers in handling significant dynamic nonlinearities,such as actuation limitations,output saturation,and filter delays.展开更多
Normally large amounts of particles are required to accurately simulate the metal cutting process,which consumes a lot of computing time and storage.Adaptive techniques can help decrease the number of particles,hence ...Normally large amounts of particles are required to accurately simulate the metal cutting process,which consumes a lot of computing time and storage.Adaptive techniques can help decrease the number of particles,hence reducing the runtime.This paper presents a novel adaptive smoothed particle hydrodynamics(SPH)method for the metal cutting simulation.The spatial resolution changes adaptively according to the distance to the tool tip by the particle splitting and merging.More particles are selected in the region where the workpiece and the tool are in contact.Since the contact region constantly changes during the cutting process,two quadrilateral frames are adopted in the adaptive algorithm to dynamically change the distribution of particles.One frame for the refinement,the other for the coarsening.These frames move at the same speed as the tool.To test the computational efficiency,the metal cutting process is simulated by using SPH with three different adaptive approaches.Numerical results show that the proposed adaptive algorithm with dynamic refinement and coarsening can significantly optimize the runtime.展开更多
Nowadays,video streaming counts for the major part of network traffic over the Internet.However,on account of the host-to-host mechanism of the traditional IP network,video distribution over IP-based Internet encounte...Nowadays,video streaming counts for the major part of network traffic over the Internet.However,on account of the host-to-host mechanism of the traditional IP network,video distribution over IP-based Internet encounters bottlenecks.Fortunately,a new proposed future Internet architecture,named data networking(NDN)can improve the performance of video distribution by its features such as in-network storage,multi-path forwarding,etc.In this paper,we design an adaptive bitrate algorithm based on Lyapunov optimization theory over NDN to optimize the long-term quality-of-experience(QoE)of video distribution while ensuring the stability of the whole system.When the network condition is abundant and stable,the problem can be simplified by approximating to a fixed-slot queuing model,but the theoretical performance will degrade when the network status is poor and fluctuate fiercely.Therefore,we divide the problem into two models of fixed time slot and non-fixed time slot and design two Lyapunov optimization algorithms to adapt different network scenarios.The proposed algorithms do not require prior knowledge of the network bandwidth and are capable of running online with the client’s available information.Simulation and realistic experiment results demonstrate that our algorithms perform better than others in NDN.展开更多
A dynamics-based adaptive control approach is proposed for a planar dual-arm space robot in the presence of closed-loop constraints and uncertain inertial parameters of the payload. The controller is capable of contro...A dynamics-based adaptive control approach is proposed for a planar dual-arm space robot in the presence of closed-loop constraints and uncertain inertial parameters of the payload. The controller is capable of controlling the po- sition and attitude of both the satellite base and the payload grasped by the manipulator end effectors. The equations of motion in reduced-order form for the constrained system are derived by incorporating the constraint equations in terms of accelerations into Kane＇s equations of the unconstrained system. Model analysis shows that the resulting equations perfectly meet the requirement of adaptive controller design. Consequently, by using an indirect approach, an adaptive control scheme is proposed to accomplish position/attitude trajectory tracking control with the uncertain parameters be- ing estimated on-line. The actuator redundancy due to the closed-loop constraints is utilized to minimize a weighted norm of the joint torques. Global asymptotic stability is proven by using Lyapunov＇s method, and simulation results are also presented to demonstrate the effectiveness of the proposed approach.展开更多
This paper studies the problem of optimal parallel tracking control for continuous-time general nonlinear systems.Unlike existing optimal state feedback control,the control input of the optimal parallel control is int...This paper studies the problem of optimal parallel tracking control for continuous-time general nonlinear systems.Unlike existing optimal state feedback control,the control input of the optimal parallel control is introduced into the feedback system.However,due to the introduction of control input into the feedback system,the optimal state feedback control methods can not be applied directly.To address this problem,an augmented system and an augmented performance index function are proposed firstly.Thus,the general nonlinear system is transformed into an affine nonlinear system.The difference between the optimal parallel control and the optimal state feedback control is analyzed theoretically.It is proven that the optimal parallel control with the augmented performance index function can be seen as the suboptimal state feedback control with the traditional performance index function.Moreover,an adaptive dynamic programming(ADP)technique is utilized to implement the optimal parallel tracking control using a critic neural network(NN)to approximate the value function online.The stability analysis of the closed-loop system is performed using the Lyapunov theory,and the tracking error and NN weights errors are uniformly ultimately bounded(UUB).Also,the optimal parallel controller guarantees the continuity of the control input under the circumstance that there are finite jump discontinuities in the reference signals.Finally,the effectiveness of the developed optimal parallel control method is verified in two cases.展开更多
The residential energy scheduling of solar energy is an important research area of smart grid. On the demand side, factors such as household loads, storage batteries, the outside public utility grid and renewable ener...The residential energy scheduling of solar energy is an important research area of smart grid. On the demand side, factors such as household loads, storage batteries, the outside public utility grid and renewable energy resources, are combined together as a nonlinear, time-varying, indefinite and complex system, which is difficult to manage or optimize. Many nations have already applied the residential real-time pricing to balance the burden on their grid. In order to enhance electricity efficiency of the residential micro grid, this paper presents an action dependent heuristic dynamic programming(ADHDP) method to solve the residential energy scheduling problem. The highlights of this paper are listed below. First,the weather-type classification is adopted to establish three types of programming models based on the features of the solar energy. In addition, the priorities of different energy resources are set to reduce the loss of electrical energy transmissions.Second, three ADHDP-based neural networks, which can update themselves during applications, are designed to manage the flows of electricity. Third, simulation results show that the proposed scheduling method has effectively reduced the total electricity cost and improved load balancing process. The comparison with the particle swarm optimization algorithm further proves that the present method has a promising effect on energy management to save cost.展开更多
In this paper,the multi-missile cooperative guidance system is formulated as a general nonlinear multi-agent system.To save the limited communication resources,an adaptive eventtriggered optimal guidance law is propos...In this paper,the multi-missile cooperative guidance system is formulated as a general nonlinear multi-agent system.To save the limited communication resources,an adaptive eventtriggered optimal guidance law is proposed by designing a synchronization-error-driven triggering condition,which brings together the consensus control with Adaptive Dynamic Programming(ADP)technique.Then,the developed event-triggered distributed control law can be employed by finding an approximate solution of event-triggered coupled Hamilton-Jacobi-Bellman(HJB)equation.To address this issue,the critic network architecture is constructed,in which an adaptive weight updating law is designed for estimating the cooperative optimal cost function online.Therefore,the event-triggered closed-loop system is decomposed into two subsystems:the system with flow dynamics and the system with jump dynamics.By using Lyapunov method,the stability of this closed-loop system is guaranteed and all signals are ensured to be Uniformly Ultimately Bounded(UUB).Furthermore,the Zeno behavior is avoided.Simulation results are finally provided to demonstrate the effectiveness of the proposed method.展开更多
The impact dynamics, impact effect, and post-impact unstable motion sup- pression of free-floating space manipulator capturing a satellite on orbit are analyzed. Firstly, the dynamics equation of free-floating space m...The impact dynamics, impact effect, and post-impact unstable motion sup- pression of free-floating space manipulator capturing a satellite on orbit are analyzed. Firstly, the dynamics equation of free-floating space manipulator is derived using the sec- ond Lagrangian equation. Combining the momentum conservation principle, the impact dynamics and effect between the space manipulator end-effector and satellite of the cap- ture process are analyzed with the momentum impulse method. Focusing on the unstable motion of space manipulator due to the above impact effect, a robust adaptive compound control algorithm is designed to suppress the above unstable motion. There is no need to control the free-floating base position to save the jet fuel. Finally, the simulation is proposed to show the impact effect and verify the validity of the control algorithm.展开更多
A policy iteration algorithm of adaptive dynamic programming（ADP） is developed to solve the optimal tracking control for a class of discrete-time chaotic systems. By system transformations, the optimal tracking prob...A policy iteration algorithm of adaptive dynamic programming（ADP） is developed to solve the optimal tracking control for a class of discrete-time chaotic systems. By system transformations, the optimal tracking problem is transformed into an optimal regulation one. The policy iteration algorithm for discrete-time chaotic systems is first described. Then,the convergence and admissibility properties of the developed policy iteration algorithm are presented, which show that the transformed chaotic system can be stabilized under an arbitrary iterative control law and the iterative performance index function simultaneously converges to the optimum. By implementing the policy iteration algorithm via neural networks,the developed optimal tracking control scheme for chaotic systems is verified by a simulation.展开更多
文摘Reliable and efficient communication is essential for Unmanned Aerial Vehicle(UAV)networks,especially in dynamic and resource-constrained environments such as disaster management,surveillance,and environmental monitoring.Frequent topology changes,high mobility,and limited energy availability pose significant challenges to maintaining stable and high-performance routing.Traditional routing protocols,such as Ad hoc On-Demand Distance Vector(AODV),Load-Balanced Optimized Predictive Ad hoc Routing(LB-OPAR),and Destination-Sequenced Distance Vector(DSDV),often experience performance degradation under such conditions.To address these limitations,this study evaluates the effectiveness of Dynamic Adaptive Routing(DAR),a protocol designed to adapt routing decisions in real time based on network dynamics and resource constraints.The research utilizes the Network Simulator 3(NS-3)platform to conduct controlled simulations,measuring key performance indicators such as latency,Packet Delivery Ratio(PDR),energy consumption,and throughput.Comparative analysis reveals that DAR consistently outperforms conventional protocols,achieving a 20%-30% reduction in latency,a 25% decrease in energy consumption,and marked improvements in throughput and PDR.These results highlight DAR’s ability to maintain high communication reliability while optimizing resource usage in challenging operational scenarios.By providing empirical evidence of DAR’s advantages in highly dynamic UAV network environments,this study contributes to advancing adaptive routing strategies.The findings not only validate DAR’s robustness and scalability but also lay the groundwork for integrating artificial intelligence-driven decision-making and real-world UAV deployment.Future work will explore cross-layer optimization,multi-UAV coordination,and experimental validation in field trials,aiming to further enhance communication resilience and energy efficiency in next-generation aerial networks.
基金the Natural Science Foundation of China(No.22301131)the Natural Science Foundation of Jiangsu Province(Nos.BK20220781,BK20240679)the National Key Research and Development Program of China(No.2024YFB3815700)are greatly acknowledged.
文摘Dynamic adaptability is a key feature in biological macromolecules,enabling selective binding and catalysis[1].From DNA supercoiling to enzyme conformational changes,biological systems have evolved intricate ways to dynamically adjust their structures to accommodate functional needs.Mimicking this adaptability in synthetic systems is an ongoing challenge in supramolecular chemistry.
基金supported by the Aeronautical Science Foundation of China(20220001057001)an Open Project of the National Key Laboratory of Air-based Information Perception and Fusion(202437)
文摘In this paper,a distributed adaptive dynamic programming(ADP)framework based on value iteration is proposed for multi-player differential games.In the game setting,players have no access to the information of others'system parameters or control laws.Each player adopts an on-policy value iteration algorithm as the basic learning framework.To deal with the incomplete information structure,players collect a period of system trajectory data to compensate for the lack of information.The policy updating step is implemented by a nonlinear optimization problem aiming to search for the proximal admissible policy.Theoretical analysis shows that by adopting proximal policy searching rules,the approximated policies can converge to a neighborhood of equilibrium policies.The efficacy of our method is illustrated by three examples,which also demonstrate that the proposed method can accelerate the learning process compared with the centralized learning framework.
基金supported in part by the National Key Research and Development Program of China(2024YFB4709100,2021YFE0206100)the National Natural Science Foundation of China(62073321)+1 种基金the National Defense Basic Scientific Research Program(JCKY2019203C029)the Science and Technology Development Fund,Macao SAR,China(0015/2020/AMJ)
文摘Learning-based methods have become mainstream for solving residential energy scheduling problems. In order to improve the learning efficiency of existing methods and increase the utilization of renewable energy, we propose the Dyna actiondependent heuristic dynamic programming(Dyna-ADHDP)method, which incorporates the ideas of learning and planning from the Dyna framework in action-dependent heuristic dynamic programming. This method defines a continuous action space for precise control of an energy storage system and allows online optimization of algorithm performance during the real-time operation of the residential energy model. Meanwhile, the target network is introduced during the training process to make the training smoother and more efficient. We conducted experimental comparisons with the benchmark method using simulated and real data to verify its applicability and performance. The results confirm the method's excellent performance and generalization capabilities, as well as its excellence in increasing renewable energy utilization and extending equipment life.
基金supported by the Key Project of the NationalLanguage Commission(No.ZDI145-110)the AcademicResearch Projects of Beijing Union University(No.ZK20202514)+1 种基金the Key Laboratory Project(No.YYZN-2024-6)the Project for the Construction and Support of High-Level Innovative Teams in Beijing Municipal Institutions(No.BPHR20220121).
文摘Gaze estimation,a crucial non-verbal communication cue,has achieved remarkable progress through convolutional neural networks.However,accurate gaze prediction in uncon-strained environments,particularly in extreme head poses,partial occlusions,and abnormal lighting,remains challenging.Existing models often struggle to effectively focus on discriminative ocular features,leading to suboptimal performance.To address these limitations,this paper proposes dual-branch gaze estimation with Gaussian mixture distribution heatmaps and dynamic adaptive loss function(DMGDL),a novel dual-branch gaze estimation algorithm.By introducing Gaussian mixture distribution heatmaps centered on pupil positions as spatial attention guides,the model is enabled to prioritize ocular regions.Additionally,a dual-branch network architecture is designed to separately extract features for yaw and pitch angles,enhancing flexibility and mitigating cross-angle interference.A dynamic adaptive loss function is further formulated to address discontinuities in angle estimation,improving robustness and convergence stability.Experimental evaluations on three benchmark datasets demonstrate that DMGDL outperforms state-of-the-art methods,achiev-ing a mean angular error of 3.98°on the Max-Planck institute for informatics face gaze(MPI-IFaceGaze)dataset,10.21°on the physically unconstrained gaze estimation in the wild(Gaze360)dataset and 6.14°on the real-time eye gaze estimation in natural environments(RT-Gene)dataset,exhibiting superior generalization and robustness.
基金supported in part by the National Natural Science Foundation of China(62222301, 62073085, 62073158, 61890930-5, 62021003)the National Key Research and Development Program of China (2021ZD0112302, 2021ZD0112301, 2018YFC1900800-5)Beijing Natural Science Foundation (JQ19013)。
文摘Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.
基金supported in part by the Science Center Program of National Natural Science Foundation of China(62373189,62188101,62020106003)the Research Fund of State Key Laboratory of Mechanics and Control for Aerospace Structures,China。
文摘In this paper,a novel adaptive Fault-Tolerant Control(FTC)strategy is proposed for non-minimum phase Hypersonic Vehicles(HSVs)that are affected by actuator faults and parameter uncertainties.The strategy is based on the output redefinition method and Adaptive Dynamic Programming(ADP).The intelligent FTC scheme consists of two main parts:a basic fault-tolerant and stable controller and an ADP-based supplementary controller.In the basic FTC part,an output redefinition approach is designed to make zero-dynamics stable with respect to the new output.Then,Ideal Internal Dynamic(IID)is obtained using an optimal bounded inversion approach,and a tracking controller is designed for the new output to realize output tracking of the nonminimum phase HSV system.For the ADP-based compensation control part,an ActionDependent Heuristic Dynamic Programming(ADHDP)adopting an actor-critic learning structure is utilized to further optimize the tracking performance of the HSV control system.Finally,simulation results are provided to verify the effectiveness and efficiency of the proposed FTC algorithm.
基金supported by the National Science Fund for Distinguished Young Scholars (62225303)the Fundamental Research Funds for the Central Universities (buctrc202201)+1 种基金China Scholarship Council,and High Performance Computing PlatformCollege of Information Science and Technology,Beijing University of Chemical Technology。
文摘In order to address the output feedback issue for linear discrete-time systems, this work suggests a brand-new adaptive dynamic programming(ADP) technique based on the internal model principle(IMP). The proposed method, termed as IMP-ADP, does not require complete state feedback-merely the measurement of input and output data. More specifically, based on the IMP, the output control problem can first be converted into a stabilization problem. We then design an observer to reproduce the full state of the system by measuring the inputs and outputs. Moreover, this technique includes both a policy iteration algorithm and a value iteration algorithm to determine the optimal feedback gain without using a dynamic system model. It is important that with this concept one does not need to solve the regulator equation. Finally, this control method was tested on an inverter system of grid-connected LCLs to demonstrate that the proposed method provides the desired performance in terms of both tracking and disturbance rejection.
基金co-supported by the National Natural Science Foundation of China(Nos.62022060,62073234,62073158,62373268,62373273)the Basic Research Project of Education Department of Liaoning Province,China(No.LJKZ0401).
文摘The helicopter Trailing-Edge Flaps(TEFs)technology is one of the recent hot topics in morphing wing research.By employing controlled deflection,TEFs can effectively reduce the vibration level of helicopters.Thus,designing specific vibration reduction control methods for the helicopters equipped with trailing-edge flaps is of significant practical value.This paper studies the optimal control problem for helicopter-vibration systems with TEFs under the framework of adaptive dynamic programming combined with Reinforcement Learning(RL).Time-delay and disturbances,caused by complexity of helicopter dynamics,inevitably deteriorate the control performance of vibration reduction.To solve this problem,a zero-sum game formulation with a linear quadratic form for reducing vibration of helicopter systems is presented with a virtual predictor.In this context,an off-policy reinforcement learning algorithm is developed to determine the optimal control policy.The algorithm utilizes only vertical vibration load data to achieve a policy that reduces vibration,attains Nash equilibrium,and addresses disturbances while compensating for time-delay without knowledge of the dynamics of the helicopter system.The effectiveness of the proposed method is demonstrated in a virtual platform.
文摘We have succeeded in 2-slit interference simulation by assuming that a travelling particle interacts with its environment, getting information on the environmental condition according to the adaptive dynamics by Ohya, thus proposed the possibility that the entanglement comes from the interaction with the environment (Ando et al., 2023). This concept means that there should be no isolated or inertial system other than our unique universe space. Taking this message into account and assuming that the signal velocity is constant against our unique universe space, we reconsidered the inertial system and relativity theory by Galilei and Einstein and found several misunderstandings and errors. Time delay and Lorentz shrinkage were derived similarly to the prediction by special relativity theory, but Lorentz transformation and 4-dimensional time/space view were not. They must have implicitly and unconsciously assumed that any signals transfer information without giving any influences to any systems different from our adaptive dynamical view. We propose that their relativity theories should be reinterpreted in view of adaptive dynamics.
文摘We applied adaptive dynamics to double slit interference phenomenon using particle model and obtained partial successful results in our previous report. The patterns qualitatively corresponded well with experiments. Several properties such as concave single slit pattern and large influence of slight displacement of the emission position were different from the experimental results. In this study we tried other slit conditions and obtained consistent patterns with experiments. We do not claim that the adaptive dynamics is the principle of quantum mechanics, but the present results support the probability of adaptive dynamics as the candidate of the basis of quantum mechanics. We discuss the advantages of the adaptive dynamical view for foundations of quantum mechanics.
基金supported by the DEEPCOBOT project under Grant 306640/O70 funded by the Research Council of Norway.
文摘This paper studies motor joint control of a 4-degree-of-freedom(DoF)robotic manipulator using learning-based Adaptive Dynamic Programming(ADP)approach.The manipulator’s dynamics are modelled as an open-loop 4-link serial kinematic chain with 4 Degrees of Freedom(DoF).Decentralised optimal controllers are designed for each link using ADP approach based on a set of cost matrices and data collected from exploration trajectories.The proposed control strategy employs an off-line,off-policy iterative approach to derive four optimal control policies,one for each joint,under exploration strategies.The objective of the controller is to control the position of each joint.Simulation and experimental results show that four independent optimal controllers are found,each under similar exploration strategies,and the proposed ADP approach successfully yields optimal linear control policies despite the presence of these complexities.The experimental results conducted on the Quanser Qarm robotic platform demonstrate the effectiveness of the proposed ADP controllers in handling significant dynamic nonlinearities,such as actuation limitations,output saturation,and filter delays.
基金the National Natural Science Foundation of China(Grant Nos.12002290 and 11772274).
文摘Normally large amounts of particles are required to accurately simulate the metal cutting process,which consumes a lot of computing time and storage.Adaptive techniques can help decrease the number of particles,hence reducing the runtime.This paper presents a novel adaptive smoothed particle hydrodynamics(SPH)method for the metal cutting simulation.The spatial resolution changes adaptively according to the distance to the tool tip by the particle splitting and merging.More particles are selected in the region where the workpiece and the tool are in contact.Since the contact region constantly changes during the cutting process,two quadrilateral frames are adopted in the adaptive algorithm to dynamically change the distribution of particles.One frame for the refinement,the other for the coarsening.These frames move at the same speed as the tool.To test the computational efficiency,the metal cutting process is simulated by using SPH with three different adaptive approaches.Numerical results show that the proposed adaptive algorithm with dynamic refinement and coarsening can significantly optimize the runtime.
基金supported by the National Key R&D Program of China under Grant 2020YFA0711400the National Science Foundation of China under Grant 61673360the CETC Joint Advanced Research Foundation under Grant 6141B08080101.
文摘Nowadays,video streaming counts for the major part of network traffic over the Internet.However,on account of the host-to-host mechanism of the traditional IP network,video distribution over IP-based Internet encounters bottlenecks.Fortunately,a new proposed future Internet architecture,named data networking(NDN)can improve the performance of video distribution by its features such as in-network storage,multi-path forwarding,etc.In this paper,we design an adaptive bitrate algorithm based on Lyapunov optimization theory over NDN to optimize the long-term quality-of-experience(QoE)of video distribution while ensuring the stability of the whole system.When the network condition is abundant and stable,the problem can be simplified by approximating to a fixed-slot queuing model,but the theoretical performance will degrade when the network status is poor and fluctuate fiercely.Therefore,we divide the problem into two models of fixed time slot and non-fixed time slot and design two Lyapunov optimization algorithms to adapt different network scenarios.The proposed algorithms do not require prior knowledge of the network bandwidth and are capable of running online with the client’s available information.Simulation and realistic experiment results demonstrate that our algorithms perform better than others in NDN.
基金supported by the National Natural Science Foundation of China(11272027)
文摘A dynamics-based adaptive control approach is proposed for a planar dual-arm space robot in the presence of closed-loop constraints and uncertain inertial parameters of the payload. The controller is capable of controlling the po- sition and attitude of both the satellite base and the payload grasped by the manipulator end effectors. The equations of motion in reduced-order form for the constrained system are derived by incorporating the constraint equations in terms of accelerations into Kane＇s equations of the unconstrained system. Model analysis shows that the resulting equations perfectly meet the requirement of adaptive controller design. Consequently, by using an indirect approach, an adaptive control scheme is proposed to accomplish position/attitude trajectory tracking control with the uncertain parameters be- ing estimated on-line. The actuator redundancy due to the closed-loop constraints is utilized to minimize a weighted norm of the joint torques. Global asymptotic stability is proven by using Lyapunov＇s method, and simulation results are also presented to demonstrate the effectiveness of the proposed approach.
基金supported in part by the National Key Reseanch and Development Program of China(2018AAA0101502,2018YFB1702300)in part by the National Natural Science Foundation of China(61722312,61533019,U1811463,61533017)in part by the Intel Collaborative Research Institute for Intelligent and Automated Connected Vehicles。
文摘This paper studies the problem of optimal parallel tracking control for continuous-time general nonlinear systems.Unlike existing optimal state feedback control,the control input of the optimal parallel control is introduced into the feedback system.However,due to the introduction of control input into the feedback system,the optimal state feedback control methods can not be applied directly.To address this problem,an augmented system and an augmented performance index function are proposed firstly.Thus,the general nonlinear system is transformed into an affine nonlinear system.The difference between the optimal parallel control and the optimal state feedback control is analyzed theoretically.It is proven that the optimal parallel control with the augmented performance index function can be seen as the suboptimal state feedback control with the traditional performance index function.Moreover,an adaptive dynamic programming(ADP)technique is utilized to implement the optimal parallel tracking control using a critic neural network(NN)to approximate the value function online.The stability analysis of the closed-loop system is performed using the Lyapunov theory,and the tracking error and NN weights errors are uniformly ultimately bounded(UUB).Also,the optimal parallel controller guarantees the continuity of the control input under the circumstance that there are finite jump discontinuities in the reference signals.Finally,the effectiveness of the developed optimal parallel control method is verified in two cases.
基金supported in part by the National Natural Science Foundation of China(61533017,U1501251,61374105,61722312)
文摘The residential energy scheduling of solar energy is an important research area of smart grid. On the demand side, factors such as household loads, storage batteries, the outside public utility grid and renewable energy resources, are combined together as a nonlinear, time-varying, indefinite and complex system, which is difficult to manage or optimize. Many nations have already applied the residential real-time pricing to balance the burden on their grid. In order to enhance electricity efficiency of the residential micro grid, this paper presents an action dependent heuristic dynamic programming(ADHDP) method to solve the residential energy scheduling problem. The highlights of this paper are listed below. First,the weather-type classification is adopted to establish three types of programming models based on the features of the solar energy. In addition, the priorities of different energy resources are set to reduce the loss of electrical energy transmissions.Second, three ADHDP-based neural networks, which can update themselves during applications, are designed to manage the flows of electricity. Third, simulation results show that the proposed scheduling method has effectively reduced the total electricity cost and improved load balancing process. The comparison with the particle swarm optimization algorithm further proves that the present method has a promising effect on energy management to save cost.
基金co-supported by the National Natural Science Foundation of China(No.62003036)China Postdoctoral Science Foundation(No.2019TQ0037)。
文摘In this paper,the multi-missile cooperative guidance system is formulated as a general nonlinear multi-agent system.To save the limited communication resources,an adaptive eventtriggered optimal guidance law is proposed by designing a synchronization-error-driven triggering condition,which brings together the consensus control with Adaptive Dynamic Programming(ADP)technique.Then,the developed event-triggered distributed control law can be employed by finding an approximate solution of event-triggered coupled Hamilton-Jacobi-Bellman(HJB)equation.To address this issue,the critic network architecture is constructed,in which an adaptive weight updating law is designed for estimating the cooperative optimal cost function online.Therefore,the event-triggered closed-loop system is decomposed into two subsystems:the system with flow dynamics and the system with jump dynamics.By using Lyapunov method,the stability of this closed-loop system is guaranteed and all signals are ensured to be Uniformly Ultimately Bounded(UUB).Furthermore,the Zeno behavior is avoided.Simulation results are finally provided to demonstrate the effectiveness of the proposed method.
基金supported by the National Natural Science Foundation of China(Nos.11072061 and 11372073)the Natural Science Foundation of Fujian Province(No.2010J01003)
文摘The impact dynamics, impact effect, and post-impact unstable motion sup- pression of free-floating space manipulator capturing a satellite on orbit are analyzed. Firstly, the dynamics equation of free-floating space manipulator is derived using the sec- ond Lagrangian equation. Combining the momentum conservation principle, the impact dynamics and effect between the space manipulator end-effector and satellite of the cap- ture process are analyzed with the momentum impulse method. Focusing on the unstable motion of space manipulator due to the above impact effect, a robust adaptive compound control algorithm is designed to suppress the above unstable motion. There is no need to control the free-floating base position to save the jet fuel. Finally, the simulation is proposed to show the impact effect and verify the validity of the control algorithm.
基金supported by the National Natural Science Foundation of China(Grant Nos.61034002,61233001,61273140,61304086,and 61374105)the Beijing Natural Science Foundation,China(Grant No.4132078)
文摘A policy iteration algorithm of adaptive dynamic programming（ADP） is developed to solve the optimal tracking control for a class of discrete-time chaotic systems. By system transformations, the optimal tracking problem is transformed into an optimal regulation one. The policy iteration algorithm for discrete-time chaotic systems is first described. Then,the convergence and admissibility properties of the developed policy iteration algorithm are presented, which show that the transformed chaotic system can be stabilized under an arbitrary iterative control law and the iterative performance index function simultaneously converges to the optimum. By implementing the policy iteration algorithm via neural networks,the developed optimal tracking control scheme for chaotic systems is verified by a simulation.
