It is common for robotic fish to generate thrust using reactive force generated by the tail’s physical motion, which interacts with the surrounding fluid. The coupling effect of the body strongly correlates with this...It is common for robotic fish to generate thrust using reactive force generated by the tail’s physical motion, which interacts with the surrounding fluid. The coupling effect of the body strongly correlates with this thrust. However, hydrodynamics cannot be wholly modeled in analytical form. Therefore, data-assisted modeling is necessary for robotic fish. This work presents the first method of its kind using Genetic Algorithm (GA)-based optimization methods for data-assistive modeling for robotic fish applications. To begin, experimental data are collected in real time with the robotic fish that has been designed and fabricated using 3D printing. Then, the model’s influential parameters are estimated using an optimization problem. Further, a model-based deep reinforcement learning (DRL) controller is proposed to track the desired speed through extensive simulation work. In addition to a deep deterministic policy gradient (DDPG), a twin delayed DDPG (TD3) is employed in the training of the RL agent. Unfortunately, due to its local optimization problem, the RL-DDPG controller failed to perform well during training. In contrast, the RL-TD3 controller effectively learns the control policies and overcomes the local optima problem. As a final step, controller performance is evaluated under different disturbance conditions. In contrast to DDPG and GA-tuned proportional-integral controllers, the proposed model with RL-TD3 controller significantly improves the performance.展开更多
The integration of artificial intelligence into the development and production of mechatronic products offers a substantial opportunity to enhance efficiency, adaptability, and system performance. This paper examines ...The integration of artificial intelligence into the development and production of mechatronic products offers a substantial opportunity to enhance efficiency, adaptability, and system performance. This paper examines the utilization of reinforcement learning as a control strategy, with a particular focus on its deployment in pivotal stages of the product development lifecycle, specifically between system architecture and system integration and verification. A controller based on reinforcement learning was developed and evaluated in comparison to traditional proportional-integral controllers in dynamic and fault-prone environments. The results illustrate the superior adaptability, stability, and optimization potential of the reinforcement learning approach, particularly in addressing dynamic disturbances and ensuring robust performance. The study illustrates how reinforcement learning can facilitate the transition from conceptual design to implementation by automating optimization processes, enabling interface automation, and enhancing system-level testing. Based on the aforementioned findings, this paper presents future directions for research, which include the integration of domain-specific knowledge into the reinforcement learning process and the validation of this process in real-world environments. The results underscore the potential of artificial intelligence-driven methodologies to revolutionize the design and deployment of intelligent mechatronic systems.展开更多
We address a state-of-the-art reinforcement learning(RL)control approach to automatically configure robotic pros-thesis impedance parameters to enable end-to-end,continuous locomotion intended for transfemoral amputee...We address a state-of-the-art reinforcement learning(RL)control approach to automatically configure robotic pros-thesis impedance parameters to enable end-to-end,continuous locomotion intended for transfemoral amputee subjects.Specifically,our actor-critic based RL provides tracking control of a robotic knee prosthesis to mimic the intact knee profile.This is a significant advance from our previous RL based automatic tuning of prosthesis control parameters which have centered on regulation control with a designer prescribed robotic knee profile as the target.In addition to presenting the tracking control algorithm based on direct heuristic dynamic programming(dHDP),we provide a control performance guarantee including the case of constrained inputs.We show that our proposed tracking control possesses several important properties,such as weight convergence of the learning networks,Bellman(sub)optimality of the cost-to-go value function and control input,and practical stability of the human-robot system.We further provide a systematic simulation of the proposed tracking control using a realistic human-robot system simulator,the OpenSim,to emulate how the dHDP enables level ground walking,walking on different terrains and at different paces.These results show that our proposed dHDP based tracking control is not only theoretically suitable,but also practically useful.展开更多
Air conditioning water systems account for a large proportion of building energy consumption.In a pressure-controlled water system,one of the key measures to save energy is to adjust the differential pressure setpoint...Air conditioning water systems account for a large proportion of building energy consumption.In a pressure-controlled water system,one of the key measures to save energy is to adjust the differential pressure setpoints during operation.Typically,such adjustments are based either on certain rules,which rely on operator experience,or on complicated models that are not easy to calibrate.In this paper,a data-driven control method based on reinforcement learning is proposed.The main idea is to construct an agent model that adapts to the researched problem.Instead of directly being told how to react,the agent must rely on its own experiences to learn.Compared with traditional control strategies,reinforcement learning control(RLC)exhibits more accurate and steady performances while maintaining indoor air temperature within a limited range.A case study shows that the RLC strategy is able to save substantial amounts of energy.展开更多
Wheeled-legged robots integrate the mobility efficiency of wheeled platforms with the terrain adaptability of legged robots,making them ideal for complex,unstructured environments.However,balancing high payload capaci...Wheeled-legged robots integrate the mobility efficiency of wheeled platforms with the terrain adaptability of legged robots,making them ideal for complex,unstructured environments.However,balancing high payload capacity with agile multimodal locomotion remains a major challenge.This paper presents a field study conducted in the high-altitude region of Golmud,Qinghai,with elevations ranging from 2800 m to 4000 m.We evaluate three wheeled-legged robot platforms of different scales on diverse terrains including Gobi,desert,grassland,and wetlands.Our experiments demonstrate the robot's robust locomotion performance across multimodal tasks such as obstacle crossing,slope climbing,and terrain classification.Moreover,we validate the performance of autonomous perception systems,including real-time localization and 3D mapping,under harsh plateau conditions.The results provide valuable insights into the deployment of wheeled-legged robots in extreme natural environments and lay a solid foundation for future applications in inspection,rescue,and transport missions in high-altitude regions.展开更多
文摘It is common for robotic fish to generate thrust using reactive force generated by the tail’s physical motion, which interacts with the surrounding fluid. The coupling effect of the body strongly correlates with this thrust. However, hydrodynamics cannot be wholly modeled in analytical form. Therefore, data-assisted modeling is necessary for robotic fish. This work presents the first method of its kind using Genetic Algorithm (GA)-based optimization methods for data-assistive modeling for robotic fish applications. To begin, experimental data are collected in real time with the robotic fish that has been designed and fabricated using 3D printing. Then, the model’s influential parameters are estimated using an optimization problem. Further, a model-based deep reinforcement learning (DRL) controller is proposed to track the desired speed through extensive simulation work. In addition to a deep deterministic policy gradient (DDPG), a twin delayed DDPG (TD3) is employed in the training of the RL agent. Unfortunately, due to its local optimization problem, the RL-DDPG controller failed to perform well during training. In contrast, the RL-TD3 controller effectively learns the control policies and overcomes the local optima problem. As a final step, controller performance is evaluated under different disturbance conditions. In contrast to DDPG and GA-tuned proportional-integral controllers, the proposed model with RL-TD3 controller significantly improves the performance.
文摘The integration of artificial intelligence into the development and production of mechatronic products offers a substantial opportunity to enhance efficiency, adaptability, and system performance. This paper examines the utilization of reinforcement learning as a control strategy, with a particular focus on its deployment in pivotal stages of the product development lifecycle, specifically between system architecture and system integration and verification. A controller based on reinforcement learning was developed and evaluated in comparison to traditional proportional-integral controllers in dynamic and fault-prone environments. The results illustrate the superior adaptability, stability, and optimization potential of the reinforcement learning approach, particularly in addressing dynamic disturbances and ensuring robust performance. The study illustrates how reinforcement learning can facilitate the transition from conceptual design to implementation by automating optimization processes, enabling interface automation, and enhancing system-level testing. Based on the aforementioned findings, this paper presents future directions for research, which include the integration of domain-specific knowledge into the reinforcement learning process and the validation of this process in real-world environments. The results underscore the potential of artificial intelligence-driven methodologies to revolutionize the design and deployment of intelligent mechatronic systems.
基金This work was partly supported by the National Science Foundation(1563921,1808752,1563454,1808898).
文摘We address a state-of-the-art reinforcement learning(RL)control approach to automatically configure robotic pros-thesis impedance parameters to enable end-to-end,continuous locomotion intended for transfemoral amputee subjects.Specifically,our actor-critic based RL provides tracking control of a robotic knee prosthesis to mimic the intact knee profile.This is a significant advance from our previous RL based automatic tuning of prosthesis control parameters which have centered on regulation control with a designer prescribed robotic knee profile as the target.In addition to presenting the tracking control algorithm based on direct heuristic dynamic programming(dHDP),we provide a control performance guarantee including the case of constrained inputs.We show that our proposed tracking control possesses several important properties,such as weight convergence of the learning networks,Bellman(sub)optimality of the cost-to-go value function and control input,and practical stability of the human-robot system.We further provide a systematic simulation of the proposed tracking control using a realistic human-robot system simulator,the OpenSim,to emulate how the dHDP enables level ground walking,walking on different terrains and at different paces.These results show that our proposed dHDP based tracking control is not only theoretically suitable,but also practically useful.
文摘Air conditioning water systems account for a large proportion of building energy consumption.In a pressure-controlled water system,one of the key measures to save energy is to adjust the differential pressure setpoints during operation.Typically,such adjustments are based either on certain rules,which rely on operator experience,or on complicated models that are not easy to calibrate.In this paper,a data-driven control method based on reinforcement learning is proposed.The main idea is to construct an agent model that adapts to the researched problem.Instead of directly being told how to react,the agent must rely on its own experiences to learn.Compared with traditional control strategies,reinforcement learning control(RLC)exhibits more accurate and steady performances while maintaining indoor air temperature within a limited range.A case study shows that the RLC strategy is able to save substantial amounts of energy.
基金supported in part by the National Key R&D Program of china(2022YFB4701500 and 2024YFB4708705)in part by the National Natural Science Foundation of China(52475021,52305024 and 52205012)+2 种基金in part by the Natural Science Foundation of jiangsu Province,China(BK20230928)in part by the China Postdoctoral Science Foundation,China(2023M731690)in part by the Fundamental Research Funds for the Central Universities,China(30923011029).
文摘Wheeled-legged robots integrate the mobility efficiency of wheeled platforms with the terrain adaptability of legged robots,making them ideal for complex,unstructured environments.However,balancing high payload capacity with agile multimodal locomotion remains a major challenge.This paper presents a field study conducted in the high-altitude region of Golmud,Qinghai,with elevations ranging from 2800 m to 4000 m.We evaluate three wheeled-legged robot platforms of different scales on diverse terrains including Gobi,desert,grassland,and wetlands.Our experiments demonstrate the robot's robust locomotion performance across multimodal tasks such as obstacle crossing,slope climbing,and terrain classification.Moreover,we validate the performance of autonomous perception systems,including real-time localization and 3D mapping,under harsh plateau conditions.The results provide valuable insights into the deployment of wheeled-legged robots in extreme natural environments and lay a solid foundation for future applications in inspection,rescue,and transport missions in high-altitude regions.
