Energy efficiency stands as an essential factor when implementing deep reinforcement learning(DRL)policies for robotic control systems.Standard algorithms,including Deep Deterministic Policy Gradient(DDPG),primarily o...Energy efficiency stands as an essential factor when implementing deep reinforcement learning(DRL)policies for robotic control systems.Standard algorithms,including Deep Deterministic Policy Gradient(DDPG),primarily optimize task rewards but at the cost of excessively high energy consumption,making them impractical for real-world robotic systems.To address this limitation,we propose Physics-Informed DDPG(PI-DDPG),which integrates physics-based energy penalties to develop energy-efficient yet high-performing control policies.The proposed method introduces adaptive physics-informed constraints through a dynamic weighting factor(λ),enabling policies that balance reward maximization with energy savings.Our motivation is to overcome the impracticality of rewardonly optimization by designing controllers that achieve competitive performance while substantially reducing energy consumption.PI-DDPG was evaluated in nine MuJoCo continuous control environments,where it demonstrated significant improvements in energy efficiency without compromising stability or performance.Experimental results confirm that PI-DDPG substantially reduces energy consumption compared to standard DDPG,while maintaining competitive task performance.For instance,energy costs decreased from 5542.98 to 3119.02 in HalfCheetah-v4 and from1909.13 to 1586.75 in Ant-v4,with stable performance in Hopper-v4(205.95 vs.130.82)and InvertedPendulum-v4(322.97 vs.311.29).Although DDPG sometimes yields higher rewards,such as in HalfCheetah-v4(5695.37 vs.4894.59),it requires significantly greater energy expenditure.These results highlight PI-DDPG as a promising energy-conscious alternative for robotic control.展开更多
Patients with complete spinal cord injury retain the potential for volitional muscle activity in muscles located below the spinal injury level.However,because of prolonged inactivity,initial attempts to activate these...Patients with complete spinal cord injury retain the potential for volitional muscle activity in muscles located below the spinal injury level.However,because of prolonged inactivity,initial attempts to activate these muscles may not effectively engage any of the remaining neurons in the descending pathway.A previous study unexpectedly found that a brief clinical round of passive activity significantly increased volitional muscle activation,as measured by surface electromyography.In this study,we further explored the effect of passive activity on surface electromyographic signals during volitional control tasks among individuals with complete spinal cord injury.Eleven patients with chronic complete thoracic spinal cord injury were recruited.Surface electromyography data from eight major leg muscles were acquired and compared before and after the passive activity protocol.The results indicated that the passive activity led to an increased number of activated volitional muscles and an increased frequency of activation.Although the cumulative root mean square of surface electromyography amplitude for volitional control of movement showed a slight increase after passive activity,the difference was not statistically significant.These findings suggest that brief passive activity may enhance the ability to initiate volitional muscle activity during surface electromyography tasks and underscore the potential of passive activity for improving residual motor control among patients with motor complete spinal cord injury.展开更多
Chronic stress affects brain function,so assessing its hazards is important for mental health.To overcome the limitations of behavioral data,we combined behavioral and event-related potentials(ERPs)in an attention net...Chronic stress affects brain function,so assessing its hazards is important for mental health.To overcome the limitations of behavioral data,we combined behavioral and event-related potentials(ERPs)in an attention network task.This task allowed us to differentiate between three specific aspects of attention:alerting,orienting,and execution.Forty-one participants under chronic stress and 31 non-stressed participants were enrolled.On the performance level,the chronically stressed group showed a significantly slower task response and lower accuracy.Concerning ERP measures,smaller cue-N1,cueN2,and larger cue-P3 amplitudes were found in the stressed group,indicating that this group was less able to assign attention to effective information,i.e.,they made inefficient use of cues and had difficulty in maintaining alerting.In addition,the stressed group showed larger target-N2 amplitudes,indicating that this group needed to allocate more cognitive resources to deal with the conflict targets task.Subgroup analysis revealed lower target-P3 amplitudes in the stressed than in the non-stressed group.Group differences associated with the attention networks were found at the ERP level.In the stressed group,excessive depletion of resources led to changes in attention control.In this study,we examined the effects of chronic stress on individual executive function from a neurological perspective.The results may benefit the development of interventions to improve executive function in chronically stressed individuals.展开更多
As a well-explored template that captures the essential dynamical behaviors of legged locomotion on sagittal plane,the spring-loaded inverted pendulum(SLIP)model has been extensively employed in both biomechanical stu...As a well-explored template that captures the essential dynamical behaviors of legged locomotion on sagittal plane,the spring-loaded inverted pendulum(SLIP)model has been extensively employed in both biomechanical study and robotics research.Aiming at fully leveraging the merits of the SLIP model to generate the adaptive trajectories of the center of mass(CoM)with maneuverability,this study presents a novel two-layered sagittal SLIP-anchored(SSA)task space control for a monopode robot to deal with terrain irregularity.This work begins with an analytical investigation of sagittal SLIP dynamics by deriving an approximate solution with satisfactory apex prediction accuracy,and a two-layered SSA task space controller is subsequently developed for the monopode robot.The higher layer employs an analytical approximate representation of the sagittal SLIP model to form a deadbeat controller,which generates an adaptive reference trajectory for the CoM.The lower layer enforces the monopode robot to reproduce a generated CoM movement by using a task space controller to transfer the reference CoM commands into joint torques of the multi-degree of freedom monopode robot.Consequently,an adaptive hopping behavior is exhibited by the robot when traversing irregular terrain.Simulation results have demonstrated the effectiveness of the proposed method.展开更多
文摘Energy efficiency stands as an essential factor when implementing deep reinforcement learning(DRL)policies for robotic control systems.Standard algorithms,including Deep Deterministic Policy Gradient(DDPG),primarily optimize task rewards but at the cost of excessively high energy consumption,making them impractical for real-world robotic systems.To address this limitation,we propose Physics-Informed DDPG(PI-DDPG),which integrates physics-based energy penalties to develop energy-efficient yet high-performing control policies.The proposed method introduces adaptive physics-informed constraints through a dynamic weighting factor(λ),enabling policies that balance reward maximization with energy savings.Our motivation is to overcome the impracticality of rewardonly optimization by designing controllers that achieve competitive performance while substantially reducing energy consumption.PI-DDPG was evaluated in nine MuJoCo continuous control environments,where it demonstrated significant improvements in energy efficiency without compromising stability or performance.Experimental results confirm that PI-DDPG substantially reduces energy consumption compared to standard DDPG,while maintaining competitive task performance.For instance,energy costs decreased from 5542.98 to 3119.02 in HalfCheetah-v4 and from1909.13 to 1586.75 in Ant-v4,with stable performance in Hopper-v4(205.95 vs.130.82)and InvertedPendulum-v4(322.97 vs.311.29).Although DDPG sometimes yields higher rewards,such as in HalfCheetah-v4(5695.37 vs.4894.59),it requires significantly greater energy expenditure.These results highlight PI-DDPG as a promising energy-conscious alternative for robotic control.
基金supported by the Fundamental Research Funds for Central Public Welfare Research Institute,No.2020CZ-5(to WS and GS)the National Natural Science Foundation of China,No.31970970(to JSR)Fundamental Research Funds for the Central Universities,No.YWF-23-YG-QB-010(to JSR)。
文摘Patients with complete spinal cord injury retain the potential for volitional muscle activity in muscles located below the spinal injury level.However,because of prolonged inactivity,initial attempts to activate these muscles may not effectively engage any of the remaining neurons in the descending pathway.A previous study unexpectedly found that a brief clinical round of passive activity significantly increased volitional muscle activation,as measured by surface electromyography.In this study,we further explored the effect of passive activity on surface electromyographic signals during volitional control tasks among individuals with complete spinal cord injury.Eleven patients with chronic complete thoracic spinal cord injury were recruited.Surface electromyography data from eight major leg muscles were acquired and compared before and after the passive activity protocol.The results indicated that the passive activity led to an increased number of activated volitional muscles and an increased frequency of activation.Although the cumulative root mean square of surface electromyography amplitude for volitional control of movement showed a slight increase after passive activity,the difference was not statistically significant.These findings suggest that brief passive activity may enhance the ability to initiate volitional muscle activity during surface electromyography tasks and underscore the potential of passive activity for improving residual motor control among patients with motor complete spinal cord injury.
基金grants from the National Natural Science Foundation of China(31771237 and 81773140)the Foundation and Advanced Research Project of Chongqing Science and Technology Commission(cstc2017shmsA130007)the Fundamental Research Funds for the Central Universities,China(SWU1709106)。
文摘Chronic stress affects brain function,so assessing its hazards is important for mental health.To overcome the limitations of behavioral data,we combined behavioral and event-related potentials(ERPs)in an attention network task.This task allowed us to differentiate between three specific aspects of attention:alerting,orienting,and execution.Forty-one participants under chronic stress and 31 non-stressed participants were enrolled.On the performance level,the chronically stressed group showed a significantly slower task response and lower accuracy.Concerning ERP measures,smaller cue-N1,cueN2,and larger cue-P3 amplitudes were found in the stressed group,indicating that this group was less able to assign attention to effective information,i.e.,they made inefficient use of cues and had difficulty in maintaining alerting.In addition,the stressed group showed larger target-N2 amplitudes,indicating that this group needed to allocate more cognitive resources to deal with the conflict targets task.Subgroup analysis revealed lower target-P3 amplitudes in the stressed than in the non-stressed group.Group differences associated with the attention networks were found at the ERP level.In the stressed group,excessive depletion of resources led to changes in attention control.In this study,we examined the effects of chronic stress on individual executive function from a neurological perspective.The results may benefit the development of interventions to improve executive function in chronically stressed individuals.
基金This work was supported by the National Natural Science Foundation of China(Grant No.51605115)State Key Laboratory of Robotics and System(Self-Planned Task No.SKLRS201719A)+1 种基金Heilongjiang Postdoctoral Financial Assistance(Grant No.LBH-Z16083)Natural Science Foundation of Heilongjiang Province(Grant No.QC2017052).
文摘As a well-explored template that captures the essential dynamical behaviors of legged locomotion on sagittal plane,the spring-loaded inverted pendulum(SLIP)model has been extensively employed in both biomechanical study and robotics research.Aiming at fully leveraging the merits of the SLIP model to generate the adaptive trajectories of the center of mass(CoM)with maneuverability,this study presents a novel two-layered sagittal SLIP-anchored(SSA)task space control for a monopode robot to deal with terrain irregularity.This work begins with an analytical investigation of sagittal SLIP dynamics by deriving an approximate solution with satisfactory apex prediction accuracy,and a two-layered SSA task space controller is subsequently developed for the monopode robot.The higher layer employs an analytical approximate representation of the sagittal SLIP model to form a deadbeat controller,which generates an adaptive reference trajectory for the CoM.The lower layer enforces the monopode robot to reproduce a generated CoM movement by using a task space controller to transfer the reference CoM commands into joint torques of the multi-degree of freedom monopode robot.Consequently,an adaptive hopping behavior is exhibited by the robot when traversing irregular terrain.Simulation results have demonstrated the effectiveness of the proposed method.
