This paper studied a supervisory control system for a hybrid off-highway electric vehicle under the chargesustaining(CS)condition.A new predictive double Q-learning with backup models(PDQL)scheme is proposed to optimi...This paper studied a supervisory control system for a hybrid off-highway electric vehicle under the chargesustaining(CS)condition.A new predictive double Q-learning with backup models(PDQL)scheme is proposed to optimize the engine fuel in real-world driving and improve energy efficiency with a faster and more robust learning process.Unlike the existing“model-free”methods,which solely follow on-policy and off-policy to update knowledge bases(Q-tables),the PDQL is developed with the capability to merge both on-policy and off-policy learning by introducing a backup model(Q-table).Experimental evaluations are conducted based on software-in-the-loop(SiL)and hardware-in-the-loop(HiL)test platforms based on real-time modelling of the studied vehicle.Compared to the standard double Q-learning(SDQL),the PDQL only needs half of the learning iterations to achieve better energy efficiency than the SDQL at the end learning process.In the SiL under 35 rounds of learning,the results show that the PDQL can improve the vehicle energy efficiency by 1.75%higher than SDQL.By implementing the PDQL in HiL under four predefined real-world conditions,the PDQL can robustly save more than 5.03%energy than the SDQL scheme.展开更多
In this paper,the problem of trajectory de-sign of unmanned aerial vehicles(UAVs)for maximizing the number of satisfied users is studied in a UAV based cellular network where the UAV works as a flying base station tha...In this paper,the problem of trajectory de-sign of unmanned aerial vehicles(UAVs)for maximizing the number of satisfied users is studied in a UAV based cellular network where the UAV works as a flying base station that serves users,and the user indicates its satis-faction in terms of completion of its data request within an allowable maximum waiting time.The trajectory design is formulated as an optimization problem whose goal is to maximize the number of satisfied users.To solve this problem,a machine learning framework based on double Q-learning algorithm is proposed.The algorithm enables the UAV tofind the optimal trajectory that maximizes the number of satisfied users.Compared to the traditional learning algorithms,such as Q-learning that selects and evaluates the action using the same Q-table,the proposed algorithm can decouple the selection from the evaluation,therefore avoid overestimation which leads to sub-optimal policies.Simulation results show that the proposed algorithm can achieve up to 19.4% and 14.1% gains in terms of the number of satisfied users compared to random algorithm and Q-learning algorithm.展开更多
The electric double layer(EDL)at the electrochemical interface is crucial for ion transport,charge transfer,and surface reactions in aqueous rechargeable zinc batteries(ARZBs).However,Zn anodes routinely encounter per...The electric double layer(EDL)at the electrochemical interface is crucial for ion transport,charge transfer,and surface reactions in aqueous rechargeable zinc batteries(ARZBs).However,Zn anodes routinely encounter persistent dendrite growth and parasitic reactions,driven by the inhomogeneous charge distribution and water-dominated environment within the EDL.Compounding this,classical EDL theory,rooted in meanfield approximations,further fails to resolve molecular-scale interfacial dynamics under battery-operating conditions,limiting mechanistic insights.Herein,we established a multiscale theoretical calculation framework from single molecular characteristics to interfacial ion distribution,revealing the EDL’s structure and interactions between different ions and molecules,which helps us understand the parasitic processes in depth.Simulations demonstrate that water dipole and sulfate ion adsorption at the inner Helmholtz plane drives severe hydrogen evolution and by-product formation.Guided by these insights,we engineered a“water-poor and anion-expelled”EDL using 4,1’,6’-trichlorogalactosucrose(TGS)as an electrolyte additive.As a result,Zn||Zn symmetric cells with TGS exhibited stable cycling for over 4700 h under a current density of 1 mA cm^(−2),while NaV_(3)O_(8)·1.5H_(2)O-based full cells kept 90.4%of the initial specific capacity after 800 cycles at 5 A g^(−1).This work highlights the power of multiscale theoretical frameworks to unravel EDL complexities and guide high-performance ARZB design through integrated theory-experiment approaches.展开更多
Wing design is a critical factor in the aerodynamic performance of flapping-wing(FW)robots.Inspired by the natural wing structures of insects,bats,and birds,we explored how bio-mimetic wing vein morphologies,combined ...Wing design is a critical factor in the aerodynamic performance of flapping-wing(FW)robots.Inspired by the natural wing structures of insects,bats,and birds,we explored how bio-mimetic wing vein morphologies,combined with a bio-inspired double wing clap-and-fling mechanism,affect thrust generation.This study focused on increasing vertical force and payload capacity.Through systematic experimentation with various vein configurations and structural designs,we developed innovative wings optimized for thrust production.Comprehensive tests were conducted to measure aerodynamic forces,power consumption,and wing kinematics across a range of flapping frequencies.Additionally,wings with different aspect ratios,a key factor in wing design,were fabricated and extensively evaluated.The study also examined the role of bio-inspired vein layouts on wing flexibility,a critical component in improving flight efficiency.Our findings demonstrate that the newly developed wing design led to a 20%increase in thrust,achieving up to 30 g-force(gf).This research sheds light on the clap-and-fling effect and establishes a promising framework for bio-inspired wing design,offering significant improvements in both performance and payload capacity for FW robots.展开更多
Soft X-ray detectors play a vital role in materials science,high-energy physics and medical imaging.Cs_(2)AgBiBr_(6),a lead-free double perovskite,has gained attention for its excellent optoelectronic properties,stabi...Soft X-ray detectors play a vital role in materials science,high-energy physics and medical imaging.Cs_(2)AgBiBr_(6),a lead-free double perovskite,has gained attention for its excellent optoelectronic properties,stability,and nontoxicity.However,its fast crystallization and requirement for high-temperature annealing(>250℃)often lead to inferior film quality,limiting its application in flexible devices.This study introduces an alloying strategy that significantly improves the quality of Cs_(2)AgBiBr_(6)thin films annealed at a reduced temperature of 150℃.Devices based on the alloyed thin films exhibit an ultra-low dark current of 0.32 nA·cm^(-2)and a quantum efficiency of 725%.Furthermore,the first successful integration of Cs_(2)AgBiBr_(6)with a thinfilm transistor backplane demonstrates its superior imaging performance,indicating that Cs_(2)AgBiBr_(6)is a promising material for next-generation soft X-ray sensors.展开更多
Co_(3)S_(4)electrocatalysts with mixed valences of Co ions and excellent structural stability possess favorable oxygen evolution reaction(OER)activity,yet challenges remain in fabricating rechargeable lithiumoxygen ba...Co_(3)S_(4)electrocatalysts with mixed valences of Co ions and excellent structural stability possess favorable oxygen evolution reaction(OER)activity,yet challenges remain in fabricating rechargeable lithiumoxygen batteries(LOBs)due to their poor OER performance,resulting from poor electrical conductivity and overly strong intermediate adsorption.In this work,fancy double heterojunctions on 1T/2H-MoS_(2)@Co_(3)S_(4)(1T/2H-MCS)were constructed derived from the charge donation from Co to Mo ions,thus inducing the phase transformation of Mo S_(2)from 2H to 1T.The unique features of these double heterojunctions endow the1T/2H-MCS with complementary catalysis during charging and discharging processes.It is worth noting that 1T-Mo S2@Co3S4could provide fast Co-S-Mo electron transport channels to promote ORR/OER kinetics,and 2H-MoS_(2)@Co_(3)S_(4)contributed to enabling moderate egorbital occupancy when adsorbed with oxygen-containing intermediates.On the basis,the Li_(2)O_(2)nucleation route was changed to solution and surface dual pathways,improving reversible deposition and decomposition kinetics.As a result,1T/2H-MCS cathodes exhibit an improved electrocatalytic performance compared with those of Co_(3)S_(4)and Mo S2cathodes.This innovative heterostructure design provides a reliable strategy to construct efficient transition metal sulfide catalysts by improving electrical conductivity and modulating adsorption toward oxygenated intermediates for LOBs.展开更多
Abiotic oxygen formation predates photosynthesis,sustaining early chemical evolution,yet its elementary mechanisms remain contested.Here,we show the production pathways for molecular oxygen from doubly ionized carbon ...Abiotic oxygen formation predates photosynthesis,sustaining early chemical evolution,yet its elementary mechanisms remain contested.Here,we show the production pathways for molecular oxygen from doubly ionized carbon dioxide upon electron-impact.Through fragment ions and electron coincidence momentum imaging,we unambiguously determine the ionization mechanism by measuring the projectile energy loss in association with the C^(+) +O_(2)^(+) channel.Further potential energy and trajectory calculations enable us to elucidate the dynamical details of this fragmentation process,in which a bond rearrangement pathway is found to proceed via the structural deformation to a triangular intermediate.Moreover,we demonstrate a further roaming pathway for the formation of O_(2)^(+) from CO_(2)^(+) 2,in which a frustrated C-O bond cleavage leaves the O atom without sufficient energy to escape.The O atom then wanders around varied configuration spaces of the flat potential energy regions and forms a C-O-O_(2)^(+) intermediate prior to the final products C^(+) +O_(2)^(+).Considering the large quantities of free electrons in interstellar space,the processes revealed here are expected to be significant and should be incorporated into atmospheric evolution models.展开更多
Aiming at the dimension disaster problem, poor model generalization ability and deadlock problem in special obstacles environment caused by the increase of state information in the local path planning process of mobil...Aiming at the dimension disaster problem, poor model generalization ability and deadlock problem in special obstacles environment caused by the increase of state information in the local path planning process of mobile robot, this paper proposed a Double BP Q-learning algorithm based on the fusion of Double Q-learning algorithm and BP neural network. In order to solve the dimensional disaster problem, two BP neural network fitting value functions with the same network structure were used to replace the two <i>Q</i> value tables in Double Q-Learning algorithm to solve the problem that the <i>Q</i> value table cannot store excessive state information. By adding the mechanism of priority experience replay and using the parameter transfer to initialize the model parameters in different environments, it could accelerate the convergence rate of the algorithm, improve the learning efficiency and the generalization ability of the model. By designing specific action selection strategy in special environment, the deadlock state could be avoided and the mobile robot could reach the target point. Finally, the designed Double BP Q-learning algorithm was simulated and verified, and the probability of mobile robot reaching the target point in the parameter update process was compared with the Double Q-learning algorithm under the same condition of the planned path length. The results showed that the model trained by the improved Double BP Q-learning algorithm had a higher success rate in finding the optimal or sub-optimal path in the dense discrete environment, besides, it had stronger model generalization ability, fewer redundant sections, and could reach the target point without entering the deadlock zone in the special obstacles environment.展开更多
针对无监督环境下传统网络异常诊断算法存在异常点定位和异常数据分类准确率低等不足,通过设计一种基于改进Q-learning算法的无线网络异常诊断方法:首先基于ADU(Asynchronous Data Unit异步数据单元)单元采集无线网络的数据流,并提取数...针对无监督环境下传统网络异常诊断算法存在异常点定位和异常数据分类准确率低等不足,通过设计一种基于改进Q-learning算法的无线网络异常诊断方法:首先基于ADU(Asynchronous Data Unit异步数据单元)单元采集无线网络的数据流,并提取数据包特征;然后构建Q-learning算法模型探索状态值和奖励值的平衡点,利用SA(Simulated Annealing模拟退火)算法从全局视角对下一时刻状态进行精确识别;最后确定训练样本的联合分布概率,提升输出值的逼近性能以达到平衡探索与代价之间的均衡。测试结果显示:改进Q-learning算法的网络异常定位准确率均值达99.4%,在不同类型网络异常的分类精度和分类效率等方面,也优于三种传统网络异常诊断方法。展开更多
基金Project(KF2029)supported by the State Key Laboratory of Automotive Safety and Energy(Tsinghua University),ChinaProject(102253)supported partially by the Innovate UK。
文摘This paper studied a supervisory control system for a hybrid off-highway electric vehicle under the chargesustaining(CS)condition.A new predictive double Q-learning with backup models(PDQL)scheme is proposed to optimize the engine fuel in real-world driving and improve energy efficiency with a faster and more robust learning process.Unlike the existing“model-free”methods,which solely follow on-policy and off-policy to update knowledge bases(Q-tables),the PDQL is developed with the capability to merge both on-policy and off-policy learning by introducing a backup model(Q-table).Experimental evaluations are conducted based on software-in-the-loop(SiL)and hardware-in-the-loop(HiL)test platforms based on real-time modelling of the studied vehicle.Compared to the standard double Q-learning(SDQL),the PDQL only needs half of the learning iterations to achieve better energy efficiency than the SDQL at the end learning process.In the SiL under 35 rounds of learning,the results show that the PDQL can improve the vehicle energy efficiency by 1.75%higher than SDQL.By implementing the PDQL in HiL under four predefined real-world conditions,the PDQL can robustly save more than 5.03%energy than the SDQL scheme.
基金supported in part by the National Natural Science Foundation of China under Grant 61671086 and Grant 61629101。
文摘In this paper,the problem of trajectory de-sign of unmanned aerial vehicles(UAVs)for maximizing the number of satisfied users is studied in a UAV based cellular network where the UAV works as a flying base station that serves users,and the user indicates its satis-faction in terms of completion of its data request within an allowable maximum waiting time.The trajectory design is formulated as an optimization problem whose goal is to maximize the number of satisfied users.To solve this problem,a machine learning framework based on double Q-learning algorithm is proposed.The algorithm enables the UAV tofind the optimal trajectory that maximizes the number of satisfied users.Compared to the traditional learning algorithms,such as Q-learning that selects and evaluates the action using the same Q-table,the proposed algorithm can decouple the selection from the evaluation,therefore avoid overestimation which leads to sub-optimal policies.Simulation results show that the proposed algorithm can achieve up to 19.4% and 14.1% gains in terms of the number of satisfied users compared to random algorithm and Q-learning algorithm.
基金supported by the National Natural Science Foundation of China(52471240)the Natural Science Foundation of Zhejiang Province(LZ23B030003)+2 种基金the Fundamental Research Funds for the Central Universities(226-2024-00075)support from the Engineering and Physical Sciences Research Council(EPSRC,UK)RiR grant-RIR18221018-1EU COST CA23155。
文摘The electric double layer(EDL)at the electrochemical interface is crucial for ion transport,charge transfer,and surface reactions in aqueous rechargeable zinc batteries(ARZBs).However,Zn anodes routinely encounter persistent dendrite growth and parasitic reactions,driven by the inhomogeneous charge distribution and water-dominated environment within the EDL.Compounding this,classical EDL theory,rooted in meanfield approximations,further fails to resolve molecular-scale interfacial dynamics under battery-operating conditions,limiting mechanistic insights.Herein,we established a multiscale theoretical calculation framework from single molecular characteristics to interfacial ion distribution,revealing the EDL’s structure and interactions between different ions and molecules,which helps us understand the parasitic processes in depth.Simulations demonstrate that water dipole and sulfate ion adsorption at the inner Helmholtz plane drives severe hydrogen evolution and by-product formation.Guided by these insights,we engineered a“water-poor and anion-expelled”EDL using 4,1’,6’-trichlorogalactosucrose(TGS)as an electrolyte additive.As a result,Zn||Zn symmetric cells with TGS exhibited stable cycling for over 4700 h under a current density of 1 mA cm^(−2),while NaV_(3)O_(8)·1.5H_(2)O-based full cells kept 90.4%of the initial specific capacity after 800 cycles at 5 A g^(−1).This work highlights the power of multiscale theoretical frameworks to unravel EDL complexities and guide high-performance ARZB design through integrated theory-experiment approaches.
基金Nguyen Tat Thanh University,Ho Chi Minh City,Vietnam for supporting this study。
文摘Wing design is a critical factor in the aerodynamic performance of flapping-wing(FW)robots.Inspired by the natural wing structures of insects,bats,and birds,we explored how bio-mimetic wing vein morphologies,combined with a bio-inspired double wing clap-and-fling mechanism,affect thrust generation.This study focused on increasing vertical force and payload capacity.Through systematic experimentation with various vein configurations and structural designs,we developed innovative wings optimized for thrust production.Comprehensive tests were conducted to measure aerodynamic forces,power consumption,and wing kinematics across a range of flapping frequencies.Additionally,wings with different aspect ratios,a key factor in wing design,were fabricated and extensively evaluated.The study also examined the role of bio-inspired vein layouts on wing flexibility,a critical component in improving flight efficiency.Our findings demonstrate that the newly developed wing design led to a 20%increase in thrust,achieving up to 30 g-force(gf).This research sheds light on the clap-and-fling effect and establishes a promising framework for bio-inspired wing design,offering significant improvements in both performance and payload capacity for FW robots.
基金supported by the NSFC under Grant No.62474169the National Key Research and Development Program of China under Grant No.2024YFB3212200the funding from USTC under Grant Nos.WK2100000025,KY2190000003,and KY2190000006。
文摘Soft X-ray detectors play a vital role in materials science,high-energy physics and medical imaging.Cs_(2)AgBiBr_(6),a lead-free double perovskite,has gained attention for its excellent optoelectronic properties,stability,and nontoxicity.However,its fast crystallization and requirement for high-temperature annealing(>250℃)often lead to inferior film quality,limiting its application in flexible devices.This study introduces an alloying strategy that significantly improves the quality of Cs_(2)AgBiBr_(6)thin films annealed at a reduced temperature of 150℃.Devices based on the alloyed thin films exhibit an ultra-low dark current of 0.32 nA·cm^(-2)and a quantum efficiency of 725%.Furthermore,the first successful integration of Cs_(2)AgBiBr_(6)with a thinfilm transistor backplane demonstrates its superior imaging performance,indicating that Cs_(2)AgBiBr_(6)is a promising material for next-generation soft X-ray sensors.
基金financially supported by the National Natural Science Foundation of China(U21A20311,U24A2040,52171141,52272117)the Natural Science Foundation of Shandong Province(ZR2022JQ19)+3 种基金the Key Technology Research Project of Shandong Province(2023CXGC010202)the Taishan Industrial Experts Program(TSCX202306142)the Core Facility Sharing Platform of Shandong Universitythe Foundation of Key Laboratory of Advanced Energy Materials Chemistry(Ministry of Education),Nankai University。
文摘Co_(3)S_(4)electrocatalysts with mixed valences of Co ions and excellent structural stability possess favorable oxygen evolution reaction(OER)activity,yet challenges remain in fabricating rechargeable lithiumoxygen batteries(LOBs)due to their poor OER performance,resulting from poor electrical conductivity and overly strong intermediate adsorption.In this work,fancy double heterojunctions on 1T/2H-MoS_(2)@Co_(3)S_(4)(1T/2H-MCS)were constructed derived from the charge donation from Co to Mo ions,thus inducing the phase transformation of Mo S_(2)from 2H to 1T.The unique features of these double heterojunctions endow the1T/2H-MCS with complementary catalysis during charging and discharging processes.It is worth noting that 1T-Mo S2@Co3S4could provide fast Co-S-Mo electron transport channels to promote ORR/OER kinetics,and 2H-MoS_(2)@Co_(3)S_(4)contributed to enabling moderate egorbital occupancy when adsorbed with oxygen-containing intermediates.On the basis,the Li_(2)O_(2)nucleation route was changed to solution and surface dual pathways,improving reversible deposition and decomposition kinetics.As a result,1T/2H-MCS cathodes exhibit an improved electrocatalytic performance compared with those of Co_(3)S_(4)and Mo S2cathodes.This innovative heterostructure design provides a reliable strategy to construct efficient transition metal sulfide catalysts by improving electrical conductivity and modulating adsorption toward oxygenated intermediates for LOBs.
基金supported by the National Natural Science Foundation of China (Grant Nos.12325406,92261201,12404305,and W2512072)the Shaanxi Province Natural Science Fundamental Research Project (Grant Nos.2023JC-XJ-03 and23JSQ013)the China Postdoctoral Science Foundation (Grant Nos.BX20240286 and 2024M7625)。
文摘Abiotic oxygen formation predates photosynthesis,sustaining early chemical evolution,yet its elementary mechanisms remain contested.Here,we show the production pathways for molecular oxygen from doubly ionized carbon dioxide upon electron-impact.Through fragment ions and electron coincidence momentum imaging,we unambiguously determine the ionization mechanism by measuring the projectile energy loss in association with the C^(+) +O_(2)^(+) channel.Further potential energy and trajectory calculations enable us to elucidate the dynamical details of this fragmentation process,in which a bond rearrangement pathway is found to proceed via the structural deformation to a triangular intermediate.Moreover,we demonstrate a further roaming pathway for the formation of O_(2)^(+) from CO_(2)^(+) 2,in which a frustrated C-O bond cleavage leaves the O atom without sufficient energy to escape.The O atom then wanders around varied configuration spaces of the flat potential energy regions and forms a C-O-O_(2)^(+) intermediate prior to the final products C^(+) +O_(2)^(+).Considering the large quantities of free electrons in interstellar space,the processes revealed here are expected to be significant and should be incorporated into atmospheric evolution models.
文摘Aiming at the dimension disaster problem, poor model generalization ability and deadlock problem in special obstacles environment caused by the increase of state information in the local path planning process of mobile robot, this paper proposed a Double BP Q-learning algorithm based on the fusion of Double Q-learning algorithm and BP neural network. In order to solve the dimensional disaster problem, two BP neural network fitting value functions with the same network structure were used to replace the two <i>Q</i> value tables in Double Q-Learning algorithm to solve the problem that the <i>Q</i> value table cannot store excessive state information. By adding the mechanism of priority experience replay and using the parameter transfer to initialize the model parameters in different environments, it could accelerate the convergence rate of the algorithm, improve the learning efficiency and the generalization ability of the model. By designing specific action selection strategy in special environment, the deadlock state could be avoided and the mobile robot could reach the target point. Finally, the designed Double BP Q-learning algorithm was simulated and verified, and the probability of mobile robot reaching the target point in the parameter update process was compared with the Double Q-learning algorithm under the same condition of the planned path length. The results showed that the model trained by the improved Double BP Q-learning algorithm had a higher success rate in finding the optimal or sub-optimal path in the dense discrete environment, besides, it had stronger model generalization ability, fewer redundant sections, and could reach the target point without entering the deadlock zone in the special obstacles environment.
文摘针对无监督环境下传统网络异常诊断算法存在异常点定位和异常数据分类准确率低等不足,通过设计一种基于改进Q-learning算法的无线网络异常诊断方法:首先基于ADU(Asynchronous Data Unit异步数据单元)单元采集无线网络的数据流,并提取数据包特征;然后构建Q-learning算法模型探索状态值和奖励值的平衡点,利用SA(Simulated Annealing模拟退火)算法从全局视角对下一时刻状态进行精确识别;最后确定训练样本的联合分布概率,提升输出值的逼近性能以达到平衡探索与代价之间的均衡。测试结果显示:改进Q-learning算法的网络异常定位准确率均值达99.4%,在不同类型网络异常的分类精度和分类效率等方面,也优于三种传统网络异常诊断方法。
