This paper introduces a novel privacy-aware Federated Proximal Policy Optimization(FPPO)method combined with action masking.As a Federated Reinforcement Learning(FRL)approach,the proposed method is used for optimizing...This paper introduces a novel privacy-aware Federated Proximal Policy Optimization(FPPO)method combined with action masking.As a Federated Reinforcement Learning(FRL)approach,the proposed method is used for optimizing the reloading of Domestic Hot Water(DHW)storage tanks,with a focus on energy savings and DHW thermal comfort in collective heating systems.The proposed approach combines FedProx as the Federated Learning(FL)method and Proximal Policy Optimization(PPO)as the Deep Reinforcement Learning(DRL)technique to address the challenges of distributed control while ensuring data privacy.Key contributions include:(1)employing action masking to guarantee compliance with comfort level,(2)designing a global reward function to align agents actions toward collective energy savings,(3)implementing a privacy-aware design where only model parameters are shared with a global aggregator,avoiding raw data transmission,and(4)optimizing PPO’s loss function for improved performance.PPO was benchmarked using a common FL method(FedAvg)alongside two other DRL methods,where PPO outperformed both in scalability and energy savings,especially in larger systems.Then,PPO-based FRL was refined into FPPO by integrating a proximal term with coefficient into the loss function to enhance the performance.Experiments were conducted with both fixed and dynamically adjusted,with the latter demonstrating better energy savings and comfort.Results show that FPPO achieves up to 10.08%energy savings while maintaining DHW discomfort below 8.72%in systems with at least 20 dwellings.These findings highlight FPPO as a scalable,privacy-aware,and energy-efficient solution for distributed control in collective heating systems.展开更多
Latex as an asphalt modifier has gained popularity in the asphalt industry as it improves the durability of asphalt pavement.However,the elastomeric properties of latex stiffen the asphalt binders,resulting in additio...Latex as an asphalt modifier has gained popularity in the asphalt industry as it improves the durability of asphalt pavement.However,the elastomeric properties of latex stiffen the asphalt binders,resulting in additional energy consumption during the production of asphalt mixtures,which may cause a higher emission of greenhouse gases.This is undesirable for sustainable development and the environment.In this study,the applicability of diluted methanol and water was comparatively evaluated as foaming agents in the production of warm mix asphalt(WMA)mixtures incorporating latex.Diluted methanol was used because it has a lower boiling point and latent heat than water,allowing the asphalt mixture to be produced at a lower temperature and thus consuming less energy.The performance of the foamed asphalt mixture was investigated through service characteristics,mechanical performance,and moisture susceptibility of mixtures.The service characteristics,on the other hand,were measured in a laboratory while preparing and compacting the asphalt mixture,which refers to the amount of energy required during the production and construction stages in the asphalt plant and on the construction site,respectively.The degree of energy required was assessed based on the workability index,coatability index,and the compaction energy index.The mechanical performance of asphalt mixtures was characterized by indirect tensile strength,resilient modulus,and dynamic creep tests.The resistance to moisture damage was evaluated based on the common parameter,indirect tensile strength ratio.The findings revealed that the use of diluted methanol foaming agent helped improve the workability of latex modified asphalt mixtures.The foamed latex-modified WMA demonstrated better performance compared to asphalt mixtures prepared using water as the foaming agent.展开更多
基金funded by a PhD fellowship of the Research Foundation Flanders(FWO)[1S08624N].
文摘This paper introduces a novel privacy-aware Federated Proximal Policy Optimization(FPPO)method combined with action masking.As a Federated Reinforcement Learning(FRL)approach,the proposed method is used for optimizing the reloading of Domestic Hot Water(DHW)storage tanks,with a focus on energy savings and DHW thermal comfort in collective heating systems.The proposed approach combines FedProx as the Federated Learning(FL)method and Proximal Policy Optimization(PPO)as the Deep Reinforcement Learning(DRL)technique to address the challenges of distributed control while ensuring data privacy.Key contributions include:(1)employing action masking to guarantee compliance with comfort level,(2)designing a global reward function to align agents actions toward collective energy savings,(3)implementing a privacy-aware design where only model parameters are shared with a global aggregator,avoiding raw data transmission,and(4)optimizing PPO’s loss function for improved performance.PPO was benchmarked using a common FL method(FedAvg)alongside two other DRL methods,where PPO outperformed both in scalability and energy savings,especially in larger systems.Then,PPO-based FRL was refined into FPPO by integrating a proximal term with coefficient into the loss function to enhance the performance.Experiments were conducted with both fixed and dynamically adjusted,with the latter demonstrating better energy savings and comfort.Results show that FPPO achieves up to 10.08%energy savings while maintaining DHW discomfort below 8.72%in systems with at least 20 dwellings.These findings highlight FPPO as a scalable,privacy-aware,and energy-efficient solution for distributed control in collective heating systems.
基金The authors express their appreciation to the National Natural Science Foundation of China(NSFC)for providing financial assistance via the Research Fund for the International Young Scientist(Grant No.51750110491)Additionally,acknowledgements are due to Universiti Sains Malaysia for providing financial support via Research University Individual(RUI)Grant 1001.PAWAM.8014140.Authors also would like to recognize supports from Chang'an University,China.Last but not least,special thanks to all technical staff of the Highway Engineering Laboratory,Universiti Sains Malaysia(USM),for their valuable help and support.
文摘Latex as an asphalt modifier has gained popularity in the asphalt industry as it improves the durability of asphalt pavement.However,the elastomeric properties of latex stiffen the asphalt binders,resulting in additional energy consumption during the production of asphalt mixtures,which may cause a higher emission of greenhouse gases.This is undesirable for sustainable development and the environment.In this study,the applicability of diluted methanol and water was comparatively evaluated as foaming agents in the production of warm mix asphalt(WMA)mixtures incorporating latex.Diluted methanol was used because it has a lower boiling point and latent heat than water,allowing the asphalt mixture to be produced at a lower temperature and thus consuming less energy.The performance of the foamed asphalt mixture was investigated through service characteristics,mechanical performance,and moisture susceptibility of mixtures.The service characteristics,on the other hand,were measured in a laboratory while preparing and compacting the asphalt mixture,which refers to the amount of energy required during the production and construction stages in the asphalt plant and on the construction site,respectively.The degree of energy required was assessed based on the workability index,coatability index,and the compaction energy index.The mechanical performance of asphalt mixtures was characterized by indirect tensile strength,resilient modulus,and dynamic creep tests.The resistance to moisture damage was evaluated based on the common parameter,indirect tensile strength ratio.The findings revealed that the use of diluted methanol foaming agent helped improve the workability of latex modified asphalt mixtures.The foamed latex-modified WMA demonstrated better performance compared to asphalt mixtures prepared using water as the foaming agent.
