In this paper,we present a Q-Learning optimization algorithm for smart home HVAC systems.The proposed algorithm combines new convex deep neural network models with model predictive control(MPC)techniques.More specific...In this paper,we present a Q-Learning optimization algorithm for smart home HVAC systems.The proposed algorithm combines new convex deep neural network models with model predictive control(MPC)techniques.More specifically,new input convex long short-term memory(ICLSTM)models are employed to predict dynamic states in an MPC optimal control technique integrated within a Q-Learning reinforcement learning(RL)algorithm to further improve the learned temporal behaviors of nonlinear HVAC systems.As a novel RL approach,the proposed algorithm generates day-ahead HVAC demand response(DR)signals in smart homes that optimally reduce and/or shift peak energy usage,reduce electricity costs,minimize user discomfort,and honor in a best-effort way the recommendations from utility/aggregator,which in turn has impact on the overall well being of the distribution network controlled by the aggregator.The proposed Q-Learning optimization algorithm,based on epsilon-model predictive control(e-MPC),can be implemented as a control agent that is executed by the smart house energy management(SHEM)system that we assume exists in the smart home,which can interact with the energy provider of the distribution network,i.e.,utility/aggregator,via the smart meter.The output generated by the proposed control agent represents day-ahead local DR signals in the form of temperature setpoints for the HVAC system that are found by the optimization process to lead to desired trade-offs between electricity cost and user discomfort.The proposed algorithm can be used in smart homes with passive HVAC controllers,which solely react to end-user setpoints,to transform them into smart homes with active HVAC controllers.Such systems not only respond to the preferences of the end-user but also incorporate an external control signal provided by the utility or aggregator.Simulation experiments conducted with a custom simulation tool demonstrate that the proposed optimization framework can offer significant benefits.It achieves 87%higher success rate in optimizing setpoints in the desired range,thereby resulting in up to 15%energy savings and zero temperature discomfort.展开更多
In this paper, we present a novel cloud-based demand side management (DSM) optimization approach for the cost reduction of energy usage in heating, ventilation and air conditioning (HVAC) systems in residential homes ...In this paper, we present a novel cloud-based demand side management (DSM) optimization approach for the cost reduction of energy usage in heating, ventilation and air conditioning (HVAC) systems in residential homes at the district level. The proposed approach achieves optimization through scheduling of HVAC energy usage within permissible bounds set by house users. House smart home energy management (SHEM) devices are connected to the utility/aggregator via a dedicated communication network that is used to enable DSM. Each house SHEM can predict its own HVAC energy usage for the next 24 h using minimalistic deep learning (DL) prediction models. These predictions are communicated to the aggregator, which will then do day ahead optimizations using the proposed game theory (GT) algorithm. The GT model captures the interaction between aggregator and customers and identifies a solution to the GT problem that translates into HVAC energy peak shifting and peak reduction achieved by rescheduling HVAC energy usage. The found solution is communicated by the aggregator to houses SHEM devices in the form of offers via DSM signals. If customers’ SHEM devices accept the offer, then energy cost reduction will be achieved. To validate the proposed algorithm, we conduct extensive simulations with a custom simulation tool based on GridLab-D tool, which is integrated with DL prediction models and optimization libraries. Results show that HVAC energy cost can be reduced by up to 36% while indirectly also reducing the peak-to-average (PAR) and the aggregated net load by up to 9.97%.展开更多
基金supported by an award from the National Science Foundation,USA grant ECCF 1936494..
文摘In this paper,we present a Q-Learning optimization algorithm for smart home HVAC systems.The proposed algorithm combines new convex deep neural network models with model predictive control(MPC)techniques.More specifically,new input convex long short-term memory(ICLSTM)models are employed to predict dynamic states in an MPC optimal control technique integrated within a Q-Learning reinforcement learning(RL)algorithm to further improve the learned temporal behaviors of nonlinear HVAC systems.As a novel RL approach,the proposed algorithm generates day-ahead HVAC demand response(DR)signals in smart homes that optimally reduce and/or shift peak energy usage,reduce electricity costs,minimize user discomfort,and honor in a best-effort way the recommendations from utility/aggregator,which in turn has impact on the overall well being of the distribution network controlled by the aggregator.The proposed Q-Learning optimization algorithm,based on epsilon-model predictive control(e-MPC),can be implemented as a control agent that is executed by the smart house energy management(SHEM)system that we assume exists in the smart home,which can interact with the energy provider of the distribution network,i.e.,utility/aggregator,via the smart meter.The output generated by the proposed control agent represents day-ahead local DR signals in the form of temperature setpoints for the HVAC system that are found by the optimization process to lead to desired trade-offs between electricity cost and user discomfort.The proposed algorithm can be used in smart homes with passive HVAC controllers,which solely react to end-user setpoints,to transform them into smart homes with active HVAC controllers.Such systems not only respond to the preferences of the end-user but also incorporate an external control signal provided by the utility or aggregator.Simulation experiments conducted with a custom simulation tool demonstrate that the proposed optimization framework can offer significant benefits.It achieves 87%higher success rate in optimizing setpoints in the desired range,thereby resulting in up to 15%energy savings and zero temperature discomfort.
基金supported by the National Science Foundation(NSF)grant ECCF 1936494.
文摘In this paper, we present a novel cloud-based demand side management (DSM) optimization approach for the cost reduction of energy usage in heating, ventilation and air conditioning (HVAC) systems in residential homes at the district level. The proposed approach achieves optimization through scheduling of HVAC energy usage within permissible bounds set by house users. House smart home energy management (SHEM) devices are connected to the utility/aggregator via a dedicated communication network that is used to enable DSM. Each house SHEM can predict its own HVAC energy usage for the next 24 h using minimalistic deep learning (DL) prediction models. These predictions are communicated to the aggregator, which will then do day ahead optimizations using the proposed game theory (GT) algorithm. The GT model captures the interaction between aggregator and customers and identifies a solution to the GT problem that translates into HVAC energy peak shifting and peak reduction achieved by rescheduling HVAC energy usage. The found solution is communicated by the aggregator to houses SHEM devices in the form of offers via DSM signals. If customers’ SHEM devices accept the offer, then energy cost reduction will be achieved. To validate the proposed algorithm, we conduct extensive simulations with a custom simulation tool based on GridLab-D tool, which is integrated with DL prediction models and optimization libraries. Results show that HVAC energy cost can be reduced by up to 36% while indirectly also reducing the peak-to-average (PAR) and the aggregated net load by up to 9.97%.
