Building heating,ventilating,and air conditioning(HVAC)systems have one of the largest energy footprint worldwide,which necessitates the design of intelligent control algorithms that improve the energy utilization whi...Building heating,ventilating,and air conditioning(HVAC)systems have one of the largest energy footprint worldwide,which necessitates the design of intelligent control algorithms that improve the energy utilization while still providing thermal comfort.In this work,the authors formulate the HVAC equipment dynamics in the setting of a two-player non-zero-sum cooperative game,which enables two decision variables(mass flow rate and supply air temperature)to perform joint optimization of the control utilization and thermal setpoint tracking by simultaneously exchanging their policies.The HVAC zone serves as a game environment for these two decision variables that act as two players in a game.It is assumed that dynamic models of HVAC equipment are not available.Furthermore,neither the state nor any estimates of HVAC disturbance(heat gains,outside variations,etc.)are accessible,but only the measurement of the zone temperature is available for feedback.Under these constraints,the authors develop a new data-driven Q-learning scheme employing policy iteration and value iteration with a bias compensation mechanism that accounts for unmeasurable disturbances and circumvents the need of full-state measurement.The proposed algorithms are shown to converge to the optimal solution corresponding to the generalized algebraic Riccati equations(GAREs)in dynamic games.展开更多
Recent advancements have shown that control strategies using Deep Reinforcement Learning(DRL)can significantly improve the management of HVAC control and energy systems in buildings,leading to significant energy savin...Recent advancements have shown that control strategies using Deep Reinforcement Learning(DRL)can significantly improve the management of HVAC control and energy systems in buildings,leading to significant energy savings and better comfort.Unlike conventional rule-based controllers,they demand considerable time and data to develop effective policies.Transfer learning using pre-trained models can help address this issue.In this work,we use imitation learning(IL)as a method of pre-training and reinforcement learning(RL)for fine-tuning.However,HVAC systems can vary depending on the location,building size,structure,construction materials and weather conditions.The diversity in HVAC control systems across different buildings complicates the use of IL and RL.Neural network weights trained on the source building cannot be directly transferred to the target building because of differences in input features and the number of control equipment.To overcome this problem,we propose a novel padding method to ensure that both the source and target buildings share the same state space dimensionality.Thus,the trained neural network weights are transferable,and only the output layer must be adjusted to fit the dimensionality of the target action space.Additionally,we evaluate the performance of an existing padding technique for comparison.Our experiments show that the novel padding technique outperforms zero padding by 1.37%and training from scratch by 4.59%on average.展开更多
This paper proposes a novel reinforcement learning(RL)architecture for the efficient scheduling and control of the heating,ventilation and air conditioning(HVAC)system in a commercial building while harnessing its de-...This paper proposes a novel reinforcement learning(RL)architecture for the efficient scheduling and control of the heating,ventilation and air conditioning(HVAC)system in a commercial building while harnessing its de-mand response(DR)potentials.With advances in automated building management systems,this can be achieved seamlessly by a smart autonomous RL agent which takes the best action,for example,a change in HVAC temper-ature set point,necessary to change the electricity usage pattern of a building in response to demand response signals,and with minimal thermal comfort impact to customers.Previous research in this area has tackled only individual aspects of the problem using RL.Specifically,due to the challenges in implementing demand response with whole-building models,simpler analytical models which poorly capture reality have been used instead.And where whole-building models are applied,RL is used for HVAC control mainly to achieve energy efficiency goals while demand response is neglected.Thus,in this research,we implement a holistic framework by designing an efficient RL controller for a whole-building model which learns to optimise and control the HVAC system for improved energy efficiency and thermal comfort levels in addition to achieving demand response goals.Our simulation results show that by applying reinforcement learning for normal HVAC operation,a maximum weekly energy reduction of up to 22%can be achieved compared to a handcrafted baseline controller.Furthermore,by employing a DR-aware RL controller during demand response periods,average power reductions or increases of up to 50%can be achieved on a weekly basis compared to the default RL controller,while keeping occupant thermal comfort levels within acceptable bounds.展开更多
We present the development of a bias compensating reinforcement learning(RL)algorithm that optimizes thermal comfort(by minimizing tracking error)and control utilization(by penalizing setpoint deviations)in a multi-zo...We present the development of a bias compensating reinforcement learning(RL)algorithm that optimizes thermal comfort(by minimizing tracking error)and control utilization(by penalizing setpoint deviations)in a multi-zone heating,ventilation,and air-conditioning(HVAC)lab facility subject to unmeasurable disturbances and unknown dynamics.It is shown that the presence of unmeasurable disturbance results in an inconsistent learning equation in traditional RL controllers leading to parameter estimation bias(even with integral action support),and in the extreme case,the divergence of the learning algorithm.We demonstrate this issue by applying the popular Q-learning algorithm to linear quadratic regulation(LQR)of a multi-zone HVAC environment and showing that,even with integral support,the algorithm exhibits bias issue during the learning phase when the HVAC disturbance is unmeasurable due to unknown heat gains,occupancy variations,light sources,and outside weather changes.To address this difficulty,we present a bias compensating learning equation that learns a lumped bias term as a result of disturbances(and possibly other sources)in conjunction with the optimal control parameters.Experimental results show that the proposed scheme not only recovers the bias-free optimal control parameters but it does so without explicitly learning the dynamic model or estimating the disturbances,demonstrating the effectiveness of the algorithm in addressing the above challenges.展开更多
In heating, ventilating and air-conditioning （HVAC） systems, there exist severe nonlinearity, time-varying nature, disturbances and uncertainties. A new predictive functional control based on Takagi-Sugeno （T-S） f...In heating, ventilating and air-conditioning （HVAC） systems, there exist severe nonlinearity, time-varying nature, disturbances and uncertainties. A new predictive functional control based on Takagi-Sugeno （T-S） fuzzy model was proposed to control HVAC systems. The T-S fuzzy model of stabilized controlled process was obtained using the least squares method, then on the basis of global linear predictive model from T-S fuzzy model, the process was controlled by the predictive functional controller. Especially the feedback regulation part was developed to compensate uncertainties of fuzzy predictive model. Finally simulation test results in HVAC systems control applications showed that the proposed fuzzy model predictive functional control improves tracking effect and robustness. Compared with the conventional PID controller, this control strategy has the advantages of less overshoot and shorter setting time, etc.展开更多
Heating, ventilation, and air conditioning (HVAC) system is significant to the energy efficiency in buildings. In this paper, temperature control of HVAC system is studied in winter operation season. The physical mode...Heating, ventilation, and air conditioning (HVAC) system is significant to the energy efficiency in buildings. In this paper, temperature control of HVAC system is studied in winter operation season. The physical model of the zone, the fan, the heating coil and sensor are built. HVAC is a non-linear, strong disturbance and coupling system. Linear active-rejection-disturbance-control is an appreciate control algorithm which can adapt to less information, strong-disturbance influence, and has relative-fixed structure and simple tuning process of the controller parameters. Active-rejection-disturbance-control of the HVAC system is proposed. Simulation in Matlab/Simulink was done. Simulation results show that linear active-rejection-disturbance-control was prior to PID and integral-fuzzy controllers in rising time, overshoot and response time of step disturbance. The study can provide fundamental basis for the control of the air-condition system with strong-disturbance and high-precision needed.展开更多
This paper proposes an event-triggered stochastic model predictive control for discrete-time linear time-invariant(LTI)systems under additive stochastic disturbances.It first constructs a probabilistic invariant set a...This paper proposes an event-triggered stochastic model predictive control for discrete-time linear time-invariant(LTI)systems under additive stochastic disturbances.It first constructs a probabilistic invariant set and a probabilistic reachable set based on the priori knowledge of system uncertainties.Assisted with enhanced robust tubes,the chance constraints are then formulated into a deterministic form.To alleviate the online computational burden,a novel event-triggered stochastic model predictive control is developed,where the triggering condition is designed based on the past and future optimal trajectory tracking errors in order to achieve a good trade-off between system resource utilization and control performance.Two triggering parametersσandγare used to adjust the frequency of solving the optimization problem.The probabilistic feasibility and stability of the system under the event-triggered mechanism are also examined.Finally,numerical studies on the control of a heating,ventilation,and air conditioning(HVAC)system confirm the efficacy of the proposed control.展开更多
针对暖通空调(HVAC)系统难以控制的问题,提出一种基于m ax-product推理的M am dan i模糊模型预测控制策略.首先利用一步模糊预测模型的结构分析得到其解析表达式,获得系统在k+1时刻的线性化预测模型;然后基于模糊线性化模型进行模型预...针对暖通空调(HVAC)系统难以控制的问题,提出一种基于m ax-product推理的M am dan i模糊模型预测控制策略.首先利用一步模糊预测模型的结构分析得到其解析表达式,获得系统在k+1时刻的线性化预测模型;然后基于模糊线性化模型进行模型预测控制器设计.对HVAC系统的仿真和实验结果表明,该算法是一种跟踪性能好且鲁棒性强的有效控制算法.展开更多
文摘Building heating,ventilating,and air conditioning(HVAC)systems have one of the largest energy footprint worldwide,which necessitates the design of intelligent control algorithms that improve the energy utilization while still providing thermal comfort.In this work,the authors formulate the HVAC equipment dynamics in the setting of a two-player non-zero-sum cooperative game,which enables two decision variables(mass flow rate and supply air temperature)to perform joint optimization of the control utilization and thermal setpoint tracking by simultaneously exchanging their policies.The HVAC zone serves as a game environment for these two decision variables that act as two players in a game.It is assumed that dynamic models of HVAC equipment are not available.Furthermore,neither the state nor any estimates of HVAC disturbance(heat gains,outside variations,etc.)are accessible,but only the measurement of the zone temperature is available for feedback.Under these constraints,the authors develop a new data-driven Q-learning scheme employing policy iteration and value iteration with a bias compensation mechanism that accounts for unmeasurable disturbances and circumvents the need of full-state measurement.The proposed algorithms are shown to converge to the optimal solution corresponding to the generalized algebraic Riccati equations(GAREs)in dynamic games.
基金financial support of TaighdeÉireann-Research Ireland under Grant No.18/CRT/6223.
文摘Recent advancements have shown that control strategies using Deep Reinforcement Learning(DRL)can significantly improve the management of HVAC control and energy systems in buildings,leading to significant energy savings and better comfort.Unlike conventional rule-based controllers,they demand considerable time and data to develop effective policies.Transfer learning using pre-trained models can help address this issue.In this work,we use imitation learning(IL)as a method of pre-training and reinforcement learning(RL)for fine-tuning.However,HVAC systems can vary depending on the location,building size,structure,construction materials and weather conditions.The diversity in HVAC control systems across different buildings complicates the use of IL and RL.Neural network weights trained on the source building cannot be directly transferred to the target building because of differences in input features and the number of control equipment.To overcome this problem,we propose a novel padding method to ensure that both the source and target buildings share the same state space dimensionality.Thus,the trained neural network weights are transferable,and only the output layer must be adjusted to fit the dimensionality of the target action space.Additionally,we evaluate the performance of an existing padding technique for comparison.Our experiments show that the novel padding technique outperforms zero padding by 1.37%and training from scratch by 4.59%on average.
基金This research was funded by Australian Renewable Energy Agency(ARENA)as part of ARENA’s Advancing Renewables Program under Grant 2018/ARP017.
文摘This paper proposes a novel reinforcement learning(RL)architecture for the efficient scheduling and control of the heating,ventilation and air conditioning(HVAC)system in a commercial building while harnessing its de-mand response(DR)potentials.With advances in automated building management systems,this can be achieved seamlessly by a smart autonomous RL agent which takes the best action,for example,a change in HVAC temper-ature set point,necessary to change the electricity usage pattern of a building in response to demand response signals,and with minimal thermal comfort impact to customers.Previous research in this area has tackled only individual aspects of the problem using RL.Specifically,due to the challenges in implementing demand response with whole-building models,simpler analytical models which poorly capture reality have been used instead.And where whole-building models are applied,RL is used for HVAC control mainly to achieve energy efficiency goals while demand response is neglected.Thus,in this research,we implement a holistic framework by designing an efficient RL controller for a whole-building model which learns to optimise and control the HVAC system for improved energy efficiency and thermal comfort levels in addition to achieving demand response goals.Our simulation results show that by applying reinforcement learning for normal HVAC operation,a maximum weekly energy reduction of up to 22%can be achieved compared to a handcrafted baseline controller.Furthermore,by employing a DR-aware RL controller during demand response periods,average power reductions or increases of up to 50%can be achieved on a weekly basis compared to the default RL controller,while keeping occupant thermal comfort levels within acceptable bounds.
文摘We present the development of a bias compensating reinforcement learning(RL)algorithm that optimizes thermal comfort(by minimizing tracking error)and control utilization(by penalizing setpoint deviations)in a multi-zone heating,ventilation,and air-conditioning(HVAC)lab facility subject to unmeasurable disturbances and unknown dynamics.It is shown that the presence of unmeasurable disturbance results in an inconsistent learning equation in traditional RL controllers leading to parameter estimation bias(even with integral action support),and in the extreme case,the divergence of the learning algorithm.We demonstrate this issue by applying the popular Q-learning algorithm to linear quadratic regulation(LQR)of a multi-zone HVAC environment and showing that,even with integral support,the algorithm exhibits bias issue during the learning phase when the HVAC disturbance is unmeasurable due to unknown heat gains,occupancy variations,light sources,and outside weather changes.To address this difficulty,we present a bias compensating learning equation that learns a lumped bias term as a result of disturbances(and possibly other sources)in conjunction with the optimal control parameters.Experimental results show that the proposed scheme not only recovers the bias-free optimal control parameters but it does so without explicitly learning the dynamic model or estimating the disturbances,demonstrating the effectiveness of the algorithm in addressing the above challenges.
基金This work was supported by Young Scientists Fundamental Research Program of Shandong Province of China (No. 031B5147).
文摘In heating, ventilating and air-conditioning （HVAC） systems, there exist severe nonlinearity, time-varying nature, disturbances and uncertainties. A new predictive functional control based on Takagi-Sugeno （T-S） fuzzy model was proposed to control HVAC systems. The T-S fuzzy model of stabilized controlled process was obtained using the least squares method, then on the basis of global linear predictive model from T-S fuzzy model, the process was controlled by the predictive functional controller. Especially the feedback regulation part was developed to compensate uncertainties of fuzzy predictive model. Finally simulation test results in HVAC systems control applications showed that the proposed fuzzy model predictive functional control improves tracking effect and robustness. Compared with the conventional PID controller, this control strategy has the advantages of less overshoot and shorter setting time, etc.
文摘Heating, ventilation, and air conditioning (HVAC) system is significant to the energy efficiency in buildings. In this paper, temperature control of HVAC system is studied in winter operation season. The physical model of the zone, the fan, the heating coil and sensor are built. HVAC is a non-linear, strong disturbance and coupling system. Linear active-rejection-disturbance-control is an appreciate control algorithm which can adapt to less information, strong-disturbance influence, and has relative-fixed structure and simple tuning process of the controller parameters. Active-rejection-disturbance-control of the HVAC system is proposed. Simulation in Matlab/Simulink was done. Simulation results show that linear active-rejection-disturbance-control was prior to PID and integral-fuzzy controllers in rising time, overshoot and response time of step disturbance. The study can provide fundamental basis for the control of the air-condition system with strong-disturbance and high-precision needed.
基金supported by the National Nature Science Foundation of China(62073194)the Natural Science Foundation of Shandong Province of China(ZR2023MF028)the Taishan Scholars Program of Shandong Province(tsqn202312008)
文摘This paper proposes an event-triggered stochastic model predictive control for discrete-time linear time-invariant(LTI)systems under additive stochastic disturbances.It first constructs a probabilistic invariant set and a probabilistic reachable set based on the priori knowledge of system uncertainties.Assisted with enhanced robust tubes,the chance constraints are then formulated into a deterministic form.To alleviate the online computational burden,a novel event-triggered stochastic model predictive control is developed,where the triggering condition is designed based on the past and future optimal trajectory tracking errors in order to achieve a good trade-off between system resource utilization and control performance.Two triggering parametersσandγare used to adjust the frequency of solving the optimization problem.The probabilistic feasibility and stability of the system under the event-triggered mechanism are also examined.Finally,numerical studies on the control of a heating,ventilation,and air conditioning(HVAC)system confirm the efficacy of the proposed control.
文摘针对暖通空调(HVAC)系统难以控制的问题,提出一种基于m ax-product推理的M am dan i模糊模型预测控制策略.首先利用一步模糊预测模型的结构分析得到其解析表达式,获得系统在k+1时刻的线性化预测模型;然后基于模糊线性化模型进行模型预测控制器设计.对HVAC系统的仿真和实验结果表明,该算法是一种跟踪性能好且鲁棒性强的有效控制算法.
