This paper develops a multi-timescale coordinated operation method for microgrids based on modern deep rein-forcement learning.Considering the complementary characteristics of different storage devices,the proposed ap...This paper develops a multi-timescale coordinated operation method for microgrids based on modern deep rein-forcement learning.Considering the complementary characteristics of different storage devices,the proposed approach achieves multi-timescale coordination of battery and supercapacitor by introducing a hierarchical two-stage dispatch model.The first stage makes an initial decision irrespective of the uncertainties using the hourly predicted data to minimize the operational cost.For the second stage,it aims to generate corrective actions for the first-stage decisions to compensate for real-time renewable generation fluctuations.The first stage is formulated as a non-convex deterministic optimization problem,while the second stage is modeled as a Markov decision process solved by an entropy-regularized deep reinforcement learning method,i.e.,the Soft Actor-Critic.The Soft Actor-Critic method can efficiently address the exploration-exploitation dilemma and suppress variations.This improves the robustness of decisions.Simulation results demonstrate that different types of energy storage devices can be used at two stages to achieve the multi-timescale coordinated operation.This proves the effectiveness of the proposed method.展开更多
基金supported by Guangdong Provincial Key Laboratory of New Technology for Smart Grid Funded Project under Grant No.2020b1212070025.
文摘This paper develops a multi-timescale coordinated operation method for microgrids based on modern deep rein-forcement learning.Considering the complementary characteristics of different storage devices,the proposed approach achieves multi-timescale coordination of battery and supercapacitor by introducing a hierarchical two-stage dispatch model.The first stage makes an initial decision irrespective of the uncertainties using the hourly predicted data to minimize the operational cost.For the second stage,it aims to generate corrective actions for the first-stage decisions to compensate for real-time renewable generation fluctuations.The first stage is formulated as a non-convex deterministic optimization problem,while the second stage is modeled as a Markov decision process solved by an entropy-regularized deep reinforcement learning method,i.e.,the Soft Actor-Critic.The Soft Actor-Critic method can efficiently address the exploration-exploitation dilemma and suppress variations.This improves the robustness of decisions.Simulation results demonstrate that different types of energy storage devices can be used at two stages to achieve the multi-timescale coordinated operation.This proves the effectiveness of the proposed method.
