摘要
讨论平均准则控制马氏链的强化学习算法.目的是寻找使得长期每阶段期望平均报酬最大的最优控制策略.由于事先未知状态转移矩阵及报酬向量,故必需使用自适应控制方法.通过引入称之为行动器和评判器的神经网络构造,使得学习单元在不断学习中,最终能发现最优策略.行动器的参数在学习中不断被修正,每一时刻的参数的值均对应着一个随机控制策略.
An average reward reinforcement learning algorithm for control Markov chains is presented.The objective is to find an optimal policy which maximizes the expected average reward per time step over infinite horizon.The transition matrices and payoff structures are not known a priori;so adaptive control methods are necessary.A neural networks structure,called actor and critic,is provided for the agent.The parameters of the actor,which determine a stochastic control strategy,are updated at each time step using a simple learning scheme.The adaptive critic is used to estimate these parameters for finding the optimal policy.
出处
《云南大学学报（自然科学版）》
CAS
CSCD
2000年第1期9-12,共4页
Journal of Yunnan University(Natural Sciences Edition)
关键词
强化学习
自适应评判
马氏链
控制问题
reinforcement learning
Markov decision processes
average reward
adaptive critic
R learning
