摘要
针对柔性作业车间中工件、AGV和机器的联合调度问题,以完工时间最小化为优化目标,提出了一种基于双重深度Q网络的分布式多智能体强化学习(DMA-DDQN)方法。创建了3类智能体,即工件分配智能体、AGV调度智能体及工序选择智能体,分别解决工件分配、AGV选择以及机器工序选择3类调度子问题。首先,双重深度Q网络(DDQN)算法用于训练3类智能体,通过捕捉生产信息和调度目标之间的关系,做出调度决策;其次,针对3类智能体,分别设计了状态和动作表示,以实现更高效的决策。其中,在设计工件分配智能体时,引入了机器评价指数,用于解决车间规模扩大时状态空间产生的维度爆炸问题;在奖励函数设计时采用了替代奖励成形技术,以提高学习效率和调度效率。最后,为了验证所提方法中各类智能体在不同规模下的性能,与现有的启发式调度算法进行了对比;进一步,与复合启发式调度算法及现有调度算法相比,验证所提方法在不同规模下的优越性。
To solve the joint scheduling problem of workpiece,AGV and machine in flexible job shop,a method of Distributed Multi-Agent reinforcement learning based on Dual Deep Q Network(DMA-DDQN)was proposed by taking the minimum completion time as the objective.Three kinds of agents were created:job assignment agent,AGV scheduling agent and process selection agent,to solve three kinds of scheduling sub-problems respectively:job assignment,AGV selection and machine process selection.The Dual Deep Q Network(DDQN)algorithm was used to train three kinds of agents and make scheduling decisions by capturing the relationship between production information and scheduling targets.The state and action representations were designed for three types of agents to achieve more efficient decision making.The machine evaluation index was introduced to solve the problem of dimensional explosion in state space when the scale of workshop was enlarged.To improve learning efficiency and scheduling efficiency,alternative reward shaping technology was used in the design of reward function.Finally,compared with the existing heuristic scheduling algorithms,the performance of various agents in the proposed method under different scales was verified.Further,compared with the compound heuristic scheduling algorithm and the existing scheduling algorithm,the advantages of the proposed method under different scales were verified.
作者
孟繁威
郭宏
延小龙
武玉鑫
张德华
罗雷
MENG Fanwei;GUO Hong;YAN Xiaolong;WU Yuxin;ZHANG Dehua;LUO Lei(School of Mechanical Engineering,Taiyuan University of Science&Technology,Taiyuan 030024,China;Shanxi Pingyang Heavy Industry Machinery Co.,Ltd.,Linfen 043000,China)
出处
《计算机集成制造系统》
北大核心
2026年第3期813-830,共18页
Computer Integrated Manufacturing Systems
基金
山西省重点研发资助项目(202102150401009)。
关键词
智能车间
多智能体
联合调度
双重深度Q网络
intelligent workshop
multi-agents
joint scheduling
dual deep Q network
