摘要
Consistency identification in task-oriented dialogue(CI-ToD)can prevent inconsistent dialogue response generation,which has recently emerged as an important and growing research area.This paper takes the first step to explore a pre-training paradigm for CI-ToD.Nevertheless,pre-training for CI-ToD is non-trivial because it requires a large amount of multi-turn KB-grounded dialogues,which are extremely hard to collect.To alleviate the data scarcity problem for pre-training,we introduce a modularized pre-training framework(MPFToD),which is capable of utilizing large amounts of KB-free dialogues.Specifically,such modularization allows us to decouple CI-ToD into three sub-modules and propose three pre-training tasks including(i)query response matching pre-training;(ii)dialogue history consistent identification pre-training;and(iii)KB mask language modeling to enhance different abilities of CI-ToD model.As different sub-tasks are solved separately,MPFToD can learn from large amounts of KB-free dialogues for different modules,which are much easier to obtain.Results on the CI-ToD benchmark show that MPFToD pushes the state-of-the-art performance from 56.3%to 61.0%.Furthermore,we show its transferability with promising performance on other downstream tasks(i.e.,dialog act recognition,sentiment classification and table fact checking).
基金
supported by the National Natural Science Foundation of China(NSFC)(Grant Nos.62306342,62176076)
the Excellent Young Scientists Fund in Hunan Province(2024JJ4070)
supported by the Natural Science Foundation of Guangdong(2023A1515012922)
Shenzhen Foundational Research Funding(JCYJ20220818102415032)
The Major Key Project of PCL(PCL2023A09)
Guangdong Provincial Key Laboratory of Novel Security Intelligence Technologies(2022B1212010005k).
