Neural machine translation(NMT)has been widely applied to high-resource language pairs,but its dependence on large-scale data results in poor performance in low-resource scenarios.In this paper,we propose a transfer-l...Neural machine translation(NMT)has been widely applied to high-resource language pairs,but its dependence on large-scale data results in poor performance in low-resource scenarios.In this paper,we propose a transfer-learning-based approach called shared space transfer for zero-resource NMT.Our method leverages a pivot pre-trained language model(PLM)to create a shared representation space,which is used in both auxiliary source→pivot(Ms2p)and(Mp2t)translation models.Specifically,we exploit pivot PLM to initialize the Ms2p decoder pivot→targetand Mp2t encoder,while adopting a freezing strategy during the training process.We further propose a feature converter to mitigate representation space deviations by converting the features from the source encoder into the shared representation space.The converter is trained using the synthetic parallel corpus.The final Ms2t model source→targetcombines the Ms2p encoder,feature converter,and Mp2t decoder.We conduct simulation experiments using English as the pivot language for and translations.We finally test our method German→French,German→Czech,Turkish→Hindion a real zero-resource language pair,with Chinese as the pivot language.Experiment results Mongolian→Vietnameseshow that our method achieves high translation quality,with better Translation Error Rate(TER)and BLEU scores compared with other pivot-based methods.The step-wise pre-training with our feature converter outperforms baseline models in terms of COMET scores.展开更多
基金funded by the National Natural Science Foundation of China(Grant number:Nos.62172341 and 12204386)Sichuan Natural Science Foundation(Grant number:No.2024NSFSC1375)+1 种基金Youth Foundation of Inner Mongolia Natural Science Foundation(Grant number:No.2024QN06017)Basic Scientific Research Business Fee Project for Universities in Inner Mongolia(Grant number:No.0406082215).
文摘Neural machine translation(NMT)has been widely applied to high-resource language pairs,but its dependence on large-scale data results in poor performance in low-resource scenarios.In this paper,we propose a transfer-learning-based approach called shared space transfer for zero-resource NMT.Our method leverages a pivot pre-trained language model(PLM)to create a shared representation space,which is used in both auxiliary source→pivot(Ms2p)and(Mp2t)translation models.Specifically,we exploit pivot PLM to initialize the Ms2p decoder pivot→targetand Mp2t encoder,while adopting a freezing strategy during the training process.We further propose a feature converter to mitigate representation space deviations by converting the features from the source encoder into the shared representation space.The converter is trained using the synthetic parallel corpus.The final Ms2t model source→targetcombines the Ms2p encoder,feature converter,and Mp2t decoder.We conduct simulation experiments using English as the pivot language for and translations.We finally test our method German→French,German→Czech,Turkish→Hindion a real zero-resource language pair,with Chinese as the pivot language.Experiment results Mongolian→Vietnameseshow that our method achieves high translation quality,with better Translation Error Rate(TER)and BLEU scores compared with other pivot-based methods.The step-wise pre-training with our feature converter outperforms baseline models in terms of COMET scores.
