摘要
针对现有多语言模型在预训练过程中对多语言数据集的利用效率低,导致跨语言上下文学习能力不足,进而产生语言偏差的问题,提出一种基于交替语言数据重构方法的跨语言文本相似度模型.该方法通过对称地替换平行语料中的中英文词语,形成重构的预训练文本对,并利用上述文本对对多语言大模型mBERT(BERT-based-multilingual)进行基于数据重构的针对性预训练和微调处理.为验证该模型的可行性,在联合国平行语料数据集上进行实验,实验结果表明,该模型的相似度查准率优于mBERT和其他两种基线模型,其不仅可以进一步提高跨语言信息检索的准确性,并且可以降低多语言自然语言处理任务的研究成本.
Aiming at the problem that existing multilingual models were inefficient in utilising multilingual datasets in the pre-training process,which led to a more insufficient cross-language contextual learning ability and thus language bias,we proposed a cross-language text similarity model based on the alternating language data reconstruction method.This method formed reconstructed pre-trained text pairs by symmetrically replacing Chinese and English words in the parallel corpus,and used the above text pairs to perform targeted pre-training and fine-tuning processing based on data reconstruction for the multilingual large model mBERT(BERT-based-multilingual).In order to verify the feasibility of the model,experiments were conducted on the United Nations parallel corpus da taset,and the experimental results show that the similarity checking accuracy of this model outperforms that of mBERT and the other two baseline models.It can not only further improve the accuracy of cross-language information retrieval,but also reduce the research cost of multilingual natural language processing tasks.
作者
王轶
王坤宁
刘铭
WANG Yi;WANG Kunning;LIU Ming(School of Foreign Languages,Changchun University of Technology,Changchun 130012,China;School of Mathematics and Statis tics,Changchun University of Technology,Changchun 130012,China)
出处
《吉林大学学报(理学版)》
北大核心
2025年第2期551-558,共8页
Journal of Jilin University:Science Edition
基金
吉林省高等教育教学改革研究课题项目(批准号:20213F2VB6S006)
吉林省发改委基本建设基金(批准号:2022C043-2)
吉林省自然科学基金(批准号:20200201157JC)
吉林省教育厅人文社科研究项目(批准号:JJKH20240835SK)
吉林省高教科研课题项目(批准号:JGJX24C051)。
