Large language models(LLMs)exhibit remarkable capabilities in various natural language processing tasks,such as machine translation.However,the large number of LLM parameters incurs significant costs during inference....Large language models(LLMs)exhibit remarkable capabilities in various natural language processing tasks,such as machine translation.However,the large number of LLM parameters incurs significant costs during inference.Previous studies have attempted to train translation-tailored LLMs with moderately sized models by fine-tuning them on the translation data.Nevertheless,when performing translations in zero-shot directions that are absent from the fine-tuning data,the problem of ignoring instructions and thus producing translations in the wrong language(i.e.,the off-target translation issue)remains unresolved.In this work,we design a twostage fine-tuning algorithm to improve the instruction-following ability of translation-tailored LLMs,particularly for maintaining accurate translation directions.We first fine-tune LLMs on the translation data to elicit basic translation capabilities.At the second stage,we construct instruction-conficting samples by randomly replacing the instructions with the incorrect ones.Then,we introduce an extra unlikelihood loss to reduce the probability assigned to those samples.Experiments on two benchmarks using the LLaMA 2 and LLaMA 3 models,spanning 16 zero-shot directions,demonstrate that,compared to the competitive baseline translation-finetuned LLaMA,our method could effectively reduce the off-target translation ratio(up to-62.4 percentage points),thus improving translation quality(up to+9.7 bilingual evaluation understudy).Analysis shows that our method can preserve the model's performance on other tasks,such as supervised translation and general tasks.Code is released at https://github.com/alphadl/LanguageAware_Tuning.展开更多
基金Project supported by the National Natural Science Foundation of China(No.62372468)the Shandong Natural Science Foundation(No.ZR2023MF008)+1 种基金the Major Basic Research Projects in Shandong Province(No.ZR2023ZD32)the Qingdao Natural Science Foundation(No.23-2-1-161-zyyd-jch)。
文摘Large language models(LLMs)exhibit remarkable capabilities in various natural language processing tasks,such as machine translation.However,the large number of LLM parameters incurs significant costs during inference.Previous studies have attempted to train translation-tailored LLMs with moderately sized models by fine-tuning them on the translation data.Nevertheless,when performing translations in zero-shot directions that are absent from the fine-tuning data,the problem of ignoring instructions and thus producing translations in the wrong language(i.e.,the off-target translation issue)remains unresolved.In this work,we design a twostage fine-tuning algorithm to improve the instruction-following ability of translation-tailored LLMs,particularly for maintaining accurate translation directions.We first fine-tune LLMs on the translation data to elicit basic translation capabilities.At the second stage,we construct instruction-conficting samples by randomly replacing the instructions with the incorrect ones.Then,we introduce an extra unlikelihood loss to reduce the probability assigned to those samples.Experiments on two benchmarks using the LLaMA 2 and LLaMA 3 models,spanning 16 zero-shot directions,demonstrate that,compared to the competitive baseline translation-finetuned LLaMA,our method could effectively reduce the off-target translation ratio(up to-62.4 percentage points),thus improving translation quality(up to+9.7 bilingual evaluation understudy).Analysis shows that our method can preserve the model's performance on other tasks,such as supervised translation and general tasks.Code is released at https://github.com/alphadl/LanguageAware_Tuning.
