摘要
基于深度学习的手语翻译(SLT)旨在使用深度学习方法对手语动作进行翻译,以提高翻译准确性。SLT降低了正常人与听障人士的沟通门槛,但由于各国手语不统一以及手语动作与口语句子的结构不匹配等问题,手语翻译面临诸多挑战。随着深度学习技术的发展,基于深度学习的手语翻译被研究人员广泛关注。通过对近年来基于深度学习的手语翻译方法进行总结,并依照模型结构与发展历程将其分为基于线性结构的手语翻译、基于编码器-解码器的手语翻译、基于大模型微调的手语翻译和基于对比学习的手语翻译四类。通过对各类方法的特点与性能进行分析,并为手语翻译方法的进展提供了全面的评估。最后,展望了未来研究的方向,重点讨论了实时翻译、基于对比学习的手语翻译和基于大模型微调的手语翻译等关键技术的潜力和发展趋势。
SLT based on deep learning aims to translate sign language gestures into natural language using deep learning techniques to improve translation’s accuracy.SLT reduces communication barriers between normal hearing individuals and those with hearing impairments.However,SLT faces numerous challenges due to the lack of standardization across different sign languages and the mismatch between sign language gestures and spoken language sentence structures.With the development of deep learning technologies,SLT has gained widespread attention from researchers.This paper summarized recent approaches on SLT based on deep learning and classified them into four categories according to model structure and development history:linear structure-based SLT,encoder-decoder architecture-based SLT,large model fine-tuning-based SLT,and contrastive learning-based SLT.By analyzing the characteristics and performance of these methods,this study provided a comprehensive evaluation of the progress in SLT methods.Finally,the paper outlined future research directions,focusing on the potential and development trends of key technologies,including real-time translation,contrastive learning-based SLT,and large model fine-tuning-based SLT.
作者
张磊
王振宇
连帅帅
蒲冰倩
刘毓涛
秦铭哲
Zhang Lei;Wang Zhenyu;Lian Shuaishuai;Pu Bingqian;Liu Yutao;Qin Mingzhe(School of Information&Electronic Technology,Jiamusi University,Jiamusi Heilongjiang 154007,China;Heilongjiang Province Key Laboratory of Autonomous Intelligence&Information Processing,Jiamusi University,Jiamusi Heilongjiang 154007,China;Jiamusi Key Laboratory of Satellite Navigation Technology&Equipment Engineering Technology,Jiamusi University,Jiamusi Heilongjiang 154007,China;Handan Polytechnic College,Handan Hebei 056046,China)
出处
《计算机应用研究》
北大核心
2025年第8期2241-2254,共14页
Application Research of Computers
基金
黑龙江省自然科学基金联合基金培育项目(PL2024F002)
黑龙江省省属高等学校基本科研业务费优秀创新团队建设项目(2022-KYYWF-0654)
佳木斯大学国家基金培育项目(JMSUGPZR2022-014)
黑龙江省自主智能与信息处理重点实验室开放课题(ZZXC202302)
佳木斯大学“东极”学术团队资助项目(DJXSTD202417)
黑龙江省省属本科高校优秀青年教师基础研究支持计划资助项目(YQJH2024239)
黑龙江省外国专家项目(G2024020)
佳木斯大学博士专项科研启动项目(JMSUBZ2024-07)
大学生创新创业计划资助项目(S202310222013)。
关键词
深度学习
手语翻译
机器翻译
对比学习
大语言模型
编码器-解码器
deep learning
sign language translation(SLT)
machine translation
contrastive learning
large language mo-dels
encoder-decoder
