摘要
中国手指语的识别使听力障碍人群与听力正常人群相互交流和人机交互更加便捷.传统的手指语识别对环境背景要求较高,为了解决复杂环境下中国手指语的识别问题,构建并扩展了手指语图片训练集,提出基于改进的区域的全卷积网络R-FCN解决复杂背景下的手指语识别任务.为适应多变的复杂场景,利用在线难例挖掘技术对手指语识别过程中产生的难例进行在线学习,结合手指语特征对网络进行优化.并与SVM分类和FasterR-CNN神经网络作对比.实验结果表明,基于改进的R-FCN在复杂环境的手语识别任务上能达到较好的识别效果.
Chinese finger-language is widely used among deaf-mute people,which makes their communication with normal people more convenient.Finger-language occupies an important position in sign language recognition and is also widely used in many fields such as human-computer interaction.Traditional finger-language recognition is highly susceptible to environmental background.To solve the recognition problem of Chinese finger-language in complex environment,the finger-language picture training set was constructed and expanded,the region-based fully convolutional networks(R-FCN) was improved and was used to recognize finger-language.To adapt to the changing complex scene,the online hard-example mining technology was used to learn the hard-examples generated during the finger-language recognition,and the network was optimized using the feature of the finger-language.In this paper,R-FCN is compared with multi-classifier based on SVM and Faster R-CNN neural network,experimental results show that the improved R-FCN can achieve better recognition effect in the task of sign language recognition in complex environment.
作者
周雷
阿里甫·库尔班
吕情深
孟欣欣
ZHOU Lei;Alifu Kuerban;LV Qingshen;MENG Xinxin(School of Software,Xinjiang University,Urumqi Xinjiang 830008,China;School of Information Science and Engineering,Urumqi Xinjiang 830008,China)
出处
《新疆大学学报（自然科学版）》
CAS
2020年第2期170-176,共7页
Journal of Xinjiang University(Natural Science Edition)
基金
国家自然科学基金资助项目金项目名称(61562084).
关键词
手指语识别
R-FCN
手语
在线难例挖掘
finger language recognition
R-FCN
sign language
online hard-examplemining
