摘要
词向量嵌入技术是研究自然语言的重要一步,通过向量化,将自然语言数字化,使自然语言能够被计算机识别和进行相关处理计算。基于Word2vec实现哈萨克文向量化,对哈萨克语机器翻译、文本分类和识别等领域研究具有重要支撑意义。本文将开源的科大讯飞哈萨克文语料数据集作为语料库,经过清洗、分词等步骤,用Word2vc实现向量化,将每一个哈萨克文词转换为一个独立的K位词向量,通过对词向量的计算,实现发现哈萨克文文本中包含的上下文语义规律、文本主题词提取、相似词计算等功能。
The word vector embedding technology is a crucial step in the study of natural language processing,which is digitized through vectorization so that natural language can be recognized by computers and relevant processing calculations.The implementation of Kazakh language vectorization based on Word2vec is important to support the research in the field of Kazakh language machine translation,text classification and recognition.In the article,the open-source iFLYTEK Kazakh corpus dataset is used as a corpus,and after cleaning,tokenization and other steps,vectorization is implemented to convert each Kazakh word into an independent K-bit word vector by using Word2vc tool.Through the computation of these word vectors,the discovery of the contextual semantic patterns contained in the Kazakh text,the extraction of the textual keywords,and the computation of the similar words can be achieved.
作者
吾塔嗯拜克·阿萨汗
亚森·艾则孜
阿依努尔·努尔太
Wutaenbaiek Asahan;Yasen Aizezi;Aynur Nurtai(People's P ublic S ecurity University of China,B eijing 100038,China;X injiang Police college,Urumqi 830013,China;Nanjing University of Posts and Telecommunications,Nanjing 210003,China)
出处
《数字通信世界》
2025年第5期148-149,166,共3页
Digital Communication World
基金
2024年新疆维吾尔自治区自然科学基金科研基金项目面向数据共享的安全电子数据存取方法研究研究成果之一(编号2024D01A55)
2024年度新疆警察学院校级科技创新团队无人机安全技术团队(编号XJKYKJTD202402)。
