摘要
新词发现一直是自然语言处理的热点问题和难点问题之一,为了提取博客语料的新词,首先对语料进行文中分词,然后根据新词的定义及新词的"时空"性质,在词串统计的基础上,通过词出现的频率,词分布的密度,上下文分析以及词在时间域上的变化分析对词进行过滤;最后通过词的统计信息和词性规则对候选词进行排序以提高准确率;在此基础上建立新词发现系统。
The discovery of new words has been one of the hot issues and difficult problems in natural language processing.In order to extract new words from blog contents,the segmentation of words in blog contents is conducted first,and then new words are filtered according to the definition,＂space-time＂ nature,statistics,appearance frequency and distribution density of the new words,as well as contextual analysis and the change of the new words in the time domain.The candidate words are sorted based on the word statistical information and part of speech（POS） rules to improve accuracy rate.By doing this,the new words discovery system was established.
出处
《现代电子技术》
2013年第2期144-146,149,共4页
Modern Electronics Technique
基金
漳州职业技术学院科研计划资助项目(ZZY1107)
关键词
新词
词串统计
上下文分析
分词
候选词
new word
word string statistics
contextual analysis
segmentation of word
candidate word
