摘要
传统的微博话题发现方法主要利用词频进行词的特征提取,然后使用单一聚类(Single-pass)方法、主题模型Latent Dirichlet Allocation(LDA)进行文本聚类,往往忽略文本之间词与词潜在的语义相似关系。在微博这种短文本表现不佳。针对这一问题进行研究,提出一种基于词向量与关键词提取微博话题检测方法。首先通过连续词袋模型(CBOW)来提取微博文本的词向量特征,对每个文档的词向量求平均并归一化得到文档向量。然后通过聚类算法得到聚类结果,并将聚类的结果与通过图排序算法(TextRank)提取的文本关键词进行共现分析和筛选得到微博话题。在真实数据集上进行实验,该方案能有效提取微博话题,在准确率和轮廓系数(CH)上优于传统方法。
The traditional methods of microblog topic discovery mainly use term Frequency–inverse document frequency(TF-IDF)to extract the fea⁃tures of the words,and use the Single-pass method and the topic model Latent Dirichlet Allocation(LDA)for text clustering,which are of⁃ten ignoring the potential semantic similarity between words in the text.They are not performing well in the short text.Aiming at this prob⁃lem,this paper proposes a method of microblog topic detection based on word vectors and keyword extraction.Firstly,the word vector fea⁃ture of the microblog text is extracted by the continuous bag of words(CBOW)model of Word2Vec,and the word vectors of each document are averaged and normalized to obtain the document vectors.Then the clustering algorithms are used to gain the clustering result,the clus⁃tering result is co-occurring with the text keywords extracted by the TextRank graph sorting algorithm.Experiment conducted on real datas⁃et,this program can effectively extract microblog topics,which is superior to the traditional methods in accuracy and Silhouette Coefficient(CH)index.
作者
王立平
赵晖
WANG Li-ping;ZHAO Hui(Department of College of Information Science and Engineering,Xinjiang University,Urumqi 830046)
出处
《现代计算机》
2020年第23期3-9,共7页
Modern Computer
基金
国家自然科学基金(No.61561047)。
关键词
话题发现
词向量
聚类
图排序算法
关键词
Topic Discovery
Word Vectors
Cluster
Graph Sorting
Keywords
