摘要
在当前信息化的年代里,文本数据在高速的增长,人们获取有用的信息犹如大海捞针.文本聚类作为文本挖掘的基础技术,发挥了很重要的作用.由于缺乏预先定义的类和类标号的训练实例,如何选择合适的数据相似度是文本聚类的关键问题.文章为此提出一种新的衡量文本相似度的方法 Adaptive Metric Selection(AMS).文章通过抓取网页内容,为聚类提供数据来源,分词和向量化是必要的转化,利用特征提取的方法获取特征项,并用Isomap进行降维,最后利用自适应选择方法 AMS对数据进行相似度衡量再进行聚类分析.实验结果表明,AMS明显优于从多种相似度独立进行聚类的平均结果.
In the current information age, text data grows at a high speed, and it is very hard for people to get useful information from huge data, which is like looking for a needle in a haystack. As the basic method in text mining, text clustering plays a very important role. Without predefined training set, it is one of the most important questions in text mining to select the suitable metric for different text data. Thus, in this thesis, we propose one novel Adaptive Metric Selection （AMS） method. The pipeline of our working includes ： （1） crawling the webpage content to prepare the data source for clustering; （2） transforming the content to separate words and then to a vector form; （3） Extracting features; （4） Reducing dimension using Isomap; and （5） Using an adaptive selection method AMS to evaluate data similarity. K means is used as the basic clustering algorithm, and we use two popular clustering quality measures to evaluate the final results： （1） Adjusted Rand Index （ARI）, and （2） Normal- ized Mutual Information （NMI）. Simulation results show the effectiveness of our proposed methods compared to the averaged
出处
《广州大学学报（自然科学版）》
CAS
2016年第1期80-89,共10页
Journal of Guangzhou University:Natural Science Edition
关键词
数据挖掘
特征提取
聚类融合
results of different metrics. data mining
feature extraction
clustering ensembles
