摘要
为解决传统大数据存储方法存在的存储响应时间长、数据聚类效果差的问题,提出云计算下分布式大数据智能存储算法。通过分析分布式大数据序列,得出归一化的RGB直方图计算大数据直方图的绝对差,映射数据序列的变化,以此对分布式大数据进行分类,再使用K-means算法选择聚类的中心,并拟定聚类数量和标准的测评阈值对数据进行聚类,把所有聚类集合内相应的数据通过长短缓存区将其补全,随后以数据流的形式存储到标签中使其组成完整的分布式大数据文件,从而达到对分布式大数据智能存储的目的。仿真结果证明,研究方法的大数据存储过程耗时更短,且不易受到外界异常干扰,有效增强了大数据的聚类效果,从而提高了分布式大数据的智能存储性能。
Due to long storage response time and poor data clustering effect in traditional methods, this article focuses on an algorithm of distributed big data intelligent storage based on cloud computing. By analyzing the distributed big data sequence, we found the normalized RGB histogram and calculated the absolute difference of big data histogram, so that the changes of data sequence were mapped. On this basis, we classified the distributed big data. Then, we used K-means algorithm to select the clustering center. Meanwhile, we adopted the evaluation threshold of clustering quantity and clustering standard to cluster the data, and complemented all the data in clustering set through the long and short data buffer cache. In the form of data stream, we stored it in the label to form a complete distributed big data file. Finally, we achieved the intelligent storage of distributed big data. Simulation results show that the proposed method has less time consumption in big data storage, so it is not easy to be interfered by external anomalies. Meanwhile, this method effectively enhances the clustering effect of big data, thus improving the intelligent storage performance for distributed big data.
作者
李劭
黄诚
LI Shao;HUANG Cheng(College of Network Space Safety,Sichuan University,Chengdu Sichuan 610227,China)
出处
《计算机仿真》
北大核心
2020年第5期443-447,共5页
Computer Simulation
基金
四川省现代教育技术研究课题(2017-R-54131)。
关键词
分布式大数据
智能存储
云计算
绝对差
Distributed big data
Intelligent storage
Cloud computing
Absolute difference
