摘要
传统的信息处理技术已难以满足现代数据处理的要求。因此,文中针对云计算时代数据处理计算量大、实时性要求高的特点,研究了基于Hadoop技术的大数据分析应用系统。该系统以Hadoop技术为基本框架,采用HDFS系统作为数据存储的基本单元。并在此基础上,利用Hive方式建立新的数据库系统,结合MySQL数据管理系统和Kylin进行数据分析系统完成对数据的获取、存储、分析、管理功能。最后,通过大数据可视化系统和大数据报表分析系统完成对数据分析结果的呈现。本研究内容对实现大数据的管理和分析具有重要意义。
Traditional information processing technology has been difficult to meet the requirements of modern data processing. Therefore,in view of the characteristics of large amount of data processing calculation and high realtime requirement in the era of cloud computing,this paper studies the application system of large data analysis based on Hadoop technology. Based on Hadoop technology and HDFS system as the basic unit of data storage,the system establishes a new database system by Hive mode,completes data acquisition,storage,analysis and management functions with MySQL data management system and Kylin data analysis system,and finally completes data visualization system and large data report analysis system. Presentation of paired data analysis results. The content of this paper is of great significance to the management and analysis of large data.
作者
鲁志芳
LU Zhi-fang(Shanghai Donghai Vocational and Technical College,Shanghai 200241,China)
出处
《电子设计工程》
2019年第16期11-14,19,共5页
Electronic Design Engineering
