摘要
通过深入研究日志的类型和特点,设计并实现了一套基于并行计算的海量日志文件分析系统。该系统采用集群方式并行地收集日志文件,采用分布式文件系统存储,最终利用并行计算对日志进行分析处理。该系统实现了日志采集、分析的完全自动化处理,在系统部署之后能够有效地进行系统安全的维护、系统性能的优化、系统故障的排查。该系统结合云计算提高了日志分析的效率,解决了海量日志处理过程中存在的问题,为海量日志分析提供了一个完整有效的解决方案。
On the basis of analyzing log type and features deeply, design and implement a massive log processing system based on parallel computing. It adopts the method of cluster to collect log in parallel way, store in the distributed file system, and analyze log by parallel computing. The system achieves log collection and analysis through automated processing, can effectively carry on security maintenance, system performance optimization, system failure check after the system deployment. The system combines distributed and cloud computing solutions to improve efficiency of log processing, solves the major pwblems of massive logs processing effectively, provides a complete and effective solutions for massive logs processing.
出处
《计算机技术与发展》
2013年第7期80-83,共4页
Computer Technology and Development
基金
安徽省科技攻关项目(07010200057)
关键词
分布式计算
分布式文件系统
云计算
日志处理
distributed computing
distributed file system
cloud computing
log processing
