期刊文献+

基于Hadoop的网络分流和流特征计算 被引量:6

Diffluent Internet Traffic and Characteristics Computation Based on Hadoop
在线阅读 下载PDF
导出
摘要 网络流量特征计算是网络流量分析的一个重要步骤,对于海量网络流量数据,并行化计算网络流量特征是高效网络流量分析的重要方法。针对传统单机处理成本高、可扩展性差的问题,提出一种基于Map Reduce编程模型的网络流量分析方法,并行实现网络分流和流量特征计算。通过使用Hadoop平台对实际数据进行分析,统计常用网络流量属性特征,实验表明,该方法分析网络流量特征的结果准确可信,且适合分析大流量数据。 Internet traffic characteristics computation is an important step of internet traffic analysis, in the face of massive internet traffic data, parallel computing internet traffic characteristics is base to prompt the performance of internet traffic analysis. In order to solve the problems of poor expansibility and high cost that caused by the traditional stand-alone, an internet traffic characteristics analysis method based on MapReduce, which parallel processes diffluent internet traffic and characteristics computing, was proposed. By using the Hadoop platform, the actual data was analyzed and the common internet traffic characteristics were computed. Experiments show that the method is reliable to analyze the characteristic of the internet traffic and it is suitable for analyzing large traffic data.
出处 《电信科学》 北大核心 2014年第12期76-81,共6页 Telecommunications Science
基金 重庆市应用开发计划基金资助项目(No.cstc2013yykf A40006)
关键词 网络分流 特征 并行计算 MapReduce, diffluent internet traffic, characteristic, parallel computing
  • 相关文献

参考文献10

  • 1周爱平,程光,郭晓军.高速网络流量测量方法[J].软件学报,2014,25(1):135-153. 被引量:30
  • 2Li W,Moore A W.A machine learning approach for efficient traffic classification.Proceedings of the 15th International Symposium on Modeling,Analysis,and Simulation of Computer and Telecommunication Systems,Istanbul,Turkey,2007:310 -317.
  • 3Li M,Ye J H,Li J L,et al.The research on network traffic management based on Hash aggregation.Journal of Jiangxi Normal University(Natural Sciences Edition),2011,35(2):174-177.
  • 4Hadoop.http://hadoop.apache.org/,2014.
  • 5Lee Y,Kang W,Lee Y.A hadoop-based packet trace processing tool.Proceedings of the 3th International Conference on Traffic Monitoring and Analysis,Vienna,Austria,2011:51-63.
  • 6Qiao Y,Lei Z,Yuan L,et al.Offline traffic analysis system based on Hadoop.Journal of China Universities of Posts and Telecommunications,2013,20(5):97-103.
  • 7杨新宇,杨树森,李娟.基于非线性预处理网络流量预测方法的泛洪型DDoS攻击检测算法[J].计算机学报,2011,34(2):395-405. 被引量:19
  • 8何高峰,杨明,罗军舟,张璐.Tor匿名通信流量在线识别方法[J].软件学报,2013,24(3):540-556. 被引量:35
  • 9Liu Z X,He M B,Liu S,et al.Research of P2P traffic comprehensive identification method.Proceedings of International Conference on Network Computing and Information Security,Guilin,China,2011:307-310.
  • 10Thiriet M J M.Classifying application flows and intrusion detection in internet traffic.Doctoral Dissertation University of Science and Technology,2013.

二级参考文献23

  • 1郑军,胡铭曾,云晓春,郑仲.基于数据流方法的大规模网络异常发现[J].通信学报,2006,27(2):1-8. 被引量:17
  • 2任勋益,王汝传,王海艳.基于自相似检测DDoS攻击的小波分析方法[J].通信学报,2006,27(5):6-11. 被引量:56
  • 3李金明,王汝传.基于VTP方法的DDoS攻击实时检测技术研究[J].电子学报,2007,35(4):791-796. 被引量:18
  • 4Nash D A,Ragsdale D J.Simulation of self-similarity in network utilization patterns as a precursor to automated testing of intrusion detection systems.IEEE Transactions on Systems,Man and Cybernetics,Part A,2001,31(4):327-331.
  • 5Yang Xin-Yu,Liu Yong,Zeng Ming,Shi Yi.A novel DDoS attack detecting algorithm based on the continuous wavelet transform//Proceedings of the Advanced Workshop on Content Computing.Zhenjiang,China,2004,3309:173-181.
  • 6Shu Yang-Tai,Jin Zhi-Gang,Zhang Lian-Fang,Wang Lei.Traffic prediction using FARIMA models//Proceedings of the IEEE International Conference on Communications.Vancouver,Canada,1999,2:891-895.
  • 7Sadek N,Khotanzad A.Multi-scale high-speed network traffic prediction using k-factor Gegenbauer ARMA model//Pro-ceedings of the IEEE International Conference on Communications.Paris,France,2004,4:2148-2152.
  • 8Ge Xiao-Hu,Zhu Guang-Xi,Zhu Yao-Ting.On the testing for alpha-stable distributions of network traffic.Computer Communications,2004,27(5):447-457.
  • 9Leland W E,Taqqu M S,Willinger W,Wilson D V.On the self-similar nature of ethernet traffic.IEEE/ACM Transactions on Networking,1994,2(1):1-15.
  • 10Gennandy Samorodnitsky.Long Memory and Self-Similar Processes.Universite Paul Sabatier.Annales de la Faculte des Sciences de Toulouse.[OL].Available:http://afst.cedram.org/item? id=AFST_2006_6_15_1_107_0,2006,XV (1):107-123.

共引文献80

同被引文献54

  • 1Callado A, Kamienski C. A survey on internet traffic identification[J]. IEEE Communications Surveys and Tutorials, 2009, 11(3): 37-52.
  • 2Nguyen T T T, Armitage G. A survey of techniques for internet traffic classification using machine learning[J]. IEEE Communications Surveys and Tutorials, 2008, 10(4): 56-76.
  • 3Pentreath N, Machine Learning with Spark[M]. [S.1.]: Packt Publishing, 2014.
  • 4M Zaharia, Das T, Li H, et al. Discretized streams: fault- tolerant streaming computation at scale[C]. The 24th ACM Symposium on Operating Systems Principles, Farmington, PA, 2013, 423-438.
  • 5Li Jun, Zhang Shunyi, Lu Yanqing, et al. Real-time P2P traffic identificafion[C]. IEEE Global Telecommunications Conference, New Orleans, LA, 2008, 2474-2478.
  • 6Li Zhu, Yuan Ruixi, Guan Xiaohong. Traffic classification- towards accurate real time network applications[J]. Lecture Notes in Computer Science, 2007, 4553: 67-76.
  • 7Li W, Abdin K, Dann R, et al. Approaching real-time network traffic classification, RR-06-12[R]. London: Queen Mary University of London, 2006.
  • 8Tavallaee M, Lu W, Ghorbani A A. Online classification of network flows[C]. The 7th Communication Networks and Services Research Conference, Moncton, NB, 2009, 78-85.
  • 9乔嫒媛.基于Hadoop的网络流量分析系统的研究与应用[D].北京:北京邮电大学,2014.
  • 10Yu Lei, data: a Liu Huan. Feature selection for high-dimensional fast correlation-based filter solution[C]. The 20th International Conference on Machine Learning, Washington, DC, 2003, 856-863.

引证文献6

二级引证文献26

相关作者

内容加载中请稍等...

相关机构

内容加载中请稍等...

相关主题

内容加载中请稍等...

浏览历史

内容加载中请稍等...
;
使用帮助 返回顶部