摘要
在Hadoop的电子政务平台中针对存储成千上万的用户上传文件,利用HDFS提高文件的存储效率。设计了一个在云存储过程中小文件上传存储的处理方案。设定临时容器来暂存上传的小文件,使用定时器或临时容器满为临界条件进行文件的合并组合。通过增加NFS服务器和分离Name Node节点的职责,建立相关子类来存储和云同步服务信息,降低客户端访问数据的压力。设计一个简单的预测系统负荷的算法,使文件的合并组合在系统负荷较小时进行,减少等待时间,提高效率。通过实验表明合理的设置临时容器的大小和增加NFS服务器,在系统负荷低时能有效地提高HDFS存储小文件的效率。
This paper improves storage efficiency by HDFS in the Hadoop e-government platform for thousands of files uploaded by users.The cloud storage and file upload storage solutions are designed.It sets up temporary container to store the small files uploaded,and uses the timer or a temporary container to combine the file with the critical conditions.By increasing the NFS server and the NameNode node of the separation of duties,it establishes the relevant subclasses to storage and cloud synchronization service information and reduces the pressure of the client access data.It designs a simple prediction system load algorithm and the file of the combined group in the system load is small,which reduces the waiting time and improve efficiency.Experimental results show that the reasonable setting temporary container size and increase the efficiency of the NFS server,can effectively improve the HDFS storage ofsmall files in the system when the load is low.
出处
《微型电脑应用》
2015年第8期76-79,6,共4页
Microcomputer Applications
基金
四川省教育厅项目(15ZB0324)
