期刊文献+

分布式存储中一种新的低修复带宽的Hitchhiker码 被引量:2

Novel Hitchhiker with Lower Bandwidth in Distributed Storage
在线阅读 下载PDF
导出
摘要 为了高效可靠的存储海量数据,分布式存储系统常利用纠删码来降低存储开销.Hitchhiker码是Piggybacking架构下易于工程实现的双条带MDS(Maximum Distance Separable Code)码,具有参数(k,r)取值任意、修复成本较低等特征.然而,目前Hitchhiker码只优化了数据单元的修复带宽,未优化校验单元的修复带宽.针对此问题,本文提出了利用LRC(Locally Repairable Code)的思想同时优化数据单元和校验单元的编码(Hitchhiker-LRC和Hitchhiker-LRC+).该方法是对第一个子条带中l个校验求局部校验,将其存放在第一个子条带的某个校验上,要求该校验的数据已通过局部校验的形式捎带在了第二个子条带的后r-1个校验中,并且对该校验单元做了横向减法.最后,理论和实验证明,Hitchhiker-LRC和Hitchhiker-LRC+这两种编码在2≤r<k/2时可降低1%~5%修复带宽和节省约10%的修复时间,在k/2≤r<k时,Hitchhiker-LRC+在r较大时,相比Hitchhiker-LRC具有更低的修复带宽;并存在l使得修复带宽率达到最优. In order to store big data efficiently and reliably,Erasure-Codes are often used to reduce storage overhead in distributed storage systems.Hitchhiker code is a double stripes coding easy to be implemented in Piggybacking architecture.It is an MDS code with arbitrary parameters and low repair cost.How ever,at present,Hitchhiker code only optimizes the data unit,while the repair of the parity unit is not improved.To solve this problem,this paper proposes two new codes(Hitchhiker-LRC and Hitchhiker-LRC+)which use LRC.They can optimize the bandwidth of repair data and parity unit simultaneously.The method is to obtain a local parity for l parity in the first substripe and store it on a parity in the first substripe.It requires that the data of the parity has been pigmented on the later parity in the second substripe by means of local parity,and the horizontal subtraction is made for the parity unit.Finally,the theory combined with the experiment proves that Hitchhiker-LRC and Hitchhiker-LRC+can reduce the repair bandw idth by 1%~5%and save about 10%of repair time at 2≤r<k/2.Hitchhiker-LRC+have lower repair bandw idth than Hitchhiker-LRC at k/2≤r<k.And there is a l can make the repair bandw idth rate reach the optimal.
作者 胡金平 李贵洋 江小玉 周悦 韩鸿宇 HU Jin-ping;LI Gui-yang;JIANG Xiao-yu;ZHOU Yue;HAN Hong-yu(Department of Computer Science,Sichuan Normal University,Chengdu 610101,China)
出处 《小型微型计算机系统》 CSCD 北大核心 2020年第7期1559-1568,共10页 Journal of Chinese Computer Systems
基金 国家自然科学基金项目(61701331)资助。
关键词 MDS码 RS码 Piggybacking码 LRC编码 Hitchhiker码 Maximum Distance Separable Reed-Solomon codes Piggybacking Locally Repairable codes Hitchhiker
  • 相关文献

参考文献2

二级参考文献22

  • 1周松,王意洁.EXPyramid:一种灵活的基于阵列结构的高容错低修复成本编码方案[J].计算机研究与发展,2011,48(S1):30-36. 被引量:5
  • 2万武南,吴震,陈运,王晓京.一种基于3容错阵列码的RAID数据布局[J].计算机学报,2007,30(10):1721-1730. 被引量:18
  • 3LUO J Q, MOCHAN S, XU L H, et al. Efficient encoding schedules for XOR-based erasure codes[J]. IEEE Transactions on Computers, 2014, 63(9): 2259-2272.
  • 4LI M, SHU J. On cyclic lowest density MDS array codes constructed using starters[J]. IEEE International Symposium on Information Theory, 2010, 41(3): 1315-1319.
  • 5KVASHENNIKOV V V. Application of fast polynoamial transforma-tions over GALOIS GF(2m) fields in Reed-Solomon coding and de-coding[J]. Telecommunications and Radio Engineering, 2012, 71(10): 85-90.
  • 6BURGISSER P, CLAUSEN M, SHOKROLLAHI MA. Algebraic complexity theory[M]. Springer Verlag Heidelberg. 1996.
  • 7LACAN J, FIMES J. Systematic MDS erasure codes based on Van-dermonde matrices[J]. IEEE Communications Letters, 2004, 8(9): 570-582.
  • 8PLANK J S, XU L.Optimizing cauchy Reed-Solomon codes for fault-tolerant network storage applications[C]//The 5th IEEE Interna-tional Symposium on Network Computing and Applications (IEEE NCA06). Cambridge, MA, 2006: 1-8.
  • 9KALCHER S, LINDENSTRUTH V. Accelerating Galois field arith-metic for Reed-Solomon erasure codes in storage applications[C]// IEEE International Conference on Cluster Computing. 2011: 290-298.
  • 10KHAN O, BURNS R, PLANK J S. Rethinking erasure codes for cloud file systems: minimizing I/O for recovery and degraded reads[C]// USENIX. FAST 2012: 10th USENIX Conference on File and Storage Technologies. San Jose, CA, 2012: 1-14.

共引文献60

同被引文献10

引证文献2

二级引证文献4

相关作者

内容加载中请稍等...

相关机构

内容加载中请稍等...

相关主题

内容加载中请稍等...

浏览历史

内容加载中请稍等...
;
使用帮助 返回顶部