摘要
为满足计算密集型大数据应用的实时处理需求,在Apache Storm基础上,研究开发H-Storm异构计算平台。通过多进程服务特性设计图形处理器(GPU)资源的量化和分布式调用机制,进而提出H-Storm异构集群的任务调度策略,实现GPU性能及负载的任务调度算法与协同计算下自适应的流分发决策机制。实验结果表明,在512×512矩阵乘法用例下,与原生Storm平台相比,H-Storm异构计算平台吞吐量提升54.9倍,响应延时下降77倍。
To meet the real-time processing needs of com pute-intensive big data applications,H-Storm heterogeneous computing platform TS developed based on Apache Storm.Through the M ulti-process Service(MPS)feature,Graphic Process Unit(GPU)resource quantization and distributed calling mechanism are designed the task scheduling strategy of H-Storm heterogeneous clusters is proposed,and the task scheduling algorithm of GPU performance and load and adaptive flow distribution decision mechanism under cooperative computing are realized.Experimental results show that in the case of 512 x 512 matrix m ultiplication,the throughput of H-Storm heterogeneous computing platform increases by 5 4.9 times and the response delay decreases by 77 times compared with that of native Storm.
作者
严健康
陈更生
YAN Jiankang;CHEN Gengsheng(State Key Laboratory of ASIC and System,Fudan University,Shanghai 200433,China)
出处
《计算机工程》
CAS
CSCD
北大核心
2018年第4期1-11,共11页
Computer Engineering
基金
上海市科委科技创新行动计划项目(14511108002)
关键词
Storm平台
异构资源
调度算法
协同计算
JCuda库
多进程服务特征
Storm platform
heterogeneous resource
scheduling algorithm
Co-computing
JCuda library
Multiprocess Service(MPS)feature
