内存计算框架的低延迟特性大幅提高了集群的计算效率,但Shuffle过程的性能瓶颈仍不可规避.宽依赖的同步操作导致大多数工作节点等待慢节点的计算结果,同步过程不仅浪费计算资源,更增加了作业延时,这一现象在异构集群环境下尤为突出.针...内存计算框架的低延迟特性大幅提高了集群的计算效率,但Shuffle过程的性能瓶颈仍不可规避.宽依赖的同步操作导致大多数工作节点等待慢节点的计算结果,同步过程不仅浪费计算资源,更增加了作业延时,这一现象在异构集群环境下尤为突出.针对内存计算框架Shuffle操作的同步问题,建立了资源需求模型、执行效率模型和任务分配及调度模型.给出了分配效能熵(allocation efficiency entropy,AEE)和节点贡献度(worker contribution degree,WCD)的定义,提出了算法的优化目标.根据模型的相关定义求解,设计了局部数据优先拉取算法(partial data shuffled first algorithm,PDSF),通过高效节点优先调度,提高流水线与宽依赖任务的时间重合度,减少宽依赖Shuffle过程的同步延时,优化集群资源利用率;通过适度倾斜的任务分配,在保障慢节点计算连续性的前提下,提高分配任务量与节点计算能力的适应度,优化作业执行效率;通过分析算法的相关优化原则,证明了算法的帕累托最优性.实验表明:PDSF算法提高了内存计算框架的作业执行效率,并使集群资源得到有效利用.展开更多
Microfiltration technique was used to measure white blood cell deformability (WCD) in one hundred and thirty patients with non-nsulin-dependent diabetes mellitus (NIDDM or type Ⅱ). It was found that WCD of the diabet...Microfiltration technique was used to measure white blood cell deformability (WCD) in one hundred and thirty patients with non-nsulin-dependent diabetes mellitus (NIDDM or type Ⅱ). It was found that WCD of the diabetics was significantly decreased compared with fifty controls (P<0.01) and further reduced with the rise of levels of plasma fibrinogen (Fg), plasma lipid, hemoglobin Ale (HbAlc) and fasting blood glucose (FBG), and also with the increase of age, the duration of diabetes, and the reduction of plasma magnesium concentration. It was shown that the decrease of WCD in diabetes mellitus was closely related to the degree of metabolic disturbance of the failure of diabetic control.展开更多
文摘内存计算框架的低延迟特性大幅提高了集群的计算效率,但Shuffle过程的性能瓶颈仍不可规避.宽依赖的同步操作导致大多数工作节点等待慢节点的计算结果,同步过程不仅浪费计算资源,更增加了作业延时,这一现象在异构集群环境下尤为突出.针对内存计算框架Shuffle操作的同步问题,建立了资源需求模型、执行效率模型和任务分配及调度模型.给出了分配效能熵(allocation efficiency entropy,AEE)和节点贡献度(worker contribution degree,WCD)的定义,提出了算法的优化目标.根据模型的相关定义求解,设计了局部数据优先拉取算法(partial data shuffled first algorithm,PDSF),通过高效节点优先调度,提高流水线与宽依赖任务的时间重合度,减少宽依赖Shuffle过程的同步延时,优化集群资源利用率;通过适度倾斜的任务分配,在保障慢节点计算连续性的前提下,提高分配任务量与节点计算能力的适应度,优化作业执行效率;通过分析算法的相关优化原则,证明了算法的帕累托最优性.实验表明:PDSF算法提高了内存计算框架的作业执行效率,并使集群资源得到有效利用.
文摘Microfiltration technique was used to measure white blood cell deformability (WCD) in one hundred and thirty patients with non-nsulin-dependent diabetes mellitus (NIDDM or type Ⅱ). It was found that WCD of the diabetics was significantly decreased compared with fifty controls (P<0.01) and further reduced with the rise of levels of plasma fibrinogen (Fg), plasma lipid, hemoglobin Ale (HbAlc) and fasting blood glucose (FBG), and also with the increase of age, the duration of diabetes, and the reduction of plasma magnesium concentration. It was shown that the decrease of WCD in diabetes mellitus was closely related to the degree of metabolic disturbance of the failure of diabetic control.
