There is a problem of real-time detection difficulty in road surface damage detection. This paper proposes an improved lightweight model based on you only look once version 5(YOLOv5). Firstly, this paper fully utilize...There is a problem of real-time detection difficulty in road surface damage detection. This paper proposes an improved lightweight model based on you only look once version 5(YOLOv5). Firstly, this paper fully utilized the convolutional neural network(CNN) + ghosting bottleneck(G_bneck) architecture to reduce redundant feature maps. Afterwards, we upgraded the original upsampling algorithm to content-aware reassembly of features(CARAFE) and increased the receptive field. Finally, we replaced the spatial pyramid pooling fast(SPPF) module with the basic receptive field block(Basic RFB) pooling module and added dilated convolution. After comparative experiments, we can see that the number of parameters and model size of the improved algorithm in this paper have been reduced by nearly half compared to the YOLOv5s. The frame rate per second(FPS) has been increased by 3.25 times. The mean average precision(m AP@0.5: 0.95) has increased by 8%—17% compared to other lightweight algorithms.展开更多
Extracting and mining social networks information from massive Web data is of both theoretical and practical significance. However, one of definite features of this task was a large scale data processing, which remain...Extracting and mining social networks information from massive Web data is of both theoretical and practical significance. However, one of definite features of this task was a large scale data processing, which remained to be a great challenge that would be addressed. MapReduce is a kind of distributed programming model. Just through the implementation of map and reduce those two functions, the distributed tasks can work well. Nevertheless, this model does not directly support heterogeneous datasets processing, while heterogeneous datasets are common in Web. This article proposes a new framework which improves original MapReduce framework into a new one called Map-Reduce-Merge. It adds merge phase that can efficiently solve the problems of heterogeneous data processing. At the same time, some works of optimization and improvement are done based on the features of Web data.展开更多
MapReduce是一个流行的并行处理大规模数据计算模型.为提升异构环境下的MapReduce性能,提出一种异构环境下基于节点作业时间感知的动态MapReduce调度策略:DTHE(Dynamic Map Reduce scheduling based on the Time-aware of node jobs in ...MapReduce是一个流行的并行处理大规模数据计算模型.为提升异构环境下的MapReduce性能,提出一种异构环境下基于节点作业时间感知的动态MapReduce调度策略:DTHE(Dynamic Map Reduce scheduling based on the Time-aware of node jobs in Heterogeneous Environments).DTHE在作业执行前,首先标记部分任务作为节点样本任务并优先处理,在执行其他任务时分析样本任务,预测节点性能和数据分布特征,动态采取相应的调度策略;在作业运行中实时监测节点任务状态,提前拉取节点下一个任务数据到本地内存.实验结果表明:在异构环境下,DTEH能够缩短5.1%的作业执行时间并减少磁盘I/O,有效提升MapReduce性能.展开更多
基金supported by the Shanghai Sailing Program,China (No.20YF1447600)the Research Start-Up Project of Shanghai Institute of Technology (No.YJ2021-60)+1 种基金the Collaborative Innovation Project of Shanghai Institute of Technology (No.XTCX2020-12)the Science and Technology Talent Development Fund for Young and Middle-Aged Teachers at Shanghai Institute of Technology (No.ZQ2022-6)。
文摘There is a problem of real-time detection difficulty in road surface damage detection. This paper proposes an improved lightweight model based on you only look once version 5(YOLOv5). Firstly, this paper fully utilized the convolutional neural network(CNN) + ghosting bottleneck(G_bneck) architecture to reduce redundant feature maps. Afterwards, we upgraded the original upsampling algorithm to content-aware reassembly of features(CARAFE) and increased the receptive field. Finally, we replaced the spatial pyramid pooling fast(SPPF) module with the basic receptive field block(Basic RFB) pooling module and added dilated convolution. After comparative experiments, we can see that the number of parameters and model size of the improved algorithm in this paper have been reduced by nearly half compared to the YOLOv5s. The frame rate per second(FPS) has been increased by 3.25 times. The mean average precision(m AP@0.5: 0.95) has increased by 8%—17% compared to other lightweight algorithms.
文摘Extracting and mining social networks information from massive Web data is of both theoretical and practical significance. However, one of definite features of this task was a large scale data processing, which remained to be a great challenge that would be addressed. MapReduce is a kind of distributed programming model. Just through the implementation of map and reduce those two functions, the distributed tasks can work well. Nevertheless, this model does not directly support heterogeneous datasets processing, while heterogeneous datasets are common in Web. This article proposes a new framework which improves original MapReduce framework into a new one called Map-Reduce-Merge. It adds merge phase that can efficiently solve the problems of heterogeneous data processing. At the same time, some works of optimization and improvement are done based on the features of Web data.
文摘MapReduce是一个流行的并行处理大规模数据计算模型.为提升异构环境下的MapReduce性能,提出一种异构环境下基于节点作业时间感知的动态MapReduce调度策略:DTHE(Dynamic Map Reduce scheduling based on the Time-aware of node jobs in Heterogeneous Environments).DTHE在作业执行前,首先标记部分任务作为节点样本任务并优先处理,在执行其他任务时分析样本任务,预测节点性能和数据分布特征,动态采取相应的调度策略;在作业运行中实时监测节点任务状态,提前拉取节点下一个任务数据到本地内存.实验结果表明:在异构环境下,DTEH能够缩短5.1%的作业执行时间并减少磁盘I/O,有效提升MapReduce性能.
