Big data is an emerging term in the storage indus- try, and it is data analytics on big storage, i.e., Cloud-scale storage. In Cloud-scale （or EB-scale） file systems, load bal- ancing in request workloads across a m...Big data is an emerging term in the storage indus- try, and it is data analytics on big storage, i.e., Cloud-scale storage. In Cloud-scale （or EB-scale） file systems, load bal- ancing in request workloads across a metadata server cluster is critical for avoiding performance bottlenecks and improv- ing quality of services. Many good approaches have been pro- posed for load balancing in distributed file systems. Some of them pay attention to global namespace balancing, making metadata distribution across metadata servers as uniform as possible. However, they do not work well in skew request dis- tributions, which impair load balancing but simultaneously increase the effectiveness of caching and replication, in this paper, we propose Cloud Cache （C2）, an adaptive and scal- able load balancing scheme for metadata server cluster in EB-scale file systems. It combines adaptive cache diffusion and replication scheme to cope with the request load balanc- ing problem, and it can be integrated into existing distributed metadata management approaches to efficiently improve their load balancing performance. C2 runs as follows： 1） to run adaptive cache diffusion first, if a node is overloaded, load- shedding will be used; otherwise, load-stealing will be used; and 2） to run adaptive replication scheme second, if there is a very popular metadata item （or at least two items） causing a node be overloaded, adaptive replication scheme will be used,in which the very popular item is not split into several nodes using adaptive cache diffusion because of its knapsack prop- erty. By conducting performance evaluation in trace-driven simulations, experimental results demonstrate the efficiency and scalability of C2.展开更多
文摘Big data is an emerging term in the storage indus- try, and it is data analytics on big storage, i.e., Cloud-scale storage. In Cloud-scale （or EB-scale） file systems, load bal- ancing in request workloads across a metadata server cluster is critical for avoiding performance bottlenecks and improv- ing quality of services. Many good approaches have been pro- posed for load balancing in distributed file systems. Some of them pay attention to global namespace balancing, making metadata distribution across metadata servers as uniform as possible. However, they do not work well in skew request dis- tributions, which impair load balancing but simultaneously increase the effectiveness of caching and replication, in this paper, we propose Cloud Cache （C2）, an adaptive and scal- able load balancing scheme for metadata server cluster in EB-scale file systems. It combines adaptive cache diffusion and replication scheme to cope with the request load balanc- ing problem, and it can be integrated into existing distributed metadata management approaches to efficiently improve their load balancing performance. C2 runs as follows： 1） to run adaptive cache diffusion first, if a node is overloaded, load- shedding will be used; otherwise, load-stealing will be used; and 2） to run adaptive replication scheme second, if there is a very popular metadata item （or at least two items） causing a node be overloaded, adaptive replication scheme will be used,in which the very popular item is not split into several nodes using adaptive cache diffusion because of its knapsack prop- erty. By conducting performance evaluation in trace-driven simulations, experimental results demonstrate the efficiency and scalability of C2.
