The reliability and high performance of metadata service is crucial to the store architecture. A novel design of a two-level metadata server file system （TTMFS） is presented, which behaves high reliability and perfo...The reliability and high performance of metadata service is crucial to the store architecture. A novel design of a two-level metadata server file system （TTMFS） is presented, which behaves high reliability and performance. The merits both centralized management and distributed management are considered simultaneously in our design. In this file system, the advanced-metadata server is responsible for manage directory metadata and the whole namespace. The double-metadata server is responsible for maintaining file metadata. And this paper uses the Markov return model to analyze the reliability of the two-level metadata server. The experiment data indicates that the design can provide high throughput.展开更多
Distribution of metadata in a metadata server cluster is important in mass storage system. A good distribution algorithm has a significant influence on the system performance, availability and scalability. Subtree par...Distribution of metadata in a metadata server cluster is important in mass storage system. A good distribution algorithm has a significant influence on the system performance, availability and scalability. Subtree partition and hash are two traditional metadata distribution algorithms used in distributed file systems. They both have a defect in system scalability. This paper proposes a new directory hash （DH） algorithm. By treating directory as the key value of hash function, implementing concentrated storage of metadata, pipelining operations and prefetching technology, DH algorithm can enhance the system scalability on the premise without sacrificing system performance.展开更多
In an object-based storage system,a novel scheme named EAP(extending attributes page) is presented to enhance the metadata reliability of the system by adding the user object file information attributes page for each ...In an object-based storage system,a novel scheme named EAP(extending attributes page) is presented to enhance the metadata reliability of the system by adding the user object file information attributes page for each user object and storing the file-related attributes of each user object in object-based storage devices.The EAP scheme requires no additional hardware equipments compared to a general method which uses backup metadata servers to improve the metadata reliability.Leveraging a Markov chain,this paper compares the metadata reliability of the system using the EAP scheme with that using only metadata servers to offer the file metadata service.Our results demonstrate that the EAP scheme can dramatically enhance the reliability of storage system metadata.展开更多
In data center, applications of big data analytics pose a big challenge to massive storage systems. It is signif- icant to achieve high availability, high performance and high scalability for PB-scale or EB-scale stor...In data center, applications of big data analytics pose a big challenge to massive storage systems. It is signif- icant to achieve high availability, high performance and high scalability for PB-scale or EB-scale storage systems. Meta- data server （MDS） cluster architecture is one of the most effective solutions to meet the requirements of applications in data center. Workload migration can achieve load balance and energy saving of duster systems. In this paper, a hybrid workload migration mechanism of MDS cluster is proposed and named as HWM. In HWM, workload of MDS is classi- fied into two categories： metadata service and state service, and they can be migrated rapidly from a source MDS to a target MDS in different ways. Firstly, in metadata service migration, all the dirty metadata of one sub file system is flushed to a shared storage pool by the source MDS, and then is loaded by the target MDS. Secondly, in state service mi- gration, all the states of that sub file system are migrated from source MDS to target MDS through network at file granular- ity, and then all of the related structures of these states are reconstructed in target MDS. Thirdly, in the process of work- load migration, instead of blocking client requests, the source MDS can decide which MDS will respond to each request according to the operation type and the migration stage. The proposed mechanism is implemented in the Blue Whale MDS cluster. The performance measurements show that the HWM mechanism is efficient to migrate the workload of a MDS cluster system and provides low-latency access to metadata and states.展开更多
基金Supported by the Industrialized Foundation ofHebei Province (F020501)
文摘The reliability and high performance of metadata service is crucial to the store architecture. A novel design of a two-level metadata server file system （TTMFS） is presented, which behaves high reliability and performance. The merits both centralized management and distributed management are considered simultaneously in our design. In this file system, the advanced-metadata server is responsible for manage directory metadata and the whole namespace. The double-metadata server is responsible for maintaining file metadata. And this paper uses the Markov return model to analyze the reliability of the two-level metadata server. The experiment data indicates that the design can provide high throughput.
基金Project supported by the National Grand Fundamental Research 973 Program of China （Grant No.2004CB318203）, and the National Natural Science Foundation of China （Grant No.60603074）
文摘Distribution of metadata in a metadata server cluster is important in mass storage system. A good distribution algorithm has a significant influence on the system performance, availability and scalability. Subtree partition and hash are two traditional metadata distribution algorithms used in distributed file systems. They both have a defect in system scalability. This paper proposes a new directory hash （DH） algorithm. By treating directory as the key value of hash function, implementing concentrated storage of metadata, pipelining operations and prefetching technology, DH algorithm can enhance the system scalability on the premise without sacrificing system performance.
基金supported by the National Natural Science Foundation of China (No.60873028)the National Basic Research Program (973) of China (No.2004CB318201)+1 种基金the Program for New Century Excellent Talents in University (No.NCET-04-0693)the Innovational Group Project (No.IRT0725),China
文摘In an object-based storage system,a novel scheme named EAP(extending attributes page) is presented to enhance the metadata reliability of the system by adding the user object file information attributes page for each user object and storing the file-related attributes of each user object in object-based storage devices.The EAP scheme requires no additional hardware equipments compared to a general method which uses backup metadata servers to improve the metadata reliability.Leveraging a Markov chain,this paper compares the metadata reliability of the system using the EAP scheme with that using only metadata servers to offer the file metadata service.Our results demonstrate that the EAP scheme can dramatically enhance the reliability of storage system metadata.
文摘In data center, applications of big data analytics pose a big challenge to massive storage systems. It is signif- icant to achieve high availability, high performance and high scalability for PB-scale or EB-scale storage systems. Meta- data server （MDS） cluster architecture is one of the most effective solutions to meet the requirements of applications in data center. Workload migration can achieve load balance and energy saving of duster systems. In this paper, a hybrid workload migration mechanism of MDS cluster is proposed and named as HWM. In HWM, workload of MDS is classi- fied into two categories： metadata service and state service, and they can be migrated rapidly from a source MDS to a target MDS in different ways. Firstly, in metadata service migration, all the dirty metadata of one sub file system is flushed to a shared storage pool by the source MDS, and then is loaded by the target MDS. Secondly, in state service mi- gration, all the states of that sub file system are migrated from source MDS to target MDS through network at file granular- ity, and then all of the related structures of these states are reconstructed in target MDS. Thirdly, in the process of work- load migration, instead of blocking client requests, the source MDS can decide which MDS will respond to each request according to the operation type and the migration stage. The proposed mechanism is implemented in the Blue Whale MDS cluster. The performance measurements show that the HWM mechanism is efficient to migrate the workload of a MDS cluster system and provides low-latency access to metadata and states.
