To reduce the time required to complete the regeneration process of erasure codes, we propose a Tree-structured Parallel Regeneration (TPR) scheme for multiple data losses in distributed storage systems. Under the sch...To reduce the time required to complete the regeneration process of erasure codes, we propose a Tree-structured Parallel Regeneration (TPR) scheme for multiple data losses in distributed storage systems. Under the scheme, two algorithms are proposed for the construction of multiple regeneration trees, namely the edge-disjoint algorithm and edge-sharing algorithm. The edge-disjoint algorithm constructs multiple independent trees, and is simple and appropriate for environments where newcomers and their providers are distributed over a large area and have few intersections. The edge-sharing algorithm constructs multiple trees that compete to utilize the bandwidth, and make a better utilization of the bandwidth, although it needs to measure the available band-width and deal with the bandwidth changes; it is therefore difficult to implement in practical systems. The parallel regeneration for multiple data losses of TPR primarily includes two optimizations: firstly, transferring the data through the bandwidth optimized-paths in a pipe-line manner; secondly, executing data regeneration over multiple trees in parallel. To evaluate the proposal, we implement an event-based simulator and make a detailed comparison with some popular regeneration methods. The quantitative comparison results show that the use of TPR employing either the edge-disjoint algorithm or edge-sharing algorithm reduces the regeneration time significantly.展开更多
This paper describes the architecture of global distributed storage system for data grid. It focue on the management and the capability for the maximum users and maximum resources on the Internet, as well as performan...This paper describes the architecture of global distributed storage system for data grid. It focue on the management and the capability for the maximum users and maximum resources on the Internet, as well as performance and other issues.展开更多
Remote data auditing becomes critical to ensure the storage reliability in distributed cloud storage.Recently,Le et al proposed an efficient private data auditing scheme NC-Audit designed for regenerating codes,which ...Remote data auditing becomes critical to ensure the storage reliability in distributed cloud storage.Recently,Le et al proposed an efficient private data auditing scheme NC-Audit designed for regenerating codes,which claimed that NC-Audit can effectively realize privacy-preserving data auditing for distributed storage systems.However,our analysis shows that NC-Audit is not secure for that the adversarial cloud can forge some illegal blocks to cheat the auditor successfully with a high probability even without storing the user’s whole data,when the coding field is large enough.展开更多
Erasure code is widely used as the redundancy scheme in distributed storage system. When a storage node fails, the repair process often requires to transfer a large amount of data. Regenerating code and hierarchical c...Erasure code is widely used as the redundancy scheme in distributed storage system. When a storage node fails, the repair process often requires to transfer a large amount of data. Regenerating code and hierarchical code are two classes of codes proposed to reduce the repair bandwidth cost. Regenerating codes reduce the amount of data transferred by each helping node, while hierarchical codes reduce the number of nodes participating in the repair process. In this paper, we propose a "sub-code nesting framework" to combine them together. The resulting regenerating hierarchical code has low repair degree as hierarchical code and lower repair cost than hierarchical code. Our code can achieve exact regeneration of the failed node, and has the additional property of low updating complexity.展开更多
Current distributed parallel file systems and database systems can not satisfy the demands of data-intensive applications, such as storage capacity, access performance, reliability, scalability, and so on. Cluster-bas...Current distributed parallel file systems and database systems can not satisfy the demands of data-intensive applications, such as storage capacity, access performance, reliability, scalability, and so on. Cluster-based storage sys tems have some shortcomings, too. To solve this kind of problems, a novel PC storage cluster solution is proposed, a distributed storage system based on 3-tiered agent architecture is designed, the system reliability model based on the masterslave backup mode is built, and the system availability is analyzed with the Markov model. According to the system availability formula and the values of the system parameters, the novel system can provide higher reliability and availability to satisfy users＇ requirements,展开更多
Recently,research on a distributed storage system that efficiently manages a large amount of data has been actively conducted following data production and demand increase.Physical expansion limits exist for tradition...Recently,research on a distributed storage system that efficiently manages a large amount of data has been actively conducted following data production and demand increase.Physical expansion limits exist for traditional standalone storage systems,such as I/O and file system capacity.However,the existing distributed storage system does not consider where data is consumed and is more focused on data dissemination and optimizing the lookup cost of data location.And this leads to system performance degradation due to low locality occurring in a Wide Area Network(WAN)environment with high network latency.This problem hinders deploying distributed storage systems to multiple data centers over WAN.It lowers the scalability of distributed storage systems to accommodate data storage needs.This paper proposes a method for distributing data in a WAN environment considering network latency and data locality to solve this problem and increase overall system performance.The proposed distributed storage method monitors data utilization and locality to classify data temperature as hot,warm,and cold.With assigned data temperature,the proposed algorithm adaptively selects the appropriate data center and places data accordingly to overcome the excess latency from the WAN environment,leading to overall system performance degradation.This paper also conducts simulations to evaluate the proposed and existing distributed storage methods.The result shows that our proposed method reduced latency by 38%compared to the existing method.Therefore,the proposed method in this paper can be used in large-scale distributed storage systems over a WAN environment to improve latency and performance compared to existing methods,such as consistent hashing.展开更多
Fraction repetition(FR)codes are integral in distributed storage systems(DSS)with exact repair-by-transfer,while pliable fraction repetition codes are vital for DSSs in which both the per-node storage and repetition d...Fraction repetition(FR)codes are integral in distributed storage systems(DSS)with exact repair-by-transfer,while pliable fraction repetition codes are vital for DSSs in which both the per-node storage and repetition degree can easily be adjusted simultaneously.This paper introduces a new type of pliable FR codes,called absolute balanced pliable FR(ABPFR)codes,in which the access balancing in DSS is considered.Additionally,the equivalence between pliable FR codes and resolvable transversal packings in combinatorial design theory is presented.Then constructions of pliable FR codes and ABPFR codes based on resolvable transversal packings are presented.展开更多
To ensure the reliability and availability of data,redundancy strategies are always required for distributed storage systems.Erasure coding,one of the representative redundancy strategies,has the advantage of low stor...To ensure the reliability and availability of data,redundancy strategies are always required for distributed storage systems.Erasure coding,one of the representative redundancy strategies,has the advantage of low storage overhead,which facilitates its employment in distributed storage systems.Among the various erasure coding schemes,XOR-based erasure codes are becoming popular due to their high computing speed.When a single-node failure occurs in such coding schemes,a process called data recovery takes place to retrieve the failed node’s lost data from surviving nodes.However,data transmission during the data recovery process usually requires a considerable amount of time.Current research has focused mainly on reducing the amount of data needed for data recovery to reduce the time required for data transmission,but it has encountered problems such as significant complexity and local optima.In this paper,we propose a random search recovery algorithm,named SA-RSR,to speed up single-node failure recovery of XOR-based erasure codes.SA-RSR uses a simulated annealing technique to search for an optimal recovery solution that reads and transmits a minimum amount of data.In addition,this search process can be done in polynomial time.We evaluate SA-RSR with a variety of XOR-based erasure codes in simulations and in a real storage system,Ceph.Experimental results in Ceph show that SA-RSR reduces the amount of data required for recovery by up to 30.0%and improves the performance of data recovery by up to 20.36%compared to the conventional recovery method.展开更多
In distributed cloud storage systems, inevitably there exist multiple node failures at the same time. The existing methods of regenerating codes, including minimum storage regenerating(MSR) codes and minimum bandwidth...In distributed cloud storage systems, inevitably there exist multiple node failures at the same time. The existing methods of regenerating codes, including minimum storage regenerating(MSR) codes and minimum bandwidth regenerating(MBR) codes, are mainly to repair one single or several failed nodes, unable to meet the repair need of distributed cloud storage systems. In this paper, we present locally minimum storage regenerating(LMSR) codes to recover multiple failed nodes at the same time. Specifically, the nodes in distributed cloud storage systems are divided into multiple local groups, and in each local group(4, 2) or(5, 3) MSR codes are constructed. Moreover, the grouping method of storage nodes and the repairing process of failed nodes in local groups are studied. Theoretical analysis shows that LMSR codes can achieve the same storage overhead as MSR codes. Furthermore, we verify by means of simulation that, compared with MSR codes, LMSR codes can reduce the repair bandwidth and disk I/O overhead effectively.展开更多
Distributed storage can store data in multiple devices or servers to improve data security.However,in today’s explosive growth of network data,traditional distributed storage scheme is faced with some severe challeng...Distributed storage can store data in multiple devices or servers to improve data security.However,in today’s explosive growth of network data,traditional distributed storage scheme is faced with some severe challenges such as insufficient performance,data tampering,and data lose.A distributed storage scheme based on blockchain has been proposed to improve security and efficiency of traditional distributed storage.Under this scheme,the following improvements have been made in this paper.This paper first analyzes the problems faced by distributed storage.Then proposed to build a new distributed storage blockchain scheme with sharding blockchain.The proposed scheme realizes the partitioning of the network and nodes by means of blockchain sharding technology,which can improve the efficiency of data verification between nodes.In addition,this paper uses polynomial commitment to construct a new verifiable secret share scheme called PolyVSS.This new scheme is one of the foundations for building our improved distributed storage blockchain scheme.Compared with the previous scheme,our new scheme does not require a trusted third party and has some new features such as homomorphic and batch opening.The security of VSS can be further improved.Experimental comparisons show that the proposed scheme significantly reduces storage and communication costs.展开更多
With the advent of the era of big data,cloud computing,Internet of things,and other information industries continue to develop.There is an increasing amount of unstructured data such as pictures,audio,and video on the...With the advent of the era of big data,cloud computing,Internet of things,and other information industries continue to develop.There is an increasing amount of unstructured data such as pictures,audio,and video on the Internet.And the distributed object storage system has become the mainstream cloud storage solution.With the increasing number of distributed applications,data security in the distributed object storage system has become the focus.For the distributed object storage system,traditional defenses are means that fix discovered system vulnerabilities and backdoors by patching,or means to modify the corresponding structure and upgrade.However,these two kinds of means are hysteretic and hardly deal with unknown security threats.Based on mimic defense theory,this paper constructs the principle framework of the distributed object storage system and introduces the dynamic redundancy and heterogeneous function in the distributed object storage system architecture,which increases the attack cost,and greatly improves the security and availability of data.展开更多
Based on the energy storage cloud platform architecture,this study considers the extensive configuration of energy storage devices and the future large-scale application of electric vehicles at the customer side to bu...Based on the energy storage cloud platform architecture,this study considers the extensive configuration of energy storage devices and the future large-scale application of electric vehicles at the customer side to build a new mode of smart power consumption with a flexible interaction,smooth the peak/valley difference of the load side power,and improve energy efficiency.A plug and play device for customer-side energy storage and an internet-based energy storage cloud platform are developed herein to build a new intelligent power consumption mode with a flexible interaction suitable for ordinary customers.Based on the load perception of the power grid,this study aims to investigate the operating state and service life of distributed energy storage devices.By selecting an integrated optimal control scheme,this study designs a kind of energy optimization and deployment strategy for stratified partition to reduce the operating cost of the energy storage device on the client side.The effectiveness of the system and the control strategy is verified through the Suzhou client-side distributed energy storage demonstration project.展开更多
There is instability in the distributed energy storage cloud group end region on the power grid side.In order to avoid large-scale fluctuating charging and discharging in the power grid environment and make the capaci...There is instability in the distributed energy storage cloud group end region on the power grid side.In order to avoid large-scale fluctuating charging and discharging in the power grid environment and make the capacitor components showa continuous and stable charging and discharging state,a hierarchical time-sharing configuration algorithm of distributed energy storage cloud group end region on the power grid side based on multi-scale and multi feature convolution neural network is proposed.Firstly,a voltage stability analysis model based onmulti-scale and multi feature convolution neural network is constructed,and the multi-scale and multi feature convolution neural network is optimized based on Self-OrganizingMaps(SOM)algorithm to analyze the voltage stability of the cloud group end region of distributed energy storage on the grid side under the framework of credibility.According to the optimal scheduling objectives and network size,the distributed robust optimal configuration control model is solved under the framework of coordinated optimal scheduling at multiple time scales;Finally,the time series characteristics of regional power grid load and distributed generation are analyzed.According to the regional hierarchical time-sharing configuration model of“cloud”,“group”and“end”layer,the grid side distributed energy storage cloud group end regional hierarchical time-sharing configuration algorithm is realized.The experimental results show that after applying this algorithm,the best grid side distributed energy storage configuration scheme can be determined,and the stability of grid side distributed energy storage cloud group end region layered timesharing configuration can be improved.展开更多
The knowledge graph with relational abundant information has been widely used as the basic data support for the retrieval platforms.Image and text descriptions added to the knowledge graph enrich the node information,...The knowledge graph with relational abundant information has been widely used as the basic data support for the retrieval platforms.Image and text descriptions added to the knowledge graph enrich the node information,which accounts for the advantage of the multi-modal knowledge graph.In the field of cross-modal retrieval platforms,multi-modal knowledge graphs can help to improve retrieval accuracy and efficiency because of the abundant relational infor-mation provided by knowledge graphs.The representation learning method is sig-nificant to the application of multi-modal knowledge graphs.This paper proposes a distributed collaborative vector retrieval platform(DCRL-KG)using the multi-modal knowledge graph VisualSem as the foundation to achieve efficient and high-precision multimodal data retrieval.Firstly,use distributed technology to classify and store the data in the knowledge graph to improve retrieval efficiency.Secondly,this paper uses BabelNet to expand the knowledge graph through multi-ple filtering processes and increase the diversification of information.Finally,this paper builds a variety of retrieval models to achieve the fusion of retrieval results through linear combination methods to achieve high-precision language retrieval and image retrieval.The paper uses sentence retrieval and image retrieval experi-ments to prove that the platform can optimize the storage structure of the multi-modal knowledge graph and have good performance in multi-modal space.展开更多
During electric vehicle(EV)-assisted grid frequency modulation,inconsistent state of charge(SOC)among EVs can result in overcharging and discharging of the batteries,affecting the stability of the electrical system.As...During electric vehicle(EV)-assisted grid frequency modulation,inconsistent state of charge(SOC)among EVs can result in overcharging and discharging of the batteries,affecting the stability of the electrical system.As a solution,this paper proposes a priority-based frequency regulation strategy for EVs.Firstly,models for the primary and secondary frequency regulation of EV-assisted power grids are established.Secondly,a consensus algorithm is used to construct a distributed com-munication system for EVs.Target SOC values are used to obtain a local frequency regulation priori-ty list.The list is used in an optimal control plan allowing individual EVs to participate in frequency regulation.Finally,a simulation of this strategy under several scenarios is conducted.The results indicate that the strategy ensures uniform SOC among the participating group of EVs,thereby avoi-ding overcharging and discharging of their batteries.It also reduces frequency fluctuations in the electrical system,making the system more robust compared with the frequency regulation strategy that is not priority-based.展开更多
With more and more distributed photovoltaic(PV)plants access to the distribution system,whose structure is changing and becoming an active network.The traditional methods of voltage regulation may hardly adapt to this...With more and more distributed photovoltaic(PV)plants access to the distribution system,whose structure is changing and becoming an active network.The traditional methods of voltage regulation may hardly adapt to this new situation.To address this problem,this paper presents a coordinated control method of distributed energy storage systems(DESSs)for voltage regulation in a distribution network.The influence of the voltage caused by the PV plant is analyzed in a simple distribution feeder at first.The voltage regulation areas corresponding to DESSs are divided by calculating and comparing the voltage sensitivity matrix.Then,a coordinated voltage control strategy is proposed for the DESSs.Finally,the simulation results of the IEEE 33-bus radial distribution network verify the effectiveness of the proposed coordinated control method.展开更多
The unbalanced state of charge(SOC)of distributed energy storage systems(DESSs)in autonomous DC microgrid causes energy storage units(ESUs)to terminate operation due to overcharge or overdischarge,which severely affec...The unbalanced state of charge(SOC)of distributed energy storage systems(DESSs)in autonomous DC microgrid causes energy storage units(ESUs)to terminate operation due to overcharge or overdischarge,which severely affects the power quality.In this paper,a fuzzy droop control for SOC balance and stability analysis of DC microgrid with DESSs is proposed to achieve SOC balance in ESUs while maintaining a stable DC bus voltage.First,the charge and discharge modes of ESUs are determined based on the power supply requirements of the DC microgrid.One-dimensional fuzzy logic is then applied to establish the relationship between SOC and the droop coefficient R,in the aforementioned two modes.In addition,when integrated with voltage-current double closed-loop control,SOC balance in different ESUs is realized.To improve the balance speed and precision,an exponential acceleration factor is added to the input variable of the fuzzy controller.Finally,based on the average model of converter,the system-level stability of microgrid is analyzed.MATLAB/Simulink simulation results verify the effectiveness and rationality of the proposed method.展开更多
This paper presents a fully distributed state-of-charge balance control (DSBC) strategy for a distributed energy storage system (DESS). In this framework, each energy storage unit (ESU) processes the state-of-charge (...This paper presents a fully distributed state-of-charge balance control (DSBC) strategy for a distributed energy storage system (DESS). In this framework, each energy storage unit (ESU) processes the state-of-charge (SoC) information from its neighbors locally and adjusts the virtual impedance of the droop controller in real-time to change the current sharing. It is shown that the SoC balance of all ESUs can be achieved. Due to virtual impedance, voltage deviation of the bus occurs inevitably and increases with load power. Meanwhile, widespread of the constant power load (CPL) in the power system may cause instability. To ensure reliable operation of DESS under the proposed DSBC, the concept of the safe region is put forward. Within the safe region, DESS is stable and voltage deviation is acceptable. The boundary conditions of the safe region are derived from the equivalent model of DESS, in which stability is analyzed in terms of modified Brayton-Moser's criterion. Both simulations and hardware experiments verify the accuracy of the safe region and effectiveness of the proposed DSBC strategy.展开更多
In distributed storage systems,file access efficiency has an important impact on the real-time nature of information forensics.As a popular approach to improve file accessing efficiency,prefetching model can fetches d...In distributed storage systems,file access efficiency has an important impact on the real-time nature of information forensics.As a popular approach to improve file accessing efficiency,prefetching model can fetches data before it is needed according to the file access pattern,which can reduce the I/O waiting time and increase the system concurrency.However,prefetching model needs to mine the degree of association between files to ensure the accuracy of prefetching.In the massive small file situation,the sheer volume of files poses a challenge to the efficiency and accuracy of relevance mining.In this paper,we propose a massive files prefetching model based on LSTM neural network with cache transaction strategy to improve file access efficiency.Firstly,we propose a file clustering algorithm based on temporal locality and spatial locality to reduce the computational complexity.Secondly,we propose a definition of cache transaction according to files occurrence in cache instead of time-offset distance based methods to extract file block feature accurately.Lastly,we innovatively propose a file access prediction algorithm based on LSTM neural network which predict the file that have high possibility to be accessed.Experiments show that compared with the traditional LRU and the plain grouping methods,the proposed model notably increase the cache hit rate and effectively reduces the I/O wait time.展开更多
We study the construction of minimum bandwidth regenerating code with combinatorial design. At first, a method of constructing minimum storage regenerating （MBR） codes is presented, which can tolerate only one-node ...We study the construction of minimum bandwidth regenerating code with combinatorial design. At first, a method of constructing minimum storage regenerating （MBR） codes is presented, which can tolerate only one-node failure. Then, we give examples to explain the code. Finally, we discuss the case of repairing multiple nodes, and analyze the performance with an example.展开更多
基金supported by the National Grand Fundamental Research of China (973 Program) under Grant No. 2011CB302601the National High Technology Research and Development of China (863 Program) under GrantNo. 2013AA01A213+2 种基金the National Natural Science Foundation of China under Grant No. 60873215the Natural Science Foundation for Distinguished Young Scholars of Hunan Province under Grant No. S2010J5050Specialized Research Fund for the Doctoral Program of Higher Education under Grant No. 20124307110015
文摘To reduce the time required to complete the regeneration process of erasure codes, we propose a Tree-structured Parallel Regeneration (TPR) scheme for multiple data losses in distributed storage systems. Under the scheme, two algorithms are proposed for the construction of multiple regeneration trees, namely the edge-disjoint algorithm and edge-sharing algorithm. The edge-disjoint algorithm constructs multiple independent trees, and is simple and appropriate for environments where newcomers and their providers are distributed over a large area and have few intersections. The edge-sharing algorithm constructs multiple trees that compete to utilize the bandwidth, and make a better utilization of the bandwidth, although it needs to measure the available band-width and deal with the bandwidth changes; it is therefore difficult to implement in practical systems. The parallel regeneration for multiple data losses of TPR primarily includes two optimizations: firstly, transferring the data through the bandwidth optimized-paths in a pipe-line manner; secondly, executing data regeneration over multiple trees in parallel. To evaluate the proposal, we implement an event-based simulator and make a detailed comparison with some popular regeneration methods. The quantitative comparison results show that the use of TPR employing either the edge-disjoint algorithm or edge-sharing algorithm reduces the regeneration time significantly.
文摘This paper describes the architecture of global distributed storage system for data grid. It focue on the management and the capability for the maximum users and maximum resources on the Internet, as well as performance and other issues.
基金Supported by the National Natural Science Foundation of China(61872088)the Science and Technology Plan Project of Xi’an(2020KJWL02,2017CGWL35)the China National Study Abroad Fund。
文摘Remote data auditing becomes critical to ensure the storage reliability in distributed cloud storage.Recently,Le et al proposed an efficient private data auditing scheme NC-Audit designed for regenerating codes,which claimed that NC-Audit can effectively realize privacy-preserving data auditing for distributed storage systems.However,our analysis shows that NC-Audit is not secure for that the adversarial cloud can forge some illegal blocks to cheat the auditor successfully with a high probability even without storing the user’s whole data,when the coding field is large enough.
基金Supported by 973 Project of China (No. 2012CB315803)Research Fund for the Doctoral Program of Higher Education of China (No. 20100002110033)Open research Fund of National Mobile Communications Research Laboratory, Southeast University (No. 2011D11)
文摘Erasure code is widely used as the redundancy scheme in distributed storage system. When a storage node fails, the repair process often requires to transfer a large amount of data. Regenerating code and hierarchical code are two classes of codes proposed to reduce the repair bandwidth cost. Regenerating codes reduce the amount of data transferred by each helping node, while hierarchical codes reduce the number of nodes participating in the repair process. In this paper, we propose a "sub-code nesting framework" to combine them together. The resulting regenerating hierarchical code has low repair degree as hierarchical code and lower repair cost than hierarchical code. Our code can achieve exact regeneration of the failed node, and has the additional property of low updating complexity.
基金Supported by the Industrialization Foundation ofHebei Province (020501)the Natural Science Foundation ofHebei University (2005Q04)
文摘Current distributed parallel file systems and database systems can not satisfy the demands of data-intensive applications, such as storage capacity, access performance, reliability, scalability, and so on. Cluster-based storage sys tems have some shortcomings, too. To solve this kind of problems, a novel PC storage cluster solution is proposed, a distributed storage system based on 3-tiered agent architecture is designed, the system reliability model based on the masterslave backup mode is built, and the system availability is analyzed with the Markov model. According to the system availability formula and the values of the system parameters, the novel system can provide higher reliability and availability to satisfy users＇ requirements,
基金This research was supported by the Chung-Ang University Graduate Research Scholarship in 2021.This study was carried out with the support of‘R&D Program for Forest Science Technology(Project No.2021338C10-2223-CD02)’provided by Korea Forest Service(Korea Forestry Promotion Institute).
文摘Recently,research on a distributed storage system that efficiently manages a large amount of data has been actively conducted following data production and demand increase.Physical expansion limits exist for traditional standalone storage systems,such as I/O and file system capacity.However,the existing distributed storage system does not consider where data is consumed and is more focused on data dissemination and optimizing the lookup cost of data location.And this leads to system performance degradation due to low locality occurring in a Wide Area Network(WAN)environment with high network latency.This problem hinders deploying distributed storage systems to multiple data centers over WAN.It lowers the scalability of distributed storage systems to accommodate data storage needs.This paper proposes a method for distributing data in a WAN environment considering network latency and data locality to solve this problem and increase overall system performance.The proposed distributed storage method monitors data utilization and locality to classify data temperature as hot,warm,and cold.With assigned data temperature,the proposed algorithm adaptively selects the appropriate data center and places data accordingly to overcome the excess latency from the WAN environment,leading to overall system performance degradation.This paper also conducts simulations to evaluate the proposed and existing distributed storage methods.The result shows that our proposed method reduced latency by 38%compared to the existing method.Therefore,the proposed method in this paper can be used in large-scale distributed storage systems over a WAN environment to improve latency and performance compared to existing methods,such as consistent hashing.
基金Supported in part by the National Key R&D Program of China(No.2020YFA0712300)NSFC(No.61872353)。
文摘Fraction repetition(FR)codes are integral in distributed storage systems(DSS)with exact repair-by-transfer,while pliable fraction repetition codes are vital for DSSs in which both the per-node storage and repetition degree can easily be adjusted simultaneously.This paper introduces a new type of pliable FR codes,called absolute balanced pliable FR(ABPFR)codes,in which the access balancing in DSS is considered.Additionally,the equivalence between pliable FR codes and resolvable transversal packings in combinatorial design theory is presented.Then constructions of pliable FR codes and ABPFR codes based on resolvable transversal packings are presented.
基金the National Natural Science Foundation of China(No.62172327)。
文摘To ensure the reliability and availability of data,redundancy strategies are always required for distributed storage systems.Erasure coding,one of the representative redundancy strategies,has the advantage of low storage overhead,which facilitates its employment in distributed storage systems.Among the various erasure coding schemes,XOR-based erasure codes are becoming popular due to their high computing speed.When a single-node failure occurs in such coding schemes,a process called data recovery takes place to retrieve the failed node’s lost data from surviving nodes.However,data transmission during the data recovery process usually requires a considerable amount of time.Current research has focused mainly on reducing the amount of data needed for data recovery to reduce the time required for data transmission,but it has encountered problems such as significant complexity and local optima.In this paper,we propose a random search recovery algorithm,named SA-RSR,to speed up single-node failure recovery of XOR-based erasure codes.SA-RSR uses a simulated annealing technique to search for an optimal recovery solution that reads and transmits a minimum amount of data.In addition,this search process can be done in polynomial time.We evaluate SA-RSR with a variety of XOR-based erasure codes in simulations and in a real storage system,Ceph.Experimental results in Ceph show that SA-RSR reduces the amount of data required for recovery by up to 30.0%and improves the performance of data recovery by up to 20.36%compared to the conventional recovery method.
基金supported in part by the National Natural Science Foundation of China (61640006, 61572188)the Natural Science Foundation of Shaanxi Province, China (2015JM6307, 2016JQ6011)the project of science and technology of Xi’an City (2017088CG/RC051(CADX002))
文摘In distributed cloud storage systems, inevitably there exist multiple node failures at the same time. The existing methods of regenerating codes, including minimum storage regenerating(MSR) codes and minimum bandwidth regenerating(MBR) codes, are mainly to repair one single or several failed nodes, unable to meet the repair need of distributed cloud storage systems. In this paper, we present locally minimum storage regenerating(LMSR) codes to recover multiple failed nodes at the same time. Specifically, the nodes in distributed cloud storage systems are divided into multiple local groups, and in each local group(4, 2) or(5, 3) MSR codes are constructed. Moreover, the grouping method of storage nodes and the repairing process of failed nodes in local groups are studied. Theoretical analysis shows that LMSR codes can achieve the same storage overhead as MSR codes. Furthermore, we verify by means of simulation that, compared with MSR codes, LMSR codes can reduce the repair bandwidth and disk I/O overhead effectively.
基金This work was supported by the National Natural Science Foundation of China under Grant 62072249,61772280,61772454,62072056.J.Wang and Y.Ren received the grants,and the URL of the sponsors’website is http://www.nsfc.gov.cn/This work was also supported by the Project of Transformation and Upgrading of Industries and Information Technologies of Jiangsu Province(No.JITC-1900AX2038/01).X.Yu received the grant,and the URL of the sponsors’website is http://gxt.jiangsu.gov.cn/.
文摘Distributed storage can store data in multiple devices or servers to improve data security.However,in today’s explosive growth of network data,traditional distributed storage scheme is faced with some severe challenges such as insufficient performance,data tampering,and data lose.A distributed storage scheme based on blockchain has been proposed to improve security and efficiency of traditional distributed storage.Under this scheme,the following improvements have been made in this paper.This paper first analyzes the problems faced by distributed storage.Then proposed to build a new distributed storage blockchain scheme with sharding blockchain.The proposed scheme realizes the partitioning of the network and nodes by means of blockchain sharding technology,which can improve the efficiency of data verification between nodes.In addition,this paper uses polynomial commitment to construct a new verifiable secret share scheme called PolyVSS.This new scheme is one of the foundations for building our improved distributed storage blockchain scheme.Compared with the previous scheme,our new scheme does not require a trusted third party and has some new features such as homomorphic and batch opening.The security of VSS can be further improved.Experimental comparisons show that the proposed scheme significantly reduces storage and communication costs.
基金National Keystone R&D Program of China(No.2017YFB0803204)Shenzhen Research Programs(JCYJ20170306092030521)+3 种基金the PCL Future Regional Network Facilities for Largescale Experiments and Applications(LZC0019)ZTE University Funding,Natural Science Foundation of China(NSFC)(No.61671001)GuangDong Prov.,R&D Key Program(No.2019B010137001)the Shenzhen Municipal Development and Reform Commission(Disciplinary Development Program for Data Science and Intelligent Computing).
文摘With the advent of the era of big data,cloud computing,Internet of things,and other information industries continue to develop.There is an increasing amount of unstructured data such as pictures,audio,and video on the Internet.And the distributed object storage system has become the mainstream cloud storage solution.With the increasing number of distributed applications,data security in the distributed object storage system has become the focus.For the distributed object storage system,traditional defenses are means that fix discovered system vulnerabilities and backdoors by patching,or means to modify the corresponding structure and upgrade.However,these two kinds of means are hysteretic and hardly deal with unknown security threats.Based on mimic defense theory,this paper constructs the principle framework of the distributed object storage system and introduces the dynamic redundancy and heterogeneous function in the distributed object storage system architecture,which increases the attack cost,and greatly improves the security and availability of data.
基金supported by the State Grid Corporation of China Science and Technological Project(Research and demonstration application of key technology of energy storage cloud for mobile energy storage application of electric vehicles 5419-20197121 7a-0-0-00)
文摘Based on the energy storage cloud platform architecture,this study considers the extensive configuration of energy storage devices and the future large-scale application of electric vehicles at the customer side to build a new mode of smart power consumption with a flexible interaction,smooth the peak/valley difference of the load side power,and improve energy efficiency.A plug and play device for customer-side energy storage and an internet-based energy storage cloud platform are developed herein to build a new intelligent power consumption mode with a flexible interaction suitable for ordinary customers.Based on the load perception of the power grid,this study aims to investigate the operating state and service life of distributed energy storage devices.By selecting an integrated optimal control scheme,this study designs a kind of energy optimization and deployment strategy for stratified partition to reduce the operating cost of the energy storage device on the client side.The effectiveness of the system and the control strategy is verified through the Suzhou client-side distributed energy storage demonstration project.
基金supported by State Grid Corporation Limited Science and Technology Project Funding(Contract No.SGCQSQ00YJJS2200380).
文摘There is instability in the distributed energy storage cloud group end region on the power grid side.In order to avoid large-scale fluctuating charging and discharging in the power grid environment and make the capacitor components showa continuous and stable charging and discharging state,a hierarchical time-sharing configuration algorithm of distributed energy storage cloud group end region on the power grid side based on multi-scale and multi feature convolution neural network is proposed.Firstly,a voltage stability analysis model based onmulti-scale and multi feature convolution neural network is constructed,and the multi-scale and multi feature convolution neural network is optimized based on Self-OrganizingMaps(SOM)algorithm to analyze the voltage stability of the cloud group end region of distributed energy storage on the grid side under the framework of credibility.According to the optimal scheduling objectives and network size,the distributed robust optimal configuration control model is solved under the framework of coordinated optimal scheduling at multiple time scales;Finally,the time series characteristics of regional power grid load and distributed generation are analyzed.According to the regional hierarchical time-sharing configuration model of“cloud”,“group”and“end”layer,the grid side distributed energy storage cloud group end regional hierarchical time-sharing configuration algorithm is realized.The experimental results show that after applying this algorithm,the best grid side distributed energy storage configuration scheme can be determined,and the stability of grid side distributed energy storage cloud group end region layered timesharing configuration can be improved.
基金This work is supported by the Fundamental Research Funds for the Central Universities(Grant No.HIT.NSRIF.201714)Weihai Science and Technology Development Program(2016DX GJMS15)+1 种基金Weihai Scientific Research and Innovation Fund(2020)Key Research and Development Program in Shandong Provincial(2017GGX90103).
文摘The knowledge graph with relational abundant information has been widely used as the basic data support for the retrieval platforms.Image and text descriptions added to the knowledge graph enrich the node information,which accounts for the advantage of the multi-modal knowledge graph.In the field of cross-modal retrieval platforms,multi-modal knowledge graphs can help to improve retrieval accuracy and efficiency because of the abundant relational infor-mation provided by knowledge graphs.The representation learning method is sig-nificant to the application of multi-modal knowledge graphs.This paper proposes a distributed collaborative vector retrieval platform(DCRL-KG)using the multi-modal knowledge graph VisualSem as the foundation to achieve efficient and high-precision multimodal data retrieval.Firstly,use distributed technology to classify and store the data in the knowledge graph to improve retrieval efficiency.Secondly,this paper uses BabelNet to expand the knowledge graph through multi-ple filtering processes and increase the diversification of information.Finally,this paper builds a variety of retrieval models to achieve the fusion of retrieval results through linear combination methods to achieve high-precision language retrieval and image retrieval.The paper uses sentence retrieval and image retrieval experi-ments to prove that the platform can optimize the storage structure of the multi-modal knowledge graph and have good performance in multi-modal space.
基金Supported by the China Postdoctoral Science Foundation(No.2022M710039).
文摘During electric vehicle(EV)-assisted grid frequency modulation,inconsistent state of charge(SOC)among EVs can result in overcharging and discharging of the batteries,affecting the stability of the electrical system.As a solution,this paper proposes a priority-based frequency regulation strategy for EVs.Firstly,models for the primary and secondary frequency regulation of EV-assisted power grids are established.Secondly,a consensus algorithm is used to construct a distributed com-munication system for EVs.Target SOC values are used to obtain a local frequency regulation priori-ty list.The list is used in an optimal control plan allowing individual EVs to participate in frequency regulation.Finally,a simulation of this strategy under several scenarios is conducted.The results indicate that the strategy ensures uniform SOC among the participating group of EVs,thereby avoi-ding overcharging and discharging of their batteries.It also reduces frequency fluctuations in the electrical system,making the system more robust compared with the frequency regulation strategy that is not priority-based.
基金This paper is supported by The National Key Research and Development Plan,Energy Storage Technology of 10MW Level Redox Battery,2017YFB0903504。
文摘With more and more distributed photovoltaic(PV)plants access to the distribution system,whose structure is changing and becoming an active network.The traditional methods of voltage regulation may hardly adapt to this new situation.To address this problem,this paper presents a coordinated control method of distributed energy storage systems(DESSs)for voltage regulation in a distribution network.The influence of the voltage caused by the PV plant is analyzed in a simple distribution feeder at first.The voltage regulation areas corresponding to DESSs are divided by calculating and comparing the voltage sensitivity matrix.Then,a coordinated voltage control strategy is proposed for the DESSs.Finally,the simulation results of the IEEE 33-bus radial distribution network verify the effectiveness of the proposed coordinated control method.
基金supported by the National Natural Science Foundation of China(No.U22B20116).
文摘The unbalanced state of charge(SOC)of distributed energy storage systems(DESSs)in autonomous DC microgrid causes energy storage units(ESUs)to terminate operation due to overcharge or overdischarge,which severely affects the power quality.In this paper,a fuzzy droop control for SOC balance and stability analysis of DC microgrid with DESSs is proposed to achieve SOC balance in ESUs while maintaining a stable DC bus voltage.First,the charge and discharge modes of ESUs are determined based on the power supply requirements of the DC microgrid.One-dimensional fuzzy logic is then applied to establish the relationship between SOC and the droop coefficient R,in the aforementioned two modes.In addition,when integrated with voltage-current double closed-loop control,SOC balance in different ESUs is realized.To improve the balance speed and precision,an exponential acceleration factor is added to the input variable of the fuzzy controller.Finally,based on the average model of converter,the system-level stability of microgrid is analyzed.MATLAB/Simulink simulation results verify the effectiveness and rationality of the proposed method.
基金supported by the National Natural Science Foundation of China under Grant 61933014 and Grant 62173243.
文摘This paper presents a fully distributed state-of-charge balance control (DSBC) strategy for a distributed energy storage system (DESS). In this framework, each energy storage unit (ESU) processes the state-of-charge (SoC) information from its neighbors locally and adjusts the virtual impedance of the droop controller in real-time to change the current sharing. It is shown that the SoC balance of all ESUs can be achieved. Due to virtual impedance, voltage deviation of the bus occurs inevitably and increases with load power. Meanwhile, widespread of the constant power load (CPL) in the power system may cause instability. To ensure reliable operation of DESS under the proposed DSBC, the concept of the safe region is put forward. Within the safe region, DESS is stable and voltage deviation is acceptable. The boundary conditions of the safe region are derived from the equivalent model of DESS, in which stability is analyzed in terms of modified Brayton-Moser's criterion. Both simulations and hardware experiments verify the accuracy of the safe region and effectiveness of the proposed DSBC strategy.
基金This work is supported by‘The Fundamental Research Funds for the Central Universities(Grant No.HIT.NSRIF.201714)’‘Weihai Science and Technology Development Program(2016DXGJMS15)’‘Key Research and Development Program in Shandong Provincial(2017GGX90103)’.
文摘In distributed storage systems,file access efficiency has an important impact on the real-time nature of information forensics.As a popular approach to improve file accessing efficiency,prefetching model can fetches data before it is needed according to the file access pattern,which can reduce the I/O waiting time and increase the system concurrency.However,prefetching model needs to mine the degree of association between files to ensure the accuracy of prefetching.In the massive small file situation,the sheer volume of files poses a challenge to the efficiency and accuracy of relevance mining.In this paper,we propose a massive files prefetching model based on LSTM neural network with cache transaction strategy to improve file access efficiency.Firstly,we propose a file clustering algorithm based on temporal locality and spatial locality to reduce the computational complexity.Secondly,we propose a definition of cache transaction according to files occurrence in cache instead of time-offset distance based methods to extract file block feature accurately.Lastly,we innovatively propose a file access prediction algorithm based on LSTM neural network which predict the file that have high possibility to be accessed.Experiments show that compared with the traditional LRU and the plain grouping methods,the proposed model notably increase the cache hit rate and effectively reduces the I/O wait time.
基金Supported by the National Natural Science Foundation of China(61271174,61301178)
文摘We study the construction of minimum bandwidth regenerating code with combinatorial design. At first, a method of constructing minimum storage regenerating （MBR） codes is presented, which can tolerate only one-node failure. Then, we give examples to explain the code. Finally, we discuss the case of repairing multiple nodes, and analyze the performance with an example.
