With the popularity of social network, the de- mand for real-time processing of graph data is increasing. However, most of the existing graph systems adopt a batch processing mode, therefore the overhead of maintainin...With the popularity of social network, the de- mand for real-time processing of graph data is increasing. However, most of the existing graph systems adopt a batch processing mode, therefore the overhead of maintaining and processing of dynamic graph is significantly high. In this pa- per, we design iGraph, an incremental graph processing sys- tem for dynamic graph with its continuous updates. The con- tribufions of iGraph include： 1） a hash-based graph partition strategy to enable fine-grained graph updates; 2） a vertex- based graph computing model to support incremental data processing; 3） detection and rebalance methods of hotspot to address the workload imbalance problem during incre- mental processing. Through the general-purpose API, iGraph can be used to implement various graph processing algo- rithms such as PageRank. We have implemented iGraph on Apache Spark, and experimental results show that for real life datasets, iGraph outperforms the original GraphX in respect of graph update and graph computation.展开更多
The rapid growth of the Internet of Things(IoTs)has resulted in an explosive increase in data,and thus has raised new challenges for data processing units.Edge computing,which settles signal processing and computing t...The rapid growth of the Internet of Things(IoTs)has resulted in an explosive increase in data,and thus has raised new challenges for data processing units.Edge computing,which settles signal processing and computing tasks at the edge of networks rather than uploading data to the cloud,can reduce the amount of data for transmission and is a promising solution to address the challenges.One of the potential candidates for edge computing is a memristor,an emerging nonvolatile memory device that has the capability of in-memory computing.In this article,from the perspective of edge computing,we review recent progress on memristor-based signal processing methods,especially on the aspects of signal preprocessing and feature extraction.Then,we describe memristor-based signal classification and regression,and end-to-end signal processing.In all these applications,memristors serve as critical accelerators to greatly improve the overall system performance,such as power efficiency and processing speed.Finally,we discuss existing challenges and future outlooks for memristor-based signal processing systems.展开更多
With the rapid growth of computer science and big data,the traditional von Neumann architecture suffers the aggravating data communication costs due to the separated structure of the processing units and memories.Memr...With the rapid growth of computer science and big data,the traditional von Neumann architecture suffers the aggravating data communication costs due to the separated structure of the processing units and memories.Memristive in-memory computing paradigm is considered as a prominent candidate to address these issues,and plentiful applications have been demonstrated and verified.These applications can be broadly categorized into two major types:soft computing that can tolerant uncertain and imprecise results,and hard computing that emphasizes explicit and precise numerical results for each task,leading to different requirements on the computational accuracies and the corresponding hardware solutions.In this review,we conduct a thorough survey of the recent advances of memristive in-memory computing applications,both on the soft computing type that focuses on artificial neural networks and other machine learning algorithms,and the hard computing type that includes scientific computing and digital image processing.At the end of the review,we discuss the remaining challenges and future opportunities of memristive in-memory computing in the incoming Artificial Intelligence of Things era.展开更多
The current DSA system used in the dispatching control centers in China is a near real-time analysis system with response speed in the order of minutes.Based on a review of the state-of-the-art in online analysis and ...The current DSA system used in the dispatching control centers in China is a near real-time analysis system with response speed in the order of minutes.Based on a review of the state-of-the-art in online analysis and discussion of distributed data processing and computation architecture patterns,a new online analysis architecture is proposed.The primary goal of the new architecture is to increase the online analysis response speed to the order of seconds.A reference implementation of the proposed online analysis architecture to validate the feasibility of implementing the architecture and some performance testing results are presented.展开更多
Approaches to apply graph computing to power grid analysis are systematically explained using real-world application examples.Through exploring the nature of the power grid and the characteristics of power grid analys...Approaches to apply graph computing to power grid analysis are systematically explained using real-world application examples.Through exploring the nature of the power grid and the characteristics of power grid analysis,the guidelines for selecting appropriate graph computing techniques for the application to power grid analysis are outlined.A custom graph model for representing the power grid for the analysis and simulation purpose and an in-memory computing(IMC)based graph-centric approach with a shared-everything architecture are introduced.Graph algorithms,including network topology processing and subgraph processing,and graph computing application scenarios,including in-memory computing,contingency analysis,and Common Information Model(CIM)model merge,are presented.展开更多
文摘With the popularity of social network, the de- mand for real-time processing of graph data is increasing. However, most of the existing graph systems adopt a batch processing mode, therefore the overhead of maintaining and processing of dynamic graph is significantly high. In this pa- per, we design iGraph, an incremental graph processing sys- tem for dynamic graph with its continuous updates. The con- tribufions of iGraph include： 1） a hash-based graph partition strategy to enable fine-grained graph updates; 2） a vertex- based graph computing model to support incremental data processing; 3） detection and rebalance methods of hotspot to address the workload imbalance problem during incre- mental processing. Through the general-purpose API, iGraph can be used to implement various graph processing algo- rithms such as PageRank. We have implemented iGraph on Apache Spark, and experimental results show that for real life datasets, iGraph outperforms the original GraphX in respect of graph update and graph computation.
基金supported in part by the National Science and Technology Major Project of China(No.2017ZX02315001-005)the National Natural Science Foundation of China(Nos.91964104 and 61974081)。
文摘The rapid growth of the Internet of Things(IoTs)has resulted in an explosive increase in data,and thus has raised new challenges for data processing units.Edge computing,which settles signal processing and computing tasks at the edge of networks rather than uploading data to the cloud,can reduce the amount of data for transmission and is a promising solution to address the challenges.One of the potential candidates for edge computing is a memristor,an emerging nonvolatile memory device that has the capability of in-memory computing.In this article,from the perspective of edge computing,we review recent progress on memristor-based signal processing methods,especially on the aspects of signal preprocessing and feature extraction.Then,we describe memristor-based signal classification and regression,and end-to-end signal processing.In all these applications,memristors serve as critical accelerators to greatly improve the overall system performance,such as power efficiency and processing speed.Finally,we discuss existing challenges and future outlooks for memristor-based signal processing systems.
基金This work was financially supported by the National Key R&D Program of China(Nos.2019YFB2205100 and 2021ZD0201201)the National Natural Science Foundation of China(Grant Nos.92064012 and 61874164).
文摘With the rapid growth of computer science and big data,the traditional von Neumann architecture suffers the aggravating data communication costs due to the separated structure of the processing units and memories.Memristive in-memory computing paradigm is considered as a prominent candidate to address these issues,and plentiful applications have been demonstrated and verified.These applications can be broadly categorized into two major types:soft computing that can tolerant uncertain and imprecise results,and hard computing that emphasizes explicit and precise numerical results for each task,leading to different requirements on the computational accuracies and the corresponding hardware solutions.In this review,we conduct a thorough survey of the recent advances of memristive in-memory computing applications,both on the soft computing type that focuses on artificial neural networks and other machine learning algorithms,and the hard computing type that includes scientific computing and digital image processing.At the end of the review,we discuss the remaining challenges and future opportunities of memristive in-memory computing in the incoming Artificial Intelligence of Things era.
基金This work was supported by the State Grid of China under the“Thousand Talents Plan”special research grant(5206001600A3).
文摘The current DSA system used in the dispatching control centers in China is a near real-time analysis system with response speed in the order of minutes.Based on a review of the state-of-the-art in online analysis and discussion of distributed data processing and computation architecture patterns,a new online analysis architecture is proposed.The primary goal of the new architecture is to increase the online analysis response speed to the order of seconds.A reference implementation of the proposed online analysis architecture to validate the feasibility of implementing the architecture and some performance testing results are presented.
基金supported by National Natural Science Foundation of China under the Grant U1766214.
文摘Approaches to apply graph computing to power grid analysis are systematically explained using real-world application examples.Through exploring the nature of the power grid and the characteristics of power grid analysis,the guidelines for selecting appropriate graph computing techniques for the application to power grid analysis are outlined.A custom graph model for representing the power grid for the analysis and simulation purpose and an in-memory computing(IMC)based graph-centric approach with a shared-everything architecture are introduced.Graph algorithms,including network topology processing and subgraph processing,and graph computing application scenarios,including in-memory computing,contingency analysis,and Common Information Model(CIM)model merge,are presented.
