计算流体动力学(computational fluid dynamics,CFD)是高性能计算重要应用领域之一,其计算涉及大量数据访问.在大规模并行计算情况下,串行I/O的性能与计算能力不匹配,I/O成为性能瓶颈.并行I/O是解决这一问题的主要途径之一.针对一个真...计算流体动力学(computational fluid dynamics,CFD)是高性能计算重要应用领域之一,其计算涉及大量数据访问.在大规模并行计算情况下,串行I/O的性能与计算能力不匹配,I/O成为性能瓶颈.并行I/O是解决这一问题的主要途径之一.针对一个真实多区结构网格CFD并行程序HOSTA(high-order simulator for aerodynamics),基于HDF5(hierarchical data format v5)数据存储格式及其并行I/O编程接口,实现了其主要数据的并行I/O.在一套有6个I/O服务器结点的高性能计算机系统上,采用实际CFD算例进行了性能测试.对一个三角翼算例,并行I/O相对于串行I/O的性能加速比达到21.27,最高获得5.81GBps的I/O吞吐率,并使程序整体性能提高10%以上;对一个网格规模更大的简单翼型算例,并行I/O最高获得了6.72GBps的I/O吞吐率.展开更多
Efficient real time data exchange over the Internet plays a crucial role in the successful application of web-based systems. In this paper, a data transfer mechanism over the Internet is proposed for real time web bas...Efficient real time data exchange over the Internet plays a crucial role in the successful application of web-based systems. In this paper, a data transfer mechanism over the Internet is proposed for real time web based applications. The mechanism incorporates the eXtensible Markup Language （XML） and Hierarchical Data Format （HDF） to provide a flexible and efficient data format. Heterogeneous transfer data is classified into light and heavy data, which are stored using XML and HDF respectively; the HDF data format is then mapped to Java Document Object Model （JDOM） objects in XML in the Java environment. These JDOM data objects are sent across computer networks with the support of the Java Remote Method Invocation （RMI） data transfer infrastructure. Client＇s defined data priority levels are implemented in RMI, which guides a server to transfer data objects at different priorities. A remote monitoring system for an industrial reactor process simulator is used as a case study to illustrate the proposed data transfer mechanism.展开更多
Scientific applications at exascale generate and analyze massive amounts of data.A critical requirement of these applications is the capability to access and manage this data efficiently on exascale systems.Parallel I...Scientific applications at exascale generate and analyze massive amounts of data.A critical requirement of these applications is the capability to access and manage this data efficiently on exascale systems.Parallel I/O,the key technology enables moving data between compute nodes and storage,faces monumental challenges from new applications,memory,and storage architectures considered in the designs of exascale systems.As the storage hierarchy is expanding to include node-local persistent memory,burst buffers,etc.,as well as disk-based storage,data movement among these layers must be efficient.Parallel I/O libraries of the future should be capable of handling file sizes of many terabytes and beyond.In this paper,we describe new capabilities we have developed in Hierarchical Data Format version 5(HDF5),the most popular parallel I/O library for scientific applications.HDF5 is one of the most used libraries at the leadership computing facilities for performing parallel I/O on existing HPC systems.The state-of-the-art features we describe include:Virtual Object Layer(VOL),Data Elevator,asynchronous I/O,full-featured single-writer and multiple-reader(Full SWMR),and parallel querying.In this paper,we introduce these features,their implementations,and the performance and feature benefits to applications and other libraries.展开更多
文摘计算流体动力学(computational fluid dynamics,CFD)是高性能计算重要应用领域之一,其计算涉及大量数据访问.在大规模并行计算情况下,串行I/O的性能与计算能力不匹配,I/O成为性能瓶颈.并行I/O是解决这一问题的主要途径之一.针对一个真实多区结构网格CFD并行程序HOSTA(high-order simulator for aerodynamics),基于HDF5(hierarchical data format v5)数据存储格式及其并行I/O编程接口,实现了其主要数据的并行I/O.在一套有6个I/O服务器结点的高性能计算机系统上,采用实际CFD算例进行了性能测试.对一个三角翼算例,并行I/O相对于串行I/O的性能加速比达到21.27,最高获得5.81GBps的I/O吞吐率,并使程序整体性能提高10%以上;对一个网格规模更大的简单翼型算例,并行I/O最高获得了6.72GBps的I/O吞吐率.
文摘Efficient real time data exchange over the Internet plays a crucial role in the successful application of web-based systems. In this paper, a data transfer mechanism over the Internet is proposed for real time web based applications. The mechanism incorporates the eXtensible Markup Language （XML） and Hierarchical Data Format （HDF） to provide a flexible and efficient data format. Heterogeneous transfer data is classified into light and heavy data, which are stored using XML and HDF respectively; the HDF data format is then mapped to Java Document Object Model （JDOM） objects in XML in the Java environment. These JDOM data objects are sent across computer networks with the support of the Java Remote Method Invocation （RMI） data transfer infrastructure. Client＇s defined data priority levels are implemented in RMI, which guides a server to transfer data objects at different priorities. A remote monitoring system for an industrial reactor process simulator is used as a case study to illustrate the proposed data transfer mechanism.
基金This research was supported by the Exascale Computing Project under Grant No.17-SC-20-SCa joint project of the U.S.Department of Energy's Office of Science and National Nuclear Security Administration,responsible for delivering a capable exascale ecosystem,including software,applications,and hardware technology,to support the nation's exascale computing imperative+4 种基金This work is also supported by the Director,Office of Science,Office of Advanced Scientific Computing Research,of the U.S.Department of Energy under Contract Nos.DE-AC02-05CH11231 and DE-AC02-06CH11357This research was funded in part by the Argonne Leadership Computing Facilitywhich is a DOE Office of Science User Facility supported under Contract No.DE-AC02-06CH11357This research used resources of the National Energy Research Scientific Computing Centerwhich is DOE Office of Science User Facilities supported by the Office of Science of the U.S.Department of Energy under Contract No.DE-AC02-05CH11231.
文摘Scientific applications at exascale generate and analyze massive amounts of data.A critical requirement of these applications is the capability to access and manage this data efficiently on exascale systems.Parallel I/O,the key technology enables moving data between compute nodes and storage,faces monumental challenges from new applications,memory,and storage architectures considered in the designs of exascale systems.As the storage hierarchy is expanding to include node-local persistent memory,burst buffers,etc.,as well as disk-based storage,data movement among these layers must be efficient.Parallel I/O libraries of the future should be capable of handling file sizes of many terabytes and beyond.In this paper,we describe new capabilities we have developed in Hierarchical Data Format version 5(HDF5),the most popular parallel I/O library for scientific applications.HDF5 is one of the most used libraries at the leadership computing facilities for performing parallel I/O on existing HPC systems.The state-of-the-art features we describe include:Virtual Object Layer(VOL),Data Elevator,asynchronous I/O,full-featured single-writer and multiple-reader(Full SWMR),and parallel querying.In this paper,we introduce these features,their implementations,and the performance and feature benefits to applications and other libraries.
