Purpose The LHAASO project collects trillions of cosmic ray events every year,generating about 10 PB of raw data annually,which brings big challenges for data processing platform.Method The LHAASO data processing plat...Purpose The LHAASO project collects trillions of cosmic ray events every year,generating about 10 PB of raw data annually,which brings big challenges for data processing platform.Method The LHAASO data processing platform is built to handle such a large amount of data,which is composed of some subsystems such as data transfer,data storage,high throughput computing and metadata management.Results and conclusions The platform was under construction since 2018 and has been working well since 2021.In this paper,the details of the design,implementation and performance of the data processing platform are presented.展开更多
Continuous global-scale mapping of human settlements in the service of international agreements calls for massive volume of multi-source,multi-temporal,and multi-scale earth observation data.In this paper,the latest d...Continuous global-scale mapping of human settlements in the service of international agreements calls for massive volume of multi-source,multi-temporal,and multi-scale earth observation data.In this paper,the latest developments in terms of processing big earth observation data for the purpose of improving the Global Human Settlement Layer(GHSL)data are presented.Two experiments with Sentinel-1 and Landsat data collections were run leveraging on the Joint Research Centre Earth Observation Data and Processing Platform.A comparative analysis of the results of built-up areas extraction from different remote sensing data and processing workflows shows how the information production supported by data-intensive computing infrastructure for optimization and multiple testing can improve the output information reliability and consistency within the GHSL scope.The paper presents the processing workflows and the results of the two main experiments,giving insights into the enhanced mapping capabilities gained by analyzing Sentinel-1 and Landsat data-sets,and the lessons learnt in terms of handling and processing big earth observation data.展开更多
基金supported by National Nature Science Foundation of China(GrantNos.12075268,12175255,12175258,12105300)the Chinese Academy of Science,Institute of High Energy Physics.
文摘Purpose The LHAASO project collects trillions of cosmic ray events every year,generating about 10 PB of raw data annually,which brings big challenges for data processing platform.Method The LHAASO data processing platform is built to handle such a large amount of data,which is composed of some subsystems such as data transfer,data storage,high throughput computing and metadata management.Results and conclusions The platform was under construction since 2018 and has been working well since 2021.In this paper,the details of the design,implementation and performance of the data processing platform are presented.
基金This work is supported by two administrative arrangements with the Directorate General of Internal Market,Industry,Entrepreneurship and SME’s(GROWTH)and the Directorate General for Regional and Urban Policy of the European Commission(REGIO).
文摘Continuous global-scale mapping of human settlements in the service of international agreements calls for massive volume of multi-source,multi-temporal,and multi-scale earth observation data.In this paper,the latest developments in terms of processing big earth observation data for the purpose of improving the Global Human Settlement Layer(GHSL)data are presented.Two experiments with Sentinel-1 and Landsat data collections were run leveraging on the Joint Research Centre Earth Observation Data and Processing Platform.A comparative analysis of the results of built-up areas extraction from different remote sensing data and processing workflows shows how the information production supported by data-intensive computing infrastructure for optimization and multiple testing can improve the output information reliability and consistency within the GHSL scope.The paper presents the processing workflows and the results of the two main experiments,giving insights into the enhanced mapping capabilities gained by analyzing Sentinel-1 and Landsat data-sets,and the lessons learnt in terms of handling and processing big earth observation data.
