摘要
Lack of rigorous reproducibility and validation are significant hurdles for scientific development across many fields.Materials science,in particular,encompasses a variety of experimental and theoretical approaches that require careful benchmarking.Leaderboard efforts have been developed previously to mitigate these issues.However,a comprehensive comparison and benchmarking on an integrated platform with multiple data modalities with perfect and defect materials data is still lacking.This work introduces JARVIS-Leaderboard,an open-source and community-driven platform that facilitates benchmarking and enhances reproducibility.The platform allows users to set up benchmarks with customtasks and enables contributions in the form of dataset,code,and meta-data submissions.We cover the following materials design categories:Artificial Intelligence(AI),Electronic Structure(ES).
基金
supported by the financial assistance award 70NANB19H117 from the U.S.Department of Commerce,National Institute of Standards and Technology
supported by the U.S.Department of Energy,Office of Science,Basic Energy Sciences,Materials Sciences and Engineering Division,as part of the Computational Materials Sciences Program and Center for Predictive Simulation of Functional Materials
supported by the Center for Nanophase Materials Sciences,which is a US Department of Energy,Office of Science User Facility at Oak Ridge National Laboratory
AHR thanks the Supercomputer Center and San Diego Supercomputer Center through allocation DMR140031 from the Advanced Cyberinfrastructure Coordination Ecosystem:Services&Support(ACCESS)program,which is supported by National Science Foundation grants#2138259,#2138286,#2138307,#2137603,and#2138296
supported by NIST award 70NANB19H005 and NSF award CMMI-2053929
S.H.W.especially thanks to the NSF Non-Academic Research Internships for Graduate Students(INTERN)program(CBET-1845531)for supporting part of the work in NIST under the guidance of K.C
A.M.K.acknowledges support from the School of Materials Engineering at Purdue University under startup account F.10023800.05.002
support by the Federal Ministry of Education and Research(BMBF)under Grant No.01DM21001B(German-Canadian Materials Acceleration Center).
