Lack of rigorous reproducibility and validation are significant hurdles for scientific development across many fields.Materials science,in particular,encompasses a variety of experimental and theoretical approaches th...Lack of rigorous reproducibility and validation are significant hurdles for scientific development across many fields.Materials science,in particular,encompasses a variety of experimental and theoretical approaches that require careful benchmarking.Leaderboard efforts have been developed previously to mitigate these issues.However,a comprehensive comparison and benchmarking on an integrated platform with multiple data modalities with perfect and defect materials data is still lacking.This work introduces JARVIS-Leaderboard,an open-source and community-driven platform that facilitates benchmarking and enhances reproducibility.The platform allows users to set up benchmarks with customtasks and enables contributions in the form of dataset,code,and meta-data submissions.We cover the following materials design categories:Artificial Intelligence(AI),Electronic Structure(ES).展开更多
基金supported by the financial assistance award 70NANB19H117 from the U.S.Department of Commerce,National Institute of Standards and Technologysupported by the U.S.Department of Energy,Office of Science,Basic Energy Sciences,Materials Sciences and Engineering Division,as part of the Computational Materials Sciences Program and Center for Predictive Simulation of Functional Materials+5 种基金supported by the Center for Nanophase Materials Sciences,which is a US Department of Energy,Office of Science User Facility at Oak Ridge National LaboratoryAHR thanks the Supercomputer Center and San Diego Supercomputer Center through allocation DMR140031 from the Advanced Cyberinfrastructure Coordination Ecosystem:Services&Support(ACCESS)program,which is supported by National Science Foundation grants#2138259,#2138286,#2138307,#2137603,and#2138296supported by NIST award 70NANB19H005 and NSF award CMMI-2053929S.H.W.especially thanks to the NSF Non-Academic Research Internships for Graduate Students(INTERN)program(CBET-1845531)for supporting part of the work in NIST under the guidance of K.CA.M.K.acknowledges support from the School of Materials Engineering at Purdue University under startup account F.10023800.05.002support by the Federal Ministry of Education and Research(BMBF)under Grant No.01DM21001B(German-Canadian Materials Acceleration Center).
文摘Lack of rigorous reproducibility and validation are significant hurdles for scientific development across many fields.Materials science,in particular,encompasses a variety of experimental and theoretical approaches that require careful benchmarking.Leaderboard efforts have been developed previously to mitigate these issues.However,a comprehensive comparison and benchmarking on an integrated platform with multiple data modalities with perfect and defect materials data is still lacking.This work introduces JARVIS-Leaderboard,an open-source and community-driven platform that facilitates benchmarking and enhances reproducibility.The platform allows users to set up benchmarks with customtasks and enables contributions in the form of dataset,code,and meta-data submissions.We cover the following materials design categories:Artificial Intelligence(AI),Electronic Structure(ES).
