Sentiment analysis,a cornerstone of natural language processing,has witnessed remarkable advancements driven by deep learning models which demonstrated impressive accuracy in discerning sentiment from text across vari...Sentiment analysis,a cornerstone of natural language processing,has witnessed remarkable advancements driven by deep learning models which demonstrated impressive accuracy in discerning sentiment from text across various domains.However,the deployment of such models in resource-constrained environments presents a unique set of challenges that require innovative solutions.Resource-constrained environments encompass scenarios where computing resources,memory,and energy availability are restricted.To empower sentiment analysis in resource-constrained environments,we address the crucial need by leveraging lightweight pre-trained models.These models,derived from popular architectures such as DistilBERT,MobileBERT,ALBERT,TinyBERT,ELECTRA,and SqueezeBERT,offer a promising solution to the resource limitations imposed by these environments.By distilling the knowledge from larger models into smaller ones and employing various optimization techniques,these lightweight models aim to strike a balance between performance and resource efficiency.This paper endeavors to explore the performance of multiple lightweight pre-trained models in sentiment analysis tasks specific to such environments and provide insights into their viability for practical deployment.展开更多
Solid electrolytes with fast ion transport are crucial for solid state lithium metal batteries.Chemical doping has been the most effective strategy for improving ion condictiviy,and atomistic simulation with machine-l...Solid electrolytes with fast ion transport are crucial for solid state lithium metal batteries.Chemical doping has been the most effective strategy for improving ion condictiviy,and atomistic simulation with machine-learning potentials helps optimize doping by predicting ion conductivity for various composition.Yet most existing machine-learning models are trained on narrow chemistry,requiring retraining for each new system,which wastes transferable knowledge and incurs significant cost.Here,we propose a pre-trained deep potential model purpose-built for sulfide solid electrolytes with attention mechanism,known as DPA-SSE.The training set includes 15 elements and consists of both equilibrium and extensive out-of-equilibrium configurations.DPA-SSE achieves a high energy resolution of less than 2 meV/atom for dynamical trajectories up to 1150 K,and reproduces experimental ion conductivity with remarkable accuracy.DPA-SSE generalizes well to complex electrolytes with mixes of cation and anion atoms,and enables highly efficient dynamical simulation via model distillation.DPA-SSE also serves as a platform for continuous learning and can be fine-tuned with minimal downstream data.These results demonstrate the possibility of a new pathway for the AIdriven development of solid electrolytes with exceptional performance.展开更多
文摘Sentiment analysis,a cornerstone of natural language processing,has witnessed remarkable advancements driven by deep learning models which demonstrated impressive accuracy in discerning sentiment from text across various domains.However,the deployment of such models in resource-constrained environments presents a unique set of challenges that require innovative solutions.Resource-constrained environments encompass scenarios where computing resources,memory,and energy availability are restricted.To empower sentiment analysis in resource-constrained environments,we address the crucial need by leveraging lightweight pre-trained models.These models,derived from popular architectures such as DistilBERT,MobileBERT,ALBERT,TinyBERT,ELECTRA,and SqueezeBERT,offer a promising solution to the resource limitations imposed by these environments.By distilling the knowledge from larger models into smaller ones and employing various optimization techniques,these lightweight models aim to strike a balance between performance and resource efficiency.This paper endeavors to explore the performance of multiple lightweight pre-trained models in sentiment analysis tasks specific to such environments and provide insights into their viability for practical deployment.
基金supported in part by the National Science and Technology Major Project(Grants No.2023ZD0120702)Key Research Program of Frontier Sciences of CAS(Grant No.ZDBS-LY-SLH008)National Nature Science Foundation of China(Grants No.12304049).We thank Bowen Deng and Dr.Peichen Zhong,the authors of CHGNet,for inspiring discussions.We also appreciate valuable advice by Dr.Qisheng Wu.The computational resource was supported by the Bohrium Cloud Platform at DP technology.
文摘Solid electrolytes with fast ion transport are crucial for solid state lithium metal batteries.Chemical doping has been the most effective strategy for improving ion condictiviy,and atomistic simulation with machine-learning potentials helps optimize doping by predicting ion conductivity for various composition.Yet most existing machine-learning models are trained on narrow chemistry,requiring retraining for each new system,which wastes transferable knowledge and incurs significant cost.Here,we propose a pre-trained deep potential model purpose-built for sulfide solid electrolytes with attention mechanism,known as DPA-SSE.The training set includes 15 elements and consists of both equilibrium and extensive out-of-equilibrium configurations.DPA-SSE achieves a high energy resolution of less than 2 meV/atom for dynamical trajectories up to 1150 K,and reproduces experimental ion conductivity with remarkable accuracy.DPA-SSE generalizes well to complex electrolytes with mixes of cation and anion atoms,and enables highly efficient dynamical simulation via model distillation.DPA-SSE also serves as a platform for continuous learning and can be fine-tuned with minimal downstream data.These results demonstrate the possibility of a new pathway for the AIdriven development of solid electrolytes with exceptional performance.
