二语写作是二语习得研究领域的重要组成部分。运用CiteSpace软件对近十年发表在Journal of Second Language Writing的231篇实证研究论文进行可视化分析,研究发现:二语写作研究整体呈波动性上升趋势,研究规模较为稳定,研究关注度逐渐提...二语写作是二语习得研究领域的重要组成部分。运用CiteSpace软件对近十年发表在Journal of Second Language Writing的231篇实证研究论文进行可视化分析,研究发现:二语写作研究整体呈波动性上升趋势,研究规模较为稳定,研究关注度逐渐提升;二语写作研究领域暂未形成明显的核心作者和机构的合作网络;研究主题主要聚焦二语写作教学方法的多元化、二语写作反馈的多焦点、二语写作评估与测试的科学化,以及学习者个体差异的多维影响等方面。基于此,提出未来该领域发展需加强学者、机构之间的相互合作;关注个体学习者写作过程的认知特征与情感因素,尤其重视青少年二语学习过程的研究;扩大二语写作纵向研究规模,推动研究的深入发展。展开更多
Large language models(LLMs)have undergone significant expansion and have been increasingly integrated across various domains.Notably,in the realm of robot task planning,LLMs harness their advanced reasoning and langua...Large language models(LLMs)have undergone significant expansion and have been increasingly integrated across various domains.Notably,in the realm of robot task planning,LLMs harness their advanced reasoning and language comprehension capabilities to formulate precise and efficient action plans based on natural language instructions.However,for embodied tasks,where robots interact with complex environments,textonly LLMs often face challenges due to a lack of compatibility with robotic visual perception.This study provides a comprehensive overview of the emerging integration of LLMs and multimodal LLMs into various robotic tasks.Additionally,we propose a framework that utilizes multimodal GPT-4V to enhance embodied task planning through the combination of natural language instructions and robot visual perceptions.Our results,based on diverse datasets,indicate that GPT-4V effectively enhances robot performance in embodied tasks.This extensive survey and evaluation of LLMs and multimodal LLMs across a variety of robotic tasks enriches the understanding of LLM-centric embodied intelligence and provides forward-looking insights towards bridging the gap in Human-Robot-Environment interaction.展开更多
Objective:This study aimed to examine the reliability and validity of the Chinese version of the Behavioral Inhibition System/Behavioral Activation System(BIS/BAS)scales among stroke survivors.Methods:The cross-sectio...Objective:This study aimed to examine the reliability and validity of the Chinese version of the Behavioral Inhibition System/Behavioral Activation System(BIS/BAS)scales among stroke survivors.Methods:The cross-sectional study was conducted at four comprehensive hospitals in Taizhou,Jiangsu,China.A sample of 232 first-ever stroke survivors were recruited from June to August 2023.Validity was examined using face validity and construct validity,which used confirmatory factor analysis(CFA)and known-group analysis.Reliability was evaluated by internal consistency and test-retest reliability.Results:The BIS/BAS scales demonstrated satisfactory face validity.The findings of CFAs supported the original four-factor structure of BAS-reward,BAS-drive,BAS-fun seeking,and BIS with acceptable model fit indices.Discriminative validity,assessed via known-group analysis,indicated that stroke survivors with probable depression had significantly lower mean BAS-reward,BAS-drive,and BAS-fun seeking scores(P<0.001)and a higher mean BIS score(P=0.028)compared to those without probable depression.The internal consistency,measured by Cronbach’s a coefficients for the subscales,ranged from 0.669 to 0.964.Test-retest reliability,assessed using intra-class correlation coefficients,ranged from 0.61 to 0.93.Conclusions:The Chinese version of the BIS/BAS scales could be a reliable and valid instrument for measuring behavioral activation among stroke survivors.展开更多
Since the pioneering work by Broca and Wernicke in the 19th century,who examined individuals with brain lesions to associate them with specific behaviors,it was evident that behaviors are complex and cannot be fully a...Since the pioneering work by Broca and Wernicke in the 19th century,who examined individuals with brain lesions to associate them with specific behaviors,it was evident that behaviors are complex and cannot be fully attributable to specific brain areas alone.Instead,they involve connectivity among brain areas,whether close or distant.At that time,this approach was considered the optimal way to dissect brain circuitry and function.These pioneering efforts opened the field to explore the necessity or sufficiency of brain areas in controlling behavior and hence dissecting brain function.However,the connectivity of the brain and the mechanisms through which various brain regions regulate specific behaviors,either individually or collaboratively,remain largely elusive.Utilizing animal models,researchers have endeavored to unravel the necessity or sufficiency of specific brain areas in influencing behavior;however,no clear associations have been firmly established.展开更多
Purpose:Evaluating the quality of academic journal articles is a time consuming but critical task for national research evaluation exercises,appointments and promotion.It is therefore important to investigate whether ...Purpose:Evaluating the quality of academic journal articles is a time consuming but critical task for national research evaluation exercises,appointments and promotion.It is therefore important to investigate whether Large Language Models(LLMs)can play a role in this process.Design/methodology/approach:This article assesses which ChatGPT inputs(full text without tables,figures,and references;title and abstract;title only)produce better quality score estimates,and the extent to which scores are affected by ChatGPT models and system prompts.Findings:The optimal input is the article title and abstract,with average ChatGPT scores based on these(30 iterations on a dataset of 51 papers)correlating at 0.67 with human scores,the highest ever reported.ChatGPT 4o is slightly better than 3.5-turbo(0.66),and 4o-mini(0.66).Research limitations:The data is a convenience sample of the work of a single author,it only includes one field,and the scores are self-evaluations.Practical implications:The results suggest that article full texts might confuse LLM research quality evaluations,even though complex system instructions for the task are more effective than simple ones.Thus,whilst abstracts contain insufficient information for a thorough assessment of rigour,they may contain strong pointers about originality and significance.Finally,linear regression can be used to convert the model scores into the human scale scores,which is 31%more accurate than guessing.Originality/value:This is the first systematic comparison of the impact of different prompts,parameters and inputs for ChatGPT research quality evaluations.展开更多
The integration of artificial intelligence(AI)technology,particularly large language models(LLMs),has become essential across various sectors due to their advanced language comprehension and generation capabilities.De...The integration of artificial intelligence(AI)technology,particularly large language models(LLMs),has become essential across various sectors due to their advanced language comprehension and generation capabilities.Despite their transformative impact in fields such as machine translation and intelligent dialogue systems,LLMs face significant challenges.These challenges include safety,security,and privacy concerns that undermine their trustworthiness and effectiveness,such as hallucinations,backdoor attacks,and privacy leakage.Previous works often conflated safety issues with security concerns.In contrast,our study provides clearer and more reasonable definitions for safety,security,and privacy within the context of LLMs.Building on these definitions,we provide a comprehensive overview of the vulnerabilities and defense mechanisms related to safety,security,and privacy in LLMs.Additionally,we explore the unique research challenges posed by LLMs and suggest potential avenues for future research,aiming to enhance the robustness and reliability of LLMs in the face of emerging threats.展开更多
Magnesium matrix composites with both high strength and ductility have been achieved by introducing pure Ti particles.However,the properties of the surfaces of the composites need to be improved by surface technology,...Magnesium matrix composites with both high strength and ductility have been achieved by introducing pure Ti particles.However,the properties of the surfaces of the composites need to be improved by surface technology,such as micro-arc oxidation(MAO).In this study,we investigated the influence of the Ti-reinforcement phase on coating growth and evolution by subjecting both AZ91 alloy and AZ91/Ti composite to MAO treatment using silicate-based and phosphate-based electrolytes.Results revealed that the Ti-reinforcement phase influenced the MAO process,altering discharge behavior,and leading to a decreased cell voltage.The vigorous discharge of the Ti-reinforcement phase induced the formation of coating discharge channels,concurrently dissolving and oxidizing Ti-reinforcement to produce a composite ceramic coating with TiO2.The MAO coating on the AZ91/Ti composite exhibited a dark blue macromorphology and distinctive local micromorphological anomalies.In silicate electrolyte,a“volcano-like”localized morphology centered on the discharge channel emerged.In contrast,treatment in phosphate-based electrolyte resulted in a coating morphology similar to typical porous ceramic coatings,with visible radial discharge micropores at the reinforcement phase location.Compared to the AZ91 alloy,the coating on the AZ91/Ti composite exhibited lower thickness and higher porosity.MAO treatment reduced the self-corrosion current density of the AZ91/Ti surface by two orders of magnitude.The silicate coating demonstrated better corrosion resistance than the phosphate coating,attributed to its lower porosity.The formation mechanism of MAO coatings on AZ91/Ti composites in phosphate-based and silicate-based electrolytes was proposed.展开更多
The performance of Mg alloys is significantly influenced by the concentrations and solid solution behavior of the alloying elements.In this work,the solid solution behavior of 20 alloying elements in 190 ternary Mg al...The performance of Mg alloys is significantly influenced by the concentrations and solid solution behavior of the alloying elements.In this work,the solid solution behavior of 20 alloying elements in 190 ternary Mg alloy systems at 500℃are systematically investigated.The solid solution behavior of a set of two different alloying elements in Mg alloy systems are suggested to be classified into three categories:inclusivity,exclusivity and proportionality.Inclusivity classification indicates that the two alloying elements are inclusive inα-Mg,increasing the joint solubility of both elements.Exclusivity classification suggests that the two alloying elements have a low joint solid solubility inα-Mg,since they prefer to form stable second phases.For the proportionality classification,the solubility curve of the ternary Mg alloy systems is a straight line connecting the solubility points of the two sub-binary systems.The proposed classification theory was validated by key experiments and the calculation of formation energies.The interaction effects between alloying elements and the preference of formation of second phases are the main factors determining the solid solution behavior classifications.Based on the observed solid solution features of multi-component Mg alloys,principles for alloy design of different types of high-performance Mg alloys were proposed in this work.展开更多
Recently,diffusion models have emerged as a promising paradigm for molecular design and optimization.However,most diffusion-based molecular generative models focus on modeling 2D graphs or 3D geom-etries,with limited ...Recently,diffusion models have emerged as a promising paradigm for molecular design and optimization.However,most diffusion-based molecular generative models focus on modeling 2D graphs or 3D geom-etries,with limited research on molecular sequence diffusion models.The International Union of Pure and Applied Chemistry(IUPAC)names are more akin to chemical natural language than the simplified molecular input line entry system(SMILES)for organic compounds.In this work,we apply an IUPAC-guided conditional diffusion model to facilitate molecular editing from chemical natural language to chemical language(SMILES)and explore whether the pre-trained generative performance of diffusion models can be transferred to chemical natural language.We propose DiffIUPAC,a controllable molecular editing diffusion model that converts IUPAC names to SMILES strings.Evaluation results demonstrate that our model out-performs existing methods and successfully captures the semantic rules of both chemical languages.Chemical space and scaffold analysis show that the model can generate similar compounds with diverse scaffolds within the specified constraints.Additionally,to illustrate the model’s applicability in drug design,we conducted case studies in functional group editing,analogue design and linker design.展开更多
Efficient lubrication of magnesium alloys is a highly challenging topic in the field of tribology.In this study,magnesium silicate hydroxide(MSH)nanotubes with serpentine structures were synthesized.The tribological b...Efficient lubrication of magnesium alloys is a highly challenging topic in the field of tribology.In this study,magnesium silicate hydroxide(MSH)nanotubes with serpentine structures were synthesized.The tribological behavior of AZ91D magnesium alloy rubbed against GCr15 steel was studied under lubricating oil with surface-modified MSH nanotubes as additives.The effects of the concentration,applied load,and reciprocating frequency on the friction and wear of the AZ91D alloy were studied using an SRV-4 sliding wear tester.Results show a decrease of 18.7–68.5%in friction coefficient,and a reduction of 19.4–54.3%in wear volume of magnesium alloy can be achieved by applying the synthetic serpentine additive under different conditions.A suspension containing 0.3 wt.%MSH was most efficient in reducing wear and friction.High frequency and medium load were more conducive to improving the tribological properties of magnesium alloys.A series of beneficial physical and chemical processes occurring at the AZ91D alloy/steel interface can be used to explain friction and wear reduction based on the characterization of the morphology,chemical composition,chemical state,microstructure,and nanomechanical properties of the worn surface.The synthetic MSH,with serpentine structure and nanotube morphology,possesses excellent adsorbability,high chemical activity,and good self-lubrication and catalytic activity.Therefore,physical polishing,tribochemical reactions,and physicalchemical depositions can occur easily on the sliding contacts.A dense tribolayer with a complex composition and composite structure was formed on the worn surface.Its high hardness,good toughness and plasticity,and prominent lubricity resulted in the improvement of friction and wear,making the synthetic MSH a promising efficient oil additive for magnesium alloys under boundary and mixed lubrication.展开更多
Software security poses substantial risks to our society because software has become part of our life. Numerous techniques have been proposed to resolve or mitigate the impact of software security issues. Among them, ...Software security poses substantial risks to our society because software has become part of our life. Numerous techniques have been proposed to resolve or mitigate the impact of software security issues. Among them, software testing and analysis are two of the critical methods, which significantly benefit from the advancements in deep learning technologies. Due to the successful use of deep learning in software security, recently,researchers have explored the potential of using large language models(LLMs) in this area. In this paper, we systematically review the results focusing on LLMs in software security. We analyze the topics of fuzzing, unit test, program repair, bug reproduction, data-driven bug detection, and bug triage. We deconstruct these techniques into several stages and analyze how LLMs can be used in the stages. We also discuss the future directions of using LLMs in software security, including the future directions for the existing use of LLMs and extensions from conventional deep learning research.展开更多
ChatGPT is a powerful artificial intelligence(AI)language model that has demonstrated significant improvements in various natural language processing(NLP) tasks. However, like any technology, it presents potential sec...ChatGPT is a powerful artificial intelligence(AI)language model that has demonstrated significant improvements in various natural language processing(NLP) tasks. However, like any technology, it presents potential security risks that need to be carefully evaluated and addressed. In this survey, we provide an overview of the current state of research on security of using ChatGPT, with aspects of bias, disinformation, ethics, misuse,attacks and privacy. We review and discuss the literature on these topics and highlight open research questions and future directions.Through this survey, we aim to contribute to the academic discourse on AI security, enriching the understanding of potential risks and mitigations. We anticipate that this survey will be valuable for various stakeholders involved in AI development and usage, including AI researchers, developers, policy makers, and end-users.展开更多
This study aims to explore the characteristics of novice teachers’inappropriate behaviors in classroom teaching and their intervention strategies.With the continuous improvement of education quality,novice teachers f...This study aims to explore the characteristics of novice teachers’inappropriate behaviors in classroom teaching and their intervention strategies.With the continuous improvement of education quality,novice teachers face increasing challenges in teaching practice.Their inappropriate behaviors not only affect the classroom atmosphere but may also negatively impact students’learning outcomes.Therefore,researching the characteristics of novice teachers’inappropriate behaviors and their intervention strategies holds significant scientific and social value.This study employs a combination of quantitative and qualitative methods to analyze the behavioral patterns of novice teachers in classroom teaching and proposes corresponding intervention strategies.The results indicate that novice teachers’inappropriate behaviors mainly manifest as poor classroom management,monotonous teaching methods,and insufficient interaction with students.Based on these findings,the study proposes a series of effective intervention strategies,including enhancing teacher training,optimizing teaching design,and promoting positive interactions between teachers and students.The conclusions of the study not only provide practical guidance for educational practice but also point out directions for future research,emphasizing the crucial role of teacher professional development in improving teaching quality.展开更多
BACKGROUND Inflammatory bowel disease(IBD)is a global health burden that affects millions of individuals worldwide,necessitating extensive patient education.Large language models(LLMs)hold promise for addressing patie...BACKGROUND Inflammatory bowel disease(IBD)is a global health burden that affects millions of individuals worldwide,necessitating extensive patient education.Large language models(LLMs)hold promise for addressing patient information needs.However,LLM use to deliver accurate and comprehensible IBD-related medical information has yet to be thoroughly investigated.AIM To assess the utility of three LLMs(ChatGPT-4.0,Claude-3-Opus,and Gemini-1.5-Pro)as a reference point for patients with IBD.METHODS In this comparative study,two gastroenterology experts generated 15 IBD-related questions that reflected common patient concerns.These questions were used to evaluate the performance of the three LLMs.The answers provided by each model were independently assessed by three IBD-related medical experts using a Likert scale focusing on accuracy,comprehensibility,and correlation.Simultaneously,three patients were invited to evaluate the comprehensibility of their answers.Finally,a readability assessment was performed.RESULTS Overall,each of the LLMs achieved satisfactory levels of accuracy,comprehensibility,and completeness when answering IBD-related questions,although their performance varies.All of the investigated models demonstrated strengths in providing basic disease information such as IBD definition as well as its common symptoms and diagnostic methods.Nevertheless,when dealing with more complex medical advice,such as medication side effects,dietary adjustments,and complication risks,the quality of answers was inconsistent between the LLMs.Notably,Claude-3-Opus generated answers with better readability than the other two models.CONCLUSION LLMs have the potential as educational tools for patients with IBD;however,there are discrepancies between the models.Further optimization and the development of specialized models are necessary to ensure the accuracy and safety of the information provided.展开更多
The pre-weld heat treatment was carried out to obtain different initial microstructures of the GH4169 superalloy,and then Linear Friction Welding(LFW)was performed.The effect of the pre-weld heat treatment on the micr...The pre-weld heat treatment was carried out to obtain different initial microstructures of the GH4169 superalloy,and then Linear Friction Welding(LFW)was performed.The effect of the pre-weld heat treatment on the microstructure evolution and mechanical properties of the joint was analyzed,and the joint electrochemical corrosion behavior as well as the hot corrosion behavior was studied.The results show that the joint hardness of Base Metal(BM)increases after pre-weld heat treatment,and the strengthening phasesγ′andγ″further precipitate.However,the precipitation phases dissolve significantly in the Weld Zone(WZ)due to the thermal process of LFW.The corrosion resistance in BM is reduced after the pre-weld heat treatment,while it is similar in WZ with a slight decrease.The surface morphology of the BM and WZ can be generally divided into a loose and porous matrix and a scattered oxide particle layer after hot corrosion.The joint cross section exhibits a Cr-depleted zone with the diffusion of Cr to form an oxide film.The corrosion product mainly consists of Fe_(2)O_(3)/Fe_(3)O_(4) as the outer layer and Cr_(2)O_(3) as the inner layer.展开更多
The undrained mechanical behavior of unsaturated completely weathered granite(CWG)is highly susceptible to alterations in the hydraulic environment,particularly under uniaxial loading conditions,due to the unique natu...The undrained mechanical behavior of unsaturated completely weathered granite(CWG)is highly susceptible to alterations in the hydraulic environment,particularly under uniaxial loading conditions,due to the unique nature of this soil type.In this study,a series of unconfined compression tests were carried out on unsaturated CWG soil in an underground engineering site,and the effects of varying the environmental variables on the main undrained mechanical properties were analyzed.Based on the experimental results,a novel constitutive model was then established using the damage mechanics theory and the undetermined coefficient method.The results demonstrate that the curves of remolded CWG specimens with different moisture contents and dry densities exhibited diverse characteristics,including brittleness,significant softening,and ductility.As a typical indicator,the unconfined compression strength of soil specimens initially increased with an increase in moisture content and then decreased.Meanwhile,an optimal moisture content of approximately 10.5%could be observed,while a critical moisture content value of 13.0%was identified,beyond which the strength of the specimen decreases sharply.Moreover,the deformation and fracture of CWG specimens were predominantly caused by shear failure,and the ultimate failure modes were primarily influenced by moisture content rather than dry density.Furthermore,by comparing several similar models and the experimental data,the proposed model could accurately replicate the undrained mechanical characteristics of unsaturated CWG soil,and quantitatively describe the key mechanical indexes.These findings offer a valuable reference point for understanding the underlying mechanisms,anticipating potential risks,and implementing effective control measures in similar underground engineering projects.展开更多
Natural cemented calcareous sand and limestone are highly complex and not well understood in terms of the me-chanical behavior due to the difficulty of obtaining undisturbed samples from far sea.This paper proposes an...Natural cemented calcareous sand and limestone are highly complex and not well understood in terms of the me-chanical behavior due to the difficulty of obtaining undisturbed samples from far sea.This paper proposes an artificial method in a laboratory setting using microbial-induced carbonate precipitation(MICP)to simulate the natural process of cementation of limestone.The artificially cemented sand has a high degree of similarity with the natural weakly limestone in three aspects:(1)the mineral composition of the cemented material is also granular calcite and acicular aragonite;(2)the microstructure in interconnected open pore network can be gradually closed and contracted with cementation.The porosity reaches to approximately 9.2%;(3)both the stress-strain relationship and the unconfined strength closely resemble that of natural weakly limestone.Furthermore,both static and dynamic behaviors of artificial limestone were studied by quasi-static compression tests and Split Hopkinson Pressure Bar(SHPB)tests,finding that the unconfined strength of weakly artifical limestone exponentially increases with increasing strain rate.A rate-dependent bond strength was proposed and implemented in software to reveal the mechanism of strain rate effects.It is found that the loading velocity is too high to keep in sync with the initiation and propagation of cracks under impact loading.This delay-induced viscosity may restrict the movement of the surrounding balls,thus increasing resistance.展开更多
In recent years,cyber threats have escalated across diverse sectors,with cybercrime syndicates increasingly exploiting system vulnerabilities.Traditional passive defense mechanisms have proven insufficient,particularl...In recent years,cyber threats have escalated across diverse sectors,with cybercrime syndicates increasingly exploiting system vulnerabilities.Traditional passive defense mechanisms have proven insufficient,particularly as Linux platforms—historically overlooked in favor of Windows—have emerged as frequent targets.According to Trend Micro,there has been a substantial increase in Linux-targeted malware,with ransomware attacks on Linux surpassing those on macOS.This alarming trend underscores the need for detection strategies specifically designed for Linux environments.To address this challenge,this study proposes a comprehensive malware detection framework tailored for Linux systems,integrating dynamic behavioral analysis with the semantic reasoning capabilities of large language models(LLMs).Malware samples are executed within sandbox environments to extract behavioral features such as system calls and command-line executions.These features are then systematically mapped to the MITRE ATT&CK framework,incorporating its defined data sources,data components,and Tactics,Techniques,and Procedures(TTPs).Two mapping constructs—Conceptual Definition Mapping and TTP Technical Keyword Mapping—are developed from official MITRE documentation.These resources are utilized to fine-tune an LLM,enabling it to semantically interpret complex behavioral patterns and infer associated attack techniques,including those employed by previously unknown malware variants.The resulting detection pipeline effectively bridges raw behavioral data with structured threat intelligence.Experimental evaluations confirm the efficacy of the proposed system,with the fine-tuned Gemma 2B model demonstrating significantly enhanced accuracy in associating behavioral features with ATT&CK-defined techniques.This study contributes a fully integrated Linux-specific detection framework,a novel approach for transforming unstructured behavioral data into actionable intelligence,improved interpretability of malicious behavior,and a scalable training process for future applications of LLMs in cybersecurity.展开更多
The influence of Nb-V microalloying on the hot deformation behavior and microstructures of medium Mn steel(MMS)was investigated by uniaxial hot compression tests.By establishing the constitutive equations for simulati...The influence of Nb-V microalloying on the hot deformation behavior and microstructures of medium Mn steel(MMS)was investigated by uniaxial hot compression tests.By establishing the constitutive equations for simulating the measured flow curves,we successfully constructed deformation activation energy(Q)maps and processing maps for identifying the region of flow instability.We concluded the following consequences of Nb-V alloying for MMS.(i)The critical strain increases and the increment diminishes with the increasing deformation temperature,suggesting that NbC precipitates more efficiently retard dynamic recrystallization(DRX)in MMS compared with solute Nb.(ii)The deformation activation energy of MMS is significantly increased and even higher than that of some reported high Mn steels,suggesting that its ability to retard DRX is greater than that of the high Mn content.(iii)The hot workability of MMS is improved by narrowing the hot processing window for the unstable flow stress,in which fine recrystallized and coarse unrecrystallized grains are present.展开更多
Large-scale Language Models(LLMs)have achieved significant breakthroughs in Natural Language Processing(NLP),driven by the pre-training and fine-tuning paradigm.While this approach allows models to specialize in speci...Large-scale Language Models(LLMs)have achieved significant breakthroughs in Natural Language Processing(NLP),driven by the pre-training and fine-tuning paradigm.While this approach allows models to specialize in specific tasks with reduced training costs,the substantial memory requirements during fine-tuning present a barrier to broader deployment.Parameter-Efficient Fine-Tuning(PEFT)techniques,such as Low-Rank Adaptation(LoRA),and parameter quantization methods have emerged as solutions to address these challenges by optimizing memory usage and computational efficiency.Among these,QLoRA,which combines PEFT and quantization,has demonstrated notable success in reducing memory footprints during fine-tuning,prompting the development of various QLoRA variants.Despite these advancements,the quantitative impact of key variables on the fine-tuning performance of quantized LLMs remains underexplored.This study presents a comprehensive analysis of these key variables,focusing on their influence across different layer types and depths within LLM architectures.Our investigation uncovers several critical findings:(1)Larger layers,such as MLP layers,can maintain performance despite reductions in adapter rank,while smaller layers,like self-attention layers,aremore sensitive to such changes;(2)The effectiveness of balancing factors depends more on specific values rather than layer type or depth;(3)In quantization-aware fine-tuning,larger layers can effectively utilize smaller adapters,whereas smaller layers struggle to do so.These insights suggest that layer type is a more significant determinant of fine-tuning success than layer depth when optimizing quantized LLMs.Moreover,for the same discount of trainable parameters,reducing the trainable parameters in a larger layer is more effective in preserving fine-tuning accuracy than in a smaller one.This study provides valuable guidance for more efficient fine-tuning strategies and opens avenues for further research into optimizing LLM fine-tuning in resource-constrained environments.展开更多
文摘二语写作是二语习得研究领域的重要组成部分。运用CiteSpace软件对近十年发表在Journal of Second Language Writing的231篇实证研究论文进行可视化分析,研究发现:二语写作研究整体呈波动性上升趋势,研究规模较为稳定,研究关注度逐渐提升;二语写作研究领域暂未形成明显的核心作者和机构的合作网络;研究主题主要聚焦二语写作教学方法的多元化、二语写作反馈的多焦点、二语写作评估与测试的科学化,以及学习者个体差异的多维影响等方面。基于此,提出未来该领域发展需加强学者、机构之间的相互合作;关注个体学习者写作过程的认知特征与情感因素,尤其重视青少年二语学习过程的研究;扩大二语写作纵向研究规模,推动研究的深入发展。
基金supported by National Natural Science Foundation of China(62376219 and 62006194)Foundational Research Project in Specialized Discipline(Grant No.G2024WD0146)Faculty Construction Project(Grant No.24GH0201148).
文摘Large language models(LLMs)have undergone significant expansion and have been increasingly integrated across various domains.Notably,in the realm of robot task planning,LLMs harness their advanced reasoning and language comprehension capabilities to formulate precise and efficient action plans based on natural language instructions.However,for embodied tasks,where robots interact with complex environments,textonly LLMs often face challenges due to a lack of compatibility with robotic visual perception.This study provides a comprehensive overview of the emerging integration of LLMs and multimodal LLMs into various robotic tasks.Additionally,we propose a framework that utilizes multimodal GPT-4V to enhance embodied task planning through the combination of natural language instructions and robot visual perceptions.Our results,based on diverse datasets,indicate that GPT-4V effectively enhances robot performance in embodied tasks.This extensive survey and evaluation of LLMs and multimodal LLMs across a variety of robotic tasks enriches the understanding of LLM-centric embodied intelligence and provides forward-looking insights towards bridging the gap in Human-Robot-Environment interaction.
文摘Objective:This study aimed to examine the reliability and validity of the Chinese version of the Behavioral Inhibition System/Behavioral Activation System(BIS/BAS)scales among stroke survivors.Methods:The cross-sectional study was conducted at four comprehensive hospitals in Taizhou,Jiangsu,China.A sample of 232 first-ever stroke survivors were recruited from June to August 2023.Validity was examined using face validity and construct validity,which used confirmatory factor analysis(CFA)and known-group analysis.Reliability was evaluated by internal consistency and test-retest reliability.Results:The BIS/BAS scales demonstrated satisfactory face validity.The findings of CFAs supported the original four-factor structure of BAS-reward,BAS-drive,BAS-fun seeking,and BIS with acceptable model fit indices.Discriminative validity,assessed via known-group analysis,indicated that stroke survivors with probable depression had significantly lower mean BAS-reward,BAS-drive,and BAS-fun seeking scores(P<0.001)and a higher mean BIS score(P=0.028)compared to those without probable depression.The internal consistency,measured by Cronbach’s a coefficients for the subscales,ranged from 0.669 to 0.964.Test-retest reliability,assessed using intra-class correlation coefficients,ranged from 0.61 to 0.93.Conclusions:The Chinese version of the BIS/BAS scales could be a reliable and valid instrument for measuring behavioral activation among stroke survivors.
基金supported by ANID Fondecyt Iniciacion 11180540(to FJB)ANID PAI 77180077(to FJB)+2 种基金UNAB DI-02-22/REG(to FJB)Exploración-ANID 13220203(to FJB)ANID-MILENIO(NCN2023_23,to FJB)。
文摘Since the pioneering work by Broca and Wernicke in the 19th century,who examined individuals with brain lesions to associate them with specific behaviors,it was evident that behaviors are complex and cannot be fully attributable to specific brain areas alone.Instead,they involve connectivity among brain areas,whether close or distant.At that time,this approach was considered the optimal way to dissect brain circuitry and function.These pioneering efforts opened the field to explore the necessity or sufficiency of brain areas in controlling behavior and hence dissecting brain function.However,the connectivity of the brain and the mechanisms through which various brain regions regulate specific behaviors,either individually or collaboratively,remain largely elusive.Utilizing animal models,researchers have endeavored to unravel the necessity or sufficiency of specific brain areas in influencing behavior;however,no clear associations have been firmly established.
文摘Purpose:Evaluating the quality of academic journal articles is a time consuming but critical task for national research evaluation exercises,appointments and promotion.It is therefore important to investigate whether Large Language Models(LLMs)can play a role in this process.Design/methodology/approach:This article assesses which ChatGPT inputs(full text without tables,figures,and references;title and abstract;title only)produce better quality score estimates,and the extent to which scores are affected by ChatGPT models and system prompts.Findings:The optimal input is the article title and abstract,with average ChatGPT scores based on these(30 iterations on a dataset of 51 papers)correlating at 0.67 with human scores,the highest ever reported.ChatGPT 4o is slightly better than 3.5-turbo(0.66),and 4o-mini(0.66).Research limitations:The data is a convenience sample of the work of a single author,it only includes one field,and the scores are self-evaluations.Practical implications:The results suggest that article full texts might confuse LLM research quality evaluations,even though complex system instructions for the task are more effective than simple ones.Thus,whilst abstracts contain insufficient information for a thorough assessment of rigour,they may contain strong pointers about originality and significance.Finally,linear regression can be used to convert the model scores into the human scale scores,which is 31%more accurate than guessing.Originality/value:This is the first systematic comparison of the impact of different prompts,parameters and inputs for ChatGPT research quality evaluations.
基金supported by the National Key R&D Program of China under Grant No.2022YFB3103500the National Natural Science Foundation of China under Grants No.62402087 and No.62020106013+3 种基金the Sichuan Science and Technology Program under Grant No.2023ZYD0142the Chengdu Science and Technology Program under Grant No.2023-XT00-00002-GXthe Fundamental Research Funds for Chinese Central Universities under Grants No.ZYGX2020ZB027 and No.Y030232063003002the Postdoctoral Innovation Talents Support Program under Grant No.BX20230060.
文摘The integration of artificial intelligence(AI)technology,particularly large language models(LLMs),has become essential across various sectors due to their advanced language comprehension and generation capabilities.Despite their transformative impact in fields such as machine translation and intelligent dialogue systems,LLMs face significant challenges.These challenges include safety,security,and privacy concerns that undermine their trustworthiness and effectiveness,such as hallucinations,backdoor attacks,and privacy leakage.Previous works often conflated safety issues with security concerns.In contrast,our study provides clearer and more reasonable definitions for safety,security,and privacy within the context of LLMs.Building on these definitions,we provide a comprehensive overview of the vulnerabilities and defense mechanisms related to safety,security,and privacy in LLMs.Additionally,we explore the unique research challenges posed by LLMs and suggest potential avenues for future research,aiming to enhance the robustness and reliability of LLMs in the face of emerging threats.
基金supported by the Guangdong Major Project of Basic and Applied Basic Research(Grant No.2020B0301030006).
文摘Magnesium matrix composites with both high strength and ductility have been achieved by introducing pure Ti particles.However,the properties of the surfaces of the composites need to be improved by surface technology,such as micro-arc oxidation(MAO).In this study,we investigated the influence of the Ti-reinforcement phase on coating growth and evolution by subjecting both AZ91 alloy and AZ91/Ti composite to MAO treatment using silicate-based and phosphate-based electrolytes.Results revealed that the Ti-reinforcement phase influenced the MAO process,altering discharge behavior,and leading to a decreased cell voltage.The vigorous discharge of the Ti-reinforcement phase induced the formation of coating discharge channels,concurrently dissolving and oxidizing Ti-reinforcement to produce a composite ceramic coating with TiO2.The MAO coating on the AZ91/Ti composite exhibited a dark blue macromorphology and distinctive local micromorphological anomalies.In silicate electrolyte,a“volcano-like”localized morphology centered on the discharge channel emerged.In contrast,treatment in phosphate-based electrolyte resulted in a coating morphology similar to typical porous ceramic coatings,with visible radial discharge micropores at the reinforcement phase location.Compared to the AZ91 alloy,the coating on the AZ91/Ti composite exhibited lower thickness and higher porosity.MAO treatment reduced the self-corrosion current density of the AZ91/Ti surface by two orders of magnitude.The silicate coating demonstrated better corrosion resistance than the phosphate coating,attributed to its lower porosity.The formation mechanism of MAO coatings on AZ91/Ti composites in phosphate-based and silicate-based electrolytes was proposed.
基金financially supported by National Natural Science Foundation of China(grant numbers:52171100,U20A20234)National Key R&D Program of China(grant number:2021YFB3701100)。
文摘The performance of Mg alloys is significantly influenced by the concentrations and solid solution behavior of the alloying elements.In this work,the solid solution behavior of 20 alloying elements in 190 ternary Mg alloy systems at 500℃are systematically investigated.The solid solution behavior of a set of two different alloying elements in Mg alloy systems are suggested to be classified into three categories:inclusivity,exclusivity and proportionality.Inclusivity classification indicates that the two alloying elements are inclusive inα-Mg,increasing the joint solubility of both elements.Exclusivity classification suggests that the two alloying elements have a low joint solid solubility inα-Mg,since they prefer to form stable second phases.For the proportionality classification,the solubility curve of the ternary Mg alloy systems is a straight line connecting the solubility points of the two sub-binary systems.The proposed classification theory was validated by key experiments and the calculation of formation energies.The interaction effects between alloying elements and the preference of formation of second phases are the main factors determining the solid solution behavior classifications.Based on the observed solid solution features of multi-component Mg alloys,principles for alloy design of different types of high-performance Mg alloys were proposed in this work.
基金supported by the Yonsei University graduate school Department of Integrative Biotechnology.
文摘Recently,diffusion models have emerged as a promising paradigm for molecular design and optimization.However,most diffusion-based molecular generative models focus on modeling 2D graphs or 3D geom-etries,with limited research on molecular sequence diffusion models.The International Union of Pure and Applied Chemistry(IUPAC)names are more akin to chemical natural language than the simplified molecular input line entry system(SMILES)for organic compounds.In this work,we apply an IUPAC-guided conditional diffusion model to facilitate molecular editing from chemical natural language to chemical language(SMILES)and explore whether the pre-trained generative performance of diffusion models can be transferred to chemical natural language.We propose DiffIUPAC,a controllable molecular editing diffusion model that converts IUPAC names to SMILES strings.Evaluation results demonstrate that our model out-performs existing methods and successfully captures the semantic rules of both chemical languages.Chemical space and scaffold analysis show that the model can generate similar compounds with diverse scaffolds within the specified constraints.Additionally,to illustrate the model’s applicability in drug design,we conducted case studies in functional group editing,analogue design and linker design.
基金support from the National Natural Science Foundation of China(grant number 52075544)Innovation Funds of Jihua Laboratory(X220971UZ230)+1 种基金Basic and Applied Basic Research Foundation of Guangdong Province(2022A1515110649)Funds from Research Platforms of Guangdong Higher Education Institutes(2022ZDJS038).
文摘Efficient lubrication of magnesium alloys is a highly challenging topic in the field of tribology.In this study,magnesium silicate hydroxide(MSH)nanotubes with serpentine structures were synthesized.The tribological behavior of AZ91D magnesium alloy rubbed against GCr15 steel was studied under lubricating oil with surface-modified MSH nanotubes as additives.The effects of the concentration,applied load,and reciprocating frequency on the friction and wear of the AZ91D alloy were studied using an SRV-4 sliding wear tester.Results show a decrease of 18.7–68.5%in friction coefficient,and a reduction of 19.4–54.3%in wear volume of magnesium alloy can be achieved by applying the synthetic serpentine additive under different conditions.A suspension containing 0.3 wt.%MSH was most efficient in reducing wear and friction.High frequency and medium load were more conducive to improving the tribological properties of magnesium alloys.A series of beneficial physical and chemical processes occurring at the AZ91D alloy/steel interface can be used to explain friction and wear reduction based on the characterization of the morphology,chemical composition,chemical state,microstructure,and nanomechanical properties of the worn surface.The synthetic MSH,with serpentine structure and nanotube morphology,possesses excellent adsorbability,high chemical activity,and good self-lubrication and catalytic activity.Therefore,physical polishing,tribochemical reactions,and physicalchemical depositions can occur easily on the sliding contacts.A dense tribolayer with a complex composition and composite structure was formed on the worn surface.Its high hardness,good toughness and plasticity,and prominent lubricity resulted in the improvement of friction and wear,making the synthetic MSH a promising efficient oil additive for magnesium alloys under boundary and mixed lubrication.
文摘Software security poses substantial risks to our society because software has become part of our life. Numerous techniques have been proposed to resolve or mitigate the impact of software security issues. Among them, software testing and analysis are two of the critical methods, which significantly benefit from the advancements in deep learning technologies. Due to the successful use of deep learning in software security, recently,researchers have explored the potential of using large language models(LLMs) in this area. In this paper, we systematically review the results focusing on LLMs in software security. We analyze the topics of fuzzing, unit test, program repair, bug reproduction, data-driven bug detection, and bug triage. We deconstruct these techniques into several stages and analyze how LLMs can be used in the stages. We also discuss the future directions of using LLMs in software security, including the future directions for the existing use of LLMs and extensions from conventional deep learning research.
文摘ChatGPT is a powerful artificial intelligence(AI)language model that has demonstrated significant improvements in various natural language processing(NLP) tasks. However, like any technology, it presents potential security risks that need to be carefully evaluated and addressed. In this survey, we provide an overview of the current state of research on security of using ChatGPT, with aspects of bias, disinformation, ethics, misuse,attacks and privacy. We review and discuss the literature on these topics and highlight open research questions and future directions.Through this survey, we aim to contribute to the academic discourse on AI security, enriching the understanding of potential risks and mitigations. We anticipate that this survey will be valuable for various stakeholders involved in AI development and usage, including AI researchers, developers, policy makers, and end-users.
文摘This study aims to explore the characteristics of novice teachers’inappropriate behaviors in classroom teaching and their intervention strategies.With the continuous improvement of education quality,novice teachers face increasing challenges in teaching practice.Their inappropriate behaviors not only affect the classroom atmosphere but may also negatively impact students’learning outcomes.Therefore,researching the characteristics of novice teachers’inappropriate behaviors and their intervention strategies holds significant scientific and social value.This study employs a combination of quantitative and qualitative methods to analyze the behavioral patterns of novice teachers in classroom teaching and proposes corresponding intervention strategies.The results indicate that novice teachers’inappropriate behaviors mainly manifest as poor classroom management,monotonous teaching methods,and insufficient interaction with students.Based on these findings,the study proposes a series of effective intervention strategies,including enhancing teacher training,optimizing teaching design,and promoting positive interactions between teachers and students.The conclusions of the study not only provide practical guidance for educational practice but also point out directions for future research,emphasizing the crucial role of teacher professional development in improving teaching quality.
基金Supported by the China Health Promotion Foundation Young Doctors'Research Foundation for Inflammatory Bowel Disease,the Taishan Scholars Program of Shandong Province,China,No.tsqn202306343National Natural Science Foundation of China,No.82270578.
文摘BACKGROUND Inflammatory bowel disease(IBD)is a global health burden that affects millions of individuals worldwide,necessitating extensive patient education.Large language models(LLMs)hold promise for addressing patient information needs.However,LLM use to deliver accurate and comprehensible IBD-related medical information has yet to be thoroughly investigated.AIM To assess the utility of three LLMs(ChatGPT-4.0,Claude-3-Opus,and Gemini-1.5-Pro)as a reference point for patients with IBD.METHODS In this comparative study,two gastroenterology experts generated 15 IBD-related questions that reflected common patient concerns.These questions were used to evaluate the performance of the three LLMs.The answers provided by each model were independently assessed by three IBD-related medical experts using a Likert scale focusing on accuracy,comprehensibility,and correlation.Simultaneously,three patients were invited to evaluate the comprehensibility of their answers.Finally,a readability assessment was performed.RESULTS Overall,each of the LLMs achieved satisfactory levels of accuracy,comprehensibility,and completeness when answering IBD-related questions,although their performance varies.All of the investigated models demonstrated strengths in providing basic disease information such as IBD definition as well as its common symptoms and diagnostic methods.Nevertheless,when dealing with more complex medical advice,such as medication side effects,dietary adjustments,and complication risks,the quality of answers was inconsistent between the LLMs.Notably,Claude-3-Opus generated answers with better readability than the other two models.CONCLUSION LLMs have the potential as educational tools for patients with IBD;however,there are discrepancies between the models.Further optimization and the development of specialized models are necessary to ensure the accuracy and safety of the information provided.
基金supported by the National Natural Science Foundation of China(Nos.52074228,52305420 and 51875470)the Practice and Innovation Funds for Graduate Students of Northwestern Polytechnical University,China(No.PF2024053)the Xi’an Beilin District Science and Technology Planning Project,China(No.GX2349).
文摘The pre-weld heat treatment was carried out to obtain different initial microstructures of the GH4169 superalloy,and then Linear Friction Welding(LFW)was performed.The effect of the pre-weld heat treatment on the microstructure evolution and mechanical properties of the joint was analyzed,and the joint electrochemical corrosion behavior as well as the hot corrosion behavior was studied.The results show that the joint hardness of Base Metal(BM)increases after pre-weld heat treatment,and the strengthening phasesγ′andγ″further precipitate.However,the precipitation phases dissolve significantly in the Weld Zone(WZ)due to the thermal process of LFW.The corrosion resistance in BM is reduced after the pre-weld heat treatment,while it is similar in WZ with a slight decrease.The surface morphology of the BM and WZ can be generally divided into a loose and porous matrix and a scattered oxide particle layer after hot corrosion.The joint cross section exhibits a Cr-depleted zone with the diffusion of Cr to form an oxide film.The corrosion product mainly consists of Fe_(2)O_(3)/Fe_(3)O_(4) as the outer layer and Cr_(2)O_(3) as the inner layer.
基金Project(42202318)supported by the National Natural Science Foundation of ChinaProject(252300421199)supported by the Natural Science Foundation of Henan Province,ChinaProject(2024JJ6219)supported by the Hunan Provincial Natural Science Foundation of China。
文摘The undrained mechanical behavior of unsaturated completely weathered granite(CWG)is highly susceptible to alterations in the hydraulic environment,particularly under uniaxial loading conditions,due to the unique nature of this soil type.In this study,a series of unconfined compression tests were carried out on unsaturated CWG soil in an underground engineering site,and the effects of varying the environmental variables on the main undrained mechanical properties were analyzed.Based on the experimental results,a novel constitutive model was then established using the damage mechanics theory and the undetermined coefficient method.The results demonstrate that the curves of remolded CWG specimens with different moisture contents and dry densities exhibited diverse characteristics,including brittleness,significant softening,and ductility.As a typical indicator,the unconfined compression strength of soil specimens initially increased with an increase in moisture content and then decreased.Meanwhile,an optimal moisture content of approximately 10.5%could be observed,while a critical moisture content value of 13.0%was identified,beyond which the strength of the specimen decreases sharply.Moreover,the deformation and fracture of CWG specimens were predominantly caused by shear failure,and the ultimate failure modes were primarily influenced by moisture content rather than dry density.Furthermore,by comparing several similar models and the experimental data,the proposed model could accurately replicate the undrained mechanical characteristics of unsaturated CWG soil,and quantitatively describe the key mechanical indexes.These findings offer a valuable reference point for understanding the underlying mechanisms,anticipating potential risks,and implementing effective control measures in similar underground engineering projects.
基金The authors would like to acknowledge the support of the National Natural Science Foundation of China(No.52279097,No.51779264)Blue and Green Project of Jiangsu Province.
文摘Natural cemented calcareous sand and limestone are highly complex and not well understood in terms of the me-chanical behavior due to the difficulty of obtaining undisturbed samples from far sea.This paper proposes an artificial method in a laboratory setting using microbial-induced carbonate precipitation(MICP)to simulate the natural process of cementation of limestone.The artificially cemented sand has a high degree of similarity with the natural weakly limestone in three aspects:(1)the mineral composition of the cemented material is also granular calcite and acicular aragonite;(2)the microstructure in interconnected open pore network can be gradually closed and contracted with cementation.The porosity reaches to approximately 9.2%;(3)both the stress-strain relationship and the unconfined strength closely resemble that of natural weakly limestone.Furthermore,both static and dynamic behaviors of artificial limestone were studied by quasi-static compression tests and Split Hopkinson Pressure Bar(SHPB)tests,finding that the unconfined strength of weakly artifical limestone exponentially increases with increasing strain rate.A rate-dependent bond strength was proposed and implemented in software to reveal the mechanism of strain rate effects.It is found that the loading velocity is too high to keep in sync with the initiation and propagation of cracks under impact loading.This delay-induced viscosity may restrict the movement of the surrounding balls,thus increasing resistance.
基金supported by the National Science and Technology Council under grant number 113-2221-E-027-126-MY3.
文摘In recent years,cyber threats have escalated across diverse sectors,with cybercrime syndicates increasingly exploiting system vulnerabilities.Traditional passive defense mechanisms have proven insufficient,particularly as Linux platforms—historically overlooked in favor of Windows—have emerged as frequent targets.According to Trend Micro,there has been a substantial increase in Linux-targeted malware,with ransomware attacks on Linux surpassing those on macOS.This alarming trend underscores the need for detection strategies specifically designed for Linux environments.To address this challenge,this study proposes a comprehensive malware detection framework tailored for Linux systems,integrating dynamic behavioral analysis with the semantic reasoning capabilities of large language models(LLMs).Malware samples are executed within sandbox environments to extract behavioral features such as system calls and command-line executions.These features are then systematically mapped to the MITRE ATT&CK framework,incorporating its defined data sources,data components,and Tactics,Techniques,and Procedures(TTPs).Two mapping constructs—Conceptual Definition Mapping and TTP Technical Keyword Mapping—are developed from official MITRE documentation.These resources are utilized to fine-tune an LLM,enabling it to semantically interpret complex behavioral patterns and infer associated attack techniques,including those employed by previously unknown malware variants.The resulting detection pipeline effectively bridges raw behavioral data with structured threat intelligence.Experimental evaluations confirm the efficacy of the proposed system,with the fine-tuned Gemma 2B model demonstrating significantly enhanced accuracy in associating behavioral features with ATT&CK-defined techniques.This study contributes a fully integrated Linux-specific detection framework,a novel approach for transforming unstructured behavioral data into actionable intelligence,improved interpretability of malicious behavior,and a scalable training process for future applications of LLMs in cybersecurity.
基金financial support from the National Natural Science Foundation of China(Nos.52233018 and 51831002)the China Baowu Low Carbon Metallurgy Innovation Foudation(No.BWLCF202213)。
文摘The influence of Nb-V microalloying on the hot deformation behavior and microstructures of medium Mn steel(MMS)was investigated by uniaxial hot compression tests.By establishing the constitutive equations for simulating the measured flow curves,we successfully constructed deformation activation energy(Q)maps and processing maps for identifying the region of flow instability.We concluded the following consequences of Nb-V alloying for MMS.(i)The critical strain increases and the increment diminishes with the increasing deformation temperature,suggesting that NbC precipitates more efficiently retard dynamic recrystallization(DRX)in MMS compared with solute Nb.(ii)The deformation activation energy of MMS is significantly increased and even higher than that of some reported high Mn steels,suggesting that its ability to retard DRX is greater than that of the high Mn content.(iii)The hot workability of MMS is improved by narrowing the hot processing window for the unstable flow stress,in which fine recrystallized and coarse unrecrystallized grains are present.
基金supported by the National Key R&D Program of China(No.2021YFB0301200)National Natural Science Foundation of China(No.62025208).
文摘Large-scale Language Models(LLMs)have achieved significant breakthroughs in Natural Language Processing(NLP),driven by the pre-training and fine-tuning paradigm.While this approach allows models to specialize in specific tasks with reduced training costs,the substantial memory requirements during fine-tuning present a barrier to broader deployment.Parameter-Efficient Fine-Tuning(PEFT)techniques,such as Low-Rank Adaptation(LoRA),and parameter quantization methods have emerged as solutions to address these challenges by optimizing memory usage and computational efficiency.Among these,QLoRA,which combines PEFT and quantization,has demonstrated notable success in reducing memory footprints during fine-tuning,prompting the development of various QLoRA variants.Despite these advancements,the quantitative impact of key variables on the fine-tuning performance of quantized LLMs remains underexplored.This study presents a comprehensive analysis of these key variables,focusing on their influence across different layer types and depths within LLM architectures.Our investigation uncovers several critical findings:(1)Larger layers,such as MLP layers,can maintain performance despite reductions in adapter rank,while smaller layers,like self-attention layers,aremore sensitive to such changes;(2)The effectiveness of balancing factors depends more on specific values rather than layer type or depth;(3)In quantization-aware fine-tuning,larger layers can effectively utilize smaller adapters,whereas smaller layers struggle to do so.These insights suggest that layer type is a more significant determinant of fine-tuning success than layer depth when optimizing quantized LLMs.Moreover,for the same discount of trainable parameters,reducing the trainable parameters in a larger layer is more effective in preserving fine-tuning accuracy than in a smaller one.This study provides valuable guidance for more efficient fine-tuning strategies and opens avenues for further research into optimizing LLM fine-tuning in resource-constrained environments.
