In this paper, a new preference multi-objective optimization algorithm called immune clone algorithm based on reference direction method （RD-ICA） is proposed for solving many-objective optimization problems. First, ...In this paper, a new preference multi-objective optimization algorithm called immune clone algorithm based on reference direction method （RD-ICA） is proposed for solving many-objective optimization problems. First, an intelligent recombination operator, which performs well on the functions comprising many parameters, is introduced into an immune clone algorithm so as to explore the potentially excellent gene segments of all individuals in the antibody pop- ulation. Second, a reference direction method, a very strict ranking based on the desire of decision makers （DMs）, is used to guide selection and clone of the active population. Then a light beam search （LBS） is borrowed to pick out a small set of individuals filling the external population. The proposed method has been extensively compared with other recently proposed evolutionary multi-objective optimization （EMO） approaches over DTLZ problems with from 4 to 100 objectives. Experimental results indicate RD-ICA can achieve competitive results.展开更多
The advancement of Large Language Models (LLMs) for domain applications in fields such as materials science and engineering depends on the development of fine-tuning strategies that adapt models for specialized, techn...The advancement of Large Language Models (LLMs) for domain applications in fields such as materials science and engineering depends on the development of fine-tuning strategies that adapt models for specialized, technical capabilities. In this work, we explore the effects of Continued Pretraining (CPT), Supervised Fine-Tuning (SFT), and various preference-based optimization approaches, including Direct Preference Optimization (DPO) and Odds Ratio Preference Optimization (ORPO), on fine-tuned LLM performance. Our analysis shows how these strategies influence model outcomes and reveals that the merging of multiple fine-tuned models can lead to the emergence of capabilities that surpass the individual contributions of the parent models. We find that model merging is not merely a process of aggregation, but a transformative method that can drive substantial advancements in model capabilities characterized by highly nonlinear interactions between model parameters, resulting in new functionalities that neither parent model could achieve alone, leading to improved performance in domain-specific assessments. We study critical factors that influence the success of model merging, such as the diversity between parent models and the fine-tuning techniques employed. The insights underscore the potential of strategic model merging to unlock novel capabilities in LLMs, offering an effective tool for advancing AI systems to meet complex challenges. Experiments with different model architectures are presented, including the Llama 3.1 8B and Mistral 7B family of models, where similar behaviors are observed. Exploring whether the results hold also for much smaller models, we use a tiny LLM with 1.7 billion parameters and show that very small LLMs do not necessarily feature emergent capabilities under model merging, suggesting that model scaling may be a key component. In open-ended yet consistent chat conversations between a human and AI models, our assessment reveals detailed insights into how different model variants perform, and shows that the smallest model achieves a high intelligence score across key criteria including reasoning depth, creativity, clarity, and quantitative precision. Other experiments include the development of image generation prompts that seek to reason over disparate biological material design concepts, to create new microstructures, architectural concepts, and urban design based on biological materials-inspired construction principles. We conclude with a series of questions about scaling and emergence that could be addressed in future research.展开更多
基金The authors would like to thank the editor and the reviewers for helpful comments that greatly improved the paper. This work was supported by the National Natural Science Foundation of China （Grant Nos. 613731 l 1, 61272279, 61003199 and 61203303） the Fundamental Re- search Funds for the Central Universities （K50511020014, K5051302084, K50510020011, K5051302049 and K5051302023）+1 种基金 the Fund for Foreign Scholars in University Research and Teaching Programs （the 111 Project） （B07048） and the Program for New Century Excellent Talents in University （NCET- 12-0920）.
文摘In this paper, a new preference multi-objective optimization algorithm called immune clone algorithm based on reference direction method （RD-ICA） is proposed for solving many-objective optimization problems. First, an intelligent recombination operator, which performs well on the functions comprising many parameters, is introduced into an immune clone algorithm so as to explore the potentially excellent gene segments of all individuals in the antibody pop- ulation. Second, a reference direction method, a very strict ranking based on the desire of decision makers （DMs）, is used to guide selection and clone of the active population. Then a light beam search （LBS） is borrowed to pick out a small set of individuals filling the external population. The proposed method has been extensively compared with other recently proposed evolutionary multi-objective optimization （EMO） approaches over DTLZ problems with from 4 to 100 objectives. Experimental results indicate RD-ICA can achieve competitive results.
基金supported in part by Google,the MIT Generative AI Initiative,USDA(grant number 2021-69012-35978)with additional support from NIH.This material is partially based upon work supported by the National Science Foundation Graduate Research Fellowship under Grant number 2141064.
文摘The advancement of Large Language Models (LLMs) for domain applications in fields such as materials science and engineering depends on the development of fine-tuning strategies that adapt models for specialized, technical capabilities. In this work, we explore the effects of Continued Pretraining (CPT), Supervised Fine-Tuning (SFT), and various preference-based optimization approaches, including Direct Preference Optimization (DPO) and Odds Ratio Preference Optimization (ORPO), on fine-tuned LLM performance. Our analysis shows how these strategies influence model outcomes and reveals that the merging of multiple fine-tuned models can lead to the emergence of capabilities that surpass the individual contributions of the parent models. We find that model merging is not merely a process of aggregation, but a transformative method that can drive substantial advancements in model capabilities characterized by highly nonlinear interactions between model parameters, resulting in new functionalities that neither parent model could achieve alone, leading to improved performance in domain-specific assessments. We study critical factors that influence the success of model merging, such as the diversity between parent models and the fine-tuning techniques employed. The insights underscore the potential of strategic model merging to unlock novel capabilities in LLMs, offering an effective tool for advancing AI systems to meet complex challenges. Experiments with different model architectures are presented, including the Llama 3.1 8B and Mistral 7B family of models, where similar behaviors are observed. Exploring whether the results hold also for much smaller models, we use a tiny LLM with 1.7 billion parameters and show that very small LLMs do not necessarily feature emergent capabilities under model merging, suggesting that model scaling may be a key component. In open-ended yet consistent chat conversations between a human and AI models, our assessment reveals detailed insights into how different model variants perform, and shows that the smallest model achieves a high intelligence score across key criteria including reasoning depth, creativity, clarity, and quantitative precision. Other experiments include the development of image generation prompts that seek to reason over disparate biological material design concepts, to create new microstructures, architectural concepts, and urban design based on biological materials-inspired construction principles. We conclude with a series of questions about scaling and emergence that could be addressed in future research.
