为改善参数量较小的大模型在逻辑推理任务中的性能不足,以及微调模型复杂度高、资源受限的问题,本文采用上下文学习(ICL)方法,通过引入思维链提示(CoT)构建示例,探索在无需微调模型参数的情况下提升通用语言模型ChatGLM2-6B推理性能的...为改善参数量较小的大模型在逻辑推理任务中的性能不足,以及微调模型复杂度高、资源受限的问题,本文采用上下文学习(ICL)方法,通过引入思维链提示(CoT)构建示例,探索在无需微调模型参数的情况下提升通用语言模型ChatGLM2-6B推理性能的可行性。以Zero-Shot-CoT生成的思维链为基准,结合随机检索与多种聚类方法优化示例选择策略。实验结果表明,不同示例选择策略可使模型推理能力平均提升10%,验证了思维链提示对推理性能的增强效果,并显示优化示例策略能够在资源受限条件下实现大模型的高效利用。本研究为提升语言模型逻辑推理能力和下游任务性能提供了新思路,并为低资源场景下的大模型应用奠定了理论基础。To improve the underperformance of large models with small parameter counts in logical reasoning tasks, as well as the high complexity and resource constraints of fine-tuning the models. This paper adopts In-Context-Learning approach to explore the feasibility of improving the reasoning performance of the General Language Model, ChatGLM2-6B, without the need of fine-tuning the parameters of the model, by introducing Chain-of-Thought (CoT) prompt to construct examples. The CoT generated by Zero-Shot-CoT are used as the benchmark, and the example selection strategy is optimized by combining random retrieval with multiple clustering methods. The experimental results show that different example selection strategies can improve the model’s reasoning ability by 10% on average, verifying the enhancing effect of CoT prompts on reasoning performance, and showing that the optimized example strategy can achieve the efficient utilization of large models under resource-constrained conditions. This study provides new ideas for improving the logical reasoning ability of language models and the performance of downstream tasks, and lays a theoretical foundation for the application of large models in low-resource scenarios.展开更多
文摘为改善参数量较小的大模型在逻辑推理任务中的性能不足,以及微调模型复杂度高、资源受限的问题,本文采用上下文学习(ICL)方法,通过引入思维链提示(CoT)构建示例,探索在无需微调模型参数的情况下提升通用语言模型ChatGLM2-6B推理性能的可行性。以Zero-Shot-CoT生成的思维链为基准,结合随机检索与多种聚类方法优化示例选择策略。实验结果表明,不同示例选择策略可使模型推理能力平均提升10%,验证了思维链提示对推理性能的增强效果,并显示优化示例策略能够在资源受限条件下实现大模型的高效利用。本研究为提升语言模型逻辑推理能力和下游任务性能提供了新思路,并为低资源场景下的大模型应用奠定了理论基础。To improve the underperformance of large models with small parameter counts in logical reasoning tasks, as well as the high complexity and resource constraints of fine-tuning the models. This paper adopts In-Context-Learning approach to explore the feasibility of improving the reasoning performance of the General Language Model, ChatGLM2-6B, without the need of fine-tuning the parameters of the model, by introducing Chain-of-Thought (CoT) prompt to construct examples. The CoT generated by Zero-Shot-CoT are used as the benchmark, and the example selection strategy is optimized by combining random retrieval with multiple clustering methods. The experimental results show that different example selection strategies can improve the model’s reasoning ability by 10% on average, verifying the enhancing effect of CoT prompts on reasoning performance, and showing that the optimized example strategy can achieve the efficient utilization of large models under resource-constrained conditions. This study provides new ideas for improving the logical reasoning ability of language models and the performance of downstream tasks, and lays a theoretical foundation for the application of large models in low-resource scenarios.
