# 大型语言模型能否超越经典的超参数优化算法？

- 来源：Hacker News 热门（buzzing.cc 中文翻译）
- 作者：galsapir
- 发布时间：2026-06-10 09:14
- AIHOT 分数：55
- AIHOT 链接：https://aihot.virxact.com/items/cmq7eksat02edsl5wegcng3tl
- 原文链接：https://arxiv.org/abs/2603.24647

## AI 摘要

一项研究将大型语言模型（LLM）应用于超参数优化任务，并与经典算法进行对比实验，检验 LLM 在该场景下是否具备超越传统方法的表现。

## 正文

Computer Science > Machine Learning

Title:Can LLMs Beat Classical Hyperparameter Optimization Algorithms? A Study on autoresearch

Abstract:The autoresearch repository enables an LLM agent to optimize hyperparameters by editing training code directly. We use it as a testbed to compare classical HPO algorithms against LLM-based methods on tuning the hyperparameters of a small language model under a fixed compute budget. When defining a fixed search space over autoresearch, classical methods such as CMA-ES and TPE consistently outperform LLM-based agents, where avoiding out-of-memory failures matters more than search diversity. Allowing the LLM to directly edit source code narrows the gap to the classical methods but does not close it, even with frontier models available at the time of writing such as Claude Opus 4.6 and Gemini 3.1 Pro Preview. We observe that LLMs struggle to track optimization state across trials. In contrast, classical methods lack the domain knowledge of LLMs. To combine the strengths of both, we introduce Centaur, a hybrid that shares CMA-ES's interpretable internal state, including mean vector, step-size, and covariance matrix, with an LLM. Centaur achieves the best result in our experiments, and a 0.8B LLM already suffices to outperform all classical and pure LLM methods. Unconstrained code editing requires larger models to be competitive with classical methods. We further analyze search diversity, model scaling from 0.8B to frontier models, and ablate the fraction of LLM-proposed trials in Centaur. All in all, our results suggest that LLMs are most effective as a complement to classical optimizers, not as a replacement. Code is available at this https URL & interactive demo at this https URL.

Subjects: Machine Learning (cs.LG); Machine Learning (stat.ML) Cite as: arXiv:2603.24647 [cs.LG] (or arXiv:2603.24647v5 [cs.LG] for this version) https://doi.org/10.48550/arXiv.2603.24647 Focus to learn more arXiv-issued DOI via DataCite

Submission history

Access Paper:

View PDF

HTML (experimental)

TeX Source

Current browse context:

References & Citations

NASA ADS

Google Scholar

Semantic Scholar

BibTeX formatted citation

Bookmark

Bibliographic and Citation Tools

Code, Data and Media Associated with this Article

Demos

Recommenders and Search Tools

Author

Venue

Institution

Topic

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.