# AutoLab 基准测试：坚持比聪明更重要

- 来源：Rohan Paul (@rohanpaul_ai)
- 发布时间：2026-07-22 17:25
- AIHOT 分数：27
- AIHOT 链接：https://aihot.virxact.com/items/cmrvwm3ng01v1bipzh95w6hfg
- 原文链接：https://x.com/rohanpaul_ai/status/2079860250425909523

## AI 摘要

新基准 AutoLab 包含 36 个长周期任务，要求模型从弱代码起步并在固定时间内改进。测试 17 个前沿模型后发现，Claude Opus 4.6 领先并非因首次猜测正确，而是持续测试并利用反馈。其他模型常因过早放弃或思考过久导致超时而失败。

## 正文

strong AI agents still struggle with long research work because they often fail to keep testing and improving.

This paper shows that today's strongest research agents win less by brilliance than by refusing to stop testing.

The paper proposes AutoLab， a benchmark with 36 tasks where each agent starts from working but weak code and must make it better within a fixed time limit.

The tasks cover system speedups， puzzles， model development， and CUDA kernel work， so the test is not just about writing code once but about managing a long work session.

The authors tested 17 strong models and found that the best results did not mainly come from the first idea being good， but from the model staying active， testing often， and using feedback well.

The best first idea was not the strongest predictor of success； persistence was.

Claude Opus 4.6 led the benchmark not because it always guessed the right move immediately， but because it kept benchmarking and folding empirical feedback into the next attempt.

Several other frontier models failed in a more revealing way： they either quit early with time left on the clock， or thought so long that they ran out of time before submitting anything useful.

----

- arxiv. org/abs/2606.05080

Title： "AutoLab： Can Frontier Models Solve Long-Horizon Auto Research and Engineering Tasks？"