# Meta 新论文：RL 优化代码为何常失败

- 来源：Rohan Paul (@rohanpaul_ai)
- 发布时间：2026-08-01 12:38
- AIHOT 分数：32
- AIHOT 链接：https://aihot.virxact.com/items/cms9wj8kj01q5ro5e9fgbx4f3
- 原文链接：https://x.com/rohanpaul_ai/status/2083412110374215893

## AI 摘要

Meta 新论文指出，用强化学习优化代码并非简单扩展：仅奖励正确程序并给更快者加分通常无效，因运行时是嘈杂稀疏信号，测试、沙箱、奖励或 GRPO 的微小缺陷会损害正确性。作者重建整个反馈路径，包括更大优化测试、校准远程执行服务、问题相对排名及正确性门控奖励，并改进 GRPO 以稳定噪声批次。Qwen 2.5 7B 在 top-50% 速度阈值下从 18.0% 提升至 31.3%。

## 正文

New Meta Paper.

Code optimization looks like an easy extension of reinforcement learning： reward correct programs， then give extra credit to the faster ones.

This paper shows why that recipe usually fails.

Says that code optimization requires co-designing tests， timing infrastructure， rewards， and GRPO.

Runtime is a noisy， sparse signal， and small flaws in the tests， sandbox， reward， or GRPO update can make the model barely faster while also hurting correctness.

The authors rebuild the entire feedback path： larger optimization tests， a calibrated remote execution service， problem-relative ranking against human solutions， correctness-gated binary rewards， and GRPO changes for noisy zero-advantage batches.

They stabilize GRPO by sampling more solutions per prompt， enlarging batches， dropping group variance normalization， and discarding stale timing data.

Qwen 2.5 7B moved from 18.0% to 31.3% at the top-50% speed threshold only after the full timing pipeline was rebuilt.

- arxiv. org/abs/2607.25970

Title： "Reinforcement Learning for Code Optimization"