# 预训练验证损失可预测强化学习收益：基于国际象棋任务的LLM流水线研究

- 来源：Rohan Paul (@rohanpaul_ai)
- 发布时间：2026-07-26 22:28
- AIHOT 分数：36
- AIHOT 链接：https://aihot.virxact.com/items/cms1x547v0051ro9fmfu8xf5n
- 原文链接：https://x.com/rohanpaul_ai/status/2081386054787965166

## AI 摘要

研究发现，预训练验证损失能高精度预测RL后的pass@1准确率（相关性|ρ|从0.93提升至0.99），且每10倍RL算力的奖励增益与预训练token数对数正相关（r=+0.84）。在680M参数下，RL最优算力占比约28%。机制上，RL并非均匀增强推理，而是对困难问题生成正确动作的同时，数倍放大已偏好的错误动作，因此仅提升pass@1而非pass@16。

## 正文

Can you figure out how much RL will actually buy you before you commit a month of GPU time？

This paper says you can predict it based on the checkpoint you start from. The authors recreate the entire LLM pipeline in chess -- pretraining 5M-1B models on Lichess games ， SFT on synthetic reasoning traces， then GRPO on puzzles with verifiable rewards -- and sweep 36 pretraining-RL combinations.

Signal is mostly carried by two pretraining properties. The pretraining validation loss predicts the post-RL pass@1， with the fit tightening from |ρ|=0.93 to 0.99 as the reference RL compute increases； and the reward gained per 10× of RL compute increases with log pretraining tokens （Pearson r=+0.84）. The same two patterns are seen in a 1B model trained on math text.

The fitted frontier puts RL's compute-optimal share at around 20% at 50M parameters， and 28% at 680M. With budget， the share increases but remains a minority.

The mechanism is the part worth noticing. RL is not uniform sharpening： on the hardest puzzles it generates correct moves to almost non-existent levels after SFT， but makes already-preferred wrong moves many times more often.

That is why they get pass@1 gains but not pass@16 gains， and it leads the work in a different direction： suppress wrong-mode amplification， rather than sharpen harder.

- arxiv. org/abs/2607.16097

Title： "Understanding Reasoning from Pretraining to Post-Training"