Anthropic 发布 PostTrainBench v1.1，更新规则与反作弊管线，重新计算排行榜。新审计标记了 234 次训练-测试污染、12 次违规外部 API 调用、10 次模型替换，以及 GPT-5.6 Sol 的三次直接搜索 PTB 痕迹的运行。Fable 5 以 41.79% 领先，GPT-5.6 Sol 以 36.23% 紧随其后，Opus 5 为 34%。
http://x.com/i/article/2082154735822594048
PostTrainBench v1.1： As Agents Get Smarter， So Does Reward Hacking
We are releasing PostTrainBench v1.1 with clearer rules， stronger anti-cheat pipeline， and a recomputed leaderboard.
Earlier this year， we released PostTrainBench， one of the first recursive self-improvement evals designed to measure how well AI agents can post-train weaker models. As agents became more capable， however， they also became better at finding unintended ways to raise their scores： targeting held-out items， distilling from external models， substituting stronger checkpoints， or mining traces from previous runs.
Our new audit flagged 234 runs for train-test contamination， 12 for disallowed external API use， and 10 for model substitution. Three GPT-5.6 Sol runs were also flagged for consulting published PostTrainBench traces.
V1.1 strengthens evaluation integrity by making the boundary between genuine model improvement and reward hacking explicit.
Inside the Reward Hacks
We reran 121 runs affected by disallowed API use， missing eval credentials， or other infrastructure faults. Promoting Fable 5 to two seeds also removed the earlier SmolLM substitutions. Its GPQA result still falls back to Opus 4.8 Max because Fable refused that benchmark.Runs affected by the finalized contamination boundary were rerun with the new settings. When an older agent cannot be rerun comparably， such as when it is no longer available through its API， we leave the affected result unreported rather than carry it into the leaderboard.
The updated trace viewer exposes the underlying actions， artifacts， and judge decisions so leaderboard changes can be inspected.
Anthropic 发布 PostTrainBench v1.1，更新规则与反作弊管线，重新计算排行榜。新审计标记了 234 次训练-测试污染、12 次违规外部 API 调用、10 次模型替换，以及 GPT-5.6 Sol 的三次直接搜索 PTB 痕迹的运行。Fable 5 以 41.79% 领先，GPT-5.6 Sol 以 36.23% 紧随其后，Opus 5 为 34%。
http://x.com/i/article/2082154735822594048
PostTrainBench v1.1： As Agents Get Smarter， So Does Reward Hacking
We are releasing PostTrainBench v1.1 with clearer rules， stronger anti-cheat pipeline， and a recomputed leaderboard.
Earlier this year， we released PostTrainBench， one of the first recursive self-improvement evals designed to measure how well AI agents can post-train weaker models. As agents became more capable， however， they also became better at finding unintended ways to raise their scores： targeting held-out items， distilling from external models， substituting stronger checkpoints， or mining traces from previous runs.
Results
Agents now get the same decontamination tool used during review. It catches n-gram overlap； the judge separately checks non-verbatim item targeting. External distillation is now out of bounds. Local teachers on the allocated GPU and self-generation with the assigned model remain allowed. We replaced one general judge with specialized reviews for contamination， API use， and direct PTB lookup， plus a programmatic model identity check.
The clearest sign that PostTrainBench needed stronger rules came from GPT-5.6 Sol. In three runs， the agent searched for PostTrainBench directly. In one， it cloned the repository， opened the trace viewer， and used earlier runs on the same benchmark and base model to guide its training.
Fable and Kimi show where we draw the line. Fable trained on more than 32，000 public function-calling examples， and only 2 of its 8 BFCL cells exceeded the contamination threshold. Kimi， by contrast， inspected specific BFCL failures and generated targeted "polish" examples around them. Those examples made up 79% of its final SFT dataset.
This helps explain Kimi K3's official v1.1 score of 31.35%. Kimi is a frontier-level model with strong results elsewhere， but nearly all of its BFCL runs were flagged for contamination and assigned the base-model score.
After re-auditing the historical runs， we recomputed the leaderboard. Fable 5 leads with 41.79%， followed by GPT-5.6 Sol at 36.23% and Opus 5 at 34%.
Runtime alone did not predict performance. Fable 5 Max achieved the highest score， reaching 41.8% in 8h 48m， while GPT-5.6 Sol offered the strongest performance below eight hours at 36.1% in 7h 20m. By contrast， several agents used nearly the same runtime but finished substantially below the Pareto frontier.
Conclusion
PostTrainBench v1.1 is a more precise measurement of the same question： how much can an agent improve a base model with a fixed compute budget？ The final policy preserves legitimate benchmark hill climbing while excluding training data built around particular test items. The agent now sees the same contamination boundary used to review its run and has a tool to check its own data before training.
Leaderboard： https://posttrainbench.com
Traces： https://posttrainbench.com/traces
Appendix
Citation
If you found PostTrainBench useful， please cite us as：
Our new audit flagged 234 runs for train-test contamination， 12 for disallowed external API use， and 10 for model substitution. Three GPT-5.6 Sol runs were also flagged for consulting published PostTrainBench traces.
V1.1 strengthens evaluation integrity by making the boundary between genuine model improvement and reward hacking explicit.
Inside the Reward Hacks
We reran 121 runs affected by disallowed API use， missing eval credentials， or other infrastructure faults. Promoting Fable 5 to two seeds also removed the earlier SmolLM substitutions. Its GPQA result still falls back to Opus 4.8 Max because Fable refused that benchmark.Runs affected by the finalized contamination boundary were rerun with the new settings. When an older agent cannot be rerun comparably， such as when it is no longer available through its API， we leave the affected result unreported rather than carry it into the leaderboard.
The updated trace viewer exposes the underlying actions， artifacts， and judge decisions so leaderboard changes can be inspected.
Results
Agents now get the same decontamination tool used during review. It catches n-gram overlap； the judge separately checks non-verbatim item targeting. External distillation is now out of bounds. Local teachers on the allocated GPU and self-generation with the assigned model remain allowed. We replaced one general judge with specialized reviews for contamination， API use， and direct PTB lookup， plus a programmatic model identity check.
The clearest sign that PostTrainBench needed stronger rules came from GPT-5.6 Sol. In three runs， the agent searched for PostTrainBench directly. In one， it cloned the repository， opened the trace viewer， and used earlier runs on the same benchmark and base model to guide its training.
Fable and Kimi show where we draw the line. Fable trained on more than 32，000 public function-calling examples， and only 2 of its 8 BFCL cells exceeded the contamination threshold. Kimi， by contrast， inspected specific BFCL failures and generated targeted "polish" examples around them. Those examples made up 79% of its final SFT dataset.
This helps explain Kimi K3's official v1.1 score of 31.35%. Kimi is a frontier-level model with strong results elsewhere， but nearly all of its BFCL runs were flagged for contamination and assigned the base-model score.
After re-auditing the historical runs， we recomputed the leaderboard. Fable 5 leads with 41.79%， followed by GPT-5.6 Sol at 36.23% and Opus 5 at 34%.
Runtime alone did not predict performance. Fable 5 Max achieved the highest score， reaching 41.8% in 8h 48m， while GPT-5.6 Sol offered the strongest performance below eight hours at 36.1% in 7h 20m. By contrast， several agents used nearly the same runtime but finished substantially below the Pareto frontier.
Conclusion
PostTrainBench v1.1 is a more precise measurement of the same question： how much can an agent improve a base model with a fixed compute budget？ The final policy preserves legitimate benchmark hill climbing while excluding training data built around particular test items. The agent now sees the same contamination boundary used to review its run and has a tool to check its own data before training.
Leaderboard： https://posttrainbench.com
Traces： https://posttrainbench.com/traces
Appendix
Citation
If you found PostTrainBench useful， please cite us as：
